The short answer
Following AI-generated security advice is risky because security is the area where a plausible, slightly wrong answer does the most damage and is the hardest to notice. An assistant can explain multi-factor authentication or email authentication very well. What it cannot do is see your tenant, confirm which policies already apply, test that a change did what it should, or warn you when a control stops working next month. Security failures are usually silent until an attacker finds them. So the safe approach is to use AI for understanding and drafting, and to have any change to live security controls reviewed, tested and recorded by someone accountable.
Why security is different from other IT questions
If an AI suggestion breaks a spreadsheet formula, you see it immediately. Security is not like that:
- Failures are invisible. A misconfigured policy still shows a green tick. You find out when it matters.
- Changes interact. A new rule can override, exclude or duplicate an existing one in ways a chat window cannot see.
- Attackers adapt faster than training data. Advice that was sound when a model was trained may be incomplete now.
- The cost of being wrong is high. Ransomware, business email compromise and privacy breaches carry recovery costs, downtime and, under state and sector breach laws, possible notification obligations.
Six ways AI security advice goes wrong
| Failure mode | Example | What it can cost |
|---|---|---|
| Exclusions that are too broad | "Add this folder to your antivirus exclusions" to stop a slow scan | A permanent blind spot where malware can sit |
| Weakening to fix friction | Disabling a sign-in prompt, legacy protocol block or security default to stop user complaints | Removes a control attackers look for |
| Wrong scope | A script meant for one mailbox applied to all, or a firewall rule opening "any" instead of one address | Wide exposure from one line |
| Outdated method | Instructions for a retired admin center, protocol or setting | Half-applied change, false confidence |
| Incomplete email authentication | An SPF or DMARC record that omits a sending service, or jumps straight to a reject policy | Lost legitimate email, or spoofing still possible |
| No monitoring step | A control is switched on, but nobody checks alerts | Detection without response |
None of these requires the AI to be absurd. Each is a reasonable-sounding answer missing context the assistant did not have.
The privacy trap
The fastest way to get a precise answer is to paste the real thing: a sign-in log, an email header, a firewall configuration, a list of flagged users. That is also how sensitive data ends up in a public tool. Users can easily and unknowingly provide sensitive corporate data or personal information in AI queries, and NIST's AI Risk Management Framework lists privacy among the risks organizations need to manage when using AI (NIST AI RMF). Using anonymized or de-identified information in prompts wherever possible is a sound habit.
Before pasting anything, remove names, email addresses, IP addresses, account identifiers and, above all, passwords, keys and tokens. If you cannot redact it, do not paste it. If the data is health, financial or student information, pasting it into a public tool may also breach your obligations under HIPAA, GLBA, FERPA or your customer contracts.
Attackers use AI too
Generative AI also helps attackers produce more convincing phishing at scale. That raises the bar for your own controls: strong authentication, email filtering, patching and staff awareness matter more, not less. CISA's Secure Our World basics (strong passwords, multi-factor authentication, software updates and recognizing phishing) are the starting point (CISA), and the NIST Cybersecurity Framework adds an incident response plan, backups, access control and management of outsourced IT services (NIST CSF). An AI assistant can explain each control; implementing and maintaining them consistently is the harder part.
Rather talk it through? If you have already changed security settings based on AI advice, or are about to, we can review them against your environment and confirm nothing important was weakened. Talk to a Promatics specialist
How to check security advice before acting
- I can explain in one sentence what this change does and why
- The advice matches the vendor's current official documentation
- I know which existing policies or rules this interacts with
- The change is scoped to what I intend (one user, one group, one address), not "all" or "any"
- It does not add exclusions, disable protections or turn off logging
- I have tested it on a pilot user or group first
- I know how to undo it, and I have recorded the previous setting
- Someone will review the related alerts or reports next week
- No personal information or credentials were pasted into the AI tool
If you cannot tick most of these, stop and get a second opinion.
When to bring in help
AI is a fine companion for security awareness: understanding phishing, drafting a password policy, learning what a control is for. Bring in a professional when:
- the change affects identity, email, firewalls, endpoint protection or backups;
- you are responding to an alert, suspicious sign-in or suspected incident;
- you handle health, financial or other sensitive personal information;
- a cyber insurer, auditor or client is asking you to evidence controls;
- nobody on your team would notice if a control silently stopped working.
A cybersecurity partner brings the context an assistant lacks: your configuration, your risks, your obligations under US privacy and sector laws, and responsibility for the result. If you want to know where you stand today, a security assessment is the usual first step.
AI tools bring security risks of their own, from staff pasting confidential data into chatbots to assistants that can be manipulated by hidden instructions. Our AI governance and security service covers policies, discovery of unapproved AI use and security testing of AI applications.
Sources and further reading
Product capabilities and guidance change. These are the primary sources this article relies on, checked on the review date above.
- AI Risk Management Framework, National Institute of Standards and Technology
- Secure Our World, Cybersecurity and Infrastructure Security Agency (CISA)
- NIST Cybersecurity Framework 2.0, National Institute of Standards and Technology
This article is general information, not legal, accounting or security advice for your specific situation. Examples are hypothetical unless stated otherwise.