A decade ago the same film played with unsanctioned cloud apps: marketing signed up for a file-sharing service, finance connected it to payroll, and IT found out during an audit. The fix that worked was not blocking everything. It was sanctioning a good option, watching the egress points, and giving people a tool that was easier than the workaround.
AI needs the same treatment, sooner — because the volume of data moving into these tools is large and the habit is already formed.
What actually leaves the building
The risky prompts are rarely the obvious ones. The same categories show up across environments:
- Customer data. "Summarize this ticket thread" carries names, financials, and sometimes health or payment details into a third party's logs.
- Intellectual property. Proposals, pricing models, roadmaps, source code — pasted in for a rewrite or a review.
- Credentials. An engineer asking a chatbot to debug a config file that still contains API keys. This one happens somewhere in most organizations every week.
- Regulated data. Anything your SOC 2, HIPAA, or contractual obligations say must stay put. Public tools rarely satisfy those terms, and "we did not know" is not a defense an auditor accepts.
The risk is not only retention. A public tool is a third-party data flow with no data processing agreement, no logging you control, and no access review — three things your own compliance evidence needs you to be able to demonstrate.
A policy people actually follow
Most AI policies fail for the same reason most acceptable-use policies fail: they say "don't" without saying "instead." A workable policy has three parts.
- Classification. One page: what may go into an AI tool (public information, drafts with no client data), and what may not (personal data, credentials, intellectual property, regulated records).
- A sanctioned path. A company-approved tool where prompts do not train external models, retention is configured, and usage is logged.
- Monitoring, not policing. Egress visibility on the network and in the proxy, so a violation surfaces as a coaching conversation rather than as a finding discovered by your auditor.
The litmus test: if you pasted this prompt into the tool and then had to hand the transcript to your largest customer, would you be comfortable? If not, it does not go into a public tool.
Private AI: the same capability, inside your walls
The capability gap has closed. Open-weight models now run on hardware you own — in an office closet, or in a rack at our facility — and handle a large share of the everyday work people were sending to public chatbots:
- On-premises models served to your team over your own network, so prompts and documents never leave your environment.
- Agents deployed sandboxed and logged, with permissions that match the person driving them rather than a shared superuser account.
- Policy gateways for the online models you do use, so requests are redacted before egress and every call is recorded.
The pattern we recommend is straightforward: sensitive workloads on private infrastructure, everything else classified by policy, and logging across all of it — so the question "where did our data go?" has an answer your compliance team can prove rather than estimate.
Curious what is leaving your network today? Ask us to look — we will review egress patterns and sanctioned-tool coverage, and tell you what we found.