A Cybersecurity Awareness Month series from Thompson Coburn’s Data Governance, Privacy and Cybersecurity practice group
Throughout Cybersecurity Awareness Month, Thompson Coburn is examining how organizations can put AI to work while keeping control of their data. So far, the series has covered what organizations retain, why retained data becomes reachable once an AI tool is connected, and whether an organization has the authority to use the data it holds for a new AI purpose.
In our first installment, we recommended a simple step before any rollout: run the AI tool under an ordinary user’s account and review what it returns. We return to that step now. What the test finds is often uncomfortable, and the organization needs a plan for those findings before the test begins.
In this fourth installment, we ask: When an AI tool surfaces information people were never supposed to see, who owns the fix?
A Pilot Turns Up More Than Anyone Expected
Consider a hypothetical. A regional lender is piloting an AI assistant connected to its document management and collaboration platforms. In the first week, a loan officer asks it to summarize her team’s notes on a commercial borrower.
The summary is accurate. It also cites two sources she did not expect: a spreadsheet of employee compensation from a human resources folder, and scanned loan applications from a branch that closed four years ago.
The assistant did what it was designed to do. Years earlier, someone shared the HR folder with everyone in the organization so managers could reach a benefits form. The closed branch’s files were never moved, restricted, or deleted. The loan officer always had access to both. She simply never had a reason, or an easy way, to find them.
Each item raises a different problem. The loan applications are customer information. The GLBA Safeguards Rule requires the lender to limit that information to employees who need it for their work and, subject to limited exceptions, to dispose of it within two years after its last legitimate business use. The compensation spreadsheet is not customer information, but it is confidential employee data the organization has its own reasons to protect. The loan officer’s discovery raises two separate issues: how many other people share the same access, and whether anyone, through the tool or directly, has actually opened these files without a business need. The first is a permissions problem to fix. The second may be an incident.
Several decisions follow, and they depend on one another. The organization must first decide whether to continue the pilot or pause it. That choice shapes how it corrects the permissions, from a quick restriction on what the tool can reach to a broader cleanup of the underlying sharing settings. It must also determine the potential scope of unauthorized access and what obligations may follow. Each of these decisions needs a designated owner. If no one holds that responsibility yet, the organization needs to assign it now. The rest of this piece takes these decisions in turn.
Why Access Gaps Become an AI Question
Many enterprise AI assistants are built to respect existing permissions. A user can retrieve through the tool only what that user could already open.
That design can be reassuring. It can also be the source of the problem. Permissions accumulate over years: folders shared broadly for convenience, links open to anyone in the organization, groups that grew as people changed roles, and access no one removed.
Organizations can also limit the tool itself. Inherited user permissions set the outer boundary, and organizations can draw a narrower one for the AI. Many enterprise AI tools let administrators choose which repositories the tool connects to or indexes, exclude locations that hold sensitive categories of information, and, where the platform supports it, block the tool from drawing on content labeled confidential. As we noted in our second installment, the tool should reach only the data its intended use requires. Tools that act through their own service account or integration, rather than a user’s credentials, need their own permissions scoped the same way.
Tool-level limits narrow what AI can surface. They leave in place what users can open on their own: the loan officer in our example could still navigate to the HR folder directly. Effective control usually requires both layers, limits on the tool and correction of the underlying access.
What Existing Law Expects of Access
The obligation to limit access is informed by existing legal frameworks. For financial institutions and the many colleges and universities covered by the GLBA Safeguards Rule, the Rule requires implementing and periodically reviewing access controls that limit authorized users to the customer information they need to perform their duties. FERPA separately requires institutions to use reasonable methods to ensure school officials reach only the education records in which they have a legitimate educational interest. NIST’s Cybersecurity Framework 2.0 calls for access permissions that are reviewed and that incorporate least privilege.
Past enforcement informs how regulators may approach the question. In 2023, the HHS Office for Civil Rights reached a $240,000 settlement with Yakima Valley Memorial Hospital after 23 emergency department security guards used their own credentials to view the records of 419 patients without a job-related purpose. The hospital’s settlement did not contain an admission of the alleged violations. Financial regulators have taken the same view. The SEC and the New York Department of Financial Services have each penalized financial firms whose controls allowed access to customer information beyond what was needed, including where a firm had not tested those controls or had not acted promptly on a known vulnerability.
While these were not AI cases, they turned on the questions a AI deployment tests will raise: whether these users, or the AI tool, needed access to these records at all.
Unauthorized access by an organizational insider may constitute a security incident. Correcting overbroad access is a control fix, usually handled by IT and the data owner. Determining whether anyone actually opened records without a business need is a separate inquiry that may carry assessment and notification obligations. That inquiry should run through the organization’s incident response process, with counsel involved, rather than be settled informally by the team adjusting permissions.
Test the Tool’s Reach Before Rollout
A reach test is a structured preview of what the tool will surface for ordinary users. It is a sample, not a complete permissions audit, but it shows quickly where the largest gaps are. A useful test has four features.
- Realistic accounts. Test with accounts that reflect actual roles: a typical employee, a recent hire, a contractor, and someone who changed departments. Administrator accounts see too much to tell you anything.
- Targeted prompts. Ask for the kinds of information that should be tightly held, such as compensation, Social Security numbers, disciplinary records, loan or financial aid files, health information, and credentials. Keep the prompt list with the test plan so the test can be repeated.
- Location, not just content. For each unexpected result, record where the item is stored, who can reach it, and how that access was granted, such as a sharing link, a broad group, or inherited folder permissions. Those details tell the remediation team exactly what to change, and they give the data owner what is needed to decide whether the item should be kept at all.
- Controlled results. Treat test results as sensitive data in their own right. The output can contain the same records the test is meant to find, and the report itself maps where sensitive information sits and who can reach it. Apply the controls the organization uses for its other sensitive data: limit who can access the results, restrict their use to remediation, store them in a protected location, and set a retention period. In reports, reference each item’s location rather than copying its contents.
Platform tools can supplement the test. Some vendors offer reports on broadly shared sites and sharing links. Those reports show the scale of the problem; the reach test shows what a user will actually see.
Commit to acting on what the test finds. A reach test creates a record of what the organization knows about specific access problems. A documented finding left unresolved can be harder to defend than the original gap, because it shows the organization knew and did not act. Before the first prompt, confirm that owners, resources, and a timeline for remediation are in place, and record how each finding is resolved.
Decide Who Owns the Response Before the Test Begins
In our first installment, we recommended deciding in advance where findings go. In practice, that means assigning five responsibilities before anyone runs a prompt.
- Report: Where does a finding go? Name the person or function that receives findings from the test and from users after launch. Give users a simple way to report what they should not have seen, and make clear that reporting is expected, not risky.
- Pause: Who can stop or narrow the deployment? Someone needs authority to pause the rollout, exclude specific locations from the tool, or proceed with conditions. Without that authority, deployment tends to continue by default while the issue is discussed.
- Assess: Was anything actually accessed? Determine whether access logs show records were opened by people without a business need. If so, route the matter into the incident response process. That assessment should be separated from, and not delayed by, the permissions fix.
- Remediate: Who corrects the underlying access? Excluding a location from the AI tool is a fast containment step, and often a control worth keeping, but the people who had access still have it. The lasting fix belongs to the data owner and IT together: correct the sharing settings and delete what the retention schedule says should already be gone.
- Verify: Who confirms the fix, and when is it tested again? Re-run the same prompts under the same accounts and document the result. Then fold the reach test into the organization’s periodic access reviews, so the next new feature or new hire does not reopen the gap.
These roles can sit within existing structures. For institutions subject to the Safeguards Rule, the Qualified Individual and the existing access-review cycle are natural homes. The point is that each responsibility has a named owner before the findings arrive.
Find the Gaps Before Your Users Do
An AI rollout is often the first time an organization sees its permissions the way a curious user could. That visibility is valuable if the organization is ready to act on it. Without clear owners, the same findings become a stalled pilot, an unexamined incident, or a gap that reopens with the next feature release.
Run the test before the rollout, and decide before the test who will act on what it finds. That is how organizations put AI to work while keeping control of their data.
Next week, we close the series with the organizations helping businesses deploy AI, their technology vendors, and what it takes to confirm that vendor commitments are being kept.
Follow along as we continue our Cybersecurity Awareness Month series, Put AI to Work. Keep Control of Your Data. To discuss how these considerations apply to your organization, reach out to Thompson Coburn’s Data Governance, Privacy and Cybersecurity team. We welcome the opportunity to continue the conversation.
