3AM Lab Notes
Read-Only Still Needs Threat Modeling
Observation is not harmless when the observer can summarize, correlate, and leak.
I’ve always been a fan of John le Carré’s spy novels and films. Sure, they are not big blockbusters like James Bond, but I would think most spy work is unglamorous. It is almost always the smiling dude no one suspects, or the wallpaper lady everyone forgets. Quiet. Helpful. Always watching.
‘Course, if you are dealing with public information, there is nothing to worry about. Everyone has access to that data. But when you get into information that matters, well, that is when people should sit up and wonder: who is on the other line?
What does the model see? Where is your prompt processed? What information does the model have access to? These are the questions an organisation has to confront in changing times.
Policies that were simply “keep it encrypted, locked, and hide the key” are no longer a valid strategy when models smarter than most people are at your beck and call. These PhD-level models can consume more data, faster, and draw more insight from your closely guarded information than a whole team could. The game’s changed. The security does not.
The moment an agent absorbs the data, it can summarize, correlate, and leak that information.
The agent is a third party gobbling down your data. You need to design with that threat model in mind. If you do not, it is no longer a question of if your data goes public. It is when.