Martin Källström
knowledge / philosophy

Privacy & surveillance

Long before "privacy by design" was a compliance checkbox, it was the founding design brief at Memoto and Narrative: a camera that photographs your life every 30 seconds has to earn the right to exist in other people's rooms. Martin's answer was architectural rather than legal — build the camera so it cannot lie about what it's doing, make sharing an active choice rather than a default, and accept that no policy will fully resolve the discomfort of being recorded. A decade later, the same instincts carried over almost unchanged into AI: local models instead of the cloud, self-hosting instead of APIs, and a running worry about what happens when agents hold data no one gave them permission to expose. Two theses anchor the whole page — privacy must be the default state, not an opt-out (Kickstarter ↗), and a visible camera is more trustworthy than a hidden one ▶ 18:32 — and almost everything else here is that logic applied to a new surface, from a shirt clip to a chatbot.

Design as the first safeguard

The founding constraint, repeated across nearly every Memoto and Narrative interview for a decade: honesty and subtlety are the two things that matter most in the design of a wearable camera ▶ 19:56 — it has to read unmistakably as a camera (not a disguised spy device) while staying unobtrusive enough not to dominate a room. Fifty hours of user interviews and iterative prototyping went into finding that balance; early prototypes were circular, with the lens deliberately centered, "primarily designed in a way that made it difficult to conceal — the last thing the designers wanted to make was a camera used to spy on others" (Fast Company ↗).

That honesty requirement shows up as a cluster of concrete choices:

The tension in this design was real, not resolved: the same device that was built to be unmistakable was also, by its own marketing, small enough that "no one would even notice it" (NBC News ↗), and its intended flagship use case — street photography — was explicitly valued because it let you "go through a crowd of people and just be taking photos surreptitiously" ▶ 1:41. Honest-by-design and unnoticeable-by-design pulled in opposite directions, and the company never fully picked one.

Visible beats hidden — but Google Glass had a different problem

Martin's clearest thesis on surveillance aesthetics: visible cameras are more trustworthy than invisible ones, because visibility enables trust ▶ 18:32. A hidden camera disguised as a pen or a pack of gum is "a lot more scary" than a Ray-Ban Meta with an obvious lens, precisely because concealment removes the possibility of acknowledgment or negotiation.

Google Glass, in his account, failed for a more specific reason than "creepy camera": it put bystanders in a position with a need they had no acceptable way to express. "Could you please take your glasses off? It's not something you ask someone, but still like you have a need and you can't express it" ▶ 25:35. Narrative's no-off-switch design was a direct answer to that failure mode: instead of the bystander having to ask, the wearer is meant to notice discomfort and remove the camera unprompted ▶ 18:04. And Glass compounded the problem because the wearer stayed visibly distracted — "I use it as a screen. I'm looking at Reddit when we're talking" ▶ 18:29 — whereas a camera alone doesn't pull attention away from the person in front of you. (Martin also floats a second, less flattering explanation: Glass's "cyborg uncanny valley feeling" may have done as much damage as the social-protocol problem ▶ 20:24.)

In practice, the design mostly worked: Martin wore the Clip on and off for about five years, and was asked to turn it off only twice ▶ 19:27 — though he adds that he usually felt uncomfortable before the other person did ▶ 19:12. Reviewers were split on whether the honesty strategy actually landed: some found the Clip more discreet and "less iconic than Glass, so it slides under the radar better" (CNET ↗); others judged it "actually a worse experience than Google Glass since the Clip is entirely passive... there's no way of knowing it's taking a picture" (iStartedSomething ↗).

Ambiguity is a signal, not a defect

Public reaction to Memoto was never a clean split between fans and critics — "everybody are both intrigued and sort of a bit creeped out about it" ▶ 25:33. Martin's read on that: that ambiguity is itself the evidence that you've found something genuinely new, not a problem to engineer away — "it's that ambiguity that tells us that we are onto something completely new. It's something that people are not used to" ▶ 25:58.

Acceptance, in his account, is mostly a function of context, not the device: "it depends very much on the context or setting" (Fast Company ↗) — people in public are already used to being observed, but "if they're in a domestic setting at home, maybe they want to be able to relax." Purpose-clarity does most of the work too: "if you see police officers wearing body cameras, it makes sense for them to capture their work," but a mysterious clip on a shirt has to earn that same legibility. Most bystanders in practice reacted with curiosity rather than alarm — "most people didn't notice or care" (Digital Trends ↗), and when confronted directly, people "usually turned out to be more interested in the technology... than they were concerned about their privacy" (SlashGear ↗). Martin's consistent bet was that norms would catch up on their own — "a protocol for when to capture and when to share will evolve naturally once devices such as this become mainstream" (GMA News ↗) — a bet futurist Paul Saffo later echoed: lifelogging devices "could make lifelogging so common in the future that few will mind" (SF Chronicle ↗).

The counter-evidence is just as present in the record. A reviewer's wife was "instantly turned off by the device when I got home, and insisted I turn it off" (CNET ↗); another called the constant capture of images "creepy" (Engadget ↗); a wearer's kindergarten banned the device outright over concerns about constant photography (Spiegel ↗). And there is real, sober academic backing for the caution: peer-reviewed bystander-privacy research on lifelogging concluded the field needs privacy-enhancing techniques "embedded into the design," dependent on "context of use, scenario... and content" (ResearchGate ↗).

The camera is honest, but the room and the law aren't always ready

Being honest about recording doesn't make it legal or welcome everywhere. Property owners can bar photography on their premises outright, though they can't compel deletion of what's already captured or seize the device ▶ 20:25. Legal scholars raised a sharper concern about the Narrative Clip 2 specifically: its inconspicuous form "could be considered a way to eavesdrop on confidential communications without the consent of the participants — a violation of California [wiretapping] law" (SF Chronicle ↗), because unlike a raised phone or a DSLR, "people don't yet recognize this as being a recording device." Even Narrative's own team acknowledged the device kept capturing in venues where photography is explicitly forbidden — cinemas, theaters, galleries — with no mechanism to stop it ▶ 10:11. Wearable-camera pioneer Steve Mann was reportedly refused permission to fly with his equipment years before Memoto shipped ▶ 6:53.

Martin's own answer to that gray zone leaned on individual responsibility rather than a technical fix: if you accidentally record somewhere inappropriate, "you will be a lawful citizen and delete them when you get home" ▶ 7:51. Memoto's published guidelines were explicit about consent as a social norm layered on top of legal permission: "if someone asks you not to use your Memoto camera — then please don't. If someone doesn't explicitly ask you, but you have reason to believe that the place or the context is inappropriate for photographing — then please don't" (Kickstarter ↗).

He also argues privacy isn't the only value in tension here — documentation is itself "an expression of freedom of speech. Sometimes you really do have the right to take someone's picture without their consent. If someone is stealing your bike or harassing you, then by all means take their photo" ▶ 13:45. He points to a train conductor who wanted the camera as protection against regular harassment on his commute — "this would transform my life" ▶ 11:34 — not to document strangers, but "just being able to point to it and say, I have a camera here" ▶ 12:01 as a deterrent.

From cameras to data: privacy in the AI era

The instincts formed on a $279 shirt clip carried straight over into how Martin thinks about AI. Local AI running on your own hardware — not the cloud — represents the future for privacy and autonomy: "it is actually a kind of cool feeling to know that you're talking to an LLM running on your own laptop locally," with Ollama plus a local frontend, no GPU or internet required, usable on a plane or "in the forest" ▶ 33:02. At Multiply, that principle became a business model: self-hosting lets you offer companies and individuals something an API can never promise — full ownership of both the software and the data, "and that benefit is impossible with APIs" ▶ 29:28. Customers coming from ChatGPT worried about training on their data even where the terms of service already prohibited it ▶ 28:16 — the anxiety outran the actual contractual protection, which itself became a sales argument: enterprise customers' fear of company data "escaping out into the wild through the use of AI models" pushed Multiply toward ISO certification and became a competitive differentiator in its own right ▶ 26:27.

The mechanics of that protection are concrete, not just contractual: if sensitive data never enters a model's training set, it cannot later be extracted from it — "the only way for that not to happen is to not have that data in the model" ▶ 27:15 — which is why one of Martin's government contacts builds AI proofs-of-concept on synthetic data instead of real records, fine-tuning on "reports that is just made up, but they're still the same structure" ▶ 12:21. The flip side of mainstream AI adoption is that the human side of every conversation becomes an attack surface: "you can no longer sort of trust the human part of the context," so "really highly resilient and robust instruction following" and strong guardrails become non-negotiable once malicious users start trying to jailbreak systems into revealing business secrets ▶ 23:27 — including mechanical exploits as crude as feeding a model 1,000 letter A's until it starts hallucinating from "some primordial soup" ▶ 24:43.

A newer, less solved problem: agentic AI that holds personal data and also acts on the open web creates a leakage risk no one has fully closed. Martin poses it directly — if a personal AI companion "gets to know me... how can I make it physically impossible" for someone else to prompt it into revealing his "deepest, most secret desires" unless they're authenticated as him ▶ 29:43? Retrieval-level access control is theoretically solvable; a trained model quietly memorizing what it shouldn't is not ▶ 31:00. The same worry surfaces closer to home, about ChatGPT's memory feature in a shared room: "what if you and I have a work meeting and I bring in my AI assistant... and you take the opportunity to ask my AI... what does Martin talk to you about when there's no one else around?" ▶ 22:07.

Trust in the surrounding information environment isn't holding up well either. Martin recounts a Swedish ambassador to Germany targeted by a Russian-made deepfake meant to "diminish her credibility" during a live political debate ▶ 18:01, a finance worker recently tricked into wiring $25 million after a deepfake Zoom call with a fabricated CFO ▶ 20:23, and — closer to home — Narrative itself being targeted by an old-fashioned social-engineering attack, where scammers phoned around the company mapping its approval chain before spoofing an email from Martin to the controller to redirect a payment ▶ 23:10. His conclusion is that trust in digital communications will need new social protocols layered on top of technical ones — better authentication, digital signatures — not a purely technical fix. He traces the anxiety back further than deepfakes: a friend's grandmother refused to trust a newly installed telephone fifty years earlier, on the same underlying logic — "if it can imitate your voice, then it can make you say anything" ▶ 14:59.

AI agents should never be allowed to impersonate humans, and that should be regulated — if an AI can respond to a text claiming to be a person, "people can't tell if they're talking to the human or a 'digital ghost,'" and disclosure of agency should be hardcoded, not optional ▶ 5:39.

There's a psychological wrinkle underneath all of this that Martin returns to more than once: people anthropomorphize conversational AI, and that makes them share more with it than they would with a faceless company doing the same data collection — "we look at ChatGPT as human in some sense, and therefore we're more... accepting of it knowing us. But Google is just a corporate, huge corporate entity... it's more like creepy when it knows us" ▶ 4:06. He's watched this play out among friends who were once "very privacy concerned with ChatGPT" and let that go once memory arrived ▶ 21:14, some of them treating ChatGPT as a personal therapist, "probably telling ChatGPT more personal things than they tell most friends." That comfort has a real cost: AI personalization can fool the emotional part of a person even while the rational part knows it's talking to software — "it doesn't even have to fool all of you. It can fool the part of you that responds to love and care" ▶ 25:48, which is why Martin argues society needs AI literacy the way it eventually needed social-media literacy — learning that "this is a research agent," "this is not a therapist," the same way people learned not to trust a single news source ▶ 26:17. Set against that emotional pull, resistance to sharing files with AI may be a bigger obstacle than engineering: the ChatGPT–Google Drive integration still hasn't gone mainstream, suggesting "there might be other hurdles than just technical" ▶ 31:27.

On the deeper question of who owns the data AI is trained on, Martin has strong, sometimes uncomfortable views — copyright shouldn't apply to a model learning from images any more than it applies to a human artist learning from other artists — worked out at more length on Data ownership & AI ethics. Here the more relevant note is his own admission that the principle frays at the edges: he holds the copyright position confidently, "but I feel conflicted" the moment the data in question is health records or brainwave data rather than pictures and text ▶ 18:41 — and he's genuinely unsettled by what a combined dataset could do: lifelogging camera footage annotated with brainwave data would let you compare "this is what the person is watching. This is what they are thinking... on a massive scale" ▶ 25:15, calling neural data the category most likely to enable something "very cool and awesome and really scary" ▶ 24:41.

Worth remembering