Cognitive security, epistemic erosion and AI detection tools

How I'm responding to the flood of AI-generated content everywhere all-at-once

Hi there,

I discovered through my extensive use of AI after quitting my corporate job a year and a half ago that my understanding of AI risk was not as deep as it should be. This is a gap to close on my part given the enormous consequences that AI will have for all of us.

One thing I’ve been doing is deeper reading, listening and thinking about these topics. One of the benefits of having the time to focus on this is the deep reading: you read the PDFs, you read the journal articles, you read some more books, you listen to some more podcasts and you take notes.

The mental models you have from your own education and life experience map to these ideas and concepts you discover in this learning process.

You see, I worked in highly regulated financial services technology and change for over 12 years. This informs the way I think about topics like risk and regulation. It also changes the way I think about how regulation actually operates in practice as opposed to in textbooks.

The business of banking is risk management. When you have been inside the beast, you know what it means to identify, manage, and mitigate risks. You understand the processes and controls and systems and audit trails and governance layers and expertise and subject matter knowledge that is important.

As I’ve watched the complete mindshare takeover that AI has had, in parallel with a lot of my deeper reading and research on topics around AI risk, it’s become kind of funny to see how cosmetic much of the public discussion is. I knew I didn’t know things, but I underestimated how little I knew about this space and how much more reading and reflection I have to do.

The urge to simplify, the urge to focus on short sentences that gloss over deep reservoirs of academic, expert and practitioner thought. The lack of interest in doing something like reading a model safety card, but feeling comfortable to make a claim about what AI can or can’t do based on something that happened over a year ago.

This is one thing which led me to reduce the level of reliance I had on AI in content creation. I use it often, especially for research, discovery, and editing. Using voice for dictation, like I am doing when I am writing this newsletter, is one of the only points of differentiation you and I have left.

If we aren’t creating content ourselves, if we aren’t leveraging our own stories and experiences, if we aren’t packaging them for our community in a way that they can enjoy or hate at the same time, then what are we even doing here?

One of the main areas of AI risk that my current research and reflection focuses on is the rise of AI-generated content. This presents cognitive security and epistemic erosion issues that relate to not knowing whether something is AI-generated or human-generated or what level of AI assistance was involved.

Is it your idea? Is it Claude’s idea? Is it DeepSeek’s idea? Increasingly as AI models improve, being able to detect this and then make decisions about what ideas you are going to let yourself get exposed to matters deeply.

From my own use of AI tools and writing experience I can generally pick up if something has AI “tells” but this is becoming harder as adversarial evasion behaviours rise and the sheer volume flooding social media feeds becomes overwhelming.

I’ve noticed and experienced the absolute explosion in AI-generated content to the point where it has become unbearable. I have taken steps to reduce my level of exposure to pure AI-generated content. I’m also reducing my own overall AI usage in content creation.

This includes installing AI detection tool Pangram, muting, blocking, unfollowing, and unsubscribing an awful lot of folks across Substack, X, and LinkedIn. It’s nothing personal. I get that AI is useful for people just like it is for me. It can help us with editing or make infographics. It doesn’t need to write the whole article!

My feeds were totally saturated with too much noise. Even outside of my terminally online social media usage, I used to use Feedly to read RSS feeds and have drastically reduced my use of that tool.

I am reviewing more written content elsewhere to check if my own priors around what AI content looks like are validated by a capable AI content detection model. Pangram reports a false positive rate of 1 in 10,000. If something is flagged by Pangram it’s likely that it is actually AI-generated or AI-assisted in some form.

Does this mean I don’t think there are enormous benefits to using AI? Of course not. I think I’m pro-AI and focused on the positive outcomes in our society over the long run from things like solving mathematical frontier problems, complex engineering and technology problems, medical innovation, scientific innovation.

All of these things I think were already starting to happen in that previously pre-GPT domain of machine learning and analytics. This was delivering results under the hood. Because people don’t see it in their daily life, they don’t have a framework for how much more of our world is already automated and improving through use of better technology.

In the past, I have definitely used AI to help generate content. I want to share my personal voice. It sounds silly, but I don’t think I would have come around to this point of view if I hadn’t used AI so much already and experienced the failure modes.

I’ve always had my own ideas and research avenues which have formed the basis of everything I’ve done. I haven’t just asked, “Generate me 90 days of LinkedIn posts”. Points of differentiation are going to be more important than ever.

On LinkedIn, the brand and reputational risk from the flood of synthetic content has made the platform essentially unusable. I think many people don’t realise that tools that work actually exist now. It’s quite easy to unfollow, block, or mute someone.

I don’t want to be in that category, so what I do is going to be either voice dictated newsletters like I’m doing now, voice dictated social media posts, or my videos where I don’t write a script. I write a sentence to act as a prompt and then lightly edit to remove verbal tics and add subtitles.

I might not get high numbers of subscribers and views, but I’m recording for posterity what I think. What my opinion is, right or wrong. I think more people are going to realise that the hyper-optimised AI-generated wave that we’re living through is a phase before more folks start to take measures to protect their social media feeds or the platforms do it for them more aggressively through suspending accounts or further limiting their algorithmic reach.

I’m interested in the pollution of the commons or cognitive security or what information we let in. It makes you appreciate how serious these risks are. Epistemic erosion is a very real risk, where our ability to tell fact from fiction deteriorates because of the deluge of AI-generated content.

Even if a proportion is from good folks “just using it for the research” or “automating my social media posts”, the sheer volume increases the amount of noise we all have to wade through each day.

At the same time, we can’t just talk about the risks and the costs. We need to think carefully about the benefits. What I’m going to attempt to do in this newsletter is share what I’m thinking, reading, and researching. I want this to be a repository for my own take on what’s happening and what I think about it in real-time.

I don’t like tribal labelling of groups or anything like that. I don’t see myself in that vein. I’m just me. I’m writing on the internet. I’m sharing what I think and I want to hear what you think.

What are the big AI risks that we’re facing?

What aren’t we doing that we should consider?

Regards,

Brennan