ChatGPT 5 Can Code a Robot, But Not My Newsletter

You probably saw the news yesterday that OpenAI released ChatGPT 5, an event that was hotly anticipated by nerds like me who live in the AI echo chamber.

I wanted to hear directly from the LLM’s mouth what ChatGPT 5 thought of itself, so I asked: Explain to me in simple language the differences between ChatGPT 5 and the previous models.

The very first item on the list was this:

Smarter, Faster, More Accurate
GPT‑5 delivers quicker responses with noticeably fewer mistakes. It's built to reduce hallucinations and provide more reliable answers across writing, coding, and complex queries.

The Foolproof Process Fools Itself

This gave me hope as I’ve been having an epic struggle to get an LLM to accomplish something I thought AI would be really good at: creating article summaries for my weekly “In the Loop” newsletter.

My process and expectations are very simple (or at least I thought they were): Each week I aggregate relevant AI-related news, put the links into an LLM and ask for an editorial style summary of the article.

At first, my instructions were pretty simple, but over time they’ve become much more specific and directive. They now span more than 50 lines, covering everything from tone to formatting to QA.

To me that seems like plenty of information, but each and every week I fight with my LLM to get it to produce summaries in the style I’ve instructed it to. I got so frustrated with ChatGPT’s inability to complete the task appropriately that I moved the operation over to Claude – which has only been slightly better.

Before beginning each week’s newsletter, I ask the LLM to re-state its instructions so I know it has the right information in its brain.

And still there are issues. Word counts are wrong. There’s no headline. The links aren’t embedded in the flow of the content. MattGPT once even entirely made up a story summary about a podcast episode that pitted tech bros against farmers.

What is going on here? Aren’t LLMs supposed to be great at 1) following instructions and 2) summarizing?

Will ChatGPT 5 Finally Ease My Pain?

I started the day today filled with hope. The release of ChatGPT 5 and its confident assertion that it was the most “reliable” model yet had me giddy with anticipation.

I decided to give MattGPT one more shot at my newsletter, so I started a new Project, fed the 50+ lines of instructions into the system, asked it to review the instructions for me, then gave it links to two stories and asked it for a combined summary.

It didn’t go well.

Notice anything? No headline, written a news style rather than an editorial style, no links to the articles within the flow of the content. In other words, it failed utterly. After literally just re-stating its instructions.

Forgive me for being completely confused and frustrated by this. OpenAI has been trumpeting its new memory capabilities, yet MattGPT is telling me it doesn’t have an “always on” memory of my project. And I love that it can come up with a metaphor that might help me understand better but can’t follow the most basic instructions.

Why This Matters Beyond My Newsletter

So why am I sharing all of this? Partly because I need to vent. I’ve been fighting with these LLMs every week for three months and can’t believe I’m still having to fight with the best models in the world to do something pretty straightforward.

But the other reason I’m sharing is that I live, eat, sleep and breathe AI; it’s my business and something I think and read about all day, every day. Yet I still sometimes struggle to use these tools effectively. Are they not as capable as the hype machine would have us believe? Definitely. Does the “release often and break things” approach these companies follow lead to suboptimal software on occasion? 100%. Has something happened to the core training data at the center of these models to make them less reliable? I have no idea.

But I do know that these systems are imperfect and can be immensely frustrating to use. For someone like me, it’s easy to find a workaround – try another model or have a sarcastic sparring match with MattGPT.

Advice for the Frustrated

But for others who are just starting to use these technologies, the challenges of using them can lead them to throwing their hands up and giving up on AI.

And with the world increasingly reliant on AI for so many aspects of our daily lives, we simply can’t allow that to happen. My suggestion? Try these tools out every day and don’t assume that, just because something doesn’t work for you the first time, it’s not necessarily user error. It could be that your friendly neighborhood LLM simply has no idea what it’s doing.