I’ve tried out most of the AI LLM products at this point. For the most part I haven’t found a use for them beyond short term entertainment. Recent releases of Claude and Gemini have gotten good enough to perform some busy-work coding for me, but anything complicated needs too much review still. But today I stumbled on an actually valuable use-case for writers.
The new Gemini 2.5 Pro preview performed substantially better for multi-file coding projects. And I was impressed with the Deep Research feature: The output stayed on track and produced a fairly intelligent report on the new MCP (Model Context Protocol) and how it could be used by my employer. The output stood out to me because it seemed like the AI’s attention didn’t wander like earlier versions of Gemini and Claude tend to do – they start out strong but end up repeating themselves or wandering off the point or spewing inane blather with no substance.
So, I uploaded my half finished NanoWriMo novel from 2009 to see if it could follow the story.
Big Gemini Improvements
Last Fall I tried Notebook LM and its “Audio Overview” that generates the well-known AI podcast / discussion on whatever you give it. I’d uploaded that same novel since it was material I was very familiar with. I wanted to see how well the AI’s understanding matched mine. That iteration of Gemini struggled to follow the plot and rationalize character actions and intentions during the discussion. It was impressive, but during every attempt I found it made substantial errors describing the story, often getting key points exactly backward.
Today I uploaded the manuscript to my Gemini Advanced account and picked the 2.5 Pro Preview model that had impressed me. I discovered the “Audio Overview” feature – not in Notebook LM but the main Gemini interface. So I requested it generate an audio overview. I presume this overview has somewhat different goals than the Notebook LM “podcast”.
What I got back was a very straight-ahead retelling of the plot, with the “hosts” making their usual asides and jokes, but toned down from the Notebook LM style. They did a nearly perfect job. They didn’t get anything wrong. It went from somebody who dozed on and off while watching a movie trying to tell you about it the next day, to someone who had a triple espresso and took lots of notes.
The really interesting and useful part? They pointed out a number of continuity errors I’d made. This was a first draft after all. Nevertheless I didn’t know these errors existed. Once pointed out I confirmed they were in the text, and fairly easily corrected.
To find these discrepancies I would have needed to read and reread myself, or have another reader go over it. Either way a substantial investment of effort.
Once I was clued in that the AI was good enough to spot these types of problems, I asked Gemini explicitly to tell me any continuity problems. It identified one or two mor than the casual audio overview, with just one point being a misunderstanding: The discrepancy is there on purpose and a human would realize it demonstrates a characters duplicity, not a factual contradiction.
Notes from Notebook LM+
Then I tried the same thing, but with the new Notebook LM which I presume is based on a newer model, maybe 2.5. Pro. The plot summary was more rushed but pretty accurate still. The hosts didn’t call out any continuity problems – the discussion seemed to be a bit too high-level to need to explain them. However, the chat for the notebook gave excellent point-by-point summaries of possible continuity problems, perhaps better than the plain 2.5 Pro Preview did. You need to ask it to distinguish apparent contradictions or erratic character behavior from discrepancies that look like simple mistakes.
Are there continuity problems that appear to be mistakes by the author?
Here are some notable points that could be viewed as continuity issues or discrepancies that don’t seem to serve a clear plot-revealing purpose within these excerpts: 1. Nicola’s Cellphone Access:
◦ Nicola initially leaves her cellphone in her car to avoid distraction.
◦ Her car is destroyed in the explosion at Claremont.
◦ She is told Featherstone’s assistant, Tiffany, called her cellphone, but Nicola states the FBI still has it and it’s likely broken from the explosion.
◦ She mentions to Laird that the FBI still has her phone and it’s probably broken.
◦ Despite this, in later sections, Nicola receives a call on her phone
◦ She uses her phone to call Laird. She also uses her cellphone in the hotel lobby. She turns off her phone to save battery.
◦ This presents a clear contradiction: Nicola repeatedly states her phone was destroyed and is held by the FBI, yet she is consistently using it in later scenes. This appears to be a discrepancy in the continuity of which characters possess which items.
Early on I mentioned that she bought a cheap new phone, but reviewing the story it seems like the recruiters would not have a way to know her new number. The loss is used to explain why her employer could not contact her, and that part works. But then in the next section she can get calls from people who could only have her old number. The AI is sort of wrong about who has which items, but it identified a lack of continuity about the phones. Buying the phone is mentioned without saying she had the number memorized for so and so, and what she did about the rest of her lost contacts.
2. Nicola’s Access to Her Sealed Apartment:
◦ After being released from custody, Nicola’s apartment is described as having been “thoroughly searched, making a mess of it”.
◦ Laird later tells Nicola he will call the police department to arrange for the apartment to be “unsealed” , implying it was still officially closed off by the authorities after the explosion and search.
◦ However, in Section 3, Nicola is suddenly back in her apartment. She hears the front door open and realizes she forgot to lock it. She describes walking through the living room and guest room. When talking to Tycho later, she mentions her apartment door was open when she got into the elevator, indicating she had been inside.
◦ This is an unexplained gap: The sources do not provide an explanation for how Nicola regained access to her apartment if it was, as Laird stated, still sealed by the police/FBI.
This sequence lacks the necessary connective tissue to maintain continuity.
The sequence is correct in its order, but I forgot to make it clear when she learned she could return. The part where she initially sees it messed up doesn’t make it clear she’s not allowed in even though it’s implied. Finally the last part where Ferris surprises her is muddled, but could be fixed easily. It was a good catch by the AI.
3. Laird’s Hotel Plan and Timing:
◦ Laird tells Ferris (the attacker) that he plans to rent a hotel room and tell Nicola to meet him there. He also tells Ferris he instructed Nicola not to stay where she was and not to trust the people helping her.
◦ Ferris and his team then stake out the parking ramp where Nicola’s car was left, based on this information and hoping she will return there.
◦ Nicola arrives at the Radison hotel. She asks at the desk for Laird, but the clerk says no one is registered under that name. Nicola thinks Laird said he reserved a room but maybe hadn’t checked in.
◦ Nicola calls Laird, who tells her he is there but used his brother’s name (an explanation Nicola finds unconvincing.) He gives her a room number (418.)
◦ Crucially, just before Nicola leaves her room (2819) towards the elevators, Laird calls Ferris to tell him to expect her in the next few minutes. This call directly precedes Ferris’s ambush attempt.
◦ This series of events presents a significant continuity problem linked to Laird’s actions: ▪ Why would Laird tell Nicola to meet him at a hotel room that she cannot find registered under his name? ▪ His explanation for the name used is flimsy and doesn’t align with standard hotel procedures or expecting a specific person to find you. ▪ Most problematically, Laird’s timing is contradictory and highly suspicious. He tells Ferris Nicola will be moving shortly and is arriving downstairs at the same moment or just before he presumably expects Nicola to arrive at his room on a different floor (418 vs Nicola’s 2819). This timing makes no logical sense unless Laird intended for Nicola to be intercepted before reaching him, which aligns with his instruction to Ferris to “make her disappear. Forever.”. While the plot reveals Laird is involved, the mechanics of his communication and arrangement with Nicola regarding the hotel room seem contradictory and poorly explained, serving as a potential authorial slip in coordinating the antagonist’s plan with his interactions with the protagonist.
These instances stand out as potential breaks in the narrative flow or logical consistency that are distinct from the plot points designed to surprise the reader or reveal new information.
The AI is correct that this whole sequence is more complex than necessary, and doesn’t ring true to how things work. I had something in mind for the room reservation name mix-up but never wrote it. It could be done – and more suspensefully – with half the words. On the last ppoint, the AI shows it’s lack of real understanding: Of course Laird’s whole goal is to eliminate Nicola which is the reason for luring her to an empty room in the first place. The AI has no theory of mind.
Spotting likely plot holes and small discrepancies in a novel length text in minutes is actually valuable.
I tried asking some other questions, particularly about possible endings for the story. There, the AI failed to impress. The answers are logical but not particularly interesting.
Some Problems Remain
One annoyance with the audio overview in the plain 2.5 Pro app was that the discussion essentially cuts off after touching on a little less than half the text I uploaded. It seems like the audio overview has an upper limit for time, and the default level of detail in the discussion simply couldn’t fit all the detail in the time (39 minutes.) I didn’t give any special prompt for the audio overview,. Even with the Notebook LM it now seems to stop the discussion prematurely, seemingly because it has talked in too much detail and runs out of time. However, the Notebook LM version does better and you can prompt it with a note on how to conduct the discussion.
Aside from the audio, the questions Gemini can answer and the timeline it constructs are very valuable for improving your draft. It doesn’t lose track of any content, at least in the 50,000 to 100,000 word range. If you notice it getting confused it may be an indication your story isn’t very clear, even if it is clear in your mind. It could be a sign that the story in your head didn’t make it into the text as well as you believed.
I found a couple of places where Gemini drew the wrong conclusion when a typical reader would not, and there was no good reason except AI making up stuff as it can from time to time. In one place, the AI failed to impute motives to a character and so misunderstood a situation a human reader would be able to follow. Even so, I probably need to review that section to see if I dropped enough clues.
The AI reading of your text will be pretty linear and can miss some context clues. Even so, the ability the reading comprehension of the new model compared to last December is huge.