The comments file

• comments.json
  • The envelope
  • A comment
  • A message
  • An event
  • A cancellation
  • States and moves
  • Elements picked in a message
  • Models
• The step files
• Images
• Hand-overs
• What the server adds

Annotant keeps two kinds of files for a project:

<state> is ANNOTANT_STATE_DIR, else $XDG_CACHE_HOME/annotant, else ~/.cache/annotant. <hash> is the first 12 hex digits of the SHA-1 of the project’s root: each project has its own folder. Deleting a comment deletes its step files and its images; comments.json never holds a step nor an image’s bytes.

The reference for the comments file is skills/annotant/scripts/lib/format.mjs: its version, states, moves and JSDoc types. The widget mirrors them in skills/annotant/widget/src/domain.ts.

comments.json

The envelope

{ "version": 2, "comments": [ /* Comment, oldest first */ ] }

Any other version, or a bare array (version 1), is refused and left untouched: the server stops with COMMENTS_REFUSED old comments format: delete <file>, and the scripts print the same message.

A comment

{
  "id": "b96fae58-5cc3-426b-9632-77f3a7f8f866",
  "route": "/",                              // "*": about the whole app
  "url": "http://localhost:5190/",           // the page and its query, no hash
  "anchor": {
    "section": "Coffee roasted the week you drink it", // the heading above it
    "quote": "",                             // the selected text; "" when none
    "occurrence": 0,                         // which occurrence of the quote
    "element": {                             // when the reader pointed at one
      "selector": "#root > main > section:nth-of-type(1) > h1",
      "tag": "h1",
      "text": "Coffee roasted the week you drink it",
      "html": "<h1>Coffee roasted the week you drink it</h1>"
    }
  },
  "state": "resolved",                       // always the last event's
  "createdAt": "2026-10-10T15:44:37.597Z",   // equals messages[0].at
  "messages": [ /* Message */ ],
  "events": [ /* Event */ ],
  "cancellation": { /* the last stopped run, when there is one */ }
}

The widget finds a quote on the only copy the page still reads around as prefix and suffix say (both sides, in full or by their part in the quote’s own block); else, while the page has occurrences copies, on the one at occurrence; else nowhere: the comment shows as detached, never on another copy.

A message

{ "id": "m1", "author": "reader", "body": "change le titre", "at": "…",
  "sentAt": "…", "notifiedAt": "…" }
{ "id": "m2", "run": "r1", "author": "claude", "question": true,
  "body": "Quel nouveau titre veux-tu ?",
  "options": [{ "label": "Café torréfié la semaine où tu le bois", "description": "traduction en français" }],
  "at": "…" }
{ "id": "m3", "author": "reader", "body": "Café torréfié la semaine où tu le bois",
  "choices": ["Café torréfié la semaine où tu le bois"], "at": "…", "sentAt": "…" }
{ "id": "m4", "run": "r2", "author": "claude", "body": "Le titre est changé.", "at": "…" }

An event

{ "at": "2026-10-10T15:44:57.469Z", "state": "working", "by": "agent", "run": "r1", "agent": "a3fbb6e8bc0e856d1",
  "model": "claude-haiku-5-5", "complexity": "trivial" }

A cancellation

{ "at": "2026-10-10T15:52:19.733Z", "changed": ["src/data.ts"], "steps": 2, "run": "r6" }

at is the time of its stopped event, the reader’s or the agent’s. changed lists the files the stopped run had written, relative to the repository: a failed write is not counted, and one cut short by the stop is. steps counts its tool calls, not its thinking or messages.

States and moves

stateDiagram-v2
    direction LR
    [*] --> open: the reader comments
    open --> working: claim.mjs (new run)
    working --> asking: ask.mjs
    working --> resolved: resolve.mjs, first time
    working --> answered: resolve.mjs, once resolved before
    working --> stopped: the reader stops, or the agent ends without finishing
    open --> stopped: the reader stops
    asking --> open: the reader answers
    answered --> open: the reader follows up
    resolved --> open: the reader follows up
    stopped --> open: the reader sends it again
    open --> resolved: the reader resolves
    asking --> resolved: the reader resolves
    answered --> resolved: the reader resolves
    stopped --> resolved: the reader resolves

The store refuses any other move, and writes nothing: the server answers 409 with the reason (comment <id> is resolved), which the widget shows. Two exceptions, as a click a poll late or twice is no error: a stop on a comment no longer open or working, and a re-send of one no longer stopped, change nothing. ask.mjs and resolve.mjs on a comment still open claim it first (open → working).

The page marks a comment’s anchor (an element’s outline, a text’s rainbow) until its thread is first resolved: a follow-up does not bring it back, unless it picks an element. The element of the reader’s latest message holding one takes the anchor’s place, while the thread is not resolved and when it was picked since the thread was last resolved (the comment itself, before any resolution). The page marks it only when it is on that page. A picked element’s chip in the thread also outlines it while hovered, and scrolls to it and flashes it once clicked.

Elements picked in a message

The reader may pick an element in the page for any message, the comment or a reply (the composer’s pointer button; the widget’s Point at element starts a comment with one): it goes with the message as elements (see A message), alone or with words, images or choices. One per message: another picked takes its place in the message box, and the server keeps the first it can, dropping one with no route. The widget no longer writes anchor.element, which older comments keep. The NEW line for the hooks names them after the comment’s body, the FOLLOWUP and EDIT lines after the reply’s choices: ` elements=[“ Help on /help”], the page added when it is not the comment's. The agent reads them whole in the thread claim.mjs` prints.

Models

Before the plugin’s hooks start a comment’s agent, they read the comment (the reader’s last message, the selected text, the element’s tag and text and those of the elements picked with the message, the route) and asks Claude Code’s small fast model for one label:

models in .annotant.json maps each label to a model alias (haiku, sonnet, opus) or id; inherit keeps the session’s model. Without it, the default is:

"models": { "trivial": "haiku", "standard": "inherit", "hard": "opus" }

A models object is read whole: a label it leaves out inherits, so { "trivial": "inherit" } turns routing off, and the classifier is not asked. A classifier that fails or takes more than 5 seconds leaves the agent on the session’s model. A comment of images alone, which the classifier cannot read, is not classified, routing on or off: its agent starts on the session’s model at a medium effort, with no label. In every case the hooks write the agent, the model it runs on and the label in the state folder (routes/<id>.json, beside progress/), and claim.mjs records them on the working event (ANNOTANT_AGENT, ANNOTANT_MODEL and ANNOTANT_COMPLEXITY override them). A follow-up resumes the same agent, with no new start: it keeps its model and its effort.

The step files

One JSON object per line. A tool writes one line when it starts and one when it ends or fails, with the same id: readers merge them into one step, in the place where it started. The agent’s thinking and messages have no id and one line each.

{"at":1791647537691,"id":"toolu_01Mk9U…","kind":"read","label":"src/data.ts","state":"running","run":"r6"}
{"at":1791647537702,"id":"toolu_01Mk9U…","kind":"read","label":"src/data.ts","state":"done","run":"r6"}
{"at":1791647539037,"kind":"thinking","label":"","run":"r6"}

<id>.jsonl exists only while a run is in progress. When the run ends (ask.mjs, resolve.mjs, a stop, the reader resolving), the store moves its lines to <id>.history.jsonl with a dev build of the widget, or drops them, and the hooks add the ending script’s own step, done, to the history.

The server never writes steps into comments.json. It serves them with the comments: progress, the last 10 steps of the run in progress, and with history=1 (the debug panel), history, every step of every run. With a dev widget, GET /api/debug/steps/<id> serves both files as they are on disk, history first, which the debug panel opens in a tab.

Images

The reader pastes or drops an image in the widget’s message box: it goes with the message, a new comment (POST /api/comments) or a reply (PATCH with followUp or choices, or alone), as images: { type, data } objects, data in base64. An image alone is a message, with an empty body. The server takes up to 4 per message, PNG, JPEG, GIF or WebP, up to 5 MB each, and checks the bytes, not only the type; it refuses any other image with 422 and its reason, which the widget shows. These limits live in scripts/lib/image-limits.json, which the widget bundles to refuse an image before sending it. A request may then hold up to 100,000 characters of text besides its images. Images sent with only another move (status, cancelled) are ignored. The server refuses any change (403) from a page whose origin it does not let in: a page of another site could send one without reading the answer.

The store writes them beside the comments file, as images/<comment id>/m<n>-<k>.<ext> (m1-1.png: the first image of the comment), and lists their names on the message. The server serves them at GET /api/images/<comment id>/<name>, for the thread’s bubbles, and adds their full paths to the comment’s line for the hooks (NEW, OPEN, FOLLOWUP, EDIT): ` images=[”/…/.annotant/images//m1-1.png"]`, at the end of the line, for the images of the message it carries. The agent opens them with Read. Deleting the comment deletes its folder.

Hand-overs

An open comment waits on the reader’s latest message, the one with the latest sentAt. Each time it opens, the server prints its line for the plugin’s hooks (OPEN, NEW or FOLLOWUP), naming that message right after the comment’s id: NEW <id> sent=m1@2026-10-10T15:44:37.597Z route=/ …, <message id>@<sentAt>. It then writes notifiedAt on the message, once: a restart prints the line again and leaves the date. The hooks start or message an agent once per comment and key, for the session, once an agent took it (started on it, or was sent it): a line about a message already taken starts nothing, whatever a restart or a reload did. A re-send dates the message anew, a key of its own; a message still waiting (queued, or whose agent could not start) starts with the next line about it. A failed write of notifiedAt prints the line again at the next read, which dates it.

The widget reads “Sent…” until the message’s notifiedAt, “Claude has your comment, an agent is starting…” once it is, “Claude has your comment, waiting for a free agent…” when no agent has claimed the comment 15 seconds later, and “No agent has started. Check the Claude Code session watching.” after 60 seconds. On the reader’s reply (an answer or a follow-up: FOLLOWUP), the widget reads “Claude has your reply, an agent is picking it up…”, “Claude has your reply, waiting for a free agent…” and “No agent has picked up your reply. Check the Claude Code session watching.” Each return to open (an answer, a follow-up, a re-send) starts again.

What the server adds

With history=1 (the debug panel), every comment the server printed a line about since it started also has lines: { at, line } objects, oldest first, each line as printed (the last 50). They live in its memory too, and go when the comment is deleted. The debug panel marks the OPEN, NEW and FOLLOWUP lines as hand-overs, dated from their opening, and dates each claim from the hand-over of its opening. With a dev widget, GET /api/debug/agent/<agent> serves what the agent of a working event was given: its first message, exactly the prompt the hooks wrote, read in Claude Code’s transcripts of the last 7 days, and its type (agentType, from the .meta.json next to the transcript); for one of the plugin’s agents, its instructions (instructions, agents/<name>.md), read as it is now: Claude Code does not keep it.

This page on GitHub