Running the monthly test

Why monthly

A month is long enough for changes to reach the answers and short enough to catch problems early. Bay Area AEO's timeline explains the rhythm: listing and data fixes can start showing up in AI answers within a few weeks, because many assistants pull live information, while content, reviews and third-party mentions build over two to four months. A monthly test catches the fast changes as they arrive and shows the slow ones as a trend.

Same conditions every month

The point of the test is comparison, so the conditions should stay as fixed as possible. Run it in the same week each month. Use the same platforms and the same versions where the platform offers a choice. Use the same kind of session: a fresh, signed-out browser window, or a test account with memory and personalization turned off, so the assistant's answers are not shaped by earlier conversations. Set the location the same way each time, either by naming the city in the question or by fixing the device's location.

Write these conditions at the top of each month's results. When something has to change (a platform renames a model, a setting disappears), note it, so a jump in the results can be traced to the change.

Running each question more than once

Assistants write each answer fresh, and the same question can produce different businesses on different runs. For the most important questions, run each one two or three times per platform and record every run. The share of runs that name the business is a steadier measure than a single yes or no. For less important questions, one run per month is enough, because the trend across months smooths out the variation.

What to record

For every answer, record the same fields as the first audit, so the months line up.

Capturing sources

Sources are some of the most useful data the test produces. Perplexity shows citations with every answer. Google's AI Overviews show linked sources. ChatGPT and Claude show sources when they search, and will usually name them when asked a follow-up such as “Which sources did you use?”. Record them all. Over months, the list shows which local guides, directories and pages the assistants trust for the business's category and area, which feeds directly into the work on listings and trusted mentions.

How long it takes

A panel of thirty questions across five platforms, with repeats for the ten most important, is about two hundred answers. Recording them by hand takes a morning for someone practiced. Some businesses and agencies use software that runs prompts automatically and records the answers; that saves time, but the output should still be read by a person, because the description and the sources matter as much as the count.

A quick consistency check before recording

Before the first answer is recorded each month, spend five minutes confirming that the master business record, the website and the four main listings still agree. If a listing has drifted since last month, note it in the results, because any wrong answer that month may trace straight back to it. This small step keeps the test from measuring an accident.

Screenshots and raw text

Save the full text of every answer, and screenshots of the most important ones. Screenshots show the answer as the customer saw it, including any map card or images, and they make the monthly report far easier to understand. Bay Area AEO's principles describe its reporting as showing the real answers before and after; the raw text and screenshots are what make that possible.

Logging the work alongside

Keep a log of the GEO work done each month next to the test results: listings corrected, pages published, markup added, reviews requested, mentions earned. When an answer changes, the log shows what changed around the same time. Over several months, the pairing of work and results is the clearest picture a business can have of what is working.

When results jump

Sometimes the results jump in a single month: a business named in twice as many answers, or suddenly missing from a platform. Before drawing conclusions, check the conditions (was the session clean, was the location the same), check for platform changes (a new model, a new search feature), and run the affected questions again a few days later. A real change persists; a fluke usually does not.

What to expect in the first months

Early tests usually show the fast changes first: corrected hours and addresses in direct questions, the business appearing in Copilot and Siri answers after its Bing and Apple listings are claimed. Changes in discovery questions tend to follow as pages, reviews and mentions build, on the two-to-four-month horizon Bay Area AEO describes. Keep the routine steady through the early months, when the trend is still forming.

Testing the AI Overviews in Google search

Google shows an AI Overview for many questions but not all, and whether one appears can change from month to month. Record whether an overview appeared, and if it did, whether the business was named and which sources were linked. Also note the map results beneath it, because customers see both together. A month where no overview appears for a question is a result in its own right, and the business's ordinary search and map visibility carry that question instead.

Testing voice answers

For businesses that depend on urgent or walk-in customers, add a short voice test: ask Siri the five most urgent panel questions from a phone with location services on, standing inside the service area. Record the business named and the details read aloud. Voice answers usually name one business, so changes are easy to see, and they depend heavily on the Apple Maps listing covered in chapter 11.

Testing from more than one place

A business that serves several cities should check that its results hold across them. Run the main discovery questions with the location set to each important city, or with the city named in the question. A plumber may be named reliably in Hayward and rarely in Castro Valley, which points to a specific gap: perhaps no page mentions Castro Valley, or few reviews come from there. Results by city turn a vague sense of progress into a list of places to work on.

Sharing results with the team

The monthly results are useful well beyond marketing. Front-desk staff learn which facts the assistants get wrong and can correct customers who arrive with them. Technicians and practitioners see the words customers and assistants use about their work, which often shows up in later reviews. Owners see which services and places are gaining. A short summary at a monthly staff meeting, with two or three real answers read aloud, turns the test into something the whole business pays attention to.

Testing a follow-up question

For a few of the most important questions, ask the natural follow-up as well: “Which of those is open on Sunday?”, “Which one takes my insurance?”, “Which is closest to the Caltrain station?”. Record whether the business survives the second question. Follow-ups test exactly the conditions that Part 3 of this handbook puts on the website, and they show whether an assistant can find them.

Keeping a clean archive

Store each month's results in the same structure: a folder per month, the results sheet, the raw text and the screenshots, and a note of the conditions. After a year, the archive tells the story of the business's AI visibility in a form anyone can check. It also protects the business if it changes agencies, because the history belongs to the business.

Who runs it

The same person, or the same agency process, should run the test each month, because consistency in how it is run matters more than who runs it. A business that works with an agency should still be able to see the raw answers and the conditions. Bay Area AEO's method is built around showing clients what AI actually says, prompt by prompt and platform by platform.

Related chapters46. Building a prompt panel · 48. Reading AI answers · 49. The monthly report · 30. Keeping facts current