AI Overviews vs Local Pack: How We Built the Report and What We Found Speaker: Niladri Sarkar, Synup Video: https://www.youtube.com/watch?v=W1w8ozJCSdk Hi, I'm Niladri. I run Marketing at Synup. And this video is about recent local overview versus local Pack study we did, So we had a question, how well does the map tell us which businesses someone will find in the answer? So we wanted to put those two lists side by side across a large set of searches and see what's going on in search results. So here's how we went about doing it. We use searches asking for the best businesses of a particular kind in a city, things like best accounting firms in Huntsville. These are kind of searches that someone might make while they are choosing between businesses or looking for a potential solution. So we collected close to 12 ,250 odd results, covering 49 different categories of businesses across 250 major US cities. We ran them on a computer in US English Then on each page, we'd capture where the map and AI answers appear. We compared the businesses in the two. Same page, same moment. That left us with a little under 3,000 pages, for the main overlap comparison that we wanted to run. For each business on the map, we checked whether it was named in the answer, appeared only among the answer’s linked sources, or appeared in neither place. Google had changed the map website links, so they no longer showed the business's website address directly. So we match businesses by name and street address instead. Our tools can miss AI answers. They can miss business names within them. So the counts of answers, names we captured are minimums. They don't tell us everything that appeared live. And the two-week source comparison gave us two snapshots. It doesn't establish an ongoing trend. We measured what appeared on the page. We did not measure which businesses people clicked, called or bought from. So here's an example from the study that makes the comparison easier for us to picture. For best accounting firms, Huntsville as an example, the map showed three businesses. The AI answer named five. On all three from the map, results were included in the AI answers, along with two additional firms. So as a marketer or a client, if you're looking at the map, you'd see three businesses. If you also read the answers, you could encounter two more. So there are two findings here that can sound contradictory until we separate them. First, 64% of the map businesses in the comparison were also named in the AI answers. Those overlap figures come from the pages where we captured both map results and then the AI answer. That's a little under 3,000 of the 12,000 searches. On 88% of those pages, the AI answer included at least one of the three businesses that were included in the map. So the AI answer and the map often had businesses in common. But let's look at it from another direction. Among the businesses named in AI answers, 67% weren't in the map pack or the three listings on the map. That measurement used a smaller group of 714 searches approximately. When every map businesses had a street address, we could match with. One number starts with the businesses on the map. The other starts with businesses in AI answer. They are answering different questions. and the samples also differ. Here's an example to make that distinction clear. Suppose a map lists three businesses and an AI answer names six. All three map businesses could be in the answer along with the three others. In that example, every map business made it into the answer but half the businesses in that answer came from outside the maps, is three listings. Both statements would be true. Practical takeaway is that appearing on map is useful information. It doesn't fully describe the business' Google names and its AI answers. So you need to complete the picture. There's another distinction that matters when you're checking an AI answer. A business might be named in the answer as a person reads or it might appear only in the panel of sources is linked to that answer or it might appear in neither place. The study calls the first state recommended and the second state as consulted. Those are internal labels from what was visible in the collected results in our study. Consulted does not give us access to Google's internal reasoning. Among the map businesses in this study, 64% were named in the visible answer. Another 26% appeared only in the sources. 10% of those appeared in neither. So for a marketing report, I show this separate results. A business owner should be able to tell whether customers would encounter business in the AI answers itself, or would it need to look through the sources to find it. The amount of overlap wasn't the same everywhere. City size made a difference in the results we observed. In the large city group, the AI answers named around 46% of the map businesses. In cities under 100,000 people, it named 69%. Mid-sized cities with between say 100,000 and a million people sat in between at 63%. The large city comparison had smaller sample. 154 searches across eight cities with more than a million people. So I'd be careful about treating the exact percentage as a number that applies to every major city. study also compared the references linked from AI answers to research runs two weeks apart in September. The first run, about 20% of those references were linked to Google's own business records. And the second, the share was about 80%. that average number of outside website references per answer also fell from about 9 to 4. A Google business record is a reference to the business within Google's own system, rather than a link to an outside website. That's the distinction between those percentages. It doesn't mean 80% of the businesses were new. It does not mean 80% of customers did on Google, and it doesn't tell us the outside websites stopped influencing the answers. You're talking about the references shown with it. This matters when you look at reports saying a particular website was cited less often. Before concluding that the site became less useful, ask whether Google showed fewer outside references overall. There's also a limit to the comparison. We have two snapshots from two weeks of September. And they show a change between those dates. They don't tell us whether it happened gradually, whether it continued or whether it later reversed. So the useful response is to keep checking. The full study is on Synup's website with the category comparison and the data and an explanation of the method. We'll link it in the description here.