How to Scrape Meta Titles and Descriptions at Scale
BackGuides

How to Scrape Meta Titles and Descriptions at Scale

Phil
Phil
8 min read

200 URLs. Every meta title and description. Under two minutes.

You have a list of competitor URLs. Or maybe your own sitemap. You need the meta title, meta description, and character counts for every page. The usual approach? Open each URL. Right-click. View source. Find the <title> tag. Copy it. Paste it into your spreadsheet. Repeat.

That works for five pages. It falls apart at fifty. At two hundred, you are losing your mind.

SEO tools like Screaming Frog, Ahrefs, and SEMrush can do this. But they cost $100+ per month. And you still end up exporting a CSV into a spreadsheet.

SheetMagic does it with one formula per URL. Directly inside Google Sheets. No exports. No extra software. Just formulas that pull meta tags straight into your cells.

Formula
=GETMETATITLE(A2)

Returns the title tag from the URL in A2. One formula, one credit, one result.

Paste that in B2. Drag it down to B201. You now have every meta title from your URL list. The same works for descriptions.

Formula
=GETMETADESCRIPTION(A2)

Returns the meta description from the URL in A2. Drag down for bulk extraction.

That is the aha moment. Two formulas. Two hundred pages audited. The rest of this guide shows you how to turn that raw data into a complete meta audit dashboard.


The meta extraction formulas

SheetMagic has dedicated formulas for every piece of page metadata. Each one targets a single tag and returns clean text.

=GETMETATITLE() for title tags

Formula
=GETMETATITLE("https://example.com/blog/post-1")

Extracts the content of the title tag. This is what shows as the blue link in Google search results.

Title tags are the first thing searchers see. Google displays roughly 50-60 characters before cutting them off. Pulling every title tag across your site (or your competitors' sites) is the foundation of any meta audit.

=GETMETADESCRIPTION() for meta descriptions

Formula
=GETMETADESCRIPTION("https://example.com/blog/post-1")

Extracts the meta description tag. This is the snippet text that appears below the title in search results.

Meta descriptions don't directly affect rankings. But they affect click-through rates. A good description convinces someone to click your result instead of the one above it. Google shows about 155 characters before truncating.

=GETSELECTOR() for Open Graph tags and beyond

What about OG titles, OG descriptions, canonical URLs, or Twitter card data? The =GETSELECTOR() formula can target any meta tag on the page using CSS selectors.

Formula
=GETSELECTOR(A2, "meta[property=og:title]")

Extracts the Open Graph title tag. Social platforms use this when someone shares the URL.

Formula
=GETSELECTOR(A2, "meta[property=og:description]")

Extracts the Open Graph description. Often different from the standard meta description.

Formula
=GETSELECTOR(A2, "link[rel=canonical]")

Extracts the canonical link element. Useful for finding duplicate content issues.

Use =GETSELECTOR() when you need meta tags beyond the basics. It can target any tag with a CSS selector, including meta[property=og:image], meta[name=robots], meta[name=twitter:card], and custom meta tags specific to a CMS.


Build a complete meta audit dashboard

Raw meta tags are useful. But a proper audit needs character counts, quality checks, and warning flags. Here is how to build the full dashboard in one sheet.

1

Set up your URL list

Paste your URLs into column A. These could come from your sitemap, a crawl export, or a =SERP() query. For this example, assume you have 200 URLs starting in A2.

If you need to pull your sitemap URLs into the sheet, try this:

Formula
=SERP("site:yourdomain.com")

Returns indexed URLs from Google for your domain. A quick way to build your URL list if you don't have a sitemap export handy.

2

Extract meta titles and descriptions

In column B, add your title extraction:

Formula
=GETMETATITLE(A2)

Drag down to B201 for all 200 URLs. Each call uses 1 integration credit.

In column C, add description extraction:

Formula
=GETMETADESCRIPTION(A2)

Drag down to C201. Now you have titles and descriptions side by side.

3

Add character counts

Title length in column D:

Formula
=LEN(B2)

Native Sheets formula. No credits needed. Google truncates titles around 60 characters.

Description length in column E:

Formula
=LEN(C2)

Google truncates descriptions around 155 characters. Flag anything over that limit.

4

Flag problems automatically

Title warnings in column F:

Formula
=IF(B2="","MISSING",IF(LEN(B2)>60,"TOO LONG",IF(LEN(B2)<30,"TOO SHORT","OK")))

Flags missing titles, titles over 60 characters, and titles under 30 characters. Native Sheets formula.

Description warnings in column G:

Formula
=IF(C2="","MISSING",IF(LEN(C2)>155,"TOO LONG",IF(LEN(C2)<70,"TOO SHORT","OK")))

Same pattern for descriptions. Missing, too long (over 155), or too short (under 70).

Filter columns F and G for anything that is not "OK" to build your fix list.

5

Add AI-powered quality checks

Character counts catch length problems. But they miss quality issues. Does the title include the target keyword? Does the description end with a call to action? AI can check for that.

Formula
=AITEXT("Review this meta title for SEO quality. Is it under 60 characters? Does it clearly describe the page topic? Is it compelling for search results? Return OK or a one-sentence fix suggestion.", B2)

AI quality review for each title. Filter for non-OK results to find pages that need rewriting.

Formula
=AITEXT("Review this meta description. Is it under 155 characters? Does it include a call to action? Does it avoid duplicate phrasing with the title? Return OK or a one-sentence fix suggestion.", C2)

Catches quality issues that character counts miss. Duplicate phrasing, missing CTAs, vague language.

Your dashboard now has eight columns: URL, title, description, title length, description length, title warning, description warning, and AI quality notes. That is a complete on-page meta audit. Sort by the warning columns to prioritize fixes.

You can try all of this on the free tier. No credit card needed. You get enough integration credits to audit a handful of pages and see how the workflow feels.


Scrape competitor meta tags for SEO research

Your own site audit is half the picture. The other half is understanding what your competitors are doing. Pull the top-ranking pages for your target keyword and compare their meta tags to yours.

1

Pull the top 10 search results

Formula
=SERP("your target keyword")

Returns up to 20 search results. The top 10 URLs are your direct competitors for this keyword.

Copy the top 10 URLs into column A of a new sheet tab.

2

Extract their meta titles and descriptions

Formula
=GETMETATITLE(A2)

Pull competitor title tags. Put in B2 and drag down to B11.

Formula
=GETMETADESCRIPTION(A2)

Pull competitor descriptions. Put in C2 and drag down to C11.

You now have the exact text that shows up in search results for your target keyword. This is what searchers see when deciding which result to click.

3

Analyze patterns across competitors

Formula
=AITEXT("Here are 10 meta titles from the top Google results for a keyword. What patterns do you see? What words or phrases appear most often? What makes the top 3 titles stand out from the rest? Return 3-4 bullet points.", B2&" | "&B3&" | "&B4&" | "&B5&" | "&B6&" | "&B7&" | "&B8&" | "&B9&" | "&B10&" | "&B11)

Pattern analysis across all competitor titles. Shows you what Google is rewarding for this keyword.

Do the same for descriptions in a separate cell. These patterns tell you what Google (and searchers) expect for this query.

4

Write better meta tags informed by the data

Formula
=AITEXT("Based on these competitor meta titles for the keyword: your target keyword, write a title tag under 60 characters that stands out while matching searcher expectations. Competitor titles: "&B2&" | "&B3&" | "&B4&" | "&B5, "")

Generates a title tag informed by real SERP data. Not generic AI copy, but data-driven copy.

Formula
=AITEXT("Write a meta description under 155 characters for a page targeting: your target keyword. It should differentiate from these competitor descriptions while ending with a clear call to action. Competitors: "&C2&" | "&C3&" | "&C4, "")

Your description is shaped by what is already ranking. Same intent, better execution.

This is competitive analysis that would normally require an Ahrefs subscription. You are doing it with spreadsheet formulas. And because everything stays in the sheet, you can rerun the analysis for any keyword by changing one cell.

For more on building full SEO workflows in Google Sheets, see the complete SEO guide.


Extract Open Graph and social meta tags

Meta titles and descriptions are for search engines. Open Graph tags are for social platforms. When someone shares your URL on Facebook, LinkedIn, or Slack, the OG tags control what shows up in the preview card.

Many sites have mismatched OG tags. The OG title says one thing, the page title says another. The OG image is missing or broken. These issues hurt social sharing but rarely show up in standard SEO audits.

Formula
=GETSELECTOR(A2, "meta[property=og:title]")

The title that appears in social media preview cards.

Formula
=GETSELECTOR(A2, "meta[property=og:description]")

The description in social media previews. Often shorter than the meta description.

Formula
=GETSELECTOR(A2, "meta[property=og:image]")

The image URL used in social cards. Missing OG images mean ugly link previews.

Add these three columns to your audit dashboard. Then flag mismatches with a simple comparison:

Formula
=IF(B2=GETSELECTOR(A2,"meta[property=og:title]"),"Match","MISMATCH")

Compares the page title tag to the OG title. Mismatches may confuse users coming from social platforms.

Some pages use JavaScript frameworks that load meta tags dynamically. If =GETMETATITLE() or =GETSELECTOR() returns empty results, the page might require JavaScript rendering. Try =VISIT(A2) to pull the rendered content, then use =AITEXT() to extract the title from the full page text.


Bulk processing tips for large URL lists

Scraping 20 URLs is simple. Scraping 2,000 takes a bit more planning. Here are the patterns that keep things smooth.

Work in batches of 50-100 rows. Write your formula in B2, drag it down to B51, and check the results. If everything looks right, continue to the next batch. This catches broken URLs and blocked sites before you burn through your credits.

Use smart caching to your advantage. SheetMagic caches scraping results for 30 minutes. If you extract a meta title from a URL, then run =GETMETADESCRIPTION() on the same URL within 30 minutes, the second call uses the cached page data. No extra credit spent on the fetch.

Handle errors gracefully. Some URLs will fail. Login walls, redirects, and bot protection all cause issues. Wrap your formulas in =IFERROR():

Formula
=IFERROR(GETMETATITLE(A2), "FAILED")

Returns FAILED instead of an error. Filter for FAILED to find problem URLs.

Use PAGEDATA for maximum efficiency. When you need title, description, H1, and HTTP status for every URL, =PAGEDATA() returns all four in a single call:

Formula
=PAGEDATA(A2)

Returns title, description, H1, and HTTP status in one row. One credit instead of four separate formula calls.

This is the most credit-efficient approach for large audits where you need multiple pieces of metadata per URL.

Check your remaining integration credits anytime in the SheetMagic sidebar under Extensions > SheetMagic, or on the dashboard. Credits reset monthly on your billing cycle.


Rate-limiting and responsible scraping

SheetMagic handles rate-limiting on the backend. You do not need to add delays between formula calls. But there are a few things to keep in mind.

Respect the sites you scrape. Public meta tags are fair game. But do not hammer a single domain with thousands of requests in a few minutes. Spread large audits across multiple sessions if you are targeting one site.

Some sites actively block scrapers. If you get empty results or errors for specific domains, the site may be blocking automated requests. Try a different URL from the same domain to confirm. If the whole domain is blocked, you can use =VISIT() (which supports JavaScript rendering) as a fallback, then extract meta tags with =AITEXT():

Formula
=AITEXT("Extract the meta title and meta description from this page content. Return them separated by a pipe character.", VISIT(A2))

Fallback approach for sites that block direct meta tag extraction. Uses the full rendered page content.

Cache your results. If you plan to rerun analysis on the same URLs, copy your extracted data as plain values first (Ctrl+Shift+V). This preserves the data without re-scraping. You only need fresh scrapes when the target pages might have changed.


Put it all together: the SEO meta audit dashboard

Here is the final column layout for a complete meta audit sheet:

ColumnFormulaWhat it shows
A(your URLs)Page URL
B=GETMETATITLE(A2)Title tag
C=GETMETADESCRIPTION(A2)Meta description
D=LEN(B2)Title character count
E=LEN(C2)Description character count
F=IF(B2="","MISSING",IF(LEN(B2)>60,"TOO LONG","OK"))Title warning
G=IF(C2="","MISSING",IF(LEN(C2)>155,"TOO LONG","OK"))Description warning
H=GETSELECTOR(A2,"meta[property=og:title]")OG title
I=GETSELECTOR(A2,"link[rel=canonical]")Canonical URL
J=GETH1(A2)H1 heading

That is ten columns per URL. Columns A through G use integration credits and native formulas. Columns H through J add deeper metadata for a full technical audit.

Sort by columns F and G to find pages with missing or oversized meta tags. Compare column B to column J to catch title/H1 mismatches. Compare column B to column H to find title/OG title discrepancies.

This dashboard replaces the meta tag audit features in tools that cost $99 to $399 per month. And because it lives in Google Sheets, you can share it with your team, add conditional formatting, build pivot tables, or connect it to your existing SEO workflow.

For content extraction beyond meta tags (headings, paragraphs, full page text), see the content extraction guide. For the full range of scraping formulas and Apify integrations, check the complete web scraping guide.


Start your meta audit now

Pick five URLs from your site. Open Google Sheets. Type =GETMETATITLE(A2) in one cell and =GETMETADESCRIPTION(A2) in the next. See the results show up in seconds.

That is all it takes to start. From there, add character counts, warning flags, and AI quality checks. Build the full dashboard from this guide. Audit your entire site in an afternoon.

You can try all of this on the free tier. No credit card needed.

Install SheetMagic free from the Google Workspace Marketplace and audit your first five meta tags in the next sixty seconds.


More SEO and scraping guides: Complete web scraping guide | AI for SEO | Extract content from any page | Getting started