SEO for a new software site: the order we’d do it in

Verify the site in both on launch day and submit your sitemap in each. They are where you see which searches find your pages, which pages are indexed and what is broken. Bing can import a site from Search Console in a few clicks.

In Search Console, open Indexing, then Sitemaps: your sitemap should show Success. Check Sitemaps in Bing Webmaster Tools the same way. Your robots.txt should point to the sitemap too:

curl -s "$site/robots.txt" | grep -i '^sitemap:'

Read it line by line, AI crawlers included. A Disallow: / left over from staging, or a rule for GPTBot, ClaudeBot or PerplexityBot copied from a template, keeps pages out of search results or AI answers.

Look for Disallow: / under User-agent: * or under a crawler you want in. The AI crawler check explains each crawler name.

curl -s "$site/robots.txt" | grep -i -E -A3 '^user-agent: *(\*|gptbot|claudebot|perplexitybot)'

List every page you want indexed and nothing else: no redirects, no missing pages, no test pages. Every URL in the sitemap should answer 200 on its own.

The first command should print 200. The second prints every listed URL that does not answer 200; on a small site it takes under a minute, and no output is what you want.

curl -s -o /dev/null -w '%{http_code} %{content_type}\n' "$site/sitemap.xml"

curl -s "$site/sitemap.xml" | grep -o '<loc>[^<]*' | sed 's/<loc>//' | while read -r u; do
  printf '%s %s\n' "$(curl -s -o /dev/null -w '%{http_code}' "$u")" "$u"
done | grep -v '^200'

The H1 and the first paragraphs should be in the HTML the server sends. Google renders JavaScript, but most AI crawlers do not, so a page drawn only in the browser reaches them empty.

This counts the words in the raw HTML. A page that reads as a full article in your browser should come back with hundreds of words, not a few dozen. Step 3 of the AI crawler check has a loop for several pages.

curl -s "$site/" | perl -0777 -pe 's/<(script|style)\b.*?<\/\1>//gis; s/<[^>]+>/ /g' | wc -w

Start with the pages people read just before they choose: “[competitor] alternative”, “[you] vs [them]” and “best [category] for [who]”. Fewer people search these than broad topics, and the ones who do are close to picking a tool.

Search “[your main competitor] alternative” in a private window and see whether you appear. Then count the buyer pages in your sitemap:

curl -s "$site/sitemap.xml" | grep -c -i -E 'alternative|-vs-|best-'

Most titles and descriptions come from one template, so one fault repeats on every page it builds. Fix the pattern there: titles of about 60 characters, descriptions of about 160, each page filling in its own name.

This prints the title and description length of every page in your sitemap. Look for titles over 60, descriptions over 160 and the same title on many pages. Counts are rough: an entity such as & counts as several characters. A full audit checks this on every page.

curl -s "$site/sitemap.xml" | grep -o '<loc>[^<]*' | sed 's/<loc>//' | while read -r u; do
  curl -s "$u" | perl -CS -0777 -ne '($t) = /<title[^>]*>(.*?)<\/title>/is; ($d) = /<meta\s+name="description"\s+content="([^"]*)"/is; printf "%3d %3d ", length($t), length($d)'
  echo "$u"
done

A short Markdown file at /llms.txt that says what the product is, who it is for, what it costs, what it does not do and which pages matter. No AI company has confirmed reading it, so treat it as a cheap test, not a ranking factor.

You want 200 and text/plain or text/markdown. The llms.txt guide shows what we put in ours.

curl -s -o /dev/null -w '%{http_code} %{content_type}\n' "$site/llms.txt"

Add the product to AlternativeTo next to the tools it replaces, give its GitHub repo topics that name the category, ship a Homebrew formula or cask for a Mac or command-line tool, and join the directories your buyers read. AI answers and alternatives searches often cite these pages instead of the product’s own site.

Search your main competitor on AlternativeTo and see whether you are in its list of alternatives. Open your repo’s About box to see its topics. For a Mac or command-line tool:

brew search your-app-name