Robots.txt: Your Website's Bouncer

The robots.txt is a small text file located in the root directory of your hosting. It tells search engine bots (e.g., Googlebot) which areas of your website they are allowed to visit and which they are not.

Why is this important?

You want to prevent Google from indexing sensitive areas like your admin login or shopping cart pages. This saves "crawl budget" and ensures that Google focuses on your important content.

Connection to the Sitemap

In the Robots.txt, Google is often also shown the path to your sitemap. We explained how to create and submit such a sitemap in our guide "XML Sitemap: The Roadmap for Google".

The MarketGoo Check: MarketGoo automatically checks if your Robots.txt is correctly configured or if you have accidentally blocked your entire website from Google.

Check SEO status now
Was this answer helpful? 0 Users Found This Useful (0 Votes)

Most Popular Articles

301 Redirects (Forwarding)

301 Redirects: Migrating Correctly Without Losing Rankings If you delete a subpage or change a...
Backlinks – The Digital Currency

Backlinks: Why External Recommendations Are So Important A backlink is a link from another...
Image SEO (File Names & Alt Texts)

Image SEO: How Google Finds Your Photos Search engines cannot "see" images like humans do....
Business Directories (Citations)

Business Directories: Why You Should Be Listed Everywhere Entries in portals such as Local.ch,...
Finding Broken Links (Dead Links)

Broken Links: Avoid Dead Ends on Your Website Nothing frustrates users (and the Google bot)...