Elementive Technical SEO Training - Search Robots & Robots.txt Controls

Matthew Edgar, Web Analytics Consultant

Play Speed
  • 0.5x
  • 1x (Normal)
  • 1.25x
  • 1.5x
  • 2x
5 Lessons (16m)
    • 1. Search Robots

    • 2. Control Methods

    • 3. When To Use Controls

    • 4. Testing Robot Controls

    • 5. Take Action


Project Description


Take the actionables that were discussed in "Part 5 - Take Action" and create or review your website’s robots.txt file.


On your website, create or review your robots.txt file with these questions in mind:

  • Do you have disallows? Are they legit?
  • Do you have allows? Are they legit?
  • Is there anything you should add?
  • What pages have you noindexed? What should you noindex?

Then, once those questions have been answered, upload your new robots.txt file and test robot controls on Google Search Console.

Feel free to reach out with any questions or concerns.

Student Projects