I'm Getting an Error When I use Robots.txt Tester
When I test our robots.txt file I get an error message "no user-agent" specified.
This is what I find when I test:
Disallow: */wp-admin/
Should I add:
User Agent: *
And how will it affect our site?
This is what I find when I test:
Disallow: */wp-admin/
Should I add:
User Agent: *
And how will it affect our site?
robots.txtfile. Yes, you can add the*and it will not affect your site. We have article explaining more aboutrobots.txtfiles and how they work — for further reference. Best, Christopher M.