Tweet
There are about 300 urls on my site that are being blocked by robot.txt, but I did not put any robot.txt files on my site at all so why is this happening? How can I stop this because the urls that are being blocked are urls that go right to my product description pages. I want these accessible for the bots to crawl. Any suggestions on how to go about this?
-
-
You could login via FTP and delete the robots.txt
However, I suggest you don't. The reason is the default robots.txt blocks the search engines from accessing files such as "add_cart.as" which should not be indexed, as well as others.----------------------------
Gonzalo Gil
3dCart Support
800-828-6650 x111 -
I am just worried that this is causing my site not to get indexed well. Do you think that it is doing that?Comment
Comment