Google Shopping -- are there special robots?

[JR123]

I'm trying to do my first Google Shopping campaign and it's been a cluster so far.

Remember when Google Shopping was free and easy to do? Now it's paid, and it's Google Merchant + Google AdWords + hair loss = Google Shopping.

Anyway, all of my products were disapproved or invalid, and when I checked the Data Quality to see why, it said that robot restrictions blocked crawling access to my product photos (image_link).

Does anyone know if Google Shopping has special robots (and if so, name?), and how I can add that so that they can crawl the images that I supplied in my data feed?

I don't know much about this stuff, but couldn't find any answers in Google's help articles.

Thanks in advance,
James

[JR123]

Ok, I might have found it...there was a link that I missed with more information on the data feed error, and it says the following:

Please update the robots.txt file on your web server to allow Google's crawler to fetch the provided images. The robots.txt file can usually be found in the root directory of the web server (e.g. http://www.example.com/robots.txt). In order for us to access your whole site, ensure that your robots.txt file allows both user-agents 'Googlebot-image' (used for images) and 'Googlebot' (used for web pages) to crawl your site. You can do this by changing your robots.txt file as follows:

User-agent: Googlebot
Disallow:

User-agent: Googlebot-image
Disallow:

So, my question is, should I just add the second one (Googlebot-image) to my robots.txt file?

I don't want to screw with much of our website, since we have REALLY good organic rankings and I'm afraid of messing with things and causing duplicate content or ANY other issues that may penalize us...

Thanks in advance!
James

[Pack Secure]

I did not do our campaign, but here is their phone number if that helps.
866-246-6453

[JR123]

Thank you Pack Secure! I spoke with both Google and got a 2nd opinion from 3D Cart. I added all of the robots that Google required and it still didn't work.

However, ONE of my items worked. It turns out that the item that did work in the data feed had the Image_Link as http:// while all of the others that weren't working had https://.

So, there was something where an https:// image URL was being blocked from having Google bots view it. I just changed all of the Image_Link fields on my data feed to http:// URLs and it works perfectly.

Thanks again!
-James

[Pack Secure]

Happy to help. Now that you are up and running, hope you get lots of sales from all that work. :)