seo - Why Google robots.txt Tester has error and it's not valid -
as can see in below image google webmaster tools robots.txt tester tell me 9 error don't know how fix , problem?
please me figure out
that valid robots.txt - you've got utf-8 bom (\xef\xbb\xbf) @ beginning of text file. that's why there's red dot next 'user' in first line. mark tells browsers , text editors interpret file utf-8 whereas robots.txt expected use ascii characters.
convert text file ascii , errors go away. or copy after red dot , try pasting in again.
i tested on live version, here's result translated byte form:
\xef\xbb\xbfuser-agent: *\r\ndisallow: /en/news/iranology/\r\ndisallow: /en/tours-services/tour-the-soul-of-iran\r\ndisallow: /en/tours-services/tour-a-whistle-stop-tour\r\ndisallow: /en/to
you can see bom @ beginning. browsers , text editors ignore may mess crawlers ability parse robots.txt. can test live version using python script:
import urllib.request text = urllib.request.urlopen('http://www.best-iran-trip.com/robots.txt') print(repr(text.read()))
if you're able install notepad++, has encoding menu lets save in format.
Comments
Post a Comment