Comments on: cross domain sitemap via robots.txt

By: Technology Transfer

Technology Transfer — Sat, 10 May 2008 15:39:21 +0000

Took me a while to successfully create a cross-domain sitemap and I had to ask for help for it, but it’s very interesting to see this working so fine. Anybody else managed to do this by himself?:)

By: kreauter

kreauter — Sun, 04 May 2008 16:07:51 +0000

okay wow fast response

the first syntax i already know but i want to use the robots.txt for more as one domain. i guess that the following syntax should be correct:

rokdd.de with Sitemap file alle-seiten.xml

however google said that is not understood that syntax

thanks for your help

By: zaki

zaki — Sun, 04 May 2008 15:32:13 +0000

What is the error message?

Btw, the declaration of you XML in your robots.txt is incorrect. It should be this way

sitemap: http://rokdd.de/alle-seiten.xml < - correct
rokdd.de with Sitemap file http://rokdd.de/alle-seiten.xml <- wrong

And you seems disallow any of you site to get indexed as well

User-agent: *
Disallow: /

By: kreauter

kreauter — Sun, 04 May 2008 14:55:07 +0000

great information thank you! so now i am able to add to live google and yahoo. but for ask.com you need the reference in robots.txt. when i had in my file the following: http://www.rokdd.de with Sitemap file alle-seiten.xml a server error occurs.. any hints? thank you

By: Raymond Chua

Raymond Chua — Fri, 07 Mar 2008 09:07:37 +0000

Too deep for a newbie like me.

By: alzack

alzack — Fri, 07 Mar 2008 06:06:27 +0000

good info

By: zaki

zaki — Thu, 06 Mar 2008 22:10:18 +0000

The advantage is only seen, if you have many websites, where you have to manually ftp or login into cpanel one by one

By: zaki

zaki — Thu, 06 Mar 2008 22:08:11 +0000

Let me know if you’re unable to do so. I check my spam folder everyday

By: Josef

Josef — Thu, 06 Mar 2008 20:57:48 +0000

You’re right, I’ve tested several sites and the robot.txt and xml sitemap can make a big difference in how deep and often bots crawl.
It’s so easy to put the robots.txt files in the folder where they go that I don’t see much advantage in putting them all on one domain.

By: cross domain sitemap via robots.txt | Domains Yahoo

cross domain sitemap via robots.txt | Domains Yahoo — Thu, 06 Mar 2008 10:12:28 +0000

[...] Read the rest of this great post here [...]

By: nUUr

nUUr — Thu, 06 Mar 2008 05:41:22 +0000

huhu..xpham jgk..erm nape kekadang xleh komen kat cni yek

By: MK

MK — Thu, 06 Mar 2008 00:56:56 +0000

uhu… SEO… those three letters bothers me, having to re-check you every move to make sure it follows ‘the right’ SEO.
nyway, since this is more on the technical site, i guess i’ll have to do it.
thanks for the info.

By: seo lad

seo lad — Thu, 06 Mar 2008 00:46:44 +0000

They should just agree on a common platform. That would make things easier.

By: cross domain sitemap via robots.txt | videositemap.com

cross domain sitemap via robots.txt | videositemap.com — Wed, 05 Mar 2008 15:58:20 +0000

[...] Original post by Blogjer is about bloggers [...]

By: azwanhadzree

azwanhadzree — Wed, 05 Mar 2008 15:26:13 +0000

at the moment, i prefer to maintain everything separately. hmm talking about sitemap and robot.txt, i haven’t got any robot.txt on any of my site yet.

By: KNizam

KNizam — Wed, 05 Mar 2008 15:00:00 +0000

this sounds like jargon to me. heheh tak paham pokcik eh