Yii Framework Forum: Robots.txt Disallow - Yii Framework Forum

Jump to content

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

Robots.txt Disallow Rate Topic: -----

#1 User is offline   darioo 

  • Junior Member
  • Pip
  • Yii
  • Group: Members
  • Posts: 61
  • Joined: 10-July 13

Posted 15 December 2013 - 05:25 PM

If I add line in my robots.txt
Disallow: /site/contact

is this the same as I put nofollow on every /site/contact link?
0

#2 User is offline   andy_s 

  • Random Member Title
  • Yii
  • Group: Moderators
  • Posts: 1,526
  • Joined: 22-June 09
  • Location:Russia, Kostroma

Posted 15 December 2013 - 05:44 PM

The topic is not Yii related, I moved it to Miscellaneous.
0

#3 User is offline   Da:Sourcerer 

  • Elite Member
  • PipPipPipPipPip
  • Yii
  • Group: Members
  • Posts: 1,222
  • Joined: 30-March 11
  • Location:Berlin, Germany

Posted 15 December 2013 - 05:45 PM

Yes. If crawlers really respect that is another issue, though.
programmer /ˈprəʊgramə/, noun: a device that converts ►coffee into ►code
0

#4 User is offline   Maurizio Domba Cerin 

  • Yii - Yesss It Is !!!
  • Yii
  • Group: Yii Dev Team
  • Posts: 4,341
  • Joined: 12-October 09
  • Location:Croatia

Posted 16 December 2013 - 03:18 AM

Not really the same... with "disallow" the crawler should not "crawl" the site/contact page at all... by having only nofollow on links the page at site/contact would be "crawled" and possibly returned as a search result.
Find more about me.... btw. Do you know your WAN IP?
1

Share this topic:


Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users