Re: [PHP] Googlebot (not another rtfm-mail)
At 17:58 28-1-04, you wrote: On Wed, Jan 28, 2004 at 11:51:06 -0500, Michael Mulligan wrote: Googlebot visits my site occasionally and follows a lot of my PHP links with long query strings... same here, google has been indexing my php site for over 3 years now :) I find the google site is NOT very helpful for PHP guidelines, considering my experiences. I used the error_document 404 redirect trick to use short links and it took some time to realise that Google boycotted that, probably because it gets a 404. Also links with a long parameter string of (5 parameters, 30 characters) were ignored. As far as I could see Google only indexed 1 index.php page. So I had to overhaul the whole link logic of my cms. I now use mod_rewrite too. But I have seen sites with only an one parameter (?ID=11) being indexed. Be aware that javascript links and form buttons are unlikely to be indexed. Chris -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Googlebot (not another rtfm-mail)
Doing that work just fine but you have to send the right heading. My sites that do that have header(HTTP/1.0 200 OK); as the first instruction and are very well indexed by Google Olwen Williams [EMAIL PROTECTED] -- -Original Message- From: Chris Hayes [mailto:[EMAIL PROTECTED] Sent: Friday, January 30, 2004 1:04 AM To: [EMAIL PROTECTED] Subject: Re: [PHP] Googlebot (not another rtfm-mail) At 17:58 28-1-04, you wrote: On Wed, Jan 28, 2004 at 11:51:06 -0500, Michael Mulligan wrote: Googlebot visits my site occasionally and follows a lot of my PHP links with long query strings... same here, google has been indexing my php site for over 3 years now :) I find the google site is NOT very helpful for PHP guidelines, considering my experiences. I used the error_document 404 redirect trick to use short links and it took some time to realise that Google boycotted that, probably because it gets a 404. Also links with a long parameter string of (5 parameters, 30 characters) were ignored. As far as I could see Google only indexed 1 index.php page. So I had to overhaul the whole link logic of my cms. I now use mod_rewrite too. But I have seen sites with only an one parameter (?ID=11) being indexed. Be aware that javascript links and form buttons are unlikely to be indexed. Chris -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php CAUTION - This message may contain privileged and confidential information intended only for the use of the addressee named above. If you are not the intended recipient of this message you are hereby notified that any use, dissemination, distribution or reproduction of this message is prohibited. If you have received this message in error please notify Safe Air Ltd immediately. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of Safe Air. _ For more information on the Safe Air Group, visit us online at http://www.safeair.co.nz/ _ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
from google http://www.google.com/webmasters/facts.html Fiction: Sites are not included in Google's index if they use ASP (or some other non-html file-type.) Fact: At Google, we are able to index most types of pages and files with very few exceptions. File types we are able to index include: pdf, asp, jsp, hdml, shtml, xml, cfm, doc, xls, ppt, rtf, wks, lwp, wri. I notice php is not mentioned in this list, surely it can index php generated pages ?? Thanks, Jon jon bennett | [EMAIL PROTECTED] new media creative _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ J b e n . n e t 91 Gloucester Rd, Trowbridge, Wilts, BA14 0AD t: +44 (0) 1225 341039 w: http://www.jben.net/ iChat (AIM): jbendotnet On 27 Jan 2004, at 14:53, Ben Ramsey wrote: Perhaps this link will help: http://www.google.com/webmasters/ Frank Tudor wrote: That's not very helpful. I am also curious about this. Anyone have a more thought provoking answer? Frank --- Raditha Dissanayake [EMAIL PROTECTED] wrote: If you read the pages on google you will know the answer. Hartley, Matt wrote: Is Googlebot (or any other bot) able to follow links that are php? e.g. a href=Contact.phpContact Us/a a href=Product.phpimg src=Images/product.jpg width=110 height=111 border=0/a Is there a way to invite bots to your site? Thanks Matt -- Raditha Dissanayake. -- -- http://www.radinks.com/sftp/ | http://www.raditha.com/megaupload Lean and mean Secure FTP applet with | Mega Upload - PHP file uploader Graphical User Inteface. Just 150 KB | with progress bar. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php __ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
is that it tends to avoid or downgrade URI's with parameters tacked onto the end. Does that go for internal links in your site then ??? news.php?start=10 etc ??? Could be tricky writing dyamic pages then. Thanks, Jon jon bennett | [EMAIL PROTECTED] new media creative _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ J b e n . n e t 91 Gloucester Rd, Trowbridge, Wilts, BA14 0AD t: +44 (0) 1225 341039 w: http://www.jben.net/ iChat (AIM): jbendotnet On 28 Jan 2004, at 01:05, Mike Migurski wrote: Is Googlebot (or any other bot) able to follow links that are php? As others have already pointed out to you, what better place to find information about google than google? That, or your server logs - googlebot is great about identifying itself. RTFM's aside, a little understanding of how HTTP request-response loop goes a long way. There's no reason why google (or any other bot) shouldn't be capable of making a GET request for a *.php resource. My understanding is that it tends to avoid or downgrade URI's with parameters tacked onto the end. - michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
Jon Bennett wrote: is that it tends to avoid or downgrade URI's with parameters tacked onto the end. Does that go for internal links in your site then ??? news.php?start=10 etc ??? Could be tricky writing dyamic pages then. Some spiders will not visit URLs with query strings, some will. Those that will generally put a limit on the number of links they will visit of that nature on a particular site to prevent it from following possibly infinite combinations. As stated in http://www.google.com/webmasters/2.html#A1... We are able to index dynamically generated pages. However, because our web crawler can easily overwhelm and crash sites serving dynamic content, we limit the amount of dynamic pages we index. Nice the way they say it's to help you not them :). As far as your tricky comment goes, dynamic content does not have to look like dynamic content. Check the archives for info on how to get rid of query strings from your site. -- Stuart -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
On Wed, 2004-01-28 at 08:34, Jon Bennett wrote: is that it tends to avoid or downgrade URI's with parameters tacked onto the end. Does that go for internal links in your site then ??? news.php?start=10 etc ??? Could be tricky writing dyamic pages then. mod_rewrite is your friend here. You can rewrite stuff like /foo/ /bar/ etc... to *.php?foo=barbar=foo. Kind of a pain in the arse but if you MUST have a spider following links then... -- BrianGnuPG - KeyID: 0x04A4F0DC | URL: www.gfx-design.com/keys Key Server: pgp.mit.edu == gpg --keyserver pgp.mit.edu --recv-keys 04A4F0DC GnuPG: http://gnupg.org http://www.biglumber.com/x/web?qs=0x2C35011004A4F0DC Linux Registered User #339825 at http://counter.li.org signature.asc Description: This is a digitally signed message part
Re: [PHP] Googlebot
Googlebot visits my site occasionally and follows a lot of my PHP links with long query strings... -Mike __ Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread! __ On Jan 28, 2004, at 8:34 AM, Jon Bennett wrote: Does that go for internal links in your site then ??? news.php?start=10 etc ??? Could be tricky writing dyamic pages then. Thanks, Jon -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
On Wed, Jan 28, 2004 at 11:51:06 -0500, Michael Mulligan wrote: Googlebot visits my site occasionally and follows a lot of my PHP links with long query strings... same here, google has been indexing my php site for over 3 years now :) -- A Pope has a Water Cannon. It is a Water Cannon. He fires Holy-Water from it.It is a Holy-Water Cannon. He Blesses it. It is a Holy Holy-Water Cannon. He Blesses the Hell out of it. It is a Wholly Holy Holy-Water Cannon. He has it pierced.It is a Holey Wholly Holy Holy-Water Cannon. He makes it official. It is a Canon Holey Wholly Holy Holy-Water Cannon. Yes, of course it's the right cabl [le0: NO CARRIER] pgp0.pgp Description: PGP signature
[PHP] Googlebot
Is Googlebot (or any other bot) able to follow links that are php? e.g. a href=Contact.phpContact Us/a a href=Product.phpimg src=Images/product.jpg width=110 height=111 border=0/a Is there a way to invite bots to your site? Thanks Matt
Re: [PHP] Googlebot
Hartley, Matt wrote: Is Googlebot (or any other bot) able to follow links that are php? e.g. a href=Contact.phpContact Us/a a href=Product.phpimg src=Images/product.jpg width=110 height=111 border=0/a Yes. Is there a way to invite bots to your site? For most, yes. -- Stuart -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
If you read the pages on google you will know the answer. Hartley, Matt wrote: Is Googlebot (or any other bot) able to follow links that are php? e.g. a href=Contact.phpContact Us/a a href=Product.phpimg src=Images/product.jpg width=110 height=111 border=0/a Is there a way to invite bots to your site? Thanks Matt -- Raditha Dissanayake. http://www.radinks.com/sftp/ | http://www.raditha.com/megaupload Lean and mean Secure FTP applet with | Mega Upload - PHP file uploader Graphical User Inteface. Just 150 KB | with progress bar. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
That's not very helpful. I am also curious about this. Anyone have a more thought provoking answer? Frank --- Raditha Dissanayake [EMAIL PROTECTED] wrote: If you read the pages on google you will know the answer. Hartley, Matt wrote: Is Googlebot (or any other bot) able to follow links that are php? e.g. a href=Contact.phpContact Us/a a href=Product.phpimg src=Images/product.jpg width=110 height=111 border=0/a Is there a way to invite bots to your site? Thanks Matt -- Raditha Dissanayake. http://www.radinks.com/sftp/ | http://www.raditha.com/megaupload Lean and mean Secure FTP applet with | Mega Upload - PHP file uploader Graphical User Inteface. Just 150 KB | with progress bar. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php __ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
Perhaps this link will help: http://www.google.com/webmasters/ Frank Tudor wrote: That's not very helpful. I am also curious about this. Anyone have a more thought provoking answer? Frank --- Raditha Dissanayake [EMAIL PROTECTED] wrote: If you read the pages on google you will know the answer. Hartley, Matt wrote: Is Googlebot (or any other bot) able to follow links that are php? e.g. a href=Contact.phpContact Us/a a href=Product.phpimg src=Images/product.jpg width=110 height=111 border=0/a Is there a way to invite bots to your site? Thanks Matt -- Raditha Dissanayake. http://www.radinks.com/sftp/ | http://www.raditha.com/megaupload Lean and mean Secure FTP applet with | Mega Upload - PHP file uploader Graphical User Inteface. Just 150 KB | with progress bar. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php __ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
Frank Tudor wrote: Anyone have a more thought provoking answer? Do some minimal research (Google is good for this) and you would find the answers yourself. Hartley, Matt wrote: Is Googlebot (or any other bot) able to follow links that are php? Yes. Most spiders will crawl over most types of URL. As Raditha said, Google has several pages of information on this for it's crawling activities, and most other legit spiders have similar resources. Is there a way to invite bots to your site? Google has a page where you can submit your site for crawling. Some other spiders also have this, but it depends on the purpose of the spider and the philosophy it follows. There are many, many, many, many, many, ..., many, many resources on search engine optimisation all over the web. Use them. Find them with... you guessed it... Google! -- Stuart -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
As others have already pointed out to you, what better place to find information about google than google? Frank Tudor wrote: That's not very helpful. I am also curious about this. Anyone have a more thought provoking answer? Frank --- Raditha Dissanayake [EMAIL PROTECTED] wrote: If you read the pages on google you will know the answer. Hartley, Matt wrote: Is Googlebot (or any other bot) able to follow links that are php? e.g. a href=Contact.phpContact Us/a a href=Product.phpimg src=Images/product.jpg width=110 height=111 border=0/a Is there a way to invite bots to your site? Thanks Matt -- Raditha Dissanayake. http://www.radinks.com/sftp/ | http://www.raditha.com/megaupload Lean and mean Secure FTP applet with | Mega Upload - PHP file uploader Graphical User Inteface. Just 150 KB | with progress bar. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php __ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ -- Raditha Dissanayake. http://www.radinks.com/sftp/ | http://www.raditha.com/megaupload Lean and mean Secure FTP applet with | Mega Upload - PHP file uploader Graphical User Inteface. Just 150 KB | with progress bar. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Googlebot
Is Googlebot (or any other bot) able to follow links that are php? As others have already pointed out to you, what better place to find information about google than google? That, or your server logs - googlebot is great about identifying itself. RTFM's aside, a little understanding of how HTTP request-response loop goes a long way. There's no reason why google (or any other bot) shouldn't be capable of making a GET request for a *.php resource. My understanding is that it tends to avoid or downgrade URI's with parameters tacked onto the end. - michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
