You need to set it in the header request you make. Putting it in the script 
you're using as a spider with ini_set won't do anything because the Target site 
doesn't know anything about it.


----- Reply message -----
From: "Ron Piggott" <>
Date: Thu, Nov 25, 2010 08:25
Subject: [PHP] Fw: Spoofing user_agent
To: <>

I have wrote a script to generate a sitemap of my web site.  It crawls all of 
the site web pages.  (About 30,000)  

I need help to spoof the user_agent variable so the stats program running in 
the background ( “AWSTATS” ) will treat the crawl as a bot, not browsing usage. 

The sitemap generator is a cron job.  I tried the syntax:
ini_set('user_agent', 'RonBot ('/);

This didn’t work.  The browsing was attributed to the dedicated IP address.

How do I get AWSTATS to access this, such as other entries under the 
“Robots/Spiders visitors” heading:
Unknown robot (identified by 'bot*') 

I don’t mean any ill will by changing this setting.  Thanks for the help.


The Verse of the Day
“Encouragement from God’s Word”

Reply via email to