On Wednesday June 15, 2005 17:22, Chris Wildman wrote: > I am trying to scrape a page on amazon that lists all the sellers selling a > given book.
Note that scraping pages may be against their terms of service (stupid, I know, but it could be); if you're just grabbing data for personal use they probably won't care. > For some reason making a request to this page(see below) with LWP returns > successful but there is never any content. Many other pages work. I can get > back the product detail page or the customer review page, etc. You might want to set the User-Agent to something browser-y. If that doesn't work, diff the headers your browser sends and the ones LWP sends and alter until you find the differentiator. -- Dave Isa. 40:31
