Re: retrieval of data from a database
* Philip Stephens ([EMAIL PROTECTED]) wrote: Hello wget users and developers, [...] [EMAIL PROTECTED]:~/Documents/projects/Boston/psdiss/code$ wget -dv http://www.bostonresourcenet.org/detail.cfm?StartRow=2EndRow=99SearchType=detailedVarZip=VarMiles=0keyword=ServID=17OrgTypeID=LangID=SpecialID=LocationID= Setting --verbose (verbose) to 1 DEBUG output created by Wget 1.11.2 on linux-gnu. --2008-06-09 14:08:08-- http://www.bostonresourcenet.org/detail.cfm?StartRow=2 Resolving www.bostonresourcenet.org... [1] 12857 [2] 12858 [3] 12859 [4] 12860 [5] 12861 [6] 12862 [7] 12863 [8] 12864 [9] 12865 [10] 12866 [2] DoneEndRow=99 [3] DoneSearchType=detailed [4] DoneVarZip= [5] DoneVarMiles=0 [6] Donekeyword= [7] DoneServID=17 [8] DoneOrgTypeID= [9] DoneLangID= [10]+ DoneSpecialID= Well, you'd better escape the '' in the shell (\)... Regards, Saint Xavier.
RE: retrieval of data from a database
Saint Xavier wrote: Well, you'd better escape the '' in your shell (\) It's probably easier to just put quotes around the entire URL than to try to find all the special characters and put backslashes in front of them. Tony
retrieval of data from a database
Hello wget users and developers, I am attempting to retrieve public information from BostonResourceNet ( http://www.bostonresourcenet.org/index.cfm ). There are four drop-down boxes that allow you to filter the database. After retrieving the results of a particular query, I can select a non-profit, for instance, and copy the link, then pass that long link back into the browser and retrieve the company's information. Of course, I could just click the link and get the info as well. My point is that I can pass the long link (as seen in the wget query below) directly to Firefox and retrieve the data. However, when I try to pass the long link to wget, I get the following: (Note that the second return to the command line was not initiated by me). Is wget an appropriate tool for this kind of operation? I plan to script this, replacing variables in the string that is passed to wget. Thanks. [EMAIL PROTECTED]:~/Documents/projects/Boston/psdiss/code$ wget -dv http://www.bostonresourcenet.org/detail.cfm?StartRow=2EndRow=99SearchType=detailedVarZip=VarMiles=0keyword=ServID=17OrgTypeID=LangID=SpecialID=LocationID= Setting --verbose (verbose) to 1 DEBUG output created by Wget 1.11.2 on linux-gnu. --2008-06-09 14:08:08-- http://www.bostonresourcenet.org/detail.cfm?StartRow=2 Resolving www.bostonresourcenet.org... [1] 12857 [2] 12858 [3] 12859 [4] 12860 [5] 12861 [6] 12862 [7] 12863 [8] 12864 [9] 12865 [10] 12866 [2] DoneEndRow=99 [3] DoneSearchType=detailed [4] DoneVarZip= [5] DoneVarMiles=0 [6] Donekeyword= [7] DoneServID=17 [8] DoneOrgTypeID= [9] DoneLangID= [10]+ DoneSpecialID= [EMAIL PROTECTED]:~/Documents/projects/Boston/psdiss/code$ 69.25.69.36 Caching www.bostonresourcenet.org = 69.25.69.36 Connecting to www.bostonresourcenet.org|69.25.69.36|:80... connected. Created socket 3. Releasing 0x08090e48 (new refcount 1). ---request begin--- GET /detail.cfm?StartRow=2 HTTP/1.0 User-Agent: Wget/1.11.2 Accept: */* Host: www.bostonresourcenet.org Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 500 Internal Server Error Connection: close Date: Mon, 09 Jun 2008 21:08:09 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET server-error: true Content-Type: text/html; charset=UTF-8 Set-Cookie: CFID=765597;expires=Wed, 02-Jun-2038 21:00:15 GMT;path=/ Set-Cookie: CFTOKEN=93009364;expires=Wed, 02-Jun-2038 21:00:15 GMT;path=/ ---response end--- 500 Internal Server Error Stored cookie www.bostonresourcenet.org -1 (ANY) / session insecure [expiry none] CFID 765597 Stored cookie www.bostonresourcenet.org -1 (ANY) / session insecure [expiry none] CFTOKEN 93009364 Closed fd 3 2008-06-09 14:08:09 ERROR 500: Internal Server Error.
Re: retrieval of data from a database
On Mon, Jun 9, 2008 at 2:56 PM, Philip Stephens [EMAIL PROTECTED] wrote: Hello wget users and developers, I am attempting to retrieve public information from BostonResourceNet ( http://www.bostonresourcenet.org/index.cfm ). There are four drop-down boxes that allow you to filter the database. After retrieving the results of a particular query, I can select a non-profit, for instance, and copy the link, then pass that long link back into the browser and retrieve the company's information. Of course, I could just click the link and get the info as well. My point is that I can pass the long link (as seen in the wget query below) directly to Firefox and retrieve the data. However, when I try to pass the long link to wget, I get the following: (Note that the second return to the command line was not initiated by me). Is wget an appropriate tool for this kind of operation? I plan to script this, replacing variables in the string that is passed to wget. Thanks. [EMAIL PROTECTED]:~/Documents/projects/Boston/psdiss/code$ wget -dv http://www.bostonresourcenet.org/detail.cfm?StartRow=2EndRow=99SearchType=detailedVarZip=VarMiles=0keyword=ServID=17OrgTypeID=LangID=SpecialID=LocationID= Setting --verbose (verbose) to 1 DEBUG output created by Wget 1.11.2 on linux-gnu. --2008-06-09 14:08:08-- http://www.bostonresourcenet.org/detail.cfm?StartRow=2 Resolving www.bostonresourcenet.org... [1] 12857 [2] 12858 [3] 12859 [4] 12860 [5] 12861 [6] 12862 [7] 12863 [8] 12864 [9] 12865 [10] 12866 [2] DoneEndRow=99 [3] DoneSearchType=detailed [4] DoneVarZip= [5] DoneVarMiles=0 [6] Donekeyword= [7] DoneServID=17 [8] DoneOrgTypeID= [9] DoneLangID= [10]+ DoneSpecialID= [EMAIL PROTECTED]:~/Documents/projects/Boston/psdiss/code$ 69.25.69.36 Caching www.bostonresourcenet.org = 69.25.69.36 Connecting to www.bostonresourcenet.org|69.25.69.36|:80... connected. Created socket 3. Releasing 0x08090e48 (new refcount 1). ---request begin--- GET /detail.cfm?StartRow=2 HTTP/1.0 User-Agent: Wget/1.11.2 Accept: */* Host: www.bostonresourcenet.org Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 500 Internal Server Error Connection: close Date: Mon, 09 Jun 2008 21:08:09 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET server-error: true Content-Type: text/html; charset=UTF-8 Set-Cookie: CFID=765597;expires=Wed, 02-Jun-2038 21:00:15 GMT;path=/ Set-Cookie: CFTOKEN=93009364;expires=Wed, 02-Jun-2038 21:00:15 GMT;path=/ ---response end--- 500 Internal Server Error Stored cookie www.bostonresourcenet.org -1 (ANY) / session insecure [expiry none] CFID 765597 Stored cookie www.bostonresourcenet.org -1 (ANY) / session insecure [expiry none] CFTOKEN 93009364 Closed fd 3 2008-06-09 14:08:09 ERROR 500: Internal Server Error. I should also mention that at this point, the process hangs and must be killed with CTRl-C. -- Philip Stephens Doctoral Student Joint Geography Program San Diego State University UC Santa Barbara
Re: retrieval of data from a database
* Philip Stephens ([EMAIL PROTECTED]) wrote: Hello wget users and developers, [...] [EMAIL PROTECTED]:~/Documents/projects/Boston/psdiss/code$ wget -dv http://www.bostonresourcenet.org/detail.cfm?StartRow=2EndRow=99SearchType=detailedVarZip=VarMiles=0keyword=ServID=17OrgTypeID=LangID=SpecialID=LocationID= Setting --verbose (verbose) to 1 DEBUG output created by Wget 1.11.2 on linux-gnu. --2008-06-09 14:08:08-- http://www.bostonresourcenet.org/detail.cfm?StartRow=2 Resolving www.bostonresourcenet.org... [1] 12857 [2] 12858 [3] 12859 [4] 12860 [5] 12861 [6] 12862 [7] 12863 [8] 12864 [9] 12865 [10] 12866 [2] DoneEndRow=99 [3] DoneSearchType=detailed [4] DoneVarZip= [5] DoneVarMiles=0 [6] Donekeyword= [7] DoneServID=17 [8] DoneOrgTypeID= [9] DoneLangID= [10]+ DoneSpecialID= Well, you'd better escape the '' in your shell (\) Regards, Saint Xavier.