Re: [plug] downloading a php result

[Daniel Escasa]

Sabi ko noong Fri, Apr 3, 2015 at 10:27 AM:

> Sometimes you don't need a script :)


Let me correct myself. You'll still need a script for changing the date
and hour parameters in the URL 

   *Daniel Escasa*
Technologist. Writer and Editor. Shakespearean actor
  

  
   "Oh, you hate your job? Why didn't you say so? There's a support group
for that. It's called EVERYBODY, and they meet at the bar." - Drew Carey

In necessariis unitas, in dubiis libertas, in omnibus caritas.
If we choose being kind over being right, we will be right every time.
 Get a signature like this:

Click
here!

_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph

Re: [plug] downloading a php result

[Roberto Verzola]

Ok thanks for the suggestion. I will try Scrapy. I tried httrack but I
got the same empty CSV file (all commas).

Obet

On Fri, 03 Apr 2015 09:49:00 +0800
w...@ateneo.edu wrote:

> You need to use a web crawler that can trigger the button pushed
> event such as Python Scrapy. If you are a Perl person then I used to
> use www-mechanize.
> 
> Sent on the move.
>   Original Message  
> From: Roberto Verzola
> Sent: Friday, April 3, 2015 9:45 AM
> To: Philippine Linux Users' Group (PLUG) Technical Discussion List
> Reply To: Philippine Linux Users' Group (PLUG) Technical Discussion
> List Subject: [plug] downloading a php result
> 
> The output of the url below is a .CSV file (hourly-updated electricity
> demand for Luzon).
> 
> http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10
> 
> This url is triggered when I click on a button online.
> 
> What I want to do is to download the resulting .csv file using wget.
> 
> But I keep getting an empty .csv file when using wget (I've tried
> several suggestion I got on the Web). I know a .csv file is generated
> somewhere because I get it when I click the CSV button online.
> 
> Any suggestions?
> 
> Have a peaceful Holy Week!
> 
> Obet
> _
> Philippine Linux Users' Group (PLUG) Mailing List
> http://lists.linux.org.ph/mailman/listinfo/plug
> Searchable Archives: http://archives.free.net.ph
> _
> Philippine Linux Users' Group (PLUG) Mailing List
> http://lists.linux.org.ph/mailman/listinfo/plug
> Searchable Archives: http://archives.free.net.ph
> 

_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph

Re: [plug] downloading a php result

[Roberto Verzola]

Got it. Thanks to all who gave suggestions! The simplest for me to
implement was Danny Escasa's. I just needed to quote the url, using
wget.

A peaceful Holy Week to all!

Obet


On Fri, 3 Apr 2015 10:27:56 +0800
Daniel Escasa  wrote:

> Sabi ni Obet noong Fri, Apr 3, 2015 at 9:45 AM:
> > The output of the url below is a .CSV file (hourly-updated
> > electricity demand for Luzon).
> >
> > http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10
> 
> >
> > This url is triggered when I click on a button online.
> >
> > What I want to do is to download the resulting .csv file using wget.
> >
> > But I keep getting an empty .csv file when using wget (I've tried
> > several suggestion I got on the Web). I know a .csv file is
> > generated somewhere because I get it when I click the CSV button
> > online.
> >
> > Any suggestions?
> 
> Worked on my FreeBSD partition. Also worked on my GNU/Linux partition,
> just tested it so's I don't have to convince you to install FreeBSD :)
> 
> Note that the URL contains *two* characters that the shell treats
> differently. You'll have to either escape them or enclose the URL in
> quotation marks, i.e:
> 
> wget "
> http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10";
> 
> Here's the result I got:
> 
> cat "luzon_dmd_csv_export.php?date=20150403&hour=10"
> HOUR,LUZDEM_DAPEL,LUZDEM_RTDEL,LUZDEM_RTXEL,LUZLWAP_DAPEL,LUZLWAP_RTDEL,LUZLWAP_RTXEL
> 
> 1,4770.1,4784.2,4775.3,1342.27,1409.69,1410.26
> 2,4411.5,4656.9,4624.7,44694.41,1632.39,1631.04
> 3,4248.6,4468.1,4474.6,46206.81,1625.77,1626.42
> 4,4150.9,4377.8,4300.2,47164.91,1449.69,1337.38
> 5,4277.8,4258.8,4188.5,1174.85,1174.34,1173.2
> 6,3864.1,4005.2,3981.3,49548.6,382.99,249.36
> 7,3704.3,3860.9,3897.2,51694.22,0,204.11
> 8,3825.3,4024.3,3982.9,58371.16,206.72,204.79
> 9,4158.1,4176.1,4169.1,516.84,518.46,508.79
> 10,4130.6,4343.2,,55169.33,810.06,
> 11,4265.5,,,53617.68,,
> 12,4259.4,,,53636.83,,
> 13,4410.7,,,52419.85,,
> 14,4481.8,,,51478.37,,
> 15,4448.4,,,51734.69,,
> 16,4339.4,,,51543.17,,
> 17,4226.3,,,52604.57,,
> 18,4361.7,,,51920.13,,
> 19,5050.8,,,56180.15,,
> 20,5148,,,55239.16,,
> 21,5234.5,,,54377.99,,
> 22,5352.8,,,53303.8,,
> 23,5186.6,,,55145.97,,
> 24,4915.6,,,49970.37,,
> 
> One other thing: I was too lazy to set an output file with -O
>  *output-file*.
> 
> 
> Sometimes you don't need a script :)
> 
>*Daniel Escasa*
> Technologist. Writer and Editor. Shakespearean actor
>   
> 
>   
>"Why can we remember the tiniest detail that has happened to us,
> and not remember how many times we have told it to the same person."
> - François de la...
> 
> In necessariis unitas, in dubiis libertas, in omnibus caritas.
> If we choose being kind over being right, we will be right every time.
>  Get a signature like this:
> 
> Click
> here!
> 

_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph

Re: [plug] downloading a php result

[Roberto Verzola]

Thanks! Will try this one too... I've used urllib2 before so I'm
familiar with some functions already. Greetings,

Obet


On Fri, 3 Apr 2015 09:55:42 +0800
Anuerin Diaz  wrote:

> Slightly better output when done using a python script:
> 
> 
> erin@Striker:/tmp$ cat wesm.py
> 
> import urllib2;
> response = urllib2.urlopen('
> http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10'
> );
> print response.read()
> 
> 
> erin@Striker:/tmp$ python wesm.py
> HOUR,LUZDEM_DAPEL,LUZDEM_RTDEL,LUZDEM_RTXEL,LUZLWAP_DAPEL,LUZLWAP_RTDEL,LUZLWAP_RTXEL
> 1,4770.1,4784.2,4775.3,1342.27,1409.69,1410.26
> 2,4411.5,4656.9,4624.7,44694.41,1632.39,1631.04
> 3,4248.6,4468.1,4474.6,46206.81,1625.77,1626.42
> 4,4150.9,4377.8,4300.2,47164.91,1449.69,1337.38
> 5,4277.8,4258.8,4188.5,1174.85,1174.34,1173.2
> 6,3864.1,4005.2,3981.3,49548.6,382.99,249.36
> 7,3704.3,3860.9,3897.2,51694.22,0,204.11
> 8,3825.3,4024.3,3982.9,58371.16,206.72,204.79
> 9,4158.1,4176.1,4169.1,516.84,518.46,508.79
> 10,4130.6,4343.2,,55169.33,810.06,
> 11,4265.5,,,53617.68,,
> 12,4259.4,,,53636.83,,
> 13,4410.7,,,52419.85,,
> 14,4481.8,,,51478.37,,
> 15,4448.4,,,51734.69,,
> 16,4339.4,,,51543.17,,
> 17,4226.3,,,52604.57,,
> 18,4361.7,,,51920.13,,
> 19,5050.8,,,56180.15,,
> 20,5148,,,55239.16,,
> 21,5234.5,,,54377.99,,
> 22,5352.8,,,53303.8,,
> 23,5186.6,,,55145.97,,
> 24,4915.6,,,49970.37,,
> 
> 
> EOF
> erin@Striker:/tmp$
> 
> 
> it can be improved further but i think this works around the blank csv
> challenge. :)
> 
> ciao!
> 
> On Fri, Apr 3, 2015 at 9:45 AM, Roberto Verzola 
> wrote:
> 
> > The output of the url below is a .CSV file (hourly-updated
> > electricity demand for Luzon).
> >
> > http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10
> >
> > This url is triggered when I click on a button online.
> >
> > What I want to do is to download the resulting .csv file using wget.
> >
> > But I keep getting an empty .csv file when using wget (I've tried
> > several suggestion I got on the Web). I know a .csv file is
> > generated somewhere because I get it when I click the CSV button
> > online.
> >
> > Any suggestions?
> >
> > Have a peaceful Holy Week!
> >
> > Obet
> > _
> > Philippine Linux Users' Group (PLUG) Mailing List
> > http://lists.linux.org.ph/mailman/listinfo/plug
> > Searchable Archives: http://archives.free.net.ph
> >
> 
> 
> 

_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph

Re: [plug] downloading a php result

[Daniel Escasa]

Sabi ni Obet noong Fri, Apr 3, 2015 at 9:45 AM:
> The output of the url below is a .CSV file (hourly-updated electricity
> demand for Luzon).
>
> http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10

>
> This url is triggered when I click on a button online.
>
> What I want to do is to download the resulting .csv file using wget.
>
> But I keep getting an empty .csv file when using wget (I've tried
> several suggestion I got on the Web). I know a .csv file is generated
> somewhere because I get it when I click the CSV button online.
>
> Any suggestions?

Worked on my FreeBSD partition. Also worked on my GNU/Linux partition,
just tested it so's I don't have to convince you to install FreeBSD :)

Note that the URL contains *two* characters that the shell treats
differently. You'll have to either escape them or enclose the URL in
quotation marks, i.e:

wget "
http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10";

Here's the result I got:

cat "luzon_dmd_csv_export.php?date=20150403&hour=10"
HOUR,LUZDEM_DAPEL,LUZDEM_RTDEL,LUZDEM_RTXEL,LUZLWAP_DAPEL,LUZLWAP_RTDEL,LUZLWAP_RTXEL

1,4770.1,4784.2,4775.3,1342.27,1409.69,1410.26
2,4411.5,4656.9,4624.7,44694.41,1632.39,1631.04
3,4248.6,4468.1,4474.6,46206.81,1625.77,1626.42
4,4150.9,4377.8,4300.2,47164.91,1449.69,1337.38
5,4277.8,4258.8,4188.5,1174.85,1174.34,1173.2
6,3864.1,4005.2,3981.3,49548.6,382.99,249.36
7,3704.3,3860.9,3897.2,51694.22,0,204.11
8,3825.3,4024.3,3982.9,58371.16,206.72,204.79
9,4158.1,4176.1,4169.1,516.84,518.46,508.79
10,4130.6,4343.2,,55169.33,810.06,
11,4265.5,,,53617.68,,
12,4259.4,,,53636.83,,
13,4410.7,,,52419.85,,
14,4481.8,,,51478.37,,
15,4448.4,,,51734.69,,
16,4339.4,,,51543.17,,
17,4226.3,,,52604.57,,
18,4361.7,,,51920.13,,
19,5050.8,,,56180.15,,
20,5148,,,55239.16,,
21,5234.5,,,54377.99,,
22,5352.8,,,53303.8,,
23,5186.6,,,55145.97,,
24,4915.6,,,49970.37,,

One other thing: I was too lazy to set an output file with -O
 *output-file*.


Sometimes you don't need a script :)

   *Daniel Escasa*
Technologist. Writer and Editor. Shakespearean actor
  

  
   "Why can we remember the tiniest detail that has happened to us, and not
remember how many times we have told it to the same person." - François de
la...

In necessariis unitas, in dubiis libertas, in omnibus caritas.
If we choose being kind over being right, we will be right every time.
 Get a signature like this:

Click
here!

_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph

Re: [plug] downloading a php result

[Anuerin Diaz]

Slightly better output when done using a python script:


erin@Striker:/tmp$ cat wesm.py

import urllib2;
response = urllib2.urlopen('
http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10'
);
print response.read()


erin@Striker:/tmp$ python wesm.py
HOUR,LUZDEM_DAPEL,LUZDEM_RTDEL,LUZDEM_RTXEL,LUZLWAP_DAPEL,LUZLWAP_RTDEL,LUZLWAP_RTXEL
1,4770.1,4784.2,4775.3,1342.27,1409.69,1410.26
2,4411.5,4656.9,4624.7,44694.41,1632.39,1631.04
3,4248.6,4468.1,4474.6,46206.81,1625.77,1626.42
4,4150.9,4377.8,4300.2,47164.91,1449.69,1337.38
5,4277.8,4258.8,4188.5,1174.85,1174.34,1173.2
6,3864.1,4005.2,3981.3,49548.6,382.99,249.36
7,3704.3,3860.9,3897.2,51694.22,0,204.11
8,3825.3,4024.3,3982.9,58371.16,206.72,204.79
9,4158.1,4176.1,4169.1,516.84,518.46,508.79
10,4130.6,4343.2,,55169.33,810.06,
11,4265.5,,,53617.68,,
12,4259.4,,,53636.83,,
13,4410.7,,,52419.85,,
14,4481.8,,,51478.37,,
15,4448.4,,,51734.69,,
16,4339.4,,,51543.17,,
17,4226.3,,,52604.57,,
18,4361.7,,,51920.13,,
19,5050.8,,,56180.15,,
20,5148,,,55239.16,,
21,5234.5,,,54377.99,,
22,5352.8,,,53303.8,,
23,5186.6,,,55145.97,,
24,4915.6,,,49970.37,,


EOF
erin@Striker:/tmp$


it can be improved further but i think this works around the blank csv
challenge. :)

ciao!

On Fri, Apr 3, 2015 at 9:45 AM, Roberto Verzola  wrote:

> The output of the url below is a .CSV file (hourly-updated electricity
> demand for Luzon).
>
> http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10
>
> This url is triggered when I click on a button online.
>
> What I want to do is to download the resulting .csv file using wget.
>
> But I keep getting an empty .csv file when using wget (I've tried
> several suggestion I got on the Web). I know a .csv file is generated
> somewhere because I get it when I click the CSV button online.
>
> Any suggestions?
>
> Have a peaceful Holy Week!
>
> Obet
> _
> Philippine Linux Users' Group (PLUG) Mailing List
> http://lists.linux.org.ph/mailman/listinfo/plug
> Searchable Archives: http://archives.free.net.ph
>



-- 
"Programming, an artform that fights back"

Anuerin G. Diaz
Registered Linux User #246176
http://ramfree17.net/capsule , when you absolutely have nothing else better
to do
_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph

Re: [plug] downloading a php result

[wyu]

You need to use a web crawler that can trigger the button pushed event such as 
Python Scrapy. If you are a Perl person then I used to use www-mechanize.

Sent on the move.
  Original Message  
From: Roberto Verzola
Sent: Friday, April 3, 2015 9:45 AM
To: Philippine Linux Users' Group (PLUG) Technical Discussion List
Reply To: Philippine Linux Users' Group (PLUG) Technical Discussion List
Subject: [plug] downloading a php result

The output of the url below is a .CSV file (hourly-updated electricity
demand for Luzon).

http://wesm.ph/chart/export/luzon_dmd_csv_export.php?date=20150403&hour=10

This url is triggered when I click on a button online.

What I want to do is to download the resulting .csv file using wget.

But I keep getting an empty .csv file when using wget (I've tried
several suggestion I got on the Web). I know a .csv file is generated
somewhere because I get it when I click the CSV button online.

Any suggestions?

Have a peaceful Holy Week!

Obet
_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph
_
Philippine Linux Users' Group (PLUG) Mailing List
http://lists.linux.org.ph/mailman/listinfo/plug
Searchable Archives: http://archives.free.net.ph