Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 05:54 PM, ToddAndMargo wrote:

On 07/28/2017 12:48 PM, ToddAndMargo wrote:

On 07/28/2017 04:31 AM, Gabor Szabo wrote:
On Fri, Jul 28, 2017 at 9:49 AM, ToddAndMargo  
wrote:

Hi All,

I am trying to convert a p5 program to p6.  What do I use in
place of `LWP::UserAgent`?

I use it for downloading files from the web.  I need to be able
to pass the following to the web page:

 Caller
 Host
 UserAgent
 Referer
 Cookies

This is the p5 code I want to convert:

http://vpaste.net/gtJgj

Any words of wisdom?

Many thanks,
-T


On 07/27/2017 10:30 PM, Gabor Szabo wrote:


LWP::Simple now allows you to set the header of your request.
See my recent article with examples:
http://perl6maven.com/simple-web-client
I hope this helps.

regards
 Gabor

On Fri, Jul 28, 2017 at 7:42 AM, Todd Chester 
wrote:




I see your article.  I do believe this is what I want.

{
   "args": {
 "language": "Perl",
 "math": "19+23=42",
 "name": "Larry Wall"
   },
   "headers": {
 "Connection": "close",
 "Host": "httpbin.org",
 "User-Agent": "LWP::Simple/0.090 Perl6/rakudo"
   },
   "origin": "17.19.208.37",
   "url": "http://httpbin.org/get?name=Larry
Wall&language=Perl&math=19%2B23%3D42"
}


Questions:

1) may I leave off the `args` and only include the `headers`?

2) I need an example with headers.  I have no clue what goes
before the first "{"

Many thanks,
-T


I think you quoted the response here and not the request.
What you need I think is the last example on that page.
Something like this:

my $html = LWP::Simple.new.get("http://httpbin.org/headers";, {
 "User-Agent" => "Perl 6 Maven articles",
 "Zone" => "q" }
);


Gabor




Thank you!

how do I save to a file?






And how do I get it to follow links, like `curl -L`?



I was just on the perl (5) chat line. One of the developers
uncovered the issue I was having following link in P5's
LWP::UserAgent.   The default is 7 redirects. I cough on
two because tcp dump truncates the return string (it is long).

They said they would look at it and see if they could fix it
and then eMail me.

Man I try across a lot of bugs!  



--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 07:03 PM, Timo Paulssen wrote:

there's an example in the readme that goes like this:

 my $curl = LibCurl::Easy.new(:verbose, :followlocation);

the followlocation part should be right for you




I love it.  Thank you!


--
~~~
Serious error.
All shortcuts have disappeared.
Screen. Mind. Both are blank.
~~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Timo Paulssen 
there's an example in the readme that goes like this:

my $curl = LibCurl::Easy.new(:verbose, :followlocation);

the followlocation part should be right for you

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 06:55 PM, Timo Paulssen wrote:

If you use the other interface where you create a curl object a la
LibCurl::Easy.new, you can just

 $my_curl_object.Host("the-host.com");
 $my_curl_object.referer("example.com");
 $my_curl_object.cookie("the-cookie");
 $my_curl_object.useragent("me");

hth
   - Timo



That is intuitive.  I love it.  Thank you!

Anything for following links?



--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Timo Paulssen 
If you use the other interface where you create a curl object a la
LibCurl::Easy.new, you can just

$my_curl_object.Host("the-host.com");
$my_curl_object.referer("example.com");
$my_curl_object.cookie("the-cookie");
$my_curl_object.useragent("me");

hth
  - Timo

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 06:49 PM, Timo Paulssen wrote:

Did you not see this part of the readme?

 # And if you need headers, pass them inside a positional Hash:
 say post 'https://httpbin.org/post?foo=42&bar=x',
%(:Some),
 :some, :42args;



Thank you.

What a nightmare to figure out.



--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 06:36 PM, Bennett Todd wrote:

As for LWP being a pain, I see it differently,  filling the specs for a web 
client --- many specs --- is a pain. The curl project is trying to wrap a 
blanket around that pain.



The pain is trying to figure out how to use the various headers.
The docs are written for those that already know how to do it.

It took me two weeks of postings and research to figure out
download to a file.

It is:
':content_file' => "$FileName",
by the way.

-T


--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Timo Paulssen 
Did you not see this part of the readme?

# And if you need headers, pass them inside a positional Hash:
say post 'https://httpbin.org/post?foo=42&bar=x',
%(:Some),
:some, :42args;

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 06:36 PM, Brian Duggan wrote:

On Friday, July 28, ToddAndMargo wrote:

I have been fighting with this all day and gave up an hour ago
and just did a system call to curl (this is P5 code):

$CurlStatus = system (
  "curl -L -b $AcceptCookie $ClickHere -o $NewFileName" );


There are also perl 6 bindings to libcurl, e.g.

 http://modules.perl6.org/dist/LibCurl

 From the examples, it looks like ':followlocation' corresponds
to '-L'.

Brian




I like it!  Thank you!



--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 06:33 PM, Bennett Todd wrote:

I just googled perl6 libcurl, and got

https://github.com/CurtTilmes/perl6-libcurl

That's definitely where I'd start.



Hi Bennett,

Looks pretty but it is too reduced function.

I need to follow links and to include the
following headers:

   Host
   UserAgent
   Referer
   Cookies

So poop!

Thank you for looking!

:-)

-T

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Bennett Todd 
Thank you! I'd have hated for my echo of your knowledge to be the only tip he 
got, and you included a pointer to the option he needed, which I didn't try 
from my phone:-)

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Bennett Todd 
As for LWP being a pain, I see it differently,  filling the specs for a web 
client --- many specs --- is a pain. The curl project is trying to wrap a 
blanket around that pain.

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Brian Duggan 
On Friday, July 28, ToddAndMargo wrote: 
> I have been fighting with this all day and gave up an hour ago
> and just did a system call to curl (this is P5 code):
> 
> $CurlStatus = system (
>  "curl -L -b $AcceptCookie $ClickHere -o $NewFileName" );

There are also perl 6 bindings to libcurl, e.g.

http://modules.perl6.org/dist/LibCurl

>From the examples, it looks like ':followlocation' corresponds
to '-L'.

Brian

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Bennett Todd 
I just googled perl6 libcurl, and got

https://github.com/CurtTilmes/perl6-libcurl

That's definitely where I'd start.

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 06:07 PM, Bennett Todd wrote:

I'm not expert in curl, nor even http client coding, so I just checked the  
curl(1) man page on my system, it says:

-L, --location
   (HTTP) If the server reports that the
   requested page has moved to a different
   location (indicated with a Location:
   header and a 3XX response code), this
   option will make curl redo the request on
   the new place. If used together with -i,
   --include or -I, --head, headers from all
   requested pages will be shown. When
   authentication is used, curl only sends
   its credentials to the initial host. If a
   redirect takes curl to a different host,
   it won't be able to intercept the
   user+password. See also --location-
   trusted on how to change this. You can
   limit the amount of redirects to follow
   by using the --max-redirs option.

If this hasn't already been done in perl6 natively, it might be nicest to just 
invoke curl.

It depends on your goal. If you want to expand the perl6 ecosystem, that'd be 
great. But if you just want to get the job done, there are advantages to just 
invoking curl; web standards evolve really fast, and curl is pretty actively 
maintained as far as I know (it's been a little while since I was actively 
tracking curl development).




In P5, LWP::UserAgent does not follow links or I do not know how.
There is a way to get it to find a redirected link, but it doesn't 
always  include anything past the "?" in the returned URL.



I have been fighting with this all day and gave up an hour ago
and just did a system call to curl (this is P5 code):

$CurlStatus = system (
 "curl -L -b $AcceptCookie $ClickHere -o $NewFileName" );

This comes from a 4986 line piece of P5 code I wrote to
download all the new revision of software I need to
place on flash drive to take to my customer sites.

I will be eventually migrating it to P6, as I adore P6 and it
will be a lot easier to maintain.  I program in Top Down and
P5's sub's annoy the pants off of me.

I may just to system calls to curl as you suggested as it is
a lot easier than LWP::UserAgent, which is a pain-in-the-ass.







--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Bennett Todd 
I'm not expert in curl, nor even http client coding, so I just checked the  
curl(1) man page on my system, it says:

-L, --location
  (HTTP) If the server reports that the
  requested page has moved to a different
  location (indicated with a Location:
  header and a 3XX response code), this
  option will make curl redo the request on
  the new place. If used together with -i,
  --include or -I, --head, headers from all
  requested pages will be shown. When
  authentication is used, curl only sends
  its credentials to the initial host. If a
  redirect takes curl to a different host,
  it won't be able to intercept the
  user+password. See also --location-
  trusted on how to change this. You can
  limit the amount of redirects to follow
  by using the --max-redirs option.

If this hasn't already been done in perl6 natively, it might be nicest to just 
invoke curl.

It depends on your goal. If you want to expand the perl6 ecosystem, that'd be 
great. But if you just want to get the job done, there are advantages to just 
invoking curl; web standards evolve really fast, and curl is pretty actively 
maintained as far as I know (it's been a little while since I was actively 
tracking curl development).

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 12:48 PM, ToddAndMargo wrote:

On 07/28/2017 04:31 AM, Gabor Szabo wrote:
On Fri, Jul 28, 2017 at 9:49 AM, ToddAndMargo  
wrote:

Hi All,

I am trying to convert a p5 program to p6.  What do I use in
place of `LWP::UserAgent`?

I use it for downloading files from the web.  I need to be able
to pass the following to the web page:

 Caller
 Host
 UserAgent
 Referer
 Cookies

This is the p5 code I want to convert:

http://vpaste.net/gtJgj

Any words of wisdom?

Many thanks,
-T


On 07/27/2017 10:30 PM, Gabor Szabo wrote:


LWP::Simple now allows you to set the header of your request.
See my recent article with examples:
http://perl6maven.com/simple-web-client
I hope this helps.

regards
 Gabor

On Fri, Jul 28, 2017 at 7:42 AM, Todd Chester 
wrote:




I see your article.  I do believe this is what I want.

{
   "args": {
 "language": "Perl",
 "math": "19+23=42",
 "name": "Larry Wall"
   },
   "headers": {
 "Connection": "close",
 "Host": "httpbin.org",
 "User-Agent": "LWP::Simple/0.090 Perl6/rakudo"
   },
   "origin": "17.19.208.37",
   "url": "http://httpbin.org/get?name=Larry
Wall&language=Perl&math=19%2B23%3D42"
}


Questions:

1) may I leave off the `args` and only include the `headers`?

2) I need an example with headers.  I have no clue what goes
before the first "{"

Many thanks,
-T


I think you quoted the response here and not the request.
What you need I think is the last example on that page.
Something like this:

my $html = LWP::Simple.new.get("http://httpbin.org/headers";, {
 "User-Agent" => "Perl 6 Maven articles",
 "Zone" => "q" }
);


Gabor




Thank you!

how do I save to a file?






And how do I get it to follow links, like `curl -L`?

--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: : question

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 02:52 PM, Brandon Allbery wrote:
On Fri, Jul 28, 2017 at 5:49 PM, ToddAndMargo > wrote:


On 07/28/2017 02:41 PM, Brandon Allbery wrote:

> That's not Perl, it's JSON generated by Perl.

Mumble, mumble ...  I have a hard enough time learning
Perl without someone throwing in another language.
Mumble ...


You're doing web stuff, it's going to get a lot worse before it gets 
better. JSON, JavaScript, possibly having to deal with web support 
components in other languages, 


I am trying to migrate the following from Perl 5 to Perl 6.
I like p5 but I adore P6.


   $ua = LWP::UserAgent->new;
   $ua->timeout ( MaxTime1 );
   $ua->show_progress ( 1 );   # 0 = do not show; 1 = show
   $response = $ua->get(
$Url,
':content_file' => "$FileName",
'Cookie'=> "$Cookies",
'Referer'   => "$Referer",
'User-Agent'=> "$UserAgent",
'Host'  => "$Host"
);

   if ( ! $response->is_success )
  { $PageStatus = DOWNLOAD_FAIL;
print STDERR BOLD RED
   "   ERROR ${Caller}.GetWebPage: unable to read ",
   $Url, RESET, "\n";
print STDERR BOLD RED Dumper($response), RESET, "\n";
return DOWNLOAD_FAIL;
  }

   return $PageStatus;

Re: : question

2017-07-28 Thread Timo Paulssen 
This article uses httpbin.org which is a nice service that'll take any
kind of request and tell you exactly what it saw.

It shows what it got as a json dictionary literal.

Here's an example for your browser, so you can just click it:

http://httpbin.org/anything?hello-todd-how-are-you=very-good-thank-you

Hope that helps
  - Timo

Re: : question

2017-07-28 Thread Brandon Allbery 
On Fri, Jul 28, 2017 at 5:49 PM, ToddAndMargo  wrote:

> On 07/28/2017 02:41 PM, Brandon Allbery wrote:

>>> > That's not Perl, it's JSON generated by Perl.
>
> Mumble, mumble ...  I have a hard enough time learning
> Perl without someone throwing in another language.
> Mumble ...
>

You're doing web stuff, it's going to get a lot worse before it gets
better. JSON, JavaScript, possibly having to deal with web support
components in other languages, 


-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net

Re: : question

2017-07-28 Thread ToddAndMargo 

Now I am really confused.  It is all over the place on
http://perl6maven.com/simple-web-client


For instance:

{
   "args": {
 "language": "Perl",
 "math": "19+23=42",
 "name": "Larry Wall"
   },
   "headers": {
 "Connection": "close",
 "Host": "httpbin.org ",
 "User-Agent": "LWP::Simple/0.090 Perl6/rakudo"
   },
   "origin": "17.19.208.37",
   "url": "http://httpbin.org/get?name=Larry

Wall&language=Perl&math=19%2B23%3D42"
}


Confusedly yours,
-T
On Fri, Jul 28, 2017 at 5:39 PM, ToddAndMargo > wrote:


On 07/28/2017 02:02 PM, Timo Paulssen wrote:

The first one is valid perl 6 code and the bottom one is not. It's
likely you were looking at JSON (or equivalently JavaScript) and
confused that with perl6 code.





On 07/28/2017 02:41 PM, Brandon Allbery wrote:
> That's not Perl, it's JSON generated by Perl.

Mumble, mumble ...  I have a hard enough time learning
Perl without someone throwing in another language.
Mumble ...

Thank you for the help.

And "=>" is just assigning values to an associative array
not assigning values to a reference pointer

Re: : question

2017-07-28 Thread Brandon Allbery 
That's not Perl, it's JSON generated by Perl.

On Fri, Jul 28, 2017 at 5:39 PM, ToddAndMargo  wrote:

> On 07/28/2017 02:02 PM, Timo Paulssen wrote:
>
>> The first one is valid perl 6 code and the bottom one is not. It's
>> likely you were looking at JSON (or equivalently JavaScript) and
>> confused that with perl6 code.
>>
>>
> Now I am really confused.  It is all over the place on
> http://perl6maven.com/simple-web-client
>
> For instance:
>
> {
>   "args": {
> "language": "Perl",
> "math": "19+23=42",
> "name": "Larry Wall"
>   },
>   "headers": {
> "Connection": "close",
> "Host": "httpbin.org",
> "User-Agent": "LWP::Simple/0.090 Perl6/rakudo"
>   },
>   "origin": "17.19.208.37",
>   "url": "http://httpbin.org/get?name=Larry Wall&language=Perl&math=19%2B2
> 3%3D42"
> }
>
>
> Confusedly yours,
> -T
>



-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net

Re: : question

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 02:02 PM, Timo Paulssen wrote:

The first one is valid perl 6 code and the bottom one is not. It's
likely you were looking at JSON (or equivalently JavaScript) and
confused that with perl6 code.



Now I am really confused.  It is all over the place on
http://perl6maven.com/simple-web-client

For instance:

{
  "args": {
"language": "Perl",
"math": "19+23=42",
"name": "Larry Wall"
  },
  "headers": {
"Connection": "close",
"Host": "httpbin.org",
"User-Agent": "LWP::Simple/0.090 Perl6/rakudo"
  },
  "origin": "17.19.208.37",
  "url": "http://httpbin.org/get?name=Larry 
Wall&language=Perl&math=19%2B23%3D42"

}


Confusedly yours,
-T

Re: : question

2017-07-28 Thread Timo Paulssen 
The first one is valid perl 6 code and the bottom one is not. It's
likely you were looking at JSON (or equivalently JavaScript) and
confused that with perl6 code.

: question

2017-07-28 Thread ToddAndMargo 

Hi All,

Is this

 "User-Agent" => "Perl 6 Maven articles"

the same as this?

 "User-Agent": "Perl 6 Maven articles"

And are we assigning to reference pointer or are
we addressing a hash pair?

Many thanks,
-T

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread ToddAndMargo 

On 07/28/2017 04:31 AM, Gabor Szabo wrote:

On Fri, Jul 28, 2017 at 9:49 AM, ToddAndMargo  wrote:

Hi All,

I am trying to convert a p5 program to p6.  What do I use in
place of `LWP::UserAgent`?

I use it for downloading files from the web.  I need to be able
to pass the following to the web page:

 Caller
 Host
 UserAgent
 Referer
 Cookies

This is the p5 code I want to convert:

http://vpaste.net/gtJgj

Any words of wisdom?

Many thanks,
-T


On 07/27/2017 10:30 PM, Gabor Szabo wrote:


LWP::Simple now allows you to set the header of your request.
See my recent article with examples:
http://perl6maven.com/simple-web-client
I hope this helps.

regards
 Gabor

On Fri, Jul 28, 2017 at 7:42 AM, Todd Chester 
wrote:




I see your article.  I do believe this is what I want.

{
   "args": {
 "language": "Perl",
 "math": "19+23=42",
 "name": "Larry Wall"
   },
   "headers": {
 "Connection": "close",
 "Host": "httpbin.org",
 "User-Agent": "LWP::Simple/0.090 Perl6/rakudo"
   },
   "origin": "17.19.208.37",
   "url": "http://httpbin.org/get?name=Larry
Wall&language=Perl&math=19%2B23%3D42"
}


Questions:

1) may I leave off the `args` and only include the `headers`?

2) I need an example with headers.  I have no clue what goes
before the first "{"

Many thanks,
-T


I think you quoted the response here and not the request.
What you need I think is the last example on that page.
Something like this:

my $html = LWP::Simple.new.get("http://httpbin.org/headers";, {
 "User-Agent" => "Perl 6 Maven articles",
 "Zone" => "q" }
);


Gabor




Thank you!

how do I save to a file?



--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Need sub for `LWP::UserAgent`

2017-07-28 Thread Gabor Szabo 
On Fri, Jul 28, 2017 at 9:49 AM, ToddAndMargo  wrote:
>>> Hi All,
>>>
>>> I am trying to convert a p5 program to p6.  What do I use in
>>> place of `LWP::UserAgent`?
>>>
>>> I use it for downloading files from the web.  I need to be able
>>> to pass the following to the web page:
>>>
>>> Caller
>>> Host
>>> UserAgent
>>> Referer
>>> Cookies
>>>
>>> This is the p5 code I want to convert:
>>>
>>>http://vpaste.net/gtJgj
>>>
>>> Any words of wisdom?
>>>
>>> Many thanks,
>>> -T
>
> On 07/27/2017 10:30 PM, Gabor Szabo wrote:
>>
>> LWP::Simple now allows you to set the header of your request.
>> See my recent article with examples:
>> http://perl6maven.com/simple-web-client
>> I hope this helps.
>>
>> regards
>> Gabor
>>
>> On Fri, Jul 28, 2017 at 7:42 AM, Todd Chester 
>> wrote:
>
>
>
> I see your article.  I do believe this is what I want.
> 
> {
>   "args": {
> "language": "Perl",
> "math": "19+23=42",
> "name": "Larry Wall"
>   },
>   "headers": {
> "Connection": "close",
> "Host": "httpbin.org",
> "User-Agent": "LWP::Simple/0.090 Perl6/rakudo"
>   },
>   "origin": "17.19.208.37",
>   "url": "http://httpbin.org/get?name=Larry
> Wall&language=Perl&math=19%2B23%3D42"
> }
> 
>
> Questions:
>
> 1) may I leave off the `args` and only include the `headers`?
>
> 2) I need an example with headers.  I have no clue what goes
>before the first "{"
>
> Many thanks,
> -T

I think you quoted the response here and not the request.
What you need I think is the last example on that page.
Something like this:

my $html = LWP::Simple.new.get("http://httpbin.org/headers";, {
"User-Agent" => "Perl 6 Maven articles",
"Zone" => "q" }
);


Gabor