Re: How to read a web page?

[ToddAndMargo via perl6-users]

> On Tue, Apr 7, 2020 at 1:58 PM ToddAndMargo via perl6-users
> mailto:perl6-users@perl.org>> wrote:
>
> On 2020-04-07 02:55, Veesh Goldman wrote:
>  > Your question is way to vague. What do you actually want to do?
> If you
>  > just wanna download a page, use curl or some other tool, no
> reason to do
>  > it from raku if there's nothing more than that.
>  > If you want more than that, then please be more specific.
>
> Hi Veesh,
>
> Sorry for the confusion.  I only wanted a small
> piece of a puzzle I am putting together.
>
> This big picture is this.
>
> I have a customer that runs RDP (Remote Desktop Protocol)
> server on Window 7.  He logs into this server with
> a Window 10 Pro laptop from all around the country using
> mstsc.exe and WFreeRDP.
>
> His new Internet provider at this house will use
> dynamic (floating) WAN (Wide Area Network) IP's.
>
> And he uses DropBox.  So I am writing a raku program
> on his RDP server to upload his WAN IP to Dropbox
> once an hour and another program on his RDP client
> to read the drop box file and wrap mstsc and WFree
> RDP with his server's WAN address.
>
> That is also why I asked the DropBox question.
> (DropBox works marvelously on Fedora 31).  But
> Since Drop Box and Cobian Backup on Windows
> don't mesh, I am being extra careful. (I think
> the issue is Cobain's use of Volume Shadow
> Copy Service.)
>
> -T
>

On 2020-04-07 05:43, Veesh Goldman wrote:
Are you familiar with dynamic dns providers like duckdns.org 
 (there are others, too). That coupled with a script 
that updates your ip with the provider, like ddclient (written in 
perl!), should be a more straightforward way of accomplishing what you want.
Also, thank you for clarifying what you are trying to do. Because the 
answer changes a lot once you want to parse the page that you download.




Hi Veesh,

I have looked at services like that.  I like the Raku
route better as I can do everything very seamlessly:
automatic WAN address check and update, top loading
and pruning log file of WAN address changes, automatic
creating of mstsc and WFreeRDP run stings, ignore
and warn if the Internet is down, etc..  All the user has
to do it click on an icon.  Other than which icon to
click on, no other user interaction.

It would help if the developers would implement:

RFE: shell / no shell switch
https://github.com/rakudo/rakudo/issues/3582

-T

And you guys wonder what I did with all the help you
have been giving me!

Re: How to read a web page?

[Veesh Goldman]

Are you familiar with dynamic dns providers like duckdns.org (there are
others, too). That coupled with a script that updates your ip with the
provider, like ddclient (written in perl!), should be a more
straightforward way of accomplishing what you want.
Also, thank you for clarifying what you are trying to do. Because the
answer changes a lot once you want to parse the page that you download.

On Tue, Apr 7, 2020 at 1:58 PM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> On 2020-04-07 02:55, Veesh Goldman wrote:
> > Your question is way to vague. What do you actually want to do? If you
> > just wanna download a page, use curl or some other tool, no reason to do
> > it from raku if there's nothing more than that.
> > If you want more than that, then please be more specific.
>
> Hi Veesh,
>
> Sorry for the confusion.  I only wanted a small
> piece of a puzzle I am putting together.
>
> This big picture is this.
>
> I have a customer that runs RDP (Remote Desktop Protocol)
> server on Window 7.  He logs into this server with
> a Window 10 Pro laptop from all around the country using
> mstsc.exe and WFreeRDP.
>
> His new Internet provider at this house will use
> dynamic (floating) WAN (Wide Area Network) IP's.
>
> And he uses DropBox.  So I am writing a raku program
> on his RDP server to upload his WAN IP to Dropbox
> once an hour and another program on his RDP client
> to read the drop box file and wrap mstsc and WFree
> RDP with his server's WAN address.
>
> That is also why I asked the DropBox question.
> (DropBox works marvelously on Fedora 31).  But
> Since Drop Box and Cobian Backup on Windows
> don't mesh, I am being extra careful. (I think
> the issue is Cobain's use of Volume Shadow
> Copy Service.)
>
> -T
>

Re: How to read a web page?

[ToddAndMargo via perl6-users]

On 2020-04-07 02:55, Veesh Goldman wrote:
Your question is way to vague. What do you actually want to do? If you 
just wanna download a page, use curl or some other tool, no reason to do 
it from raku if there's nothing more than that.

If you want more than that, then please be more specific.


Hi Veesh,

Sorry for the confusion.  I only wanted a small
piece of a puzzle I am putting together.

This big picture is this.

I have a customer that runs RDP (Remote Desktop Protocol)
server on Window 7.  He logs into this server with
a Window 10 Pro laptop from all around the country using
mstsc.exe and WFreeRDP.

His new Internet provider at this house will use
dynamic (floating) WAN (Wide Area Network) IP's.

And he uses DropBox.  So I am writing a raku program
on his RDP server to upload his WAN IP to Dropbox
once an hour and another program on his RDP client
to read the drop box file and wrap mstsc and WFree
RDP with his server's WAN address.

That is also why I asked the DropBox question.
(DropBox works marvelously on Fedora 31).  But
Since Drop Box and Cobian Backup on Windows
don't mesh, I am being extra careful. (I think
the issue is Cobain's use of Volume Shadow
Copy Service.)

-T

Re: How to read a web page?

[ToddAndMargo via perl6-users]

On 2020-04-07 02:38, Shlomi Fish wrote:

Hi Todd!

On Tue, 7 Apr 2020 02:15:13 -0700
ToddAndMargo via perl6-users  wrote:


How do I read (download) a web page?  get?


See https://github.com/raku-community-modules/raku-lwp-simple and
https://github.com/shlomif/Freenode-programming-channel-FAQ/blob/master/FAQ_with_ToC__generated.md#how-can-i-write-code-to-perform-operations-on-web-sites-for-me-that-otherwise-should-be-done-manually
.

Note that there are many file types and formats that can be found on the WWW
and downloaded using HTTP/HTTPS.


Thank you!  I figured it out.

Re: How to read a web page?

[Shlomi Fish]

Hi Todd!

On Tue, 7 Apr 2020 02:15:13 -0700
ToddAndMargo via perl6-users  wrote:

> How do I read (download) a web page?  get?

See https://github.com/raku-community-modules/raku-lwp-simple and
https://github.com/shlomif/Freenode-programming-channel-FAQ/blob/master/FAQ_with_ToC__generated.md#how-can-i-write-code-to-perform-operations-on-web-sites-for-me-that-otherwise-should-be-done-manually
.

Note that there are many file types and formats that can be found on the WWW
and downloaded using HTTP/HTTPS.



-- 

Shlomi Fish   https://www.shlomifish.org/
NSA Factoids - https://www.shlomifish.org/humour/bits/facts/NSA/

Barth's Distinction:  There are two types of people: those who divide people
into two types, and those who don't.
— via fortune-mod.

Please reply to list if it's a mailing list post - https://shlom.in/reply .

Re: How to read a web page?

[ToddAndMargo via perl6-users]

> On Tue, Apr 7, 2020 at 12:15 PM ToddAndMargo via perl6-users
> mailto:perl6-users@perl.org>> wrote:
>
> How do I read (download) a web page?  get?

On 2020-04-07 02:55, Veesh Goldman wrote:
Your question is way to vague. What do you actually want to do? If you 
just wanna download a page, use curl or some other tool, no reason to do 
it from raku if there's nothing more than that.

If you want more than that, then please be more specific.


Hi Veesh,

My question sounded vague as it was meant to be simple.
I wanted to give a url and get back the HTML of the url.

As far as curl goes, I wrote a module that
interfaces with curl.  It would do this perfectly,
except I wanted something very simple in this instance:
a pea shooter instead of a howitzer (my curl module).


$ raku -e 'use HTTP::Client; my $client=HTTP::Client.new;
 my $response =  $client.get("http://checkip.dyndns.org/";);
 say $response.content;
 (my $x=$response.content) ~~ s/.*? "Address: "//;
 $x~~s/\< .* //;
 say "WAN IP Address is " ~ $x;'

 Current IP CheckCurrent IP 
Address: 50.37.24.122


 WAN IP Address is 50.37.24.122

Now is there were nasty redirects to follow or cookie or
referrers or headers to input, yada, yada yada, my curl
module would work perfect.  I just wanted something simple
this time.

-T

Re: How to read a web page?

[ToddAndMargo via perl6-users]

On 2020-04-07 02:15, ToddAndMargo via perl6-users wrote:

How do I read (download) a web page?  get?


Came up with this:

  # zef --force-test install HTTP::Client

  $ raku -e 'use HTTP::Client; my $client=HTTP::Client.new; my 
$response = $client.get("http://checkip.dyndns.org/";); say 
$response.content;'


   Current IP CheckCurrent IP 
Address: 50.37.24.122

Re: How to read a web page?

[Veesh Goldman]

Your question is way to vague. What do you actually want to do? If you just
wanna download a page, use curl or some other tool, no reason to do it from
raku if there's nothing more than that.
If you want more than that, then please be more specific.

On Tue, Apr 7, 2020 at 12:15 PM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> How do I read (download) a web page?  get?
>

How to read a web page?

[ToddAndMargo via perl6-users]

How do I read (download) a web page?  get?