Re: [Mojolicious] curl works, Mojo::UserAgent doesn't

2020-05-15 Thread lackhoff via Mojolicious 
Am 14.05.2020 um 05:08 schrieb Stefan Adams:
> I'm certain you'll be able to accomplish this with M::UA. Can you share

Just to reduce complexity I changed everything to plain HTTP, so no SSL,
keys or certificates.

> what the HTML error produced is?

It is not really a normal error page, just a page from the UI. The most
relevant lines are these:
Invalid variable name.
Wiederholen Sie das Update oder starten Sie die FRITZ!Box neu.
[German for: repeat the update(sic! I wanted to backup the box, not
update) or restart your FRITZ!Box]

> Can you share the output that you are comparing that you are noticing is so
> similar between curl and M::UA? You're using -v for curl. Also set the
> MOJO_CLIENT_DEBUG env variable to 1.

Here is what curl gives (with --trace-ascii):
=> Send header, 200 bytes (0xc8)
: POST /cgi-bin/firmwarecfg HTTP/1.1
0024: Host: fritz.box
0035: User-Agent: curl/7.58.0
004e: Accept: */*
005b: Content-Length: 370
0070: Content-Type: multipart/form-data; boundary=
00b0: 565a580ebeb0c567
00c6:
=> Send data, 370 bytes (0x172)
: --565a580ebeb0c567
002c: Content-Disposition: form-data; name="sid"
0058:
005a: 4e240384c30c909d
006c: --565a580ebeb0c567
0098: Content-Disposition: form-data; name="ImportExportPassword"
00d5:
00d7: backup
00df: --565a580ebeb0c567
010b: Content-Disposition: form-data; name="ConfigExport"
0140:
0142:
0144: --565a580ebeb0c567--
<= Recv header, 17 bytes (0x11)
: HTTP/1.1 200 OK


> See how it works comparing curl to:
> 
>$ env MOJO_CLIENT_DEBUG=1 mojo get -v -M POST -f sid=$SID -f
> ImportExportPassword=$BAKPWD -f ConfigExport= http
> ://fritz.box/cgi-bin/firmwarecfg 

The empty ConfigExport parameter gets lost with this command and the
Content-Type is different:

-- Blocking request (http://fritz.box/cgi-bin/firmwarecfg)
-- Connect 7c84d2d7396d9d2ff9571982f73240cb (http://fritz.box:80)
-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
POST /cgi-bin/firmwarecfg HTTP/1.1\x0d
Content-Length: 48\x0d
User-Agent: Mojolicious (Perl)\x0d
Accept-Encoding: gzip\x0d
Content-Type: application/x-www-form-urlencoded\x0d
Host: fritz.box\x0d
\x0d
ImportExportPassword=backup=4e240384c30c909d
-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)

-- Client <<< Server (http://fritz.box/cgi-bin/firmwarecfg)
HTTP/1.1 200 OK\x0d



With DEBUG on and my code (s. my original message and below) I get this:

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
POST /cgi-bin/firmwarecfg HTTP/1.1\x0d
Accept-Encoding: gzip, deflate\x0d
Content-Type: multipart/form-data; boundary=tyUPX\x0d
Accept: */*\x0d
Host: fritz.box\x0d
Content-Length: 230\x0d
User-Agent: Mojolicious (Perl)\x0d
\x0d
--tyUPX\x0d

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
Content-Disposition: form-data; name="ConfigExport"\x0d
\x0d

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
\x0d
--tyUPX\x0d

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
Content-Disposition: form-data; name="ImportExportPassword"\x0d
\x0d

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
backup
-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
\x0d
--tyUPX\x0d

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
Content-Disposition: form-data; name="sid"\x0d
\x0d

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
33cb272bcd34940b
-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)
\x0d
--tyUPX--\x0d

-- Client >>> Server (http://fritz.box/cgi-bin/firmwarecfg)

-- Client <<< Server (http://fritz.box/cgi-bin/firmwarecfg)
HTTP/1.1 200 OK\x0d

which for me looks very much the same as the curl version but as I said,
curl gives the backup file whereas M::U just gives the HTML page back,
indicating that something went wrong.
I can only spot very minor differences that is why I am running out of
ideas:
- the boundary used by M::U is quite short
- the form parameters are not in the same order
- Accept-Encoding header (is also sent from Firefox where it works)
- the different SID is normal (my script always makes a fresch login)

Looks very strange to me
-Michael

> On Wed, May 13, 2020, 4:56 PM 'Michael Lackhoff' via Mojolicious <
> mojolicious@googlegroups.com> wrote:
> 
>> I had quite a bit of success recently using Mojo::UserAgent so I tried to
>> replace a curl command to do a backup of my Fritz.box router with M::U.
>>
>> Here is the curl command:
>> curl -s -k -o $OUT --form sid=$SID --form ImportExportPassword=$BAKPWD \
>> --form ConfigExport= http://fritz.box/cgi-bin/firmwarecfg
>>
>> It should be equivalent to this M::U request:
>>
>> my $tx = $ua->build_tx(
>> POST => 'http://fritz.box/cgi-bin/firmwarecfg' =>
>> {
>> 'Accept'   => '*/*',
>> 'Content-Type' => 'multipart/form-data',
>> } => form => {
>> sid

Re: [Mojolicious] curl works, Mojo::UserAgent doesn't

2020-05-13 Thread Stefan Adams 
I'm certain you'll be able to accomplish this with M::UA. Can you share
what the HTML error produced is?

Can you share the output that you are comparing that you are noticing is so
similar between curl and M::UA? You're using -v for curl. Also set the
MOJO_CLIENT_DEBUG env variable to 1.

See how it works comparing curl to:

   $ env MOJO_CLIENT_DEBUG=1 mojo get -v -M POST -f sid=$SID -f
ImportExportPassword=$BAKPWD -f ConfigExport= http
://fritz.box/cgi-bin/firmwarecfg 

On Wed, May 13, 2020, 4:56 PM 'Michael Lackhoff' via Mojolicious <
mojolicious@googlegroups.com> wrote:

> I had quite a bit of success recently using Mojo::UserAgent so I tried to
> replace a curl command to do a backup of my Fritz.box router with M::U.
>
> Here is the curl command:
> curl -s -k -o $OUT --form sid=$SID --form ImportExportPassword=$BAKPWD \
> --form ConfigExport= http://fritz.box/cgi-bin/firmwarecfg
>
> It should be equivalent to this M::U request:
>
> my $tx = $ua->build_tx(
> POST => 'http://fritz.box/cgi-bin/firmwarecfg' =>
> {
> 'Accept'   => '*/*',
> 'Content-Type' => 'multipart/form-data',
> } => form => {
> sid  => $SID,
> ImportExportPassword => $BAKPWD,
> ConfigExport => '',
> }
> );
>
> # for debugging:
> print $tx->req->to_string;
>
> $tx = $ua->start($tx);
> $tx->res->save_to($OUT);
>
> As far as I can tell both the headers and the POST body is very much the
> same (except the boundary value to separate the form fields) but to my
> surprise the curl command works ($OUT is the backup file) but with the M::U
> version $OUT consists of some HTML output indicating an error.
>
> If I could see a difference I could try to better adjust my script but as
> I said, they look very much the same (I compared it with the -v and
> --trace-ascii output of curl), so I run out of ideas what could trigger the
> differnt response of my Fritz.box.
> Any ideas? At the moment I just solve it by using the curl command with
> "system" but I would prefer a Perl-only solution and what is even more
> important to me: I want to understand what is going on here.
>
> -Michael
>
> --
> You received this message because you are subscribed to the Google Groups
> "Mojolicious" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mojolicious+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mojolicious/14df452c-e7a4-4b7c-90a2-9e1f5e15becc%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/mojolicious/CACyQ%2BFR1F7fqH3%3DFJCMftHuwH1rEKwyPZ%2BO0g7d%3DJcRaWYqDtw%40mail.gmail.com.