Decoding base64 in parameter parsing

2018-09-02 Thread Philipp Hagemeister via curl-library
I would like to propose an option --data-as-base64 which allows users to
pass in data containing null bytes, something which is otherwise
impossible without the use of temporary files or pipelines, both of
which are not options for me. See https://github.com/curl/curl/pull/2931
for more details.

However, it seems I messed up my pull request, as linking crashes with:

/usr/bin/ld: curl-tool_getparam.o: in function `getparameter':
tool_getparam.c:(.text+0x2f2f): undefined reference to `Curl_base64_decode'

I can now reproduce the problem, but I'm not sure how I did manage to
build a working version of curl with my changes (it even passed the
automated and a number of manual tests I did!).

So, how can I use the Curl_base64_decode function in the tool?

1) Copy code, but that seems wrong.
2) Writing and including a smaller base64 decoder seems like a bad idea
as well.
3) Curl_base64_decode is (rightly) not part of the public libcurl API,
or is it?
4) curlx seems to be made for these cases, but I'm not sure how to
properly configure that. If I just add base64.c to CURLX_CFILES in
Makefile.inc, the linker is missing Curl_cmalloc . How do I get _that_?
5) I could switch from base64 to another encoding, such as URL-encoding.
Maybe that's simpler to parse. But base64 seems best for binary data
("only" 1/3 overhead).
6) Is there any other option?

Any help would be much appreciated,

Philipp



signature.asc
Description: OpenPGP digital signature
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: libssh2 optimization [was: Re: Windows users! Help us test upload performance tuning?]

2018-09-02 Thread Daniel Jeliński via curl-library
niedz., 2 wrz 2018 o 13:33 Jan Ehrhardt via curl-library
 napisał(a):
>
> Do you have a compiled version somewhere?

I'm hacking this on Linux, I don't have a proper testing environment
on my Windows machine.

> I tried to build my own with the 3 patches:
>
> 1. winsock
> 2. oploadbuffer 512 KB
> 3. sftp writeback in libssh2
>
> Dissappointing results for a 274MB sftp upload against a remote CoreFTP 
> server:
> - bash / ssh / lftp: 68 seconds
> - openssh portable sftp: 50 seconds
> - curl triple patched: 734 seconds
> ping of the remote machine: 3-4 ms.

That's like 5MB/s best case, 300KB/s worst case; what's your limiting
factor? Network?

Daniel

---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: libssh2 optimization [was: Re: Windows users! Help us test upload performance tuning?]

2018-09-02 Thread Jan Ehrhardt via curl-library
Jan Ehrhardt via curl-library (Sun, 26 Aug 2018 16:03:16 +0200):
>Daniel Jelinski via curl-library (Sun, 26 Aug 2018 09:09:58 +0200):
>>added 100ms: original 310kB/sec, patched 9900kB/sec
>
>Impressive results!

Do you have a compiled version somewhere?

I tried to build my own with the 3 patches:

1. winsock
2. oploadbuffer 512 KB
3. sftp writeback in libssh2

Dissappointing results for a 274MB sftp upload against a remote CoreFTP server:
- bash / ssh / lftp: 68 seconds
- openssh portable sftp: 50 seconds
- curl triple patched: 734 seconds
ping of the remote machine: 3-4 ms. Used cipher in openssh portable sftp:
AES128-CBC. In production we have an old version of the CoreFTP server. On our
dev machine I am going to install the most recent CoreFTP server version.
-- 
Jan

---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html