On 26 Apr, Damjan Jovanovic wrote:
> On Mon, Nov 15, 2021 at 9:57 PM Jim Jagielski <j...@jagunet.com> wrote:
>
>> I'm gonna look into the serf->(lib)curl option... Since we don't use any
>> of the fancy features of serf, I'm thinking that the easy option might be
>> best
>
>
>
> Hi
>
> I've ported our WebDAV content provider module from Serf to Curl.
>
> While it ended well, and several other bugs were found and fixed, it
> definitely wasn't the "easy option" Jim ;). Starting with conservative
> changes, it ended up needing total restructuring, and became more of a
> rewrite. The crashes were frequent and hung connections many, and I had to
> read up on the HTTP protocol, and read Curl and Serf's source code, but
> eventually I prevailed, and a clean elegant stable Curl WebDAV module
> emerged.
>
> The huge patch is attached for anyone wishing to review and test. Unless
> there are major objections, I'll push it in a couple of days.
>
> STATUS
>
> It builds and works well on FreeBSD and Windows.
>
> Most of the code was reused, and all the operations and semantics
> previously present with Serf, should have been preserved.
>
> Browsing WebDAV files and directories, loading files, overwriting them
> ("Save"), creating them ("Save As"), renaming and deleting them, all works.
>
> HTTP and HTTPS proxies work. Unlike Serf, Curl could also support SOCKS
> proxies (with minimal further changes), but AOO lacks that setting in the
> proxy UI and configuration.
>
> Authentication works, both to the destination server and to the proxy
> server. I've successfully tested Basic and Digest authentication. Curl
> supports every authentication method Serf does and more.
>
> HTTPS works, with a custom certificate verification function, using our own
> certificate store from NSS and its API (like the Serf code used). A bug was
> discovered (which is in the Serf implementation too) where self-signed
> certificates were being unconditionally rejected; apparently NSS wants to
> see that a copy of that certificate in its certificate chain parameter
> too. Now they work, and the user gets prompted to allow access.
>
> HTTPS and authentication can be used together on the same connection and
> work well, both bringing up their UI dialogs as needed.
>
> A bug was fixed where when username and password were both present in the
> URL (dav://user:pass@host/path), the code was trying to split them at the
> "@" instead of ":".
>
> Unnecessary base64 encoding and decoding was removed, when importing the
> TLS connection's certificates into our XSecurityEnvironment. They now come
> in directly as ASN.1 DER, and still work.
>
> The code was greatly restructured and cleaned up, as Curl's API is
> synchronous and blocking, with parameters set in advance instead of through
> many callbacks, which has allowed using short clear methods, and clean
> separation between the session and request classes. The WebDAV content
> provider has shrunk from 35 to 21 C++ files, 43 to 29 header files, and
> 19129 to 15991 lines of code. With WebDAV methods centralized and
> refactored into only 10-20 lines of code each, instead of scattered across
> 4 files, it is much more understandable and maintainable now.
>
> Curl is vastly more portable than Serf. We should build easily now even on
> OS/2. We can remain with existing build tools instead of needing scons or
> cmake just to build Serf.
>
> 3 now unused dependencies were removed: apr, apr-util, and serf. Serf isn't
> so bad. APR's pool idea is an ingenious way of doing resource management in
> C. However Curl has excellent documentation, guides, examples, and detailed
> explanations and even example code for each setting, while Serf has no
> documentation. Serf might be worth it in a project that already uses APR a
> lot, but we don't.
>
> Instead of the historical, crippled forms of logging like OSL_TRACE(),
> which don't appear in release builds, I've made it use the newer
> com.sun.star.logging UNO component (wrapped in comphelper::EventLogger),
> which was inspired by java.util.logging, with configurable verbosity
> levels, handlers (file and console) and output formats (plain, csv), and
> importantly, which produces output in release builds too. I've also made it
> so that on LogLevel::FINEST, Curl's verbose mode is enabled and Curl's
> debug output is also logged through us, with descriptions of what Curl is
> doing, and logs of all HTTP traffic including headers and bodies, before
> encryption and after decryption in the case of HTTPS, giving us tremendous
> detail that can be used for troubleshooting problems.
>
> CURL CHANGED TO USE OPENSSL AND ZLIB
>
> Curl only supports the custom TLS certificate verification function we use
> (the CURLOPT_SSL_CTX_FUNCTION option) when built with OpenSSL, wolfSSL or
> mbedTLS providers. We currently use schannel on Windows instead, which had
> to be changed. I also made it use zlib, which generally helps, and WebDAV
> uses XML which is very verbose and benefits from compression. On other OSes
> with system curl, it is now checked for its SSL provider, and configure
> fails if it isn't OpenSSL.
>
> The new WebDAV module successfully builds and runs with both OpenSSL 1.0.2
> or 1.1.1. However 1 function was renamed between those versions, so the
> OpenSSL version at runtime probably has to match the one used at compile
> time (although building with 1.0.2 headers might allow running with 1.1.1 -
> not tested).
>
> (After completing development and testing, it dawned on me there is a
> completely different way to do the certification verification, which should
> allow other SSL providers to be used and might be better in various ways.
> See later.)
>
> We currently build zlib as a static library only, and on Windows its C
> runtime library is linked statically (_MT), which can't be mixed with
> curl's dynamic linking of the runtime library (_MD). Thus curl was made to
> link it statically too. Most if not all of our modules link the runtime
> library statically too (which begs the question, why do we ship the msvcr*
> redistributables to users at all then?).
>
> ISSUES
>
> The file open dialog (Ctrl+O) can hang for several minutes when first
> connecting to a server. This is not new - it happens with Serf as well.
> This appears to be caused by autocompletion in the file dialog. When typing
> in a URL like "davs://127.0.0.1", the WebDAV content provider is first
> called with a partial "https://1";, before the rest of the URL is entered.
> That "1" is (somehow) treated as IP address "0.0.0.1", and a TCP connection
> to 0.0.0.1 is started. Only after several minutes, when that connection
> times out and fails, does the content provider get another request with the
> complete URL, which succeeds. The distinctly unpleasant wait, is luckily
> only present that the first time that server is used, as caching remembers
> the URL even across AOO restarts, so the "1" is automatically expanded to
> the full URL and the content provider never sees "https://1"; again.
>
> You can only enter credentials for the HTTP(S) proxy or the destination,
> never both, as the credential manager caches credentials per URL, not per
> host/realm, so while you are prompted for credentials for both, and Curl is
> told about both, the destination credentials overwrite the proxy
> credentials in the cache, so one Curl request works but future Curl
> requests use the wrong credentials to the proxy. This doesn't matter, as
> AOO doesn't support passwords for proxy servers anyway, and Serf has the
> same issue.
Some of the CVEs mentioned by https://curl.se/docs/vuln-7.82.0.html may
be now relavent for this new usage of curl.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org
