On 26 Apr, Damjan Jovanovic wrote: > On Mon, Nov 15, 2021 at 9:57 PM Jim Jagielski <j...@jagunet.com> wrote: > >> I'm gonna look into the serf->(lib)curl option... Since we don't use any >> of the fancy features of serf, I'm thinking that the easy option might be >> best > > > > Hi > > I've ported our WebDAV content provider module from Serf to Curl. > > While it ended well, and several other bugs were found and fixed, it > definitely wasn't the "easy option" Jim ;). Starting with conservative > changes, it ended up needing total restructuring, and became more of a > rewrite. The crashes were frequent and hung connections many, and I had to > read up on the HTTP protocol, and read Curl and Serf's source code, but > eventually I prevailed, and a clean elegant stable Curl WebDAV module > emerged. > > The huge patch is attached for anyone wishing to review and test. Unless > there are major objections, I'll push it in a couple of days. > > STATUS > > It builds and works well on FreeBSD and Windows. > > Most of the code was reused, and all the operations and semantics > previously present with Serf, should have been preserved. > > Browsing WebDAV files and directories, loading files, overwriting them > ("Save"), creating them ("Save As"), renaming and deleting them, all works. > > HTTP and HTTPS proxies work. Unlike Serf, Curl could also support SOCKS > proxies (with minimal further changes), but AOO lacks that setting in the > proxy UI and configuration. > > Authentication works, both to the destination server and to the proxy > server. I've successfully tested Basic and Digest authentication. Curl > supports every authentication method Serf does and more. > > HTTPS works, with a custom certificate verification function, using our own > certificate store from NSS and its API (like the Serf code used). A bug was > discovered (which is in the Serf implementation too) where self-signed > certificates were being unconditionally rejected; apparently NSS wants to > see that a copy of that certificate in its certificate chain parameter > too. Now they work, and the user gets prompted to allow access. > > HTTPS and authentication can be used together on the same connection and > work well, both bringing up their UI dialogs as needed. > > A bug was fixed where when username and password were both present in the > URL (dav://user:pass@host/path), the code was trying to split them at the > "@" instead of ":". > > Unnecessary base64 encoding and decoding was removed, when importing the > TLS connection's certificates into our XSecurityEnvironment. They now come > in directly as ASN.1 DER, and still work. > > The code was greatly restructured and cleaned up, as Curl's API is > synchronous and blocking, with parameters set in advance instead of through > many callbacks, which has allowed using short clear methods, and clean > separation between the session and request classes. The WebDAV content > provider has shrunk from 35 to 21 C++ files, 43 to 29 header files, and > 19129 to 15991 lines of code. With WebDAV methods centralized and > refactored into only 10-20 lines of code each, instead of scattered across > 4 files, it is much more understandable and maintainable now. > > Curl is vastly more portable than Serf. We should build easily now even on > OS/2. We can remain with existing build tools instead of needing scons or > cmake just to build Serf. > > 3 now unused dependencies were removed: apr, apr-util, and serf. Serf isn't > so bad. APR's pool idea is an ingenious way of doing resource management in > C. However Curl has excellent documentation, guides, examples, and detailed > explanations and even example code for each setting, while Serf has no > documentation. Serf might be worth it in a project that already uses APR a > lot, but we don't. > > Instead of the historical, crippled forms of logging like OSL_TRACE(), > which don't appear in release builds, I've made it use the newer > com.sun.star.logging UNO component (wrapped in comphelper::EventLogger), > which was inspired by java.util.logging, with configurable verbosity > levels, handlers (file and console) and output formats (plain, csv), and > importantly, which produces output in release builds too. I've also made it > so that on LogLevel::FINEST, Curl's verbose mode is enabled and Curl's > debug output is also logged through us, with descriptions of what Curl is > doing, and logs of all HTTP traffic including headers and bodies, before > encryption and after decryption in the case of HTTPS, giving us tremendous > detail that can be used for troubleshooting problems. > > CURL CHANGED TO USE OPENSSL AND ZLIB > > Curl only supports the custom TLS certificate verification function we use > (the CURLOPT_SSL_CTX_FUNCTION option) when built with OpenSSL, wolfSSL or > mbedTLS providers. We currently use schannel on Windows instead, which had > to be changed. I also made it use zlib, which generally helps, and WebDAV > uses XML which is very verbose and benefits from compression. On other OSes > with system curl, it is now checked for its SSL provider, and configure > fails if it isn't OpenSSL. > > The new WebDAV module successfully builds and runs with both OpenSSL 1.0.2 > or 1.1.1. However 1 function was renamed between those versions, so the > OpenSSL version at runtime probably has to match the one used at compile > time (although building with 1.0.2 headers might allow running with 1.1.1 - > not tested). > > (After completing development and testing, it dawned on me there is a > completely different way to do the certification verification, which should > allow other SSL providers to be used and might be better in various ways. > See later.) > > We currently build zlib as a static library only, and on Windows its C > runtime library is linked statically (_MT), which can't be mixed with > curl's dynamic linking of the runtime library (_MD). Thus curl was made to > link it statically too. Most if not all of our modules link the runtime > library statically too (which begs the question, why do we ship the msvcr* > redistributables to users at all then?). > > ISSUES > > The file open dialog (Ctrl+O) can hang for several minutes when first > connecting to a server. This is not new - it happens with Serf as well. > This appears to be caused by autocompletion in the file dialog. When typing > in a URL like "davs://127.0.0.1", the WebDAV content provider is first > called with a partial "https://1", before the rest of the URL is entered. > That "1" is (somehow) treated as IP address "0.0.0.1", and a TCP connection > to 0.0.0.1 is started. Only after several minutes, when that connection > times out and fails, does the content provider get another request with the > complete URL, which succeeds. The distinctly unpleasant wait, is luckily > only present that the first time that server is used, as caching remembers > the URL even across AOO restarts, so the "1" is automatically expanded to > the full URL and the content provider never sees "https://1" again. > > You can only enter credentials for the HTTP(S) proxy or the destination, > never both, as the credential manager caches credentials per URL, not per > host/realm, so while you are prompted for credentials for both, and Curl is > told about both, the destination credentials overwrite the proxy > credentials in the cache, so one Curl request works but future Curl > requests use the wrong credentials to the proxy. This doesn't matter, as > AOO doesn't support passwords for proxy servers anyway, and Serf has the > same issue.
Some of the CVEs mentioned by https://curl.se/docs/vuln-7.82.0.html may be now relavent for this new usage of curl. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org