Re: Dynamic OpenSSL causes Curl regression on *nix
Hi All, Am 02.05.24 um 14:02 schrieb Matthias Seidel: Hi Arrigo, Am 01.05.24 um 20:31 schrieb Arrigo Marchiori: Hello All, TL;DR: I think that keeping distinct branches for trunk and AOO42X is good practice, and we shall work on stabilizing AOO42X. More below. To be clear, I am not thinking of build or stability problems when I say that we cannot release AOO 4.2.0. Even if all release blockers in AOO42X are fixed, we do not have a working release process! Translations are outdated and we have no way to get them into our code or new strings into our translation process. This should be a primary problem to be solved. I would have expected that we start a discussion about improving the release process at this point. Or at least start with a new "Roadmap" for 2024? Regards, Matthias Regards, Matthias On Wed, May 01, 2024 at 03:52:19PM +0200, Matthias Seidel wrote: Hi Pedro, Am 01.05.24 um 15:39 schrieb Pedro Lino: Hi Matthias On 05/01/2024 1:30 PM WEST Matthias Seidel wrote: Maybe it is easier to delete the current 42x branch and start a new 42X branch? If any patches were added to 42X only (and they are still relevant), then these should be cherry picked to trunk before splitting... +1. The AOO42X branch is quite old. Development only really happens on trunk. The best maintained branch is trunk, and any branches should be as close to trunk as possible. Trunk is an unstable branch. That is the nature of Trunk ;) Yes! And should not be the nature of AOO42X, IMHO. Everything we want to have in AOO42X (which was branched from trunk some years ago) should have been cherry-picked early. Do you mean that all patches/fixes in 42X are in Trunk also? Trunk has all patches, AOO42X only those that were cherry-picked from trunk. I confirm this. I personally made some patch ``backwards'' (first in another branch, and then cherry-picked into trunk) but it was my mistake: our best practice shall be as Matthias described above. Therefore there would not be a problem in deleting the current branch 42X and start a new one? This would make sense because Damjan mentioned that many relevant fixes in trunk are not included in 42X If he didn't cherry-pick them (for a reason?) they are not included. ...and if no one else did ;-) I personally cherry-picked the SSL- and curl-related commits because of some tests I am doing on AOO42X. I think that it makes sense to cherry-pick also his latest change of 16-bit into 32-bit indices. Also why is the next release 4.2.x? Haven't we made enough changes for 5.0.x? If not, what should be in 5.0.x? It is quite hypothetical to discuss about versions we are NOT capable to release. It is. But Damjan's question is still valid? What would be the change needed to consider this 5.0? I would say that having a Winx64 would be such a condition. As it is, I still see this as 42X For AOO 5.0 I would expect at least that we support ODF 1.3. A Win64 build would be great for AOO 4.5.0. I personally would see both these improvements good for AOO 4.2.0, but I think it's only a matter of... marketing and promotion. This topic is worth another thread. But again, we are not able to do a release, we should work on that problem first. If we are talking about ``technical'' issues, then I agree that AOO42X is unstable and for this reason it cannot be released. The next version we CAN release is 4.1.16 (AOO41X). That is true! If we become able to do a release for a newer branch (AOO42X), we should simply release what we have... Exactly, but including all valid updates that have been added to trunk but not cherry picked to 42X +1 If they were not cherry-picked I would expect there was a reason for it? I think that we have to discuss the meaning of the ``valid'' word as used above by Pedro. Almost all commits I saw to trunk would be IMHO applicable to AOO42X. I do not think there is anything so ``revolutionary'' or ``experimental'', that I would rather delay to 4.3.0, 4.5.0, 5.0.0 or whatever future AOO version. However, the author of each commit has IMHO the right to the first opinion about the opportunity of cherry-picking into AOO42X, AOO41X etc. _My_ latest commits to trunk were all targeted to AOO42X. If I did not cherry-pick them is just because... I forgot to. And Matthias, sometimes, kindly reminded me about them. Only trunk should be allowed to be blatantly unstable. The reason for AOO42X also being unstable is because we inherited it this way. This is what I understood, at least, and I believe we have to fix this. I hope I could explain myself clearly. Best regards, smime.p7s Description: Kryptografische S/MIME-Signatur
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Arrigo, Am 01.05.24 um 20:31 schrieb Arrigo Marchiori: Hello All, TL;DR: I think that keeping distinct branches for trunk and AOO42X is good practice, and we shall work on stabilizing AOO42X. More below. To be clear, I am not thinking of build or stability problems when I say that we cannot release AOO 4.2.0. Even if all release blockers in AOO42X are fixed, we do not have a working release process! Translations are outdated and we have no way to get them into our code or new strings into our translation process. This should be a primary problem to be solved. Regards, Matthias On Wed, May 01, 2024 at 03:52:19PM +0200, Matthias Seidel wrote: Hi Pedro, Am 01.05.24 um 15:39 schrieb Pedro Lino: Hi Matthias On 05/01/2024 1:30 PM WEST Matthias Seidel wrote: Maybe it is easier to delete the current 42x branch and start a new 42X branch? If any patches were added to 42X only (and they are still relevant), then these should be cherry picked to trunk before splitting... +1. The AOO42X branch is quite old. Development only really happens on trunk. The best maintained branch is trunk, and any branches should be as close to trunk as possible. Trunk is an unstable branch. That is the nature of Trunk ;) Yes! And should not be the nature of AOO42X, IMHO. Everything we want to have in AOO42X (which was branched from trunk some years ago) should have been cherry-picked early. Do you mean that all patches/fixes in 42X are in Trunk also? Trunk has all patches, AOO42X only those that were cherry-picked from trunk. I confirm this. I personally made some patch ``backwards'' (first in another branch, and then cherry-picked into trunk) but it was my mistake: our best practice shall be as Matthias described above. Therefore there would not be a problem in deleting the current branch 42X and start a new one? This would make sense because Damjan mentioned that many relevant fixes in trunk are not included in 42X If he didn't cherry-pick them (for a reason?) they are not included. ...and if no one else did ;-) I personally cherry-picked the SSL- and curl-related commits because of some tests I am doing on AOO42X. I think that it makes sense to cherry-pick also his latest change of 16-bit into 32-bit indices. Also why is the next release 4.2.x? Haven't we made enough changes for 5.0.x? If not, what should be in 5.0.x? It is quite hypothetical to discuss about versions we are NOT capable to release. It is. But Damjan's question is still valid? What would be the change needed to consider this 5.0? I would say that having a Winx64 would be such a condition. As it is, I still see this as 42X For AOO 5.0 I would expect at least that we support ODF 1.3. A Win64 build would be great for AOO 4.5.0. I personally would see both these improvements good for AOO 4.2.0, but I think it's only a matter of... marketing and promotion. This topic is worth another thread. But again, we are not able to do a release, we should work on that problem first. If we are talking about ``technical'' issues, then I agree that AOO42X is unstable and for this reason it cannot be released. The next version we CAN release is 4.1.16 (AOO41X). That is true! If we become able to do a release for a newer branch (AOO42X), we should simply release what we have... Exactly, but including all valid updates that have been added to trunk but not cherry picked to 42X +1 If they were not cherry-picked I would expect there was a reason for it? I think that we have to discuss the meaning of the ``valid'' word as used above by Pedro. Almost all commits I saw to trunk would be IMHO applicable to AOO42X. I do not think there is anything so ``revolutionary'' or ``experimental'', that I would rather delay to 4.3.0, 4.5.0, 5.0.0 or whatever future AOO version. However, the author of each commit has IMHO the right to the first opinion about the opportunity of cherry-picking into AOO42X, AOO41X etc. _My_ latest commits to trunk were all targeted to AOO42X. If I did not cherry-pick them is just because... I forgot to. And Matthias, sometimes, kindly reminded me about them. Only trunk should be allowed to be blatantly unstable. The reason for AOO42X also being unstable is because we inherited it this way. This is what I understood, at least, and I believe we have to fix this. I hope I could explain myself clearly. Best regards, smime.p7s Description: Kryptografische S/MIME-Signatur
Re: Dynamic OpenSSL causes Curl regression on *nix
Hello All, TL;DR: I think that keeping distinct branches for trunk and AOO42X is good practice, and we shall work on stabilizing AOO42X. More below. On Wed, May 01, 2024 at 03:52:19PM +0200, Matthias Seidel wrote: > Hi Pedro, > > Am 01.05.24 um 15:39 schrieb Pedro Lino: > > Hi Matthias > > > > > On 05/01/2024 1:30 PM WEST Matthias Seidel > > > wrote: > > > > > Maybe it is easier to delete the current 42x branch and start a new > > > > > 42X > > > > > branch? If any patches were added to 42X only (and they are still > > > > > relevant), then these should be cherry picked to trunk before > > > > > splitting... > > > > > > > > > > > > > > +1. > > > > > > > > The AOO42X branch is quite old. Development only really happens on > > > > trunk. > > > > The best maintained branch is trunk, and any branches should be as > > > > close to > > > > trunk as possible. > > > > > > Trunk is an unstable branch. > > > > That is the nature of Trunk ;) Yes! And should not be the nature of AOO42X, IMHO. > > > Everything we want to have in AOO42X (which was branched from trunk some > > > years ago) should have been cherry-picked early. > > > > Do you mean that all patches/fixes in 42X are in Trunk also? > > Trunk has all patches, AOO42X only those that were cherry-picked from trunk. I confirm this. I personally made some patch ``backwards'' (first in another branch, and then cherry-picked into trunk) but it was my mistake: our best practice shall be as Matthias described above. > > Therefore there would not be a problem in deleting the current branch 42X > > and start a new one? > > This would make sense because Damjan mentioned that many relevant fixes in > > trunk are not included in 42X > > If he didn't cherry-pick them (for a reason?) they are not included. ...and if no one else did ;-) I personally cherry-picked the SSL- and curl-related commits because of some tests I am doing on AOO42X. I think that it makes sense to cherry-pick also his latest change of 16-bit into 32-bit indices. > > > > Also why is the next release 4.2.x? Haven't we made enough changes for > > > > 5.0.x? If not, what should be in 5.0.x? > > > > > > It is quite hypothetical to discuss about versions we are NOT capable to > > > release. > > > > It is. But Damjan's question is still valid? What would be the change > > needed to consider this 5.0? I would say that having a Winx64 would be such > > a condition. As it is, I still see this as 42X > > For AOO 5.0 I would expect at least that we support ODF 1.3. > > A Win64 build would be great for AOO 4.5.0. I personally would see both these improvements good for AOO 4.2.0, but I think it's only a matter of... marketing and promotion. This topic is worth another thread. > But again, we are not able to do a release, we should work on that problem > first. If we are talking about ``technical'' issues, then I agree that AOO42X is unstable and for this reason it cannot be released. > > > The next version we CAN release is 4.1.16 (AOO41X). > > > > That is true! > > > > > If we become able to do a release for a newer branch (AOO42X), we should > > > simply release what we have... > > > > Exactly, but including all valid updates that have been added to trunk but > > not cherry picked to 42X +1 > If they were not cherry-picked I would expect there was a reason for it? I think that we have to discuss the meaning of the ``valid'' word as used above by Pedro. Almost all commits I saw to trunk would be IMHO applicable to AOO42X. I do not think there is anything so ``revolutionary'' or ``experimental'', that I would rather delay to 4.3.0, 4.5.0, 5.0.0 or whatever future AOO version. However, the author of each commit has IMHO the right to the first opinion about the opportunity of cherry-picking into AOO42X, AOO41X etc. _My_ latest commits to trunk were all targeted to AOO42X. If I did not cherry-pick them is just because... I forgot to. And Matthias, sometimes, kindly reminded me about them. Only trunk should be allowed to be blatantly unstable. The reason for AOO42X also being unstable is because we inherited it this way. This is what I understood, at least, and I believe we have to fix this. I hope I could explain myself clearly. Best regards, -- Arrigo - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Pedro, Am 01.05.24 um 15:39 schrieb Pedro Lino: Hi Matthias On 05/01/2024 1:30 PM WEST Matthias Seidel wrote: Maybe it is easier to delete the current 42x branch and start a new 42X branch? If any patches were added to 42X only (and they are still relevant), then these should be cherry picked to trunk before splitting... +1. The AOO42X branch is quite old. Development only really happens on trunk. The best maintained branch is trunk, and any branches should be as close to trunk as possible. Trunk is an unstable branch. That is the nature of Trunk ;) Everything we want to have in AOO42X (which was branched from trunk some years ago) should have been cherry-picked early. Do you mean that all patches/fixes in 42X are in Trunk also? Trunk has all patches, AOO42X only those that were cherry-picked from trunk. Therefore there would not be a problem in deleting the current branch 42X and start a new one? This would make sense because Damjan mentioned that many relevant fixes in trunk are not included in 42X If he didn't cherry-pick them (for a reason?) they are not included. Also why is the next release 4.2.x? Haven't we made enough changes for 5.0.x? If not, what should be in 5.0.x? It is quite hypothetical to discuss about versions we are NOT capable to release. It is. But Damjan's question is still valid? What would be the change needed to consider this 5.0? I would say that having a Winx64 would be such a condition. As it is, I still see this as 42X For AOO 5.0 I would expect at least that we support ODF 1.3. A Win64 build would be great for AOO 4.5.0. But again, we are not able to do a release, we should work on that problem first. The next version we CAN release is 4.1.16 (AOO41X). That is true! If we become able to do a release for a newer branch (AOO42X), we should simply release what we have... Exactly, but including all valid updates that have been added to trunk but not cherry picked to 42X If they were not cherry-picked I would expect there was a reason for it? Regards, Matthias Best, Pedro - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org smime.p7s Description: Kryptografische S/MIME-Signatur
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Matthias > On 05/01/2024 1:30 PM WEST Matthias Seidel wrote: > >> Maybe it is easier to delete the current 42x branch and start a new 42X > >> branch? If any patches were added to 42X only (and they are still > >> relevant), then these should be cherry picked to trunk before splitting... > >> > >> > > +1. > > > > The AOO42X branch is quite old. Development only really happens on trunk. > > The best maintained branch is trunk, and any branches should be as close to > > trunk as possible. > Trunk is an unstable branch. That is the nature of Trunk ;) > Everything we want to have in AOO42X (which was branched from trunk some > years ago) should have been cherry-picked early. Do you mean that all patches/fixes in 42X are in Trunk also? Therefore there would not be a problem in deleting the current branch 42X and start a new one? This would make sense because Damjan mentioned that many relevant fixes in trunk are not included in 42X > > Also why is the next release 4.2.x? Haven't we made enough changes for > > 5.0.x? If not, what should be in 5.0.x? > It is quite hypothetical to discuss about versions we are NOT capable to > release. It is. But Damjan's question is still valid? What would be the change needed to consider this 5.0? I would say that having a Winx64 would be such a condition. As it is, I still see this as 42X > The next version we CAN release is 4.1.16 (AOO41X). That is true! > If we become able to do a release for a newer branch (AOO42X), we should > simply release what we have... Exactly, but including all valid updates that have been added to trunk but not cherry picked to 42X Best, Pedro - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Damjan, Pedro, All, Am 01.05.24 um 03:42 schrieb Damjan Jovanovic: On Sun, Apr 28, 2024 at 10:55 AM Pedro Lino wrote: Hi Arrigo, all On 04/27/2024 5:36 PM WEST Arrigo Marchiori wrote: The build can be downloaded from here: https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-27-x86_64-installed.tar.bz2 Tested in Ubuntu 20.04 x64 and WebDAV works perfectly! Cherry-picking was not a trivial task, because AOO42X was quite behind. It still is, WRT Damjan's last work on 32 bit indices. One step at a time... :-) Maybe it is easier to delete the current 42x branch and start a new 42X branch? If any patches were added to 42X only (and they are still relevant), then these should be cherry picked to trunk before splitting... +1. The AOO42X branch is quite old. Development only really happens on trunk. The best maintained branch is trunk, and any branches should be as close to trunk as possible. Trunk is an unstable branch. Everything we want to have in AOO42X (which was branched from trunk some years ago) should have been cherry-picked early. Also why is the next release 4.2.x? Haven't we made enough changes for 5.0.x? If not, what should be in 5.0.x? It is quite hypothetical to discuss about versions we are NOT capable to release. The next version we CAN release is 4.1.16 (AOO41X). If we become able to do a release for a newer branch (AOO42X), we should simply release what we have... Regards, Matthias smime.p7s Description: Kryptografische S/MIME-Signatur
Re: Dynamic OpenSSL causes Curl regression on *nix
On Sun, Apr 28, 2024 at 10:55 AM Pedro Lino wrote: > Hi Arrigo, all > > > On 04/27/2024 5:36 PM WEST Arrigo Marchiori wrote: > > > The build can be downloaded from here: > > > https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-27-x86_64-installed.tar.bz2 > > Tested in Ubuntu 20.04 x64 and WebDAV works perfectly! > > > Cherry-picking was not a trivial task, because AOO42X was quite > > behind. It still is, WRT Damjan's last work on 32 bit indices. One > > step at a time... :-) > > Maybe it is easier to delete the current 42x branch and start a new 42X > branch? If any patches were added to 42X only (and they are still > relevant), then these should be cherry picked to trunk before splitting... > > +1. The AOO42X branch is quite old. Development only really happens on trunk. The best maintained branch is trunk, and any branches should be as close to trunk as possible. Also why is the next release 4.2.x? Haven't we made enough changes for 5.0.x? If not, what should be in 5.0.x?
Re: Dynamic OpenSSL causes Curl regression on *nix
Dear All, On Sat, Apr 27, 2024 at 06:36:29PM +0200, Arrigo Marchiori wrote: > Dear All, > > On Tue, Apr 23, 2024 at 04:48:13PM +, Damjan Jovanovic wrote: > > [...] > > I've now verified it works on Windows too, and have pushed the commits to > > trunk. > > > > Here they are, in case you want to cherry-pick: > > > > commit f7b97bf7d9139c8b602d3da3aadbeef0631e39c1 (HEAD -> trunk, > > origin/trunk, origin/HEAD) > [...] > > commit e469ab6aed23a1b38f105a944997af16e61071d0 > > I cherry-picked the commits into AOO42X. I built that branch with our > Docker image and it seems to work fine, too! > > The build can be downloaded from here: > https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-27-x86_64-installed.tar.bz2 Windows build available here: https://home.apache.org/~ardovm/openoffice/windows/Apache_OpenOffice-2024-04-28_4.2.0_Win_x86_install_en-US.exe Best regards, -- Arrigo - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Arrigo, all > On 04/27/2024 5:36 PM WEST Arrigo Marchiori wrote: > The build can be downloaded from here: > https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-27-x86_64-installed.tar.bz2 Tested in Ubuntu 20.04 x64 and WebDAV works perfectly! > Cherry-picking was not a trivial task, because AOO42X was quite > behind. It still is, WRT Damjan's last work on 32 bit indices. One > step at a time... :-) Maybe it is easier to delete the current 42x branch and start a new 42X branch? If any patches were added to 42X only (and they are still relevant), then these should be cherry picked to trunk before splitting... (Again I'm just suggesting a logic path, I have no idea how this is done :) ) Thank you again to all that contribute with new code/patches! All the best, Pedro - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
On Fri, Apr 26, 2024 at 9:06 PM Pedro Lino wrote: > Hi Arrigo, Damjan, all > > > On 04/26/2024 8:16 PM WEST Arrigo Marchiori wrote: > > > The build can be downloaded from here: > > > https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-26-x86_64-installed.tar.bz2 > > Just tested the build on Ubuntu 20.04 x64 and it works perfectly. > Opens WebDAV folders and files smoothly. > > Thanks! > > Thank you for testing, that's great news! Damjan
Re: Dynamic OpenSSL causes Curl regression on *nix
Dear All, On Tue, Apr 23, 2024 at 04:48:13PM +, Damjan Jovanovic wrote: [...] > I've now verified it works on Windows too, and have pushed the commits to > trunk. > > Here they are, in case you want to cherry-pick: > > commit f7b97bf7d9139c8b602d3da3aadbeef0631e39c1 (HEAD -> trunk, > origin/trunk, origin/HEAD) [...] > commit e469ab6aed23a1b38f105a944997af16e61071d0 I cherry-picked the commits into AOO42X. I built that branch with our Docker image and it seems to work fine, too! The build can be downloaded from here: https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-27-x86_64-installed.tar.bz2 Cherry-picking was not a trivial task, because AOO42X was quite behind. It still is, WRT Damjan's last work on 32 bit indices. One step at a time... :-) Best regards, -- Arrigo - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Arrigo, Damjan, all > On 04/26/2024 8:16 PM WEST Arrigo Marchiori wrote: > The build can be downloaded from here: > https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-26-x86_64-installed.tar.bz2 Just tested the build on Ubuntu 20.04 x64 and it works perfectly. Opens WebDAV folders and files smoothly. Thanks! Best, Pedro - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
Dear All, On Tue, Apr 23, 2024 at 04:48:13PM +, Damjan Jovanovic wrote: [...] > I've now verified it works on Windows too, and have pushed the commits to > trunk. > > Here they are, in case you want to cherry-pick: > > commit f7b97bf7d9139c8b602d3da3aadbeef0631e39c1 (HEAD -> trunk, > origin/trunk, origin/HEAD) [...] > commit e469ab6aed23a1b38f105a944997af16e61071d0 I built trunk with our Docker image and it seems to work fine! Good work, Damjan! The build can be downloaded from here: https://home.apache.org/~ardovm/openoffice/linux/openoffice4-2024-04-26-x86_64-installed.tar.bz2 Best regards, -- Arrigo - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
On Tue, Apr 23, 2024 at 11:27 AM Pedro Lino wrote: > Hi Damjan > > > On 04/22/2024 6:21 PM WEST Damjan Jovanovic wrote: > > > Now what would you guys prefer: > > - Should I do more testing, on Windows and Linux, and push my changes in > a > > few days? > > - Should I push my changes now, and let you guys test too, and fix any > > problems as we discover them? > > I prefer B. More people testing in different scenarios is probably better. > > Best, > Pedro > > I've now verified it works on Windows too, and have pushed the commits to trunk. Here they are, in case you want to cherry-pick: commit f7b97bf7d9139c8b602d3da3aadbeef0631e39c1 (HEAD -> trunk, origin/trunk, origin/HEAD) Author: Damjan Jovanovic Date: Sun Apr 21 17:07:24 2024 +0200 Override OpenSSL's certificate verification with our own, instead of using its verification and selectively overriding the result. - A nonsense self-signed expired certificate is fed into Curl to get it to initialize even when the certificates in its expected system path are missing or elsewhere. - In Curl's CURLOPT_SSL_CTX_FUNCTION, our Curl_SSLContextCallback, we then completely override OpenSSL's verification process with ours, using SSL_CTX_set_cert_verify_callback() (instead of the previous SSL_CTX_set_verify() which just allows us to override OpenSSL's verification result). - The verification is largely the same as before, we just have to call slightly different functions to retrieve the certificate to verify and the untrusted chain. - Create components using the component context, not the legacy multi service factory. - Various other cleanups, better logging, etc. were made in the process. Patch by: me commit e469ab6aed23a1b38f105a944997af16e61071d0 Author: Damjan Jovanovic Date: Mon Apr 22 19:23:06 2024 +0200 Upgrade Curl to version 8.7.1. Patch by: me Regards Damjan
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Damjan > On 04/22/2024 6:21 PM WEST Damjan Jovanovic wrote: > Now what would you guys prefer: > - Should I do more testing, on Windows and Linux, and push my changes in a > few days? > - Should I push my changes now, and let you guys test too, and fix any > problems as we discover them? I prefer B. More people testing in different scenarios is probably better. Best, Pedro - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Damjan, Am 22.04.24 um 19:21 schrieb Damjan Jovanovic: On Tue, Apr 9, 2024 at 9:21 PM Arrigo Marchiori wrote: I think this thread excerpt explains it: https://lists.apache.org/thread/3rvvgxnws9867krk75rw6bvhmds1t2co Is that horrible bug fixed now, or does Curl/OpenSSL still crash AOO in certain setups? Now when I check for update I see a dialog box with the following error message: Status Checking for an update failed. Description Error reading data from the Internet. Server error message: Problem with the SSL CA cert (path? access rights?). I think that on trunk we had solved the crash, but automatic updates were just failing silently. I need to build ``untouched'' trunk and try again the above. In curl lib/strerror.c, function curl_easy_strerror(), that message comes from: ---snip--- case CURLE_SSL_CACERT_BADFILE: return "Problem with the SSL CA cert (path? access rights?)"; ---snip--- Searching for where CURLE_SSL_CACERT_BADFILE appears then takes us to lib/vtls/openssl.c (since we use OpenSSL). In that file, it's only set in one region of the lengthy ossl_connect_step1() function, when ssl_cafile or ssl_capath are set, verifypeer is set, and calling SSL_CTX_load_verify_file() or SSL_CTX_load_verify_dir() on them fails. The ssl_cafile and ssl_capath come from options to ./configure, but those are impossible to change, read acinclude.m4 around CURL_CHECK_CA_BUNDLE, where if neither --with-ca-bundle nor --with-ca-path are given (or both are given but disabled), it auto-detects the local location of the bundle. Even a wrong path is overridden by the auto-detected one. And this auto-detected one won't work on other Linux distributions. So one way to fix the error would be to patch Curl's configure and/or code. But there is another way, and that is to always call: curl_easy_setopt( m_pCurl, CURLOPT_CAINFO, "/path/to/certificates" ); at run-time, so that the compile-time path is overridden by the one we specify there. Curl version 7.77.0 also added the even better option CURLOPT_CAINFO_BLOB, which can use an in-memory PEM list, instead of a file. But here we face a problem. Our ::com::sun::star::xml::crypto::XSecurityEnvironment is intended to own the certificate verification process. We just ask it whether a certificate chain is valid, using the verifyCertificate() method. But no method exists, to tell us what certificates are trusted. Thus we cannot populate our trusted certificates from Mozilla/Windows, into Curl/OpenSSL. There are 2 ways to get it working: 1. Patch XSecurityEnvironment to add further methods to retrieve the trusted CA root certificates from Mozilla/Windows, and use these to build the initial certificate list to pass to Curl/OpenSSL. Then use OpenSSL to do verification, but on certificates that fail OpenSSL's verification, use our own verification process instead, like we do now. 2. Specify an initial nonsense certificate to Curl and OpenSSL, just to get them to initialize, then completely override the certificate verification with our own using the XSecurityEnvironment, before that nonsense certificate can be used. Option 1 seems counter-productive: why do certificate verification twice, first in OpenSSL, and then again in XSecurityEnvironment (via NSS or Windows's mscrypt)? Also it's quite a pain to develop, UNO interfaces are immutable so we'd need new interfaces, new services, and many code changes. Instead, I have successfully developed option 2: - Curl has been upgraded to the latest version, 8.7.1. - A nonsense self-signed expired certificate is fed into Curl to get it to initialize. - In Curl's CURLOPT_SSL_CTX_FUNCTION, our Curl_SSLContextCallback, we then completely override OpenSSL's verification process with ours, using SSL_CTX_set_cert_verify_callback() (instead of the previous SSL_CTX_set_verify() which just allows us to override OpenSSL's verification result). - The verification is largely the same as before, we just have to call slightly different functions to retrieve the certificate to verify and the untrusted chain. - Various other cleanups, better logging, etc. were made in the process. And it's working! We should now be able to use HTTPS and WebDAV on any Linux distribution, including those whose system trusted CA certificates are in a different location or missing entirely, as long as they have the Mozilla certificates. Now what would you guys prefer: - Should I do more testing, on Windows and Linux, and push my changes in a few days? - Should I push my changes now, and let you guys test too, and fix any problems as we discover them? Can you create a Pull Request? Then I could try to build it on Windows. Regards, Matthias Regards Damjan smime.p7s Description: Kryptografische S/MIME-Signatur
Re: Dynamic OpenSSL causes Curl regression on *nix
On Tue, Apr 9, 2024 at 9:21 PM Arrigo Marchiori wrote: > > > > > I think this thread excerpt explains it: > > > https://lists.apache.org/thread/3rvvgxnws9867krk75rw6bvhmds1t2co > > > > > > > > Is that horrible bug fixed now, or does Curl/OpenSSL still crash AOO in > > certain setups? > > Now when I check for update I see a dialog box with the following > error message: > > Status > Checking for an update failed. > Description > Error reading data from the Internet. > Server error message: Problem with the SSL CA cert (path? access > rights?). > > I think that on trunk we had solved the crash, but automatic updates > were just failing silently. > > I need to build ``untouched'' trunk and try again the above. > > In curl lib/strerror.c, function curl_easy_strerror(), that message comes from: ---snip--- case CURLE_SSL_CACERT_BADFILE: return "Problem with the SSL CA cert (path? access rights?)"; ---snip--- Searching for where CURLE_SSL_CACERT_BADFILE appears then takes us to lib/vtls/openssl.c (since we use OpenSSL). In that file, it's only set in one region of the lengthy ossl_connect_step1() function, when ssl_cafile or ssl_capath are set, verifypeer is set, and calling SSL_CTX_load_verify_file() or SSL_CTX_load_verify_dir() on them fails. The ssl_cafile and ssl_capath come from options to ./configure, but those are impossible to change, read acinclude.m4 around CURL_CHECK_CA_BUNDLE, where if neither --with-ca-bundle nor --with-ca-path are given (or both are given but disabled), it auto-detects the local location of the bundle. Even a wrong path is overridden by the auto-detected one. And this auto-detected one won't work on other Linux distributions. So one way to fix the error would be to patch Curl's configure and/or code. But there is another way, and that is to always call: curl_easy_setopt( m_pCurl, CURLOPT_CAINFO, "/path/to/certificates" ); at run-time, so that the compile-time path is overridden by the one we specify there. Curl version 7.77.0 also added the even better option CURLOPT_CAINFO_BLOB, which can use an in-memory PEM list, instead of a file. But here we face a problem. Our ::com::sun::star::xml::crypto::XSecurityEnvironment is intended to own the certificate verification process. We just ask it whether a certificate chain is valid, using the verifyCertificate() method. But no method exists, to tell us what certificates are trusted. Thus we cannot populate our trusted certificates from Mozilla/Windows, into Curl/OpenSSL. There are 2 ways to get it working: 1. Patch XSecurityEnvironment to add further methods to retrieve the trusted CA root certificates from Mozilla/Windows, and use these to build the initial certificate list to pass to Curl/OpenSSL. Then use OpenSSL to do verification, but on certificates that fail OpenSSL's verification, use our own verification process instead, like we do now. 2. Specify an initial nonsense certificate to Curl and OpenSSL, just to get them to initialize, then completely override the certificate verification with our own using the XSecurityEnvironment, before that nonsense certificate can be used. Option 1 seems counter-productive: why do certificate verification twice, first in OpenSSL, and then again in XSecurityEnvironment (via NSS or Windows's mscrypt)? Also it's quite a pain to develop, UNO interfaces are immutable so we'd need new interfaces, new services, and many code changes. Instead, I have successfully developed option 2: - Curl has been upgraded to the latest version, 8.7.1. - A nonsense self-signed expired certificate is fed into Curl to get it to initialize. - In Curl's CURLOPT_SSL_CTX_FUNCTION, our Curl_SSLContextCallback, we then completely override OpenSSL's verification process with ours, using SSL_CTX_set_cert_verify_callback() (instead of the previous SSL_CTX_set_verify() which just allows us to override OpenSSL's verification result). - The verification is largely the same as before, we just have to call slightly different functions to retrieve the certificate to verify and the untrusted chain. - Various other cleanups, better logging, etc. were made in the process. And it's working! We should now be able to use HTTPS and WebDAV on any Linux distribution, including those whose system trusted CA certificates are in a different location or missing entirely, as long as they have the Mozilla certificates. Now what would you guys prefer: - Should I do more testing, on Windows and Linux, and push my changes in a few days? - Should I push my changes now, and let you guys test too, and fix any problems as we discover them? Regards Damjan
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Damjan, Thanks, it builds fine now! https://www.dropbox.com/scl/fi/mee9swkwgpa1gsw0axhl1/Apache_OpenOffice_4.5.0_Win_x86_install_en-US_openssl.exe?rlkey=3f9kyqdh4fmztllsda9pqbhmx=0 Regards, Matthias Am 18.04.24 um 03:39 schrieb Damjan Jovanovic: On Wed, Apr 17, 2024 at 7:13 PM Matthias Seidel wrote: Hi Damjan, I just tried to build trunk on Windows and it stops in "curl": ... LINK : fatal error LNK1181: cannot open input file 'libeay32.lib' NMAKE : fatal error U1077: 'C:\PROGRA~2\MICROS~1.0\VC\bin\link.exe' : return code '0x49d' Stop. NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\nmake.exe"' : return code '0x2' Stop. dmake: Error code 2, while making './wntmsci12.pro/misc/build/so_built_so_curl' 1 module(s): curl need(s) to be rebuilt Regards, Matthias Hi Matthias Please try again now, this might fix it: commit 9b51720274ee0b7c1ade0e9b4cd4b8417efd1b6c (HEAD -> trunk, origin/trunk, origin/HEAD) Author: Damjan Jovanovic Date: Thu Apr 18 03:38:14 2024 +0200 Fix a regression in 8eb9a7e66a3128669216ddb884f844d50ac59fb9, which broke delivering libcrypto.lib and libssl.lib on Windows. Regards Damjan smime.p7s Description: Kryptografische S/MIME-Signatur
Re: Dynamic OpenSSL causes Curl regression on *nix
On Wed, Apr 17, 2024 at 7:13 PM Matthias Seidel wrote: > Hi Damjan, > > I just tried to build trunk on Windows and it stops in "curl": > > ... > > LINK : fatal error LNK1181: cannot open input file 'libeay32.lib' > NMAKE : fatal error U1077: 'C:\PROGRA~2\MICROS~1.0\VC\bin\link.exe' : > return code '0x49d' > Stop. > NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual > Studio 9.0\VC\bin\nmake.exe"' : return code '0x2' > Stop. > dmake: Error code 2, while making > './wntmsci12.pro/misc/build/so_built_so_curl' > > 1 module(s): > curl > need(s) to be rebuilt > > Regards, > > Matthias > > Hi Matthias Please try again now, this might fix it: commit 9b51720274ee0b7c1ade0e9b4cd4b8417efd1b6c (HEAD -> trunk, origin/trunk, origin/HEAD) Author: Damjan Jovanovic Date: Thu Apr 18 03:38:14 2024 +0200 Fix a regression in 8eb9a7e66a3128669216ddb884f844d50ac59fb9, which broke delivering libcrypto.lib and libssl.lib on Windows. Regards Damjan
Re: Dynamic OpenSSL causes Curl regression on *nix
Hi Damjan, I just tried to build trunk on Windows and it stops in "curl": ... LINK : fatal error LNK1181: cannot open input file 'libeay32.lib' NMAKE : fatal error U1077: 'C:\PROGRA~2\MICROS~1.0\VC\bin\link.exe' : return code '0x49d' Stop. NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\nmake.exe"' : return code '0x2' Stop. dmake: Error code 2, while making './wntmsci12.pro/misc/build/so_built_so_curl' 1 module(s): curl need(s) to be rebuilt Regards, Matthias Am 15.04.24 um 19:38 schrieb Damjan Jovanovic: On Tue, Apr 9, 2024 at 10:14 PM Arrigo Marchiori wrote: Hello Damjan, All, replying to this other message. On Mon, Apr 08, 2024 at 02:42:01PM +, Damjan Jovanovic wrote: [...] Here's how you set RPATH in Curl: ---snip--- diff --git a/main/curl/makefile.mk b/main/curl/makefile.mk index 044bf4d8c9..ecef11820a 100644 --- a/main/curl/makefile.mk +++ b/main/curl/makefile.mk @@ -59,6 +59,7 @@ curl_LDFLAGS+:=$(ARCH_FLAGS) ssl_param=--with-ssl .ELSE ssl_param=--with-ssl=$(OUTDIR) +curl_LDFLAGS+=-Wl,-z,origin -Wl,-rpath,\\\$$\$$ORIGIN PATCH_FILES+= curl-bundled_openssl.patch .ENDIF ---snip--- which gets libcurl.so to search for the OpenSSL dynamic libraries in its own directory before the system directories: $ ldd solver/450/unxfbsdx.pro/lib/libcurl.so ... libssl.so.3 => /path/to/openoffice-git/main/solver/450/ unxfbsdx.pro/lib/libssl.so.3 (0x299f766bc000) libcrypto.so.3 => /path/to/openoffice/openoffice-git/main/solver/450/ unxfbsdx.pro/lib/libcrypto.so.3 (0x299f77747000) It works under Linux too! Great, thank you for testing. If we want to use the dynamically linked OpenSSL, then, we have to amend 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 and have main/scp2/source/ooo/file_library_ooo.scp include "libssl.so.3" and "libcrypto.so.3" instead of "libssl.so" and "libcrypto.so". I think the same should happen under FreeBSD, so you may want to commit all these edits together after testing them? Here is my change: commit 8eb9a7e66a3128669216ddb884f844d50ac59fb9 (HEAD -> trunk, origin/trunk, origin/HEAD) Author: Damjan Jovanovic Date: Sun Apr 7 10:41:42 2024 +0200 Build OpenSSL as a dynamic link library, instead of a static library. Patch its users to use an RPATH of $ORIGIN, so they use the correct copy. This reduces the size of the build by about 4615 KiB, or 3.78%. Next step will be asking p11-kit for the CA certificates, as you proposed here: https://lists.apache.org/thread/3rb1t9jf5fnp4nfxr2z9dxmzt9l61tjq otherwise Linux builds may be unable to validate certificate chains, unfortunately. No, that was a separate issue. I wanted to make a new xmlsecurity provider that could be used on Linux instead of Mozilla (which requires category B nss) and mscrypto for Windows. As currently, --disable-category-b badly breaks AOO: the macro security dialog won't open, the document signatures dialog won't open, and a few other features probably break. We'd need to provide at least the following: - Cryptographic functions. We could do that with OpenSSL. - The read-only trusted system certificate store. We could do that with p11-kit's trust policy module. - The writable user certificate store, with client certificates. There is currently no way to do that on the Linux desktop. See my detailed investigation on https://gitlab.gnome.org/GNOME/seahorse/-/issues/205 for details, which found several bugs and even design defects that stop it from working. However what I am now thinking, is that we could make a partially working provider, with just the cryptographic functions and system certificate store, and add the user certificates later. It should only break document signatures, rather than the macro dialog. As for Linux validating certificate chains, I'll cover that in another email. Best regards, -- Arrigo Regards Damjan smime.p7s Description: Kryptografische S/MIME-Signatur
Re: Dynamic OpenSSL causes Curl regression on *nix
On Tue, Apr 9, 2024 at 9:21 PM Arrigo Marchiori wrote: > > > > I think this thread excerpt explains it: > > > https://lists.apache.org/thread/3rvvgxnws9867krk75rw6bvhmds1t2co > > > > > > > > Is that horrible bug fixed now, or does Curl/OpenSSL still crash AOO in > > certain setups? > > Now when I check for update I see a dialog box with the following > error message: > > Status > Checking for an update failed. > Description > Error reading data from the Internet. > Server error message: Problem with the SSL CA cert (path? access > rights?). > > I think that on trunk we had solved the crash, but automatic updates > were just failing silently. > Thank you, I know how to reproduce this issue even on FreeBSD, which should make testing easier. The problem is that when building Curl, it detects one path to the system certificate store, but at runtime, when it can't find it there, it fails with that error. I am investigating. Regards Damjan
Re: Dynamic OpenSSL causes Curl regression on *nix
On Tue, Apr 9, 2024 at 10:14 PM Arrigo Marchiori wrote: > Hello Damjan, All, > > replying to this other message. > > On Mon, Apr 08, 2024 at 02:42:01PM +, Damjan Jovanovic wrote: > > [...] > > Here's how you set RPATH in Curl: > > > > ---snip--- > > diff --git a/main/curl/makefile.mk b/main/curl/makefile.mk > > index 044bf4d8c9..ecef11820a 100644 > > --- a/main/curl/makefile.mk > > +++ b/main/curl/makefile.mk > > @@ -59,6 +59,7 @@ curl_LDFLAGS+:=$(ARCH_FLAGS) > > ssl_param=--with-ssl > > .ELSE > > ssl_param=--with-ssl=$(OUTDIR) > > +curl_LDFLAGS+=-Wl,-z,origin -Wl,-rpath,\\\$$\$$ORIGIN > > PATCH_FILES+= curl-bundled_openssl.patch > > .ENDIF > > > > ---snip--- > > > > which gets libcurl.so to search for the OpenSSL dynamic libraries in its > > own directory before the system directories: > > > > $ ldd solver/450/unxfbsdx.pro/lib/libcurl.so > > ... > > libssl.so.3 => /path/to/openoffice-git/main/solver/450/ > > unxfbsdx.pro/lib/libssl.so.3 (0x299f766bc000) > > libcrypto.so.3 => /path/to/openoffice/openoffice-git/main/solver/450/ > > unxfbsdx.pro/lib/libcrypto.so.3 (0x299f77747000) > > It works under Linux too! > Great, thank you for testing. > If we want to use the dynamically linked OpenSSL, then, we have to > amend 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 and have > main/scp2/source/ooo/file_library_ooo.scp include "libssl.so.3" and > "libcrypto.so.3" instead of "libssl.so" and "libcrypto.so". > > I think the same should happen under FreeBSD, so you may want to > commit all these edits together after testing them? > Here is my change: commit 8eb9a7e66a3128669216ddb884f844d50ac59fb9 (HEAD -> trunk, origin/trunk, origin/HEAD) Author: Damjan Jovanovic Date: Sun Apr 7 10:41:42 2024 +0200 Build OpenSSL as a dynamic link library, instead of a static library. Patch its users to use an RPATH of $ORIGIN, so they use the correct copy. This reduces the size of the build by about 4615 KiB, or 3.78%. > > Next step will be asking p11-kit for the CA certificates, as you > proposed here: > https://lists.apache.org/thread/3rb1t9jf5fnp4nfxr2z9dxmzt9l61tjq > otherwise Linux builds may be unable to validate certificate chains, > unfortunately. > No, that was a separate issue. I wanted to make a new xmlsecurity provider that could be used on Linux instead of Mozilla (which requires category B nss) and mscrypto for Windows. As currently, --disable-category-b badly breaks AOO: the macro security dialog won't open, the document signatures dialog won't open, and a few other features probably break. We'd need to provide at least the following: - Cryptographic functions. We could do that with OpenSSL. - The read-only trusted system certificate store. We could do that with p11-kit's trust policy module. - The writable user certificate store, with client certificates. There is currently no way to do that on the Linux desktop. See my detailed investigation on https://gitlab.gnome.org/GNOME/seahorse/-/issues/205 for details, which found several bugs and even design defects that stop it from working. However what I am now thinking, is that we could make a partially working provider, with just the cryptographic functions and system certificate store, and add the user certificates later. It should only break document signatures, rather than the macro dialog. As for Linux validating certificate chains, I'll cover that in another email. > > Best regards, > -- > Arrigo > > Regards Damjan
Re: Dynamic OpenSSL causes Curl regression on *nix
Hello Damjan, All, replying to this other message. On Mon, Apr 08, 2024 at 02:42:01PM +, Damjan Jovanovic wrote: [...] > Here's how you set RPATH in Curl: > > ---snip--- > diff --git a/main/curl/makefile.mk b/main/curl/makefile.mk > index 044bf4d8c9..ecef11820a 100644 > --- a/main/curl/makefile.mk > +++ b/main/curl/makefile.mk > @@ -59,6 +59,7 @@ curl_LDFLAGS+:=$(ARCH_FLAGS) > ssl_param=--with-ssl > .ELSE > ssl_param=--with-ssl=$(OUTDIR) > +curl_LDFLAGS+=-Wl,-z,origin -Wl,-rpath,\\\$$\$$ORIGIN > PATCH_FILES+= curl-bundled_openssl.patch > .ENDIF > > ---snip--- > > which gets libcurl.so to search for the OpenSSL dynamic libraries in its > own directory before the system directories: > > $ ldd solver/450/unxfbsdx.pro/lib/libcurl.so > ... > libssl.so.3 => /path/to/openoffice-git/main/solver/450/ > unxfbsdx.pro/lib/libssl.so.3 (0x299f766bc000) > libcrypto.so.3 => /path/to/openoffice/openoffice-git/main/solver/450/ > unxfbsdx.pro/lib/libcrypto.so.3 (0x299f77747000) It works under Linux too! If we want to use the dynamically linked OpenSSL, then, we have to amend 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 and have main/scp2/source/ooo/file_library_ooo.scp include "libssl.so.3" and "libcrypto.so.3" instead of "libssl.so" and "libcrypto.so". I think the same should happen under FreeBSD, so you may want to commit all these edits together after testing them? Next step will be asking p11-kit for the CA certificates, as you proposed here: https://lists.apache.org/thread/3rb1t9jf5fnp4nfxr2z9dxmzt9l61tjq otherwise Linux builds may be unable to validate certificate chains, unfortunately. Best regards, -- Arrigo - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Dynamic OpenSSL causes Curl regression on *nix
Hello Damjan, All, On Sun, Apr 07, 2024 at 05:22:43PM +, Damjan Jovanovic wrote: > On Sun, Apr 7, 2024 at 2:26 PM Arrigo Marchiori wrote: > > > Hello Damjan, All, > > > > On Sun, Apr 07, 2024 at 01:53:08PM +, Damjan Jovanovic wrote: > > > > > Hi > > > > > > With Pedro Lino's help, I've discovered a serious regression, where Curl > > > (and libraries that use Curl) can fail to load, because they've been > > > dynamically linked to OpenSSL but cannot find the copy of OpenSSL shipped > > > with OpenOffice. > > > > [big snip] > > > > [...] > > > > > When main/oox linked to internal OpenSSL, it did dynamically on Windows, > > > and on *nix, dynamically to system OpenSSL, and statically to internal > > > OpenSSL. > > > > I might have missed the end of this sentence: "statically to internal > > OpenSSL". > > > > > > > --- > > > 2022-05-06, commit 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 > > > > > --- > > > "Fix including OpenSSL dynamic libraries on Unix (#147)" by Arrigo. > > > > > > OpenSSL, on *nix, now starts delivering static and dynamic libraries, and > > > packaging dynamic libraries at the end of the build. > > > Curl continues using OpenSSL, but now since both static and dynamic > > OpenSSL > > > libraries are available, it favours the dynamic. > > > main/oox still links to OpenSSL statically outside Windows. > > > main/ucb still links to OpenSSL statically outside Windows. > > > > [...] > > > > > > > > CURL LOADS WRONG OPENSSL > > > > > > > > > Now the combination of these last 2 commits broke Curl (and possibly > > python > > > and redland as well) as follows: > > > - Internal modules have their RPATH set to $ORIGIN (among others), and > > thus > > > always search for dependencies in the same directory as themselves at > > > runtime. > > > - Third-party modules usually do not set their RPATH. In libcurl.so, it's > > > unset. > > > - When only static libraries are present at link-time, the binary being > > > linked, is linked to them statically. So before > > > 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3, Curl was linking statically to > > > OpenSSL. > > > > Apparently not. My purpose, with that commit, was to include the > > OpenSSL libraries _linked by the AOO executable_, that were: dynamic. > > > > In other words, the SSL libraries were not linked statically when > > building on my system. They were linked dynamically instead. > > > > > What happens currently, with OpenSSL 3, if you revert most of > 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 (I think the makefile.mk change is > still correct)? Does Curl still link to OpenSSL dynamically, or does it > start linking statically? For me, on FreeBSD, it links statically. Apparently, this is also happening on Linux. After reverting 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3: $ ldd solver/450/unxlngx6.pro/lib/libcurl.so linux-vdso.so.1 (0x7ffe59dab000) libidn2.so.0 => /usr/lib64/libidn2.so.0 (0x7f123640) libzstd.so.1 => /usr/lib64/libzstd.so.1 (0x7f12362cf000) libbrotlidec.so.1 => /usr/lib64/libbrotlidec.so.1 (0x7f123600) libz.so.1 => /usr/lib64/libz.so.1 (0x7f12366f) libdl.so.2 => /lib64/libdl.so.2 (0x7f12366eb000) libpthread.so.0 => /lib64/libpthread.so.0 (0x7f12366c5000) libc.so.6 => /lib64/libc.so.6 (0x7f1235e09000) libunistring.so.2 => /usr/lib64/libunistring.so.2 (0x7f1235a0) libbrotlicommon.so.1 => /usr/lib64/libbrotlicommon.so.1 (0x7f123560) /lib64/ld-linux-x86-64.so.2 (0x7f1236c2e000) OpenSSL libraries are not listed! They must be linked statically! > > I think this thread excerpt explains it: > > https://lists.apache.org/thread/3rvvgxnws9867krk75rw6bvhmds1t2co > > > > > Is that horrible bug fixed now, or does Curl/OpenSSL still crash AOO in > certain setups? Now when I check for update I see a dialog box with the following error message: Status Checking for an update failed. Description Error reading data from the Internet. Server error message: Problem with the SSL CA cert (path? access rights?). I think that on trunk we had solved the crash, but automatic updates were just failing silently. I need to build ``untouched'' trunk and try again the above. [...] > > Honestly, I don't know in detail what RPATH is and how to change it, > > and I would like to adopt the simplest approach to the problem. > > > > It is a ":"-separated list of paths where the binary's dependencies are > searched for, before trying the system paths. [...] I thought the "soffice" shell script set the LD_LIBRARY_PATH environment variable just for this purpose... but now I see it's just used for Java. [...] > > > Maybe it's safe to link to OpenSSL dynamically now? >
Re: Dynamic OpenSSL causes Curl regression on *nix
On Sun, Apr 7, 2024 at 5:22 PM Damjan Jovanovic wrote: > > > On Sun, Apr 7, 2024 at 2:26 PM Arrigo Marchiori wrote: > > Honestly, I don't know in detail what RPATH is and how to change it, >> and I would like to adopt the simplest approach to the problem. >> > > It is a ":"-separated list of paths where the binary's dependencies are > searched for, before trying the system paths. If it contains $ORIGIN at the > beginning, then in the case of executables, it is the directory containing > the executable, and in the case of libraries, it is the directory of the > executable that loaded them. You can see how this is useful: it allows > OpenOffice to be installed anywhere in the filesystem, and binaries can > still find their internal dependencies, because their paths are relative to > wherever the executable is, instead of some hardcoded location. > > In theory, you pass these options to the compiler when linking: > -Wl,-z,origin -Wl,-rpath,$ORIGIN > where -Wl,-z,origin enables the use of $ORIGIN in the path, and > -Wl,-rpath,$ORIGIN specifies the path, in this case being the directory > containing the executable. > In practice, however, getting that "$" past dmake and 3 layers of shell > quoting and escaping, is impossibly difficult. > Here's how you set RPATH in Curl: ---snip--- diff --git a/main/curl/makefile.mk b/main/curl/makefile.mk index 044bf4d8c9..ecef11820a 100644 --- a/main/curl/makefile.mk +++ b/main/curl/makefile.mk @@ -59,6 +59,7 @@ curl_LDFLAGS+:=$(ARCH_FLAGS) ssl_param=--with-ssl .ELSE ssl_param=--with-ssl=$(OUTDIR) +curl_LDFLAGS+=-Wl,-z,origin -Wl,-rpath,\\\$$\$$ORIGIN PATCH_FILES+= curl-bundled_openssl.patch .ENDIF ---snip--- which gets libcurl.so to search for the OpenSSL dynamic libraries in its own directory before the system directories: $ ldd solver/450/unxfbsdx.pro/lib/libcurl.so ... libssl.so.3 => /path/to/openoffice-git/main/solver/450/ unxfbsdx.pro/lib/libssl.so.3 (0x299f766bc000) libcrypto.so.3 => /path/to/openoffice/openoffice-git/main/solver/450/ unxfbsdx.pro/lib/libcrypto.so.3 (0x299f77747000) Regards Damjan
Re: Dynamic OpenSSL causes Curl regression on *nix
On Sun, Apr 7, 2024 at 2:26 PM Arrigo Marchiori wrote: > Hello Damjan, All, > > On Sun, Apr 07, 2024 at 01:53:08PM +, Damjan Jovanovic wrote: > > > Hi > > > > With Pedro Lino's help, I've discovered a serious regression, where Curl > > (and libraries that use Curl) can fail to load, because they've been > > dynamically linked to OpenSSL but cannot find the copy of OpenSSL shipped > > with OpenOffice. > > [big snip] > > [...] > > > When main/oox linked to internal OpenSSL, it did dynamically on Windows, > > and on *nix, dynamically to system OpenSSL, and statically to internal > > OpenSSL. > > I might have missed the end of this sentence: "statically to internal > OpenSSL". > > > > --- > > 2022-05-06, commit 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 > > > --- > > "Fix including OpenSSL dynamic libraries on Unix (#147)" by Arrigo. > > > > OpenSSL, on *nix, now starts delivering static and dynamic libraries, and > > packaging dynamic libraries at the end of the build. > > Curl continues using OpenSSL, but now since both static and dynamic > OpenSSL > > libraries are available, it favours the dynamic. > > main/oox still links to OpenSSL statically outside Windows. > > main/ucb still links to OpenSSL statically outside Windows. > > [...] > > > > > CURL LOADS WRONG OPENSSL > > > > > > Now the combination of these last 2 commits broke Curl (and possibly > python > > and redland as well) as follows: > > - Internal modules have their RPATH set to $ORIGIN (among others), and > thus > > always search for dependencies in the same directory as themselves at > > runtime. > > - Third-party modules usually do not set their RPATH. In libcurl.so, it's > > unset. > > - When only static libraries are present at link-time, the binary being > > linked, is linked to them statically. So before > > 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3, Curl was linking statically to > > OpenSSL. > > Apparently not. My purpose, with that commit, was to include the > OpenSSL libraries _linked by the AOO executable_, that were: dynamic. > > In other words, the SSL libraries were not linked statically when > building on my system. They were linked dynamically instead. > > What happens currently, with OpenSSL 3, if you revert most of 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 (I think the makefile.mk change is still correct)? Does Curl still link to OpenSSL dynamically, or does it start linking statically? For me, on FreeBSD, it links statically. > I think this thread excerpt explains it: > https://lists.apache.org/thread/3rvvgxnws9867krk75rw6bvhmds1t2co > > Is that horrible bug fixed now, or does Curl/OpenSSL still crash AOO in certain setups? > [...] > > > == > > LINK STATICALLY OR SET RPATH? > > == > > > > There are 2 ways to fix this: > > 1. Provide only static OpenSSL libraries at build and run time, to force > > Curl to link to OpenSSL statically. > > 2. Provide OpenSSL dynamically, but set RPATH to $ORIGIN in all binaries > > that use it (including Curl), so they can find the local OpenSSL copy > that > > ships with OpenOffice. > > > > Arrigo: why did you ever make OpenSSL deliver dynamic libraries? Were you > > planning to implement option 2? > > Not at all. I just wanted to fix the problem of the (required) OpenSSL > libraries missing from the AOO distribution. > > I now agree that 1. is the best solution, and that my commit including > the .so files was sort of a wrong solution to the problem. > > Honestly, I don't know in detail what RPATH is and how to change it, > and I would like to adopt the simplest approach to the problem. > It is a ":"-separated list of paths where the binary's dependencies are searched for, before trying the system paths. If it contains $ORIGIN at the beginning, then in the case of executables, it is the directory containing the executable, and in the case of libraries, it is the directory of the executable that loaded them. You can see how this is useful: it allows OpenOffice to be installed anywhere in the filesystem, and binaries can still find their internal dependencies, because their paths are relative to wherever the executable is, instead of some hardcoded location. In theory, you pass these options to the compiler when linking: -Wl,-z,origin -Wl,-rpath,$ORIGIN where -Wl,-z,origin enables the use of $ORIGIN in the path, and -Wl,-rpath,$ORIGIN specifies the path, in this case being the directory containing the executable. In practice, however, getting that "$" past dmake and 3 layers of shell quoting and escaping, is impossibly difficult. > > > Why was OpenOffice ever using option 1, always linking statically to > > OpenSSL on *nix? We can't tell from our Git log, because it was done > before > > we
Re: Dynamic OpenSSL causes Curl regression on *nix
Hello Damjan, All, On Sun, Apr 07, 2024 at 01:53:08PM +, Damjan Jovanovic wrote: > Hi > > With Pedro Lino's help, I've discovered a serious regression, where Curl > (and libraries that use Curl) can fail to load, because they've been > dynamically linked to OpenSSL but cannot find the copy of OpenSSL shipped > with OpenOffice. [big snip] [...] > When main/oox linked to internal OpenSSL, it did dynamically on Windows, > and on *nix, dynamically to system OpenSSL, and statically to internal > OpenSSL. I might have missed the end of this sentence: "statically to internal OpenSSL". > --- > 2022-05-06, commit 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 > --- > "Fix including OpenSSL dynamic libraries on Unix (#147)" by Arrigo. > > OpenSSL, on *nix, now starts delivering static and dynamic libraries, and > packaging dynamic libraries at the end of the build. > Curl continues using OpenSSL, but now since both static and dynamic OpenSSL > libraries are available, it favours the dynamic. > main/oox still links to OpenSSL statically outside Windows. > main/ucb still links to OpenSSL statically outside Windows. [...] > > CURL LOADS WRONG OPENSSL > > > Now the combination of these last 2 commits broke Curl (and possibly python > and redland as well) as follows: > - Internal modules have their RPATH set to $ORIGIN (among others), and thus > always search for dependencies in the same directory as themselves at > runtime. > - Third-party modules usually do not set their RPATH. In libcurl.so, it's > unset. > - When only static libraries are present at link-time, the binary being > linked, is linked to them statically. So before > 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3, Curl was linking statically to > OpenSSL. Apparently not. My purpose, with that commit, was to include the OpenSSL libraries _linked by the AOO executable_, that were: dynamic. In other words, the SSL libraries were not linked statically when building on my system. They were linked dynamically instead. I think this thread excerpt explains it: https://lists.apache.org/thread/3rvvgxnws9867krk75rw6bvhmds1t2co [...] > == > LINK STATICALLY OR SET RPATH? > == > > There are 2 ways to fix this: > 1. Provide only static OpenSSL libraries at build and run time, to force > Curl to link to OpenSSL statically. > 2. Provide OpenSSL dynamically, but set RPATH to $ORIGIN in all binaries > that use it (including Curl), so they can find the local OpenSSL copy that > ships with OpenOffice. > > Arrigo: why did you ever make OpenSSL deliver dynamic libraries? Were you > planning to implement option 2? Not at all. I just wanted to fix the problem of the (required) OpenSSL libraries missing from the AOO distribution. I now agree that 1. is the best solution, and that my commit including the .so files was sort of a wrong solution to the problem. Honestly, I don't know in detail what RPATH is and how to change it, and I would like to adopt the simplest approach to the problem. > Why was OpenOffice ever using option 1, always linking statically to > OpenSSL on *nix? We can't tell from our Git log, because it was done before > we got the code, and we don't have history going that far back. But if I > had to guess, I would say that the OpenSSL API is unstable, and yet OpenSSL > is widely used, and when different versions of OpenSSL (eg. one internal, > one system) were loaded into memory by different dependencies, it led to > conflicts. That seems reasonable to me. > Maybe it's safe to link to OpenSSL dynamically now? I think that our ``official'' Linux build machines are quite outdated, and for this reason it makes sense not to rely on their system libraries. I hope this helps making my intentions at the time more clear now. Thank you for the efforts you are putting into this issue! Best regards, -- Arrigo - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Dynamic OpenSSL causes Curl regression on *nix
Hi With Pedro Lino's help, I've discovered a serious regression, where Curl (and libraries that use Curl) can fail to load, because they've been dynamically linked to OpenSSL but cannot find the copy of OpenSSL shipped with OpenOffice. == TIMELINE == 2011, commit 72ebe82f8122a4cf6f6f9eea0178f07d12863421 Initial import from Oracle. Curl was built with --without-ssl (no OpenSSL support). OpenSSL, on *nix, delivered static libraries only. On Windows, it delivered both static and dynamic. As per main/solenv/inc/libs.mk, dmake could link to OpenSSL statically (OPENSSL_ST) or dynamically (OPENSSL). When main/oox linked to internal OpenSSL, it did dynamically on Windows, and on *nix, dynamically to system OpenSSL, and statically to internal OpenSSL. - 2016, commit b63233d868a9af170b0457a7aa0c5809011cc2c1 - gbuid-reintegration branch, based on Oracle's gbuild branch, is merged to trunk. gbuild gains the ability to link to OpenSSL. OpenSSL, on *nix, still delivers only static libraries. Curl still doesn't use OpenSSL. main/oox still links to OpenSSL statically outside Windows, despite now doing it through gbuild. - 2022-04-04, commit 51ba086bf122dbb5b50fd813e5b9a81c051aa16b - "Port our WebDAV content provider from serf/apr/apr-util, to curl." OpenSSL, on *nix, still delivers only static libraries. Curl now begins using OpenSSL. main/oox still links to OpenSSL statically outside Windows. main/ucb links to OpenSSL statically outside Windows. --- 2022-05-06, commit 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3 --- "Fix including OpenSSL dynamic libraries on Unix (#147)" by Arrigo. OpenSSL, on *nix, now starts delivering static and dynamic libraries, and packaging dynamic libraries at the end of the build. Curl continues using OpenSSL, but now since both static and dynamic OpenSSL libraries are available, it favours the dynamic. main/oox still links to OpenSSL statically outside Windows. main/ucb still links to OpenSSL statically outside Windows. --- 2024-03-18, commit 4c5b548fb6ece87dd30bbf720aca0d994a749167 --- "Upgrade OpenSSL to version 3.0.13." OpenSSL, on *nix, continues delivering static and dynamic libraries, and packaging dynamic libraries at the end of the build. These libraries however have a different name on OpenSSL 3. Curl continues using OpenSSL, but now since both static and dynamic OpenSSL libraries are available, it favours the dynamic. main/oox still links to OpenSSL statically outside Windows. main/ucb still links to OpenSSL statically outside Windows. CURL LOADS WRONG OPENSSL Now the combination of these last 2 commits broke Curl (and possibly python and redland as well) as follows: - Internal modules have their RPATH set to $ORIGIN (among others), and thus always search for dependencies in the same directory as themselves at runtime. - Third-party modules usually do not set their RPATH. In libcurl.so, it's unset. - When only static libraries are present at link-time, the binary being linked, is linked to them statically. So before 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3, Curl was linking statically to OpenSSL. - When dynamic libraries for OpenSSL became present in 0ca5b4b7b8e66fbc937f89173ce45fcc179e72b3, Curl began linking to OpenSSL dynamically. - Curl thus began using the Linux distribution's OpenSSL, despite us shipping our own, because libcurl.so's RPATH isn't set, so it doesn't search its own directory for OpenSSL. - Many operating systems still have OpenSSL 1 installed, so the fact Curl gained a hidden runtime dependency on system OpenSSL wasn't immediately apparent... - When we upgraded to OpenSSL 3, Curl began linking to OpenSSL 3, and thus needing the Linux distribution to have OpenSSL 3. - Older Linux distributions, like Ubuntu 20.04, don't have OpenSSL 3 yet. So Curl refuses to load. Anything needing Curl (WebDAV, automatic updates) also refuses to load. == LINK STATICALLY OR SET RPATH? == There are 2 ways to fix this: 1. Provide only static OpenSSL
