Thank you Troy! I have now tried to set the CURLAVAILABLE define via the cmake call (-DCURLAVAILABLE=TRUE), but that did not help.
Is there anything else I can to help you debug this? Best regards, Tobias From: Troy A. Griffitts Sent: Montag, 22. Januar 2024 05:57 To: Tobias Klein Subject: Re: [sword-devel] HTTPS Transport Hi Tobias, Having a quick look at your build output, I don't use cmake so I'm not sure how this all works, but I am suspicious with this: D:\a\sword-build-win32\sword-build-win32>cmake -B build\sword -G "Visual Studio "16 2019"" -A Win32 -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE -DZLIB_LIBRARY=D:\a\sword-build-win32\sword-build-win32\dist\lib\zlib.lib -DZLIB_INCLUDE_DIR=D:\a\sword-build-win32\sword-build-win32\zlib -DBZIP2_LIBRARY=D:\a\sword-build-win32\sword-build-win32\dist\lib\libbz2.lib -DBZIP2_INCLUDE_DIR=D:\a\sword-build-win32\sword-build-win32\bzip2 -DXZ_LIBRARY=D:\a\sword-build-win32\sword-build-win32\dist\lib\liblzma.lib -DXZ_INCLUDE_DIR=D:\a\sword-build-win32\sword-build-win32\xz\src\liblzma\api -DCURL_LIBRARY=D:\a\sword-build-win32\sword-build-win32\dist\lib\libcurl_imp.lib -DCURL_INCLUDE_DIR=D:\a\sword-build-win32\sword-build-win32\curl\include -DCMAKE_DISABLE_FIND_PACKAGE_ICU=TRUE -DCMAKE_BUILD_TYPE=Release sword I don't see the -DCURLAVAILABLE define which tells SWORD to use libCURL. Again, it should default to the old behavior if it can't use libcurl to include the http transport to do the new check, but I am wondering if you have been building SWORD without CURL support (this would mean that it would fall back to libftp and should have still worked for you). @Greg Hellings, any ideas about the cmake system? Does it ever define CURLAVAILABLE when it finds libcurl? (most interestingly on Windows) Troy On 1/21/24 13:28, Tobias Klein wrote: Hi Troy, my Windows build of SWORD has already been including CURL (currently used tag: curl-7_68_0). Here is what CMake outputs (extract from build log on GitHub). 2024-01-21T14:39:13.3799672Z -- CONFIGURING SOURCE LIST 2024-01-21T14:39:13.3800942Z -- ZLib: system D:\a\sword-build-win32\sword-build-win32\dist\lib\zlib.lib 2024-01-21T14:39:13.3802219Z -- bzip2: system D:/a/sword-build-win32/sword-build-win32/dist/lib/libbz2.lib 2024-01-21T14:39:13.3803606Z -- xz: system D:/a/sword-build-win32/sword-build-win32/dist/lib/liblzma.lib 2024-01-21T14:39:13.3804732Z -- cURL: system D:\a\sword-build-win32\sword-build-win32\dist\lib\libcurl_imp.lib and D:/a/sword-build-win32/sword-build-win32/curl/include 2024-01-21T14:39:13.3805695Z -- CLucene: no 2024-01-21T14:39:13.3806800Z -- PkgConfig: yes 2024-01-21T14:39:13.3807142Z -- ICU: no 2024-01-21T14:39:13.3807647Z -- Regex.h: internal 2024-01-21T14:39:13.3814178Z -- Building Shared library. 2024-01-21T14:39:13.3818932Z -- Building Static library. 2024-01-21T14:39:13.3821940Z -- Setting SOVERSION to 1.9.0 2024-01-21T14:39:13.3972004Z -- Setting link libraries to D:\a\sword-build-win32\sword-build-win32\dist\lib\zlib.lib;D:/a/sword-build-win32/sword-build-win32/dist/lib/libbz2.lib;D:/a/sword-build-win32/sword-build-win32/dist/lib/liblzma.lib;D:\a\sword-build-win32\sword-build-win32\dist\lib\libcurl_imp.lib I am attaching the full build log from GitHub for reference, not sure if it helps. In a moment I will share screenshots from Ezra Bible App (before and after) with the available repository list – maybe that helps. Best regards, Tobias From: Troy A. Griffitts Sent: Sonntag, 21. Januar 2024 19:57 To: SWORD Developers' Collaboration Forum; Tobias Klein Subject: Re: [sword-devel] HTTPS Transport Hi Tobias, Thank you so much for testing. My guess is that the difference is likely due to the way SWORD is compiled-- which dependencies SWORD is using. My speculation is that libcurl is probably available on Linux and your Windows build probably isn't using this library. Can you let me know if SWORD is compiled with libcurl on your Windows build? I will try to test a Linux build without libcurl. I tried to make the new feature fallback to previous behavior if it couldn't use the new feature successfully. I obviously have failed in my backward compatibility effort. 😊 I will try to get it right. Troy On January 21, 2024 09:44:12 MST, Tobias Klein <cont...@tklein.info> wrote: Hi Troy, I have just done some testing with the latest changes in sword (SVN Rev. 3894). I have not observed any issues on Linux. However, on Windows 10 the following happens: Repository data does not seem to be returned in the same way as before (I was previously on SVN Rev. 3873, which is from November 7 2021). E.g. a helper function in node-sword-interface that returns the number of modules per repository suddenly returns 0 modules for the CrossWire repository. Is there any explanation for that? I did not change anything in node-sword-interface, but only updated the sword dependency to the latest version 3894. Best regards, Tobias On 12/16/23 11:11 PM, Troy A. Griffitts wrote: Hey guys, For a while now, we've been working on a new mechanism for allowing a remote module installation repository to use only HTTPS traffic to supply their modules, if they wish. A little history and how things work in the released SWORD engine and what we have in SVN trunk now... (skip to === NEW === if you don't care about the history and why) >From the beginning of SWORD we have had as a core value the simple enabling of >Bible distribution. The very first versions of our installer could use as an >installation source any working installation of a SWORD library. E.g., user 1 >sets up Xiphos and manually unzips 100 Bibles, commentaries, lexicons, >dictionaries, etc. for use with Xiphos. User 1 can then share (network drive, >USB stick, FTP) their installation folder where they have unzipped all the >data for their library, and user 2 can come along and install Bibletime or >Xiphos or any other SWORD application and point their installer to this shared >location and install from there any Bible, commentary, lexicon, dictionary, >etc., from user 1's working installation. Then user 2 can travel to their >school in Zimbabwe, plug into their school's network and share their data >folder from their working SWORD application and students on that network can >install Bibles from them. None of this has changed. This is still a core value and still works with all the same mechanisms. Over the years, we have added on top of this behavior optional optimizations for remote repositories. For example, instead of looking for the mods.d/ folder and downloading individually all the .conf files found there to present to a user a list of which Bibles, commentaries, etc. are available, we first look for a mods.d.tar.gz file with all the .conf files bundled into a single download. This saves a lot of time working with large remote repositories. If we don't find this file, we still fallback to downloading the individual files. We don't want a failure to happen when passing along Bibles if this optimization is not in place, but we do want to speed things up if the manager of the remote repository knows how to manage their repository optimally and is willing to do this extra work to keep this file in place and up to date. Over the years, the FTP protocol, which SWORD has primarily used for remote module installation over the internet, has seen data providers block traffic due to its unencrypted nature. Being Bible distributors, in most cases we don't care if anyone snoops on our data packets. Generally, again in most cases, we WANT people to snoop. We don't require user / password for distribution so the security issues involved in sending those in plain text don't apply to our applications, generally. Now, of course there are scenarios which people may wish to distribute Bibles without public knowledge, and sometimes users may wish to protect their modules with username / password credentials, and for this we have historically also supported SFTP. One driving factor for the latest improvement described below is that we have found some mobile data providers blocking FTP traffic on their network, requiring our users to get to a WiFi connection before they can install Bibles, etc. === NEW === In SVN trunk there is code to handle a new facility for remote module installation. Like the optional optimization with the mods.d.tar.gz file, this new mechanism is optional. All will work as before if nothing is changed. Fully enabling the new mechanism consists of 4 steps: 1. assuring https access to the root folder of your module repository. 2. mods.d.tar.gz is required for this mechanism to be successful. 3. module.zip files must be available from a packages/ folder at the root of your module repository folder. These .zip files have been historically required for JSword-based apps because JSword does not yet know how to install from an working installation of modules, as described at the beginning of this email. So because many of our repository maintainers support JSword, this step might be as simple as creating a packages/ -> symbolic link to your JSword .zip module files folder, if you are already maintaining zip files. 4. adding an HTTPSPackagePreference entry into our master repository list telling us the server, and path on that server, to find your repository with https The main CrossWire repository now has this mechanism enabled and can be used as a reference to test frontends and can be used as an example for remote module installation repository maintainers. For CrossWire main, #1 is available here, and at the root of this location you can also see #2 mods.d.tar.gz and #3 packages/ : https://crosswire.org/ftpmirror/pub/sword/raw/ Step 4 can be seen in our master repo list, the first entry under [Repos] here: https://crosswire.org/ftpmirror/pub/sword/masterRepoList.conf Any SWORD app compiled against SVN trunk should now only use HTTPS when installing modules from CrossWire main. May I ask to please test and give feedback? Thank you for all the advice and encouragement to add this functionality. I pray this enhances our ability to distribute more Bibles to those who have yet to hear the Good New of Jesus Christ and to be used by Him to build up His church, Troy sword-devel mailing list: sword-devel@crosswire.org http://crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page sword-devel mailing list: sword-devel@crosswire.org http://crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page -- Sent from my Android device with K-9 Mail. Please excuse my brevity. _______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
_______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
