On Fri, Dec 23, 2011 at 6:03 PM, Ivo Benner <skyt...@web.de> wrote: > Hello, > > I want to write a javascript based bible application that should be able > to download and use SWORD modules. > Its javascript because it should run as offline app on Linux, OSX, > Windows and even on all major mobile devices. > > The Sword modules are binary files on a ftp server, so i've got some > issues so solve. > Binary file reading is possible with modern browsers and on mobile > devices with the FileReader API or AJAX with ArrayBuffers. > There are even libraries for zip and zlib compressions.
Since your application will be running in one of these and will not be on a web page, those APIs should be available to you. > > Since it's a Javascript application the best format would probably be > JSON and I already wrote a converter to convert bible modules to JSON, > but the files are quite huge. JSON might be a good way to do the import, but you probably don't want to actually store and manipulate the modules this way. Reading a full-sized JSON file into a JavaScript environment (a full module like the KJV can run up into the 5-10MB range or more) will take a very long time to parse. When you're dealing with embedded worlds like cellphones and tablets the loading and parse time for a JSON file of that size can be as high as a few minutes depending on many different variables. After retrieving the file in JSON from the server you probably will want to store it in a JavaScript friendly database environment if you're not going to be working with a SWORD engine embedding. Most modern web browsers (that is to say: those other than IE) come equipped with the IndexDB which is akin to schema-less/NoSQL databases like MongoDB. Some of them also have the older but deprecated WebDB which is a SQL database (defined in the specification to be compatible with SQLite and usually implemented by an actual instance of SQLite itself). Most embedded environments also have access to SQLite (it is iPhone's preferred method for applications to store user data, and I'm fairly sure its available on Android as well). You will find that lookup, retrieval and search times in a database will be MUCH better than loading and parsing a JSON file for embedded devices usually by several orders of magnitude. The static JSON files are probably a great method of distribution, but I wouldn't rely on storing modules in that format. > > My current problem with the binary format is that i cannot access the > files on http://crosswire.org with an AJAX request since there is an > "Origin Policy" which doesn't let other websites or local javascript > files fetch data from it. This is a limitation in the browser. If you are running in a stand-alone JavaScript environment, like v8 from the command line, a Chrome browser plugin, etc., there are settings you can activate to allow you access to domains other than the origin domain. I have a browser plugin for Chrome that is able to access pages from http://www.biblegateway.com even though that is not its origin domain. The cross-domain limitation is a browser-enforced thing and is not a limitation of JavaScript. The mechanism to allow arbitrary domain access is probably different for each environment you will target, but the cross-domain origin policy is pretty much limited only to web pages you load and not to anything else. > > So here are my questions: > Am I allowed to convert and host JSON converted Sword modules or are > there any licensing issues? Each module will be different. You are welcome to look through the list of modules on the CrossWire repositories. Many of them are public domain or have open distribution rights. Those you are free to distribute as you would like. There are others which are listed as under copyright however CrossWire has been given the rights to distribute the files - those you would not be legally able to host on another site. However, if you get to that point, I would be willing to work with you to do format-shifting of those which are permissible and host them on CrossWire's site. If format shifting is allowed for the CrossWire-only modules, then the effort to host on CrossWire's site in JSON format would be beneficial. As I mentioned above, I have a start on a Chrome plugin that is able to access BibleGateway, and the effort to put SWORD modules into a JavaScript-friendly format is one I've already put some brain power into and would benefit my plugin, so I'd be happy to work with you on that. > Can you allow AJAX cross origin requests for the sword files? For an > Apache webserver this would be: > > Header set Access-Control-Allow-Origin "*" This is not a server thing. This is a client-side thing as I address above. > > within the directory rule. I'm accessing the files via HTTP: > http://www.crosswire.org/ftpmirror/pub/sword/raw/ > Cross domain/origin FTP access is not possible with AJAX as far as i know. I am unfamiliar with a JavaScript environment that supports FTP at all. However, if we format shift to JSON we can host the files under any scheme that we like. HTTP would obviously be what we would target so as to avoid the difficulty of FTP. I'm sure in some environments there are FTP libraries that can be accessed from embedded JavaScript interpreters, but for the broadest appeal HTTP would be much more desirable. --Greg > > If you can help me or if you have any questions concerning my > application please let me know. > > Ivo > > _______________________________________________ > sword-devel mailing list: sword-devel@crosswire.org > http://www.crosswire.org/mailman/listinfo/sword-devel > Instructions to unsubscribe/change your settings at above page _______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
