Re: [fossil-users] Fossil is Awesome
On Thu, Oct 27, 2011 at 1:15 AM, Caleb Gray ca...@calebgray.com wrote: Awesome, I didn't find either the JSON demo or The Doc while reading through everything. Thanks for the links! They're still very much in development, so don't get too attached to any specific behaviours :). If you find pieces you especially dislike, let us know - most of the details are still flexible at this point. If you'd like to help out, just get in touch once Richard has set up your account. You are exactly on track with what I was envisioning for the JS enhancements with: http://mbostock.github.com/d3/talk/20111018/#8 i just came across d3 yesterday, actually. He's done some amazing stuff. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On 27.10.2011 02:15, Caleb Gray wrote: @ Stephan Beal: I see the appeal in creating a separate HTML application. I will take this approach, and will see how everyone feels about having installable skins in Fossil: shipping it with only the Default skin. +1: Absolutely - optional AJAX applications as installable (in fossil DB) skins with JSON datasources, I am sure that you also follow the progress of ace.ajax.org and the other JS projects, which are good candidates for employment ;-). I think that the only missing part is feature for stored (parametrized) SQL queries, which can be called with lower privileges via JSON interface. You are exactly on track with what I was envisioning for the JS enhancements with: http://mbostock.github.com/d3/talk/20111018/#8 +1: d3 is the best for the SVG part in my eyes too, it seems even more powerful than his famous predecessor - protovis.org (and can be seen also as smart XSLT like engine - thus, can have more applications than just SVG drawing for fossil UX applications - i.e. live JSON-DOM templates for non-SVG enhanced pages also) Kind Regards, Alek ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On 26/10/2011, at 5:59 PM, Nolan Darilek wrote: On 10/26/2011 04:50 PM, Stephan Beal wrote: On Wed, Oct 26, 2011 at 11:37 PM, Nolan Darilek no...@thewordnerd.info wrote: like to see: hooks, and the ability to query the internal database and output the results into a page. The JSON API provides a query command[1] which allows you to run arbitrary queries and get the results as JSON, but it requires admin access (because it allows one to do anything with the db). Right, I understand that. I was hoping for an extension of TH1 that would allow for something like selecting the most X Y (where Y == commits, events, tickets, etc.) and displaying properties of those objects, such that the default generated page automatically included the most recent news items or whatever. This would eliminate the need for client-side rendering, and would make those pages searchable via Google, etc. I know that I've mentioned this before, and there has seemed to be some interest, but not enough to take it up and I unfortunately lack the skill/time. So it isn't a complaint, more a here's something I wish I could have. I've been talking with Richard about replacing TH1 with Jim Tcl. Proof of concept seems to work just fine. The idea would be to allow access back through sqlite to the repository. Jim Tcl is much more functional than TH1. Combined with the sqlite access, should provide many possibilities... Cheers, Steve -- µWeb: Embedded Web Framework - http://uweb.workware.net.au/ WorkWare Systems Pty Ltd W: www.workware.net.au P: +61 434 921 300 E: ste...@workware.net.au F: +61 7 3391 6002 ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
Hi, Caleb! i can't say much to points 1 and 2, but... 3) The web interface could use a face lift, as well as some HTML5 functionality. I've got a lot of web development experience and would love to contribute in this area, also. All of the work on the JSON APIs is a great step toward making the entire interface XHR compatible. What are the community's feelings on jQuery? I get the gist that externals are trying to be avoided, so that's why I'm asking, The JSON API itself doesn't need any 3rd-party JS libs (it's jot JS-specific, some of my test code is written in Java, and Zach's working on Python code). That said, jQuery is (by far) my favourite of the various JS libs, and is the only one i use with any regularity. For any given HTML interface, of course, jQuery/mojo/whatever would be essentially necessary. Rather than initially re-implement the existing HTML interface, my suggestion would be that we create an external HTML app, independent of fossil. The main advantages would be: - no concerns about dependencies vis-a-vis the fossil core - it would help us figure out what the JSON API needs (and how what it currently can do might need to change) - releases would be independent of fossil There's a live prototype/demo here: http://fossil.wanderinghorse.net/repos/fossil-sgb/json/ That copy is a week or 10 days old, but not much has changed in that time. I would love to write a library that turns the current site in to a highly interactive version without touching the HTML or CSS at all. That's one of the goals of the JSON API. In case you haven't see The Doc yet: https://docs.google.com/document/d/1fXViveNhDbiXgCuE7QDXQOKeFzf2qNUkBEgiUvoqFN4/edit?hl=en_US That contains the current status, the TODOs, etc. We're of course more than happy to have collaborators on that. i'm running out of steam, going through one of my slow periods (which typically last 4-6 weeks), so collaborators could really help move it along (== get it out the door more quickly). In particular, your experience with web apps would be very welcomed :). The JSON API still has lots of room for change, so now's the time to get your opinions voiced. The only small catch is that the repo requires one to sign a copyright waiver and send it to Richard before commit access will be granted or patches accepted: http://www.fossil-scm.org/fossil/doc/trunk/www/copyright-release.html it's a painless process, though. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Tue, 25 Oct 2011 18:51:05 -0700 Caleb Gray ca...@calebgray.com wrote: [...] 3) The web interface could use a face lift, as well as some HTML5 functionality. I've got a lot of web development experience and would love to contribute in this area, also. All of the work on the JSON APIs is a great step toward making the entire interface XHR compatible. What are the community's feelings on jQuery? I get the gist that externals are trying to be avoided, so that's why I'm asking, I would love to write a library that turns the current site in to a highly interactive version without touching the HTML or CSS at all. I strongly disagree. First, please don't fix what's not broken. Web UI does what it's intended to do, and does it well. JS usage is constrained to what can't be done without it (those arrows in the timeline view), and it's even usable with JS turned off. P.S. I'm one of those crazy folks who usually has NoScript turned on except for the intranet sites, so yes, I'm biased. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On 10/26/2011 10:59 AM, Konstantin Khomoutov wrote: I strongly disagree. First, please don't fix what's not broken. Agree 100% P.S. I'm one of those crazy folks who usually has NoScript turned on except for the intranet sites, so yes, I'm biased. Yes, so am I ... ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 12:59:05PM +0400, Konstantin Khomoutov wrote: [...] I'm one of those crazy folks who usually has NoScript turned on except for the intranet sites, so yes, I'm biased. Same here. I don't think we should require c89 and html5-browsers-with-javascript at once. Regards, Lluís. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 11:02 AM, Ron Aaron r...@ronware.org wrote: On 10/26/2011 10:59 AM, Konstantin Khomoutov wrote: I strongly disagree. First, please don't fix what's not broken. Agree 100% FWIW, i think we're all agreed that retrofitting the main HTML interface is not The Right Thing to do (nor feasible - it brings with it too many compatibility constraints for this purposes of making it XHR-friendly). A colleague of mine just showed me this: http://mbostock.github.com/d3/ex/ http://mbostock.github.com/d3/talk/20111018/#0 (thought it's not obvious: use the left/right arrow keys to navigate the 2nd link) imagine what we could do for version, file/dir, and diff browsing with something like: http://mbostock.github.com/d3/talk/20111018/#8 -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On 10/26/2011 11:15 AM, Stephan Beal wrote: imagine what we could do for version, file/dir, and diff browsing with something like: http://mbostock.github.com/d3/talk/20111018/#8 I think that things of that nature would be computationally intensive, and better suited to a separate utility using e.g. your JSON interface to query data to present. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
Ditto. Please resist the temptation to make Fossil into bloatware. The best thing about the app, in my opinion, is the amount of features it has all contained in a single binary that can be deployed practically anywhere. On Wed, Oct 26, 2011 at 2:02 AM, Ron Aaron r...@ronware.org wrote: On 10/26/2011 10:59 AM, Konstantin Khomoutov wrote: I strongly disagree. First, please don't fix what's not broken. Agree 100% P.S. I'm one of those crazy folks who usually has NoScript turned on except for the intranet sites, so yes, I'm biased. Yes, so am I ... ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users -- Michael Barrow michael at barrow dot me +1 (408) 782-4249 ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Tue, 25 Oct 2011 18:51:05 -0700 Caleb Gray ca...@calebgray.com wrote: [trimmed ...] What are the community's feelings on jQuery? I get the gist that externals are trying to be avoided, so that's why I'm asking, I would love to write a library that turns the current site in to a highly interactive version without touching the HTML or CSS at all. I think a well-made js-enabled UI could be a wonderful improvement over the current static one. I'd love to see what you have in mind. (in other words, I'm not one of those noscript people..;) ) ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 3:51 AM, Caleb Gray ca...@calebgray.com wrote: 2) Add (or replace zlib with) LZMA. Some of my repositories are very large; the compression difference between the DEFLATE algorithm and the LZMA algorithm is not negligible, and can even be substantial. Recently, even the Linux kernel has officially begun using lzmalib to compress their releases: http://www.kernel.org/pub/linux/kernel/v3.0/ (even the difference between BZ2 and XZ is about 13M, or 31M smaller than GZ). A quick implementation of LZMA (by editing auto.def and blob.c's blob_compress, blob_compress2, and blob_uncompress) resulted in the following repository sizes for the latest trunk of SQLite: sqlite-zlib.fossil (4.8M) (~5.0K per file) sqlite-lzma.fossil (3.5M) (~3.6K per file) How's that for bandwidth efficient enough for dial-up? LZMA is good for release tarballs, but has unacceptable speed and memory requirements for version control systems. Note that due to self-checks, Fossil compresses and then extracts and verifies content before committing it: http://www.fossil-scm.org/index.html/doc/trunk/www/selfcheck.wiki I'd say DEFLATE is a good compromise between LZO/SNAPPY and LZMA for our use. -- Dmitry Chestnykh ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 3:51 AM, Caleb Gray ca...@calebgray.com wrote: 1) Compress the releases found on http://www.fossil-scm.org/download.html using UPX ( http://upx.sourceforge.net/). UPX is technology from the good ol' days, when people had tiny floppies and hard drives. There's no reason to use it nowadays. It slows down execution, takes more memory, may introduce vulnerabilities, etc. Just read about all the disadvantages of UPX here http://linux.die.net/man/1/upx. Note that Fossil in CGI mode is executed for each request, so with UPX it would have to decompress itself on each request. The first time you run it, it may be faster (due to smaller disk reads compared to uncompressed version), but for each subsequent request, once Fossil is in RAM, it would be slower and require more CPU. -- Dmitry Chestnykh ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 05:27:41PM +0200, Dmitry Chestnykh wrote: On Wed, Oct 26, 2011 at 3:51 AM, Caleb Gray ca...@calebgray.com wrote: 1) Compress the releases found on http://www.fossil-scm.org/download.html using UPX ( http://upx.sourceforge.net/). Note that Fossil in CGI mode is executed for each request, so with UPX it would have to decompress itself on each request. The first time you run it, it may be faster (due to smaller disk reads compared to uncompressed version), but for each subsequent request, once Fossil is in RAM, it would be slower and require more CPU. Forking and executing is not the same thing. I don't know in Windows, but in unix, a fork of a upx program should be the same as a non-upx program. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
2011/10/26 Lluís Batlle i Rossell virik...@gmail.com: Forking and executing is not the same thing. I don't know in Windows, but in unix, a fork of a upx program should be the same as a non-upx program. Fork (on *nix) applies to fossil server. Fossil's CGI is, obviously, run by webserver with fork/exec. Also, Windows doesn't have fork. -- Dmitry Chestnykh ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 05:47:22PM +0200, Dmitry Chestnykh wrote: 2011/10/26 Lluís Batlle i Rossell virik...@gmail.com: Forking and executing is not the same thing. I don't know in Windows, but in unix, a fork of a upx program should be the same as a non-upx program. Fork (on *nix) applies to fossil server. Fossil's CGI is, obviously, run by webserver with fork/exec. Also, Windows doesn't have fork. Ah right, for the CGI yes. My fault :) ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 10:57 PM, Remigiusz Modrzejewski l...@maxnet.org.pl wrote: I'm one of the noscript people, but I'm also for a nice html5 ui. But it'd better be standalone using the json api. There are _no_ plans to replace the current HTML interface. In the scope of the JSON API we'll almost certainly create a separate interface (quite possibly as a separate project), or possibly just smaller components (think widgets, like timeline/ticket views) which could be plugged in to script-capable sites. Some of my test code is in Java, and i'd eventually like to create a webstart-launchable UI which could run alongside any other HTML interface(s) for the repo. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 11:30 PM, Stephan Beal sgb...@googlemail.comwrote: API we'll almost certainly create a separate interface (quite possibly as a separate project) Just to be clear: that would not mean a fork or other fundamental split from fossil. One of the main purposes of the JSON API is to make such external UIs possible on top of the core fossil functionality. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
Just to chime in here: I like the JSON work, but I hope we'll eventually see a more dynamic means of creating internal HTML pages. After about a year of using Fossil, there are two features I'd dearly like to see: hooks, and the ability to query the internal database and output the results into a page. So, for instance, I'm currently storing one of my projects' sets of release notes as events. I'd like to pull the most recent 5, or 10, or whatever and display them on the main page such that http://spielproject.info doesn't look incredibly bare. :) Anyhow, I mention the year bit because I'm sensitive to not wanting to bloat Fossil, and to establish that I don't have ten other feature requests waiting in the wings. The new JSON support is awesome, but I also hope that we see some way to make the current HTML interface a bit more dynamic, if only a little. Thanks. On 10/26/2011 04:31 PM, Stephan Beal wrote: On Wed, Oct 26, 2011 at 11:30 PM, Stephan Beal sgb...@googlemail.com mailto:sgb...@googlemail.com wrote: API we'll almost certainly create a separate interface (quite possibly as a separate project) Just to be clear: that would not mean a fork or other fundamental split from fossil. One of the main purposes of the JSON API is to make such external UIs possible on top of the core fossil functionality. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, Oct 26, 2011 at 11:37 PM, Nolan Darilek no...@thewordnerd.infowrote:
like to see: hooks, and the ability to query the internal database and
output the results into a page.
The JSON API provides a query command[1] which allows you to run arbitrary
queries and get the results as JSON, but it requires admin access (because
it allows one to do anything with the db).
Delayed hooks could be added on top of the JSON API. e.g. poll a repo,
checking for new commits, and send an email when the timeline reveals new
info.
The topic of real hooks has come up many times, and the main reason it
hasn't been added so far is platform portability. (Sorry, i don't mean to
start another dead-horse-beating thread.)
Anyhow, I mention the year bit because I'm sensitive to not wanting to
bloat Fossil, and to establish that I don't have ten other feature requests
waiting in the wings. The new JSON support is awesome, but I also hope that
we see some way to make the current HTML interface a bit more dynamic, if
only a little.
Once the JSON API is more or less feature-complete, one logical next step
would be to touch up some of the features in the static interface, e.g.
drilling down in the timeline or file/dir list could potentially be done
more easily (==more user friendly) in an XHR-based interface. But
compatibility with a wide range of clients is important, and i doubt (but
cannot rule out) that the default HTML interface will undergo any such
radical changes.
[1] =
stephan@tiny ~/cvs/fossil/fossil-json $ f json query 'select uid, login from
user' -f a -indent 0
{fossil:23a35ba1cccad37844f23867319668dafe6430b4,
timestamp:1319665780, command:query, procTimeMs:12,
payload:{columns:[uid, login], rows:[[1, stephan], [2,
anonymous], [3, nobody], [4, json-demo]]}}
--
- stephan beal
http://wanderinghorse.net/home/stephan/
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Oct 26, 2011, at 11:50 PM, Stephan Beal wrote: The topic of real hooks has come up many times, and the main reason it hasn't been added so far is platform portability. (Sorry, i don't mean to start another dead-horse-beating thread.) Nope, this has already been resolved. The reason now is there is not enough programmer throughput in the project. Or however it was worded. Kind regards, Remigiusz Modrzejewski ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On 10/26/2011 04:50 PM, Stephan Beal wrote: On Wed, Oct 26, 2011 at 11:37 PM, Nolan Darilek no...@thewordnerd.info mailto:no...@thewordnerd.info wrote: like to see: hooks, and the ability to query the internal database and output the results into a page. The JSON API provides a query command[1] which allows you to run arbitrary queries and get the results as JSON, but it requires admin access (because it allows one to do anything with the db). Right, I understand that. I was hoping for an extension of TH1 that would allow for something like selecting the most X Y (where Y == commits, events, tickets, etc.) and displaying properties of those objects, such that the default generated page automatically included the most recent news items or whatever. This would eliminate the need for client-side rendering, and would make those pages searchable via Google, etc. I know that I've mentioned this before, and there has seemed to be some interest, but not enough to take it up and I unfortunately lack the skill/time. So it isn't a complaint, more a here's something I wish I could have. The topic of real hooks has come up many times, and the main reason it hasn't been added so far is platform portability. (Sorry, i don't mean to start another dead-horse-beating thread.) No worries. I've been on-list for those discussions and understand the issues involved. I just mentioned them for the sake of completeness. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
@ Stephan Beal: I see the appeal in creating a separate HTML application. I will take this approach, and will see how everyone feels about having installable skins in Fossil: shipping it with only the Default skin. Awesome, I didn't find either the JSON demo or The Doc while reading through everything. Thanks for the links! I've already mailed my copyright release last week sometime, so it should be arriving any day now. :D You are exactly on track with what I was envisioning for the JS enhancements with: http://mbostock.github.com/d3/talk/20111018/#8 @Konstantin Khomoutov, @Ron Aaron, @Lluís Batlle i Rossell: You're absolutely right, the web interface is fine. What I'm saying that it could be enhanced with complimentary JavaScript, leaving all of the previous CSS/HTML alone. Disabled JS wouldn't be a problem at all; you just wouldn't see the enhancements. @Ron Aaron: I definitely agree that features like syntax highlighting can be very expensive. If the Fossil server is running a JS enhanced skin, the end-user will be presented with a way to disable features they feel aren't worth their computational time. @Michael Barrow: I'm hoping that we can abstract all of Fossil's hard-coded pages and skins, and have installable skins to dodge bloating Fossil with any and all unnecessary data. @Dmitry Chestnykh: I just wrote a script for testing the speed and size difference between the different compressions available, find the results here: http://uploads.calebgray.com/contributions/compression/index.html As far as I know, memory usage depends entirely on dictionary size... so shouldn't DEFLATE and LZMA use the same amount of memory if configured correctly? I'm used to 500KiB/s download speed, but my only choice at home is Clearwire (which is true, I'm sure, for more than just myself). Unfortunately, it's not rare for me to get 20KiB/s download speeds on it, if the Fossil releases were UPX compressed, that would have saved me ~5 seconds over the ZIP. Obviously, this isn't doesn't seem like a big deal, but keep in mind that the people in Australia/New Zealand have to pay for their bandwidth. It's not just time we're saving people, it's money too, in the long run. UPX has zero effect on memory usage, and would probably add a millisecond or two to each request, leaving CPU as the only truly impacted factor... I suppose that if a Pentium 133 uncompressed at ~10MB/s as they claim on their homepage... then if you're getting 10 requests a second on a 1MB executable... your server would begin seeing the performance impact of Fossil being compressed using UPX. Anyway, I'm not a huge proponent to the idea, it was just a thought. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil is Awesome
On Wed, 26 Oct 2011 16:15:11 -0700 Caleb Gray ca...@calebgray.com wrote: @Dmitry Chestnykh: I just wrote a script for testing the speed and size difference between the different compressions available, find the results here: http://uploads.calebgray.com/contributions/compression/index.html Thanks! I grepped a bit, and found out that Fossil uses 9 level for deflate, which seem to me _not_ a good balance; and LZMA 1 level outperforms it in both compression ratio and speed. Plus I've heard that xz is for some reason slower than 7zip: http://ck-hack.blogspot.com/2011/04/quick-lrzip-comparison.html BTW, how LZMA performs for tiny files? I ask because most blobs in Fossil in normal operation (source code) are tiny due to delta compression. As far as I know, memory usage depends entirely on dictionary size... so shouldn't DEFLATE and LZMA use the same amount of memory if configured correctly? Not sure, but I thought a good part of why LZMA is so good is that it has a large dictionary size. Though, maybe low compression levels don't require a lot of memory. I'm used to 500KiB/s download speed, but my only choice at home is Clearwire (which is true, I'm sure, for more than just myself). Unfortunately, it's not rare for me to get 20KiB/s download speeds on it, if the Fossil releases were UPX compressed, that would have saved me ~5 seconds over the ZIP. Obviously, this isn't doesn't seem like a big deal, but keep in mind that the people in Australia/New Zealand have to pay for their bandwidth. It's not just time we're saving people, it's money too, in the long run. Sure, I understand. When I lived in Russia, the first Ethernet connection I got was pretty expensive: $0.15 per megabyte or so (with average income in Moscow ~$500 at that time). Today a lot of places in the world have expensive mobile internet. Still, 500 KB difference in a world of torrents, 500 MB text editors, and 2 GB software updates? Come on! :-) UPX has zero effect on memory usage, and would probably add a millisecond or two to each request, leaving CPU as the only truly impacted factor... I suppose that if a Pentium 133 uncompressed at ~10MB/s as they claim on their homepage... then if you're getting 10 requests a second on a 1MB executable... your server would begin seeing the performance impact of Fossil being compressed using UPX. Anyway, I'm not a huge proponent to the idea, it was just a thought. UPX works by allocating a buffer of original executable size, then uncompressing the file to it, and executing this buffer. The statically compiled fossil running on my server (amd64) is ~5 MB. This means that on each request, if I packed it with UPX, I'd get 5 MB allocation -- not good for my little VPS :-) -- Dmitry Chestnykh http://www.codingrobots.com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Fossil is Awesome
I've recently discovered Fossil, and have been migrating all of my repositories over to it. I was wondering about a few questions that I've come up with since I started using Fossil in my day to day life and wanted to see what the community's thoughts were on them. 1) Compress the releases found on http://www.fossil-scm.org/download.html using UPX ( http://upx.sourceforge.net/). While deploying Fossil to my home computers, work machines, and servers, I found the extra step of unzipping the self-contained executable unnecessary. Speaking from personal experience: UPX does its job flawlessly every time. There are entire Linux distributions compressed using UPX proving that it has no effect on executable integrity. UPX even does a better job of compressing the binary than ZIP does: linux-x86-20111021125253 (ZIP: 477K) (UPX: 383K) macosx-x86-20111021125253 (ZIP: 550K) (UPX: 473K) openbsd-x86-20111021125253 (ZIP: 537K) (UPX: 416K) w32-20111021125253.exe(ZIP: 498K) (UPX: 402K) Although, one caveat to this is that many Windows users have Anti-virus solutions installed that claim UPX compressed executables are either viruses, or possible threats. Perhaps it would be best to leave the Windows release as a ZIP. 2) Add (or replace zlib with) LZMA. Some of my repositories are very large; the compression difference between the DEFLATE algorithm and the LZMA algorithm is not negligible, and can even be substantial. Recently, even the Linux kernel has officially begun using lzmalib to compress their releases: http://www.kernel.org/pub/linux/kernel/v3.0/ (even the difference between BZ2 and XZ is about 13M, or 31M smaller than GZ). A quick implementation of LZMA (by editing auto.def and blob.c's blob_compress, blob_compress2, and blob_uncompress) resulted in the following repository sizes for the latest trunk of SQLite: sqlite-zlib.fossil (4.8M) (~5.0K per file) sqlite-lzma.fossil (3.5M) (~3.6K per file) How's that for bandwidth efficient enough for dial-up? 3) The web interface could use a face lift, as well as some HTML5 functionality. I've got a lot of web development experience and would love to contribute in this area, also. All of the work on the JSON APIs is a great step toward making the entire interface XHR compatible. What are the community's feelings on jQuery? I get the gist that externals are trying to be avoided, so that's why I'm asking, I would love to write a library that turns the current site in to a highly interactive version without touching the HTML or CSS at all. Thanks for your time! I look forward to your replies. :D Sincerely, Caleb ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
