Re: FSFS format7 and compressed XML bundles

On 2013-03-06 18:55:55 +, Julian Foad wrote: > I don't know if anything like that would be feasible.  It may be > possible in theory but too complex in practice.  The parameters we > need to extract would include such things as the Huffman coding > tables used and also parameters that influence

Re: FSFS format7 and compressed XML bundles

Vincent Lefevre wrote: > On 2013-03-05 16:52:30 +, Julian Foad wrote: >> Vincent Lefevre wrote: > [about server-side vs client-side] [...] > Because the diff between two huge compressed files is generally huge > (unless some rsync-friendly option has been applied, when available). > So, if th

Re: FSFS format7 and compressed XML bundles

This is all very insightful and informative. For fun, I threw together a quick script which commits a series of extremely minor changes to a MS Word file and monitors how the repository size evolves. I then added the following lines to the script to commit not the original Word file but an unzip

Re: FSFS format7 and compressed XML bundles

On 2013-03-05 16:52:30 +, Julian Foad wrote: > Vincent Lefevre wrote: [about server-side vs client-side] > > But even if there would be no problems with the > > construction/reconstruction, it would be a bad solution, IMHO. > > Indeed, for a commit, it is the client that is supposed to expand >

Re: FSFS format7 and compressed XML bundles

I (Julian Foad) wrote: > Vincent Lefevre wrote: >>  On 2013-03-05 13:30:28 +, Julian Foad wrote: >>> Vincent Lefevre wrote: On 2013-03-01 14:58:10 +, Philip Martin wrote: > A server-side solution is difficult.  Suppose the client has some > uncompressed content U which it compr

Re: FSFS format7 and compressed XML bundles

Vincent Lefevre wrote: > On 2013-03-05 13:30:28 +, Julian Foad wrote: >> Vincent Lefevre wrote: >> > On 2013-03-01 14:58:10 +, Philip Martin wrote: >> >>  A server-side solution is difficult.  Suppose the client has some >> >>  uncompressed content U which it compresses to C and sends

Re: FSFS format7 and compressed XML bundles

Hi Julian, On 2013-03-05 13:30:28 +, Julian Foad wrote: > Vincent Lefevre wrote: > > > On 2013-03-01 14:58:10 +, Philip Martin wrote: > >> A server-side solution is difficult.  Suppose the client has some > >> uncompressed content U which it compresses to C and sends to the server. > >>

Re: FSFS format7 and compressed XML bundles

Vincent Lefevre wrote: > On 2013-03-01 14:58:10 +, Philip Martin wrote: >> A server-side solution is difficult.  Suppose the client has some >> uncompressed content U which it compresses to C and sends to the server. >> The server can uncompress C to get U but unless the compression scheme

Re: FSFS format7 and compressed XML bundles

On 2013-03-01 14:58:10 +, Philip Martin wrote: > A server-side solution is difficult. Suppose the client has some > uncompressed content U which it compresses to C and sends to the server. > The server can uncompress C to get U but unless the compression scheme > has a canonical compressed for

Re: FSFS format7 and compressed XML bundles

On Fri, Mar 1, 2013 at 5:49 PM, Julian Foad wrote: > No, that's not true. I think the article Ben read was inaccurate. The > '--rsyncable' option doesn't reset the compression after a fixed number of > bytes, but rather at every point where a rolling checksum of the last N bytes > leading up

Re: FSFS format7 and compressed XML bundles

> Vincent Lefevre wrote: >> Ben Reser wrote: >>> It resets the compression algorithm every 1000 bytes and thus makes >>> blocks that can be saved between revisions of the file. >> >> Wouldn't this work only when data are appended to the file? >> If data are inserted or deleted, this would cha

Re: FSFS format7 and compressed XML bundles

On Thu, Feb 28, 2013 at 12:08 PM, Stefan Fuhrmann wrote: > ZIP - in contrast to .tar.gz - compresses each of these files > individually and then mainly concatenates them into the > result file. As long as you don't change the template or > any of the existing pictures, for instance, larger parts o

Re: FSFS format7 and compressed XML bundles

On Fri, Mar 1, 2013 at 5:44 AM, Vincent Lefevre wrote: > Wouldn't this work only when data are appended to the file? > If data are inserted or deleted, this would change the block > boundaries. Instead of fixed-length blocks, I'd rather see > boundaries based on the file contents. That's true, th

Re: FSFS format7 and compressed XML bundles

On Fri, Mar 1, 2013 at 6:30 AM, Vincent Lefevre wrote: > On 2013-03-01 14:24:07 +, Philip Martin wrote: >> $ gzip --help | grep rsync >> --rsyncable Make rsync-friendly archive >> $ dpkg -s gzip | grep Version >> Version: 1.5-1.1 > > OK, then it seems that its man page is out-of-date.

Re: FSFS format7 and compressed XML bundles

Julian Foad writes: > Yes, a client-side plug-in -- either to Subversion or to OpenOffice -- > seems to me the best practical solution. A server-side solution is difficult. Suppose the client has some uncompressed content U which it compresses to C and sends to the server. The server can uncomp

Re: FSFS format7 and compressed XML bundles

On 2013-03-01 14:24:07 +, Philip Martin wrote: > $ gzip --help | grep rsync > --rsyncable Make rsync-friendly archive > $ dpkg -s gzip | grep Version > Version: 1.5-1.1 OK, then it seems that its man page is out-of-date. -- Vincent Lefèvre - Web: 100% access

Re: FSFS format7 and compressed XML bundles

Vincent Lefevre writes: > On 2013-02-28 10:58:07 -0800, Ben Reser wrote: >> Speaking with Julian here at ApacheCon he mentioned that gzip has a >> rsyncable option. Looking into this turns out that there is a patch >> applied to Debian's gzip that provides this option. > > I can't see such an op

Re: FSFS format7 and compressed XML bundles

On 2013-02-28 10:58:07 -0800, Ben Reser wrote: > Speaking with Julian here at ApacheCon he mentioned that gzip has a > rsyncable option. Looking into this turns out that there is a patch > applied to Debian's gzip that provides this option. I can't see such an option in Debian's gzip (from unstab

Re: FSFS format7 and compressed XML bundles

On 2013-02-28 08:37:51 -0800, Ben Reser wrote: > 3) You'd be saving storage at the expense of using time (read: CPU) on > every client that's working with those files when checking out. So > the end result may be worse than the current problem. But storage is permanent, while a checkout may occur

RE: FSFS format7 and compressed XML bundles

> -Original Message- > From: Julian Foad [mailto:[email protected]] > Sent: donderdag 28 februari 2013 20:54 > To: Ben Reser > Cc: Magnus Thor Torfason; Subversion Development > Subject: Re: FSFS format7 and compressed XML bundles > > Ben Reser wro

Re: FSFS format7 and compressed XML bundles

On Thu, Feb 28, 2013 at 5:04 PM, Magnus Thor Torfason < [email protected]> wrote: > Hey all, > Sorry that I have to disagree with what most people said. I guess, Mark got closed to the what the current intend is. I've been following the discussion about FSFS format7, and had a question: > I

Re: FSFS format7 and compressed XML bundles

Ben Reser wrote: > Speaking with Julian here at ApacheCon he mentioned that gzip has a > rsyncable option.  Looking into this turns out that there is a patch > applied to Debian's gzip that provides this option.  It resets the > compression algorithm every 1000 bytes and thus makes blocks that can

Re: FSFS format7 and compressed XML bundles

On Thu, Feb 28, 2013 at 1:45 PM, Ben Reser wrote: > On Thu, Feb 28, 2013 at 8:28 AM, Mark Phippard wrote: > > FWIW, the Branch Readme does imply he intends to work on some things that > > might have an impact here. > I pasted the contents of the readme merely to point out that he indicates that

Re: FSFS format7 and compressed XML bundles

On Thu, Feb 28, 2013 at 8:37 AM, Ben Reser wrote: > I just don't see this happening unless someone has a very clever idea > that I haven't thought of. Speaking with Julian here at ApacheCon he mentioned that gzip has a rsyncable option. Looking into this turns out that there is a patch applied t

Re: FSFS format7 and compressed XML bundles

On Thu, Feb 28, 2013 at 8:28 AM, Mark Phippard wrote: > FWIW, the Branch Readme does imply he intends to work on some things that > might have an impact here. Specifically: > > TxDelta v2 > -- > > Version 1 of txdelta turns out to be limited in its effectiveness for > larger files when da

Re: FSFS format7 and compressed XML bundles

On Thu, Feb 28, 2013 at 8:04 AM, Magnus Thor Torfason wrote: > I've been following the discussion about FSFS format7, and had a question: > Is there any chance that the format would improve storage efficiency for > documents that are stored as compressed (zipped) bundles of XML files and > other r

Re: FSFS format7 and compressed XML bundles

On Thu, Feb 28, 2013 at 11:25 AM, Branko Čibej wrote: > On 28.02.2013 08:04, Magnus Thor Torfason wrote: > > Hey all, > > > > I've been following the discussion about FSFS format7, and had a > > question: Is there any chance that the format would improve storage > > efficiency for documents that

Re: FSFS format7 and compressed XML bundles

On 28.02.2013 08:04, Magnus Thor Torfason wrote: > Hey all, > > I've been following the discussion about FSFS format7, and had a > question: Is there any chance that the format would improve storage > efficiency for documents that are stored as compressed (zipped) > bundles of XML files and other r