Re: [pacman-dev] delta support in libalpm

2009-03-14 Thread Allan McRae
Xavier wrote: On Fri, Feb 20, 2009 at 8:04 AM, Allan McRae al...@archlinux.org wrote: Would it be useful if I put xdelta3 into the repos to help testing things out for this? Short answer : yes, I believe it would be nice to have xdelta3 in the repos snip xdelta3 is now in the

Re: [pacman-dev] delta support in libalpm

2009-02-27 Thread Aaron Griffin
On Fri, Feb 27, 2009 at 6:18 AM, Xavier shinin...@gmail.com wrote: On Fri, Feb 27, 2009 at 12:37 AM, Aaron Griffin aaronmgrif...@gmail.com wrote: On Thu, Feb 26, 2009 at 5:21 PM, Xavier shinin...@gmail.com wrote: On Thu, Feb 26, 2009 at 11:34 PM, Aaron Griffin aaronmgrif...@gmail.com wrote:

Re: [pacman-dev] delta support in libalpm

2009-02-26 Thread Aaron Griffin
On Thu, Feb 26, 2009 at 4:19 PM, Xavier shinin...@gmail.com wrote: On Mon, Feb 23, 2009 at 6:48 PM, Aaron Griffin aaronmgrif...@gmail.com wrote: Questions which make the implementation complex: * When do we generate deltas? As part of the db scripts? Well I think that would be practical.

Re: [pacman-dev] delta support in libalpm

2009-02-26 Thread Xavier
On Thu, Feb 26, 2009 at 11:34 PM, Aaron Griffin aaronmgrif...@gmail.com wrote: So, ok, from a db-scripts point of view, we're going to have to do the following: when a new package is added:   copy old package file from ftp to build dir   generate delta from old file - new file (in staging)

Re: [pacman-dev] delta support in libalpm

2009-02-26 Thread Aaron Griffin
On Thu, Feb 26, 2009 at 5:37 PM, Aaron Griffin aaronmgrif...@gmail.com wrote: correctly construct deltas in different directories, or do we also need to copy the file next to the new package before running pkgdelta need to be next to each other? Ignore the last need to be next to each other, I

Re: [pacman-dev] delta support in libalpm

2009-02-23 Thread Brendan Hide
Xavier wrote: There has never been any real official interests for delta. This seems to make a requirement the ability to make a separate delta server. This seems to require a separate delta database. This implies a new level of complexity and code bloat in pacman. Now maybe it is worth it, I

Re: [pacman-dev] delta support in libalpm

2009-02-23 Thread Xavier
On Mon, Feb 23, 2009 at 9:57 AM, Brendan Hide bren...@swiftspirit.co.za wrote: Xavier wrote: There has never been any real official interests for delta. This seems to make a requirement the ability to make a separate delta server. This seems to require a separate delta database. This implies

Re: [pacman-dev] delta support in libalpm

2009-02-23 Thread Brendan Hide
b Xavier wrote: Everything is already implemented in pacman, with a more complex logic (which might be totally useless after all) For each package in a sync db, there is a deltas file besides the depends and desc one which basically contains the list of deltas for that package and their size.

Re: [pacman-dev] delta support in libalpm

2009-02-23 Thread Xavier
On Mon, Feb 23, 2009 at 12:27 PM, Brendan Hide bren...@swiftspirit.co.za wrote: This makes a lot more sense to me now. Thank you for the clarification, Xavier. It is the most efficient way, end-user-wise, despite the possibly-excessive metadata. It isn't necessarily efficient for the server.

Re: [pacman-dev] delta support in libalpm

2009-02-23 Thread Brendan Hide
Xavier wrote: how Garns answered to them: ... For Arch this would mean creating deltas on Gerolde, which seems to be fairly strained already. ... http://www.archlinux.org/pipermail/pacman-dev/2008-November/007672.htm Is Gerolde separate from the server that serves the FTP and HTTP traffic? If

Re: [pacman-dev] delta support in libalpm

2009-02-23 Thread Xavier
On Mon, Feb 23, 2009 at 12:43 PM, Brendan Hide bren...@swiftspirit.co.za wrote: Xavier wrote: how Garns answered to them: ... For Arch this would mean creating deltas on Gerolde, which seems to be fairly strained already. ...

Re: [pacman-dev] delta support in libalpm

2009-02-23 Thread Aaron Griffin
On Mon, Feb 23, 2009 at 5:58 AM, Xavier shinin...@gmail.com wrote: On Mon, Feb 23, 2009 at 12:43 PM, Brendan Hide bren...@swiftspirit.co.za wrote: Xavier wrote: how Garns answered to them: ... For Arch this would mean creating deltas on Gerolde, which seems to be fairly strained already.

Re: [pacman-dev] delta support in libalpm

2009-02-20 Thread Xavier
On Fri, Feb 20, 2009 at 8:04 AM, Allan McRae al...@archlinux.org wrote: Would it be useful if I put xdelta3 into the repos to help testing things out for this? Short answer : yes, I believe it would be nice to have xdelta3 in the repos Long answer : Well I did have something running, but I

Re: [pacman-dev] delta support in libalpm

2009-02-19 Thread Brendan Hide
Hi guys I'm new here so I'm asking in advance that you forgive my ignorance. Even before having clicked send, I feel like I'm spamming... o.O Xavier wrote: On Sat, Nov 8, 2008 at 12:47 AM, Henning Garus henning.ga...@googlemail.com wrote: 2. create the package, but don't compress it with

Re: [pacman-dev] delta support in libalpm

2009-02-19 Thread Xavier
On Thu, Feb 19, 2009 at 11:06 AM, Brendan Hide bren...@swiftspirit.co.za wrote: Xavier Wrote: So we don't even have extra decompression/recompression steps, there is no loss. + snprintf(command, PATH_MAX, xdelta3 -d -R -c -s %s %s | gzip -n %s, from, delta, to);

Re: [pacman-dev] delta support in libalpm

2009-02-19 Thread Xavier
On Sat, Nov 8, 2008 at 12:47 AM, Henning Garus henning.ga...@googlemail.com wrote: Delta creation in makepkg seems somehow ok (its already in there after all). But what I would really like is a separate tool for delta creation, which would allow the separation of building packages and

Re: [pacman-dev] delta support in libalpm

2009-02-19 Thread Allan McRae
Xavier wrote: On Sat, Nov 8, 2008 at 12:47 AM, Henning Garus henning.ga...@googlemail.com wrote: Delta creation in makepkg seems somehow ok (its already in there after all). But what I would really like is a separate tool for delta creation, which would allow the separation of building

Re: [pacman-dev] delta support in libalpm

2009-02-18 Thread Xavier
On Sat, Nov 8, 2008 at 12:47 AM, Henning Garus henning.ga...@googlemail.com wrote: 2. create the package, but don't compress it with bsdtar, use gzip -n instead. This means we have to use gzip again, in libalpm, when we apply the delta. Seems better than 1, but makes makepkg and libalpm

Re: [pacman-dev] delta support in libalpm

2008-11-14 Thread Nagy Gabor
Idézet Xavier [EMAIL PROTECTED]: On Fri, Nov 14, 2008 at 1:02 PM, Nagy Gabor [EMAIL PROTECTED] wrote: I guess I spoke to soon. I was right concerning xdelta3 using gzip for handling gzipped files, however it doesn't use the -n flag. This gives us the same behaviour as xdelta1, with one minor

Re: [pacman-dev] delta support in libalpm

2008-11-14 Thread Henning Garus
2008/11/14 Nagy Gabor [EMAIL PROTECTED]: Idézet Xavier [EMAIL PROTECTED]: On Fri, Nov 14, 2008 at 1:02 PM, Nagy Gabor [EMAIL PROTECTED] wrote: I guess I spoke to soon. I was right concerning xdelta3 using gzip for handling gzipped files, however it doesn't use the -n flag. This gives us

Re: [pacman-dev] delta support in libalpm

2008-11-13 Thread Henning Garus
On Mon, Nov 10, 2008 at 8:38 AM, Allan McRae [EMAIL PROTECTED] wrote: Henning Garus wrote: Yes you do. libalpm uses system() to execute: xdelta patch [deltafile] [oldpkg] [newpkg] xdelta will unzip the old package, apply the patch and rezip the new package. Due to the zlib/gzip

Re: [pacman-dev] delta support in libalpm

2008-11-10 Thread Allan McRae
Henning Garus wrote: On Sat, Nov 8, 2008 at 1:38 PM, Nagy Gabor [EMAIL PROTECTED] wrote: Hi, I have been looking through the current delta implementation in libalpm and have put some thought into changing makepkg/repo-add to support delta creation. However, I'm running into some problems,

Re: [pacman-dev] delta support in libalpm

2008-11-10 Thread Xavier
On Sat, Nov 8, 2008 at 12:47 AM, Henning Garus [EMAIL PROTECTED] wrote: Hi, I have been looking through the current delta implementation in libalpm and have put some thought into changing makepkg/repo-add to support delta creation. However, I'm running into some problems, mostly due to

Re: [pacman-dev] delta support in libalpm

2008-11-08 Thread Henning Garus
On Sat, Nov 8, 2008 at 1:38 PM, Nagy Gabor [EMAIL PROTECTED] wrote: Hi, I have been looking through the current delta implementation in libalpm and have put some thought into changing makepkg/repo-add to support delta creation. However, I'm running into some problems, mostly due to md5sums

[pacman-dev] delta support in libalpm

2008-11-07 Thread Henning Garus
Hi, I have been looking through the current delta implementation in libalpm and have put some thought into changing makepkg/repo-add to support delta creation. However, I'm running into some problems, mostly due to md5sums and gzip. The current implementation works as follows. On a sync