Bug#602711: pbuilder: please add support for debdelta

2015-10-06 Thread Ritesh Raj Sarraf 
On Mon, 2015-10-05 at 22:47 +, Mattia Rizzolo wrote:
> > Based on our conversation on IRC, please consider the attached
> patch.
> 
> I merged it, and made a couple of follow-up commit improving its
> integration and robustness (e.g. a debdelta failure won't fail
> anything).
> Also, I tried to deal with the (imho) messed up dependecies of
> debdelta-upgrade.
> 

THanks.

> Please, try out a pbuilder checkout and see whether it suites you :)
> Also, please point out if you see anything that could be improved, or
> if
> I did something wrong or you don't like.
> 

I just tested both, the option and the config file setting. And both
look good to me.


For the case where there is no debdelta pre-installed, I like the way
it handles now. Thanks.

I'm not sure of the reason for the Recommends: on all those different
types of compression tools, but IIRC the idea was that on server side,
the delta generation could be from any of those tools. And so, all were
added to Recommends.


> Personally I won't use this, mainly because most of the time here I
> have
> a good enough connection, and while trying it it turned out to
> actually
> be slower than it does without (I have a local squid proxy and I
> already
> downloaded all of them for the host, fwiw), and the "recreate the
> .deb"
> step (which involves a lot of xz) takes a lot of time and makes my
> CPU
> boil with src:gcc-5 binaries
> 
> So, takes this time as a good moment to improve the implementation,
> that
> at least in this very moment is there only for you :)


Thanks.

-- 
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."



signature.asc
Description: This is a digitally signed message part

Bug#602711: pbuilder: please add support for debdelta

2015-10-05 Thread Mattia Rizzolo 
control: tag -1 pending

On Sat, Oct 03, 2015 at 06:34:23PM +0530, Ritesh Raj Sarraf wrote:
> On Sat, 2014-03-22 at 13:50 +0530, Ritesh Raj Sarraf wrote:
> > Please consider integrating it as it would save a lot of resources
> > both,
> > for our users/developers, and our infrastructure.
> > 
> > Appended below is the log after applying this patch, and debdelta
> > into
> > action (ofcourse one is expected to install debdelta and its
> > dependencies in the chroot first)
> 
> 
> Hello Mattia,
> 
> Based on our conversation on IRC, please consider the attached patch.

I merged it, and made a couple of follow-up commit improving its
integration and robustness (e.g. a debdelta failure won't fail
anything).
Also, I tried to deal with the (imho) messed up dependecies of
debdelta-upgrade.

Please, try out a pbuilder checkout and see whether it suites you :)
Also, please point out if you see anything that could be improved, or if
I did something wrong or you don't like.

Personally I won't use this, mainly because most of the time here I have
a good enough connection, and while trying it it turned out to actually
be slower than it does without (I have a local squid proxy and I already
downloaded all of them for the host, fwiw), and the "recreate the .deb"
step (which involves a lot of xz) takes a lot of time and makes my CPU
boil with src:gcc-5 binaries

So, takes this time as a good moment to improve the implementation, that
at least in this very moment is there only for you :)

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  http://mapreri.org  : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Bug#602711: pbuilder: please add support for debdelta

2015-10-03 Thread Ritesh Raj Sarraf 
On Sat, 2014-03-22 at 13:50 +0530, Ritesh Raj Sarraf wrote:
> Please consider integrating it as it would save a lot of resources
> both,
> for our users/developers, and our infrastructure.
> 
> Appended below is the log after applying this patch, and debdelta
> into
> action (ofcourse one is expected to install debdelta and its
> dependencies in the chroot first)


Hello Mattia,

Based on our conversation on IRC, please consider the attached patch.



Here's the result.

rrs@learner:/var/tmp/Debian-Build/Result$ sudo pbuilder --update --
debdelta
I: Current time: Sat Oct  3 18:17:41 IST 2015
I: pbuilder-time-stamp: 1443876461
I: Building the build Environment
I: extracting base tarball [/var/cache/pbuilder/sid-amd64-base.tgz]
I: copying local configuration
I: mounting /proc filesystem
I: mounting /run/shm filesystem
I: mounting /dev/pts filesystem
I: Mounting /var/cache/apt/archives/
I: policy-rc.d already exists
I: Refreshing the base.tgz 
I: upgrading packages
Hit http://ftp.debian.org sid InRelease
Get:1 http://ftp.debian.org sid/main amd64 Packages/DiffIndex [4540 B]
Get:2 http://ftp.debian.org sid/contrib amd64 Packages/DiffIndex [919
B]
Get:3 http://ftp.debian.org sid/non-free amd64 Packages/DiffIndex [1057
B]
Get:4 http://ftp.debian.org sid/contrib Translation-en/DiffIndex [367
B]
Get:5 http://ftp.debian.org sid/main Translation-en/DiffIndex [4401 B]
Get:6 http://ftp.debian.org sid/non-free Translation-en/DiffIndex [1057
B]
Fetched 12.3 kB in 2s (4859 B/s)
Reading package lists...
I: Using debdelta-upgrade for available deltas
Delta-upgrade
statistics:
   
   
 total resulting debs, size 0B time 2sec virtual speed 0B/sec
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists...
Building dependency tree...
Reading state information...
Package 'ccache' is not installed, so not removed
apt-utils is already the newest version.
aptitude is already the newest version.
build-essential is already the newest version.
debdelta is already the newest version.
dpkg-dev is already the newest version.
eatmydata is already the newest version.
python-apt is already the newest version.
xdelta3 is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
I: unmounting /var/cache/apt/archives/ filesystem
I: unmounting dev/pts filesystem
I: unmounting run/shm filesystem
I: unmounting proc filesystem
I: creating base tarball [/var/cache/pbuilder/sid-amd64-base.tgz]
I: cleaning the build env 
I: removing directory /tmp/27612 and its subdirectories
2015-10-03 / 18:19:31 ♒♒♒  ☺    


-- 
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."

From fc9544ab825c111c1ad137ae9f431dde90fb7e09 Mon Sep 17 00:00:00 2001
From: Ritesh Raj Sarraf 
Date: Sat, 3 Oct 2015 16:09:43 +0530
Subject: [PATCH 2/2] Add debdelta as a feature, which can be enabled in
 pbuilderrc

Fix typo in variable reference

Signed-off-by: Ritesh Raj Sarraf 
---
 pbuilder-checkparams| 4 
 pbuilder-updatebuildenv | 5 +
 2 files changed, 9 insertions(+)

diff --git a/pbuilder-checkparams b/pbuilder-checkparams
index 07534ba..15f26cc 100755
--- a/pbuilder-checkparams
+++ b/pbuilder-checkparams
@@ -217,6 +217,10 @@ while [ -n "$1" ]; do
 	ALLOWUNTRUSTED=yes;
 	shift;
 	;;
+	--debdelta)
+	DEBDELTA=yes;
+	shift;
+	;;
 	--keyring)
 	APTKEYRINGS[${#APTKEYRINGS[@]}]="$2";
 	shift; shift;
diff --git a/pbuilder-updatebuildenv b/pbuilder-updatebuildenv
index 13dd043..713ad6e 100755
--- a/pbuilder-updatebuildenv
+++ b/pbuilder-updatebuildenv
@@ -41,6 +41,11 @@ log "I: Refreshing the base.tgz "
 log "I: upgrading packages"
 $CHROOTEXEC /usr/bin/apt-get -q update
 
+if [ "$DEBDELTA" = "yes" ]; then
+	log "I: Using debdelta-upgrade for available deltas"
+	$CHROOTEXEC /usr/bin/debdelta-upgrade
+fi
+
 case "$(readlink -e "$PBUILDERSATISFYDEPENDSCMD")" in
   *-aptitude)
 EXTRAPACKAGES="$EXTRAPACKAGES aptitude"
-- 
2.6.0

From 02d5e0706f95f75a76b99037cfa003ebda42ccad Mon Sep 17 00:00:00 2001
From: Ritesh Raj Sarraf 
Date: Sat, 3 Oct 2015 18:33:17 +0530
Subject: [PATCH] Also update the manpage

Signed-off-by: Ritesh Raj Sarraf 
---
 pbuilderrc.5 | 5 +
 1 file changed, 5 insertions(+)

diff --git a/pbuilderrc.5 b/pbuilderrc.5
index 5a1f6aa..b0f19d1 100644
--- a/pbuilderrc.5
+++ b/pbuilderrc.5
@@ -188,6 +188,11 @@ may also be used to reset the list of options.
 
 The default value is to build source and binary package.
 .TP
+
+.BI "DEBDELTA=" "yes/no"
+Whether to use debdelta or

Bug#602711: pbuilder: please add support for debdelta

2014-03-22 Thread Ritesh Raj Sarraf 
Control: tag -1 patch

On 11/07/2010 07:13 PM, Ritesh Raj Sarraf wrote:
 Package: pbuilder
 Version: 0.199
 Severity: wishlist


 debdelta helps lower down the install size of packages. It only
 downloads the delta. debdelta is now available at debdelta.debian.net

 Please add support for it into pbuilder.

13:46:31 rrs@zan:/var/tmp$ diff /tmp/pbuilder-updatebuildenv
/usr/lib/pbuilder/pbuilder-updatebuildenv
68a69   

 

 $CHROOTEXEC /usr/bin/debdelta-upgrade ||
true
  



It is a one liner so I didn't bother to prepare a proper patch.

Please consider integrating it as it would save a lot of resources both,
for our users/developers, and our infrastructure.

Appended below is the log after applying this patch, and debdelta into
action (ofcourse one is expected to install debdelta and its
dependencies in the chroot first)


13:38:33 rrs@zan:/var/tmp$ sudo DIST=experimental pbuilder update
I: Current time: Sat Mar 22 13:38:35 IST 2014
I: pbuilder-time-stamp: 1395475715
I: Building the build Environment
I: extracting base tarball [/var/cache/pbuilder/experimental-amd64-base.tgz]
I: creating local configuration
I: copying local configuration
I: mounting /proc filesystem
I: mounting /run/shm filesystem
I: mounting /dev/pts filesystem
I: Mounting /var/cache/apt/archives/
I: policy-rc.d already exists
I: Refreshing the base.tgz
I: upgrading packages
Hit http://ftp.debian.org sid InRelease
Hit http://ftp.debian.org experimental InRelease
Hit http://ftp.debian.org sid/main amd64 Packages/DiffIndex
Hit http://ftp.debian.org sid/contrib amd64 Packages/DiffIndex
Hit http://ftp.debian.org sid/non-free amd64 Packages/DiffIndex
Hit http://ftp.debian.org sid/contrib Translation-en/DiffIndex
Hit http://ftp.debian.org sid/main Translation-en/DiffIndex
Hit http://ftp.debian.org sid/non-free Translation-en/DiffIndex
Hit http://ftp.debian.org experimental/main amd64 Packages/DiffIndex
Hit http://ftp.debian.org experimental/contrib amd64 Packages/DiffIndex
Hit http://ftp.debian.org experimental/non-free amd64 Packages/DiffIndex
Hit http://ftp.debian.org experimental/contrib Translation-en/DiffIndex
Hit http://ftp.debian.org experimental/main Translation-en/DiffIndex
Hit http://ftp.debian.org experimental/non-free Translation-en/DiffIndex
Reading package lists...
Created,time  0.29sec, speed 500kB/sec,
gcc-4.7-base_4.7.3-12_amd64.deb 
 

Created,time  0.17sec, speed 866kB/sec,
gcc-4.8-base_4.8.2-17_amd64.deb 
 

Created,time  0.12sec, speed 509kB/sec,
libasan0_4.8.2-17_amd64.deb 
 

Delta is not present: libatomic1_4.8.2-16_4.8.2-17_amd64.debdelta
Created,time  0.13sec, speed 348kB/sec,
libdebconfclient0_0.189_amd64.deb   
 

Created,time  0.15sec, speed 236kB/sec,
libgcc1_1%3a4.8.2-17_amd64.deb  
 

Delta is too big:
libitm1_4.8.2-16_4.8.2-17_amd64.debdelta
   

Created,time  0.12sec, speed 1014kB/sec,
libquadmath0_4.8.2-17_amd64.deb 


Created,time  0.24sec, speed 384kB/sec,
libtsan0_4.8.2-17_amd64.deb 
 

Created,time  0.17sec, speed 332kB/sec,
readline-common_6.3-4_all.deb   
 

Delta is not present: tzdata_2013i-1_2014a-1_all.debdelta
Downloaded, time  0.53sec, speed 2856B/sec,
libgomp1_4.8.2-16_4.8.2-17_amd64.debdelta   
 

Created,time  0.12sec, speed 183kB/sec,
libgomp1_4.8.2-17_amd64.deb 
 

Downloaded, time  0.42sec, speed 11kB/sec,
libapt-inst1.5_0.9.15.5+b1_0.9.16.1_amd64.debdelta  
  

Created,time  0.23sec, speed 678kB/sec,
libapt-inst1.5_0.9.16.1_amd64.deb   
 

Downloaded, time  0.71sec, speed 23kB/sec,

Bug#602711: pbuilder: please add support for debdelta

2010-11-07 Thread Ritesh Raj Sarraf 
Package: pbuilder
Version: 0.199
Severity: wishlist


debdelta helps lower down the install size of packages. It only
downloads the delta. debdelta is now available at debdelta.debian.net

Please add support for it into pbuilder.



-- System Information:
Debian Release: squeeze/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.36-trunk-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages pbuilder depends on:
ii  coreutils 8.5-1  GNU core utilities
ii  debconf [debconf-2.0] 1.5.36 Debian configuration management sy
ii  debianutils   3.4.1  Miscellaneous utilities specific t
ii  debootstrap   1.0.25 Bootstrap a basic Debian system
ii  wget  1.12-2.1   retrieves files from the web

Versions of packages pbuilder recommends:
ii  devscripts2.10.69scripts to make the life of a Debi
ii  fakeroot  1.14.4-1   Gives a fake root environment
ii  sudo  1.7.4p4-4  Provide limited super user privile

Versions of packages pbuilder suggests:
ii  cowdancer 0.62   Copy-on-write directory tree utili
pn  gdebi-corenone (no description available)
ii  pbuilder-uml  0.199  user-mode-linux version of pbuilde

-- debconf information:
* pbuilder/mirrorsite: http://ftp.debian.org/debian/
  pbuilder/nomirror:
* pbuilder/rewrite: false



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org