Re: [gentoo-portage-dev] emerge-webrsync patch
On Wed, 2005-12-28 at 17:38 +0100, Johannes Fahrenkrug wrote: > Good point. Is this patch better? Or should it rather be _exactly_ as it > is in revdep-rebuild? I personally would do it the same way as revdep-rebuild since that causes the entire script and anything it calls to be run at the value set in PORTAGE_NICENESS. However, the way you have done it works just as well. As far as your patch goes, you don't need to call the emerge metadata with a nice value since the emerge command reads and uses the PORTAGE_NICENESS value. Finally, I am not a portage developer, so if this gets accepted and used depends upon them and not me. Regards, Paul -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] emerge-webrsync patch
On Wed, Dec 28, 2005 at 05:38:02PM +0100, Johannes Fahrenkrug wrote: > Paul Varner wrote: > > > > >Instead of hardcoding the nice value, use PORTAGE_NICENESS. Here is how > >it is done in revdep-rebuild > > > ># Obey PORTAGE_NICENESS > >PORTAGE_NICENESS=$(portageq envvar PORTAGE_NICENESS) > >[ ! -z "$PORTAGE_NICENESS" ] && renice $PORTAGE_NICENESS $$ > /dev/null > > > > > > > Good point. Is this patch better? Or should it rather be _exactly_ as it > is in revdep-rebuild? I'd suggest raiding from emerge-delta-webrsync for the portageq call; it's a bit nasty, but it's a single call rather then multiple. I'd also raid the tarsync call- this is something I was intending on doing but have't yet. It will cut out the untarring/rsyncing call to 2 read throughs of the tarball, and single run through the tree. Fair bit faster, especially if the user's box doesn't have the ram to buffer the tree/tarball in memory. Tagging portage_niceness into it, just create a var with the appropriate nice call- if no PORTAGE_NICENESS, then the var is empty. ~harring pgpNUrhTiBo25.pgp Description: PGP signature
Re: [gentoo-portage-dev] emerge-webrsync patch
Paul Varner wrote: Instead of hardcoding the nice value, use PORTAGE_NICENESS. Here is how it is done in revdep-rebuild # Obey PORTAGE_NICENESS PORTAGE_NICENESS=$(portageq envvar PORTAGE_NICENESS) [ ! -z "$PORTAGE_NICENESS" ] && renice $PORTAGE_NICENESS $$ > /dev/null Good point. Is this patch better? Or should it rather be _exactly_ as it is in revdep-rebuild? - Johannes. --- /usr/sbin/emerge-webrsync 2005-12-28 11:49:18.0 +0100 +++ emerge-webrsync 2005-12-28 17:33:50.179723896 +0100 @@ -18,9 +18,19 @@ cd "$DISTDIR" found=0 +rsyncops="-av --progress" + +PORTAGE_NICENESS=$(portageq envvar PORTAGE_NICENESS) +if [ -z "$PORTAGE_NICENESS" ] ; then +PORTAGE_NICENESS=0 +fi + if [ "$1" == "-v" ] ; then wgetops= else +if [ "$1" == "-q" ] ; then + rsyncops="-aq" +fi #this sucks. probably better to do 1> /dev/null #that said, waiting on the refactoring. if [ "${FETCHCOMMAND/wget}" != "${FETCHCOMMAND}" ]; then @@ -42,7 +52,7 @@ sync_local() { echo Syncing local tree... - if ! tar jxf $FILE; then + if ! nice -n $PORTAGE_NICENESS tar jxf $FILE; then echo "Tar failed to extract the image. Please review the output." echo "Executed command: tar jxf $FILE" exit 1 @@ -51,14 +61,14 @@ # Make sure user and group file ownership is root chown -R 0:0 portage cd portage - rsync -av --progress --stats --delete --delete-after \ + nice -n $PORTAGE_NICENESS rsync ${rsyncops} --stats --delete --delete-after \ --exclude='/distfiles' --exclude='/packages' \ --exclude='/local' . ${PORTDIR%%/} cd .. echo "cleaning up" rm -rf portage echo "transferring metadata/cache" - emerge metadata + nice -n $PORTAGE_NICENESS emerge metadata } echo "Fetching most recent snapshot"
Re: [gentoo-portage-dev] emerge-webrsync patch
On Wed, 2005-12-28 at 13:04 +0100, Johannes Fahrenkrug wrote: > I put a "nice -n 19" in front of the tar, rsync and emerge metadata > commands because normally calling emerge-webrsync renders my box > unusable for 15 to 20 minutes. You still notice a difference when using > "nice" but everything seems to be at least noticably smoother than before. Instead of hardcoding the nice value, use PORTAGE_NICENESS. Here is how it is done in revdep-rebuild # Obey PORTAGE_NICENESS PORTAGE_NICENESS=$(portageq envvar PORTAGE_NICENESS) [ ! -z "$PORTAGE_NICENESS" ] && renice $PORTAGE_NICENESS $$ > /dev/null Regards, Paul -- gentoo-portage-dev@gentoo.org mailing list
[gentoo-portage-dev] emerge-webrsync patch
Hi, I'm new to this list put I have been using Gentoo for several years. Thank you for all the hard work you put into this beautiful distribution. Since I'm behind a quite restrictive proxy at work, I have to frequently use emerge-webrsync. I looked at the script today and thought that 2 things might make sense/be nice. I put a "nice -n 19" in front of the tar, rsync and emerge metadata commands because normally calling emerge-webrsync renders my box unusable for 15 to 20 minutes. You still notice a difference when using "nice" but everything seems to be at least noticably smoother than before. The other change is that you can call emerge-webrsync with the "-q" option to suppress the rsync output. Please tell me what you think! - Johannes. --- /usr/sbin/emerge-webrsync 2005-12-28 11:49:18.0 +0100 +++ emerge-webrsync 2005-12-28 12:41:58.838852352 +0100 @@ -18,9 +18,14 @@ cd "$DISTDIR" found=0 +rsyncops="-av --progress" + if [ "$1" == "-v" ] ; then wgetops= else +if [ "$1" == "-q" ] ; then + rsyncops="-aq" +fi #this sucks. probably better to do 1> /dev/null #that said, waiting on the refactoring. if [ "${FETCHCOMMAND/wget}" != "${FETCHCOMMAND}" ]; then @@ -42,7 +47,7 @@ sync_local() { echo Syncing local tree... - if ! tar jxf $FILE; then + if ! nice -n 19 tar jxf $FILE; then echo "Tar failed to extract the image. Please review the output." echo "Executed command: tar jxf $FILE" exit 1 @@ -51,14 +56,14 @@ # Make sure user and group file ownership is root chown -R 0:0 portage cd portage - rsync -av --progress --stats --delete --delete-after \ + nice -n 19 rsync ${rsyncops} --stats --delete --delete-after \ --exclude='/distfiles' --exclude='/packages' \ --exclude='/local' . ${PORTDIR%%/} cd .. echo "cleaning up" rm -rf portage echo "transferring metadata/cache" - emerge metadata + nice -n 19 emerge metadata } echo "Fetching most recent snapshot"