Re: [CVS] RPM: rpm/ CHANGES rpm/scripts/ dbconvert.sh

2010-11-11 Thread Jeff Johnson 
Nice! I knew if I remained being a lazy schmuck long
enough a check-in would appear ;-\

What remains is to figger what invokes the script ...

%posttrans is feasible (but if you miss you're toast).

And URPMI can (of course) do whatever it chooses to do.
The problem there is that its not just URPM, but also apt-rpm
and yum and smart and zypp and PackageKit and ... that
all need to do similar. Not gonn happen imho.

I'd still suggest thinking carefully about use manual intervention
and an opt-in approach. Automation (when it misses) tends to
inflict pain, unsurprisingly.

jmho

73 de Jeff


On Nov 11, 2010, at 4:14 PM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   11-Nov-2010 22:14:09
  Branch: HEAD Handle: 201021140900
 
  Added files:
rpm/scripts dbconvert.sh
  Modified files:
rpm CHANGES
 
  Log:
add a first prototype scripts/dbconvert.sh for handling rpmdb
conversion during upgrades.
 
  Summary:
RevisionChanges Path
1.3494  +2  -0  rpm/CHANGES
1.1 +139 -0 rpm/scripts/dbconvert.sh
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3493 -r1.3494 CHANGES
  --- rpm/CHANGES  3 Nov 2010 14:07:35 -   1.3493
  +++ rpm/CHANGES  11 Nov 2010 21:14:09 -  1.3494
  @@ -1,4 +1,6 @@
   5.3.5 - 5.4a1:
  +- proyvind: add a first prototype scripts/dbconvert.sh for handling 
 rpmdb
  +conversion during upgrades.
   - jbj: rpmdb: double the defaults  for locking tunables.
   - jbj: rpmdb: fix: don't bother excluding RPM_CHAR_TYPE sanity check.
   - jbj: rpmdb: fix: don't bother converting to RPM_UINT8_TYPE.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/scripts/dbconvert.sh
  
  $ cvs diff -u -r0 -r1.1 dbconvert.sh
  --- /dev/null2010-11-11 22:11:01.0 +0100
  +++ dbconvert.sh 2010-11-11 22:14:09.227464410 +0100
  @@ -0,0 +1,139 @@
  +TIMESTAMP=$(LC_TIME=C date -u +%F_%R)
  +BACKUP=/var/tmp/rpmdb-$TIMESTAMP
  +OLDDB=/var/tmp/olddb
  +DBPATH=${DBPATH:-/var/lib/rpm}
  +
  +DB_DUMP=$(which db51_dump 2 /dev/null)
  +if [ -z $DB_DUMP -a ! -x $DB_DUMP ]; then
  +DB_DUMP=$(which db_dump 2 /dev/null)
  +fi
  +if [ -x $DB_DUMP ]; then
  +echo Found $DB_DUMP
  +else
  +echo Unable to locate db_dump
  +exit 1
  +fi
  +
  +DB_DUMP_VERSION=$($DB_DUMP -V |sed 's/^Berkeley DB 
 \([0-9]\+\.[0-9]\+\).*/\1/')
  +if [ $DB_DUMP_VERSION == 5.1 ]; then
  +echo $DB_DUMP version $DB_DUMP_VERSION ok
  +else
  +echo Incompatible db_dump version ($DB_DUMP_VERSION) found, 5.1.* 
 required
  +exit 1
  +fi
  +
  +DB_LOAD=$(which db51_load 2 /dev/null)
  +if [ -z $DB_LOAD -a ! -x $DB_LOAD ]; then
  +DB_LOAD=$(which db_load 2 /dev/null)
  +fi
  +if [ -x $DB_LOAD ]; then
  +echo Found $DB_LOAD
  +else
  +echo Unable to locate db_load
  +exit 1
  +fi
  +
  +DB_LOAD_VERSION=$($DB_LOAD -V |sed 's/^Berkeley DB 
 \([0-9]\+\.[0-9]\+\).*/\1/')
  +if [ $DB_LOAD_VERSION == 5.1 ]; then
  +echo $DB_LOAD version $DB_LOAD_VERSION ok
  +else
  +echo Incompatible db_load version ($DB_LOAD_VERSION) found, 5.1.* 
 required
  +exit 1
  +fi
  +
  +echo Converting system database.
  +rm -rf $OLDDB
  +mkdir -p {$OLDDB,$DBPATH}/{log,tmp}
  +if [ $DBPATH != /var/lib/rpm ]; then
  +if [ -f /var/lib/rpm/DB_CONFIG ]; then
  +cp /var/lib/rpm/DB_CONFIG $DBPATH/DB_CONFIG
  +else
  +tee $DBPATH/DB_CONFIG  EOH
  +#  Environment
  +#add_data_dir   .
  +set_data_dir.
  +set_create_dir  .
  +set_lg_dir  ./log
  +set_tmp_dir ./tmp
  +
  +# -- thread_count must be = 8
  +set_thread_count64
  +
  +#set_verboseDB_VERB_DEADLOCK
  +#set_verboseDB_VERB_FILEOPS
  +#set_verboseDB_VERB_FILEOPS_ALL
  +#set_verboseDB_VERB_RECOVERY
  +#set_verboseDB_VERB_REGISTER
  +#set_verboseDB_VERB_REPLICATION
  +#set_verboseDB_VERB_REP_ELECT
  +#set_verboseDB_VERB_REP_LEASE
  +#set_verboseDB_VERB_REP_MISC
  +#set_verboseDB_VERB_REP_MSGS
  +#set_verboseDB_VERB_REP_SYNC
  +#set_verboseDB_VERB_REP_TEST
  +#set_verboseDB_VERB_REPMGR_CONNFAIL
  +#set_verboseDB_VERB_REPMGR_MISC
  +#set_verboseDB_VERB_WAITSFOR
  +
  +#

Re: [CVS] RPM: rpm/ CHANGES rpm/scripts/ dbconvert.sh

2010-11-11 Thread Per Øyvind Karlsen 
Could you verify and make any appropriate modifications to it, and
also integrate the one referred to (if necessar/possible/relevant?) in
the mail with Shlomi?

Then I'll upload a new version to cooker. :)
2010/11/11 Per Ųyvind Karlsen pkarl...@rpm5.org:
  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  

  Server: rpm5.org                         Name:   Per Ųyvind Karlsen
  Root:   /v/rpm/cvs                       Email:  pkarl...@rpm5.org
  Module: rpm                              Date:   11-Nov-2010 22:14:09
  Branch: HEAD                             Handle: 201021140900

  Added files:
    rpm/scripts             dbconvert.sh
  Modified files:
    rpm                     CHANGES

  Log:
    add a first prototype scripts/dbconvert.sh for handling rpmdb
    conversion during upgrades.

  Summary:
    Revision    Changes     Path
    1.3494      +2  -0      rpm/CHANGES
    1.1         +139 -0     rpm/scripts/dbconvert.sh
  

  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3493 -r1.3494 CHANGES
  --- rpm/CHANGES       3 Nov 2010 14:07:35 -       1.3493
  +++ rpm/CHANGES       11 Nov 2010 21:14:09 -      1.3494
  @@ -1,4 +1,6 @@
   5.3.5 - 5.4a1:
  +    - proyvind: add a first prototype scripts/dbconvert.sh for handling 
 rpmdb
  +     conversion during upgrades.
       - jbj: rpmdb: double the defaults  for locking tunables.
       - jbj: rpmdb: fix: don't bother excluding RPM_CHAR_TYPE sanity check.
       - jbj: rpmdb: fix: don't bother converting to RPM_UINT8_TYPE.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/scripts/dbconvert.sh
  
  $ cvs diff -u -r0 -r1.1 dbconvert.sh
  --- /dev/null 2010-11-11 22:11:01.0 +0100
  +++ dbconvert.sh      2010-11-11 22:14:09.227464410 +0100
  @@ -0,0 +1,139 @@
  +TIMESTAMP=$(LC_TIME=C date -u +%F_%R)
  +BACKUP=/var/tmp/rpmdb-$TIMESTAMP
  +OLDDB=/var/tmp/olddb
  +DBPATH=${DBPATH:-/var/lib/rpm}
  +
  +DB_DUMP=$(which db51_dump 2 /dev/null)
  +if [ -z $DB_DUMP -a ! -x $DB_DUMP ]; then
  +    DB_DUMP=$(which db_dump 2 /dev/null)
  +fi
  +if [ -x $DB_DUMP ]; then
  +    echo Found $DB_DUMP
  +else
  +    echo Unable to locate db_dump
  +    exit 1
  +fi
  +
  +DB_DUMP_VERSION=$($DB_DUMP -V |sed 's/^Berkeley DB 
 \([0-9]\+\.[0-9]\+\).*/\1/')
  +if [ $DB_DUMP_VERSION == 5.1 ]; then
  +    echo $DB_DUMP version $DB_DUMP_VERSION ok
  +else
  +    echo Incompatible db_dump version ($DB_DUMP_VERSION) found, 5.1.* 
 required
  +    exit 1
  +fi
  +
  +DB_LOAD=$(which db51_load 2 /dev/null)
  +if [ -z $DB_LOAD -a ! -x $DB_LOAD ]; then
  +    DB_LOAD=$(which db_load 2 /dev/null)
  +fi
  +if [ -x $DB_LOAD ]; then
  +    echo Found $DB_LOAD
  +else
  +    echo Unable to locate db_load
  +    exit 1
  +fi
  +
  +DB_LOAD_VERSION=$($DB_LOAD -V |sed 's/^Berkeley DB 
 \([0-9]\+\.[0-9]\+\).*/\1/')
  +if [ $DB_LOAD_VERSION == 5.1 ]; then
  +    echo $DB_LOAD version $DB_LOAD_VERSION ok
  +else
  +    echo Incompatible db_load version ($DB_LOAD_VERSION) found, 5.1.* 
 required
  +    exit 1
  +fi
  +
  +echo Converting system database.
  +rm -rf $OLDDB
  +mkdir -p {$OLDDB,$DBPATH}/{log,tmp}
  +if [ $DBPATH != /var/lib/rpm ]; then
  +    if [ -f /var/lib/rpm/DB_CONFIG ]; then
  +     cp /var/lib/rpm/DB_CONFIG $DBPATH/DB_CONFIG
  +    else
  +     tee $DBPATH/DB_CONFIG  EOH
  +#  Environment
  +#add_data_dir                .
  +set_data_dir         .
  +set_create_dir               .
  +set_lg_dir           ./log
  +set_tmp_dir          ./tmp
  +
  +# -- thread_count must be = 8
  +set_thread_count     64
  +
  +#set_verbose         DB_VERB_DEADLOCK
  +#set_verbose         DB_VERB_FILEOPS
  +#set_verbose         DB_VERB_FILEOPS_ALL
  +#set_verbose         DB_VERB_RECOVERY
  +#set_verbose         DB_VERB_REGISTER
  +#set_verbose         DB_VERB_REPLICATION
  +#set_verbose         DB_VERB_REP_ELECT
  +#set_verbose         DB_VERB_REP_LEASE
  +#set_verbose         DB_VERB_REP_MISC
  +#set_verbose         DB_VERB_REP_MSGS
  +#set_verbose         DB_VERB_REP_SYNC
  +#set_verbose         DB_VERB_REP_TEST
  +#set_verbose         DB_VERB_REPMGR_CONNFAIL
  +#set_verbose         DB_VERB_REPMGR_MISC
  +#set_verbose         DB_VERB_WAITSFOR
  +
  +#  Logging
  +
  +#  Memory Pool
  +#XXX initializing dbenv with set_cachesize has unimplemented prerequsites
  +#set_cachesize               0 1048576 0
  +set_mp_mmapsize              16777216
  +
  +#  Locking
  +set_lk_max_locks     16384
  +set_lk_max_lockers   16384
  +set_lk_max_objects   16384
  +mutex_set_max                163840
  +
  +#  Replication
  +EOH
  +    fi
  +fi
  +
  +echo -- copy the system rpmdb
  +cp $DBPATH/Packages

Re: [CVS] RPM: rpm/ CHANGES rpm/scripts/ dbconvert.sh

2010-11-11 Thread Jeff Johnson 

On Nov 11, 2010, at 4:24 PM, Per Øyvind Karlsen wrote:

 Could you verify and make any appropriate modifications to it, and
 also integrate the one referred to (if necessar/possible/relevant?) in
 the mail with Shlomi?
 
 Then I'll upload a new version to cooker. :)

Hint: Watch the buildbot's ... next time its you, not me, who does the change.

73 de Jeff

 2010/11/11 Per Ųyvind Karlsen pkarl...@rpm5.org:
  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Ųyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   11-Nov-2010 22:14:09
  Branch: HEAD Handle: 201021140900
 
  Added files:
rpm/scripts dbconvert.sh
  Modified files:
rpm CHANGES
 
  Log:
add a first prototype scripts/dbconvert.sh for handling rpmdb
conversion during upgrades.
 
  Summary:
RevisionChanges Path
1.3494  +2  -0  rpm/CHANGES
1.1 +139 -0 rpm/scripts/dbconvert.sh
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3493 -r1.3494 CHANGES
  --- rpm/CHANGES   3 Nov 2010 14:07:35 -   1.3493
  +++ rpm/CHANGES   11 Nov 2010 21:14:09 -  1.3494
  @@ -1,4 +1,6 @@
   5.3.5 - 5.4a1:
  +- proyvind: add a first prototype scripts/dbconvert.sh for handling 
 rpmdb
  + conversion during upgrades.
   - jbj: rpmdb: double the defaults  for locking tunables.
   - jbj: rpmdb: fix: don't bother excluding RPM_CHAR_TYPE sanity check.
   - jbj: rpmdb: fix: don't bother converting to RPM_UINT8_TYPE.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/scripts/dbconvert.sh
  
  $ cvs diff -u -r0 -r1.1 dbconvert.sh
  --- /dev/null 2010-11-11 22:11:01.0 +0100
  +++ dbconvert.sh  2010-11-11 22:14:09.227464410 +0100
  @@ -0,0 +1,139 @@
  +TIMESTAMP=$(LC_TIME=C date -u +%F_%R)
  +BACKUP=/var/tmp/rpmdb-$TIMESTAMP
  +OLDDB=/var/tmp/olddb
  +DBPATH=${DBPATH:-/var/lib/rpm}
  +
  +DB_DUMP=$(which db51_dump 2 /dev/null)
  +if [ -z $DB_DUMP -a ! -x $DB_DUMP ]; then
  +DB_DUMP=$(which db_dump 2 /dev/null)
  +fi
  +if [ -x $DB_DUMP ]; then
  +echo Found $DB_DUMP
  +else
  +echo Unable to locate db_dump
  +exit 1
  +fi
  +
  +DB_DUMP_VERSION=$($DB_DUMP -V |sed 's/^Berkeley DB 
 \([0-9]\+\.[0-9]\+\).*/\1/')
  +if [ $DB_DUMP_VERSION == 5.1 ]; then
  +echo $DB_DUMP version $DB_DUMP_VERSION ok
  +else
  +echo Incompatible db_dump version ($DB_DUMP_VERSION) found, 5.1.* 
 required
  +exit 1
  +fi
  +
  +DB_LOAD=$(which db51_load 2 /dev/null)
  +if [ -z $DB_LOAD -a ! -x $DB_LOAD ]; then
  +DB_LOAD=$(which db_load 2 /dev/null)
  +fi
  +if [ -x $DB_LOAD ]; then
  +echo Found $DB_LOAD
  +else
  +echo Unable to locate db_load
  +exit 1
  +fi
  +
  +DB_LOAD_VERSION=$($DB_LOAD -V |sed 's/^Berkeley DB 
 \([0-9]\+\.[0-9]\+\).*/\1/')
  +if [ $DB_LOAD_VERSION == 5.1 ]; then
  +echo $DB_LOAD version $DB_LOAD_VERSION ok
  +else
  +echo Incompatible db_load version ($DB_LOAD_VERSION) found, 5.1.* 
 required
  +exit 1
  +fi
  +
  +echo Converting system database.
  +rm -rf $OLDDB
  +mkdir -p {$OLDDB,$DBPATH}/{log,tmp}
  +if [ $DBPATH != /var/lib/rpm ]; then
  +if [ -f /var/lib/rpm/DB_CONFIG ]; then
  + cp /var/lib/rpm/DB_CONFIG $DBPATH/DB_CONFIG
  +else
  + tee $DBPATH/DB_CONFIG  EOH
  +#  Environment
  +#add_data_dir.
  +set_data_dir .
  +set_create_dir   .
  +set_lg_dir   ./log
  +set_tmp_dir  ./tmp
  +
  +# -- thread_count must be = 8
  +set_thread_count 64
  +
  +#set_verbose DB_VERB_DEADLOCK
  +#set_verbose DB_VERB_FILEOPS
  +#set_verbose DB_VERB_FILEOPS_ALL
  +#set_verbose DB_VERB_RECOVERY
  +#set_verbose DB_VERB_REGISTER
  +#set_verbose DB_VERB_REPLICATION
  +#set_verbose DB_VERB_REP_ELECT
  +#set_verbose DB_VERB_REP_LEASE
  +#set_verbose DB_VERB_REP_MISC
  +#set_verbose DB_VERB_REP_MSGS
  +#set_verbose DB_VERB_REP_SYNC
  +#set_verbose DB_VERB_REP_TEST
  +#set_verbose DB_VERB_REPMGR_CONNFAIL
  +#set_verbose DB_VERB_REPMGR_MISC
  +#set_verbose DB_VERB_WAITSFOR
  +
  +#  Logging
  +
  +#  Memory Pool
  +#XXX initializing dbenv with set_cachesize has unimplemented prerequsites
  +#set_cachesize   0 1048576 0
  +set_mp_mmapsize  16777216
  +
  +#  Locking
  +set_lk_max_locks 16384
  +set_lk_max_lockers   16384
  +set_lk_max_objects   16384

Re: [CVS] RPM: rpm/ CHANGES rpm/scripts/ dbconvert.sh

2010-11-11 Thread Per Øyvind Karlsen 
2010/11/11 Jeff Johnson n3...@mac.com:

 On Nov 11, 2010, at 4:24 PM, Per Øyvind Karlsen wrote:

 Could you verify and make any appropriate modifications to it, and
 also integrate the one referred to (if necessar/possible/relevant?) in
 the mail with Shlomi?

 Then I'll upload a new version to cooker. :)

 Hint: Watch the buildbot's ... next time its you, not me, who does the change.
Didn't get the hint.. :/

--
Regards,
Per Øyvind
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/scripts/ dbconvert.sh

2010-11-11 Thread Jeff Johnson 

On Nov 11, 2010, at 4:42 PM, Per Øyvind Karlsen wrote:

 
 Didn't get the hint.. :/
 

OK.

See this link

http://pm:8010/builders/MDV1x_rpm_release/builds/11/steps/compile_5/logs/stdio

You are looking at Mandriva 2010.1, and make check is being run.

Down toward'd the bottom, you will see this:

...
-rw-r--r-- 1 buildbot buildbot 8192 2010-11-11 13:33 Triggername
-rw-r--r-- 1 buildbot buildbot90112 2010-11-11 13:33 Version
-- test the conversion
rootcerts-java-20100827.00-1mdv2011.0.i586
libgdkimlib1-1.9.15-9mdv2010.1.i586
...

Stare at that space until you see your scripts/dbconvert.sh check-in appear,
and being invoked.

Then look at all the other waterfalls, where you will _ALSO_ see 
scripts/dbconvert.sh
being run.

Soon ... time for hamachi  sake first, I loathe scripting ...

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/scripts/ dbconvert.sh

2010-11-11 Thread Jeff Johnson 

On Nov 11, 2010, at 4:56 PM, Jeff Johnson wrote:

 
 On Nov 11, 2010, at 4:42 PM, Per Øyvind Karlsen wrote:
 
 
 Didn't get the hint.. :/
 
 
 OK.
 
 See this link
   
 http://pm:8010/builders/MDV1x_rpm_release/builds/11/steps/compile_5/logs/stdio
 

Sorry pm == plymouth aka buildbot.rpm5.org. My DNS needs some loving ...

Welcome to continuous integration on the rpm-5_3 branch!

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org