Re: Making rpm5 work with dnf

[Mark Hatle]

On 12/28/16 8:25 AM, Jeffrey Johnson wrote:
> 
>> On Dec 28, 2016, at 7:02 AM, Alexander Kanavin 
>>  wrote:
>>
>> On 12/27/2016 08:00 PM, Jeffrey Johnson wrote:
>>
>>> FYI: most of the openssl-1.1.0 port in rpm is now done.
>>>
>>> I’ve done “Do no harm testing.” with openssl-1.0.2j, will get to
>>> detailed openssl-1.1.0 testing as soon as I see a platform that
>>> distributes with openssl-1.1.0 (likely Fedora 26, not yet Fedora 25).
>>
>> Thanks! If you need a platform for testing, then debian testing (stretch) 
>> does ship with both openssl 1.1 and 1.0. You can have both libraries 
>> installed at the same time, but development packages mutually exclude each 
>> other. Despite the name, debian testing is fairly stable.
>>
> 
> Two versions of openssl installed isn’t what is hard, setting up a platform
> with one version of openssl well integrated is what is hard.
> 
> RPM links many libraries, some of which link openssl, and recompiling
> all RPM prerequisites to use a single version of openssl is very time 
> consuming.
> 
> Then there are issues of how openssl is installed: e.g. testing ECDSA
> usually requires rebuilding openssl and reading removed curves.
> 
> Then there is openssl-fips which RPM5 uses (or used, I’ve not checked 
> recently).
>  openssl-fips-2.0.13 functions with openssl-1.0.x, but not openssl-1.1.x. 
> FIPS 140-2
> seems to be quite a mess these days, sigh. Getting all the HMAC’s installed 
> and
> verified for FIPS 140-2 is always a chore.

AFAIK, only openssl 1.0.x supports the FIPS module.  There are a few folks
looking at private implementations of the older module with OpenSSL 1.1.x, but
definitely not official.

--Mark

> Meanwhile I have checked that RPM builds/links against openssl-1.1.0, and
> the code in ramie/rpmssl.c has rather simple usages of openssl.
> 
> hth
> 
> 73 de Jeff
> 
> __
> RPM Package Managerhttp://rpm5.org
> Developer Communication Listrpm-devel@rpm5.org
> 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Fix for rpmpgp in the rpm 5_4 branch

[Mark Hatle]

On 7/18/14, 3:08 PM, Jeffrey Johnson wrote:


On Jul 1, 2014, at 10:12 AM, Mark Hatle wrote:


The recent changes to add variable encryption have left one compilation issue 
that I found.  Attached is a patch that adds the missing ifdefs to resolve the 
issue.

--Mark
rpm-rpmpgp-fix.patch


This patch finally checked in, thank you.

There's _STILL_ some problem with RSA using openssl:


:)  At least you finally can reproduce it.  We did get the Yocto Project up to 
rpm 5.4.14.  (I also have a 'CVS' recipe for the newer stuff..)  So we are 
testing much newer code at least now.


--Mark


[jbj@ha tests]$ while true; do make check-sign; done
...
=== check-sign ===
-- DSA:
-- RSA:
-- rpmsslVerify(0x224ec30) BAD  RSA/SHA1
error: edos-test/tyre-2-0.noarch.rpm: Header V4 RSA/SHA1 signature: BAD, key ID 
36528658
error: reading edos-test/tyre-2-0.noarch.rpm manifest, non-printable characters 
found
make: *** [check-sign] Error 1

This is just a loop resigning with a constant private key (using gpg2),
and verifying using rpm internal openssl.

The error is specific to openssl ... likely guess is that the
bit count has gone awry somehow. Dunno yet.

At least I have a reproducer ... digging.

73 de Jeff



__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Fix for rpmpgp in the rpm 5_4 branch

[Mark Hatle]

The recent changes to add variable encryption have left one compilation issue 
that I found.  Attached is a patch that adds the missing ifdefs to resolve the 
issue.


--Mark
rpmpgp.c: Add missing if defs around crypto implementations

Without these, the system will error trying to find the correct crypto 
library to use.

Signed-off-by: Mark Hatle mark.ha...@windriver.com

Index: rpm/rpmio/rpmpgp.c
===
--- rpm.orig/rpmio/rpmpgp.c
+++ rpm/rpmio/rpmpgp.c
@@ -1339,16 +1339,26 @@ int pgpExportPubkey(pgpDig dig)
 {
 int rc = 0;/* assume failure */
 
+#if defined(WITH_BEECRYPT)
 if (pgpImplVecs == rpmbcImplVecs)
rc = rpmbcExportPubkey(dig);
+#endif
+#if defined(WITH_SSL)
 if (pgpImplVecs == rpmsslImplVecs)
rc = rpmsslExportPubkey(dig);
+#endif
+#if defined(WITH_NSS)
 if (pgpImplVecs == rpmnssImplVecs)
rc = rpmnssExportPubkey(dig);
+#endif
+#if defined(WITH_GCRYPT)
 if (pgpImplVecs == rpmgcImplVecs)
rc = rpmgcExportPubkey(dig);
+#endif
+#if defined(WITH_TOMCRYPT)
 if (pgpImplVecs == rpmltcImplVecs)
rc = rpmltcExportPubkey(dig);
+#endif
 return rc;
 }
 
@@ -1356,16 +1366,26 @@ int pgpExportSignature(pgpDig dig, DIGES
 {
 int rc = 0;/* assume failure */
 
+#if defined(WITH_BEECRYPT)
 if (pgpImplVecs == rpmbcImplVecs)
rc = rpmbcExportSignature(dig, ctx);
+#endif
+#if defined(WITH_SSL)
 if (pgpImplVecs == rpmsslImplVecs)
rc = rpmsslExportSignature(dig, ctx);
+#endif
+#if defined(WITH_NSS)
 if (pgpImplVecs == rpmnssImplVecs)
rc = rpmnssExportSignature(dig, ctx);
+#endif
+#if defined(WITH_GCRYPT)
 if (pgpImplVecs == rpmgcImplVecs)
rc = rpmgcExportSignature(dig, ctx);
+#endif
+#if defined(WITH_TOMCRYPT)
 if (pgpImplVecs == rpmltcImplVecs)
rc = rpmltcExportSignature(dig, ctx);
+#endif
 return rc;
 }
 

Re: rpmdb.c:2323: rpmmiNext: Assertion `0' failed (DB_BUFFER_SMALL)

[Mark Hatle]

 information that we're both seeing.

Attached is the patch I came up with.. like I said, it's not a fix -- it's a 
workaround and a rough one at that.


--Mark
In certain cases with BerkleyDB 5.3.x we are getting the error:

db3.c:1443: dbcursor-pget(-30999): BDB0063 DB_BUFFER_SMALL: User memory too 
small for return value

See https://bugs.launchpad.net/rpm/+bug/934420 for more information.

It appears to be some type of a bug in the BerkleyDB 5.3.x.  In an attempt
to workaround the problem, when we encounter this situation we attempt
to adjust the size of the mmap buffer until the call works, or we
end up trying 10 times.  The new size is either the updated vp-size
from the failed pget call, or the previous size + 1024.

If DBI debugging is enabled, additional diagnostics are printed, otherwise
a basic retry and success message is added to show that the failure was
resolved.

Signed-off-by: Mark Hatle mark.ha...@windriver.com

Index: rpm-5.4.9/rpmdb/rpmdb.c
===
--- rpm-5.4.9.orig/rpmdb/rpmdb.c
+++ rpm-5.4.9/rpmdb/rpmdb.c
@@ -2212,8 +2212,12 @@ static int rpmmiGet(dbiIndex dbi, DBC * 
vp-flags |= DB_DBT_USERMEM;
rc = dbiGet(dbi, dbcursor, kp, vp, flags);
if (rc == DB_BUFFER_SMALL) {
+   int retry = 0;
+   size_t origlen = vp-size;
size_t uhlen = vp-size;
-   void * uh = mmap(NULL, uhlen, _prot, _flags, _fdno, _off);
+   void * uh;
+retry_get:
+   uh = mmap(NULL, uhlen, _prot, _flags, _fdno, _off);
if (uh == NULL || uh == (void *)-1)
fprintf(stderr,
== mmap(%p[%u], 0x%x, 0x%x, %d, 0x%x) error(%d): %s\n,
@@ -2235,6 +2239,25 @@ static int rpmmiGet(dbiIndex dbi, DBC * 
if (munmap(uh, uhlen) != 0)
fprintf(stderr, == munmap(%p[%u]) error(%d): %s\n,
uh, (unsigned)uhlen, errno, strerror(errno));
+   /* We want to be sure to limit the number of retry attempts to 
avoid a loop! */
+   if (rc == DB_BUFFER_SMALL  retry  10) {
+  /* If we got a largr vp-size back, use that, otherwise 
increment the size by 1k */
+  uhlen = vp-size  uhlen ? vp-size : uhlen + 1024;
+  retry++;
+  if ((dbi)-dbi_debug)
+  fprintf(stderr, == DB_BUFFER_SMALL orig requested 
(%d), configured (%d), forcing larger buffer (%d), new size (%d)\n,
+   origlen, vp-ulen, uhlen, vp-size);
+  else
+  fprintf(stderr, == retry (%d) db3cpget (%d)\n, retry, 
uhlen);
+  goto retry_get;
+   }
+   }
+   if (retry) {
+   if ((dbi)-dbi_debug)
+  fprintf(stderr, == success orig requested (%d), configured 
buffer (%d), buffer (%d), size after dbiGet (%d)\n,
+   origlen, vp-ulen, uhlen, vp-size);
+   else
+  fprintf(stderr, == success\n);
}
}
 } else
Index: rpm-5.4.9/rpmdb/db3.c
===
--- rpm-5.4.9.orig/rpmdb/db3.c
+++ rpm-5.4.9/rpmdb/db3.c
@@ -1452,7 +1452,7 @@ assert(db != NULL);
 #endif
 }
 
-DBIDEBUG(dbi, (stderr, -- %s(%p,%p,%p,%p,%p,0x%x) rc %d %s%s\n, 
__FUNCTION__, dbi, dbcursor, key, pkey, data, flags, rc, _DBCFLAGS(flags), 
_KEYDATA(key, pkey, data, NULL)));
+DBIDEBUG(dbi, (stderr, -- %s(%p,%p,%p,%p,%p,0x%x) rc %d %s%s\n, 
__FUNCTION__, dbi, dbcursor, key, pkey, data, flags, rc, _DBCFLAGS(flags), 
_KEYDATA(key, pkey, rc == DB_BUFFER_SMALL ? NULL : data, NULL)));
 return rc;
 }
 /*@=mustmod@*/

Re: ossp uuid support

[Mark Hatle]

On 5/17/12 8:13 AM, Jeffrey Johnson wrote:


On May 17, 2012, at 8:25 AM, Anders F Björklund wrote:


Mark Hatle wrote:


I'm having problems enabling ossp uuid support with RPM5.

Both util-linux and ossp-uuid want to provide a /usr/lib/pkgconfig/uuid.pc file.  Since 
ossp-uuid is the alternative on my system, I'd like to avoid the conflict by 
renaming it.


If you rename the library, you should probably rename the .pc too ?

At least that is the way it is being installed in Redhat and Debian.


However, I can't figure out any way to get RPM's configure to look at a 
different uuid.pc file for configuration.. so it keeps pulling up the 
util-linux version and failing.


I think it will always look for uuid and libuuid, as it is now.



Yes: RPM_CHECK_LIB() will look for -lossp-uuid first, but
adding a second library to search for will continue and
find uuid.pc not ossp-uuid.pc.


Ya, it's finding the libossp-uuid.so no problem, but when it goes to find the 
.pc file for the cflags, it finds the uuid version and does a 
-I/usr/include/uuid, which is the wrong version.  I as Anders mentioned 
previously, the .pc file has been renamed ossp-uuid.pc.


Looking at the .m4 code in RPM5, I see the second argument to the RPM_CHECK_LIB 
is what is used to find the .pc file as well as what variable to stuff the 
result into.  I tried changing it to ossp-uuid (and it refused to pass 
autoreconf).. changed it to ossp_uuid and it worked, but then the variables were 
wrong.


In the end I had to put in a sed on the generated configure:

sed -e 's/pkg-config --exists uuid/pkg-config --exists ossp-uuid/' -e 
's/pkgconfig uuid/pkgconfig ossp-uuid/' -i configure


And at least it's doing what I want right now, but needless to say, I don't 
really like that method of configuring... ;)



Choosing the One True Name in AutoFu would require an
extra level of testing of *.pc/*-config/*.la to tell if the just
found information is useful (and that isn't erasable to attempt).


But if you use system instead of external, it should use flags.



Hmm, I didn't know system was an option.  I could certainly do this.  I'll give 
it a try and see what results.



Does anyone have any suggestions on how I can resolve this issue?

(I'm doing my best to avoid hacking up the generated configure...)


You could pass the output from `pkg-config ossp-uuid` in as flags ?

So instead of having configure call it, you call it yourself before.

CPPFLAGS += `pkg-config ossp-uuid --cflags-only-I`
CFLAGS   += `pkg-config ossp-uuid --cflags-only-other`
LDFLAGS  += `pkg-config ossp-uuid --libs-only-other`
LIBS += `pkg-config ossp-uuid --libs-only-l`



Passing flags in from external configuration will always work.


Or just avoid pkg-config and pass the good ole -I and -L flags...

Something like -I/usr/include/ossp is probably all that is needed.



I'm sure there will be some distro that prefers
-I${prefix}/ossp-uuid
as a ritual sacrifice to the Name Space Gods soon.


;)

--Mark


73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org



__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

ossp uuid support

[Mark Hatle]

I'm having problems enabling ossp uuid support with RPM5.

Both util-linux and ossp-uuid want to provide a /usr/lib/pkgconfig/uuid.pc file. 
 Since ossp-uuid is the alternative on my system, I'd like to avoid the 
conflict by renaming it.


However, I can't figure out any way to get RPM's configure to look at a 
different uuid.pc file for configuration.. so it keeps pulling up the util-linux 
version and failing.


Does anyone have any suggestions on how I can resolve this issue?

(I'm doing my best to avoid hacking up the generated configure...)

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Requires: python(foo) = bar

[Mark Hatle]

On 5/11/12 1:11 PM, Jeffrey Johnson wrote:

Matthew Dawkins mentioned an interest in better python module
dependencies in RPM earlier today.

A google search for python module dependency graph shows
some reasonably serious efforts to create reliable (i.e. not
*RE's scraping use … statements out of *.pl) dependencies.


I'd love better dependencies, but most of the time the implementations I see are 
focused around attempting to run the module in python and having it return back 
usage information.


This doesn't always work for me in my cross-install case.  (Note, I can 
certainly make it a requirement that the same version of python be available on 
the host (build system) as on the target system.  But I don't believe that is 
enough.)


--Mark


Perhaps best-of-breed (based on 10 mins of surfing) appears to be snake food
http://furius.ca/snakefood/
There's also (perhaps more recent/active)
http://pypi.python.org/pypi/modulegraph/

There's also (useful for verifying but not for generating dependencies
because it requires loading modules)
http://www.indelible.org/ink/python-reloading/
with a perhaps even simpler script here

http://code.activestate.com/recipes/577222-create-module-dependency-graph/

All of these are python scripts are generating *.dot output. So one approach
to better
Requires: python(foo) = bar
would be to write a parser for *.dot format (which is pretty damn trivial, even
sed(1) is almost enough) to automagically generate better python module
dependencies.

And the verification could/should/would proceed by capturing reality
(i.e. overloading builtins.__import__ to log what modules are actually
loaded for comparison with python(foo) dependencies in *.rpm headers)

Any interest? If so, the starting point would be to find the 2-3 best
python dependency graphs and have a go with sed(1) on the generated
*.dot spewage to see how good the dependencies being found actually are.

JMHO, dunno if I have time to look at python(…) reps for a couple of months ...

73 de Jeff


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Listing hard dependencies of a package

[Mark Hatle]

On 12/22/11 10:22 AM, Jeffrey Johnson wrote:


On Dec 22, 2011, at 10:40 AM, Mark Hatle wrote:


On 12/22/11 7:56 AM, Jeffrey Johnson wrote:


On Dec 22, 2011, at 7:24 AM, Paul Eggleton wrote:


Hi there,

Whilst working on Poky/OpenEmbedded I found the need to list all of the hard
dependencies of an RPM package within a shell script. Unfortunately --requires
or the REQUIRES tag seem to also include soft dependencies (i.e.
suggests) which is not really what I need. FYI we're using RPM 5.4.0.



Yes: Unlike Debian, Suggests: is an attribute, not a separate tag, that
modifies how a Requires: assertion is to be interpreted. An unsatisfied
Requires: marked with RPMSENSE_MISSINGOK will ignore failed dependencies.


After a little digging I found REQUIREFLAGS, here's an example listing from
our system:



Good (though working with shell processing --qf output is gonna be AWKward ;-)

SHort answer:
Use the --qf output as below but filter with awk to drop any Requires:
that has RPMSENSE_MISSINGOK (1   19) set in REQUIREFLAGS.


 snip 
paul@helios:~/poky/poky/build$ 
/home/paul/poky/poky/build/tmp/sysroots/i686-linux/usr/bin/rpm --dbpath /var/lib/rpm 
--root=/home/paul/poky/poky/build/tmp/work/qemux86-poky-linux/core-image-sato-1.0-r0/rootfs
 -q --qf [%{REQUIRENAME} %{REQUIREFLAGS}\n] connman-gnome
libglib-2.0-0 12
hicolor-icon-theme 0
libdbus-glib-1-2 12
libgtk-2.0 12
libc6 12
connman-plugin-loopback 524288
connman 524288
python 524288
python-dbus 524288
connman-plugin-dnsproxy 524288
connman-plugin-bluetooth 524288
connman-plugin-udhcp 524288
connman-plugin-ofono 524288
connman-plugin-wifi 524288
connman-plugin-ethernet 524288
connman-plugin-fake 524288
/bin/sh 1280
/bin/sh 4352
rtld(GNU_HASH) 16384
libgtk-x11-2.0.so.0 16384
libdbus-glib-1.so.2 16384
libc.so.6(GLIBC_2.0) 16384
libc.so.6(GLIBC_2.2) 16384
libgdk-x11-2.0.so.0 16384
libgobject-2.0.so.0 16384
libpthread.so.0 16384
libglib-2.0.so.0 16384
libc.so.6 16384
 snip 



Yes, REQUIREFLAGS here is a bit mask displayed as a decimal integer
which is difficult to filter with a *RE expression.

The most important bits are these (from rpmdb/rpmevr.h)
 RPMSENSE_LESS  = (1   1),
 RPMSENSE_GREATER   = (1   2),
 RPMSENSE_EQUAL = (1   3),
so a mask of 0x0e needs to be applied to REQUIREFLAGS.


These values are not ideal for filtering via a shell script and I don't know
what the values mean. Mark Hatle pointed me to its fflags modifier which does
appear to translate some of the flags but not for the suggests (e.g.
connman-plugin-* above), here's what it says for the same package:



Here is the Suggests: flags
 RPMSENSE_MISSINGOK  = (1   19),/*!   suggests/enhances hint. */

The other flags are context markers for how/why the dependency was added
and should not be used or relied upon. Note that there are no more available
bits in the int32_t so any additional need is going to require changes
to the existing definitions to recycle (and perhaps redefine) the bit
meanings.


jbj, would it be reasonable to add a context format specific to the dependency flags?  
Similar to the fflags so we get a pretty print out of the flag values?



Well there are several implementations already and Per Oyvind's patch
is entirely acceptable (because its an add-on and did not change existing 
production behavior).

My comments weren't negative: I was merely suggesting alternatives by pointing 
out
1) a --nosuggests modifier to all dependency formatting is arguably 
more general.
2) a popt alias is dirt simple to configure as needed (like --last)

There's also already XML/YAML/JSON formatting that can be used per-tag,
and there is a RPN calculator that can be used for simple arithmetic/logical 
transforms
(one of 4 RPM calculators in RPM+POPT ;-) that likely could achieve the 
necessary
test of (1  19) in a narrow --query display context easily.

(aside)
Making the RPN calculator generally useful will
predictably lead to in-fix notation RFE's, sigh.
Its all about Have it your own way! aesthetics these days, and
Consensus is futile.


(asking within the context of rpm5 development... not the oe-core work)

When I was trying to figure out if one already existed, it looks like it's 
fairly simple to add additional rendering formats.  (I or someone else could 
probably whip up something fairly quickly to render the flags..)



Yep: see lots of examples in /usr/lib/rpm/qf/* … note that if you are
going to attempt to use better spewage in --query output displays, you likely
want to pick up some of Anders Bjorkland's refactoring (from last May iirc) of 
--qf
that isn't in rpm-5.4.0. Its just a patch to (mostly) rpmdb/hdrfmt.c if you
need to retrofit.


Good to know.  I'm intending to uprev to newer rpm5 come the new year.. I've 
just been too busy to do it so far.


I think the popt solution is likely more then adequate for now.. but I'll look

Simple? query question

[Mark Hatle]

I'm working on multilib/multiarch support in the oe-core stuff.  And I've hit an
issue.  I'm trying to query for a package of a specific arch.

rpm -q zlib

returns:

zlib-1.2.5-r0.x86_64

if I do:

rpm -q zlib-1.2.5 or zlib-1.2.5-r0 or zlib-1.2.5-r0.x86_64

I get the same result..

however, I thought I could do:

rpm -q zlib.x86_64

and it would also work, but that doesn't seem to be the case.  Is there any way
to do rpm -q zlib.x86_64 -- or do I need to do rpm -q zlib, and through is some
grep commands on the output?

(The current case only has one instance of zlib, but eventually there could be 2
or 3 instances.)

Thanks!

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Simple? query question

[Mark Hatle]

On 6/30/11 11:37 AM, Jeff Johnson wrote:
 
 On Jun 30, 2011, at 12:29 PM, Mark Hatle wrote:
 
 I'm working on multilib/multiarch support in the oe-core stuff.  And I've 
 hit an
 issue.  I'm trying to query for a package of a specific arch.

 rpm -q zlib

 returns:

 zlib-1.2.5-r0.x86_64

 if I do:

 rpm -q zlib-1.2.5 or zlib-1.2.5-r0 or zlib-1.2.5-r0.x86_64

 I get the same result..

 however, I thought I could do:

 rpm -q zlib.x86_64

 
 That's a bug (and likely related to Distepoch: turn on
 but might be RPM ACID related, dunno)
 
 Meanwhile there's lots of ways to achieve that same query:
 
   rpm -q zlib | grep x86_64
   rpm -q '^zlib.*x86_64$'
   rpm -qa zlib 'arch=x86_64'

Thanks, it looks like I can use the -qa one easily enough.

 and it would also work, but that doesn't seem to be the case.  Is there any 
 way
 to do rpm -q zlib.x86_64 -- or do I need to do rpm -q zlib, and through is 
 some
 grep commands on the output?

 
 DO what works.
 
 FYI: The
   rpm -q N.A
 form of query is likely doomed in RPM. It was an expedient hack
 when multilib was first deployed: I was given 4 hours to
 devise, implement, test, and deliver a solution with mad manglers
 watching my every key stroke.
 
 (The current case only has one instance of zlib, but eventually there could 
 be 2
 or 3 instances.)

 
 Lemme see if I can fix this evening … tomorrow is 7/1 monthly release time.
 No promises.

Thanks!

BTW what I'm doing in the code is simply trying to find a package out of my list
of available packages to build the beginning of a manifest.. I may simply be
doing this a poor way right now, but I'm trying to hack in support to get a
proof of concept for multiarch RPM..

--Mark

 73 de Jeff
 
 Thanks!

 --Mark
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Yocto, directory ownership, RPM deb packaging

[Mark Hatle]

On 6/10/11 6:33 PM, Jeff Johnson wrote:
 
 On Jun 10, 2011, at 4:39 PM, Mark Hatle wrote:
 
 I've been trying to reconcile the behavior between rpm and deb packages in 
 the
 way their directories are handled.

 In rpm, we can set the directories to own in various ways.  If there is a
 conflict between two packages owning the same directory -- first-in wins and
 that's where the permissions/owner/group are selected.  (correct me if I'm 
 wrong)

 
 That's likely (but based on ancient memories of a performance speed-up
 to avoid creating already existent directories ... not that mkdir is slow, 
 just
 that is what was originally implemented like 1997).
 
 But that explains first rather than last. I could look at the code: so can you
   rpm -Uvv --fsmdebug *.rpm
 should spew every system call performed.
 
 
 In deb, directories are not owned by any package, they're just created as an
 artifact of the package installation.  All of the packages in the system are
 expected to have directories with the same permissions, owners, group or the
 package is broken.  I assume this means a similar first-in wins strategy as 
 well.

 Is there a way in RPM to change this strategy to create a true conflict 
 (install
 failure) to identify these situations?

 
 There's no configurable way ... but changing opens a walloping amount
 of pain because RPM will *again* have to justify its reliability if
 change occurs.

I was wondering because we could optionally enable it as a form of QA.
Otherwise we'll have to dump the data from the RPM packages and externally run 
QA...

 I plain and simply don't give a hoot unless a sound
 engineering reason is given for the change.
 
 Change for the sake of change isn't a sound reason, nor is consistency with
 the Debian Borg mind meld, nor is introducing Yet Another Way That RPM
 Doesn't Install Anything because of lack of vendor QA.
 
 Also, what is the impact to RPM if a lot of packages all own the same
 directories (such as /bin, /lib, /usr, etc..)?  [or alternatively have no
 packages own the directories and fix the owner/group/mode with some type of
 scripting]
 
 Lemme answer a question with a question:
 
   Why are you asking? Are you seeing impact or just enquiring?

More inquiring at this point...  This all stems from two issues that I have to
solve.  The first is pragmatic, we had a package that was incorrectly creating
the /var/lib portion of a directory as 0700, it just happened it was being
installed before the base-files package (which normally creates those
directories as 0755...)  It caused a number of minor problems that took a while
to identify.

In the Debian world the folks I asked basically said this doesn't happen.. I
asked how they were sure and they spouted some stuff that IMHO really means
cause we treat it as a bug and fix it, well duh..

In the RPM world, I would have solved the issue different, I would have owned
the directory and files within the /var/lib, but not /var/lib itself -- that
would have been owned by the base-files package

 Owning all directories in every package so that
   rpm -qa | wc -l
 and
   rpm -qf / | wc -l
 are identical is what I think should be done on even days of the month.

Currently I'm leaning toward having to have all packages own all files and
directories, in order to match the deb behavior -- I don't see a reason why
this isn't possible, just not what I'm used to doing.

My concern with this approach was performance, dependency issues, etc.. (I'm not
sure the ramifications of a package owning all of it's directories, as I've
never implemented a system that was meant to be that way.)

 On odd days of the month, I tend towards the schizoid alterantive POV, that
 no directory should be in packages.
 
 And on leap days I use dpkg ...

;)  I try to avoid it, unless I don't have a choice..


 Any help understanding RPM's behavior in these conditions performance or
 otherwise will help me figure out how to reconcile the behaviors.. thanks!

 
 Everything that rpm does while installing is displayable with
 
   rpm -Uvv --fsmdebug *.rpm
 
 Any quirks aren't from the state machine, but rather from foolish
 fiddle ups for unowned directory behavior w SELinux and other insanities.

Ya, I figured RPM isn't at fault... I just need a way to reconcile the behavior
of RPM, dpkg, opkg and the OpenEmbedded build environment.  This is new
territory for me.

My current thinking is to add some QA and helper routines into OpenEmbedded to
ensure that the core base set of directories are always correct and matching..
 The QA routines would somehow check the directories owned by all of the binary
packages and ensure they don't conflict...  (as well as don't collide with the
preset list from the helper)

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Fix linking error on Ubuntu 11.04-alpha

[Mark Hatle]

On 3/4/11 2:54 AM, devzero2000 wrote:
 On Thu, Mar 3, 2011 at 9:09 PM, Mark Hatle mark.ha...@windriver.com wrote:
 On 3/3/11 2:00 PM, Jeff Johnson wrote:

 On Mar 3, 2011, at 2:54 PM, Mark Hatle wrote:

 On 3/3/11 1:44 PM, Jeff Johnson wrote:
 Hmmm ... I'l try, but I'll bet that both
 beecrypt and syck are external in this bug report.


 (See above, it's internal)

 If its internal, then BeeCrypt/syck are in -lrpmmisc and
 the fix needs to be spun slightly differently.

 My understanding is that rpmmisc has a reference to the libbeecrypt.so, but 
 it
 doesn't contain a copy of it.  It's this referencing that is breaking on 
 newer
 systems.  If you don't include both the reference and implementation when
 linking the final executable, newer versions of the linker will intentionally
 fail to link.  (There is an --as-needed flag that can be added to the link
 line.. this will revert the linker behavior to the old way of bringing in
 libraries as needed based on run-time references.)

 Note that the patch has been put in and taken out
 a couple times already. RSE's intent was
 to strip all external library linkages and
 then use -lrpmmisc as a staging area for
 internal - external.

 But s modern tool chains start to try to mininimize
 library linkages, well, it gets tricky to keep everyone
 happy.

 I'll either apply the quick-and-dirty path again or try
 to figger what isn't right. Its likely linker stoopidness
 with transitive dependencies not exposing unneeded symbols.
 
 Hi, perhaps useful but these links describe the ChangeInImplicitDSOLinking
 https://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking
 and UnderstandingDSOLinkChange
 https://fedoraproject.org/wiki/UnderstandingDSOLinkChange
 for FC13. I have to check if UBUNTU 11.04 have this change in binutils
 but Mandriva
 cooker DOESN'T have it (checked now). I have to check also RHEL6 last
 update, probably not.
 
 Is this the problem perhaps ?

Whats odd is that we don't appear to be having any problems on FC13 -- only
Ubuntu 11.04-alpha.

--Mark

 

 todo++.

 73 de jeff

 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Fix linking error on Ubuntu 11.04-alpha

[Mark Hatle]

On 3/4/11 7:16 AM, devzero2000 wrote:
 On Fri, Mar 4, 2011 at 2:15 PM, devzero2000 pinto.e...@gmail.com
 mailto:pinto.e...@gmail.com wrote:
 
 Just for security can you download foo{1,2,4}.c and compile.sh from
 
 https://github.com/yersinia/junkcode/tree/master/rpm/rpm5/understandingdsochangeFC13
 
 and execute compile.sh ? I have problem to upgrade  lucid to maverick now 
 .
 
 Typo.  download foo{1,2,3}.c and compile.sh

+ gcc -g -fPIC -c foo1.c foo2.c foo3.c
+ gcc -shared -o foo3.so foo3.o
+ gcc -shared -o foo2.so foo2.o foo3.so
+ gcc -o foo1 foo1.o foo2.so -Wl,--rpath-link=.
/usr/bin/ld: foo1.o: undefined reference to symbol 'foo'
/usr/bin/ld: note: 'foo' is defined in DSO ./foo3.so so try adding it to the
linker command line
./foo3.so: could not read symbols: Invalid operation
collect2: ld returned 1 exit status

That is FC13.. So it appears that it does have at least the Fedora version of
the as-needed.  I'll try to get the person who reported the problem to me on
Ubuntu to try the same test and see what result he gets.

--Mark

 
  
 
 
  --Mark
 
 
 
  todo++.
 
  73 de jeff
 
  
 __
  RPM Package Manager
 http://rpm5.org
  Developer Communication List
 rpm-devel@rpm5.org
 mailto:rpm-devel@rpm5.org
 
  __
  RPM Package Managerhttp://rpm5.org
  Developer Communication Listrpm-devel@rpm5.org
 mailto:rpm-devel@rpm5.org
 
 
 
 
 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Fix linking error on Ubuntu 11.04-alpha

[Mark Hatle]

From the Ubuntu 11.04-alpha machine:

 I get
 
 $ sh compile.sh
 foo1.o: In function `main':
 /tmp/understandingdsochangeFC13/foo1.c:7: undefined reference to `foo'
 collect2: ld returned 1 exit status

So it certainly looks like as-needed is involved.

--Mark

On 3/4/11 8:28 AM, devzero2000 wrote:
 
 
 On Fri, Mar 4, 2011 at 2:27 PM, Mark Hatle mark.ha...@windriver.com
 mailto:mark.ha...@windriver.com wrote:
 
 On 3/4/11 7:16 AM, devzero2000 wrote:
  On Fri, Mar 4, 2011 at 2:15 PM, devzero2000 pinto.e...@gmail.com
 mailto:pinto.e...@gmail.com
  mailto:pinto.e...@gmail.com mailto:pinto.e...@gmail.com wrote:
 
  Just for security can you download foo{1,2,4}.c and compile.sh from
 
 
 https://github.com/yersinia/junkcode/tree/master/rpm/rpm5/understandingdsochangeFC13
 
  and execute compile.sh ? I have problem to upgrade  lucid to 
 maverick
 now .
 
  Typo.  download foo{1,2,3}.c and compile.sh
 
 + gcc -g -fPIC -c foo1.c foo2.c foo3.c
 + gcc -shared -o foo3.so foo3.o
 + gcc -shared -o foo2.so foo2.o foo3.so
 + gcc -o foo1 foo1.o foo2.so -Wl,--rpath-link=.
 /usr/bin/ld: foo1.o: undefined reference to symbol 'foo'
 /usr/bin/ld: note: 'foo' is defined in DSO ./foo3.so so try adding it to 
 the
 linker command line
 ./foo3.so: could not read symbols: Invalid operation
 collect2: ld returned 1 exit status
 
 That is FC13.. So it appears that it does have at least the Fedora 
 version of
 the as-needed.  I'll try to get the person who reported the problem to 
 me on
 Ubuntu to try the same test and see what result he gets.
 
 Ubuntu 10.04, RHEL6 and Mandriva cooker don't have any problem. AIX i am 
 pretty
 sure have the same problem, dunno Solaris (but i will check also
 belenix/openindiana)
 
 --Mark
 
 
 
 
  
   --Mark
  
  
  
   todo++.
  
   73 de jeff
  
  
 __
   RPM Package Manager  
  http://rpm5.org
   Developer Communication List  
  rpm-devel@rpm5.org mailto:rpm-devel@rpm5.org
  mailto:rpm-devel@rpm5.org mailto:rpm-devel@rpm5.org
  
   
 __
   RPM Package Manager
 http://rpm5.org
   Developer Communication List  
  rpm-devel@rpm5.org mailto:rpm-devel@rpm5.org
  mailto:rpm-devel@rpm5.org mailto:rpm-devel@rpm5.org
  
  
  
 
 
 
 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Fix linking error on Ubuntu 11.04-alpha

[Mark Hatle]

The following patch was sent to the oe-core mailing list.  This fixes a problem
when RPM5 (5.4.0) is linked on an Ubuntu 11.04-alpha host.

--Mark

 Original Message 
Subject: [OE-core] [PATCHv4 6/6] rpm: Fix linking error encountered in  
rpm-native
Date: Thu, 3 Mar 2011 08:09:06 -0800
From: Khem Raj raj.k...@gmail.com
Reply-To: Patches and discussions about the oe-core layer
openembedded-c...@lists.openembedded.org
To: openembedded-c...@lists.openembedded.org

* This patch passes the correct LDFLAGS to account for
  additional dependencies of librpmio on libbeecrypt and libsyck
  and hence fixes the build error.

Signed-off-by: Khem Raj raj.k...@gmail.com
---
 .../rpm/rpm/rpm-tools-mtree-LDFLAGS.patch  |   24 
 meta/recipes-devtools/rpm/rpm_5.4.0.bb |3 +-
 2 files changed, 26 insertions(+), 1 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch

diff --git a/meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch
b/meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch
new file mode 100644
index 000..9e7bba0
--- /dev/null
+++ b/meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch
@@ -0,0 +1,24 @@
+Source: None
+Status: Needs to go Upstream
+Author: Khem Raj raj.k...@gmail.com
+Date: 2011-03-03
+Description:
+This patch fixes build issue found when compiling rpm-natibe on ubuntu
+11.04-alpha. Problem is that linker does not get correct set of dependent
+libraries when linking mtree binary which results in undefined symbols
+in librpmio.so. This patch adds correct flags to mtree_LDADD to account
+for rpmio deps
+
+Index: rpm-5.4.0/tools/Makefile.am
+===
+--- rpm-5.4.0.orig/tools/Makefile.am
 rpm-5.4.0/tools/Makefile.am
+@@ -169,7 +169,7 @@ grep_LDADD =   $(RPMIO_LDADD_COMMON)
+
+ mtree_SOURCES =   rpmmtree.c
+ mtree_LDFLAGS =   @LDFLAGS_STATIC@ $(LDFLAGS)
+-mtree_LDADD = $(RPM_LDADD_COMMON)
++mtree_LDADD = $(RPMIO_LDADD_COMMON)
+
+ rpmrepo_SOURCES = rpmrepo.c
+ rpmrepo_CPPFLAGS =$(AM_CPPFLAGS) -I$(top_srcdir)/scripts
-I$(top_builddir)/scripts $(CPPFLAGS)
diff --git a/meta/recipes-devtools/rpm/rpm_5.4.0.bb
b/meta/recipes-devtools/rpm/rpm_5.4.0.bb
index 0003f48..3704816 100644
--- a/meta/recipes-devtools/rpm/rpm_5.4.0.bb
+++ b/meta/recipes-devtools/rpm/rpm_5.4.0.bb
@@ -43,7 +43,7 @@ LICENSE = LGPL 2.1
 LIC_FILES_CHKSUM = file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1

 DEPENDS = bzip2 zlib python perl db openssl elfutils expat libpcre attr acl 
popt
-PR = r13
+PR = r14

 # rpm2cpio is a shell script, which is part of the rpm src.rpm.  It is needed
 # in order to extract the distribution SRPM into a format we can extract...
@@ -56,6 +56,7 @@ SRC_URI =
http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.0-0.20101229.src.rpm;ex
   file://rpm-showrc.patch \
   file://rpm-nofsync.patch \
   file://rpm-solvedb.patch \
+  file://rpm-tools-mtree-LDFLAGS.patch \
  

 #   file://hdraddorappend.patch \
-- 
1.7.4.1


___
Openembedded-core mailing list
openembedded-c...@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Fix linking error on Ubuntu 11.04-alpha

[Mark Hatle]

On 3/3/11 1:44 PM, Jeff Johnson wrote:
 Hmmm ... I'l try, but I'll bet that both
 beecrypt and syck are external in this bug report.

They shouldn't be.  The issue is something to do with stricter library link
checking on newer systems.  I.e. if you simply link to librpmio.so you get a
link error, because the symbols for the beecrypt and related used in librpmio.so
can't be resolved -- you have to include all of the components on the link line 
now.

 Which is kinda pointless since ...
 
 afaik: nothing but rpm uses beecrypt, and anything using syck is
 better off with libyaml.
 
 So its m00t whether internal - external (and the rpm tarball ships
 with both beecrypt/syck inside).

(See above, it's internal)

 But I WILL try to figger something.
 
 BTW, mtree is quite useful. Are you distributing in Poky/yacto?
 The alternative is of course to just include the *BSD mtree
 outside of RPM.

It's of course being built with Poky/Yocto -- I think we're also packaging it,
but I'm not sure about that.  (We more or less follow the same package breakup
as the rpm 5.4.0 .spec file...  So if it's packaged in there, then we're likely
packaging as well.)

--Mark

 73 de Jeff
 On Mar 3, 2011, at 2:28 PM, Mark Hatle wrote:
 
 The following patch was sent to the oe-core mailing list.  This fixes a 
 problem
 when RPM5 (5.4.0) is linked on an Ubuntu 11.04-alpha host.

 --Mark

  Original Message 
 Subject: [OE-core] [PATCHv4 6/6] rpm: Fix linking error encountered in   
 rpm-native
 Date: Thu, 3 Mar 2011 08:09:06 -0800
 From: Khem Raj raj.k...@gmail.com
 Reply-To: Patches and discussions about the oe-core layer
 openembedded-c...@lists.openembedded.org
 To: openembedded-c...@lists.openembedded.org

 * This patch passes the correct LDFLAGS to account for
  additional dependencies of librpmio on libbeecrypt and libsyck
  and hence fixes the build error.

 Signed-off-by: Khem Raj raj.k...@gmail.com
 ---
 .../rpm/rpm/rpm-tools-mtree-LDFLAGS.patch  |   24 
 
 meta/recipes-devtools/rpm/rpm_5.4.0.bb |3 +-
 2 files changed, 26 insertions(+), 1 deletions(-)
 create mode 100644 
 meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch

 diff --git a/meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch
 b/meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch
 new file mode 100644
 index 000..9e7bba0
 --- /dev/null
 +++ b/meta/recipes-devtools/rpm/rpm/rpm-tools-mtree-LDFLAGS.patch
 @@ -0,0 +1,24 @@
 +Source: None
 +Status: Needs to go Upstream
 +Author: Khem Raj raj.k...@gmail.com
 +Date: 2011-03-03
 +Description:
 +This patch fixes build issue found when compiling rpm-natibe on ubuntu
 +11.04-alpha. Problem is that linker does not get correct set of dependent
 +libraries when linking mtree binary which results in undefined symbols
 +in librpmio.so. This patch adds correct flags to mtree_LDADD to account
 +for rpmio deps
 +
 +Index: rpm-5.4.0/tools/Makefile.am
 +===
 +--- rpm-5.4.0.orig/tools/Makefile.am
  rpm-5.4.0/tools/Makefile.am
 +@@ -169,7 +169,7 @@ grep_LDADD =$(RPMIO_LDADD_COMMON)
 +
 + mtree_SOURCES =rpmmtree.c
 + mtree_LDFLAGS =@LDFLAGS_STATIC@ $(LDFLAGS)
 +-mtree_LDADD =  $(RPM_LDADD_COMMON)
 ++mtree_LDADD =  $(RPMIO_LDADD_COMMON)
 +
 + rpmrepo_SOURCES =  rpmrepo.c
 + rpmrepo_CPPFLAGS = $(AM_CPPFLAGS) -I$(top_srcdir)/scripts
 -I$(top_builddir)/scripts $(CPPFLAGS)
 diff --git a/meta/recipes-devtools/rpm/rpm_5.4.0.bb
 b/meta/recipes-devtools/rpm/rpm_5.4.0.bb
 index 0003f48..3704816 100644
 --- a/meta/recipes-devtools/rpm/rpm_5.4.0.bb
 +++ b/meta/recipes-devtools/rpm/rpm_5.4.0.bb
 @@ -43,7 +43,7 @@ LICENSE = LGPL 2.1
 LIC_FILES_CHKSUM = file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1

 DEPENDS = bzip2 zlib python perl db openssl elfutils expat libpcre attr acl 
 popt
 -PR = r13
 +PR = r14

 # rpm2cpio is a shell script, which is part of the rpm src.rpm.  It is needed
 # in order to extract the distribution SRPM into a format we can extract...
 @@ -56,6 +56,7 @@ SRC_URI =
 http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.0-0.20101229.src.rpm;ex
 file://rpm-showrc.patch \
 file://rpm-nofsync.patch \
 file://rpm-solvedb.patch \
 +   file://rpm-tools-mtree-LDFLAGS.patch \


 #   file://hdraddorappend.patch \
 -- 
 1.7.4.1


 ___
 Openembedded-core mailing list
 openembedded-c...@lists.openembedded.org
 http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org
 
 __
 RPM Package Managerhttp://rpm5.org

Re: Fix linking error on Ubuntu 11.04-alpha

[Mark Hatle]

On 3/3/11 2:00 PM, Jeff Johnson wrote:
 
 On Mar 3, 2011, at 2:54 PM, Mark Hatle wrote:
 
 On 3/3/11 1:44 PM, Jeff Johnson wrote:
 Hmmm ... I'l try, but I'll bet that both
 beecrypt and syck are external in this bug report.


 (See above, it's internal)
 
 If its internal, then BeeCrypt/syck are in -lrpmmisc and
 the fix needs to be spun slightly differently.

My understanding is that rpmmisc has a reference to the libbeecrypt.so, but it
doesn't contain a copy of it.  It's this referencing that is breaking on newer
systems.  If you don't include both the reference and implementation when
linking the final executable, newer versions of the linker will intentionally
fail to link.  (There is an --as-needed flag that can be added to the link
line.. this will revert the linker behavior to the old way of bringing in
libraries as needed based on run-time references.)

 Note that the patch has been put in and taken out
 a couple times already. RSE's intent was
 to strip all external library linkages and
 then use -lrpmmisc as a staging area for
 internal - external.
 
 But s modern tool chains start to try to mininimize
 library linkages, well, it gets tricky to keep everyone
 happy.
 
 I'll either apply the quick-and-dirty path again or try
 to figger what isn't right. Its likely linker stoopidness
 with transitive dependencies not exposing unneeded symbols.
 
 todo++.
 
 73 de jeff
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: librpmio memory allocation issue

[Mark Hatle]

On 3/1/11 1:27 PM, Jeff Johnson wrote:
 
 On Mar 1, 2011, at 2:20 PM, Mark Hatle wrote:
 
 I think jbj might have figured out the problem.  Zypper is not using pcre, 
 but
 is instead causing mire to use the system regex.

 I made some local changes in an attempt to point the #include of regex.h in 
 mire
 with a #error.  I'm in the process of building both RPM and Zypper now to 
 see if
 this causes a build failure -- if it does, we need to figure out a Zypper
 solution because it's bringing in mire.h.

 
 If you can show me the Zypper code that's trying to use miRE, I can
 likely suggest a clean fix.

I am not finding any direct inclusions of miRE within Zypper or libzypper.
However, I am finding an inclusion of regex.h within libzypper.

I am still trying to track down the call order, but if I had to guess.. zypper
is calling into RPM with a regex object, which is getting fed to mire
(internally) and because the sizes are different we're getting a failure of some
type.

--Mark

 The traditional use of mire was tied to
   rpm -qa foo*
 as a selection filter on packages returned from a sequential
 iteration through Packages.
 
 The problem there is that *its Pig Slow* to load every
 installed header just to apply a pattern to NVRA.
 Its equally opig slow on @rpm.org or @rpm5.org code
 (though @rpm5.org will be faster than @rpm.org for other reasons ;-)
 
 But _NOT_ doing
   #include mire.h
 and once again _NOT_ installing
   /usr/include/rpm/mire.h
 (and thereby implicitly exporting a very messy API issue)
 is The Right Thing To Do.
 
 Anything that mire.h can also be done in zypper without
 a whole lotta work.
 
 Can you snip out and send along the zypper usage case for mire.h please?
 
 73 de Jeff
 --Mark

 On 3/1/11 1:15 PM, pinto.e...@gmail.com wrote:
 Hi, i have checked mire.c and there is none difference from 5.3 and 5.4. So 
 perhaps a autofu issue is possible, difficult that i can do a 
 MandrivaUpdate without a problem on cokker with a such apparently evident 
 problem with 5.3.6. Can you post your configure invocation, configure log. 
 Regards
 -Original Message-
 From: Paul Eggleton
 Sent:  01/03/2011, 17:04 
 To: rpm-devel@rpm5.org
 Subject: librpmio memory allocation issue


 Hi there,

 In Poky we're currently seeing a crash of zypper search in conjunction 
 with 
 rpm 5.4.0 [1]. Using valgrind I tracked the issue down to rpmio/mire.c line 
 361:

 mire-preg = xcalloc(1, sizeof(*mire-preg));

 If I hack this line to specify 64 as the size (the expected sizeof(regex_t) 
 for x86_64, as opposed to 24 reported by valgrind) then the crash 
 disappears 
 and valgrind stops reporting invalid memory accesses.

 I don't have much knowledge of the rpm codebase, but a bit of header 
 grepping 
 shows me that libpcre's pcreposix.h has a regex_t which differs quite 
 considerably from regex_t in regex.h (and matches the smaller size reported 
 by 
 valgrind), and therefore I strongly suspect that the culprit is that pcre's 
 regex_t is being used when allocating the struct in mire.c which is then 
 passed to regcomp. FWIW we are enabling pcre support at configure time.

 I could hack this to work, but since we may have dueling headers here the 
 solution might not be trivial. Any suggestions?

 Thanks,
 Paul

 [1] http://bugzilla.pokylinux.org/show_bug.cgi?id=721


 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: prelinked binaries failing validation

[Mark Hatle]

On 2/24/11 10:41 PM, Jeff Johnson wrote:
 
 On Feb 24, 2011, at 11:14 PM, Mark Hatle wrote:
 
 I'm trying to validate prelinked binaries and I'm getting a somewhat strange 
 result:

 /bin
 S.?./bin/bash
 /bin/bashbug

 The macro setting being used is:

 %__prelink_undo_cmd /usr/sbin/prelink prelink -y library

 I'm not sure what the '?' indicates, but I suspect it's some type of a 
 failure
 to get the information from the prelinker.

 
 Hmmm ... I'm pretty sure prelink affects only libraries, not executables.
 It has been years ...

Prelink affects any ELF executable.  But it doesn't do much good on executables
unless the libraries are already prelinked...  All the prelinker does is
predefine the ELF relocations so that ld.so doesn't need to compute them at load
time.  (ld.so does however still have a check the relocations are valid, if
they're not then it falls back to the old way of computing them all at load 
time.)

 I have a strong suspicion the problem is actually on the prelink side -- but 
 I'm
 not sure how to debug what RPM is doing that is ending up with the '?'.

 
 So first question -- what does the '?' indicate?  Second, is there any 
 debugging
 available that can help me understand what type of output the prelinker may 
 be
 sending back to RPM so I can diagnose what may be wrong there?

 
 ? == indeterminate like 700 root.root read by non-root.

I'm wondering if the pipe closed early (i.e. prelink didn't actually return any
data..)

 rpm is just trying to verify a digest on plaintext.
 
 Usually its just the content as plaintext.
 
 But prelink alters the content, so the digest has to be run off a pipe
 to prelink undo.
 
 prelink has builtin sha1/md5 options: try those.

I have run those, and I'm getting the results that I believe are correct.   This
is why I'm not sure whats going wrong -- but I still suspect it's on the prelink
side of thins.

 And you can use /usr/bin/{md5sum.sha1sum,sha256sum} with prelink undo
 to duplicate exactly what rpm should be doing.
 
 Extract the digest and compare with the manual results.
 
 I hope you're not putting prelinked content in a *.rpm. That has
 a really obscure failure mode because rpmbuild assumes unprelinked
 plaintext which prelink undo tries to restore. If the content
 was already prelinked, then the digest will never match.

Nope, the prelinker is run after the filesystem is constructed...

--Mark

 But that doesn't sound like your problem, and its libraries, not executables,
 affected by prelink I'm almost certain (run readelf -a to tell whether 
 prelinked
 or not. compare with the check implemented).
 
 hth
 
 73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: prelinked binaries failing validation

[Mark Hatle]

On 2/25/11 8:40 AM, Mark Hatle wrote:
 On 2/24/11 10:41 PM, Jeff Johnson wrote:

 On Feb 24, 2011, at 11:14 PM, Mark Hatle wrote:

 I'm trying to validate prelinked binaries and I'm getting a somewhat 
 strange result:

 /bin
 S.?./bin/bash
 /bin/bashbug

Ok, I've done more digging.  If I run this within gdb, I'm suddenly seeing an
error message appear on the screen:

prelink: Couldn't write file to standard output: Broken pipe
S.?. filename

However, if I use gdb and step through the fork/exec in rpmdb/legacy.c at around
like 128..  I don't get the error message -- and it comes back with a . for a
valid md5um.  (If I run this without GDB, I get the '?' and no error message
from prelink.)

So it looks like we have some type of pipe issue.

Any suggestions?

 The macro setting being used is:

 %__prelink_undo_cmd /usr/sbin/prelink prelink -y library

 I'm not sure what the '?' indicates, but I suspect it's some type of a 
 failure
 to get the information from the prelinker.


 Hmmm ... I'm pretty sure prelink affects only libraries, not executables.
 It has been years ...
 
 Prelink affects any ELF executable.  But it doesn't do much good on 
 executables
 unless the libraries are already prelinked...  All the prelinker does is
 predefine the ELF relocations so that ld.so doesn't need to compute them at 
 load
 time.  (ld.so does however still have a check the relocations are valid, if
 they're not then it falls back to the old way of computing them all at load 
 time.)
 
 I have a strong suspicion the problem is actually on the prelink side -- 
 but I'm
 not sure how to debug what RPM is doing that is ending up with the '?'.


 So first question -- what does the '?' indicate?  Second, is there any 
 debugging
 available that can help me understand what type of output the prelinker may 
 be
 sending back to RPM so I can diagnose what may be wrong there?


 ? == indeterminate like 700 root.root read by non-root.
 
 I'm wondering if the pipe closed early (i.e. prelink didn't actually return 
 any
 data..)
 
 rpm is just trying to verify a digest on plaintext.

 Usually its just the content as plaintext.

 But prelink alters the content, so the digest has to be run off a pipe
 to prelink undo.

 prelink has builtin sha1/md5 options: try those.
 
 I have run those, and I'm getting the results that I believe are correct.   
 This
 is why I'm not sure whats going wrong -- but I still suspect it's on the 
 prelink
 side of thins.
 
 And you can use /usr/bin/{md5sum.sha1sum,sha256sum} with prelink undo
 to duplicate exactly what rpm should be doing.

 Extract the digest and compare with the manual results.

 I hope you're not putting prelinked content in a *.rpm. That has
 a really obscure failure mode because rpmbuild assumes unprelinked
 plaintext which prelink undo tries to restore. If the content
 was already prelinked, then the digest will never match.
 
 Nope, the prelinker is run after the filesystem is constructed...
 
 --Mark
 
 But that doesn't sound like your problem, and its libraries, not executables,
 affected by prelink I'm almost certain (run readelf -a to tell whether 
 prelinked
 or not. compare with the check implemented).

 hth

 73 de Jeff
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Platform scoring..

[Mark Hatle]

I'm getting a message:

warning: package rpm-5.4.0-r13.ppc603e is intended for a ppc603e-pc-linux-gnu
platform

In my /etc/rpm/platform file I have:

qemuppc-poky-linux-gnu
ppc603e-poky-linux-gnu
powerpc-poky-linux-gnu
noarch-poky-linux-gnu
any-poky-linux-gnu
all-poky-linux-gnu

the package was generated using --target ppc603e-poky-linux-gnu, but the
platform that got embedded into package is: ppc603e-pc-linux-gnu

I'm looking at the rpmPlatformScore function and I don't really understand how
it's supposed to work.  But I would have expected that ppc603e-pc-linux-gnu to
be compatible with ppc603e-poky-linux-gnu in my platform file.

Any suggestions as to what might be going wrong and how to correct it?  (and
yes, if I put ppc603e-pc-linux-gnu into the platform file, the warning goes 
away!)

Note: it says it's a warning but it's actually an error during install -- it
silently prevents you from performing the install.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Platform scoring..

[Mark Hatle]

On 2/24/11 3:39 PM, Jeff Johnson wrote:
 
 On Feb 24, 2011, at 4:18 PM, Mark Hatle wrote:
 
 I'm getting a message:

 warning: package rpm-5.4.0-r13.ppc603e is intended for a ppc603e-pc-linux-gnu
 platform

 In my /etc/rpm/platform file I have:

 qemuppc-poky-linux-gnu
 
 This one is a string.

So would:

qemuppc-poky-linux-gnu
qemuppc
ppc603e
powerpc
noarch
any
all

Be correct, or should I wild card these or?  Sorry, I'm a bit confused
(obviously) as to the format and expectations.

 ppc603e-poky-linux-gnu
 powerpc-poky-linux-gnu
 noarch-poky-linux-gnu
 any-poky-linux-gnu
 all-poky-linux-gnu

 
 these are patterns (but strings are subsets of patterns).
 
 the package was generated using --target ppc603e-poky-linux-gnu, but the
 platform that got embedded into package is: ppc603e-pc-linux-gnu

 I'm looking at the rpmPlatformScore function and I don't really understand 
 how
 it's supposed to work.  But I would have expected that ppc603e-pc-linux-gnu 
 to
 be compatible with ppc603e-poky-linux-gnu in my platform file.

 
 The score should end up being the line no. of the 1st pattern that
 matches from /etc/rpm/platform. So arrange your patterns in priority order.
 
 Meanwhile the whole basis for scoring was based on trying to
 track
   i386 - i486 - i586 - i686
 kernel/openssl optimized packages (which there are almost none),
 and to prevent n00by lusers from attempting (by returning a negative score)
 to install ppc on i386.
 
 Are those RPM training wheels still needed a decade later?
 
 Any suggestions as to what might be going wrong and how to correct it?  (and
 yes, if I put ppc603e-pc-linux-gnu into the platform file, the warning goes 
 away!)

 
 I never use anything but a single COU-VENDOR-OS line in /etc/rpm/platform
 (which is why the functionality never really gets tested by me).
 
 Adding --miredebug is the 1st step towards debugging. You should see
 a 1 (RPMMIRE_STRCMP) for the first line in /etc/rpm/platform, and 2
 (RPMMIRE_REGEX) for the remaining lines when the patterns are created.

-- mireRegexec(0x10017348, 0x1002e808[20]) rc -1 mode 1 ppc603e-pc-linux-gnu
-- mireRegexec(0x100173a0, 0x1002e808[20]) rc -1 mode 2 ppc603e-pc-linux-gnu
-- mireRegexec(0x100173f8, 0x1002e808[20]) rc 0 mode 2 ppc603e-pc-linux-gnu

I assume this is the debug.. it didn't match the first line, so it went to the
second, didn't match.. then the third and hit with the rc 0.

A different package with the arch of qemuppc:

-- mireRegexec(0x10017348, 0x1002fd00[20]) rc -1 mode 1 qemuppc-pc-linux-gnu
-- mireRegexec(0x100173a0, 0x1002fd00[20]) rc 0 mode 2 qemuppc-pc-linux-gnu

Again same assumption, only it didn't hit the qemuppc-poky-linux-gnu, but did
the following line.

 You should also be able to read the results of all pattern matching.
 
 But this code breaks every 6-9 months regular as clockwork largely
 because everyone wants RPMTAG_ARCH compatibility and that code
 was removed in rpm-4.4.7 and I'm NOT going back there to trapping
 SIGILL to tell a ppc64 with faulty flash is running (because of setarch)
 in ppc32 mode. Life's too short for SIGILL ick.

Ya, I disabled all of the SIGILL PPC stuff.. it's completely the wrong way to do
it.. (I understand the legacy reasons why it was done, but for me it'll never
trigger.)

 Note: it says it's a warning but it's actually an error during install -- it
 silently prevents you from performing the install.

 
 Change the error to a warning if you wish. I personally see no reason why RPM
 needs to protect lusers from themselves to the extent that has always
 been implemented in RPM.

Thats the issue.  It looks like a warning, but RPM -is- protecting the user by
not installing.. the behavior I want is for RPM to warn (if it's concerned) but
do the install.

 The world of emulation and cross-compilation was NOT what was anticipated
 when RPM started to use uname(2) to determine arch in 1997.
 
 And the flaws in RPM (and the right answer, to use /proc/cpuinfo attributes 
 instead)
 were pointed out by Alan Cox in 1998.
 
 And so it goes ... but I'll dig out the issue for you if you need. I'll
 need the package and /etc/rpm/platform contents and the --target
 CLI option (2 or 3 you've already given me) in order to
 reproduce.

Sure...  The current /etc/rpm/platform I have is:

qemuppc-poky-linux-gnu
qemuppc
ppc603e
powerpc
noarch
any
all

The command line for rpmbuild is:

rpmbuild --nodeps --short-circuit
 --target ${pkgarch}-poky-linux-gnu
 --buildroot ${pkgd}
 --define '_topdir ${workdir}' --define '_rpmdir ${pkgwritedir}'
 --define '_build_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'
 --define '_use_internal_dependency_generator 0'
 --define '__find_requires ${outdepends}'
 --define '__find_provides ${outprovides}'
 --define '_unpackaged_files_terminate_build 0'
 --define 'debug_package %{nil}'
 -bb ${outspecfile}

(If you are wondering debug_packages are generated as part of the spec file --
so auto generation is disabled...  the find_requires/provides

prelinked binaries failing validation

[Mark Hatle]

I'm trying to validate prelinked binaries and I'm getting a somewhat strange 
result:

/bin
S.?./bin/bash
/bin/bashbug

The macro setting being used is:

%__prelink_undo_cmd /usr/sbin/prelink prelink -y library

I'm not sure what the '?' indicates, but I suspect it's some type of a failure
to get the information from the prelinker.

I have a strong suspicion the problem is actually on the prelink side -- but I'm
not sure how to debug what RPM is doing that is ending up with the '?'.

So first question -- what does the '?' indicate?  Second, is there any debugging
available that can help me understand what type of output the prelinker may be
sending back to RPM so I can diagnose what may be wrong there?

Thanks!
--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Need help with rpmdsRpmlib

[Mark Hatle]

On 2/16/11 11:22 AM, Jeff Johnson wrote:
 
 On Feb 15, 2011, at 7:18 PM, Mark Hatle wrote:
 
 I'm currently using rpm-5.4.0, and I've encountered a few issues.. however
 during this I'm trying to diagnose loading of certain files using --showrc.  
 But
 when I do, I get a segfault.

 Looking at valgrind output I get:

 ==3492== Invalid read of size 8
 ==3492==at 0x53E862A: rpmioGetPool (rpmmalloc.c:223)
 ==3492==by 0x4E73A9B: rpmdsGetPool (rpmds.c:196)
 ==3492==by 0x4E76209: rpmdsMerge (rpmds.c:950)
 ==3492==by 0x4E782B2: rpmdsRpmlib (rpmds.c:1567)
 ==3492==by 0x4E88966: rpmShowRC (rpmrc.c:1243)


Ahh I saw that below as well, but figured they were an artifact of the above.  I
didn't realize it was an additional problem.

 There's one more peice that needs doing on rpm -v --showrc
 
 ==32637== 1 errors in context 1 of 2:
 ==32637== Invalid free() / delete / delete[]
 ==32637==at 0x423A471: rpmioFreePoolItem (rpmmalloc.c:201)
 ==32637==by 0x417DD48: mireFreeAll (mire.c:102)
 ==32637==by 0x409566F: rpmFreeRpmrc (rpmrc.c:1005)
 ==32637==by 0x406EB65: rpmcliFini (poptALL.c:550)
 ==32637==by 0x406EF8C: rpmcliAllArgCallback (poptALL.c:302)
 ==32637==by 0xD49AC0: invokeCallbacksOPTION (in /lib/libpopt.so.0.0.0)
 ==32637==by 0xD49B1A: invokeCallbacksOPTION (in /lib/libpopt.so.0.0.0)
 ==32637==by 0xD4B3EC: poptGetNextOpt (in /lib/libpopt.so.0.0.0)
 ==32637==by 0x406F582: rpmcliInit (poptALL.c:766)
 ==32637==by 0x804AC75: main (rpmqv.c:385)
 ==32637==  Address 0x444a53c is 12 bytes inside a block of size 176 alloc'd
 ==32637==at 0x4005F94: realloc (vg_replace_malloc.c:525)
 ==32637==by 0x417DBB5: mireAppend (mire.c:503)
 ==32637==by 0x4096ABB: defaultMachine (rpmrc.c:469)
 ==32637==by 0x4096ED1: rpmSetMachine (rpmrc.c:836)
 ==32637==by 0x409701E: rpmRebuildTargetVars.clone.0 (rpmrc.c:920)
 ==32637==by 0x4097532: rpmReadConfigFiles (rpmrc.c:1108)
 ==32637==by 0x406EC9B: rpmcliConfigured (poptALL.c:196)
 ==32637==by 0x406EF74: rpmcliAllArgCallback (poptALL.c:300)
 ==32637==by 0xD49AC0: invokeCallbacksOPTION (in /lib/libpopt.so.0.0.0)
 ==32637==by 0xD49B1A: invokeCallbacksOPTION (in /lib/libpopt.so.0.0.0)
 ==32637==by 0xD4B3EC: poptGetNextOpt (in /lib/libpopt.so.0.0.0)
 ==32637==by 0x406F582: rpmcliInit (poptALL.c:766)
 
 The 12 bytes is (my guess) the size of the 3 pointers carried by all rpm 
 objects
 that needs to be fiddled in somewhere. There's some subtleties with miRE
 reference counts shared between the array and the array elements that are
 the root cause. There are only 2 obscure code paths affected: here with
 rpm -v --showrc and rpm -qa 'arch=i[3456]86' pattern queries. Might
 be time to rip out both functionalities in favor of the new code
 in RPM ACID that permits patterns as arguments like
   rpm -e '^python-.*'
 Note that the above is implemented but largely untested. WORKSFORME is hardly
 finished, I use RPM rather differently than most.
 
 I'll dig the rest of the fix out over the next few days.

So what I was working on was a situation where the automatic dep solver wasn't
finding packages.  It turns out that there were some compatible packages
problems going on.  The patch located at:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/recipes-devtools/rpm/rpm/rpm-platform.patch

fixes the issue for Poky.

Majority of it simply disables some hacked that were RPM_VENDOR_WINDRIVER,
around the isCompatible.  These might have to come back in the future -- or an
alternative to fix the multilib issues WR had (vs poky).. but I'm going to try
to avoid them in the future.

The remainder of the patch fixes a problem I found with the sysinfo loading...
and then the real issue I was having.

I needed a way to pass the path to the platform file while I was coming up with
an install solution.  This also cleaned up the platform file path in the same
was as sysinfo loading worked.

I do note that under the OPENPKG vendor, there was previously a value of
__platform defined for a similar purpose, but the name I choose
_rpmrc_platform_path seems to more closely match the sysinfo path.

Any comments?

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Need help with rpmdsRpmlib

[Mark Hatle]

On 2/16/11 12:21 PM, Jeff Johnson wrote:
 
 On Feb 16, 2011, at 12:35 PM, Mark Hatle wrote:
 

 I'll dig the rest of the fix out over the next few days.

 So what I was working on was a situation where the automatic dep solver 
 wasn't
 finding packages.  It turns out that there were some compatible packages
 problems going on.  The patch located at:

 http://git.pokylinux.org/cgit.cgi/poky/tree/meta/recipes-devtools/rpm/rpm/rpm-platform.patch

 fixes the issue for Poky.

 Majority of it simply disables some hacked that were RPM_VENDOR_WINDRIVER,
 around the isCompatible.  These might have to come back in the future -- 
 or an
 alternative to fix the multilib issues WR had (vs poky).. but I'm going to 
 try
 to avoid them in the future.

 
 Passing SYSCONFIGDIR as (possibly) an rpm macro %{_etcrpm}  and then using
 rpmExpand() instead of xstrdup() might flush some of the details from the
 code. Dunno if you can get the rpm macro passed from AutoFu - ... - C 
 defines
 straightforwardly.

hadn't considered that, but ya as long as _etcrpm is always defined somewhere
that should work, and reduce some of the complexity.

 The remainder of the patch fixes a problem I found with the sysinfo 
 loading...
 and then the real issue I was having.

 I needed a way to pass the path to the platform file while I was coming up 
 with
 an install solution.  This also cleaned up the platform file path in the same
 was as sysinfo loading worked.

 I do note that under the OPENPKG vendor, there was previously a value of
 __platform defined for a similar purpose, but the name I choose
 _rpmrc_platform_path seems to more closely match the sysinfo path.

 
 I do wish Poky and Mandriva and Openpkg could converge to a common solution
 for /etc/rpm/platform.
 
 Per Oyvind's libcpuinfo is essentially re-writing platform based on whether
 its elf32 or elf64, where scoring differs between.
 
 Poky (and likely OpenPKG I've forgot) need precise control for
 cross-platform images.
 
 If up to me, I'd add some macro magic madness to stop with the path
 rewrites, and then add an immediate %{load:...} where macros are
 already expanded in paths.
 
 The whole issue is bootstrapping: evaluating macros wile initializing macros
 requires a certain discipline and methodology.

We're not using PREMACROFILES.. I forgot to mention, in that patch I had to call
the PREMACROFILES loader so that the early default values (of things like
_etcrpm) were defined.. calling it with a NULL accomplished it when
PREMACROFILES was not defined.

 Also please note that PREMACROFILES just won't ever solve the bootstrapping
 issue. I just hven't gotten to pushing the attempts under
   #ifdef RPM_VENDOR_MANDRIVA
 on HEAD and on rpm-5_4 (I believe I ripped out PREMACROS in rpm-5_3)
 
 I tried with --predefine and --define years ago. Dinna work then, will
 not work now.

I'm using --predefine to specify the location of the sysinfo directory and the
platform file.  Works great, I just didn't know --predefine even existed before
this exercise!

 The whole issue is bootstrapping ... and it gets really nasty/ugly
 when bindings like rpm-python -- which also call rpmReadConfig() --
 are considered.

Ya, luckily I don't have to do that -- yet -- but I suspect with the
Zypper/sat-solver work I'm going to have to figure out how to pull that in..
(the platform file also needs to be wired into the sat-solver/zypper so it can
also determine which packages are available/allowed)...  We're working on that
for Poky...

--Mark

 hth
 
 73 de Jeff
 Any comments?

 --Mark
 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Need help with rpmdsRpmlib

[Mark Hatle]

I disabled the rpmdsFree in the previous chunk of code, and now showrc works as
expected.

In lib/rpmrc.c

@@ -1216,11 +1229,15 @@
if (DNEVR != NULL)
fprintf(fp, %s\n, DNEVR+2);
}
+#if 0
(void)rpmdsFree(ds);
ds = NULL;
+#endif
fprintf(fp, \n);
}
+#if 0
PRCO = rpmdsFreePRCO(PRCO);
+#endif
 }

 if (rpmIsVerbose()) {

 I'm currently using rpm-5.4.0, and I've encountered a few issues.. however
 during this I'm trying to diagnose loading of certain files using --showrc.  
 But
 when I do, I get a segfault.
 
 Looking at valgrind output I get:
 
 ==3492== Invalid read of size 8
 ==3492==at 0x53E862A: rpmioGetPool (rpmmalloc.c:223)
 ==3492==by 0x4E73A9B: rpmdsGetPool (rpmds.c:196)
 ==3492==by 0x4E76209: rpmdsMerge (rpmds.c:950)
 ==3492==by 0x4E782B2: rpmdsRpmlib (rpmds.c:1567)
 ==3492==by 0x4E88966: rpmShowRC (rpmrc.c:1243)
 
 And further items that seem to indicate I've got memory corruption.
 
 if (rpmIsVerbose()) {
 fprintf(fp, _(Features provided by rpmlib installer:\n));
 xx = rpmdsRpmlib(ds, NULL);

Within rpmdsRpmlib, it does a merge of ds  the new internal ds from the rpmlib
instantiation.  This is what is triggering the valgrind messages, but again I'm
at a loss to figure out where the real bug is.. but based on the above it seems
like it's in the merge code.

--Mark

 ds = rpmdsInit(ds);
 while (rpmdsNext(ds) = 0) {
 
 the rpmdsRpmlib is the function causing the issue.  Prior to this ds is freed,
 and I'm wondering if that is causing the problem.  But I'm not sure how to
 diagnose any of the memory pool functions or even if Merge is doing the right
 thing.  Any suggestions would be appreciated.
 
 --Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Need help with rpmdsRpmlib

[Mark Hatle]

The reproducer for me is simple.

Setup a /etc/rpm/platform with some entries (not sure if this matters), have an
/etc/rpm/sysinfo/Provides with a / as a provide.

Then run rpm -vv --showrc.

I get a double free and crash.. (which is why I started digging into it with
Valgrind.)

On 2/15/11 8:03 PM, Jeff Johnson wrote:
 
 On Feb 15, 2011, at 7:27 PM, Mark Hatle wrote:
 

 the rpmdsRpmlib is the function causing the issue.  Prior to this ds is 
 freed,
 and I'm wondering if that is causing the problem.  But I'm not sure how to
 diagnose any of the memory pool functions or even if Merge is doing the 
 right
 thing.  Any suggestions would be appreciated.
 
 There are programming restriction on how rpmdsMerge() is done as well.
 
 The core issue is that -- while iterating -- the loop
 index needs to be saved and restored.
 
 IOW: If you nest iterations using rpmds objects, you have
 to be careful about how its done, protecting the ds-i object.
 
 This is likely even more important if you are using rpmdsRpmlib
 in complex ways.
 
 The easy fox for merging in a single item is construct a
 carrier using rpmdsSingle() for that specific dependency
 before merging.

I can't call rpmdsNew() from within the rpmrc.c function, is there an altenative
way of doing this?  or is the rpmdspool corrupt by this point and someone how to
be restarted?

 I'd need more context to diagnose what precisely is happening.
 
 73 de Jeff
 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Need help with rpmdsRpmlib

[Mark Hatle]

I'm currently using rpm-5.4.0, and I've encountered a few issues.. however
during this I'm trying to diagnose loading of certain files using --showrc.  But
when I do, I get a segfault.

Looking at valgrind output I get:

==3492== Invalid read of size 8
==3492==at 0x53E862A: rpmioGetPool (rpmmalloc.c:223)
==3492==by 0x4E73A9B: rpmdsGetPool (rpmds.c:196)
==3492==by 0x4E76209: rpmdsMerge (rpmds.c:950)
==3492==by 0x4E782B2: rpmdsRpmlib (rpmds.c:1567)
==3492==by 0x4E88966: rpmShowRC (rpmrc.c:1243)

And further items that seem to indicate I've got memory corruption.

if (rpmIsVerbose()) {
fprintf(fp, _(Features provided by rpmlib installer:\n));
xx = rpmdsRpmlib(ds, NULL);
ds = rpmdsInit(ds);
while (rpmdsNext(ds) = 0) {

the rpmdsRpmlib is the function causing the issue.  Prior to this ds is freed,
and I'm wondering if that is causing the problem.  But I'm not sure how to
diagnose any of the memory pool functions or even if Merge is doing the right
thing.  Any suggestions would be appreciated.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

RPM 5.4.0

[Mark Hatle]

I've just started working with RPM 5.4.0.  Some of this strange things I've been
doing with RPM has me creating databases while never installing any files to the
disk.

Before I could use:

-D _dbi_tags_3 Packages:Name:Basenames:Providename:Nvra \

but now if I enable that RPM doesn't return any error messages but fails with a
non-zero return code.  Any suggestions?

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: What to do about RPM file conflicts?

[Mark Hatle]

I've always thought fileconflict mechanism that simply checked md5/sha1, mode,
owner and group would be adequate.  The creation time I would expect to be
different between packages.

In the case of compressed man pages, this seems like it would require special
processing to be able to deal with the same file, that was compressed slightly
differently.  I'm not sure if it makes sense to write a generic hook for this
(decompress  then compare), or if it's just a system packaging error.

--Mark

On 12/28/10 10:19 AM, Jeff Johnson wrote:
 Might as well open this up to wider discussion:
   What SHOULD be done with RPM file conflicts?
 
 
 (aside to Per Oyvind)
 
 Yes I agree there's no actual need.
 
 And the real fix is in XZ (if interested). This isn't
 the first time that file conflicts have shown up
 in multilib packaging (or with dueling man pages).
 
 The fix (sic) at the time was to invoke gzip -n
 during rpmbuild to remove the time stamp (and path)
 from the compressed man page. That was almost good enough too,
 except that some builds installed their own compressed
 man pages, and so man pages needed to be recompressed with -n.
 
 I'm almost certain (but have lost track) that XZ has the same issue,
 that there's a time stamp in *.xz/*.lzma that forces content differences.
 
 
 
 
 Begin forwarded message:
 
 From: Jeff Johnson n3...@mac.com
 Date: December 28, 2010 10:18:40 AM EST
 To: Eugeni Dodonov eug...@mandriva.com
 Subject: Re: [Cooker] Replaced files in *.rpm packages


 On Dec 28, 2010, at 6:48 AM, Eugeni Dodonov wrote:

 On 12/28/2010 08:11 AM, Jeff Johnson wrote:
 Hi --

 In order to figure out how to fix an issue with
 RPM, I need some input on Mandriva (or any other)
 system regarding replaced files.

 A replaced file is a path contained in more than
 one package with different content or stat(2) metadata.

 What I need is this output (before upgrading to rpm-5.3.6+):
rpm -qas | grep ^replaced | wc -l

 Private replies are likely preferred, the output is technically
 mind numbing and otherwise stultifying and utterly uninteresting.

 (aside)
 I'd also be interested in the actual paths if the output
 isn't huge.

 My hunch is that there aren't any replaced file states, and
 I need a confirmation before ripping out some ancient code.


 Hi,

 here are my stats:
 [eug...@eugeni-x86_64 09:44:23 ~] $ rpm -qas | grep ^replaced | wc -l
 8
 [eug...@eugeni-x86_64 09:44:38 ~] $ rpm -qas | grep ^replaced
 replaced  /usr/share/man/man3/ExtUtils::CBuilder.3pm.xz
 replaced
 /usr/share/man/man3/ExtUtils::CBuilder::Platform::Windows.3pm.xz
 replaced  /usr/share/man/man3/Compress::Raw::Zlib.3pm.xz
 replaced  /usr/share/man/man8/ifdown.8.xz
 replaced  /usr/share/man/man8/ifup.8.xz
 replaced  /usr/share/man/man3/crypt.3.xz
 replaced  /usr/share/man/man3/crypt_r.3.xz
 replaced  /usr/share/man/man1/gpg-zip.1.lzma


 Thanks.

 All the examples I've seen in Mandriva are in man pages.

 (aside)
 I can guess from the paths in Per Oyvind's filtering patch that there may be
 a few more cases.

 But that's enough data for me to ascertain what to do (likely rip-out some
 ancient code. Alternatively, there's an rpmdb access deadlock on this
 same code path).

 Which brings me to the next level
  How/when/where should file conflicts be detected?

 Any interest in expressing Mandriva Cooker needs? The two
 extreme view points on file conflicts are:

  1) rpm should _NEVER_ do anything unexpected, should always FULLSTOP
  when unexpected is detected (which has a certain cognitive dissonance).

  The problem here is that FULLSTOP creates a support issue, and
  it takes a bit to fix, build, distribute, install a new package,
  so the problem is usually reported multiple times, and often
  reappears.

  One solution here is to attempt a check while building. In the 
 case
  of file conflicts, 2 packages are independently constructed. 
 What
  could be done is to automate file conflict detection during 
 rpmbuild
  using a reference database (i.e. an rpmdb) of everything. The
  difficulty then is maintaing a proper reference database: any
  check will be only as good/bad/ugly as the reference database 
 is.

  2) rpm should do best effort and attempt some reasonable resolution, 
 possibly
  logging the problem case.

  The problem here is that noone will ever read the log, nor 
 understand
  and fix the problem.

 What's your Mandriva Cooker POV?

 73 de Jeff


 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer 

Re: [Fwd: [Rpm-maint] [RFC PATCH] install selinux policies from package header]

[Mark Hatle]

Jeff Johnson wrote:


On Jul 6, 2009, at 4:03 PM, Mark Hatle wrote:


I saw the forwarded come across the rpm-maint list.

My understanding is there is currently not any SE Linux integration in 
rpm5 correct?




The logic test for SELinux was inverted. So --nofcontexts enabled rather
than disabled.

Fixed a couple weeks back.

That's hardly not any ... integration.


Ahh then I had misunderstood, I was under the impression the SE Linux 
integration had gone stale and was not longer functional.


Is there any desire to include support for the %policy that they seem 
to be adding below?




All depends on what include support means.

I did -- in fact -- implement all of the SELinux within RPM packaging.


Two issues, the first is does rpm5 support the %policy that appears to be in 
rpm.org?  If not this could turn into a compatibility issue.. (but I assume it's 
just a new file attribute, and as such won't cause problems...)


(I agree incorporating selinux into RPM is dangerous..  but it would 
be nice to take the policy changes in some way that they can be 
processed as a batch, and avoid the whole %post thing.)




I did -- in fact -- implement all of the SELinux within RPM packaging,
working 14+ hours a day over Xmas without pay @redhat.com, and have
multiple times discussed what was desired for SELinux in RPM, usually
at my own expense. But I digress ...


Do you have any pointers to the code?  In the embedded / cross-install 
situation, we've got the problem where we can't actually initialize the SE Linux 
subsystem at install time, but have to do it as a separate task.  I'm curious if 
this would be possible.  (The %post scripts don't make this possible for us.)


--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [Fwd: [Rpm-maint] [RFC PATCH] install selinux policies from package header]

[Mark Hatle]

Jeff Johnson wrote:

I did -- in fact -- implement all of the SELinux within RPM packaging,
working 14+ hours a day over Xmas without pay @redhat.com, and have
multiple times discussed what was desired for SELinux in RPM, usually
at my own expense. But I digress ...


Do you have any pointers to the code?  In the embedded / cross-install 
situation, we've got the problem where we can't actually initialize 
the SE Linux subsystem at install time, but have to do it as a 
separate task.  I'm curious if this would be possible.  (The %post 
scripts don't make this possible for us.)




Pointers to *what* code? I'd suggest asking on a SELinux list -- can't 
actually

initialize needs to be pinned down to something less vague.


The code within rpm5 that handles the special actions of the %policy flags.


I'm not sure what %post scripts don't make this possible means. Clearly
that has been the recommended approach to installing per-package policy 
for years.


In the cross install, I can't execute %post scripts, only lua..  So from a 
trying to inherit [and not reinvent the wheel] point of view, I have to change 
any spec files that using scripting to lua (or simply gut them...)



Whether the recommendation is wise or viable is anyone's guess since there
aren't any real world examples that I'm aware of.


:)  Ya, I'm just trying to avoid the future..  I'm comfortable right now.. 
change is scary.. /sarcasm


I'm just trying to get a basic grasp of the problem so when this becomes a 
problem, I have at least a basic understanding where to look.


Thanks!

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

[Fwd: [Rpm-maint] [RFC PATCH] install selinux policies from package header]

[Mark Hatle]

I saw the forwarded come across the rpm-maint list.

My understanding is there is currently not any SE Linux integration in rpm5 
correct?

Is there any desire to include support for the %policy that they seem to be 
adding below?


(I agree incorporating selinux into RPM is dangerous..  but it would be nice to 
take the policy changes in some way that they can be processed as a batch, and 
avoid the whole %post thing.)


--Mark

 Original Message 
Subject: [Rpm-maint] [RFC PATCH] install selinux policies from package header
Date: Mon, 6 Jul 2009 15:42:28 -0400
From: Stephen Lawrence slawre...@tresys.com
To: rpm-ma...@lists.rpm.org
CC: Eamon Walsh ewa...@tycho.nsa.gov, Joshua Brindle jbrin...@tresys.com

RPM currently has support for security policies to be stored in an rpm
header but it doesn't currently do anything with the policies. We'd like
to get some feedback on a prototype implementation that adds support for
using those policies in an SELinux environment.

First, a bit of background. SELinux policy is currently installed
through %post scripts. This presents several problems. First, this means
that policy for a given application may not be loaded at the time the
files are written to disk, preventing those files from being labeled
properly, because the symbols used to label files need to be in the
policy loaded into the kernel. Secondly, this means that if multiple
packages install policy, each of their %post scripts will reload the
policy, which is a very expensive operation. Consequently, policy is
generally kept in a single package to avoid this, despite containing
many application specific policy modules that would be more suited to be
included in their application package.

So, what we would like to do is to start including SELinux policy as
part of the rpm and have rpm install all policies together before files
start to hit the disk. To do this, we would like to use the already
supported %policy directive, which stores the policy in the archive
header.

We would then install the policy before pretrans. This policy load would
involve gathering all the policies to be installed from all packages,
writing them to a temporary location, and calling out to semodule to
install the SELinux policy modules.

Obviously I'm glossing over many implementation details that would need
to be worked out. The point of this email is strictly to get feedback on
our approach. Below is a patch that implements the beginnings of what I
describe above. Any and all feedback is appreciated.

Included in this patch is a spec file and sources for creating a test
rpm. When this rpm is installed, a policy module called poltest will be
loaded. To verify that the module is sucessfully installed, you can run

   # semodule -l | grep poltest

---
 lib/psm.c |   85 +
 lib/psm.h |6 ++-
 lib/rpmte.c   |   10 +++
 lib/rpmte_internal.h  |3 +
 lib/transaction.c |   83 
 tests/data/SOURCES/poltest-1.0.tar.bz2|  Bin 0 - 384 bytes
 tests/data/SOURCES/poltest-policy-1.0.tar.bz2 |  Bin 0 - 446 bytes
 tests/data/SPECS/poltest.spec |   36 +++
 8 files changed, 222 insertions(+), 1 deletions(-)

diff --git a/lib/psm.c b/lib/psm.c
index e552c6c..c8ba1c6 100644
--- a/lib/psm.c
+++ b/lib/psm.c
@@ -17,6 +17,7 @@
 #include rpm/argv.h

 #include rpmio/rpmlua.h
+#include rpmio/base64.h
 #include lib/cpio.h
 #include lib/fsm.h /* XXX CPIO_FOO/FSM_FOO constants */
 #include lib/psm.h
@@ -60,6 +61,8 @@ struct rpmpsm_s {
 pkgStage nstage;   /*! Next psm stage. */

 int nrefs; /*! Reference count. */
+
+ARGV_t policyFiles;	/*! Locations of temporary policy files on disk, 
extracted from this package */

 };

 int rpmVersionCompare(Header first, Header second)
@@ -869,6 +872,76 @@ exit:
 }

 /**
+ * Write policies from the header to temporary files and store the
+ * tmp file locations in the psm data
+ * @param psm  package state machine data
+ * @return rpmRC return code
+ */
+static rpmRC installPolicy(rpmpsm psm)
+{
+   rpmRC rc = RPMRC_OK;
+   Header h = NULL;
+   FD_t fd = NULL;
+   char *tmpFile = NULL;
+   struct rpmtd_s policies;
+   const char *data = NULL;
+   uint8_t *policy = NULL;
+   size_t policylen = 0;
+
+   rpmtdReset(policies);
+   h = rpmteHeader(psm-te);
+
+   if (h == NULL) {
+   rpmlog(RPMLOG_ERR, _(Failed to get header\n));
+   rc = RPMRC_FAIL;
+   goto exit;
+   } else if (!headerIsEntry(h, RPMTAG_POLICIES)) {
+   // header does not contain any policy
+   rc = RPMRC_OK;
+   goto exit;
+   } else if (!headerGet(h, RPMTAG_POLICIES, policies, HEADERGET_ALLOC)) {
+   

Re: A setup - filesystem loop because of parentdir ordering

[Mark Hatle]

We handle this by doing installs in the following order:
*) passwd/group (rpm2cpio)
*) filesystem/basesystem/setup
*) Everything else

This appeared to be the only safe way to setup the environment (even prior to us 
using RPM5.)


--Mark

Jeff Johnson wrote:
This issue was reported on #rpm, and is likely gonna be commonly 
encountered.


Traditionally, RPM dependency relations order packages on bootstrap 
installs like this:


setupcontains /etc/passwd and nothing else
basesystemno files, the version was supposed to be the distro 
release

filesystemcontaining the directory skeleton for the file systems.

But note that when parentdir relations are used for ordering (as in 
rpm-5.1.7+), that

setup implicitly acquires an ordering hint equivalent to
Requires: /etc
because setup contains the file /etc/passwd, which will depend on its 
parentdir.


Since /etc is contained in the filesystem package, the filesystem
package is ordered before setup. That's almost perfectly OK, but
the file system package also needs to look up users/groups, which
need /etc/passwd installed, so ideally setup (or whatever installs
/etc/passwd) must be first. OTOH, the filesystem package (or
whatever carries the directory skeleton) typically has root:root
ownership which is hardwired within rpmio/ugid.c cached lookups.

But if anything whatsoever in the filesystem package depends on
the setup package, then there's a loop, and rpmtsOrder() will delay
attempting to install order either setup - filesystem until its way
way too late. The failure symptom will include many obscure messages 
from lots of

other packages. The flaw is that setup/filesystem _MUST_ be installed
as early as possible.

One can either fix setup/filesystem packages to not loop, or
one can add dependency ordering whiteout to specify which
of setup or filesystem should be installed first.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Implementing EVR comparisons using *RE's ?

[Mark Hatle]

While I don't have an answer for you, my concern with any change to rpmvercmp is 
around compatibility.. bugs become features


What I've never seen well documented is the actual algorithm that rpmvercmp uses 
for it's comparison (in a human readable form).  I attempted to write one once 
and I can dig it up if it will help... but I'm not even 100% sure it's really 
correct.


As for using RE, it should be possible to simplify the rpmvercmp and do it for 
same pattern upgrades, but when the pattern changes thats where we would need to 
fall back to rpmvercmp rules.  (note, I think it's important someone document 
the deb vercmp as well..  I know we had a discussion on this a few years back 
and they were ALMOST the same, but not exactly...)


--Mark

Jeff Johnson wrote:

While muddling through how to add rpmvercmp to JavaScript,
I find myself thinking Yet Again about how silly and feeble
and fragile and deficient rpmvercmp actually is.

So I ask the question:
Can *RE patterns that match newer but not older
packages be devised?

The general answer is boring: No, of course not.

But %track (and /usr/lib/rpm/vcheck.pl) are based
on *RE's and Get It Right! (i.e. detecting newer
upstream versions of tarballs) sufficiently often
that newer EVR comnparisons might be done with *RE's
rather than the usual slice-n-dice of alpha/digit/other
character sets and using strcmp(3).

Perhaps a better (as in easier to answer) question is
How many types of versioning pattern templates would
need to recognized for rpmvercmp to be done
using *RE's instead?

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Paths for per-interpreter initialization?

[Mark Hatle]

I like the idea of changing the locations to be loaded into macros.  This helps 
my relocatable RPM mechanism.. ;)


My only concern is how do we deal with working with roots or not?  In my case, I 
doubt I ever want to load interpreters from inside of a chroot, however I think 
I can fairly easily see a need for doing that within an installer framework.


Perhaps what is needed is a macro, convention, etc that indicates either to 
perform the chroot before the load.. or contains the path of the chroot for use 
within a macro  %{nil} if not set?  (Just thinking out loud of how I can use 
this for the cross-architecture installs..)


--Mark

Jeff Johnson wrote:

So far I've (quite successfully ;-) avoided the issue
of where/how/what/when per-interpreter initialization
should come from.

Currently, the initialization is compiled in with the
interpreter wrappers, which was quick and easy devel.

E.g., here's what %{python:...} loads when instantiated:

#if defined(WITH_PYTHONEMBED)
static const char * rpmpythonInitStringIO = \
import sys\n\
from cStringIO import StringIO\n\
sys.stdout = StringIO()\n\
import rpm\n\
;
#endif

That goop needs to be slurped from an
external configure file somehow.

Which means that RPM needs to commit to
some convention for per-interpreter initialization.

What has been wired up with Lua was perfectly OK when
just LUA was embedded. But those conventions, and
--rpmlua, just aren't gonna scale to 5 interpreters.

What I'd like to see is a per-interpreter config macro
carrying the usual colon separated paths that are slurped
and loaded when an embedded interpreter is instantiated.

What I'd like to fix, is the hardwired compiled in paths
for LUA, and remove --rpmlua as a compiled in option.
A popt alias should suffice for all the emebedded interpreters,
the only feature reversion is attempting to run
the rpm executable without _ANY_ configuration. These
days, while you can still override everything in
RPM from the CLI, its highly unlikely you will achieve
much joy, as way too much rpm configuration is already
being loaded from all over the place. JMHO, YMMV.

I'll wire up some KISS convention like
/usr/lib/rpm/embed/{ruby,...}.init
in a flash if there's consensus.

(new topic)
I haven't a clue where to install js/libjsm.so, JavaScript
has no clear conventional guidance for paths to search for
extension modules. Shall I just dump libjsm.so into
/usr/lib/rpm, or should I attempt /usr/lib/rpm/js/*
instead?

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Paths for per-interpreter initialization?

[Mark Hatle]

Then I misunderstood the original question.  I was reading it as to what path(s) 
are used to initialize the interpreters.. i.e. load includes, modules, etc. 
That is what I was referring to.


As for the interpreters themselves, I'm not really sure.  I can see a case for 
both system level paths and paths inside of the /usr/lib*/rpm framework.


--Mark

Jeff Johnson wrote:


On May 7, 2009, at 9:15 AM, Mark Hatle wrote:

I like the idea of changing the locations to be loaded into macros.  
This helps my relocatable RPM mechanism.. ;)




Using a macro for the path doesn't tell me what I asked:

Where should per-interpreter initialization and
private JS extensions be installed?

My only concern is how do we deal with working with roots or not?  In 
my case, I doubt I ever want to load interpreters from inside of a 
chroot, however I think I can fairly easily see a need for doing that 
within an installer framework.




If the stuff isn't installed (because no path has been chosen),
then its impossible to consider using the stuff outside
of a rpm build tree.

Perhaps what is needed is a macro, convention, etc that indicates 
either to perform the chroot before the load.. or contains the path of 
the chroot for use within a macro  %{nil} if not set?  (Just thinking 
out loud of how I can use this for the cross-architecture installs..)




I need answers for what path, not how to Have it your own way! using
macros.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Paths for per-interpreter initialization?

[Mark Hatle]

Ok.. I understand now.. then my suggestion is that these should be macros so 
that individual distributions can set specific initialization options.  A 
reasonable default macro should be provided as well.


(In many ways I equate this to the various pre/post-prep, build, and install 
macros that already exist.)


--Mark

Jeff Johnson wrote:


On May 7, 2009, at 9:57 AM, Mark Hatle wrote:

Then I misunderstood the original question.  I was reading it as to 
what path(s) are used to initialize the interpreters.. i.e. load 
includes, modules, etc. That is what I was referring to.




ATM, the embedded interpreters have initializiation strings
that are eval'd when an embedded interpreter is instantiated.

I'm looking for where to put those strings so that the
strings can be changed without recompiling rpm.

As for the interpreters themselves, I'm not really sure.  I can see a 
case for both system level paths and paths inside of the /usr/lib*/rpm 
framework.




There are many issues. I'm asking the very narrow issue of where to put,
say, this text that is eval'd every time an embedded perl interpreter
is fired up:

/*...@unchecked@*/
static const char * rpmperlInitStringIO = \
use strict;\n\
use IO::String;\n\
our $io = IO::String-new;\n\
select $io;\n\
use RPM;\n\
;
#endif

The other issues of integrating with existing path conventions
of interpreters are equally important.

Guess what I ask next ;-)

73 de Jeff


--Mark

Jeff Johnson wrote:

On May 7, 2009, at 9:15 AM, Mark Hatle wrote:
I like the idea of changing the locations to be loaded into macros.  
This helps my relocatable RPM mechanism.. ;)



Using a macro for the path doesn't tell me what I asked:
   Where should per-interpreter initialization and
   private JS extensions be installed?
My only concern is how do we deal with working with roots or not?  
In my case, I doubt I ever want to load interpreters from inside of 
a chroot, however I think I can fairly easily see a need for doing 
that within an installer framework.



If the stuff isn't installed (because no path has been chosen),
then its impossible to consider using the stuff outside
of a rpm build tree.
Perhaps what is needed is a macro, convention, etc that indicates 
either to perform the chroot before the load.. or contains the path 
of the chroot for use within a macro  %{nil} if not set?  (Just 
thinking out loud of how I can use this for the cross-architecture 
installs..)



I need answers for what path, not how to Have it your own way! using
macros.
73 de Jeff
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Paths for per-interpreter initialization?

[Mark Hatle]

Jeff Johnson wrote:


On May 7, 2009, at 12:42 PM, Mark Hatle wrote:


Jeff Johnson wrote:

On May 7, 2009, at 10:55 AM, Mark Hatle wrote:
Ok.. I understand now.. then my suggestion is that these should be 
macros so that individual distributions can set specific 
initialization options.  A reasonable default macro should be 
provided as well.



Look, you will get a macro, likely with colon separation for multiple
paths, and globbing, and URI's, and security checks with digital
signatures/digests as well as simple owner/permission checks ...


Macros I was referring to are the __build_pre, __build_post and 
similar.  This style would allow basic initialization sequences such 
as the example you provided:


use strict;
use IO::String;
our $io = IO::String-new;
select $io;
use RPM;



AH, I refer to this as templating, and rpm is doing lots and
lots of templating these days, usually with macros or queryformats.

But yes -- if embedded interpreters are a keeper, still unclear --
RPM will proceed through templating to assemble per-interpreter
scripts much like is being done with shell on the build side.

But increasingly, eliminating user scripting, or at least enriching
the set of scripts used by packaging ala debhelper, is more
important (imho) than providing a flexible means to assemble
complex scripts for extending package installs.

YMMV, everyone's does. But ISTRC reading a post from you suggesting
that its not worth the effort of customizing package scripts on
the list that has chosen to censor me to prevent my participation
in RPM discusssions.


Not sure exactly which message you are referring to.. But my experience is that 
if you are basing your distribution off of another distribution, the more 
customization you do to the spec files, the more long term maintenance you have.


However, modifying the master rpm-macros can allow you to globally change 
default behaviors with minimal modifications to the spec files.  (This is the 
technique we are using when integrating Moblin packages into the Wind River 
build system.  We use the templating macros above to pre-define CC=, LD=, etc to 
the cross compilers.  Then adjust the %configure macro to include a reference to 
our config site file(s).)


I'm not sure anyone is going to be able to provide a definitive answer 
as to what text everyone wants to use.. so pick a default, pick a way 
for distros to customize and make the assumption nobody will.. ;)




OK, let's try multiple choice ...

Where should embedded interpreters find their default initialization in 
RPM?

Should the, say, ruby initialization be stored in:
   a) /usr/lib/rpm/embedded/ruby.init
   b) /usr/lib/rpm/ruby/init
   c) all of the above
   d) none of the above
   e) a Gucci purse


Umm, my daughter choses 'e'.

However either A or B would be fine with me.  :)

--Mark

I will tally the votes and set about implementing a reasonable default 
path ...


73 de Jeff
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: rpm API available ?

[Mark Hatle]

Just an FYI, we're interesting in similar partial download and update mechanisms 
for embedded systems.  If you want to bounce ideas off me, feel free.  I can't 
guaranty I'll have time to contribute with code, but I can certainly vet ideas.


Most of the systems I'm currently working with have large amounts of memory and 
disk (relatively speaking in the embedded world), but the read/write window on 
the disk is kept very short.. so we need to use limited ramdisks to download 
updates.


Most of the systems I am familar with do this by managing bundles of packages 
and install scripts today.  (I.e. it downloads a shell script, when proceeds to 
download/install the packages based on known constraints of the device...)


--Mark

Sven Luther wrote:

On Wed, Apr 15, 2009 at 09:03:03AM -0400, Jeff Johnson wrote:

On Apr 15, 2009, at 4:54 AM, Sven Luther wrote:


On Wed, Apr 15, 2009 at 03:31:22AM -0400, Jeff Johnson wrote:

On Apr 15, 2009, at 1:34 AM, Sven Luther wrote:


Hi, ...

I am about to use rpm 5.1 in an embedded context, where we need to  
do

some special treatment before installing the packages.


Can you supply details about special treatment?
Not yet :) But the idea is to do some of the stuff apt-rpm or yum  
does,

but with a smaller footprint. Keep in mind that i only recently dived
into rpm stuff, having mostly done debian stuff previously.


Hint: RPM can do depsolving too. Already. For years.


But not the more advanced downloading strategies we are planning
(because we may not have enough free flash space to download all RPMs in
one go). Anyway, i will need to look at this in more detail.


As i was investigating, various options are open to us, like
writting a
rpmlib based application, or patching rpm. At this point i am only
investigating, but i noticed that the rpm5.org web site has a link  
to

API reference, but it only is rpm 4.5, not rpm 5.


Current CVS can generate API docs using doxygen. I can generate
a tarball of current docs if you want pretty easily.

Ok, found it.


Note that I do maintain the inline doxygen
fairly accurately at the function level and I find the
doxygen doco useful for detailed browsing. But there
hasn't been much need (imho) so far to organize the
per-function doco into subsystems with examples
and more.

Well, i was coming from the C rpmlib code from chapter 15 of the RPM
guide linked from the docs section of rpm5.org.


There's better than Chpter 15 these days. I can whip out
an example for typical operations in an hour or so
if needed.


Yes, this would be helpful, maybe linking to some such examples from the
documentation web page would be helpful too.


Jeff, i also saw in the mailing list archive, that you made some
comment
about thinking there is no need for custom binaries, or something
similar. Does the fact that there is no api reference for rpm 5.1 or
5.2
available and that comment mean that custm binaries using rpmlib is
not
favoured ?


All depends on what you need or want. The context was probably
(dunno) just a statement of fact, that almost nothing uses
-lrpmlib for years, and most of what does link -lrpmlib
is pretty broken or overly complicated or should be done
a different way. That's what I see in C code when asked.
And most code these days is now done in python/perl, not C.

python/perl is no option for embedded targets with low memory/storage
space and cross compilation.


But embedded lua is an option.


I am no fan of interpreted languages, especially in the embedded world,
but i will have a look at this also.


But I'll be happy to help with most of any patches that
need doing if/when API does shift.

In particular, i followed the example of the chapter 15 rpmguide which
allows one to investigate the internals of an rpm package. This  
failed,

as ...


The rpmlib API/ABI changes as little as possible, but that
doesn't mean no change, or guranteed to be supported forevermore.

... the lead.h stuff is now no more exported (struct rpmlead for
example) is defined in pkgio.c only, and the readLead function is no
more present (altough pkgio has a rdLead function which is also not
exported).


Yup. Here's what replaces


Yep, found the below (by looking at the source code :)


   {const char item[] = Lead;
msg = NULL;
rc = rpmpkgRead(item, fd, NULL, msg);


mmm, using a string for what is basically a switch/case operation seems
to me like a waste of cpu cycles.


switch (rc) {
default:
   rpmlog(RPMLOG_ERR, %s: %s: %s\n, fn, item, msg);
   /*...@fallthrough@*/
case RPMRC_NOTFOUND:
   msg = _free(msg);


I am no more at the office, and i don't have the API doc here, but i
think i missed the information in the rpmpkgRead description that you
have to free the msg string. It is evident from the source code, but
still ...


   goto exit;
   /*...@notreached@*/ break;
case RPMRC_OK:
   break;
}
msg = _free(msg);
}


Ok, i did 

Re: [CVS] RPM: rpm/perl/ Makefile.PL.in

[Mark Hatle]

I have some concern that this could still cause problems on other systems.. but 
of course at a much lower rate then before.


Is there any way to only run this when we're using Mac OS X to eliminate the 
remaining possibility?


--Mark

Anders F. Björklund wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  

  Server: rpm5.org Name:   Anders F. Björklund
  Root:   /v/rpm/cvs   Email:  a...@rpm5.org
  Module: rpm  Date:   24-Jan-2009 16:58:54
  Branch: HEAD Handle: 2009012415585300

  Modified files:
rpm/perlMakefile.PL.in

  Log:
fix regexp matching bug with macosx ldflags

  Summary:
RevisionChanges Path
1.25+1  -1  rpm/perl/Makefile.PL.in
  

  patch -p0 '@@ .'
  Index: rpm/perl/Makefile.PL.in
  
  $ cvs diff -u -r1.24 -r1.25 Makefile.PL.in
  --- rpm/perl/Makefile.PL.in   25 Dec 2008 17:44:44 -  1.24
  +++ rpm/perl/Makefile.PL.in   24 Jan 2009 15:58:53 -  1.25
  @@ -32,7 +32,7 @@
   # teach MakeMaker some Mac OS X LDFLAGS:
   my @LDFLAGS = qw(@LDFLAGS@);
   foreach (@LDFLAGS) {
  -if (m/-arch|ppc|i386|ppc64|x86_64/) {
  +if (m/^(-arch|ppc|i386|ppc64|x86_64)$/) {
push (@ldflags, $);
   } elsif (m/-Wl,-search_paths_first/) {
push (@ldflags, $);
  @@ .
__
RPM Package Managerhttp://rpm5.org
CVS Sources Repositoryrpm-...@rpm5.org


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

[Fwd: [Rpm-maint] [patch] warnings for rpmExpand(), rpmlog(), rpmGetPath()]

[Mark Hatle]

This is a fairly interesting patch that was sent to the rpm.org list. 
Is this something we're interested in?  It may point out some issues 
with rpm5, or future development.. (At the expense of gcc 4...)


--Mark

 Original Message 
Subject: [Rpm-maint] 	[patch] warnings for rpmExpand(), rpmlog(), 
rpmGetPath()

Date: Wed, 13 Feb 2008 16:19:41 +0100
From: Michal Marek [EMAIL PROTECTED]
To: [EMAIL PROTECTED]

Hi,

gcc allows to mark variadic functions that expect a terminating NULL as
such and issues a warning if the caller doesn't pass the NULL. A similar
mechanism is available for printf-like functions. What about using this
for the few variadic functions in the rpmlib API? It would help to catch
some mistakes at compile-time rather than at run-time, e.g:

  char *test1 = rpmExpand(%_sourcedir);
  char *test2 = rpmExpand(%_topdir,  );

$ gcc -Wall -I ./include -c test.c
test.c: In function 'main':
test.c:9: warning: not enough variable arguments to fit a sentinel
test.c:10: warning: missing sentinel in function call

Attached are two patches, the first one marks rpmExpand() and
rpmGetPath() with __attribute__((sentinel)) for gcc 4.0+ and rpmlog()
with __attribute__((format (printf, 2, 3))). The second patch silences
warnings in rpmlog() calls that now show up.

Comments?

Michal

diff -r 5165a84e8400 rpmio/rpmfileutil.h
--- a/rpmio/rpmfileutil.h	Mon Feb 11 20:47:03 2008 +0200
+++ b/rpmio/rpmfileutil.h	Wed Feb 13 15:55:03 2008 +0100
@@ -83,7 +83,12 @@ char * rpmGenPath	(const char * urlroot,
  * @param path		macro(s) to expand (NULL terminates list)
  * @return		canonicalized path (malloc'ed)
  */
-char * rpmGetPath (const char * path, ...);
+char * rpmGetPath (const char * path, ...)
+#if defined(__GNUC__)  __GNUC__ = 4
+ /* issue a warning if the list is not  NULL-terminated */
+  __attribute__((sentinel))
+#endif
+  ;
 
 /** \ingroup rpmfileutil
  * Return URL path(s) from a (URL prefixed) pattern glob.
diff -r 5165a84e8400 rpmio/rpmlog.h
--- a/rpmio/rpmlog.h	Mon Feb 11 20:47:03 2008 +0200
+++ b/rpmio/rpmlog.h	Wed Feb 13 15:55:03 2008 +0100
@@ -220,7 +220,12 @@ int rpmlogSetMask (int mask);
 /** \ingroup rpmlog
  * Generate a log message using FMT string and option arguments.
  */
-void rpmlog (int code, const char *fmt, ...);
+void rpmlog (int code, const char *fmt, ...)
+#if defined(__GNUC__)  __GNUC__ = 2
+  /* issue a warning if the format string doesn't match arguments */
+  __attribute__((format (printf, 2, 3)))
+#endif
+  ;
 
 /** \ingroup rpmlog
  * Return text of last rpmError() message.
diff -r 5165a84e8400 rpmio/rpmmacro.h
--- a/rpmio/rpmmacro.h	Mon Feb 11 20:47:03 2008 +0200
+++ b/rpmio/rpmmacro.h	Wed Feb 13 15:55:03 2008 +0100
@@ -124,7 +124,12 @@ void	rpmFreeMacros	(rpmMacroContext mc);
  * @param arg		macro(s) to expand (NULL terminates list)
  * @return		macro expansion (malloc'ed)
  */
-char * rpmExpand	(const char * arg, ...);
+char * rpmExpand	(const char * arg, ...)
+#if defined(__GNUC__)  __GNUC__ = 4
+ /* issue a warning if the list is not  NULL-terminated */
+  __attribute__((sentinel))
+#endif
+  ;
 
 /** \ingroup rpmmacro
  * Return macro expansion as a numeric value.

diff -r 5165a84e8400 lib/query.c
--- a/lib/query.c	Mon Feb 11 20:47:03 2008 +0200
+++ b/lib/query.c	Wed Feb 13 15:55:00 2008 +0100
@@ -641,12 +641,12 @@ int rpmQueryVerify(QVA_t qva, rpmts ts, 
 	rpmlog(RPMLOG_NOTICE, _(invalid package number: %s\n), arg);
 	return 1;
 	}
-	rpmlog(RPMLOG_DEBUG, _(package record number: %u\n), recOffset);
+	rpmlog(RPMLOG_DEBUG, _(package record number: %lu\n), recOffset);
 	/* RPMDBI_PACKAGES */
 	qva-qva_mi = rpmtsInitIterator(ts, RPMDBI_PACKAGES, recOffset, sizeof(recOffset));
 	if (qva-qva_mi == NULL) {
 	rpmlog(RPMLOG_NOTICE,
-		_(record %u could not be read\n), recOffset);
+		_(record %lu could not be read\n), recOffset);
 	res = 1;
 	} else
 	res = rpmcliShowMatches(qva, ts);
diff -r 5165a84e8400 lib/rpminstall.c
--- a/lib/rpminstall.c	Mon Feb 11 20:47:03 2008 +0200
+++ b/lib/rpminstall.c	Wed Feb 13 15:55:00 2008 +0100
@@ -1006,7 +1006,7 @@ int rpmRollback(rpmts ts, struct rpmInst
 	tid = (time_t)thistid;
 	rpmlog(RPMLOG_NOTICE,
 		_(Rollback packages (+%d/-%d) to %-24.24s (0x%08x):\n),
-			numAdded, numRemoved, ctime(tid), tid);
+			numAdded, numRemoved, ctime(tid), thistid);
 
 	rc = rpmtsCheck(ts);
 	ps = rpmtsProblems(ts);
diff -r 5165a84e8400 lib/rpmts.c
--- a/lib/rpmts.c	Mon Feb 11 20:47:03 2008 +0200
+++ b/lib/rpmts.c	Wed Feb 13 15:55:00 2008 +0100
@@ -205,7 +205,7 @@ rpmdbMatchIterator rpmtsInitIterator(con
 	case '(':
 		/* XXX Fail if nested parens. */
 		if (level++ != 0) {
-		rpmlog(RPMLOG_ERR, _(extra '(' in package label: %s\n), keyp);
+		rpmlog(RPMLOG_ERR, _(extra '(' in package label: %s\n), (const char*)keyp);
 		return NULL;
 		}
 		/* Parse explicit epoch. */
@@ -223,7 +223,7 @@ rpmdbMatchIterator rpmtsInitIterator(con
 	case ')':
 		/* XXX Fail if nested parens. */
 		if (--level != 0) {
-		

Re: Sharing some experiences with RPM and Lua...

[Mark Hatle]

Just an FYI of what we are doing here at Wind River with Lua and RPM.

We have a need to install packages that will run on a target machine
perhaps running an ARM process on a ia32 host system.  So we run into
all sorts of issues where transaction scripts (pre/post install/remove,
etc) are not capable of being run properly during the install phase.

The majority of those scripts serve two purposes:

1) user/group add or removal
2) initscript setup (i.e. chkconfig)

In order to do this, we use the lua mechanism in RPM5 and have extended
it by adding a wrs namespace.  For example:

%pre
# Add the dbus user
/usr/sbin/useradd -c 'System message bus' -u %{dbus_user_uid} \
   -s /sbin/nologin -r -d '/' dbus 2 /dev/null || :

would become:

%pre -p lua
# Add the dbus user
wrs.useradd('-c System message bus -u %{dbus_user_uid} -s
/sbin/nologin -r -d / dbus')



or a slightly more complicated one:

%post
/sbin/chkconfig --add crond

%preun
if [ $1 = 0 ]; then
/sbin/chkconfig --del crond
fi

would become:

%post -p lua
wrs.chkconfig('--add crond')

%preun -p lua
if (tonumber(arg[2]) == 0) then
  wrs.chkconfig('--del crond')
end

(note it's arg 2 not $1 as in shell.. arg1 in lua is lua... so it's
a difference that you need to be aware of.)



We intend to expand the wrs namespace over time with other essential
install functions that simply can't be run in a cross compilation
environment.  But for now, these simple replacements cover 90% of the
cases we care about.. the others we have to deal with outside of the RPM
environment.

As an aside..  the following is a neat trick one of our folks does to
run some basic lua scripts:

#!/bin/sh
rpm -E %{lua:[EMAIL PROTECTED]

Then he can just do:

script wrs.chkconfig('--del crond')

to verify if things will do what he expects.. ;)

--Mark

Ralf S. Engelschall wrote:
 As you have certainly recognized on this list, I've recently developed
 a larger set features for RPM 5 with the help of RPM's integrated Lua.
 Although I'm still far away from ever being a Lua-fan (see also my BLOG
 under http://trainofthoughts.org/blog/2007/09/16/lua-popularity/), I
 have to admit that the availability of Lua in RPM is some sort of a
 killer-feature -- most people certainly just will not recognize.
 
 But already the %{lua: } macro functionality is the ultimative
 escape to implement really sophisticated RPM macro hacks. Well, ok, I
 first had to extend the Lua rpm.* namespace with additional functions
 and also implement a bunch of reusable functions in Lua itself under the
 util.* namespace, but now it is really a powerful and useful tool.
 
 Initially I was very sceptic and wanted to avoid Lua in RPM, but
 once I recognized that this way I was finally able to implement some
 long-awaited essential RPM features for OpenPKG, I have to admit that
 IMHO Lua really is a great benefit for RPM. Ok, actually every embedded
 scripting language would be as beneficial as Lua here, of course. But
 RPM only has Lua support, so the hero is Lua.
 
 In other words, let me just share my experience: if you want to power
 up your RPM 5, you really have to get used to Lua and the %{lua: ...}
 functionality! It opens you a large window to new possibilities.
 
 One final tip: when developing in RPM Lua, do not forget that you
 have my handy util.dump() function at hand for dumping and debugging
 arbitrary complex Lua data structures (in Lua syntax). Also, I strongly
 recommend you to build RPM 5 with --with-lua --with-pcre in order to get
 PCRE support in my implemented util.r{match,subst,split}() functions.
 Believe me, you don't want to hack RPM in Lua without these tools... ;-)
 
Ralf S. Engelschall
[EMAIL PROTECTED]
www.engelschall.com
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Emulating rootdir for non-root chroot(2) emulation.

[Mark Hatle]

Two items from extensive use of fakeroot and fakechroot (from Debian).

fakeroot has some many gotcha's.. it in essence has to be built to match
the glibc that it will be overriding.  There are a million and one
overrides.  We've recently been fighting a problem where building
fakeroot for RHEL 4 won't work on Fedora 7.. because Fedora 7's glibc
has additional interfaces that didn't exist in RHEL 4..  so the problem
growns...

The fakechroot issues are quite as bad, but there are still some
problems.  The biggest of them is the behavior where a chroot runs and
the cwd doesn't change.. but PATHs and related now reference the new
root.  So this isn't trivial either.

My best recommendation at this time is to use the debian
fakeroot/fakechroot and document for people how to use them to achieve
what they want.  I'm not sure integrating them into RPM is worth the
effort.. (or more to the point the maintenance headaches!)

--Mark

Jeff Johnson wrote:
 Since the discussion this week is on --root, there are
 several other related issues and/or implementations.
 
 The core feature request related to --root is how to
 permit non-root installs into home directories.
 
 Since chroot(2) is root-only, that means (to me) that all
 system calls which mention a path will need wrapping
 in order to prefix an emulated chroot path.
 
 It also means that the wrapped system calls must be used
 consistently throughout rpm.
 
 The issue of prefixing current working directory to change
 a relative into an absolute path is a closely related problem
 (without the root-only showstopper flaw) and likely should be attempted
 at the same time.
 
 There have also been suggestions about fakeroot emulation
 using rpmlib which are closely related.
 
 FInally, there is the choice of names for the syscall wrappers.
 Currently, rpm is changing the first letter to uppercase, as in
 the Open() system wrapper. That's fine for rpm code itself, but will
 not permit applications to use an emulated chroot(2) as non-root.
 
 So the eventual implementation of an emulated chroot(2) and
 fakeroot will have to replace the system calls in libc much like
 current implementations of fakeroot do.
 
 Whether rpm should attempt chroot(2) emulation at all is an open question.
 
 The RFE for the ability to be able to install using --root as non-root
 user has been around forever, and (imho) an implementation needs
 to be attempted somewhen.
 
 73 de Jeff
 
 
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: db4 vs. sqlite

[Mark Hatle]

We use sqlite, but my recommendation is unless there is a very specific
reason to use sqlite, don't.

In my case I need to use sqlite due to multiple glibc locking issues.
(db4 is really unhappy when built w/ one version of glibc and run on
another.. most likely due to various locking issues.)

Also I have a requirement to do basic database access over NFS.  While
this is generally unsafe, sqlite makes it more reliable as long as you
don't attempt concurrent access.  For my environment the benefits
outweigh the risks.

(sqlite is also considerbly slower then db4, due to the emulation
aspects..  it also takes more space.. etc..)

--Mark

Bernhard Rosenkraenzer wrote:
 Hi,
 we're just about to switch Ark Linux over to using rpm5 - so this would be 
 the 
 right time for us to make other related changes.
 
 The first thing coming to mind is the rpm database format -- we've been using 
 db4 so far, has anyone compared db4 and sqlite to see if switching to sqlite 
 has any benefits/drawbacks (speed, db file size, ...)?
 
 Also, are there any known db backend specific bugs?
 
 Thanks,
 bero
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: db4 vs. sqlite

[Mark Hatle]

Jeff Johnson wrote:
 2) Supporting asm locking on a myriad of embedded tool chains. Of course
 fcntl locking with BDB would have solved exactly the same problem.

I tried this a month or so ago and it was incredibly unreliable.  I
didn't have the time to track down the issues.

The symptom was that you could open and work w/ the database once.. then
the next time you tried it went into a lock recovery loop until you kill
-9'ed RPM.  (none of this was over NFS BTW.)  So for fcntl locking in
db4 to be usable, someone will really need to try and document the
gotchas.  (Unfortunately I didn't have time.)

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Mechanism to generate *only* pkg NEVR dependencies

[Mark Hatle]

This would be useful to us.  I'm not sure we'll use it right away, but I
would expect we'll use it within a year.

--Mark

Jeff Johnson wrote:
 Right next to the mechanism for filtering internally generated soname
 dependencies in the PLD patch, is a means to map all dependencies back
 to package NEVRA.
 
 The rationale for the mapping (as far as I understand) is to simplify
 finding the name of the object that contains the provides that satisfies
 a require.
 
 WIthout getting into semantic usage cases (e.g. whether soname and
 file dependencies are a good or a bad idea), I ask
 
 Should the mechanism be added to HEAD?
 
 73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Filtering automagically generated dependencies

[Mark Hatle]

I know there have been an awful lot of people who want to do this kind
of filtering.  perl, dlopen have been the primary ones I know about.

Does the filtering affect the internal per-file dependencies or just the
overall package dependencies once they are rolled together?

personally I'd like a mechanism to do this kind of filtering on a per
package basis... but so far you've talked me out of it.

--Mark

Jeff Johnson wrote:
 PLD has a patch adding a macro to specify patterns to filter internally
 generated soname dependencies.
 
 Basically
 %define _noautorequires libA.so libB.so
 with some regex gunk in lib/rpmfc.c to do essentially grep -v.
 
 Should it be added?
 
 My personal opinion is that per-pkg filtering of dependencies that are
 either mis-generated in the first place, e.g.
 Requires: perl(the)
 or that achieve some marginally useful goal, like filtering a soname
 that is generated for a dlopen'd module that loads dynaically, is of
 marginal utility. There are many ways to solve the problem with
 existiung mechanisms in rpm.
 
 
 OTOH, I've always believed that there needs to be a general per-distro
 (i.e. no specfile syntax changes) filtering mechanism with patterns read
 from a file. The stopping point for the implementation is lack of an
 answer to
 Who maintains the patterns in the file?
 
 The PLD patch hurts nothing, but does establish a Newer! Better! Bestest!
 convention for packaging that may last for years and years, just like
 rpmrc files.
 
 73 de Jeff
 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Logging rpm install/erase operations

[Mark Hatle]

I think this would be useful functionality to track package movement in
a system.  So it's worth exploring to me.. but I'm on the fence on if it
should really be in or not..  It would be nice to understand the
ramifications of the patch to how RPM operates.  (Also any logging needs
to deal with programs that directly use rpmlib vs the command line so
the log is complete at all times..)

--Mark

Jeff Johnson wrote:
 PLD wishes a historical log of package install/erase operations.
 
 So Yet Another Path needs to be agreed upon as a place to
 append a time-stamp'ed message.
 
 So I'm looking for consensus on the path to be used as de in rpm.
 
 Ideas?
 
 73 de Jeff
 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Filtering automagically generated dependencies

[Mark Hatle]

Arkadiusz Miskiewicz wrote:
 Does the filtering affect the internal per-file dependencies or just the
 overall package dependencies once they are rolled together?
 
 PLD patch applies to overall deps for entire spec. So it's not per-pkg. It's 
 per-spec. Both per-{pkg,spec} are actually useful but we implemented per-spec 
 only.

(Without seeing the patch), does this affect the per-file dependency
generator, or is the filter applied when it's rolled up into the pkg
dependencies?  I'm concerned about this because (and MV) use the
per-file dependencies as hints on how to construct partial rpm package
installs for embedded systems.  If we're not filtering on the file deps,
then the hints will be invalid, or won't match the packages overall deps.

 personally I'd like a mechanism to do this kind of filtering on a per
 package basis... but so far you've talked me out of it.
 
 per-spec, too. per-pkg only would require to put filtering 10 times in one 
 spec if it generates 10 pkgs from single spec.

Actually thats what I wanted, was per-spec.  I meant per source
package.  Same thing :)
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: chroot(2) - db-4.6.19 problems

[Mark Hatle]

Jeff Johnson wrote:
 Because of the often requested feature to be able to install using rpm
 --root as an
 unprivileged user (chroot(2) is limited to uid=0), the forward looking
 development
 answer will be to attempt to emulate chroot prefixing for unprivileged
 users,
 but that too is a largish change that assumes that all file paths pass only
 through a syscall wrapper, not through the syscall itself.

We actually do this.  We have three programs that work as intercepters:

fakeroot (from Debian)
fakechroot (from Debian)
fakepw (I wrote this)

Basically fakeroot, makes the system think it's root and has a backend
database to capture file information such as mode, user, group, device,
fifo, etc..

fakechroot, intercepts all of the filesystem calls and maps them into a
fakeroot environment.  Short of one bug I found in the lua support on
Friday this works properly w/ RPM.

fakepw, intercepts all of the group and password commands and checks the
filesystem and avoids nss lookups.

I can write up some basic instructions in a couple of week (when I get
back from business trips) if anyone is interested.  While I'd prefer to
be able to do some of this right inside of RPM, this was the only way I
could think of to deal w/ special files and lua scripts.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Choosing a test harness ...

[Mark Hatle]

Personally.. I like make check.  With the belief that it would run
through James Olin Odin's test suite as one of the checks, as well as
the trpm stuff.

--Mark

Jeff Johnson wrote:
 Its likely time to commit to some test harness methodolgy for rpm-5.0.
 
 There are several candidate choices (imho):
 
 0) Do nothing, just use rpm.
This isn't as crazy as it sounds. A week underneath a distro
 usually shakes out any/all rpm problems, and was the de facto
 criteria for releasing rpm for years and years.
 
 1) make check
There's a great deal of activity dusting off what I originally wrote
in 1998 at rpm.org which could be re-cycled, if for no other
 reason, to
attempt to maintain compatible behavior in rpm5.org. OTOH, I'm
not sure testing the build-install-erase w queries is worth the
 effort, none
of the above breaks (except for easily fixed brain farts) at the
 single package
level any more. The cost of maintaining the make check Auto Fu
 is non-trivial as well.
 
 2) ./test-harness
 James Olin Odin's test harness was essential in capturing certain
--rollback problem cases reproducibly. The harness is easily
 extensible
to other tests. OTOH, the test-harness is not as standalone as it
 should be,
and expects additional environmental configgery to function.
 
 3) devtool standalone
 
   devtool standalone has sanity checks that are likely sufficient to
 exercise
   misbuilds. devtool can be extended in many ways rather easily, it
 would
   not be impossible to try to capture more extensive tests within
 devtool
   stanzas
 
 4) trpm
While trpm is an absolutely grotty script, the script WORKSFORME.
 The
   script is crap, but the ability to generate transaction manifests
 and configuration
   with the property of dependency closure from globs against a
 steaming pile of packages
   is unparalleled. All of the major changes in rpm, including adding
 signatures,
   --rollback, and multlib, were developed using trpm test cases.
 
 My personal preference (today expect quixotic behavior) would be to replace
 the test-harness driver with devtool, and add some additional stanzas to
 generate
 test manifests using the globs from trpm.
 
 But I'd like to see any harness rather than none.
 
 73 de Jeff
 
  
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Should we enforce src.rpm as arch independent ?

[Mark Hatle]

From somebody who has done a lot of this type of target compilation.
Just limit to one target argument and make the builder issue multiple
rpmbuild commands.  IMHO thats the only reasonable solution to this.
There is little to gain from having the ability to build multiple target
packages w/ a single build call.

--Mark

Jeff Johnson wrote:
 
 On Aug 17, 2007, at 9:51 PM, Olivier Thauvin wrote:
 
 After a short discussion on irc with Jeff, I decide to work on the
 part I know
 the best: build/.

 My goal is to provide a way to cleanly build a packages (so many
 packages) for
 many architectures (--target foo --target bar) in envirronement and
 w/o the
 current recursivity.

 
 OK. So you want a build system tool able to build package(s) for
 multiple target platforms.
 
 You need to look at the fundamental design issue first:
 
 Multiple target platforms (in general) is more than i[3456]86 targets,
 and that means remote execution (or emulation).
 
 You also should think through what a build environment means. ATM,
 rpmbuild's connection to managing the environment is nothing more than
 setting up per-target macros, verifying build dependencies, and reaping
 scriptlet(s) status for a single build.
 
 Very specifically _NOT_ attempted are the mechanics of entering/leaving
 a chroot,
 installing missing build elements, remote scriptlet execution, logging
 output, etc.
 
 My though is to create a struct to hold all info related to build:

 struct Spec_s {
 const char specfile;
 [...]
 }

 struct buildenv_s {
 Spec spec;
 scripts ...
 Packages ...
 }

 
 These are the wrong abstraction(s).
 
 For starters, package is way too imprecise a concept to
 define object methods.
 
 Now the problem... In an ideal world, rpm -ba pkg.spec --target foo
 --target
 bar should build one src.rpm and for each platform one or more binary.
 Currently a src.rpm is build for each platform, normally overriding
 each one
 overriding the previous.

 But in real world, nothing can prevent this kind of spec (worst case):

 Name: foo-%{_target_arch}
 Source: foo-%{_target_arch}.tar.gz
 Buildrequires: bar-%{_target_arch}

 Reviewing:
 Name: this will works, creating foo-i586.src.rpm, foo-ppc.src.rpm, etc...
 Source: this will probably cause issue, as only one source will packed
 Buildrequires: this will works and is probably use, this end as
 Requires in
 the src.rpm.

 The spec should be expand for each architecture, so in case I rework
 the build
 part,, I am thinking to first parse the spec (preamble only) to generate
 src.rpm with a fake platform to really deny spec like I exposed
 previously.

 
 There already is a loop over multiple target platforms in build.c, and the
 entire configuration is trashed and reloaded for each loop traversal.
 
 Far more reliable is the simple shell loop
 for t in target1 target2 target3; do rpmbuild -ba --target $t
 foo.spec; done
 
 OTOH, I do believe that generating a *.src.rpm immediately after parsing,
 not after building (as currently), starts to address the fundamental
 issue of
 multiple srpm's, with possibly differing content, being generated on
 multiple
 target(s).
 
 The fundamenatl design decision will be whether there is one single
 canonical
 master src.rpm, or whether srpm's indeed should be considered per-platform
 with potentially different content.
 
 I'm much less enthusiastic about fake platform ...
 
 73 de Jeff
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Should we enforce src.rpm as arch independent ?

[Mark Hatle]

Jeff Johnson wrote:
 Very specifically _NOT_ attempted are the mechanics of entering/leaving
 a chroot,
 installing missing build elements, remote scriptlet execution, logging
 output, etc.

 None of that is needed for a cross-compile rpmbuild.

 
 Yep. But the major impediment I hear of is that /bin/sh is not native in
 a chroot, and so rpmbuild (and rpm install) scriptlets will not run.
 
 Well, duh ...

/bin/sh though is only needed for package INSTALL, not building.  I've
never encountered a problem that couldn't be patched away during a cross
compile.

When installing packages to a sysroot to make headers and libraries
available for linking you don't need scriptlets to execute.  (At last I
never have.)  It's only for a runnable system that scriptlets are
useful.  (i.e. adduser, chkconfig, etc.)

 Far more reliable is the simple shell loop
 for t in target1 target2 target3; do rpmbuild -ba --target $t
 foo.spec; done

 Yep that is what worked for me.

This is more or less what I have always done.

...

 Another fundamental question here is about whether crosscompile is the
 official way to build for other targets rather than entering to
 half-working morass of remote scriplet execution and other stickyness.
   I really recommend cross compile and it's reasonably well supported
 already, although the dependency stuff is inadequate to deal with the
 situation that you have an RPM database that talks about the build host
 environment AND an RPM database that talks about the cross build
 environment for the current target.

 
 There are many reasons to attempt remote execution of build scriptlets, not
 just to have %{_target_cpu} on remote != local.

There may be reasons, but I've never encountered a situation that really
requires it.  The times where it would have been nice, generally I can
run the item on the target system, capture the configuration and use
that configuration in the future.

 There is nothing whatsoever stopping you from using --dbpath (or change
 %_dbpath)
 to have rpmbuild use multiple rpmdb's. Throwing massive amounts of
 cross-packaged garbage into
 a single rpmdb and expecting rpm to sort out the mess is naive. I
 certainly cannot justify
 my time expenditure to support the vanishingly small needs of cross
 compilation build hosts.

This is kind of what I did at MV.  In hindsight.. it was a little heavy
weight for what is really required.  A more reasonable solution in my
experience is to use a custom database that captures the state of the
target _only_ during compilation.  Build dependencies can either be
resolved or discovered though the existing rpm mechanisms.  This also
does a good job of pointing out bogus build dependencies.  :)

 Meanwhile I'd certainly like to see more cros-compilation used. What I
 actually see is
 that few have the time/energy to maintain cross-tool chains, and who
 have the methodolgy
 and discipline to setup cross build machines, all using *.rpm packages.
 
 Note all using *.rpm packages qualifier.

It's definitely not as easy at customers would like, but with the power
of the macros already in use.. it's not difficult to import a lot of
packages with patches that only address the cross compilation failures
in the package itself, with little to no changes in the spec file.

One bit of advice to folks doing this..  use the %build and %install
override macros and seed your environment w/ CC, LD, etc values there...
 Override the %configure macro w/ one that specifies the correst build
and target..  and many things just work.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Should we enforce src.rpm as arch independent ?

[Mark Hatle]

Jeff Johnson wrote:
 
 On Aug 20, 2007, at 11:53 AM, Mark Hatle wrote:
 

 One bit of advice to folks doing this..  use the %build and %install
 override macros and seed your environment w/ CC, LD, etc values there...
  Override the %configure macro w/ one that specifies the correst build
 and target..  and many things just work.

 
 Um, %build and %install are not macros at all, but rather section markers.

Yes I know, I wasn't be pedantic enough sorry.

 Overloading section markers like %build with additional goop will
 eventually collide with some other overloading.
 
 There is already mechanism to add additional shell to rpm scriptlets.
 
 Configure %___build_pre to add addition envvar's to all build scriptlets.

This is what we use, and what I had meant.  Same goes w/ install_pre.

 Or change the envvar's for a specific section by configuring the
 specific section pre scriptlet macro. E.g. to add an envvar to only
 the %prep section, one would modify
 
 %__spec_build_pre   %{___build_pre}

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Lua -- how to extend?

[Mark Hatle]

Ralf S. Engelschall wrote:
 On Wed, Aug 08, 2007, Mark Hatle wrote:
 
 I'm trying to use lua to implement some complex transaction scripts..
 specifically adduser, addgroup and chkconfig functionalities.  Due to my
 cross compile environment, I can't actually execute the adduser,
 addgroup or chkconfig command line programs as they may not execute on
 the machine I'm populating the chroot on.

 What I would like to do is provide a C function for each of those items
 and add it into the lua grammar using a namespace of wrs or something
 similar.

 I've looked at rpmio/rpmlua.c and see how the rpm namespace is added..
 but isn't there a more standard way to load a lua module and extend the
 grammar with additional commands?  (I'm trying to avoid modifying RPM to
 add the functionality I need for transaction scripts.)

 So how do I add lua modules into the lua that RPM uses?  Any help would
 be appreciated.
 
 Lua has a require directive which can load your extension code
 from a DSO...

Where does this DSO live?  If it's inside of the chroot, I'm back to
where I started because the chroot contains code that I can't possibly
execute.

Also do you have an example of this, or can you point me somewhere with
an example?

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Lua -- how to extend?

[Mark Hatle]

Jeff Johnson wrote:
 
 On Aug 8, 2007, at 4:19 PM, Mark Hatle wrote:
 

 Where does this DSO live?  If it's inside of the chroot, I'm back to
 where I started because the chroot contains code that I can't possibly
 execute.

 
 If all you're trying to do is run in a chroot where /bin/sh is
 non-functional
 because not the right target arch, then try
%post -p %%{some_path_to_functional_shell_macro}
 and override the scritlet PATH macro.

Same problem though.  the -p is executed inside of the chroot..  Which
means I still need to have enough of a (host) environment somehow
prepopulated in the chroot to work.  And I'm really trying to avoid
mixing host and target binaries.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Lua -- how to extend?

[Mark Hatle]

Jeff Johnson wrote:
 
 On Aug 8, 2007, at 4:53 PM, Mark Hatle wrote:
 
 Jeff Johnson wrote:


 Same problem though.  the -p is executed inside of the chroot..  Which
 means I still need to have enough of a (host) environment somehow
 prepopulated in the chroot to work.  And I'm really trying to avoid
 mixing host and target binaries.

 
 Having a directory with an executable shell, and a handful of
 useful helpers like grep, in /host/bin is not exactly mixing.
 
 rewriting adduser() in lua as anything more complicated than
 (minimal necessary to define a uglix user)
 echo root:*:0:0:System Administrator:/var/root:/bin/sh  /etc/passwd
 is gonna be nontrivial no matter what. There are likely dlopen's
 of nss modules if you go through glibc, none of which are gonna
 function in a cross chroot.

The existing plan is to take the already C version of adduser, and put
it into a dso, and load it via lua so I can just call the function with
the right arguments.  This may not actually happen however...

We're only concerned with writing to the
/etc/{passwd,shadow,group,gshadow} files.. so nss is not an issue for us
here.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

RPM python module

[Mark Hatle]

When cross-compiling the rpm python module I'm getting a bogus -rpath
being added to the compilation.

The problem is the code is designed to be run on the target and is
picking up host libraries and corrupting my build.

The python/Makefile.am has no references to -rpath $(rpmdir), but the
Makefile.in does.  So where is this wonderfully incorrect -rpath
$(rpmdir) coming from?

Any help would be appreciated.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: database support in rpm

[Mark Hatle]

Jeff Johnson wrote:
 
 On Aug 1, 2007, at 4:03 PM, Mark Hatle wrote:
 

 SQLite has a locking mechanism that will work on shared filesystems (w/
 proper locking.)  Ignore NFS.. thats just one way to get a shared
 filesystem, but not the only way.)  The assumption here is we have a
 shared filesystem w/ proper file based locking.  In that environment
 sqlite can do the basic tasks of keeping things from colliding.

 
 Presumably proper locking is fcntl locking. Or are you calling
 dot-locked files proper?

I'm referring to fcntl locking working in a filesystem.

 If using BerkleyDB either need a master database with locks held
 elsewhere, or change the locking method to file based.

 
 Have you actually tried Berkeley DB with file control locking?
 
 Nevertheless, back to the subject: I have taken notice that the rpm5.org
 user community has need for networked DB support. So I will adjust my
 proposal.

 I do think it's a really good idea to have a networked DB available for
 configurations that need it.  These aren't common in the
 workstation/server world.. but are in the telco world.  Some system
 designs are better then others.  :)

 
 Networked DB and file systems with proper locking are way different issues.

Correct different issues, but I see them as related.  They are design
tradeoffs in this use case.  Either I have to have a filesystem w/
functional fcntl locking (and configure RPM to use fcntl locking), or I
need a remote DB.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: database support in rpm

[Mark Hatle]

Thomas Lotterer wrote:
 On Monday, 30. July 2007 at 9:26 pm, Jeff Johnson [EMAIL PROTECTED] wrote:
 On Jul 30, 2007, at 2:58 PM, Thomas Lotterer wrote:
 Decision:
 - exclusively use Berkeley DB
 I'd change exclusively to primarily. Otherwise we have an
 external feature regression.

 That's exactly the intention! My observations are there is not enough
 interest in supporting a different DB. The current incomplete support
 for SQLite sends a false signal to those who are interested in it.
 Everyone should know for sure: BDB only.

Make a list of what is incomplete and as I have time I will attempt to
implement the missing feature(s).  As far as my usage scenario w/
SQLite, there are no incomplete features.

 The engineering issues wrto a SQL db should be addressed. No matter
 what, a reference SQL schema for rpm package metadata has been needed
 for years. Candidates are [...] Rolling a SQL schema from scratch is
 not impossible either.

 Can be done later if Reasons are reviewed and found to be obsolete.

This is off into the realm of a properly schema vs the key/value pair
BerkleyDB emulation.. so it's a bit of a switch as to the current
emulation implementation.

 Alternatives to Berkeley DB for the 3-4 usage cases:
 1) licensing

 What's the issue here?
 
 2) embedded and -NPTL locking (although fcntl with BDB likely  
 addresses)

 So BDB is fine.

BDB is not fine in all cases.  For embedded, the locking mechanisms may
vary wildly from architecture to architecture.  I have the need to
create a database on x86_32 and use it on mips64.  While fcntl may be
able to address this w/ BDB, sqlite has proven (for our usage) to be a
reliable alternative to hacking on BDB.)

 3) NFS support

 The can be rewritten to networked filesystem support. BDB, SQLite and
 any embedded DB require proper filesystem locking. Given the
 environmental issues of a networked filesystem this will never work
 reliably. Many NFS admins have disabled locking to avoid client hangs on
 server outages and the price is obviously that locking doesn't work. If
 they would use locking then this problem is gone at the price of
 unpredictable client hangs. I spent years of my life trying to improve
 DOS/Windows applications with embedded DB like M$ Access in networked
 environments using CIFS and NCP - no way to ever fix 'em through
 filesystem access. The only real fix is to use applications that are
 designed for client/server environments. If rpm is ever split into a
 client/server application then the server still is likely to use BDB.

The goal is always to require proper filesystem locking, but reality
kicks in and you live with what you've been given.  Using RPC or
whatever for BDB is one solution, and one I'd personally like to see
documented and tested, it's too heavy weight for some environments.
Good enough is frankly good enough for many people, and thats what
SQLite appears to be.

 So 2) and 3) do not qualify for a BDB alternative.
 Let's see what 1) brings up.

They don't quality in your Workstation environment.  They do in the
embedded development environment.  NFS is used extensively for
development and prototyping.  It may even be used in some production
environments.  Don't misunderstand me.  I never intended SQLite's BDB
emulation to replace BDB... it was intended to work around other system
issues that didn't have simple solutions where good enough was what as
needed.  SQLite is much slower the BDB, and we believe it may have
concurrent access issues...(which we've never proven or disproven BTW).

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Support for multiple rpmdb's

[Mark Hatle]

Jeff Johnson wrote:
 There is also the existing %_solve_dbpath. There is  a need for a per-rpmts
 database for headers passed to rpmtsAddInstallElement that could/should
 be used to eliminate the nasty code with a largish in-memory footprint in
 lib/rpmal.c.
 
 But if other means to an integrated dependency solver for rpm-5.0 are
 desired, it's likely time to warn me now ;-)

(Before I begin, I realize it's sick, but I just wanted to describe a
usage that would be nice to keep in mind.)

For MontaVista's environment, there were multiple RPM databases that
revolved around different namespaces.  The four namespaces are common
host cross and target.

MontaVista's environment is a development environment that internally
uses RPM to manage software packaging, installation and upgrades.  This
is all done w/o having the ability to run software as root!

Each of the namespaces correspond to software that is related in some
way, the rules are:

common software is common among all MV products, and executes on the
host system

host software is specific to a single MV product and executes on the
host system

cross software is specific to a single MV product, and a specific target
CPU type, but executes on the host system

target software is specific to a single MV product, and a specific
target CPU, and executes on the target system.

common software can only require other common software and host system
software

host software can require other host software, common software, and host
system software

cross software can require other cross software, host software, common
software and host system software

target software can only require other target software

---

The way this was solved (when I was at MV) was to catch the case where
dependencies were not solved.  Instead of running an external dep
solver, we would run a script that would attempt to resolve the
software.  The script was passed the dependency and the namespace the
request came from.

So if someone was trying to install cross software, it would use the
internal dependency stuff, then fall back to the script which would
check the host, then common rpm databases.. if neither of those
satisfied the dependency it would then do a runtime probe on the system.
 Due to the name space issue you could have a dependency that specified
the namespace of the item you were interested in.  I.e. cross foo could
require host(foo).

...

So anyway, as I noted above the point of this is to give you a usage
scenario that exists for multiple rpm databases, and run time dependency
solutions.  The idea of a namespace to group software is interesting to
me, but not critical (there are other ways then namespace to manage
this.)  I know of another use (outside of MV) of that patch.  Their dep
solver is used to manage cross-machine dependencies.  I.e. packageB
requires that packageA is installed somewhere on the cluster.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: database support in rpm

[Mark Hatle]

Thomas Lotterer wrote:
 On Tuesday, 31. July 2007 at 3:04 pm, Jeff Johnson wrote:
 On Jul 31, 2007, at 5:02 AM, Thomas Lotterer wrote:
 On Monday, 30. July 2007 at 9:26 pm, Jeff Johnson wrote:
 On Jul 30, 2007, at 2:58 PM, Thomas Lotterer wrote:
 Decision:
 - exclusively use Berkeley DB
 3) NFS support

 This can be rewritten to networked filesystem support. BDB, SQLite
 and any embedded DB require proper filesystem locking. Given the
 environmental issues of a networked filesystem this will never work
 reliably.

 I don't disagree. However, an answer for client/server rpmdb needs to
 be identified.

 No use case for client/server rpmdb has been brought to my attention.

You have a chassis w/ 8 blades.  1 blade acts as an internal server for
the other 7 blades.  It serves a kernel and filesystem to the other 7.

The other 7 blades boot and have to do things related to the rpm
database.  Over NFS the blades can blow up.. using RPC they will be able
to manipulate the database as necessary.

That is a very common usage scenario in the Telecommunication space.  7
blades do the work, 1 (or 2) blades act as a control infrastructure.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ COPYING COPYING.LIB rpm/lib/ rpmqv.c rpm/ rpmqv.c

[Mark Hatle]

I strongly suggest that the Copyright be amended to either be you jbj or
rpm5.org or something.  Leaving it at 2002 - Red Hat may give the wrong
impression as to the status of this fork.

--Mark

Jeff Johnson wrote:
   Index: rpm/rpmqv.c
   
   $ cvs diff -u -r1.118 -r1.119 rpmqv.c
   --- rpm/rpmqv.c 16 Jul 2007 01:32:25 -  1.118
   +++ rpm/rpmqv.c 31 Jul 2007 22:21:13 -  1.119
   @@ -155,7 +155,7 @@
   /[EMAIL PROTECTED] *fp, fileSystem @*/
{
fprintf(fp, _(Copyright (C) 1998-2002 - Red Hat, Inc.\n));
   -fprintf(fp, _(This program may be freely redistributed under the 
 terms of the GNU GPL\n));
   +fprintf(fp, _(This program may be freely redistributed under the 
 terms of the GNU LGPL\n));
}

static void printUsage(poptContext con, FILE * fp, int flags)
   @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repository[EMAIL PROTECTED]

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: database support in rpm

[Mark Hatle]

Jeff Johnson wrote:
 
 On Jul 31, 2007, at 6:39 PM, Mark Hatle wrote:
 
 Thomas Lotterer wrote:
 On Tuesday, 31. July 2007 at 3:04 pm, Jeff Johnson wrote:
 On Jul 31, 2007, at 5:02 AM, Thomas Lotterer wrote:
 On Monday, 30. July 2007 at 9:26 pm, Jeff Johnson wrote:
 On Jul 30, 2007, at 2:58 PM, Thomas Lotterer wrote:
 Decision:
 - exclusively use Berkeley DB
 3) NFS support

 This can be rewritten to networked filesystem support. BDB, SQLite
 and any embedded DB require proper filesystem locking. Given the
 environmental issues of a networked filesystem this will never work
 reliably.

 I don't disagree. However, an answer for client/server rpmdb needs to
 be identified.

 No use case for client/server rpmdb has been brought to my attention.

 You have a chassis w/ 8 blades.  1 blade acts as an internal server for
 the other 7 blades.  It serves a kernel and filesystem to the other 7.

 The other 7 blades boot and have to do things related to the rpm
 database.  Over NFS the blades can blow up.. using RPC they will be able
 to manipulate the database as necessary.

 That is a very common usage scenario in the Telecommunication space.  7
 blades do the work, 1 (or 2) blades act as a control infrastructure.

 
 And in the telco space one often desires 5 9's behavior.
 
 I'm not sure that NFS write's qualify. Try NFS to multiple clients, then
 random removes/writes on clients of  those files. The server than has
 multiple
 copies of files with identical paths, but differing per-client content.
 That's
 the NFS behavior I remember, not looked or cared for years.

There are other shared filesystem (think Fibre channel) that don't rely
on NFS.. but can't use NPTL locking because they're done on separate
machines.

I certainly don't endorse NFS for 5 9's.  :)

 Putting an rpmdb on NFS, with associated write's and rpm --rebuilddb
 behavior, is likely to cause unexpected problems. And rpmdb database
 problems are far more likely to be noticed than random write corruption
 for, say, log files or backup's.
 
 I've also debug'ed a handful of problems with rpm installing onto large
 NFS trees, some that were quite reproducible. The last problem I remember
 was at NCSU quite a while ago, should be in bugzilla.
 
 I doubt NFS has gotten significantly better since I last looked.

I'm pretty sure it hasn't.. :P

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

MIPS64 n32 files

[Mark Hatle]

I was looking at something today, and noticed that the MIPS64 n32
dependencies did not get (32bit) appended to them... I looked briefly
and found the place where that would need to be done.  But what I didn't
understand is how to tell if the item was N32 or not?

(I'm also not finding where the RPMFC_ELFMIPSN32 is set.. maybe my grep
is failing me?)

My guess as to what is required is in lib/rpmds.c:sonameDep, either add
an N32 argument or use the existing isElf64 and then add the (32bit) (or
maybe (n32)?) to the dependencies based on that flag.



The problem I have is some external dep solvers (I hate them) are
querying the packages and telling the person if they have all of the
deps resolved or not.  When installing a mixture of MIPS32 (o32) and
MIPS64 (n32).. the deps all appear to have the same names so it can't
find any problems until the install occurs.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM 5 status quo

[Mark Hatle]

Ralf S. Engelschall wrote:
 Some status quo on RPM 5 from my personal point of view:
 
 As you perhaps recognized yourself, just about 4 weeks ago building
 RPM was (sorry that I have to say this) still more or less a real
 nightmare -- even for a hard-core developers.
 
...
 
 After about 6 (unfortunately unpaid) man-weeks of very extensive
 development on RPM 5, I personally have soon to throttle my amount
 of development time I spend on RPM 5 as I've to finish some
 business-related project management certifications first and I also
 have to earn a living (at least until someone is willing to pay me for
 further hacking on RPM 5 ;-). Nevertheless, I'm very pleased with the
 strong progress and especially the resulted very good shape of RPM 5.
 
 Jeff, now it's your turn to also blow up
 the RPM5-internal feature volume... ;-)

Thank you for all of the hard work.  I know for the first time ever I
haven't had to carry extensive hacking to the build system in order to
cross compile RPM.

...

From a status point of view the only remaining issue I am aware of is
the whole library linking issue with the --as-needed and libtool .la
file linkage information.  I'm not sure we have a definitive answer for
this yet.  (We do however have documentation on the approach taken with
valid reasons.)


Again, thanks for all the hard work.. it's made the configuration much
better then it has been in many years.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM 5 status quo

[Mark Hatle]

Olivier Thauvin wrote:
 Le mardi 17 juillet 2007, Ralf S. Engelschall a écrit :
 1. except for very small and harmless hacks, the RPM 5 build environment
is now really acceptably clean and mostly untwisted
again. Especially, mostly all hard-coded (and often even
distribution-specific) installation paths and other nasty hard-coded
things are gone. One even can install RPM now with even a GNU-style
filesystem layout instead of the default Linux/LSB filesystem layout.
And not to mention the fact that RPM 5 since recently finally
provides a very flexible and clean way build against the numerious
third-party libraries. Something which required really lots of
hacking for distribution vendors in the past.
 
 In the case you wonder, I am in favor of killing all hardcode specific 
 distribution value. I think rpm should just provide option a build to please 
 everyone, but it is to vendor to tweak it.

Just an FYI, this is the tactic I am approaching RPM with (for the Wind
River distribution.)  I've got 3 local (very small) Wind River
configuration patches here.  And at this point they really aren't
applicable to anyone else.

From a development point of view the tactic is bugs are bugs.. lets fix
them.  :)

This does make my brain come up with the question.  Would it make sense
to add a contrib-patches dir?  I.e. changes that can be used as examples
of vendor tailoring and/or patches that may be useful to someone but
don't fall under the generally acceptable category?

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM_CHECK_LIB vs older autotools

[Mark Hatle]

Ralf S. Engelschall wrote:
 Jeff: what about rolling daily _SNAPSHOT_ tarballs for RPM 5 (like
 rpm-5.0-SNAPSHOT-MMDD.tar.gz)? These would already contain all the
 pre-generated files from ./devtool autogen and this way we could let
 people more easy try out our latest sources without the need for all the
 development tools? And as they are clearly marked as SNAPSHOTSs, I see
 no problem with them. If nobody has any objections, I'll investigate and
 setup a cron-job on rpm5.org for providing those daily snapshots, ok?

I would like to see these.. I'm doing that virtually on my own machine
for my own work..

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: ugly but necessary platform define

[Mark Hatle]

Ralf S. Engelschall wrote:
 On Tue, Jul 17, 2007, Peter O'Gorman wrote:
 
 On Tue, 2007-07-17 at 09:37 -0500, Peter O'Gorman wrote:
 On Tue, 2007-07-17 at 10:19 +0200, Ralf S. Engelschall wrote:
 Well, I do have to ask why you're using  here, I think it
 ought to be
 http://www.gnu.org/software/autoconf/manual/autoconf.html#Specifying-Names.

 Not that they will be different :)

 Help! Something ate target_os and host_os. I'll quote the $ with \, and
 see if it gets eaten again.

 Well, I do have to ask why you're using \$target_os here, I think it
 ought to be \$host_os.
 
 Ops, you're right. Yes, has to be fixed. Too much copy  paste from
 other parts of configure.ac (which are similarily wrong as far as I
 see it). I'll review the other cases soon, too.

Actually I was just going to ask the same.  What are we identifying
here.. the OS that the software is going to run on?  or the OS that the
software is being compiled on?  If it's the former target_os is correct,
if its the later then it's correct as it appears to be checked in now.

--Mark

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM 5 status quo

[Mark Hatle]

Jeff Johnson wrote:
 
 On Jul 17, 2007, at 12:10 PM, Mark Hatle wrote:
 
 Jeff Johnson wrote:

 On Jul 17, 2007, at 11:37 AM, Mark Hatle wrote:
 *) file based provide and requires (again WR is not using this... yet)..
 but having a mechanism to define dependencies on individual files vs on
 a per package basis is useful to us.

 
 There is no known usage case for manually (and likely erroneously)
 specified
 file dependencies. You are the only person -- EVER -- to ask for the
 facilty, and
 your embedded needs are significantly different than what is useful for
 a multlib distro.

The usage case is that we have mechanisms to install subsets of
packages.  So we need to regenerate the dependencies based on the
per-file dependencies that are already present in RPM.

Occasionally, we have the need to specify an individual dependency that
was not automatically identified on a per-file basis.  (This is _RARE_!)
 But either the items need to be in .spec file/and thus binary RPM.. or
an external file..  I really loath the idea of the external file to
track this because there is no way to sync or update when the binary RPM
changes.

One simple example is glibc... libc requires /etc/libnss.conf..
/etc/libnss.conf (depending on the configuration) requires one or more
libnss libraries..  none of this can be automatically determined today.

(The usual case I saw at MV who was actively using this patch, was to
specify a given file required a given dlopened library and/or
configuration file.)

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/lib/ depends.c

[Mark Hatle]

Jeff Johnson wrote:
 
 On Jul 17, 2007, at 4:29 PM, Mark Hatle wrote:
 

 I agree, doesn't make sense to have multiple occurrences, but I need to
 be able to change the location based on run-time path. :P

 
 Why? You plain and simply shouldn't be using per-system configuration paths
 if the paths need to change based on run-time path. Use a different
 run-time
 configuration path instead.

That is EXACTLY what I'm trying to do.  Use a different run-time
configuration path.

The way our system works:

We ship prebuilt tools to enable cross-compiled Linux OS creation.
These prebuilt tools include build system components, RPM, cross
compilers, etc.

The user configures their build environment with all of the information
for the system they're going to build.  The result of this is a series
of RPM target configurations (one for each multilib supported on the
platform), as well as a platform file identifying the platform.

These files are now static to the configuration for that cross-compiled
Linux OS.

The need for the run-time relocation is that rpm is a tool provided as
part of the build system.  Once configured.. the run-time relocation is
now set and all of the paths are known.

Once the system is built, a filesystem is generated using rpm's --root=
ability (and fakechroot  fakeroot).  Resulting in a filesystem in an
NFS mountable image.

Does this make more sense on how I'm using rpm for cross compiling?

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: rpm interoperability

[Mark Hatle]

Thomas Lotterer wrote:
 I tried to understand the view points of all authors in recent
 discussions on this list regarding the developer build environment,
 handling of extra metadata, hooks and calls to external helper
 applications and support for embedded script languages. I failed.
 In the vast majority of cases I end up with a heretic question:

My expectations:

 Do you expect a source or binary RPM created for one distro
 will or should work when used unmodified with another distro?

A package created for a distribution may not work unmodified on another
distribution.

However, for application created outside of the distribution space using
reasonable practices, it should be possible to create a package that
will run on multiple distributions.  (Think LSB as an example.)

So we really have two problem sets.  Distro creators and external
software vendors.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES Makefile.am configure.ac macros.in

[Mark Hatle]

The other values you added to popt and such.. if we could macrotize them
all it would help me and my run-time relocation issues.. (then I can do
it with a simple wrapper.)

The only real issue w/ macrotizing everything is some of these values
are used before the macros are set.  So I'm not yet sure how to deal
with that in a general case.

So the values that I am using in a modifiable way are:

/etc/rpm/sysinfo - %{_etcrpm}/sysinfo
LOCALEDIR
RPMPOPTFILE
_usrlibrpm
_etcrpm
/etc/rpm/platform - %{_etcrpm}/platform

--Mark

Ralf S. Engelschall wrote:
 On Mon, Jul 16, 2007, Mark Hatle wrote:
 
 I'm not sure how we can cleanup the rpm/Makefile.am fragment (noted
 below), to allow us to pass --with-path-database=%{foo}/rpm.  (I need
 to be able to do that for various reasons)

 Any suggestions?
 
 Ah, I see what you want to achieve. I'll try to come up with a solution
 for this use case once I've finished my work on the build RPM fully
 standalone and in parallel stuff...
 
Ralf S. Engelschall
[EMAIL PROTECTED]
www.engelschall.com
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Import /usr/lib/rpm/check-buildroot from cvs.rpm.org?

[Mark Hatle]

I think this is a good check to add.  Basically what it does is check
the files for embedded references to the buildroot location vs the
expected install location.

--Mark

Robert Scheck wrote:
 On Mon, 16 Jul 2007, Jason Corley wrote:
 So being far too lazy to look on my own, what is
 /usr/lib/rpm/check-buildroot supposed to accomplish?  URL to [EMAIL 
 PROTECTED]
 mercurial repo ok if need be.
 
 Sorry, forgot this. And I'm to lazy to look for repo link, just attaching
 from localhost ;-)
 
 
 Robert
 
 
 
 
 #! /bin/sh
 
 # Copyright (C) 2004 Enrico Scholz [EMAIL PROTECTED]
 #  
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation; version 2 of the License.
 #  
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #  
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
 
 
 test -z $QA_SKIP_BUILD_ROOT || exit 0
 
 if [ -z $RPM_BUILD_ROOT -o $RPM_BUILD_ROOT = / ]; then
 exit 0
 fi
 
 tmp=$(mktemp /tmp/cbr.XX)
 trap rm -f $tmp EXIT
 
 find $RPM_BUILD_ROOT \! \( \
 -name '*.pyo' -o -name '*.pyc' -o -name '*.elc' -o -name '.packlist' \
 -o -name '*.jar.so.debug' \) -type f -print0 | \
 LANG=C xargs -0r grep -F $RPM_BUILD_ROOT $tmp
 
 test -s $tmp  {
 cat $tmp
 echo $Found '$RPM_BUILD_ROOT' in installed files; aborting
 exit 1
 } || :

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

One more %patch oddity

[Mark Hatle]

Ok, I'm not sure how to resolve this other then in the specfile.

The vim package (from Fedora Core) is doing:

PATCH001: foo

%patch001

This is failing with:

cat: %PATCH001: No such file or directory

Any suggestions?  (Any easy way to trim leading zeros?)

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Configuring rpm for cross compilation

[Mark Hatle]

I just noticed the macro files was getting host information instead of
target information.  Attached is a patch to change references from build
to target.

Since I'm not the autoconf wizard, I didn't want to commit this until it
was reviewed.

Ralf?

--Mark
Index: configure.ac
===
RCS file: /v/rpm/cvs/rpm/configure.ac,v
retrieving revision 2.178
diff -u -r2.178 configure.ac
--- configure.ac15 Jul 2007 21:50:16 -  2.178
+++ configure.ac16 Jul 2007 02:31:55 -
@@ -447,10 +447,10 @@
 esac
 
 dnl # FIXME: platform-hack: SCO and SunOS 4.x
-if echo $build_os | grep sco  /dev/null; then
+if echo $target_os | grep sco  /dev/null; then
AC_DEFINE(NEED_STRINGS_H, 1, [Define as one if we need to include 
strings.h (along with string.h)])
AC_DEFINE(HAVE_STRUCT_MNTTAB, 1, [Define as 1 if you have struct 
mnttab (only sco?)])
-elif echo $build_os | grep sunos  /dev/null; then
+elif echo $target_os | grep sunos  /dev/null; then
CFLAGS=$CFLAGS -D__USE_FIXED_PROTOTYPES__
AC_DEFINE(NEED_STRINGS_H, 1, [Define as one if we need to include 
strings.h (along with string.h)])
AC_DEFINE(NEED_MYREALLOC, 1, [Define as 1 if we need myrealloc])
@@ -901,38 +901,38 @@
 AC_SUBST(ROOT_GROUP)
 
 dnl # provide platform-specific find-provides and find-requires script 
symlinks
-if echo $build_os | grep '.*-gnulibc1'  /dev/null ; then
-   build_os=`echo ${build_os} | sed 's/-gnulibc1$//'`
+if echo $target_os | grep '.*-gnulibc1'  /dev/null ; then
+   target_os=`echo ${target_os} | sed 's/-gnulibc1$//'`
 fi
-if echo $build_os | grep '.*-gnu'  /dev/null ; then
-   build_os=`echo ${build_os} | sed 's/-gnu$//'`
+if echo $target_os | grep '.*-gnu'  /dev/null ; then
+   target_os=`echo ${target_os} | sed 's/-gnu$//'`
 fi
 changequote(, )
-build_os_exact=${build_os}
-build_os_major=`echo ${build_os} | sed 's/\..*$//'`
-build_os_noversion=`echo ${build_os} | sed 's/[0-9]*\..*$//'`
+target_os_exact=${target_os}
+target_os_major=`echo ${target_os} | sed 's/\..*$//'`
+target_os_noversion=`echo ${target_os} | sed 's/[0-9]*\..*$//'`
 changequote([, ])
 rm -f ./find-provides
-if test -f ${srcdir}/autodeps/${build_cpu}-${build_os_exact}.prov ; then
-ln -s ${srcdir}/autodeps/${build_cpu}-${build_os_exact}.prov 
./find-provides
-elif test -f ${srcdir}/autodeps/${build_os_exact}.prov ; then
-ln -s ${srcdir}/autodeps/${build_os_exact}.prov ./find-provides
-elif test -f ${srcdir}/autodeps/${build_os_major}.prov ; then
-ln -s ${srcdir}/autodeps/${build_os_major}.prov ./find-provides
-elif test -f ${srcdir}/autodeps/${build_os_noversion}.prov ; then
-ln -s ${srcdir}/autodeps/${build_os_noversion}.prov ./find-provides
+if test -f ${srcdir}/autodeps/${target_cpu}-${target_os_exact}.prov ; then
+ln -s ${srcdir}/autodeps/${target_cpu}-${target_os_exact}.prov 
./find-provides
+elif test -f ${srcdir}/autodeps/${target_os_exact}.prov ; then
+ln -s ${srcdir}/autodeps/${target_os_exact}.prov ./find-provides
+elif test -f ${srcdir}/autodeps/${target_os_major}.prov ; then
+ln -s ${srcdir}/autodeps/${target_os_major}.prov ./find-provides
+elif test -f ${srcdir}/autodeps/${target_os_noversion}.prov ; then
+ln -s ${srcdir}/autodeps/${target_os_noversion}.prov ./find-provides
 else
 ln -s ${srcdir}/autodeps/none ./find-provides
 fi
 rm -f ./find-requires
-if test -f ${srcdir}/autodeps/${build_cpu}-${build_os_exact}.req ; then
-ln -s ${srcdir}/autodeps/${build_cpu}-${build_os_exact}.req ./find-requires
-elif test -f ${srcdir}/autodeps/${build_os_exact}.req ; then
-ln -s ${srcdir}/autodeps/${build_os_exact}.req ./find-requires
-elif test -f ${srcdir}/autodeps/${build_os_major}.req ; then
-ln -s ${srcdir}/autodeps/${build_os_major}.req ./find-requires
-elif test -f ${srcdir}/autodeps/${build_os_noversion}.req ; then
-ln -s ${srcdir}/autodeps/${build_os_noversion}.req ./find-requires
+if test -f ${srcdir}/autodeps/${target_cpu}-${target_os_exact}.req ; then
+ln -s ${srcdir}/autodeps/${target_cpu}-${target_os_exact}.req 
./find-requires
+elif test -f ${srcdir}/autodeps/${target_os_exact}.req ; then
+ln -s ${srcdir}/autodeps/${target_os_exact}.req ./find-requires
+elif test -f ${srcdir}/autodeps/${target_os_major}.req ; then
+ln -s ${srcdir}/autodeps/${target_os_major}.req ./find-requires
+elif test -f ${srcdir}/autodeps/${target_os_noversion}.req ; then
+ln -s ${srcdir}/autodeps/${target_os_noversion}.req ./find-requires
 else
 ln -s ${srcdir}/autodeps/none ./find-requires
 fi
@@ -941,7 +941,7 @@
 autorelocate_path='%{nil}'
 autorelocate_dcolor='0'
 RPMCANONCOLOR=3
-case ${build_cpu} in
+case ${target_cpu} in
 *86)   RPMCANONARCH=i386 ;;
 ia32e*)RPMCANONARCH=ia32e ;;
 amd64*)RPMCANONARCH=amd64 ;;
@@ -959,17 +959,17 @@
 armv3l*)   RPMCANONARCH=armv3l ;;
 armv4l*)   RPMCANONARCH=armv4l ;;
 armv4b*)   

Re: [CVS] RPM: rpm/ CHANGES rpm/rpmio/ macro.c

[Mark Hatle]

Jeff Johnson wrote:
 This patch is pretty close but is not expanding the parameterized %patch
 correctly after the rewrite.
 
 Specifically -b .foo is not becoming -b --suffix .foo with
 the rewrite %patchNN - %patch -P NN.

What should be responsible for the -b .foo - -b --suffix .foo?

I don't believe it's possible to do that in the parser, so it would have
to be done later where the %patch is actually evaluated into the
corresponding command.

 There's an extra level of recursion involved with the rewrite, so its
 likely that
 the auto clean up of parameters is causing the parameterized options to
 disappear.

I don't see a simple implementation for this, so should I back
out/disable by change for now?

--Mark

 Hmmm ...
 
 73 de Jeff
 
 On Jul 12, 2007, at 10:11 PM, Mark Hatle wrote:
 
   RPM Package Manager, CVS Repository
   http://rpm5.org/cvs/
  
 


   Server: rpm5.org Name:   Mark Hatle
   Root:   /v/rpm/cvs   Email:  [EMAIL PROTECTED]
   Module: rpm  Date:   13-Jul-2007 04:11:31
   Branch: HEAD Handle: 2007071303113001

   Modified files:
 rpm CHANGES
 rpm/rpmio   macro.c

   Log:
 Transform %patchNN to %patch -P NN

 Based on suggestions from jbj.

 This should allow anyone to override the way the patch operations by
 supplying %patch as a macro!

   Summary:
 RevisionChanges Path
 1.1457  +1  -0  rpm/CHANGES
 2.122   +18 -0  rpm/rpmio/macro.c
  
 


   patch -p0 '@@ .'
   Index: rpm/CHANGES
  
 

   $ cvs diff -u -r1.1456 -r1.1457 CHANGES
   --- rpm/CHANGES12 Jul 2007 22:25:30 -1.1456
   +++ rpm/CHANGES13 Jul 2007 02:11:31 -1.1457
   @@ -1,4 +1,5 @@
4.5 - 5.0:
   +- mgh: transform %patchNN to %patch -P NN
- jbj: add mono automagic dependency extractors.
- jbj: insure that RPMFILE_EXISTS is set for %config files.
- jbj: revert the _GetPass() hack-a-round.
   @@ .
   patch -p0 '@@ .'
   Index: rpm/rpmio/macro.c
  
 

   $ cvs diff -u -r2.121 -r2.122 macro.c
   --- rpm/rpmio/macro.c12 Jul 2007 02:02:34 -2.121
   +++ rpm/rpmio/macro.c13 Jul 2007 02:11:30 -2.122
   @@ -1212,6 +1212,11 @@
} else if (STREQ(F, f, fn)) {
b = buf + strlen(buf) + 1;
sprintf(b, file%s.file, buf);
   +} else if (fn  5  STREQ(patch, f, fn = 5 ? 5 : fn)) {
   +strncpy(buf, f, fn);
   +buf[fn]='\0';
   +b = buf + fn + 1;
   +sprintf(b, %%patch -P %s, buf+5);
}

if (b) {
   @@ -1494,6 +1499,19 @@
mep = findEntry(mb-mc, f, fn);
me = (mep ? *mep : NULL);

   +/* XXX necessary but clunky
   + * We have to do this after the findEntry because there may be a
   + * macro such as %patchversion, if it evaluates ignore it!
   + */
   +if (me == NULL 
   + (fn  5  STREQ(patch, f, fn = 5 ? 5 : fn))) {
   +/[EMAIL PROTECTED]@*/ /* FIX: verbose may be set */
   +doFoo(mb, negate, f, fn, g, gn);
   +/[EMAIL PROTECTED]@*/
   +s = se;
   +continue;
   +}
   +
/* XXX Special processing for flags */
if (*f == '-') {
if (me)
   @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repository[EMAIL PROTECTED]
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/build/ parsePreamble.c rpm/lib/ rpmfc.c

[Mark Hatle]

Ralf S. Engelschall wrote:
 On Sat, Jul 14, 2007, Jeff Johnson wrote:
 
 [...]
 I don't agree, but I do not wish to discuss further. The cost of
 the discussion is already higher than the cost of the preemptive
 implementation.

 Lusers asked for the implementation, now deal with the consequences of
 what was requested. Arbitrary means exactly that, unspecified and
 unknown, typo's are not an rpm problem, nor will naming conventions
 solve the typo problem, consider
 X-Reqiures: foo
 Note swapped 'i' and 'u'.
 
 Ok, but even if provide _BY DEFAULT_ the possibility for _arbitrary_
 tags, I still think it is reasonable to nevertheless provide a mechanism
 so a distribution vendor can switch from arbitrary tags to fixed set
 of extra tags. My suggestion of using %_extra_tags * as the default
 in macros and using %_extra_tags Foo:Bar:Quux:X-* for a particular
 distribution vendor would provide this. I still think this approach
 could make both parties happy...

I'm inclined to agree w/ Ralf.  I have no desire to allow anyone here to
use these extra tags.  If a customer wants to use them, so be it.. but
here I want the old behavior of no extra tags.. (Easy enough to re-add
via a patch..  but it should be fairly simple to implement the
%_extra_tags as well...

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ macros.in

[Mark Hatle]

Jeff Johnson wrote:
 Thanks for the %patchNN rewrite. Holler if %patch breaks.

I'm about to run it through another build sanity check.. (thats how I
noticed the '\' was missing suddenly nothing was patched..)

--Mark

 On Jul 14, 2007, at 7:03 PM, Mark Hatle wrote:
 
   RPM Package Manager, CVS Repository
   http://rpm5.org/cvs/
  
 


   Server: rpm5.org Name:   Mark Hatle
   Root:   /v/rpm/cvs   Email:  [EMAIL PROTECTED]
   Module: rpm  Date:   15-Jul-2007 01:03:09
   Branch: HEAD Handle: 2007071500030900

   Modified files:
 rpm macros.in

   Log:
 Fix missing \

   Summary:
 RevisionChanges Path
 1.174   +2  -2  rpm/macros.in
  
 


   patch -p0 '@@ .'
   Index: rpm/macros.in
  
 

   $ cvs diff -u -r1.173 -r1.174 macros.in
   --- rpm/macros.in14 Jul 2007 20:58:04 -1.173
   +++ rpm/macros.in14 Jul 2007 23:03:09 -1.174
   @@ -1,7 +1,7 @@
#/*! \page config_macros Default configuration: @USRLIBRPM@/macros
# \verbatim
#
   -# $Id: macros.in,v 1.173 2007/07/14 20:58:04 jbj Exp $
   +# $Id: macros.in,v 1.174 2007/07/14 23:03:09 fray Exp $
#
# This is a global RPM configuration file. All changes made here will
# be lost when the rpm package is upgraded. Any per-system
 configuration
   @@ -389,7 +389,7 @@

%patch(b:p:P:REz:F:d:) \
%define patch_file%{P:%{-P:%{-P*}}%{!-P:%%PATCH0}} \
   -%define patch_suffix%{-b:-b --suffix %{-b*}}%{-z:--suffix %{-z*}}
   +%define patch_suffix%{-b:-b --suffix %{-b*}}%{-z:--suffix
 %{-z*}} \
  echo Patch #%{-P:%{-P*}}%{!-P:0} (%{basename:%patch_file}): \
  %{uncompress:%patch_file} | %{__patch} -s %{-p:-p%{-p*}}
 %{-F:-F%{-F*}} %{-d:-d%{-d*}} %patch_suffix %{-R} %{-E}

   @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repository[EMAIL PROTECTED]
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [Suspected Spam][Blocked Sender]Re: [CVS] RPM: rpm/scripts/ mono-find-provides

[Mark Hatle]

I'll leave that to someone else.. probably should file a defect in the
tracker on this.  I just copied it wholesale so I could build HEAD again.

--Mark

Arkadiusz Miskiewicz wrote:
 On Friday 13 of July 2007, Mark Hatle wrote:
 
  
 ===
 = $ cvs diff -u -r0 -r1.1 mono-find-provides
   --- /dev/null  2007-07-13 04:56:02 +0200
   +++ mono-find-provides 2007-07-13 04:57:03 +0200
   @@ -0,0 +1,42 @@
   +#!/bin/bash
   +#
 
 
 Ehem, could you fix that script to be POSIX shell compatibile? I don't even 
 want to have bash on my system 8-)
 

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Network transport for rpm depsolving

[Mark Hatle]

My preference is external helpers.  I'm not sure if curl, wget, and/or
rsync is the right interface though.

I'd like to see something that goes out and says I need the following
information, and then the external interface can then use curl, wget,
rsync, neon, or something custom to do what it wants.  (The default
implemented should be http of course with whatever the standard
protocols decided are..)  but I'd like this modular.

--Mark

Jeff Johnson wrote:
 On the feature set of rpm-5.0 is an integrated dependency solver.
 
 A dependency solver needs information about the package universe.
 
 The information is time sensitive, and is usually not resident on the
 local client.
 
 Which means that an integrated dependency solver must undertake transport
 of the information necessary to perform depsolving.
 
 Since network transport within rpm (I am not the one who added FTP
 transport
 to rpm) has always been controversial, its time to start the discussions
 now.
 
 The end-point of the discussion will be a choice of network transport or
 a feature reversion for the rpm-5.0 roadmap. No other choice is possible
 imho.
 
 The protocol choice is HTTP, duh. No other protocol traverses firewalls.
 
 The implementation choice is either internal through rpmio, or external
 through a curl/wget/rsync helper invocation.
 
 If internal rpmio is used (my preference) then neon becomes mandatory
 and the linking decisions move to whether https through openssl/gnutls
 is necessary underneath neon.
 
 If external helper is chosen, then the ufdio layer in rpmio becomes
 optional
 and external helpers are what will be used for future rpm development.
 
 What say ye?
 
 73 de Jeff
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

mono-find-provides is missing

[Mark Hatle]

The mono-find-provides file appears to have been missed in the previous
check in.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: librpmmisc approach for handling shared external libraries doesn't look sane

[Mark Hatle]

Arkadiusz Miskiewicz wrote:
 Try this:
 ./configure LDFLAGS=-Wl,--as-needed 
 
 It will fail miserably because things are not linked properly.
 
 Do we actually need separate librpmXYZ for anything?
 
 Doing single librpm.so would solve the problem.

quick summary of the discussion we had on #rpm about this.

The key problem here is that librpm* all require similar elements, such
as neon.  These elements if linked to a shared library work fine because
the run-time linker resolves duplicates and only loads one copy.

However, if an element (like neon) is statically linked, we have to make
sure we only load one copy of the item, otherwise the multiple copies
would get out of sync with the way the various librpm elements pass data
around.

So this all comes down to:

if librpm* has a requirement on librpmmisc, and librpmmisc requires the
libraries (static or shared) it works w/o --as-needed.

--as-needed however, does not recursively verify linkage, so when used
it fails because the common elements are not provided directly by
librpmmisc, but by it's requirements.

I think I captured the important bits.  In this case, if --as-needed is
supposed to work.. one size doesn't fit all..

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPMTAG_FILEBUILDID?

[Mark Hatle]

Jeff Johnson wrote:
 I suspect that at least for elf, RPMTAG_FILEBUILDID should be added to
 package
 metadata, if for no other reason, so that the 2 proposed symlinks can be
 created as
 a side effect of installing the file without the additional files being
 added to all packages.
 
 Are non-elf file build instance identifiers worth pursuing as well?

There are a few cases I'd like them.. but I've found that falling back
on md5sum and/or sha1 seems to resolve any issues.. I just have to have
those identifiers associated with a given build.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: Manifest not working

[Mark Hatle]

My manifest has 500 entries of full paths about 45 characters each.. the
problem was as I said.. it got the ENOENT and skipped the manifest
loading.  I removed that check and now it works.

I just haven't been able to track down where the ENOENT is set.

Attached is the workaround patch I'm currently using...

--Mark

Michael Schroeder wrote:
 On Tue, Jul 03, 2007 at 04:04:22PM -0500, Mark Hatle wrote:
 The manifest files don't seem to be working for me.  But I can't debug
 rpm due to it being stripped.
 
 A manifest file needs to be at least 96 bytes long (the size of
 the rpm lead). Maybe yours is too short?
 
 Cheers,
   Michael.
 

When working with a manifest, somehow errno is set to ENOENT.  Disable 
the optimization, and always look for a manifest file.
Index: rpm5-20070703/lib/rpmgi.c
===
--- rpm5-20070703.orig/lib/rpmgi.c  2007-05-25 12:36:02.0 -0500
+++ rpm5-20070703/lib/rpmgi.c   2007-07-03 16:33:55.0 -0500
@@ -204,8 +204,10 @@
 
if (rpmrc == RPMRC_OK || gi-flags  RPMGI_NOMANIFEST)
break;
+#if 0
if (errno == ENOENT)
break;
+#endif
 
/* Not a header, so try for a manifest. */
gi-argv[gi-i] = NULL; /* Mark the insertion point */

Re: Berkley DB errors w/ HEAD

[Mark Hatle]

Jeff Johnson wrote:
 
 On Jul 4, 2007, at 3:54 AM, Mark Hatle wrote:
 
 I am getting the following error after using rpm to install source
 packages:

 
 Is the srpm signed?
 
 Add --nosignature to not access a rpmdb installing a src.rpm.
 
 Alternatively, try importing the pubkey.
 
 The above fixes nothing of course, but may permit you
 to get farther along. Installing a srpm ain't the place to debug
 rpmdb problems, as an rpmdb is hardly needed for installing
 srpm's.

Bingo you figured out what was happening.  The SRPM was signed.  I'll
add in the --nosignature when this build is over..

At least I should get points for the most unique use case in finding a
bug in the rpmdb.  Installing a signed source package w/ an empty
database.  :P

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/lib/ rpmgi.c

[Mark Hatle]

Mark Hatle wrote:
 Jeff Johnson wrote:
 Actually I think this patch is unneeded.

 I do believe Mark has a typo in his manifest that he needs to correct
 before we starting mucking about with rpmgi behavior.

Wait.. the system (due to the ENOENT) never got to the point of loading
the manifest.. so a typo there can't be the problem.  The ENOENT showed
up earlier when it was trying to load the manifest file as a package.
Something in there causes ENOENT to kick off.

I just haven't had the time to try to track this down.  (gdb can't trace
errno anymore AFAIK.. which makes this even more difficult to diagnose.)

--Mark

 If having a typo in the manifest (on a query) causes a failure, this is
 new behavior.  The commands I am executing are the same as my old rpm
 4.4.2.  It worked in rpm 4.4.2.
 
 --Mark
 
 73 de Jeff


 On Jul 4, 2007, at 12:16 PM, Ralf S. Engelschall wrote:

   RPM Package Manager, CVS Repository
   http://rpm5.org/cvs/
  
 


   Server: rpm5.org Name:   Ralf S. Engelschall
   Root:   /v/rpm/cvs   Email:  [EMAIL PROTECTED]
   Module: rpm  Date:   04-Jul-2007 18:16:50
   Branch: HEAD Handle: 2007070417165000

   Modified files:
 rpm/lib rpmgi.c

   Log:
 apply the temporary fix for the manifest problems

   Summary:
 RevisionChanges Path
 2.25+3  -0  rpm/lib/rpmgi.c
  
 


   patch -p0 '@@ .'
   Index: rpm/lib/rpmgi.c
  
 

   $ cvs diff -u -r2.24 -r2.25 rpmgi.c
   --- rpm/lib/rpmgi.c25 May 2007 17:36:02 -2.24
   +++ rpm/lib/rpmgi.c4 Jul 2007 16:16:50 -2.25
   @@ -204,8 +204,11 @@

if (rpmrc == RPMRC_OK || gi-flags  RPMGI_NOMANIFEST)
break;
   +#if 0
   +/* FIXME (see http://rpm5.org/community/rpm-devel/0523.html) */
if (errno == ENOENT)
break;
   +#endif

/* Not a header, so try for a manifest. */
gi-argv[gi-i] = NULL;/* Mark the insertion point */
   @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repository[EMAIL PROTECTED]
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/lib/ rpmgi.c

[Mark Hatle]

Jeff Johnson wrote:
 Actually I think this patch is unneeded.
 
 I do believe Mark has a typo in his manifest that he needs to correct
 before we starting mucking about with rpmgi behavior.

If having a typo in the manifest (on a query) causes a failure, this is
new behavior.  The commands I am executing are the same as my old rpm
4.4.2.  It worked in rpm 4.4.2.

--Mark

 73 de Jeff
 
 
 On Jul 4, 2007, at 12:16 PM, Ralf S. Engelschall wrote:
 
   RPM Package Manager, CVS Repository
   http://rpm5.org/cvs/
  
 


   Server: rpm5.org Name:   Ralf S. Engelschall
   Root:   /v/rpm/cvs   Email:  [EMAIL PROTECTED]
   Module: rpm  Date:   04-Jul-2007 18:16:50
   Branch: HEAD Handle: 2007070417165000

   Modified files:
 rpm/lib rpmgi.c

   Log:
 apply the temporary fix for the manifest problems

   Summary:
 RevisionChanges Path
 2.25+3  -0  rpm/lib/rpmgi.c
  
 


   patch -p0 '@@ .'
   Index: rpm/lib/rpmgi.c
  
 

   $ cvs diff -u -r2.24 -r2.25 rpmgi.c
   --- rpm/lib/rpmgi.c25 May 2007 17:36:02 -2.24
   +++ rpm/lib/rpmgi.c4 Jul 2007 16:16:50 -2.25
   @@ -204,8 +204,11 @@

if (rpmrc == RPMRC_OK || gi-flags  RPMGI_NOMANIFEST)
break;
   +#if 0
   +/* FIXME (see http://rpm5.org/community/rpm-devel/0523.html) */
if (errno == ENOENT)
break;
   +#endif

/* Not a header, so try for a manifest. */
gi-argv[gi-i] = NULL;/* Mark the insertion point */
   @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repository[EMAIL PROTECTED]
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/lib/ rpmgi.c

[Mark Hatle]

Reproducer.  Build rpm5 HEAD as of about 15 minutes ago...

(remove the workaround for the ENOENT)

rpm5/HEAD/rpm/configure --disable-nls --with-libelf --without-selinux
--without-perl --without-python --with-zlib --with-bzip2 --with-beecrypt
--without-neon --with-file=internal --with-popt --with-db=internal
--with-lua=internal --without-apidocs

thats my configure line..

have a big directory of packages somewhere..

find dir -type f  manifest

./rpm -qp --qf %{NAME}\n manifest

that will result in nothing.. if you trace w/ gdb you'll see it gets
to the ENOENT check and drops out.

--Mark

Jeff Johnson wrote:
 
 On Jul 4, 2007, at 12:28 PM, Mark Hatle wrote:
 
 Mark Hatle wrote:
 Jeff Johnson wrote:
 Actually I think this patch is unneeded.

 I do believe Mark has a typo in his manifest that he needs to correct
 before we starting mucking about with rpmgi behavior.

 Wait.. the system (due to the ENOENT) never got to the point of loading
 the manifest.. so a typo there can't be the problem.  The ENOENT showed
 up earlier when it was trying to load the manifest file as a package.
 Something in there causes ENOENT to kick off.

 I just haven't had the time to try to track this down.  (gdb can't trace
 errno anymore AFAIK.. which makes this even more difficult to diagnose.)

 
 ;-)
 
 Show me the reproducer and I'll give you the fix.
 
 73 de Jeff
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/lib/ rpmgi.c

[Mark Hatle]

The only thing I can think of is it's either configuration or OS
dependent.  I'm running FC7 currently.

--Mark

Jeff Johnson wrote:
 
 On Jul 4, 2007, at 12:52 PM, Mark Hatle wrote:
 
 Reproducer.  Build rpm5 HEAD as of about 15 minutes ago...

 (remove the workaround for the ENOENT)

 rpm5/HEAD/rpm/configure --disable-nls --with-libelf --without-selinux
 --without-perl --without-python --with-zlib --with-bzip2 --with-beecrypt
 --without-neon --with-file=internal --with-popt --with-db=internal
 --with-lua=internal --without-apidocs

 thats my configure line..

 have a big directory of packages somewhere..

 find dir -type f  manifest

 ./rpm -qp --qf %{NAME}\n manifest

 that will result in nothing.. if you trace w/ gdb you'll see it gets
 to the ENOENT check and drops out.

 
 [EMAIL PROTECTED] wdj]$ rpm --version# -- w/o ENOENT hackery
 RPM version 5.0
 [EMAIL PROTECTED] wdj]$ cd /var/ftp/pub/ppc-macports1.4/
 [EMAIL PROTECTED] ppc-macports1.4]$ find . -type f  /tmp/manifest
 [EMAIL PROTECTED] ppc-macports1.4]$ rpm -qp /tmp/manifest
 ...
 long list of package NVR deleted
 
 The pkgs used are online at http://wraptastic.org/pub/ppc-macports1.4
 
 73 de Jeff
 
 
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Internal Dependency generator

[Mark Hatle]

I don't think the internal dependency generator is working in HEAD.

I'm currently tracing through the code trying to understand what is
happening..  as far as I can tell it gets into rpmfcApply and the ddict
is empty.  So it never generated the per file dependencies.

Any pointers on where to look on where it processes each file independently?

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: One more magic bug...

[Mark Hatle]

Jeff Johnson wrote:
 
 On Jul 3, 2007, at 1:09 PM, Mark Hatle wrote:
 
 Jeff Johnson wrote:

 What behavior is expected if the _rpmfc_magic_path is not set, but yet
 rpm was compiled w/ HAVE_MAGIC_H?

 
 UNSPECIFIED ;-)

Ok, I'll go with the pretend there is no magic_h/file available then.
:)  It should be the easiest for me to implement.

 Either we need to asset here (which is what happens if the magic file
 can't be opened in the next step), or we need to continue as if there is
 no magic file available.  Pick a behavior and I'll implement it.  :)

 But
 if (fc-fn != NULL)
 for (fc-ix = 0; fc-fn[fc-ix] != NULL; fc-ix++) {
 might fix the segfault.

 This should be an assert IMHO.  rpmfcApply shouldn't be able to run if
 fc-fn is null.  rpmfcApply becomes a no-op in that case.

 
 assert's just cause luser complaints for something better imho. luser's
 just don't understand ;-)

Only if the assert is hit.. in this case I think it's better then
silently going through rpmfcApply and not doing anything in the event of
an error condition.. but a segfault is as good as anything here.. ;)

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: SQLite compile/exec [was: RPM 5 distribution tarball size?]

[Mark Hatle]

Ralf S. Engelschall wrote:
 On Mon, Jul 02, 2007, Mark Hatle wrote:
 
 Ralf S. Engelschall wrote:
 On Mon, Jul 02, 2007, Jeff Johnson wrote:

 On Jul 2, 2007, at 5:23 AM, Ralf S. Engelschall wrote:
 [...]

 it is time to start the engineering to get
 (*dbiCompile) and (*dbiExec) methods implemented.
 Can you be more specific, please? What exactly
 is missing here for the SQLite RPM DB backend?
 In SQLite we can precompile expressions to make the queries faster.  We
 don't have this implemented because there is no way to Compile and
 Execute the queries except in real time.  In some informal testing
 way back when the initial implementation was done, precompiling and
 executing the queries resulted in a performance increase.  (I don't
 remember if it was significant or not.. but there was an increase.)
 
 Ah, ok, we are talking here about preparing the SQL statements _once_
 and then just executing them _multiple_ times. Well, as you said: it
 can only be prepared under RPM's runtime (and not stored externally for
 reuse). But if the SELECTs are done really a lot of times, how about
 this: add a small _CACHE_ to rpmdb/sqlite.c which is established in
 rpmdb/sqlite.c:sql_open() and destroyed in rpmdb/sqlite.c:sql_open()
 and to which all sqlite3_prepare()'ed statements are stored once and
 taken for all subsequent executions. OTOH as the used SQL statements are
 rather simple ones (no complex queries which need to be optimized by the
 SQL engine), I'm not quite sure whether this _really_ provides a major
 speedup. Or did I still miss a point?

I wish I could remember the specifics.. but it's been a few years now
since I did the original work.  There are actually a surprisingly large
number of selects and inserts when working with a package installation.
 Some query operations might be extensive as well, I don't remember.

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES macros.in

[Mark Hatle]

For the case w/o db3 header being available, there is the db_emu.h
file.  Put that value in there, and it is supposed to be picked up if db
items are not defined.

--Mark

Ralf S. Engelschall wrote:
 On Thu, Jun 28, 2007, Mark Hatle wrote:
 
 Is TEMPORARY TABLE something special?  I knew about the :memory:
 before, but not the other.
 
 Excerpt from http://www.sqlite.org/lang_createtable.html:
 
If the TEMP or TEMPORARY keyword occurs in between CREATE
and TABLE then the table that is created is only visible within
that same database connection and is automatically deleted when the
database connection is closed. Any indices created on a temporary
table are also temporary. Temporary tables and indices are stored in
a separate file distinct from the main database file.
 
 Otherwise this looks good to me.  It may even speed things up slightly.
 
 But before I can commit I at least wish to find a better approach for
 the DB_PRIVATE value. The current hack is not the best since sliced
 bread:
 
 +#if defined(WITH_SQLITE)  !defined(DB_PRIVATE)
 +#define DB_PRIVATE 0x020 /* shameless hack for shared option */
 +#endif
 
Ralf S. Engelschall
[EMAIL PROTECTED]
www.engelschall.com
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: XAR development starts next weekend.

[Mark Hatle]

Jeff Johnson wrote:
 Originally, I planned to do the development on a branch.
 
 However, after seeing recent breakages, I will be doing my
 development on HEAD, just like everyone else.
 
 I will be building rpm with the same stuff that has been in rpm
 for years. That includes internal db, internal lua, internal neon,
 and sqlite.

From my current testing, internal db, internal lua, internal zlib and
internal file are all working ... with external neon and external sqlite.

There doesn't appear to be a way to do internal neon or sqlite at the
moment.  If this is required, it should be a minor patch to the
configure.ac.


__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

db3 configure wrapper

[Mark Hatle]

Any objections to the attached change?

I believe this will work in any POSIX shell.  The idea is instead of
processing all of the arguments and possibly blowing away quoting and
such, we process them one at a time and use shift and set to recreate
the argument list.

In my testing, this fixed the configuration problem I was having.

My configuration is:

CC=ccache gcc \
CFLAGS=-O2 -march=i486 -mtune=i686 -D__USE_STRING_INLINES
-fomit-frame-pointer -pipe
-I/home/mhatle/build/host-tools/host-cross/include \
CXX=ccache g++ \
CXXFLAGS=-O2 -march=i486 -mtune=i686 -D__USE_STRING_INLINES -fno-rtti
-fno-implicit-templates -pipe \
CXXCPP=g++ -E \
CPP=gcc -E \
LD=ld \
LDFLAGS=-s -L/home/mhatle/build/host-tools/host-cross/lib \
AR=ar \
AS=as \
NM=nm \
OBJCOPY=objcopy \
OBJDUMP=objdump \
RANLIB=ranlib \
STRIP=strip \
/home/mhatle/build/host-tools/build-tools/rpm5-20070629/configure \
--cache-file=wrs_config_cache \
--prefix=/home/mhatle/build/host-tools/host-cross \
--build=i686-pc-linux-gnu --disable-nls --without-python \
--without-perl --without-selinux \
--without-apidocs

This results in:

configure: running /bin/sh ./configure
'--prefix=/home/mhatle/build/host-tools/h
ost-cross'  '--build=i686-pc-linux-gnu' '--disable-nls'
'--without-python' '--wi
thout-perl' '--without-selinux' '--without-apidocs'
'build_alias=i686-pc-linux-g
nu' 'CC=ccache gcc' 'CFLAGS=-O2 -march=i486 -mtune=i686
-D__USE_STRING_INLINES -
fomit-frame-pointer -pipe
-I/home/mhatle/build/host-tools/host-cross/include' 'L
DFLAGS=-s -L/home/mhatle/build/host-tools/host-cross/lib' 'CPP=gcc -E'
'CXX=ccac
he g++' 'CXXFLAGS=-O2 -march=i486 -mtune=i686 -D__USE_STRING_INLINES
-fno-rtti -
fno-implicit-templates -pipe' 'CXXCPP=g++ -E'
--cache-file=../wrs_config_cache -
-srcdir=.
configure: WARNING: you should use --build, --host, --target
configure: WARNING: invalid host type: g++
checking build system type... i686-pc-linux-gnu
checking host system type... Invalid configuration `g++': machine `g++'
not reco
gnized
configure: error: /bin/sh ./../db/dist/config.sub g++ failed
mv: cannot stat `Makefile': No such file or directory
cat: Makefile.orig: No such file or directory
mv: cannot stat `db.h': No such file or directory
cat: db.h.orig: No such file or directory



--Mark
Index: CHANGES
===
RCS file: /v/rpm/cvs/rpm/CHANGES,v
retrieving revision 1.1418
diff -u -r1.1418 CHANGES
--- CHANGES 29 Jun 2007 20:23:06 -  1.1418
+++ CHANGES 29 Jun 2007 20:57:53 -
@@ -1,4 +1,5 @@
 4.5 - 5.0:
+- mgh: change db3 configure wrapper to process one argument at a time
 - mgh: allow rpm to be built w/o neon
 - rse: remove unused rpmio/sexp/ sub-library
 - rse: add errno information to be more informative in error message 
when lstat(2) on verification fails.
Index: db3/configure
===
RCS file: /v/rpm/cvs/rpm/db3/configure,v
retrieving revision 1.32
diff -u -r1.32 configure
--- db3/configure   27 Jun 2007 21:18:48 -  1.32
+++ db3/configure   29 Jun 2007 20:57:53 -
@@ -7,7 +7,12 @@
 
 rm -f config.cache
 
-ARGS=`echo $* |\
+# We iterate over the argument list.  Processing the arguments using 
+# echo $* can fail with the sed chopping out parts of unrelated 
+# arguments
+set - $@ eoa_marker
+while [ $1 != eoa_marker ]; do
+   ARG=`echo $1 | \
sed -e 's% [^ ]*CFLAGS=[^ ]*%%' \
-e 's% [^ ]*CC=[^ ]*ccache [^ ]*%%' \
-e 's% [^ ]*CC=[^ ]*%%' \
@@ -24,6 +29,11 @@
-e 's%--with-db-mutex%%' \
-e 's%--without-db-mutex%%' \
 `
+   shift
+   set - $@ $ARG
+done
+shift
+
 
 #   NOTICE:
 #   1. --enable-shared is there for enforcing GNU libtool usage only

Re: FYI: migrating the RPM DB between Berkeley-DB and SQLite

[Mark Hatle]

I'd have to double check this in the code, but if the __dbabi is 0 (or
maybe it was -1), rpm will work w/ both sqlite and db databases..
basically it'll try to open it as one, if it can't it'll fall back to
the other format.  (I don't remember the order it looks in.)

--Mark

Ralf S. Engelschall wrote:
 As it is AFAIK not documented anyhwere and also far away from being
 obvious, here is a short tutorial for the RPM hackers on how to migrate
 the RPM DB of RPM 5 between Berkeley-DB and SQLite.
 
 The obvious pre-requisite for an in-place migration obviously is that
 RPM 5 was built with _both_ Berkeley-DB _and_ SQLite, of course. In our
 newer RPM 5 world-order this is just a matter of having SQLite installed
 under /path/to/sqlite and then build RPM with the Autoconf option set
 --with-db=internal --with-sqlite=/path/to/sqlite
 
 After you have this multi-DB-lingual RPM available, here is how one
 can migrate from Berkeley-DB to SQLite and vice versa:
 
 1. start with a standard Berkeley-DB RPM DB:
 
 | $ /tmp/rpm/bin/rpm --initdb
 | $ /tmp/rpm/bin/rpm --import pubkeys/JBJ-GPG-KEY
 | $ /tmp/rpm/bin/rpm -qai
 | Name: gpg-pubkey   Relocations: (not relocatable)
 | Version : 2039b291  Vendor: (none)
 | Release : 3dbaae72  Build Date: Thu Jun 28 11:20:48 
 2007
 | Install Date: Thu Jun 28 11:20:48 2007  Build Host: localhost
 | Group   : Public Keys   Source RPM: (none)
 | Size: 0License: pubkey
 | Signature   : (none)
 | Summary : gpg(Jeff Johnson (ARS N3NPQ) [EMAIL PROTECTED])
 | [...]
 | $ file /tmp/rpm/var/lib/rpm/*
 | /tmp/rpm/var/lib/rpm/Group:  Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Installtid: Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Name:   Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Packagecolor:   Berkeley DB (Hash, version 8, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Packages:   Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Providename:Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Provideversion: Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Pubkeys:Berkeley DB (Hash, version 8, 
 little-endian)
 
 2. migrate RPM DB from Berkeley-DB to SQLite:
 
 | $ /tmp/rpm/bin/rpm --rebuilddbapi 4 --rebuilddb
 | $ file /tmp/rpm/var/lib/rpm/*
 | /tmp/rpm/var/lib/rpm/Group:  SQLite 3.x database
 | /tmp/rpm/var/lib/rpm/Installtid: SQLite 3.x database
 | /tmp/rpm/var/lib/rpm/Name:   SQLite 3.x database
 | /tmp/rpm/var/lib/rpm/Packagecolor:   SQLite 3.x database
 | /tmp/rpm/var/lib/rpm/Packages:   SQLite 3.x database
 | /tmp/rpm/var/lib/rpm/Providename:SQLite 3.x database
 | /tmp/rpm/var/lib/rpm/Provideversion: SQLite 3.x database
 | /tmp/rpm/var/lib/rpm/Pubkeys:SQLite 3.x database
 | $ /tmp/rpm/bin/rpm --dbapi 4 -qai
 | Name: gpg-pubkey   Relocations: (not relocatable)
 | Version : 2039b291  Vendor: (none)
 | Release : 3dbaae72  Build Date: Thu Jun 28 11:20:48 
 2007
 | Install Date: Thu Jun 28 11:20:48 2007  Build Host: localhost
 | Group   : Public Keys   Source RPM: (none)
 | Size: 0License: pubkey
 | Signature   : (none)
 | Summary : gpg(Jeff Johnson (ARS N3NPQ) [EMAIL PROTECTED])
 | [...]
 | -END PGP PUBLIC KEY BLOCK-
 
 3. migrate RPM DB back from SQLite to Berkeley-DB:
 
 | $ /tmp/rpm/bin/rpm --dbapi 4 --rebuilddbapi 3 --rebuilddb
 | $ file /tmp/rpm/var/lib/rpm/*
 | /tmp/rpm/var/lib/rpm/Group:  Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Installtid: Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Name:   Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Packagecolor:   Berkeley DB (Hash, version 8, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Packages:   Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Providename:Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Provideversion: Berkeley DB (Btree, version 9, 
 little-endian)
 | /tmp/rpm/var/lib/rpm/Pubkeys:Berkeley DB (Hash, version 8, 
 little-endian)
 | $ /tmp/rpm/bin/rpm -qai
 | Name: gpg-pubkey   Relocations: (not relocatable)
 | Version : 2039b291  Vendor: (none)
 | Release : 3dbaae72  Build Date: Thu Jun 28 11:20:48 
 2007
 | Install Date: Thu Jun 28 11:20:48 2007  Build Host: localhost
 | Group   : Public Keys   Source RPM: (none)
 | Size: 0License: pubkey
 | Signature   : (none)
 | Summary : gpg(Jeff Johnson (ARS N3NPQ) [EMAIL 

Re: [CVS] RPM: rpm/ CHANGES macros.in

[Mark Hatle]

Ahh you just reminded me of a bug.  I'm not sure how to fix this using
the dynamic values..  But basically, Depends can't be in the _dbi_tags
if you are using sqlite.  If you notice in the fragment below, on db3
it's setup as a temporary private database used for single-run
optimization.  On the sqlite side, it becomes a permanent cache..  The
problem with that is that it only holds partial dependency data.. so you
suddenly start to get failures.

Could we make the _dbi_tags variable depending on the configuration?
And we need ot be sure that Depends never gets used on sqlite.

   -# Depends is a per-transaction cache of known dependency resolutions.
   -%_dbi_config_Depends   %{_dbi_btconfig} temporary private
   +# database tags
   +%_dbi_tags  
 Packages:Name:Basenames:Group:Requirename:Providename:Conflictname:Triggername:Dirnames:Requireversion:Provideversion:Installtid:Sigmd5:Sha1header:Filemd5s:Depends:Pubkeys:Packagecolor

   -# The default database configuration to use
   -%_dbi_config%{_dbi_btconfig}
   +# database configuration: Berkeley-DB [dbapi 3 hooks]
   +%_dbi_config_3_Depends  %{_dbi_btconfig} temporary private
   +
   +# database configuration: SQLite [dbapi 4 hooks]
   +%_dbi_config_4_Depends  %{_dbi_sqlconfig}
   +
   +# database configuration [code entry hooks]
   +%_dbi_config_Depends
 %{expand:%%{_dbi_config_%{_dbapi_used}_Depends}}

--Mark
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES macros.in

[Mark Hatle]

Is TEMPORARY TABLE something special?  I knew about the :memory:
before, but not the other.

Otherwise this looks good to me.  It may even speed things up slightly.

--Mark

Ralf S. Engelschall wrote:
 On Thu, Jun 28, 2007, Mark Hatle wrote:
 
 Ahh you just reminded me of a bug.  I'm not sure how to fix this using
 the dynamic values..  But basically, Depends can't be in the _dbi_tags
 if you are using sqlite.  If you notice in the fragment below, on db3
 it's setup as a temporary private database used for single-run
 optimization.  On the sqlite side, it becomes a permanent cache..  The
 problem with that is that it only holds partial dependency data.. so you
 suddenly start to get failures.
 [...]
 
 How about the particular (still fully untested) change appended below?
 
 It enables private and temporary for SQLite and implements
 temporary via SQLite's special :memory: database and private via
 CREATE TEMPORARY TABLE.
 
 Feedback?
Ralf S. Engelschall
[EMAIL PROTECTED]
www.engelschall.com
 Index: macros.in
 ===
 RCS file: /v/rpm/cvs/rpm/macros.in,v
 retrieving revision 1.167
 diff -u -d -u -d -r1.167 macros.in
 --- macros.in 28 Jun 2007 16:41:51 -  1.167
 +++ macros.in 28 Jun 2007 21:15:38 -
 @@ -648,7 +648,7 @@
  # database tag configuration
  %_dbi_tags  %{expand:%%{_dbi_tags_%{_dbapi_used}}}
  %_dbi_tags_3
 Packages:Name:Basenames:Group:Requirename:Providename:Conflictname:Triggername:Dirnames:Requireversion:Provideversion:Installtid:Sigmd5:Sha1header:Filemd5s:Pubkeys:Packagecolor:Depends
 -%_dbi_tags_4
 Packages:Name:Basenames:Group:Requirename:Providename:Conflictname:Triggername:Dirnames:Requireversion:Provideversion:Installtid:Sigmd5:Sha1header:Filemd5s:Pubkeys:Packagecolor
 +%_dbi_tags_4
 Packages:Name:Basenames:Group:Requirename:Providename:Conflictname:Triggername:Dirnames:Requireversion:Provideversion:Installtid:Sigmd5:Sha1header:Filemd5s:Pubkeys:Packagecolor:Depends
 
  # database configuration: Berkeley-DB [dbapi 3 hooks]
  %_dbi_config_3  %{_dbi_btconfig}
 @@ -692,7 +692,7 @@
  %_dbi_config_4_Sigmd5   %{_dbi_sqlconfig}
  %_dbi_config_4_Triggername  %{_dbi_sqlconfig}
  %_dbi_config_4_Packages %{_dbi_sqlconfig}
 -%_dbi_config_4_Depends  %{_dbi_sqlconfig}
 +%_dbi_config_4_Depends  %{_dbi_sqlconfig} temporary private
 
  # database configuration [code entry hooks]
  %_dbi_config%{expand:%%{_dbi_config_%{_dbapi_used}}}
 Index: rpmdb/dbconfig.c
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/dbconfig.c,v
 retrieving revision 1.39
 diff -u -d -u -d -r1.39 dbconfig.c
 --- rpmdb/dbconfig.c  28 Jun 2007 15:52:05 -  1.39
 +++ rpmdb/dbconfig.c  28 Jun 2007 21:15:39 -
 @@ -112,7 +112,7 @@
   { lockdown,   0,POPT_BIT_SET, db3dbi.dbi_eflags, DB_LOCKDOWN,
   NULL, NULL },
  #endif
 -#if defined(WITH_DB)  defined(DB_PRIVATE)
 +#if (defined(WITH_DB) || defined(WITH_SQLITE))  defined(DB_PRIVATE)
   { private,0,POPT_BIT_SET, db3dbi.dbi_eflags, DB_PRIVATE,
   NULL, NULL },
  #endif
 @@ -288,8 +288,10 @@
   NULL, NULL },
   { lockdbfd,   0,POPT_ARG_NONE,db3dbi.dbi_lockdbfd, 0,
   NULL, NULL },
 +#endif
   { temporary,  0,POPT_ARG_NONE,db3dbi.dbi_temporary, 0,
   NULL, NULL },
 +#if defined(WITH_DB)
   { debug,  0,POPT_ARG_NONE,db3dbi.dbi_debug, 0,
   NULL, NULL },
  #endif
 Index: rpmdb/rpmdb.h
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/rpmdb.h,v
 retrieving revision 1.59
 diff -u -d -u -d -r1.59 rpmdb.h
 --- rpmdb/rpmdb.h 28 Jun 2007 15:52:05 -  1.59
 +++ rpmdb/rpmdb.h 28 Jun 2007 21:15:39 -
 @@ -15,6 +15,9 @@
  #else
  #include db_emu.h
  #endif
 +#if defined(WITH_SQLITE)  !defined(DB_PRIVATE)
 +#define DB_PRIVATE 0x020 /* shameless hack for shared option */
 +#endif
 
  /[EMAIL PROTECTED]@*/
  /[EMAIL PROTECTED]@*/
 Index: rpmdb/sqlite.c
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/sqlite.c,v
 retrieving revision 1.6
 diff -u -d -u -d -r1.6 sqlite.c
 --- rpmdb/sqlite.c19 Jun 2007 09:34:45 -  1.6
 +++ rpmdb/sqlite.c28 Jun 2007 21:15:39 -
 @@ -729,13 +729,15 @@
   }
  if (_debug)
  fprintf(stderr, \t%s(%d) type(%d) keytype %s\n, tagName(dbi-dbi_rpmtag), 
 dbi-dbi_rpmtag, (tagType(dbi-dbi_rpmtag)  RPM_MASK_TYPE), keytype);
 - sprintf(cmd, CREATE TABLE '%s' (key %s, value %s),
 + sprintf(cmd, CREATE %sTABLE '%s' (key %s, value %s),
 + dbi-dbi_eflags  DB_PRIVATE ? TEMPORARY  : ,
   dbi-dbi_subfile, keytype, valtype);
   rc