Re: Making rpm5 work with dnf

[Jeff Johnson]

Sent from my iPad

> On Nov 25, 2016, at 11:42 PM, Mark Hatle  wrote:
> 
>> On 11/25/16 5:07 PM, Jeffrey Johnson wrote:
>> 
>>> On Nov 24, 2016, at 8:50 AM, Alexander Kanavin
>>> >> >
>>> wrote:
>>> 
 On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:
 
 Again: I cannot do much more than suggest porting approaches unless I can
 attempt reproducers.
 Please try to expose your git repositories somehow, either publicly or 
 privately.
>>> 
>>> Here it is:
>>> 
>>> https://github.com/kanavin/libhif/commits/master
>>> 
>> 
>> Good. I will take a look at your libhif efforts. Note that libhif is only
>> one of several repositories that are going to be needed.
>> 
>> FWIW, your invitation expired or is otherwise unusable (but at least
>> I can read your code, todo++).
>> 
>> (aside)
>> Before we go too much further here:
>> 
>> What is the intent of this collaboration?
> 
> The functional goal in this work is to stop using smart in the Yocto Project 
> and
> switch to DNF.  (We've found nothing else that is a reasonable alternative 
> that
> is still under active development.)
> 

This part I understand (from conversations mid-summer).

> As far as the way things are done, I'll leave that between you, Alexander and
> anyone else interested.
> 

We're getting there ... perhaps some OpenMandriva involvement after
their weekly meeting 11am EST next Wed. I believe there is PLD interest
as well, perhaps a few other efforts interested in collaborating too.

hth

73 de Jeff

> --Mark
> 
>> You have chosen to fork libhif from
>> rpm-software-management/libhif 
>> 
>> rather than a fork (of a fork) from
>> https://github.com/rpm5/libhif
>> 
>> That forces our coordination to be pulled from the only common
>> root at
>> rpm-software-management/libhif 
>> 
>> which almost certainly precludes any participation from me and rpm5.org
>> 
>> for various reasons.
>> 
>> Note that there are several other efforts attempting a dnf->…->rpm5 tool 
>> chain
>> that I
>> am aware of. Which is why I attempted RPM5 repositories to permit 
>> collaboration, and
>> am perfectly willing to give write access to anyone who wishes.
>> 
>> I am also perfectly willing to let someone other than rpm5.org 
>> 
>> administrate the mess if that
>> is what is desired. I do encourage all of you to collaborate early and work
>> forward from
>> working tools. There’s a fair amount of subtle work that will be needed imho.
>> 
>> What is your intent: collaboration with rpm5.org  or
>> collaboration with rpm-software-management?
>> 
>>> I've fixed what I could by adding rpm5 includes, but the remaining build
>>> issues are all caused by actual API incompatibilities between 4 and 5. I can
>>> file them as separate github issues if you want.
>>> 
>> 
>> Um rpm5.org  and rpm.org  have different 
>> API’s,
>> and are most definitely different
>> implementations these days. Its not just include files, and more than libhif 
>> is
>> going to
>> be needed to build a working dnf->…->rpm5 tool chain.
>> 
>> I’ll take a look at your libhif this weekend.
>> 
>> 73 de Jeff
>>> Alex
>>> 
>>> __
>>> RPM Package Managerhttp://rpm5.org
>>> User Communication List rpm-users@rpm5.org
>>> 
>> 
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm 5.1.9 issues on powerPC architecture

[Jeff Johnson]

On Apr 14, 2016, at 12:15 PM, Jeff Johnson wrote:

> 
>> >>>> rpm files are generated on x86_64 machines but querying it on ppc 
>> >>>> machines (arch within rpm is ppc). Inbuilt metadata can be queried 
>> >>>> using --qf but not custom metadata. I guessed that endianness should 
>> >>>> not have impact rpm metadata content. But, even if it has, then it 
>> >>>> should give garbled values and not "(none)" as its result. However, 
>> >>>> querying custom metadata works fine in 5.4.14.
>>  
> 
> Yes arbitrary tags in metadata should be platform independent as a goal. But 
> not in rpm-5.1.9 == a bug.
> 

I should state the problem more carefully:

The meatadata and the tags assigned to the arbitrary metadata ARE independent
of platform in the sense that that are marked with 0xc000. What is/was 
broken
is that the truncated SHA1 for the remaining 30bits of the tag assignment is 
little
endian.

The fix on PPC and big-endian machines is to swab the 30bits before generating
the tag assignment.

> The tag itself, not its content, is what is not platform independent. Hence 
> not "garbled".
> 

The fix on PPC and big-endian machines is to swab the 30bits before generating
the tag assignment.

So its a fairly simple one line patch in rpmio/tagname in _tagGenerate() in 
rpm-5.1.9
that end up looking should look like this:

if (digest && digestlen > 4) {
  /* The tag is stored in a uniform byte order for cross-endian 
compatib  ility. 
 Swap to the host uses. */
  memcpy(, digest + (digestlen - 4), 4);
  tag = le32toh(tag);
  tag = (rpmTag) (tag & 0x3fff);
  tag = (rpmTag) (tag | 0x4000);
  }   

73 de Jeff

Re: rpm 5.1.9 issues on powerPC architecture

[Jeff Johnson]

On Apr 12, 2016, at 2:14 PM, Rajul Bhavsar wrote:
> 
> Hmm why rpm-5.1.9, which was released quite some time ago?
>  Yes, its older version. But that is what current version we had. Will 
>  like to move to 5.4.14 in near future and that rpm version performs all 
>  required queries perfectly fine.
>> $ rpm -qai
>> rpm: -qai: unknown option
>> $
>> 
> 
> Try a simpler query first: does rpm -qa "work"?
>  
>  Yes, -qa works, but --xml or --yaml do not work along with -qpi. Also, 
>  use of --force give same error in rpm -iv command.
>  

OK. All of -i/--info, --xml and --yaml are popt aliases.

Make sure (with strace -e open) that rpm can read /usr/lib/rpm/rpmpopt.

You can also (for --xml and --yaml at least) verify with --queryformat.

All that --yaml does is add this --queryformat option:

rpm -q --qf '[%{*:yaml}\n]' bash

> The -i option is contextually dependent with multiple meanings in rpm
> and the processing is painfully complex. Identifying what does and does not
> work will be helpful in finding a fix.
> 
>> When we try to query custom metadata (implemented using arbitrary tags) then 
>> also it is failing (giving output as (none)). We have proper macro file in 
>> /usr/lib/rpm-5.1.9/macros and rpm is referring to it.
>> 
> 
> (from memory of a recent patch from Poky/Yocto)
> 
> The arbitrary tag values are likely in native endian.
> Did you build and query both on the same machine, or are you
> trying to query a package built on x86_64 on a ppc* machine?
>  
>  rpm files are generated on x86_64 machines but querying it on ppc 
>  machines (arch within rpm is ppc). Inbuilt metadata can be queried using 
>  --qf but not custom metadata. I guessed that endianness should not have 
>  impact rpm metadata content. But, even if it has, then it should give 
>  garbled values and not "(none)" as its result. However, querying custom 
>  metadata works fine in 5.4.14.
>  

Yes arbitrary tags in metadata should be platform independent as a goal. But 
not in rpm-5.1.9 == a bug.

The tag itself, not its content, is what is not platform independent. Hence not 
"garbled".

With arbitrary tags, there are never any missing tags, any tag name will do.
RPM queries have always returned '(none)' in-band (i.e. in the actual output 
string)
for optional tags. That behavior has been extended to arbitrary tags.

hth

73 de Jeff

Re: rpm 5.1.9 issues on powerPC architecture

[Jeff Johnson]

On Apr 11, 2016, at 1:25 PM, Rajul Bhavsar wrote:

> Hi,
> 
> We are trying to use rpm 5.1.9 on powerPC architecture. But, we see that 
> there are issues with basic querying:
> 

Hmm why rpm-5.1.9, which was released quite some time ago?

> $ rpm -qai
> rpm: -qai: unknown option
> $
> 

Try a simpler query first: does rpm -qa "work"?

The -i option is contextually dependent with multiple meanings in rpm
and the processing is painfully complex. Identifying what does and does not
work will be helpful in finding a fix.

> When we try to query custom metadata (implemented using arbitrary tags) then 
> also it is failing (giving output as (none)). We have proper macro file in 
> /usr/lib/rpm-5.1.9/macros and rpm is referring to it.
> 

(from memory of a recent patch from Poky/Yocto)

The arbitrary tag values are likely in native endian.
Did you build and query both on the same machine, or are you
trying to query a package built on x86_64 on a ppc* machine?

> $ rpm --showrc | grep macros
> macrofiles: 
> /usr/lib/rpm-5.1.9/macros:/usr/lib/rpm-5.1.9/ppc-linux/macros:/etc/rpm/5.1.9/macros.*:/etc/rpm/5.1.9/macros:/etc/rpm/5.1.9/ppc-linux/macros:~/.rpmmacros-5.1.9
> $
> 
> We see that on x86_64 these things work fine. We do not see anything specific 
> to architecture in the build of rpm, but only difference is that - host's 
> arch is x86_64.
> 
> Can anybody tell what is peculiar about rpm on powerPC arch?
> 

hth

73 de Jeff
> Thanks,
> Rajul

Re: RPM sqlite3 support

[Jeff Johnson]

On Jan 13, 2016, at 6:34 AM, devzero2000 wrote:

> 
> On Tue, Jan 12, 2016 at 10:06 PM, Mark Hatle  wrote:
> On 1/12/16 2:06 PM, Tim Mooney wrote:
> > In regard to: Re: RPM sqlite3 support, Jeffrey Johnson said (at 12:46pm 
> > on...:
> >
> >> The sqlite3 code (and support) in rpm5 was abandoned in favor of
> >> Berkeley DB ACID transactional support quite some years ago
> >
> > I've been meaning to ask about this for a while, and this provides a
> > good segue...
> >
> > With Oracle's license change on BDB 6.x (or 12.x, or whatever they're
> > calling it) to AGPL, does that impact rpm5's long-term use of BDB?
> 
> I know we and many of our commercial customers has rejected BDB 6.x because of
> the change, so we've been forced to 'support' BDB 5.x.
> 
> Personally I would like to get rid of anything that has to do with BerkeleyDB,
> just to get rid of this or future license questions from customers.  (But
> unfortunately I don't know what can reasonably replace BDB.)
> 
> Hello 
> 
> FWIW, @rpm.org started a plan to replace the rpmdb format
> 
> https://fedoraproject.org/wiki/Changes/NewRpmDBFormat
> 
> Here the original announcement
> 
> http://comments.gmane.org/gmane.linux.redhat.fedora.devel/215427
> 
> Currently no details are known (to me)

There aren't many details, certainly the reasons for change are not clearly
expressed in the description:

The current implementation of the RPM Database is based on Berkeley DB. 
There are doubts about the its future
 and level of maintenance. In addition rpm's use of the database has 
multiple issues on its own. As a result RPMx
 upstream is working to replace the database format with a new 
implementation.

The implementation (assuming rpm/lib/backend/ndb is the new format) is a rather
straightforward hash based database afaict. I am not seeing the necessary
interconversion tools between the formats, not hard to write when the time 
comes.
I'm also not seeing provisions for transactions and ACID and caching and
all the other sophistication/complexity that Berkeley DB provides, but
perhaps I am not appreciating the ndb implementation from a single
pass through the code.

There is also no obvious signs of remote access and/or replication. The API is
simple enough that any form of RPC will serve for remoting. OTOH, there's
much much more to replication than RPC.

The discussions are mostly legitimate concerns about compatibility and its 
pretty
clear that the proposed solution for other projects is
Use rpmlib API's if you wish compatibility with an rpmdb.
Which isn't bad advice, just opposite from the entire intent of using a 
"standard"
API in Berkeley DB way back when so that other/better implementations than rpm
could be developed (of course the main problem there is that the Packages 
database
contains a header blob rather than simple/rich data types that 
applications/developers expect).

Most of the applications that use an rpmdb appear to be a 1-time data scrape of 
an rpmdb
(libguestfs with db_dump(1) and what has been done in yum/zypper in the past). 
A 1-time
data scrape of an entire rpmdb perhaps indicates that no database whatsoever in 
rpm
is what developers wish: there is no reason I know that a pile of *.rpm files 
cannot replace
the /var/lib/rpm/Packages store and leave indexing to applications (which are 
doing the indexing
into the data scrape already).

*shrug*

C+ would be my grade for what I am seeing implemented in rpm/lib/backend/ndb

hth

73 de Jeff
> 
> Elia 
>  
> 
> --Mark
> 
> > Tim
> >
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org
> 

Re: How to detect rpm post scriptlet failure

[Jeff Johnson]

On Oct 13, 2011, at 6:13 AM, Levon Poghosyan wrote:

 It seems to be there is no chance to do this in rpm 4.8.1, am I right ? 

rpm-4.8.1 (and later/earlier versions from @rpm.org) does not save scriptlet
exit codes persistently.

There are also side-effects introduced by ignoring scriptlet exit codes like 
here:

https://bugzilla.redhat.com/show_bug.cgi?id=569930 

If you have an application that is installing a bunch of rpm packages, and
some of those packages have a %post scriptlet failing, then the failures need
to be anaylized, classified, and remedied somehow.

If you can detect failure, then you should be able to diagnose and fix the
package scriptlets.

Asking RPM to detect failures, when the only information
available is the scriptlet exit code, which is ignored and not saved 
persistently in
rpm-4.8.1, simply isn't a feasible approach.

73 de Jeff

 
 On 13 October 2011 11:04, devzero2000 pinto.e...@gmail.com wrote:
 On Thu, Oct 13, 2011 at 9:15 AM, Levon Poghosyan deimusmeis...@gmail.com 
 wrote:
 Thanks for the reply.
 
 The situation is following. I've got an application which installs bunch of 
 rpm packages. 
 Some of those packages have post scriptlet failing, so I need to know/figure 
 out which are failing. 
 Those packages are many, so I think it wont be the best solution to modify 
 all off them to store their post install scriptlet execution status in /var, 
 any other solution for this case ? 
 
 Regards,
 Levon
 
 FWIW, there is a related bugzilla for @rpm.org 
 
 https://bugzilla.redhat.com/show_bug.cgi?id=569930 
 
 (rpm exit 0 always on scriptlet execution)
 
 In other package management system exists state as half installed or so. But
 here are two issue relevant i think :
 
 - Should be failing script relevant or not (see the bugzilla for a discussion 
 on this, in
 particular for %post) ? @rpm.org choose to exit always with 0
 
 - How should applications or users know that there is some errors ?
 
 In any case @rpm5.org add the scriplets exit code to the header registrated 
 in a rpmdb and also to the source rpm (for detecting a wrong use of 
 --short-circuit). 
 
 rpm -q --yaml arbitrarytags
 (snip)
 .
 
 Scriptstates:
 - 0
 - 0
 - 0
 - 131072  %post exit code : this number mean exit 0
 
 
 you can also use 
 
 rpm -q --qf '[%{SCRIPTSTATES}]\n' arbitrarytag
 000131072
   
 ^
 |
 
 Hope this help
 
 
 Best Regards
 
 
 
 
 
 
 here are two issues relevant (imho
Should failing scripts be permitted?
 
 and a obscurely related general design issue
 
How should applications and users and ... be notified of errors?
 
 
   
 
 On 12 October 2011 21:22, Jeff Johnson n3npq@gmail.com wrote:
 
 On Oct 12, 2011, at 11:38 AM, Levon Poghosyan wrote:
 
  Hello,
 
  How can I detect if the execution of post scriptlet of the rpm package 
  failed ?
 
 The simple answer here is:
You don't.
 
 All I mean by that is that the %post script needs to be written robustly
 so that the exit code is always 0.
 
 If you need/want to tell whether some specific operation worked,
 then write a test after rpm has run for that specific operation.
 
 Another approach would be to have the %post script register its
 state somewhere in /var so that you can easily test whether the script
 worked or not.
 
  I've generated the and rpm package which has a test command lalala in 
  post install section. So during the installation it prints out information 
  that command lalala was not found but the installation is still successful.
  How do I identify this failure from post install section.
 
 A %post scriptlet (the only difference between script and scriptlet is that 
 a
 scriplet is macro expanded and may eventually have some envvar's
 prepended instead of having RPM add to the environ directly) is
 just a script.
 
 SO use test(1) to test for existence and executability, and write that into
 the %post section directly.
 
 Because a %post is part of a package install state machine, the script
 SHOULD return 0 for all but catasstrophic faiulures.
 
 There are side-effects of returning failure from %post, the most important
 of which is that on an upgrade, the erase will be skipped if/when the install 
 fails.
 
 
  Please note I'm not interested in failure in other places, I just need to 
  be informed in post install scriptlet failed.
 
 
 Personal;ly, I'd just write the %post script to write 1 line into 
 /var/lib/application/state
 with the message
The %post script succeeded
 or (on failure)
The %post script failed.
 
 I'd have to know more about what is implied by a %post success/failure
 in order to suggest some other approach.
 
 hth
 
 73 de Jeff
 
 __
 RPM Package Managerhttp://rpm5.org
 User Communication List

Re: How would one extract the trigger scripts from a spec file.

[Jeff Johnson]

On Oct 12, 2011, at 4:36 PM, James Olin Oden wrote:

 I figured out that there are a bunch of tags with TRIGGER at the
 begining of the name that form some set of parallel arrays of
 information on all the triggers:
 
   TRIGGERFLAGS
   TRIGGERINDEX
   TRIGGERNAME
   TRIGGERSCRIPTPROG
   TRIGGERSCRIPTS
   TRIGGERVERSION
   TRIGGERCONDS
   TRIGGERTYPE
 

You've mixed in some header tag extensions with
actual tags that are present in packages.

The extensions are there to make it easier to use
--queryformat to display trigger scripts, not anything
deep.

Triggers have:
   1) a firing condition
   2) a script and interpreter to run when fired

 In my case I just want to run the scripts through bash -c and validate
 that there syntax is good, and show the information on the script.

Likely
rpm -qp --triggers *.rpm | …
into a script is the easiest path for a bash -c syntax check.

(aside)
Note that the same syntax check is automated by
invoking bash --rpm-requires @rpm5.org (and with triggers
as well iirc). The back port likely isn't hard, and it is as easy
to use bash -c as it is to do bash --rpm-requires if desired.

The goal would be better automation: fail the build on scriptlet syntax errors.

 However I can't seem to locate the scripts, and I figure the name
 would be a concatenation of the trigger name an d the type, but I'm
 not sure how to interpret the type.   I can dig into the source and
 figure this out, but before that can anyone point me to a quick
 explanation of the tags involving triggers or could they explain them?
 

The basic change for RPMTAG_TRIGGER* tag storage is merely this:
For most scriptlets packages - script relation is 1-to-1.
But there can be multiple triggers in a single package so the
the store is 1-to-N (i.e. one package can have multiple triggers).
The TRIGGERINDEX is glue to make 1-to-N possible (and also handles
the various types of triggers which are distinguished by bits in *FLAGS.

(aside)
I used to think it was a good idea to move all scriptlets into a storage
like used by triggers to reduce clutter in the RPMTAG_* name space.

Then along came LSB demanding standard compliance for no real purpose
and it became infeasible to change anything at all in RPM without also
attempting parallel change to LSB standards and politcs took over …

*shrug*

Holler if you have a specific question and I'll look at code.

hth

73 de Jeff



__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm install and folder creation

[Jeff Johnson]

On Jun 4, 2010, at 12:04 PM, Belal Salem wrote:

 Thanks for explaining that!
 But now, all the RPM packages that I have this issue with are installing 
 correctly with RPM-4.x as that the directory structure are listed when using 
 -qlp, but with RPM-5.3.11, the package manager complain about the directories 
 until I use the --nodeps to workaround it, hence it installs correctly and it 
 does create the needed directories.
 

Instead of --nodeps, try --noparentdirs/--nolinktos disablers.

Yes. I've described another recommended procedure to track
the orphan (i.e. not explicitly included in any package) directories
and dangling (i.e. the end-point is not in any package) symlinks.

To mask a single directory during development/bootstrapping
until you can correct what is intrinsically a packaging flaw (imho):

mkdir -p /etc/rpm/sysinfo
echo /some/orphan/directory/  /etc/rpm/sysinfo/Dirnames
echo /some/dangling/symlink:  /etc/rpm/sysinfo/Filelinktos

You -- in fact -- can satisfy any missing dependency using /etc/rpm/sysinfo:
echo libmissing.so  /etc/rpm/sysinfo/Providename
echo awol-package = 1:2.3  /etc/rpm/sysinfo/Providename

If you don't wish the manual one-by-one setup tedium, then
use --noparentdirs --nolinktos (or --nodeps) while installing into a chroot
(with --root /some/chroot) and then extract *all* of the
parentdir/linkto flaws like
rpm -Va --nofiles | grep '^/' | sort -u  /tmp/Dirnames.new

Compiling the disabler directly into RPM prevents usage. You can do
that as well if you wish: just its Have it your own way! supported
(and I am not the person to ask for support of what I believe isn't
a feature).

hth

73 de Jeff
 
 On Sun, 04 Sep 2011 16:47:38 +0200, Jeff Johnson n3...@mac.com wrote:
 
 
 On Jun 4, 2010, at 2:55 AM, Belal Salem wrote:
 
 Hi there!
 I issued the same issue before, when installing some packages, the RPM 
 package manager doesn't create the required folders and ask for the folders 
 as unresolved dependencies, although those folders are part of the package 
 being installed.
 
 Its part of the package which is confusing.
 
 There are two meanings for part of the package:
 
 1) directory components as part of file paths
 2) directory explicitly listed in rpm -qpl *.rpm
 
 If its not explicitly in the file manifest, its not part of the package
 and you *will* see what you are reporting.
 
 Recompiling RPM with the options: --disable-dirname-and-symlink-deps didn't 
 solve the problem, anyway through that?!
 
 
 I'm not the person to fix --disable-dirname-and-symlink-deps.
 
 My fix will be to rip out the Have it your own way! functionality that 
 isn't working
 and remove the
  --disable-dirname-and-symlink-deps
 in order to simplify RPM's build and clarify supported functionality. I see
 no future in carrying around functionality that doesn't work as it should
 and is vendor supported by others here @rpm5.org.
 
 I will rip out the option if it isn't fixed by someone else @rpm5.org
 this month.
 
 73 de Jeff
 
 __
 RPM Package Managerhttp://rpm5.org
 User Communication List rpm-users@rpm5.org
 
 
 
 -- 
 Using Opera's revolutionary email client: http://www.opera.com/mail/
 __
 RPM Package Managerhttp://rpm5.org
 User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Why rpm5 and smart for Belenix

[Jeff Johnson]

On Sep 3, 2011, at 3:48 PM, pinto.e...@gmail.com wrote:

 Sorry for the top posting. I think @rpm5.org is glad of this. Just a, perhaps 
 naive, question : What mean for a package manager to be more zfs-aware ? 
 Thanks and best regards.

For starters:

ZFS has snapshots for --rollback.

And Solaris has SEEK_HOLE, useful if sparse files ever need to be packaged.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm -U completes with non-zero return code but no error message

[Jeff Johnson]

On Aug 20, 2011, at 3:31 AM, akiyoshi kawamura wrote:

 Hello,
 
 I tried to upgrage mysql_devel package by having all required rpms in
 a directory. I used the following command to do the upgrade
 
 /bin/rpm --replacepkgs --oldpackage -Uvh /v/rpm/sw/var/petidomo/*.rpm; echo 0
 
 Here is the output; there are many warning messages but no error
 message. The return code is 93. What does the return code imply?

The return code in rpm is the number of failed items, in this case
packages that you asked to be installed that were not.

 Non-zero return code means there are failures during the upgrade. But
 without error messages I am not sure what to look for.
 

Because users tend to do
rpm -Uvh … *.rpm
(as you have done) rpm filters the given arguments when there are multiple 
packages
with identical names, choosing the newest package.

The warnings are exactly the error messages that you are seeking.

73 de Jeff


 warning: 
 /root/e-admin_5.0/rpms/cs.latest/mysql_devel_all/apr-1.3.9-3.fc13.x864.rpm:
 Header V3 RSA/SHA256 Signature, key ID 97a1071f: NOKEY
 warning: package apr = 1.3.9-3.fc13 was already added, replacing with
 apr  1.5-1.fc14
 warning: package apr-util = 1.3.12-1.fc14 was already added, skipping
 apr-util 1.3.9-3.fc13
 warning: package apr-util-ldap = 1.3.12-1.fc14 was already added,
 skipping aprtil-ldap  1.3.9-3.fc13
 warning: package glibc = 2.12.90-17 was already added, replacing with
 glibc 13-1
 warning: package glibc-common = 2.12.90-17 was already added,
 replacing with gbc-common  2.13-1
 warning: package glibc-devel = 2.12.90-17 was already added, replacing
 with glc-devel  2.13-1
 warning: package glibc-headers = 2.12.90-17 was already added, replacing with
 ibc-headers  2.13-1
 warning: package httpd = 2.2.16-1.fc14 was already added, replacing with httpd
  2.2.17-1.fc14
 warning: package httpd-manual = 2.2.16-1.fc14 was already added, replacing wit
 httpd-manual  2.2.17-1.fc14
 warning: package httpd-tools = 2.2.16-1.fc14 was already added, replacing with
 ttpd-tools  2.2.17-1.fc14
 warning: package krb5-devel = 1.8.2-10.fc14 was already added, skipping krb5-d
 el  1.8.2-6.fc14
 warning: package krb5-libs = 1.8.2-10.fc14 was already added, skipping krb5-li
   1.8.2-6.fc14
 warning: package krb5-workstation = 1.8.2-10.fc14 was already added, skipping
 b5-workstation  1.8.2-6.fc14
 warning: package libcom_err = 1.41.12-5.fc14 was already added, replacing with
 ibcom_err  1.41.12-6.fc14
 warning: package libcom_err-devel = 1.41.12-5.fc14 was already added, replacin
 with libcom_err-devel  1.41.12-6.fc14
 warning: package libselinux = 2.0.96-5.fc14 was already added, replacing with
 bselinux  2.0.96-6.fc14.1
 warning: package libselinux-devel = 2.0.96-5.fc14 was already added, replacing
 ith libselinux-devel  2.0.96-6.fc14.1
 warning: package libselinux-python = 2.0.96-5.fc14 was already added, replacin
 with libselinux-python  2.0.96-6.fc14.1
 warning: package libselinux-utils = 2.0.96-5.fc14 was already added, replacing
 ith libselinux-utils  2.0.96-6.fc14.1
 warning: package mysql = 5.1.51-2.fc14 was already added, replacing with mysql
  5.1.56-1.fc14
 warning: package mysql-libs = 5.1.51-2.fc14 was already added, replacing with
 sql-libs  5.1.56-1.fc14
 warning: package mysql-server = 5.1.51-2.fc14 was already added, replacing wit
 mysql-server  5.1.56-1.fc14
 warning: package ncurses = 5.7-8.20100703.fc14 was already added, replacing wi
  ncurses  5.7-9.20100703.fc14
 warning: package ncurses-base = 5.7-8.20100703.fc14 was already added, replaci
  with ncurses-base  5.7-9.20100703.fc14
 warning: package ncurses-libs = 5.7-8.20100703.fc14 was already added, replaci
  with ncurses-libs  5.7-9.20100703.fc14
 warning: package nss-softokn-freebl = 3.12.10-1.fc14 was already added, skippi
  nss-softokn-freebl  3.12.7-6.fc14
 warning: package openssl = 1.0.0a-2.fc14 was already added, replacing with ope
 sl  1.0.0d-1.fc14
 warning: package openssl-devel = 1.0.0a-2.fc14 was already added, replacing wi
  openssl-devel  1.0.0d-1.fc14
 warning: package php = 5.3.3-1.fc14 was already added, replacing with php  5.
 6-1.fc14
 warning: package php-cli = 5.3.3-1.fc14 was already added, replacing with php-
 i  5.3.6-1.fc14
 warning: package php-common = 5.3.3-1.fc14 was already added, replacing with p
 -common  5.3.6-1.fc14
 warning: package php-mysql = 5.3.3-1.fc14 was already added, replacing with ph
 mysql  5.3.6-1.fc14
 warning: package php-pdo = 5.3.3-1.fc14 was already added, replacing with php-
 o  5.3.6-1.fc14
 Preparing...### [100%]
1:libgcc ### [  2%]
2:glibc-common   ### [  3%]
3:nss-softokn-freebl ### [  5%]
4:glibc  ### [  7%]
5:libselinux ### [  

Re: rpm-5.3.11

[Jeff Johnson]

On Aug 14, 2011, at 1:15 PM, Belal Salem wrote:

 Thank for the perfect details!
 since I'm running RHEL-5 (as my system base and based on rpm-4.4) but updated 
 from RHEL-6 (glibc and its deps and python),
 and according to your explanation, I guess I would better recompile rpm with 
 the --disable-dirname-and-symlink-deps
 specially that adding the folders to _SOME_ will not be suitable for my case 
 since I have requests for different folder whenever I'm trying to install 
 packages from RHEL-5 or RHEL-6.
 

Actually its likely easier to use a run-time rather than a compile-time
setting. The parentdir/linto dependencies are implicit
in the sense that there's no Requires: that is explicitly needed.
The dependencies are synthesized from pre-existing information in
a package.

To configure parentdir/linkto dependencies at run-time:

1) Make sure that RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS is enabled.
That is likely already the case if you haven't disabled. Check
config.h after running ./configure (and you can always edit config.h
after running configure if all else fails).

 
 The code to Rip It Out! Have it your own way! is (from lib/depends.c):
  #if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
 defined(RPM_VENDOR_OPENMAMBA) || 
 defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
 optional-dirname-and-symlink-deps */
  int dirname_deps;
  int symlink_deps;
  #endif
 …
  #if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
 defined(RPM_VENDOR_OPENMAMBA) || 
 defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
 optional-dirname-and-symlink-deps */
  dirname_deps = 
 rpmExpandNumeric(%{?_check_dirname_deps}%{?!_check_dirname_deps:1});
  if (dirname_deps) {
  #endif
 …
  #if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
 defined(RPM_VENDOR_OPENMAMBA) || 
 defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
 optional-dirname-and-symlink-deps */
  }
 
  symlink_deps = 
 rpmExpandNumeric(%{?_check_symlink_deps}%{?!_check_symlink_deps:1});
  if (symlink_deps) {
  #endif
 …
  }
 

2) Edit /etc/rpm/macros (create if not existent) to add
%_check_dirname_deps0
%_check_symlink_deps0
to disable.

3) Turn on symlink dependencies: its less annoying than parentdir deps and
is almost always a package failure when encountered (aka a dangling symlink
that points to a non-existent end-point).

4) When you have a mostly stable/unchanging set of packages installed, then
turn on parentdirectiory dependencies. Then do
mkdir -p /etc/rpm/sysinfo
rpm -Va --nofiles | grep '^/' | sort -u  /etc/rpm/sysinfo/Dirnames
(there is a --orphandirs option to rpm -Va that does the above, I'm just too 
lazy to check
how to use it: its a popt alias that does essentially the above).

5) Run
rpm -Va --nofiles
periodically, and when you see a orphan directory, i.e. a directory that
isn't explicitly in any package file manifest, then just add that directory
to /etc/rpm/sysinfo/Dirnames.

The annoyance is mostly that RHEL5/RHEL6 never bothered to do the QA to
ensure that rpm --erase would also remove created directories.

BTW, if you are going to use rpm-5.3.11` on RHEL5/RHEL6, then you need
to convert your rpmdb so that header instance numbers are always in
network order (needed for optimal btree access).

hth

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm-5.3.11

[Jeff Johnson]

On May 13, 2010, at 5:46 PM, Belal Salem wrote:

 So! Are there another solution by recompiling 'rpm' source code to just work 
 as we used to!
 means, are there any configure script options that I can use to override 
 that?!
 

Sure you can recompile to change two control bits and their
default setting compiled into RPM to work as it used to (this change
was made 7+ years ago in like rpm-4.4.6 or something).

Here are the bits (from lib/rpmts.h):
RPMDEPS_FLAG_NOPARENTDIRS = (1  4), /*! from --noparentdirs */
RPMDEPS_FLAG_NOLINKTOS = (1  5),/*! from --nolinktos */ 
Here is where the default values for dependency flags are set (lib/poptALL.c):
/*@unchecked@*/ int global_depFlags = RPMDEPS_FLAG_ADDINDEPS; 
Add those two disabler bits as you please.

There are macros that can be set to change the default
values of those two bits so that you -- like every other vendor --
can revert to traditional rpm behavior from 7+ years ago without
recompiling.

The code to Rip It Out! Have it your own way! is (from lib/depends.c):
#if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
defined(RPM_VENDOR_OPENMAMBA) || 
defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
optional-dirname-and-symlink-deps */
int dirname_deps;
int symlink_deps;
#endif
…
#if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
defined(RPM_VENDOR_OPENMAMBA) || defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) 
/* optional-dirname-and-symlink-deps */
dirname_deps = 
rpmExpandNumeric(%{?_check_dirname_deps}%{?!_check_dirname_deps:1});
if (dirname_deps) {
#endif
…
#if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
defined(RPM_VENDOR_OPENMAMBA) || 
defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
optional-dirname-and-symlink-deps */
}

symlink_deps = 
rpmExpandNumeric(%{?_check_symlink_deps}%{?!_check_symlink_deps:1});
if (symlink_deps) {
#endif
…
}

And there is AutoFu to set the default value in Have it your own way! 
fashion
in several ways. The RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS functionality is 
tied to
./configure --help
…
  --disable-dirname-and-symlink-deps
  permit to disable via macro the automatic dirname
  and symlink dependecy

The other way to disable -- because vendors MUST see their branding 
everywhere and
can't really figure out how to build RPM unless they can see/find their name
in the code -- is tied to

  --with-vendor=VENDORbuild with a supported vendor's specific set 
of
  changes: mandriva, ark, fedora, openmamba

Note that per-vendor changes are _NOT_ supported through @rpm5.org. I can't
support code that is never enabled in my development versions (and that is
controversial for some reason, in this case because the issue was discussed
and decided 7+ years ago and no one has bothered to reopen discussions about
what SHOULD be done since).

But the better way to fix instead of recompiling RPM is to add
%files
…
%dir /etc/fonts
%dir /etc/fonts/conf.d

to _SOME_ package so that the driectores are actually present in _SOME_ package.
(the directories can be in multiple packages too).

Adding the directories to _SOME_ package ensures that the directories will
be removed by --erase, which is/was the reason for adding the annoying
parent directory dependency in the first place.

hth

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 10, 2011, at 6:56 AM, Anders F Björklund wrote:

 Jeff Johnson:
 
 6. Configure RPM:
 -
 # ./configure --prefix=/usr --libdir=/usr/lib64 --with-uuid=/usr/lib64 
 --with-xar=/usr/lib64 \
   --enable-rpm-lua-extensions-based-on-rpmlib 
 --enable-fast-install --enable-shared \
  --enable-rpmvercmp-digits-beat-alpha --with-expat=/usr/lib64 
 --with-neon=/usr/lib64 \
 
 Hmmm … I did not know there is a --enable-rpmvercmp-digits-beat-alpha
 or a --enable-rpm-lua-extensions-based-on-rpmlib option.
 
 I'm not at all sure what they are supposed to do or who added (its likely 
 from RSE and OpenPKG several
 years back). It's sure to be  some pretty arcane functionality ...
 
 Actually it's quite new...
 http://rpm5.org/cvs/chngview?cn=15680  optional-dirname-and-symlink-deps
 http://rpm5.org/cvs/chngview?cn=15735  
 rpm-lua-extensions-based-on-rpm-lib-functionality
 http://rpm5.org/cvs/chngview?cn=16017  old-comparision-behaviour
 
 It came from splitting vendor configuration into autotools configuration:
 https://blueprints.launchpad.net/rpm/+spec/rpm-split-vendor-config-in-autofu
 
 Actual config is the same.
 
 Not sure it's a good idea.
 

Could you be a bit more specific about what isn't a good idea?

Per-vendor configuration: an excuse to not work through
the best solution. Its always based on a Not time, we know what we are doing …
mind-set. I don't mind the divergence if we did not include me doing
support and development.

AutoFu: I've always believed that compiled in options need to become run-time
options. The problem with compiled in options is the assumption that you
can re-compile. That is often not the case any more. And its not as simple
as adding a macro somewhere when there are deep semantic changes like version
comparison, or removing colors and not attaching dependencies to files etc:
configuration should be about file paths, or verbosity levels, and other UI
issues, not fundamental changes to RPM.

But somehow per-vendor configuration needs to be merged/dropped imho: blaming
RPM (and me) for bugs and lack of support on code that isn't well used/tested,
and where vendor support isn't an actuality, is, well, not such a good idea.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 10:43 AM, Belal Salem wrote:

 Hi all,
 compiling the source for any rpm-5.x version doesn't success, I encounter the 
 following error:
 
 rpmaug.c:650: error: 'POPT_ARG_MAINCALL' undeclared here (not in a function)

POPT_ARG_MAINCALL is in popt-1.14 and later. Upgrade your version of popt.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 11:37 AM, Belal Salem wrote:

 Great!
 Updating to popt-1.14 works.
 
 Now,rpmaug.c with POPT_ARG_MAINCALL has passed correctly, but I encounter the 
 same error type different place:
 rpmzq.c:179: error: 'POPT_ARGFLAG_TOGGLE' undeclared here (not in a function)
 
 Are there any other packages that needs updating in addition to popt?
 

All depends on how what build options you choose for rpm: RPM is *NOT*
a build that you can just do
./configure --prefix=/usr
and hope for the best. There's simply too many build options to even
attempt the needed QA coverage explicitly.

The prerequisites for RPM are in INSTALL where it says:

POPTmandatory 1.151.16http://rpm5.org/files/popt/

INSTALL is maintained accurately for the most common build options, certainly 
the MANDATORY ones.

There's also an internal popt distributed with rpm for most of the last year.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 12:39 PM, Belal Salem wrote:

 Yes!
 The repository you give to me is a good place to compare the versions I have 
 and to upgrade what needs upgrading.
 
 Thanks for your time.
 

NP. If you catch some annoyance, its not personal:

I maintain both RPM5 and POPT and yours isn't the first
report of build failures.

The problem isn't simple: RPM uses POPT in easy that no other program does.

E.g.POPT_ARGFLAG_TOGGLE adds the ability to set/clear a set of flags from
CLI options with both
--thisoption
--nothisoption
Since RPM is chock full of disablers (like --nodeps etc) which are attempting
to use POPT_ARGFLAG_TOGGLE in reasonable ways, well, the inability to upgrade
POPT by distros in 3-4 years is, well, a tad bit disturbing.

POPT is *NOT* a hard upgrade, nowhere near as complicated as upgrading RPM has 
become.

Anyways holler at me (and ignore my mutterings) if you need help. I'll try
to get you fixed up no matter what …

(aside to Robert Xu):
Getting there just buried …

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 9, 2011, at 12:54 AM, Belal Salem wrote:

 Thanks for the detailed explanation.
 So, here after I satisfied all the prerequisites, when compiling the RPM 
 source, I encounter the following compilation errors:
 
 In file included from dbconfig.c:14:
 ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'
 ./rpmdb.h:490: error: expected specifier-qualifier-list before 'DB_LOGC'
 ./rpmdb.h: In function 'dbiCopen':
 ./rpmdb.h:589: error: 'struct _dbiIndex' has no member named 'dbi_vec'
 ./rpmdb.h: In function 'dbiCclose':
 .
 .
 
 what do you think that is?!
 

This is the same problem that is being reported multiple times.

In order to do a release -- using --with-db=external as everyone wishes --
rpm has
#include db51/db.h
hardwired in the code.

What everyone wishes is that rpm Just Works with whatever Berkeley DB
happens to be installed *for all possible distros and versions*.

The simplest fix is to build/install db-5.1.25 externally. The portable
(in the sense you will have to edit to taste, the basic build is fine)
that I used is available here

cvs -d :pserver:anonym...@rpm5.org:/v/rpm/cvs get distro/db51

Add the official release of db-5.1.25 from Oracle, and the ancient db.1.85 
tar ball,
and build/install the db51 packages.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Detection of bad packages

[Jeff Johnson]

On Aug 1, 2011, at 4:42 AM, Michael Baudisch wrote:

 Hello,
 i do some tests with RPM and somehow bad packages. When I pass a bad test 
 package to RPM I observered that there's an error like error: open of 
 blablabla failed.  The package is a plain text file  with .rpm extension 
 containing only blablabla.
 But when I put this blablabla in squared brackets ([Blablabla]) there's 
 no error.
 So, what is the magic that does RPM not detect/report a corrupt package? Are 
 there other circumstances under which RPM does not detect/report a bad 
 package? Thanks for help in advance.
 

What rpm version?

All input arguments that are not *.rpm packages
are treated as a manifest of *.rpm packages.

The manifest is split on white space and each item is added
at that position in the argument list.

Not reporting is not the same as not detecting.
Did rpm actually attempt to install anything?

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: libzypp and rpm-5.3.10

[Jeff Johnson]

On Jul 30, 2011, at 1:47 PM, Eric MSP Veith wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Hello,
 
 did somebody have any luck with building libzypp against rpm-5.3.10? I've 
 weeded out most of the simple compile errors, but I'm stuck here:
 

Poky-Yacto has succeeded using zipper - rpm-5.4.0 (API is nearly the same,
rpm-5.4.0 was just to get out of Mandriva's way).

The patches were pretty minor iirc.

 cd /usr/src/gnyubuild/build/libzypp-9.8.9/build/zypp  /usr/bin/c++   -
 Dzypp_EXPORTS -DHAVE_PIPE2 -D_FILE_OFFSET_BITS=64 -DVERSION=\9.9.0\ -
 D_RPM_5 -DHAVE_UDEV -DLOCALEDIR=\/usr/share/locale\ -DTEXTDOMAIN=\zypp\ 
 - -O2 -g -m32 -mtune=generic -march=i686 -fasynchronous-unwind-tables -
 D_RPMDB_INTERNAL -D_RPMDB_INTERNAL  -fno-strict-aliasing -fPIC -g -Wall -
 Woverloaded-virtual -Wnon-virtual-dtor -Wl,-as-needed -fPIC -
 I/usr/src/gnyubuild/build/libzypp-9.8.9 -
 I/usr/src/gnyubuild/build/libzypp-9.8.9/build -I/usr/include/rpm -
 I/usr/include/libxml2 -I/usr/src/gnyubuild/build/libzypp-9.8.9/build/zypp
 - -DZYPP_BASE_LOGGER_LOGGROUP=\zypp\ -o 
 CMakeFiles/zypp.dir/target/rpm/BinHeader.cc.o -c 
 /usr/src/gnyubuild/build/libzypp-9.8.9/zypp/target/rpm/BinHeader.cc
 In file included from /usr/include/rpm/rpm4compat.h:60:0,
 from 

Avoid rpm*compat.h and go native rpm-5.3.x is what I would suggest (what 
Poky-Yacto did).

There is no desire to maintain a compatible API though what's
there mostly works. But you likely need to pick-n-choose because
there's no such thing as a rp[m4 API anymore than there is an rpm5 API.

Things change when necessary: and having an API isn't that high a priority
because most interest is in bindings, not in the API.

 /usr/src/gnyubuild/build/libzypp-9.8.9/zypp/target/rpm/librpm.h:19,
 from 
 /usr/src/gnyubuild/build/libzypp-9.8.9/zypp/target/rpm/BinHeader.cc:12:
 /usr/include/rpm/rpmdb.h: In function 'DB_TXN* dbiTxnid(_dbiIndex*)':
 /usr/include/rpm/rpmdb.h:863:50: error: invalid conversion from 'void*' to 
 'DB_TXN*'
 /usr/include/rpm/rpmdb.h: In function 'int rpmlkId(rpmdb_s*, uint32_t*)':
 /usr/include/rpm/rpmdb.h:873:29: error: invalid conversion from 'void*' to 
 'DB_ENV*'
 /usr/include/rpm/rpmdb.h: In function 'int rpmlkIdFree(rpmdb_s*, uint32_t)':
 /usr/include/rpm/rpmdb.h:884:29: error: invalid conversion from 'void*' to 
 'DB_ENV*'
 /usr/include/rpm/rpmdb.h: In function 'int rpmlkGet(rpmdb_s*, DBT*, 
 uint32_t, void*)':
 /usr/include/rpm/rpmdb.h:895:29: error: invalid conversion from 'void*' to 
 'DB_ENV*'
 /usr/include/rpm/rpmdb.h:899:69: error: invalid conversion from 'uint32_t' 
 to 'db_lockmode_t'
 /usr/include/rpm/rpmdb.h:899:69: error: invalid conversion from 'void*' to 
 'DB_LOCK*'
 

Theses aren't difficult issues to fix: they are all Berkeley DB typedef's 
missing,
grab whatever you need from whatever db.h is in use and bigger out how to
include when compiling.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM Stub Package

[Jeff Johnson]

On Jul 22, 2011, at 10:13 AM, Miller, Vincent (Rick) wrote:

 Hi Anders,
 
 I installed sysutils/file and received an error message indicating that
 file 5.3 supports only version 7 magic files and that the one installed
 was version 8.  Perhaps this is the reason that RPM 5.2.1 did not install
 sysutils/file?
 

If there is a version involved, then see if there isn't a way to rebuild the
*.mgc files for the distributed rules.

Checking … yes here's the file(1) option:

-C, --compile Write a magic.mgc output file that contains a pre-parsed 
version of the magic file or directory.

Unless something is seriously different in modern file(1), one can simply
remove/rename the *.mgc compiled version and the plain text rules file
will be used.

Again, if you have a version mismatch in *.mgc, then there are two
versions of file(1) floating about, one that has zigged, the other zagged,
and that is a distro problem and needs a bug report as Anders suggested.

73 de Jeff

 [root@bsdsandbox2 /usr/ports/sysutils/file]# find /lib |
 /usr/local/lib/rpm/bin/rpmdeps --provides
 error: magic_load(ms, /usr/local/share/file/magic) failed: File 5.3
 supports only version 7 magic files. `/usr/local/share/file/magic.mgc' is
 version 8
 Assertion failed: (mg != NULL), function rpmfcClassify, file rpmfc.c, line
 1132.
 Abort trap: 6 (core dumped)
 
 ==
 
 Vincent (Rick) Miller
 Systems Engineer
 vmil...@verisign.com
 
 t: 703-948-4395
 21345 Ridgetop Cir Dulles, VA 20166
 
 VerisignInc.com
 
 
 
 
 
 On 7/22/11 7:47 AM, Anders F Björklund a...@rpm5.org wrote:
 
 Miller, Vincent (Rick) wrote:
 
 I get an error when I attempt to run rpmdeps.  It complains of a
 missing magic file (see error below).  I can't say this surprises me,
 but I am hoping there is a simple fix, instead of recompiling/installing
 from ports with different optionsŠ
 
 [root@bsdsandbox2 /home/vmiller]# find /lib |
 /usr/local/lib/rpm/bin/rpmdeps --provides
 error: magic_load(ms, /usr/local/share/file/magic) failed: could not
 find any magic files!
 Assertion failed: (mg != NULL), function rpmfcClassify, file rpmfc.c,
 line 1132.
 Abort trap: 6 (core dumped)
 
 I think the magic file it needs is /usr/share/misc/magic
 
 Thoughts?
 
 The archivers/rpm5 port was intended to link against sysutils/file,
 rather than the system version of libmagic that you ended up with...
 
 $ /usr/bin/file --version
 file-5.03
 magic file from /usr/share/misc/magic
 $ /usr/local/bin/file --version
 file-5.05
 magic file from /usr/local/share/file/magic
 
 You should file a Problem Report (PR) against the port, or perhaps
 just install sysutils/file and rebuild. It should match the library:
 
 LIB_DEPENDS=
  magic:${PORTSDIR}/sysutils/file \
 CONFIGURE_ARGS+=
  --with-path-magic=${LOCALBASE}/share/file/magic
 
 --anders
 
 
 __
 RPM Package Managerhttp://rpm5.org
 User Communication List rpm-users@rpm5.org
 
 __
 RPM Package Managerhttp://rpm5.org
 User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM Stub Package

[Jeff Johnson]

On Jul 21, 2011, at 6:59 AM, Miller, Vincent (Rick) wrote:

 Hi,
 
 I've got RPM installed via FreeBSD ports.  The next step is to create the 
 stub package which contains all the files that are already installed.  The 
 vpkg-provides.sh script does not specifically support FreeBSD.  I've 
 considered modifying it to support it.
 

Good.

You won't need … all the files .. generally (but you can do that if you wish).

RPM uses a database to find Provides: that match Requires:.

The stub package is a means to do a database import of certain items.

You will almost certainly need
Provides: /bin/sh
Most other dependencies that are needed are library so names.

You can see what will be needed using /usr/lib/rpm/bin/rpmdeps.

E.g. here are the provides from /lib/* libraries:

find /lib | /usr/lib/rpm/bin/rpmdeps --provides

Note that there is a --requires option, and rpmdeps can take files either
from stdin, or form the CLI argv list.

Building the stub package is little more than capturing output from rpmdeps
into a *.spec file used for markup
Provides: libc.so.6
which when installed will satisfy dependencies of other packages.

(aside)
The vpkg generated stub pkg will also generate md5 digests, and has
the ability to verify those digests using a %verifyscript, which isn't
strictly needed for stubbing out a dependencies, but may be useful.

There's other means to verify digests on a file using a probe dependency.
E.g.
Requires: digest(/path/to/file) = 123456789….
can be added to any @rpm5.org package, and will verify the content digest on
that file path.


 How much of the script will need to be modified to support FreeBSD?  
 Obviously, the case() setting OS defaults would be the first modification 
 necessary, but is there more beyond this that should be modified?
 

Its hard to guess  before hand because the hard task isn't modifying the
script, but rather identifying all the dependencies will be needed.

Simpler than the vpkg script is to supply Provides: dependencies from a file
(1 per line please):

mkdir -p /etc/rpm/sysinfo
echo /bin/sh  /etc/rpm/sysinfo/Providename

and edit in whatever dependencies you need as found.

Its more tedious than hard. But it might help to see what needs to be done
before attempting to automate with a script.

hth

73 de Jeff

 
 ==
 Vincent (Rick) Miller
 Systems Engineer
 vmil...@verisign.com
 
 t: 703-948-4395
 21345 Ridgetop Cir Dulles, VA 20166
 
 VerisignInc.com   

Re: ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'

[Jeff Johnson]

On Jun 27, 2011, at 10:11 AM, Sriram Narayanan wrote:

 Hi Jeff:
 
 Some questions below:
 
 On Mon, Jun 27, 2011 at 7:30 PM, Jeff Johnson n3...@mac.com wrote:
 
 
 There is no simple solution that I see to the above: either more resources 
 @rpm5.org
 are needed (which is unlikely to happen), or users are going to have
 to patch the released version to accommodate their own specific build needs.
 
 
 What are these resources ? People or computers ?
 

Interest mostly. Its utterly trivial to find hardware assets,
just the sysadmin maintenance and QA is already eating up
2-3 weeks per month of my time doing tedious due diligence.

I can't justify that time expenditure any further.

 The necessary changes are not all that hard:
 
There are two places that
#include db51/db.h
is used that need to be changed. Its the db51/ portion
of the include that causes build failures.
 
 
 You've answered this before, but could you please tell us once again
 what would be the repercussions of changing this to
 
 #include db.h
 

That *is* what should be done. Adding any sort of hierarchical
structure to #include's causes issues. E.g. building rpm from
a cvs checkout with
#include rpm/rpmlib.h
is impossible because of the name collision on rpm. Sure
the problem could be fixed by rearranging how directories
in RPM sources (in CVS) are created. But the general fix is
Never do
#include foo/bar.h
Always add -I/prefix/some/where/include/foo to CFLAGS and
#include bar.h

Anyways I'm well aware of the problem, there's a snarl of build
issues conflicting with features related to
Which sqlite should be used: from db-5.x.y or external?
that instantly appear if you try
#include db.h
Been there, done that, wrestled for a month last September with
the issues that cannot be resolved until
Should rpm embed sqlite? How?
have clearer answers.

 And there are these 3 approaches to building RPM:
 
1) install db-5.1.x into
/usr/include/db51/db.h
/usr/lib/libdb-5.1.*
 
 The default make install for bdb creates some other folder structures.
 I've had to symlink from those locations to the above.
 

Yup. Everyone does things differently.

FYI: The *.spec I use on other platforms that do not have
Berkeley DB db-5.1.x is available by checking out the top
level distro module from @rpm5.org cvs. Download the necessary
official release tar ball's from Oracle and build. WORKSFORME
almost everywhere (though I somethings have to adjust paths to
accomodate uglix vendor divergence: *BSD tends to choose different
paths than linux does hysterically while installing Berkeley DB).

 
2) patch RPM to use a different version of Berkeley DB
 
3) re-add Berkeley DB db-5.1.x internal to RPM.
 
 
 What is the reason that you'd removed this ? How about making this the
 default again, with us distro folk having to explicitly specify
 --with-db=external ?
 

The positive justification for removal is
The released tar ball is 50% smaller without Berkeley DB included.

Sure I can flip … flop … flip … flop … forever. I have gone along
with the de facto chosen conventions of
Always use system Berkeley DB.
largely because its a waste of time to continue the discussion.

If the choice was up to me, I'd _ALWAYS_ embed Berkeley DB everywhere
needed. Meanwhile I don't care to endlessly restate my engineering reasons to
linux communityadvocates  that just want to chant
Bloat! Bloat! Bloat! Bloat! Bloat! Bloat!
in spite of what is recommended by Oracle (nee Sleepycat) for 10 years.

 @rpm5.org cannot dictate which of those approaches end-users wish to choose 
 in RPM.
 
 
 I feel that getting rpm5 to build easily is a good goal to have. I've
 learned that if I read your INSTALL document, and then follow
 devtool.conf, then life becomes easier.
 

The flaw here is that RPM has a whopping amount of
Have it your own way!
flexibility. SO there's insufficient context to define easily
meaningfully, there's already a combinatorial failure of the
dozens of build options that cannot ever be properly tested
(even if there was a desire to do so).

E.g. its quite possible to build RPM easily with
no database support
no compression support
no crypto digest support
and have an utterly useless executable that builds easily.

Please note that I'm _NOT_ disagreeing with the goal. Just that
how rpm builds is way way way too complex, and its the
Have it your own way!
complexity that needs to be ripped out imho, not made Yet More Adaptable!

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM5 on FreeBSD: hto* functions

[Jeff Johnson]

On Jun 27, 2011, at 1:59 PM, Miller, Vincent (Rick) wrote:

 On 6/25/11 4:39 AM, Anders F Björklund a...@rpm5.org wrote:
 
 
 
 
 The BSD and Mac ports are currently using 5.2.1, that is the pre-ACID RPM:
 http://www.freebsd.org/cgi/cvsweb.cgi/ports/archivers/rpm5/
 https://trac.macports.org/browser/trunk/dports/sysutils/rpm52/
 
 
 I would be happy with the FreeBSD port of RPM.  I did do an install of it
 and it did not install the rpmbuild binary.  This was one of the main
 reasons for downloading and installing from source…
 

Are you trying to build *.rpm's on FreeBSD? That is
doable, but the binary *.rpm's produced are likely
not going to usable anywhere except on FreeBSD (which
doesn't use rpmbuild) without a fair amount of work.

What problem are you trying to solve?

 Perhaps I'll setup another VM and give the FreeBSD port another try…
 

I can turn on my FreeBSD VM which worked with the %fbsd8x
build options in devtool.conf under buildbot's for several
weeks running make check if there's really a need.

You can have the whole VM if you prepared for a 20+ GB VMFusion download.

I likely can reinstall and redo the entire VM and build RPM from
a cvs checkout in less time than the download takes.

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM5 on FreeBSD: hto* functions

[Jeff Johnson]

On Jun 27, 2011, at 2:39 PM, Miller, Vincent (Rick) wrote:

 On 6/27/11 2:18 PM, Jeff Johnson n3...@mac.com wrote:
 
 
 
 On Jun 27, 2011, at 1:59 PM, Miller, Vincent (Rick) wrote:
 
 On 6/25/11 4:39 AM, Anders F Björklund a...@rpm5.org wrote:
 
 
 
 
 The BSD and Mac ports are currently using 5.2.1, that is the pre-ACID
 RPM:
 http://www.freebsd.org/cgi/cvsweb.cgi/ports/archivers/rpm5/
 https://trac.macports.org/browser/trunk/dports/sysutils/rpm52/
 
 
 I would be happy with the FreeBSD port of RPM.  I did do an install of
 it
 and it did not install the rpmbuild binary.  This was one of the main
 reasons for downloading and installing from sourceŠ
 
 
 Are you trying to build *.rpm's on FreeBSD? That is
 doable, but the binary *.rpm's produced are likely
 not going to usable anywhere except on FreeBSD (which
 doesn't use rpmbuild) without a fair amount of work.
 
 What problem are you trying to solve?
 
 The problem being solved is not really a problem.  A decision was made
 that ops should employ a unified approach to managing software on a
 system.  By unified approach, I mean issuing the same command to install
 something, despite the platform.  The task to integrate RPM into FreeBSD
 was handed to me.
 

Ah, k. Your easiest path by far is going to
be to use Anders build of rpm-5.2.1 in FreeBSD
ports. That SHOULD just drop-in without the
issues you are reporting, and is likely all
you need (and likely prefer) for unified management.

The harder issue with management will be to teach
rpm about what F*BSD provides in ports and natively.
Your unified approach to managing software likely
has reasonably sane exclusions for native software.

There are 2 approaches to teaching rpm about what rpm provides.

1) build a stub pkg automatically that provides all
the usual libraries in an empty (i.e. no files) package.

2) add provides as needed to /etc/rpm/sysinfo/Providename.
E.g. you are almost certainly going to have to do
mkdir -p /etc/rpm/sysinfo
echo /bin/sh  /etc/rpm/sysinfo/Providename
in order to install any package that includes a shell script let.

The approach in 1) is a little bit (but only a little bit) harder
to get into place than the drudgery of stubbing out every provides
encountered.

See scripts/vpkg_provides.sh for the script that automates
generation of the the empty package that provides all the
usual native FreeBSD libraries. The script will generate
a *.spec file that needs
Name:
Version:
Release:
added in order to be built and installed. You need to choose
what you want to call the package.

 
 RPMs that get built on FreeBSD will only be installed in FreeBSD and vice
 versa.
 

That works. If you were interested in build *.rpm packages for
linux from FreeBSD hosting, well, that too is doable just trickier.

 
 
 Perhaps I'll setup another VM and give the FreeBSD port another tryŠ
 
 
 I can turn on my FreeBSD VM which worked with the %fbsd8x
 build options in devtool.conf under buildbot's for several
 weeks running make check if there's really a need.
 
 You can have the whole VM if you prepared for a 20+ GB VMFusion download.
 
 I likely can reinstall and redo the entire VM and build RPM from
 a cvs checkout in less time than the download takes.
 
 Don't worry about your FreeBSD VM at the moment.  I'm just interested in
 the simplest way to get rpm and rpmbuild to function on FreeBSD.  If I can
 accomplish this with the BSD port, that will likely be the method I use to
 integrate.
 

Again rpm-5.2.1 from ports is likely the minimal effort path.

rpm-5.3.x is better code and higher performing, but trickier to get built.

I'd choose the path of least resistance initially to achieve your goal.

I can/will deliver you a rpm-5.3.x package built on FreeBSD if you can
state how you want that package built. It is in fact harder to walk
through the entire set of build options than it is to do the build.

See the %fbsd8x stanza in devtool.conf for what I last used. Note
that as a developer I'm forced to maximally featured (like 4-5 crypto
stacks and 4-5 embeddings and 3+ compressions all on best effort
in order to achieve code coverage and to find flaws. A production
choice of build options could/should be simpler than what is in %fbsd8x.

 I must admit, I am unfamiliar with devtool and devtool.conf.  Google
 returns lots of results.  Is there an official project page for this?
 

Yes devtool is rather unique.

The issue that had to be solved is splitting trees (like Berkeley DB)
out from rpm code in cvs.

So devtool is mostly about reforming a tree from multiple checkouts:
./devtool checkout
The other main aspect of devtool is building with options enumerated
in devtool.conf like this (for a build on FreeBSD from cvs)
./devtool fbsd8x
starting from a bare checkout.

There's also functionality to bootstrap an entire rpm build, also
downloading and building most of the essential build prereqs, that
is *very* nicely done: bootstrapping

Re: RPM5 on FreeBSD: hto* functions

[Jeff Johnson]

On Jun 23, 2011, at 2:24 PM, Miller, Vincent (Rick) wrote:

 Hi,
 
 I'm compiling RPM 5.3.11 on FreeBSD 8.2-RELEASE and encountered an error that 
 is more fundamental than previous errors.  dbconvert.c  makes calls to 
 various hto*() functions, which are not defined in FreeBSD's libc.
 

Replace dbconvert.c with dbconvert.sh.

The program is NOT essential to RPM, nor is the executable used anywhere.

The sources are included for reference.



 When compiling dbconvert.c, the compiler throws an error indicating that 
 htobe32 and htole32 are undefined symbols (see below).
 
 dbconvert.o(.text+0x894):/home/vmiller/rpm-5.3.11/tools/dbconvert.c:184: 
 undefined reference to `htobe32'
 dbconvert.o(.text+0x8a9):/home/vmiller/rpm-5.3.11/tools/dbconvert.c:176: 
 undefined reference to `htole32'
 dbconvert.o(.text+0x8cb):/home/vmiller/rpm-5.3.11/tools/dbconvert.c:176: 
 undefined reference to `htole32'
 
 A site (http://www.man-online.org/page/3-htobe32)  explained that including 
 sys/endian.h would be necessary.  I modified the tools/dbconvert.c to add 
 #include sys/endian.h.  It resolved the initial errors, but produced 
 another indicating bswap32 was an undefined symbol.  However, sys/endian.h 
 does define bswap32.
 

The dbconver.c code is pretty recent and hasn't been compiled much on
non-linux. So I'm not too surprised at portability issue with endian issues.

 Why is the compiler unable to reference the bswap32 definition in 
 sys/endian.h?  Is there a way to force this or is this a sign that installing 
 RPM5 onto FreeBSD will be more of a pain than I had initially anticipated?
 

I can't easily answer programming questions about platforms that I
do not currently have development access to.

Don't build dbconvert.c would be my suggestion.

And reading signs instead of try-and-see can easily trick you imho.

When you finish building, do
cd tests
make check
and report what works and what doesn't.

From the output of make check, I will be able to tell you how
much pain you are in for, and whether you have built successfully.

hth

73 de Jeff

Re: rpm errors upon running

[Jeff Johnson]

On Jun 23, 2011, at 1:04 PM, Miller, Vincent (Rick) wrote:

 I had not loaded berkely-db because I thought I had read somewhere that there 
 was a bundled berkley-db with RPM5.  I have since loaded Berkley-DB 5.1.x 
 from FreeBSD ports and was able to get the compile to move along further.
 

Good.

 It failed later when attempting to link against libgomp which is not 
 installed on the system.  I am planning to reconfigure disabling OpenMP, 
 unless it provides a feature we might be able to utilize.  Can you clarify 
 the features OpenMP provides to RPM5?
 

RPM is headed towards multi-threaded installs.

One way to achieve parallelism would be to use OpenMP.

When looking at feasibility, I parallelized the loop over multiple digests
(not the digest algorithms per se which are nearly impossible to parallelize)
used by RPM and benchmarked rpm digest (which can run 100 digests
in parallel directly off of input/output buffers) and measured a 7x speed-up
on a 4-way.

Since a 7x speedup from a single #pragma is a quite remarkable gain,
I parallelized --verify to see how much gain might be achieved in a
non-artificial real world case. The results there were less impressive
(~1.5x faster from a primitive implementation, since improved to ~2x
back in Jabuary in rpm-5.4.0).

The refactoring cost to use OpenMP #pragma in RPM code would
be considerable, and so likely OpenMP will NOT be used when
RPM becomes multi-threaded.

So the feature provided by using OpenMP in 5.3.1 is that --verify
would run ~1.6x faster on SMP.

I personally wouldn't bother too much with OpenMP on FreeBSD based
on my experiences compiling and using RPM on Mac OS X, where attempting
to use GOMP and GCC is a rather rugged wilderness trek, paddling upstream and 
swatting
lots and lots of mosquitoes while trying to avoid more dangerous fauna.

YMMV of course ;-)

hth

73 de Jeff

Re: Compile Problems on FreeBSD 8.2

[Jeff Johnson]

On Jun 25, 2011, at 4:13 AM, Anders F Björklund wrote:

 vrwmil...@gmail.com wrote:
 
 I am compiling RPM 5.3.4 on FreeBSD 8.2-RELEASE and have encountered a 
 compile time error. I am hoping someone might be able to help resolve the 
 error condition. The error is below. Is the error a result of missing or 
 incorrect versions or libraries?
 
 The easiest way to build RPM on FreeBSD is to use the port*, but if you want 
 a more recent version than 5.2.1 the second easiest is to use devtool. If 
 you follow the %ports comments in devtool.conf, running ./devtool freebsd 
 should configure and make it for you...
 
 * http://www.freebsd.org/cgi/cvsweb.cgi/ports/archivers/rpm5/
 

Actually the standalone %freebsd target that re-invokes devtool
is more likely to have bit rot than building from a cvs checkout and using
./devtool checkout
./devtool fbsd8x
which was running under buildbot's more recently than whatever
is in FreeBSD ports.

And the %fbsd8x target removes a level of recursion that is
quite tricky to explain and support imho.

No matter what:
Have it your won way!
has made RPM nearly impossible to build reliably anywhere, not otherwise.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm errors upon running

[Jeff Johnson]

On Jun 20, 2011, at 10:57 AM, Miller, Vincent (Rick) wrote:

 Hi Jeff…thanks for the reply.  See my replies inline.  Also, I have an 
 rpmbuild error at the tail of this message.
 
 
 From: Jeff Johnson n3...@mac.com
 Reply-To: rpm-users@rpm5.org
 Date: Fri, 17 Jun 2011 19:10:33 -0400
 To: rpm-users@rpm5.org
 Subject: Re: rpm errors upon running
 
 
 On Jun 17, 2011, at 1:43 PM, Miller, Vincent (Rick) wrote:
 
 Hi,
 
 I have installed RPM5 on FreeBSD 8.2-RELEASE and am now unable to do 
 anything useful with it due to errors.  I tried searching the mail archives 
 for similar issues, but only found a reference to DB_CONFIG in reference to 
 initializing a the rpm db.  Since this is a brand new install and there were 
 no db files on the host, I figured it might apply.  It mentions mmapsize, 
 which is definitely set, but offered no recommendations into calculating 
 what to set it to.
 
 
 I don't know how you installed, but
 make install
 from a build SHOULD copy a /var/lib/rpm/DB_CONFIG into place.
 
 The DB_CONFIG file is MANDATORY with @rpm5.org.
 
 The DB_CONFIG file was put into place and the truss output shows DB_CONFIG 
 being read.
 
 
 The contents (from the rpm-5_3 production branch) that pertain to locking 
 tunables are
 
 #  Locking
 set_lk_max_locks16384
 set_lk_max_lockers  16384
 set_lk_max_objects  16384
 mutex_set_max   163840
 
 The heuristic (that I use) is that the mutex_set_max should be ~10x larger
 than the other values, which is about the right scaling.
 
 The size of the others depends on how many packages are managed,
 which is an extrinsic quantity that isn't guessable a priori. 16K - 64K are
 about the right range, and conservative. There's likely something else
 wrong if you appear to need 64K.
 
 How did you build rpm? From CVS? rpm-5.4.1 or rpm-5.3.11?
 
 I downloaded a source tarball for 5.3.4 and installed with the commands below 
 after installing various ports  that appeared to be required.  The ports I 
 installed were popt, beecrypt, pcre, gss, and expat.  Some of these ports had 
 their own dependencies  they installed.
 

You likely want to use the latest production rpm-5.3.11 release instead of 
rpm-5.3.4.
The @rpm5.org project does tick,tock monthly releases.

 ./configure —with-expat CPPFLAGS=-I/usr/local/include 
 LDFLAGS=-L/usr/local/lib LIBS=-lgss
 gmake
 gmake install
 
 
 How many packages do you have?
 
 
 I currently do not have any RPM packages.  There are a couple hundred ports 
 installed, but this should have no bearing on RPM.
 

OK. In order to use RPM on FreeBSD as an add-on you will almost certainly
need to supply some dependencies.

There are two ways to do this:

1) capture the usual provides (like /bin/sh) in a virtual package and
install that first.

2) stub-in dependencies as needed. E.g. you will almost certainly have
to provide /bin/sh somehow, which can be done like this
mkdir -p /etc/rpm/sysinfo
echo /bin/sh  /etc/rpm/sysinfo/Providename

You are likely to find that the two rules
1) Every file requires its parent directory
2) Every symlink requires its endpoint.
to be a bit bothersome as well. Stub those out in /etc/rpm/sysinfo
(or what ever path you have chosen) using
echo /usr/bin  /etc/rpm/sysinfo/Dirnames
echo /path/to/endpoint  /etc/rpm/sysinfo/Filelinktos


 
 
 I'll include as much info as I can here, but it'll mean the length of this 
 mail may be substantial.
 
 The error I receive when I attempt to rpm –qa is:
 
 
 snipped but not ignored
 
 
 #  Locking
 set_lk_max_locks  8000
 set_lk_max_lockers  8000
 set_lk_max_objects  8000
 mutex_set_max 8000
 
 
 Try setting mutex_set_max to 8 not 8000.
 
 I changed mutex_set_max to 8 and ran rpm –qa and it did not produce the 
 original error.  Additionally, it appears to have created the __db files in 
 /var/local/lib/rpm/.  This is a good start.
 
 My next test is to attempt to build an RPM.  I am going with a very simple 
 test, initially, which is to create a new /etc/motd.  When I run rpmbuild on 
 the spec file, I get the following error.  Do you have any thoughts on this?
 
 [root@freebsd /usr/local/src/rpm/SPECS]# rpmbuild -bb 
 vrsnconfig-FreeBSD-amd64-1.0.spec 
 Executing(%prep): /bin/sh -e /var/local/tmp/rpm-tmp.447
 + umask 022
 + cd /usr/local/src/rpm/BUILD
 + rm -rf /var/local/tmp/vrsnconfig-FreeBSD-amd64-root
 + cd /usr/local/src/rpm/BUILD
 + rm -rf vrsnconfig-FreeBSD-amd64-1.0
 + /bin/mkdir -p vrsnconfig-FreeBSD-amd64-1.0
 + cd vrsnconfig-FreeBSD-amd64-1.0
 + + /usr/bin/gzip/usr/bin/tar -dc -xvvf 
 /usr/local/src/rpm/SOURCES/vrsnconfig-FreeBSD-amd64-1.0.tar.gz -
 
 x etc/motd
 + STATUS=0
 + [ 0 -ne 0 ]
 + exit 0
 Executing(%build): /bin/sh -e /var/local/tmp/rpm-tmp.447
 + umask 022
 + cd /usr/local/src/rpm/BUILD
 + cd vrsnconfig-FreeBSD-amd64-1.0
 + exit 0
 Executing(%install): /bin/sh -e /var/local/tmp/rpm-tmp.447
 + umask 022
 + cd /usr/local/src/rpm/BUILD

Re: rpm errors upon running

[Jeff Johnson]

On Jun 20, 2011, at 1:48 PM, Miller, Vincent (Rick) wrote:

 Ok.  Based on your input with regards to the error I am seeing with the 
 rpmbuild issues.  I'll need to rebuild RPM5.  So, I was going to download it, 
 but there are no source tarballs (just source rpms) for newer versions and I 
 cannot download newer sources from CVS due to a firewall.
 
 How do you recommend I download a newer version?

Find scripts/rpm2cpio.sh (its also on the web site).

Download the *.src.rpm and run

rpm2cpio.sh rpm-*.src.rpm | cpio -dim

You will end up w 2 tar balls and one spec file jun the current directory.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: rpm errors upon running

[Jeff Johnson]

 use in this function)
 ./rpmdb.h:924: error: 'struct rpmdb_s' has no member named '_dbi'
 ./rpmdb.h:925: error: 'DB_ENV' has no member named 'log_cursor'
 ./rpmdb.h:926: error: 'struct rpmdb_s' has no member named 'db_logc'
 ./rpmdb.h: At top level:
 ./rpmdb.h:933: error: expected declaration specifiers or '...' before 'DB_LSN'
 ./rpmdb.h: In function 'rpmlgcGet':
 ./rpmdb.h:935: error: 'DB_LOGC' undeclared (first use in this function)
 ./rpmdb.h:935: error: '_logc' undeclared (first use in this function)
 ./rpmdb.h:935: error: 'struct rpmdb_s' has no member named 'db_logc'
 ./rpmdb.h:936: error: '_lsn' undeclared (first use in this function)
 ./rpmdb.h: In function 'rpmlgcClose':
 ./rpmdb.h:945: error: 'DB_LOGC' undeclared (first use in this function)
 ./rpmdb.h:945: error: '_logc' undeclared (first use in this function)
 ./rpmdb.h:945: error: 'struct rpmdb_s' has no member named 'db_logc'
 ./rpmdb.h:948: error: 'struct rpmdb_s' has no member named 'db_logc'
 ./rpmdb.h: At top level:
 ./rpmdb.h:955: error: expected ';', ',' or ')' before '*' token
 ./rpmdb.h:965: error: expected ';', ',' or ')' before '*' token
 ./rpmdb.h: In function 'rpmlgcPrintf':
 ./rpmdb.h:979: error: 'DB_ENV' has no member named 'log_printf'
 ./rpmdb.h: At top level:
 ./rpmdb.h:986: error: expected declaration specifiers or '...' before 'DB_LSN'
 ./rpmdb.h: In function 'rpmlgcPut':
 ./rpmdb.h:989: error: 'DB_ENV' has no member named 'log_put'
 ./rpmdb.h:989: error: '_lsn' undeclared (first use in this function)
 ./rpmdb.h: In function 'rpmmpfSyncAll':
 ./rpmdb.h:999: error: 'DB_LSN' undeclared (first use in this function)
 ./rpmdb.h:999: error: '_lsn' undeclared (first use in this function)
 ./rpmdb.h:1000: error: 'struct rpmdb_s' has no member named '_dbi'
 ./rpmdb.h:1001: error: 'DB_ENV' has no member named 'memp_sync'
 ./rpmdb.h: In function 'rpmmpfTrickle':
 ./rpmdb.h:1013: error: 'struct rpmdb_s' has no member named '_dbi'
 ./rpmdb.h:1014: error: 'DB_ENV' has no member named 'memp_trickle'
 ./rpmdb.h: In function 'rpmmpfClose':
 ./rpmdb.h:1023: error: 'DB_MPOOLFILE' undeclared (first use in this function)
 ./rpmdb.h:1023: error: 'mpf' undeclared (first use in this function)
 ./rpmdb.h:1023: error: 'struct rpmdb_s' has no member named 'db_mpf'
 ./rpmdb.h:1026: error: 'struct rpmdb_s' has no member named 'db_mpf'
 ./rpmdb.h: In function 'rpmmpfGet':
 ./rpmdb.h:1035: error: 'DB_MPOOLFILE' undeclared (first use in this function)
 ./rpmdb.h:1035: error: 'mpf' undeclared (first use in this function)
 ./rpmdb.h:1035: error: 'struct rpmdb_s' has no member named 'db_mpf'
 ./rpmdb.h: In function 'rpmmpfOpen':
 ./rpmdb.h:1047: error: 'DB_MPOOLFILE' undeclared (first use in this function)
 ./rpmdb.h:1047: error: 'mpf' undeclared (first use in this function)
 ./rpmdb.h:1048: error: 'struct rpmdb_s' has no member named '_dbi'
 ./rpmdb.h:1051: error: 'struct rpmdb_s' has no member named '_dbi'
 ./rpmdb.h:1052: error: 'DB_ENV' has no member named 'memp_fcreate'
 ./rpmdb.h:1058: error: 'struct rpmdb_s' has no member named 'db_mpf'
 ./rpmdb.h: In function 'rpmmpfPut':
 ./rpmdb.h:1069: error: 'DB_MPOOLFILE' undeclared (first use in this function)
 ./rpmdb.h:1069: error: 'mpf' undeclared (first use in this function)
 ./rpmdb.h:1069: error: 'struct rpmdb_s' has no member named 'db_mpf'
 ./rpmdb.h:1070: error: 'DB_PRIORITY_DEFAULT' undeclared (first use in this 
 function)
 ./rpmdb.h:1071: error: 'DB_CACHE_PRIORITY' undeclared (first use in this 
 function)
 ./rpmdb.h:1071: error: expected ')' before '_priority'
 ./rpmdb.h: In function 'rpmmpfSync':
 ./rpmdb.h:1080: error: 'DB_MPOOLFILE' undeclared (first use in this function)
 ./rpmdb.h:1080: error: 'mpf' undeclared (first use in this function)
 ./rpmdb.h:1080: error: 'struct rpmdb_s' has no member named 'db_mpf'
 gmake[4]: *** [librpmdb_la-dbconfig.lo] Error 1
 gmake[4]: Leaving directory `/usr/home/vmiller/rpm-5.3.11/rpmdb'
 gmake[3]: *** [all-recursive] Error 1
 gmake[3]: Leaving directory `/usr/home/vmiller/rpm-5.3.11/rpmdb'
 gmake[2]: *** [all] Error 2
 gmake[2]: Leaving directory `/usr/home/vmiller/rpm-5.3.11/rpmdb'
 gmake[1]: *** [all-recursive] Error 1
 gmake[1]: Leaving directory `/usr/home/vmiller/rpm-5.3.11'
 gmake: *** [all] Error 2
 
 ==
 Vincent (Rick) Miller
 Systems Engineer
 vmil...@verisign.com
 
 t: 703-948-4395
 21345 Ridgetop Cir Dulles, VA 20166
 
 VerisignInc.com   
 
 From: Jeff Johnson n3...@mac.com
 Reply-To: rpm-users@rpm5.org
 Date: Mon, 20 Jun 2011 13:56:34 -0400
 To: rpm-users@rpm5.org
 Subject: Re: rpm errors upon running
 
 
 On Jun 20, 2011, at 1:48 PM, Miller, Vincent (Rick) wrote:
 
 Ok.  Based on your input with regards to the error I am seeing with the 
 rpmbuild issues.  I'll need to rebuild RPM5.  So, I was going to download 
 it, but there are no source tarballs (just source rpms) for newer versions 
 and I cannot download newer sources from CVS due to a firewall.
 
 How do you recommend I download a newer version?
 
 Find scripts/rpm2cpio.sh (its also

Re: rpm errors upon running

[Jeff Johnson]

On Jun 17, 2011, at 1:43 PM, Miller, Vincent (Rick) wrote:

 Hi,
 
 I have installed RPM5 on FreeBSD 8.2-RELEASE and am now unable to do anything 
 useful with it due to errors.  I tried searching the mail archives for 
 similar issues, but only found a reference to DB_CONFIG in reference to 
 initializing a the rpm db.  Since this is a brand new install and there were 
 no db files on the host, I figured it might apply.  It mentions mmapsize, 
 which is definitely set, but offered no recommendations into calculating what 
 to set it to.
 

I don't know how you installed, but
make install
from a build SHOULD copy a /var/lib/rpm/DB_CONFIG into place.

The DB_CONFIG file is MANDATORY with @rpm5.org.

The contents (from the rpm-5_3 production branch) that pertain to locking 
tunables are

#  Locking
set_lk_max_locks16384
set_lk_max_lockers  16384
set_lk_max_objects  16384
mutex_set_max   163840

The heuristic (that I use) is that the mutex_set_max should be ~10x larger
than the other values, which is about the right scaling.

The size of the others depends on how many packages are managed,
which is an extrinsic quantity that isn't guessable a priori. 16K - 64K are
about the right range, and conservative. There's likely something else
wrong if you appear to need 64K.

How did you build rpm? From CVS? rpm-5.4.1 or rpm-5.3.11?

How many packages do you have?


 I'll include as much info as I can here, but it'll mean the length of this 
 mail may be substantial.
 
 The error I receive when I attempt to rpm –qa is:
 

snipped but not ignored

 
 #  Locking
 set_lk_max_locks  8000
 set_lk_max_lockers  8000
 set_lk_max_objects  8000
 mutex_set_max 8000
 

Try setting mutex_set_max to 8 not 8000.

 
73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: I killed my RPMDB --- any rescue possible?

[Jeff Johnson]

On Jun 13, 2011, at 9:58 AM, Eric MSP Veith wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On Sunday 12 June 2011, Jeff Johnson n3...@mac.com wrote:
 However, after using rpm -r /my/chroot -Uvh ..., the RPMDB *outside*
 the chroot is corrupted. When running rpm -qa, this is what I get:
 
 So which rpmdb needs fixing? The rpm inside or outside the chroot?
 
 Outside. The host system's RPMDB is broken (after rpm -r ... -Uvh ...). The 
 chroot's RPMDB is intact, but the package I wanted to install is missing, 
 not so surprisingly.  :-)
 

OK.

 And what versions of rpm are in use inside and outside the chroot?
 
 5.3.10 in both cases.
 

OK.

 Underlying what?
 
 Sorry, wrong wording: The bdb version linked with the current rpm-5.3.10 
 install is 5.1.25. The RPMDB was originally created using bdb 4.7.something, 
 and after installing rpm-5.3.10 (with bdb-5.1.25), I immediateley ran 
 dbconvert.sh.
 

OK. There are no significant format changes in Berkeley DB
that are NOT handled by db-utils, specifically db_dump … | db_load.

What is subtle is that one MUST choose the correct version specific
db_dump/db_load to upgrade (and downgrade). I'm not at all
sure that dbconvert.sh is properly implemented to Just Work with
1) a outer - inner chroot system
2) multiple rpmdb paths
3) multiple database versions

The safest way to avoid problems is to do
chroot /your/chroot /bin/bash
and just run dbconvert.sh within.


 Clean-up and save what you have in /var/lib/rpm (...)
 Create a new directory and copy what you have (...)
 Check (as root) that Packages is at least mostly intact:
  rpm -qavv
 
 I did as advised, and rpm -qavv outputs this:
 
 - ---%---
 D: pool fd: created size 208 limit -1 flags 0
 D: pool iob:created size 20 limit -1 flags 0
 D: pool mire:   created size 84 limit -1 flags 0
 D: pool lua:created size 32 limit -1 flags 0
 D: pool ts: created size 720 limit -1 flags 0
 D: pool gi: created size 92 limit -1 flags 0
 D: pool db: created size 184 limit -1 flags 0
 D: pool dbi:created size 284 limit -1 flags 0
 D: opening  db index   /var/lib/rpm/Packages thread:rdonly mode=0x0
 D: pool mi: created size 88 limit -1 flags 0
 D: pool h:  created size 216 limit -1 flags 0
 libXfixes-4.0.3-1ev.i686
 error: rpmdb: header #33554432 cannot be loaded -- skipping.
 (none)-(none)-(none)
 == munmap(0xb65d[180811702]) error(22): Invalid argument
 error: rpmdb: header #67108864 cannot be loaded -- skipping.
 error: rpmdb: header #83886080 cannot be loaded -- skipping.
 error: rpmdb: header #100663296 cannot be loaded -- skipping.
 error: rpmdb: header #117440512 cannot be loaded -- skipping.
 error: rpmdb: header #134217728 cannot be loaded -- skipping.
 error: rpmdb: header #150994944 cannot be loaded -- skipping.
 error: rpmdb: header #167772160 cannot be loaded -- skipping.
 error: rpmdb: header #201326592 cannot be loaded -- skipping.
 error: rpmdb: header #218103808 cannot be loaded -- skipping.
 error: rpmdb: header #234881024 cannot be loaded -- skipping.
 error: rpmdb: header #268435456 cannot be loaded -- skipping.
 error: rpmdb: header #285212672 cannot be loaded -- skipping.
 error: rpmdb: header #301989888 cannot be loaded -- skipping.
 (none)-(none)-(none)
 (none)-(none)-(none)
 (none)-(none)-(none)
 error: rpmdb: header #385875968 cannot be loaded -- skipping.
 (none)-(none)-(none)
 Segmentation fault
 - ---%---
 

OK. You have a damaged database for whatever reason. All of those
header #12345678 are too big so you likely have NOT succeeded
in byte swabbing the header instances (i.e. what dbconvert.sh is doing).

(alternatively) You have run dbconvert.sh an even number of times
and so two applications of dbconvert.sh has undone the byte swabbing
originally intended.

You can try reading the rpmdb with pre-rpm-5.3 to assess how much data
is savable.

Doing rpm --rebuilddb (possibly with --nosignature --nodigest --nohdrchk 
disablers
to get past the segfault) WILL discard damaged headers that you can then 
replace later.

While doing --rebuilddb, make sure you do either
cd /var/lib/rpm
rm -f __db*
or
cd /var/lib/rpm
db_recover -v
(note no -e passed to db_recover to remove the __db cache)

A damaged cache in __db* files can/will screw a --rebuilddb worser.


 
 Re-create the indices (and check that dbconvert.sh swapped the bytes):
  rpm --rebuilddb --vv
 
 - ---%---
 D: pool fd: created size 208 limit -1 flags 0
 D: pool iob:created size 20 limit -1 flags 0
 D: pool mire:   created size 84 limit -1 flags 0
 D: pool lua:created size 32 limit -1 flags 0
 D: pool ts: created size 720 limit -1 flags 0
 D: pool db: created size 184 limit -1 flags 0
 D: pool dbi:created size 284 limit -1 flags 0
 D: rpmdb: cpus 2 physmem 3026Mb
 D: opening  db environment /var/lib/rpm/Packages thread:lock:log:mpool:txn
 rpmdb: Thread/process 18824/3062732608 failed: Thread

Re: I killed my RPMDB --- any rescue possible?

[Jeff Johnson]

On Jun 13, 2011, at 11:05 AM, Eric MSP Veith wrote:

 
 What's the preferred way to initially populate a chroot? 
 

The only difference in rpm-5.3 related ro your issue
is running dbconvert.sh in %posttrans.

The whole idea of converting an *active* rpmdb *automagically*
is _INSANE_. There are far far too many error cases for any
dbconvert.sh script to _GUARANTEE_ success _ALWAYS_.

Everything is _EXACTLY_ as has always been implemented with
rpm and chroot(2). There are no changes:
A script/executable invoked while installing with rpm -r
can escape a chroot and do whatever it wishes.

73 de Jeff




smime.p7s
Description: S/MIME cryptographic signature

Re: yum support in rpm5 ?

[Jeff Johnson]

On Feb 23, 2011, at 12:39 PM, Sriram Narayanan wrote:

 Is there some form of yum support in rpm5 ?
 

Not in, but yum works fine with rpm-python from @rpm5.org.

(aside)
There's one minor issue with a StopIteration exception
from a python iterator for which both a fix and a 3 line
change to yum are known. There's no place to put the fix.

There are features in @rpm5.org that will never be supported
by yum. You can either disable the feature, or otherwise
work around yum developers recalcitrance.

Note carefully:
Yum will NEVER accept any patches from @rpm5.org.
Clearly and publically and repeatedly stated by yum developers.

But I use yum with @rpm5.org all the time, and IDMS linux ships
with y...@rpm5.org as its depsolver. I.e. works.

hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: Info needed - Packages vs compiling on production

[Jeff Johnson]

On Feb 21, 2011, at 1:16 PM, Sriram Narayanan wrote:

 Hi everyone:
 
 I'm asking this question here since there are many distribution
 builders on this list.
 
 I'm having a discussion with some colleagues about the merits of
 installing native libraries via a package manager than by compiling
 source on production.
 
 Compiling source on production is how most Ruby application developers
 deploy rubygems having native bindings onto production servers.
 

SHort answer: both get the job done.

Package management scales, rebiuilding does not.

And package management tracks software over its entire life-cycle,
its not just installing.

(aside)
In fact, the poorly understood reason for RPM's success -- everyone
is so so busy chanting dependency hell instead -- is that rpm
has flawless erases, cleans up everything, never misses.

 My reasoning for using package managers includes the following:
 - the various benefits listed at wikipedia
 (http://en.wikipedia.org/wiki/Package_management_system)
 - dependency resolution
 - file verification (e.g. rpm -V)
 - roll forward - roll back - to the extent possible by various package 
 managers
 

Well --rollback is (mostly) not part of package management. There are only
a few package managers (nix and conary come to mind) that can really do
--rollback without caveats like ... to the extent possible 

The killer for --rollback in package management is the instantaneous
(and quite predictable) RFE from users:
How do I rollback everything except blah?
and the short answer
You don't, that's not how --rollback works.
pleases noone.


 My reasoning against compiling on production:
 - doesn't help resolve dependency hell when you start to upgrade
 libraries partly (when A depends on J1 and B1, while B1 depends on
 J2). A package manager would tell you very quickly when you run into
 such situations.
 - There's some message somewhere about installing only the bare
 minimum files onto a production server. Compilers and Devel files are
 not required on production.
 
 Have such topics been discussed somewhere ? Have you faced such
 questions from your end users or in life ?
 

The archetype for your question goes way way way back to *BSD
make world
as opposed to binary package management (there are source (my words) package 
managers
like MacPorts too).

The best answer I (I'm an old school *BSD developer) know of to
make world - rpm
I got from Erik Troan a long time ago and illustrates the support scaling
that comes from doing binary package management:

Let's say you build  test a package and it works.

Now put that package on a DVD, wait a year or two.

Go find the DVD, install the package, and horrors! the package DOES NOT 
WORK.

The trick question is this
Q: Where do you look to fix the problem?
A: Not the binary package: it was known good and verifiably hasn't 
changed.

Its the long shelf life, and minimizing support necessary for make world 
distros
that scales and makes RPM worth the pain of wrestling with dependency hell.

Caveat:
My opinions aren't typical of most RPM users.

73 de jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: RPM Installation status

[Jeff Johnson]

On Feb 18, 2011, at 10:28 AM, Mircea Lemnaru wrote:

 Hi all,
 
  
 I having some issues with a project that I am working on, in short , the user 
 installs 2 rpm’s , using the command say: “rpm –I pkg1 pkg2” . The first 
 package contains a post install script that fails , this results in an 
 general error / non zero return code so my wrapper over the rpm command 
 assumes the command failed and the rpms did not get successfully installed, 
 but it turns out that if I query the database after the command ended , the 
 packages show up as installed , no mention of the failed post install script.
 
 
 

What version of RPM? This sounds like its _NOT_ the ACID behavior
that is implemented in rpm-5.3.x, where the package header registered
in an rpmdb is part of a transaction, and will be backed out, not left
in an rpmdb, if/when a package scriptlet fails.

(aside)
Please note that there's 2 other parts of what is known as
Transactionally Protected Package Management
that are _NOT_ turned on yet, and so the files installed are the files installed
by the package with the failing %post script. But the database registration 
SHOULD
be fully functional now.

But otherwise the behavior is as expected.

 package with a failing %post will have a header in an rpmdb, and content
on the file system. All that RPM does get right -- mostly -- is that if a %post 
fails,
then the other half of an upgrade aka the erasure of the old files will
be skipped if the install fails for any reasons.

This behavior prevents, say, RPM from accidentally rendering a machine
unbbotable if, say, a glibc upgrade fails for whatever reason. But
its purely disaster avoidance, not anything else. The behavior
you are reporting is exactly what RPM has been doing for years.
 So I need a way to query the database , or look at the output of the initial 
 RPM command to determine which packages failed, and which actually got 
 installed properly - and by properly I mean a package that got installed AND 
 all the script got executed without any error.
 
 

The best answer that RPM has atm is this:
Do proper QA on the packaging to make sure scriptlets don't fail.
Write the scriptlets to take into account every failure mode.

Yes this is hard Q and hard to get prefect scripting.

Meanwhile, there's
rpm -qa --last
that will sort by installation time, and
rpm -qa --dupes
each of which helps find the duplicate partially installed/registered
packages from a failed best effort attempt to upgrade.

 Does anyone know of any way for me do determine if the rpm’s got successfully 
 installed (that includes also the script phase) ? Shouldn’t the rpm database 
 contain also that info ? I mean it seems to me that’s a pretty important 
 information , to know if a post install script executed successfully for a 
 certain installation …
 

The problem here is that successfully installed does not have any semantic 
meaning for
RPM. RPM has only the exit code from a scriptlet, cannot undertake any 
meaningful --rollback
action based purely on an exit code. E.g. package scriptlets can have huge 
side-effects that
are opaquely implemented in (possibly buggy) scriptlets.

All that RPM has is a non-zero exit code to indicate failed to install. And 
the most useful
behavior that most users expect is best effort which is proceeding with all 
the
rest of the installation as far as possible. Manual intervention is always 
needed
for failed to install.

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: RPM Installation status

[Jeff Johnson]

On Feb 18, 2011, at 4:14 PM, pinto.e...@gmail.com wrote:

 In Some rpm version it is possible that a post fail could return always a 
 zero status.there is bugzilla for this. Regards-Original 

TIts possible to mask the exit status, and its also possible for a script to
return 0 and do nothing useful.

But logic and convention sez' that:
All scripts have an exit status in uglix.
An exit status of 0 == success, else failure.

You know all this ... or perhaps you are referring to the daft (imho)
@rpm.org decision to _IGNORE_ scriptt return codes.

Might as well just add
%post -p /dev/random
and start the betting pol to see how long it takes for the
first successful return code from /dev/random.

73 de Jeff  my opinions are all mine



smime.p7s
Description: S/MIME cryptographic signature

Re: Question about rpmbuild --root

[Jeff Johnson]

On Feb 14, 2011, at 3:43 AM, Eric MSP Veith wrote:

 Am Sonntag, 13. Februar 2011, 21:07:45 schrieb Jeff Johnson:
 Depends. --root (as wired in rpmbuild) isn't doing what you think
 its doing. What rpmbuild --root is actually attempting is considerably
 harder to explain, dates back to 2000 or so ...
 
 Hm, well, the manpage says it'll chroot() before running any scriptlets. But 
 skimming through the sources, I found no call to chroot(2). Hum. Manpage 
 bug?
 

Yes but ... rpmbuild --root is not doing chroot(2), never has done chroot(2),
and cannot do chroot(2) unless uid == 0.

It in the man page because someone insisted:
Its a CLI option so it MUST be in the man page.
and it was easier to oblige than explain.

  chroot /wherever rpmbuild ...
 if you want/need single invocation functionality. But you
 likely just want to enter the chroot, and use rpmbuild inside.
  chroot /wherever
  rpmbuild ...
 
 Which, however, will force me to install RPM with all deps inside the chroot, 
 right? 
 

Essentially yes.

 And without having tried it I guess that doing it programatically I could 
 call 
 chroot() before buildSpec() and it *should* work? If so, I'm back working on 
 my small little build system... :-)
 

File path's get implicitly prefixed when chroot(2) is called.

There's harder issues, like readong configuration (from inside or outside the 
chroot?)
and finding the rpmdb (inside or outside?) and writing *.rpm output (inside or 
outside),
but otherwise yes.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: Question about rpmbuild --root

[Jeff Johnson]

On Feb 14, 2011, at 8:31 AM, Eric MSP Veith wrote:

 Am Montag, 14. Februar 2011, 13:26:32 schrieb Jeff Johnson:
 Yes but ... rpmbuild --root is not doing chroot(2), never has done
 chroot(2), and cannot do chroot(2) unless uid == 0.
 
 The latter one I solved by calling sudo rpmbuild. 
 

So no figger where rpmbuild is supposed to read its configuration from.
That's a bit harder than using sudo to get uid == 0.

 It in the man page because someone insisted:
  Its a CLI option so it MUST be in the man page.
 and it was easier to oblige than explain.
 
 What's the difference between '--root' and '--dbpath' then?
 

Depends on context.

Traditionally, RPM implicitly added --root to --dbpath when opening
an rpmdb and expected the user to know when to add the chroot
prefix to the --dbpath argument for non-root execution.

But PLD wanted --rebuilddb --root instead of --rebuildb --dbpath
(where one had to manually insert the root prefix in the --dbpath argument).

So --root and --dbpath behavior was changed.

The change has broken 4-5 times over the years and is way too complex
and undocumented and more.

 File path's get implicitly prefixed when chroot(2) is called.
 
 There's harder issues, like readong configuration (from inside or outside
 the chroot?) and finding the rpmdb (inside or outside?) and writing *.rpm
 output (inside or outside), but otherwise yes.
 
 I can answer these questions in case of what I'm planning to do. I mainly 
 don't want to populate the buildroot with all dependencies of RPM, that's 
 all. 
 But I can live with copying macro files and other config data.
 
 Ok, off trying... :-)
 

Good luck!

FYI: --root takes a /string which is used to construct a command that looks like
rsh -c chroot /string ...
that isn't what anyone is expecting but is exactly what was intended.

73 de jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: Question about rpmbuild --root

[Jeff Johnson]

On Feb 13, 2011, at 2:23 PM, Eric MSP Veith wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Hello list,
 
 I try to build in a chroot with an old RPM-5.1.9 using 
 
   sudo rpmbuild --root=/var/tmp/eglibc-2.13.buildroot -ba eglibc.spec
 
 Basically, what I want to know, is: Am I doing something completely wrong 
 besides using an old and dusty RPM version? ;-)

Depends. --root (as wired in rpmbuild) isn't doing what you think
its doing. What rpmbuild --root is actually attempting is considerably
harder to explain, dates back to 2000 or so ...


Try
chroot /wherever rpmbuild ...
if you want/need single invocation functionality. But you
likely just want to enter the chroot, and use rpmbuild inside.
chroot /wherever
rpmbuild ...

hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: Recommendations on initializing the system rpmdb ?

[Jeff Johnson]

On Jan 2, 2011, at 12:48 AM, Sriram Narayanan wrote:

 Hello:
 
 1. What are the recommendations on initializing the rpmdb ?
 

No initialization per se is needed:

[...@rhel6 wdj]$ rpm -qa --dbpath /tmp/blah
[...@rhel6 wdj]$ ls -al /tmp/blah
total 13720
drwxr-xr-x   2 jbj  jbj  4096 Jan  2 02:45 .
drwxrwxrwt. 19 root root20480 Jan  2 02:45 ..
-rw-r--r--   1 jbj  jbj 24576 Jan  2 02:45 __db.001
-rw-r--r--   1 jbj  jbj  10493952 Jan  2 02:45 __db.002
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.003
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.004
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.005
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.006
-rw-r--r--   1 jbj  jbj163840 Jan  2 02:45 __db.007
-rw-r--r--   1 jbj  jbj   4554752 Jan  2 02:45 __db.008
-rw-r--r--   1 jbj  jbj 40960 Jan  2 02:45 __db.009
-rw-r--r--   1 jbj  jbj  10485760 Jan  2 02:45 log.01
-rw-r--r--   1 jbj  jbj  8192 Jan  2 02:45 Packages

You WILL need to configure cache/mmap sizes and particularly
the number of locks by adding a DB_CONFIG file.

 2. I've tried to use /usr/local/lib/rpm/vpkg-provides.sh
 This asks for --spec_header '/path/to/os-base-header.spec'
 

vpkg-provides.sh is for a installation model where a traditional software vendor
(like in Apple Mac OS X) provides a native OS and *.rpm packages are added
on top of the native os.

You will need to add a few tags like Name:, Version:, and Release:
(i.e. that's what is in os-base-header.spec if you wish to use
vpkg-provides.sh. The rest of the script should generate a virtual
package with library sonames and a %verifyscript to check library
file digests.

 a. What are the recommended contents of this spec file ?
 

Essentially just the tags that prevent the vpkg-provides.sh
output spec file from building.

 b. What do other distros do ?
 

Most distros use RPM for everything, in which case vpkg-provides.sh
is unnecessary. I've done packaging on Mac OS X, using vpkg-provides.sh
to jump start a spec file.

Note that Provides: can be added (1-per-line) to a file. E.g.
mkdir -p /etc/rpm/sysinfo
echo /bin/sh  /etc/rpm/sysinfo/Providename
supplies the essential pre-requsite to run *.rpm scriptlets.

The ability to configure Provides: in a flat-file was re-added because
there were too many details to explain about how to use RPM packages
such as what vpkg-provides.sh does.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How do I set the _topdir ?

[Jeff Johnson]

On Jan 1, 2011, at 12:46 PM, Sriram Narayanan wrote:
snip

 /usr/local/lib/rpm/%{_target}/rpmpopt

Note the unexpanded macro here. That's Have it your own way!
flexibility as compiled.

snip

 
  0   2237 open64:entry rpmbuild /home/sriram/.rpmmacros
 

This SHOULD have read wyat is in ~/.rpmmacros.

You can verify with
rpm --showrc | grep _topdir

 If I want to set /workspace/rpmwork as the topdir (which would contain
 SPECS, SOURCES, RPMS, etc), what should the contents of .rpmmacros be
 ?
 
 I've tried the following:
 topdir=/workspace/rpmwork
 _topdir=/workspace/rpmwork
 topdir:/workspace/rpmwork
 _topdir:/workspace/rpmwork
 

All macros in configuration file loads have to start with '%' in
1st column.

So you need this

%_topdir/workspace/rpmwork

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Wondering status of rpm5 in Mandriva?

[Jeff Johnson]

On Dec 17, 2010, at 10:29 AM, Dwight Paige wrote:

 On 12/16/2010 11:28 PM, Jeff Johnson wrote:
 
 There's a few issues that have been fixed this week:
  /etc/passwd (and %config handling) was screwed
  parentdir dependencies used for ordering were screwed
  file triggers and/or %posttrans wasn't functional w --root
 Reported by IDMS but confirmed in Mandriva.
 
 The recent segfaults I've heard (but not personally seen) in Mandriva were
 with DistEpoch: which is a bit tricky to deploy but since
 every package will use, issues are easy to see and fix.
 
 Per Oyvind knows details better than I. I just sweep up the bugs ;-)
 
 hth
 
 73 de Jeff
 
 __
 RPM Package Managerhttp://rpm5.org
 User Communication List rpm-users@rpm5.org
 
 Thanks. As I understand we are basically awaiting perl-URPM 4.3. Per Oyvind 
 has reported difficulties getting it through build system. Most recent is:
 
 perl-URPM-4.2-1mdv2011.0.x86_64.rpm 03-Dec-2010
 
 and there doesn't appear to be anything here:
 
 http://kenobi.mandriva.com/queue/queue/cooker/main/testing/
 
 So I'll just wait and see what happens.

Careful: Some of the rpm-5.3.6+ changes -- particularly %config handling -- WILL
affect your Mandriva upgrades. All I'm saying is there is a rpm-5.3.7 package
from the distribution mirrors that is needed in addition to perl-URPM-4.3.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Building a 32-bit RPM on a 64-bit machine?

[Jeff Johnson]

On Oct 29, 2010, at 9:07 AM, Joe Flowers wrote:

 Hello again,
 
 I would like to be able to create a 32-bit RPM (with no 64-bit library 
 dependencies) on a 64-bit machine.
 
 I have a 64-bit, SuSE development machine, and I would like to create an RPM 
 on this 64-bit machine that will install 32-bit software on a 32-bit SuSE 
 machine.
 

SuSE does multilib differently than RedHat afaik. IIRC, there are 3 products
1) ELF32 only
2) ELF64 only
3) multilib.
What's missing is all the needed other persuasion packages on 1) and 2).

So what you want is no different than setting up a cross-compiler, or
other wise setting up host - target infrastucture.

But its rather easy to set up whatever you need with an install into a 
chroot(2).
That does need root access to execute chrott(2).

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Creating an RPM to install a daemon

[Jeff Johnson]

On Oct 28, 2010, at 11:10 AM, Joe Flowers wrote:

 Hello Everyone,
 
 I am trying to create an RPM that will install a daemon correctly, but I'm 
 not sure if or where I should put the command:
 
 chkconfig --level 345 /etc/rc.d/mydaemon on
 

(aside)
Well first of all, chkconfig doesn't do the right
thing in many cases like installing into a chroot,
or with SELinux enabled, or if networking isn't
configured, etc etc etc.

So one can make a strong argument (I just tried, YMMV) for _NOT_
doing chkconfig in package scriptlets whatsoever, but rather
configure daemon startup (if changed) in some other way,

The underlying design principle that is violated with chkconfig in packaging
scriptlets is that
package management != configuration management
the important difference is the persistence involved:
The time scale and state for package management is rather different
than that for configuration management.
And in many cases :One size does not fit all usage cases.

But the specific answer to your question is usually to add to %post/%preun.

You likely have examples on your system using chkconfig already installed:

rpm -qa --scripts  /tmp/scripts

and then look for an example of what is being attempted with chkconfig.

Do the same thing.

 
 Should this line go somewhere in the Makefile (like the install section), 
 or should it go in the RPM .spec file somewhere?
 

Not the Makefile, but in the %post or %preun scriptlet in the spec file.

 Is there some other command I should use rather than this external 
 chkconfig program?
 

For configuring a daemon to start at a run level, chkconfig is what you want.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Finding out triggered package version

[Jeff Johnson]

On Jul 14, 2010, at 9:28 PM, Jeff Johnson wrote:

 
 The package that was just installed is what fired the trigger
 script. Perhaps easiest to get the last package installed is
   rpm -qa --last | gead -n 1
--- head of course
 but there's likely simpler if I thought a bit.
 

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Replacing a file already owned by another rpm package

[Jeff Johnson]

On Jul 12, 2010, at 1:53 AM, Oguz Yilmaz wrote:

 Hi all,
 
 I am on a task for packaging a kernel module.
 kernel-xx.rpm has already e1000e.ko module in it.
 
 I will make a new rpm with e1000e.ko with the same path.
 
 What do you suggest for installing new rpm without conflicts and
 force? There may be some hacks in my mind:
 For example:
 Adding e1000e.ko file with another path in the rpm and use postinstall
 script to replace old file.
 

Using %post is one way. The usual failure point with %post is
that the effect of replacing a file in another package
doesn't survive an upgrade. i Then a %trigger* script is needed.

But for replacing e1000e.ko a %post will work, kernel packages
are never upgraded.

See also the Dell Kernel Module System (DKMS if I've remembered
the name). DKMS is an orderly way to add a 3rd party driver to
an existing kernel package, including rebuilding when the kernel
is replaced.

 However, if exists and if possible, I want to use most standard way of
 doing this.
 

There is no 1 standard way to replace a file in another package using RPM.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Replacing a file already owned by another rpm package

[Jeff Johnson]

On Jul 12, 2010, at 8:14 AM, Oguz Yilmaz wrote:

 
 I think %post will not prevent the conflict error, and package will
 not be installed without --force. However, because I am using yum
 update for my systems, I have to have all packages installing without
 any --force.
 

Yes there will be a file conflict (which is different than a
package Conflicts: assertion) when a package install is attempted
and some other package has different content on the identical path.

Rename the e1000e.ko in your package, and use %post to copy
to the final path.

 If the is not a legal method for replacing a file of another rpm, I
 will add a postinstall script while adding the file with another path;
 and move the file to the original location in postinstall script.
 

The is *NO* supported means in RPM to replace a file in another
package. RPM is a package, not a file, manager is the deep reason.

The correct/supported RPM packaging solution is to patch the kernel
package however you wish and replace the entire kernel package.

But %post or %trigger or DKMS or alternatives(1) are all means that will work:

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: About the consistency of /var/lib/rpm/__db.00?

[Jeff Johnson]

On May 25, 2010, at 2:02 PM, Jeff Johnson wrote:
 
 There are an immense number of off buttons in RPM, none
 of them very useful or correct, but I can/will tell you
 what to do.
 


OK I have a checkout from @rpm5.org onto the tag
rpm-4_0_4-release
and here's the gory (untested, I'm just reading code) details
about how to make __db* files go away with RPM macro configuration.

(aside)
I suggest the KISS approach of shell wrapping
rm -f /var/lib/rpm/__db*
files becuase none of rpm-4.04 is maintable or fixable or anything else.
JMHO, YMMV, everyone's does.

The basic configuration details are attributes while opening
tables (what Berkeley DB calls databases) are resident in
/usr/lib/rpm/macros. Change the configuration to taste.

E.g. I'm referring to lines in /usr/lib/rpm/macros that
look like
%__dbi_rebuild  nofsync !log !txn !cdb
%__dbi_transient%{__dbi_rebuild} temporary private
%__dbi_permsperms=0644
Note the syntax:
1) attributes are enabled(or disabled) like
nofsync
of (for disabling)
!log
2) there are KEY=VALUE parameters like
perms=0644

1) private
This disables _ALL_ locking by Berkeley DB,

2) usedbenv
This disables using a Berkeley DB environment _WHATSOEVER_.

3) cachesize=nnnMb
This is one (of several) Berkeley DB tunables related to
not happening I/O and the size of __db* files. Some of
the others are nofsync, mp_mmapsize, mp_size, and pagesize.
I personally haven't seen any use (in rpm-4.0.x) of changing
any of the above, but I know quite well what to do with the above
in rpm-5.3.

4) verify
This enables calling (*DB-verify) (which I believe also forces
__db* removal but these are dim memories) when a rpmdb is closed.

5) teardown
This (IIRC) will remove __db* files at end of every RPM invocation.

hth Have fun! All of the above turned out Not To Be The Right Thing To Do.

73 de Jeff

4) 
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: About the consistency of /var/lib/rpm/__db.00?

[Jeff Johnson]

On May 25, 2010, at 2:59 PM, Jeff Johnson wrote:

 
 hth   Have fun! All of the above turned out Not To Be The Right Thing To Do.
 

One last note ...

Berkeley DB for many years now will read its own configuration
from a DB_CONFIG file (i.e. /var/lib/rpm/DB_CONFIG) that will
override any tunable that RPM attempts from the macros I've
just described.

The contents of DB_CONFIG file are quite completely documented in every
released version of Berkeley DB I've used for years, certainly for all
of db-4.x.y versions. But yes you will need to dig out the details.

SO if you are attempting to make not happening I/O happen, I suggest
just configuring Berkeley DB using DB_CONFIG, not RPM macros.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: About the consistency of /var/lib/rpm/__db.00?

[Jeff Johnson]

On May 21, 2010, at 7:35 PM, Marc MERLIN wrote:

 Howdy,
 
 So running most rpm versions from this century :) creates
 /var/lib/rpm/__db.00? when accessing the rpm dbs.
 
 My understanding is that they are a temporary index created by the DB
 libraries and I know they get recreated as needed.
 

Nope. There are (at least) 3 RPM user visible uses of __db* files, none of them
temporary or index:
1) a version stamp
2) names of locks (and with thread_count, stale lock registration
for clean-up after exceptional conditions like reboot and segfault
and programmer error). Note names, not the locks themselves.
3) data caching

(there are other uses, but the above are the most important for a DB_INIT_CDB 
concurrent
access model used in legacy rpmdb's. (@rpm5.org uses a different model now,
with full ACID proerties).

 I however have 3 questions I wasn't able to answer because I manage
 machines where most of the system is rsynced (including packages and the
 rpm DB).
 
 1) can I configure rpm and/or the DB lib not to create those files?
   (they cause issues when we look at a 32bit image on a 64bit system
   for instance since rpm64 will try to read them and fail with a weird
   error message, also they are bloat on the image syncing process that
   I'd rather do without)
 

Remove the files when moving from 32 - 64 bit. This is/was done automagically
several years now for the DB_INIT_CDB model. While the error message
may be weird, it is in fact the error message that indicates that
you need to remove the __db* files.

 2) if I can't stop them from being created, would I be able to look for
   some exit code path in the rpm source and just delete them there?
 

You don't want to delete the files because that _IS_ where the locks
are found. Removing the files introduces races (but is perfectly okay
if you are _SURE_ that nothing else is using an rpmdb at that moment,
which is often a pretty safe assumption).

 3) what if a system has an rpmdb with those files and I revert the rpm
   db files (excluding those specific __db.00? files from the sync) so
   that the rpm db is now different form what it was when they were
   created. 
   Does that create an inconsistency that could severly confuse RPM?
 

What happens if you do
cp /dev/zero /dev/mem
;-)

What is the sync? Are you using rsync to copy between machines?
Don't copy __db* files, they are useless on any other machine,
can only be used on the local machine. (Well one _CAN_ have
concurrent access through NFS+fcntl from multiple sufficiently
similar machines, but let's not go there ...)

There are failure modes if you screw around, typically DB_PAGE_NOTFOUND
when a page has gone AWOL in the cache. But rpm needs very little
from an rpmdb and deliberately has a KISS schema so that --rebuilddb
regenerates all the secondary indices, any time you want, and
the primary store of headers is both digest/signature checked, so
inconsistency is largely limited to dropping a registered package
that fails signature/digest/sanity checks during --rebuilddb.

Again, rpm-5.3 has full transactional protection and ACID properties,
so none of the above comments apply to rpm-5.3.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: About the consistency of /var/lib/rpm/__db.00?

[Jeff Johnson]

On May 21, 2010, at 7:35 PM, Marc MERLIN wrote:

 
 I however have 3 questions I wasn't able to answer because I manage
 machines where most of the system is rsynced (including packages and the
 rpm DB).
 

Database backups are a bit trickier than just blasting with rsync because
all the files need to be consistent.

Likely the best method is to do
rsync /var/lib/rpm/package
and then do
rpm --rebuilddb

One can rebuild into a different directory and check if paranoid;
but that's largely what --rebuilddb is already doing.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: About the consistency of /var/lib/rpm/__db.00?

[Jeff Johnson]

On May 21, 2010, at 10:52 PM, Marc MERLIN wrote:

 
 Right. So it sounds like
 1) __db* files can't be turned off since for most users they are useful (not
   for us though). 

I'm not sure how you arrive at this conclusion. Dataabses (with concurrent 
access) locks,
the locks are in __db* files, and only userst need locks?!?

But there are many alternatives, including just disabling all locks.

 2) I could hack rpm to find all the exit paths or wrap it and auto delete
   the db files on exit

Its easier than that. There is code that removed __db* files at end-of
execution. It very much was not the right thing to do, but Have fun!

 3) if I have an rpmdb synced over a machine that has non matching __db*
   files, nothing really bad will happen, but rpm may act weird and it's
   just better not to get in that situation, potentially via #2 or other
   means.
 

Why sync files that aren't useful in the first place?

73 de jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: package foo-1.1.1.x86_64.rpm is intended for a x86_64 architecture

[Jeff Johnson]

On May 12, 2010, at 2:55 PM, Marc MERLIN wrote:

 
 Any idea how I can tell the new rpm to rely on my arch as in uname -m as
 opposed to whatever else it is doing?
 
 Oh, I forgot, 
 eou15:/tmp# rpm -qRp foo-1.1.1.x86_64.rpm 
 monitoring  
 rpmlib(CompressedFileNames) = 3.0.4-1
 rpmlib(PayloadFilesHavePrefix) = 4.0-1
 
 The binary does require a 64libc, but I have one.
 

Hint: If you want to see all the metadata to, say, diagnose
which tags contain what, try adding --xml.

73 de Keff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Glibc %post

[Jeff Johnson]

On May 10, 2010, at 9:50 AM, Eric MSP Veith wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On Monday 10 May 2010, Jeff Johnson n3...@mac.com wrote:
 If you want *RE's you might try embedded perl instead of lua.
 
 I missed something, please enlighten me: %post -p perl does not work here 
 with rpm 5.1.9? If embedding Perl is as easy as embedding lua, I'll use 
 perl, since what I want to archive already has some working Perl code. I 
 mean, using %{perl: ...} will *expand* whatever I produce and put it into 
 the spec file, I don't want this. I merely want to run some Perl/Lua code 
 from the spec file and have access to the specs %defines.
 

Yup, no embeddings in 5.1.9. The mebeddings were added in rpm-5.2.0 which
was supposed to be released 2 weeks after rpm-5.1.9 was.

But the upgrade to rpm-5.2.1 released today should have all the embeddings,
and should be quite straight forward.

The other approach would be back porting, its quite easy to drop-in
rpmio/rpmperl.[ch] and patch the 4-5 needed places for embedding in RPM.

What is _MUCH_ harder is to get the infrastructure in place to build
with an embedded language. Embedding API's are usually buried deeply
in per-language trees with obscure and frustrating compilation issues.

But I've managed to sort most of the details without too much pain. But
it usually takes me quite a bit longer to figger the linkage than it does
to implement Yet Another Embedding in RPM, the implementation is mostly
cookie cutter cut-n-paste.

 OTOH, if you need legacy portable RPM embedding, there's only lua,
 but you will likely find that legacy applied to the embedding is
 not as important as looking carefully at what Lua extensions are
 implemented.
 
 I'm not into legacy, not with this distro of my own creation. ;-)
 

Well, there's another for of legacy called future proofing. One
of these days I'm gonna have to rework how lua is embedded, what's
implemented currently doesn't have strong enough conventions
for loading from sysem - vendor - application - ... paths,
and could/should be simplified to use an rpmiob rather than a rpmluab

(An rpmiob is juts a container for octets, related to the macro expansion
limit you asked about).

rpmiob == I/O Buffer is the acronym.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Fwd: rpmdb: c_put: attempt to modify a read-only tree

[Jeff Johnson]

I replied privately but perhaps useful.

Begin forwarded message:
 
 - Forwarded message from Marc MERLIN marc_...@merlins.org -
 
 From: Marc MERLIN marc_...@merlins.org
 To: rpm-users@rpm5.org
 Subject: rpmdb: c_put: attempt to modify a read-only tree
 
 After upgrading rpm on a system from 4.0.2 to 4.4.1, I'm seeing this:
 
 muny20:/lib# rpm -e --test sed
 rpmdb: c_put: attempt to modify a read-only tree
 error: db4 error(13) from dbcursor-c_put: Permission denied
 rpmdb: c_put: attempt to modify a read-only tree
 error: db4 error(13) from dbcursor-c_put: Permission denied
 rpmdb: c_put: attempt to modify a read-only tree
 error: db4 error(13) from dbcursor-c_put: Permission denied
 error: Failed dependencies:
   sed is needed by (installed) console-tools-19990829-34.i386
   sed is needed by (installed) pam-0.74-22.i386
   /bin/sed is needed by (installed) initscripts-5.84-1.i386
 
 
 Hmmm ... I personally have never seen this error msg that I recall ...
 
 what platform and vendor distro?
 
 what version of rpm is in use for upgrade, 4.0.2 or 4.4.1?
 
 what version of Berkeley DB? These are really old RPM versions and I've 
 forgotten details.
 
 (aside)
 Hmmm, I'd skip rpm-4.4.1 (was it SuSE who used? I fergit ...) not
 my best effort ... and 4.4.2 should be easy to upgrade to.
 
 
 It doesn't quite seem to stop rpm from working, but it's worrisome and I 
 can't
 quite find what it means.
 
 
 There's gonna be data loss (because db-c_put was involved).
 
 But --rebuilddb afterwards will fix all the indices (except for Packages,
 do rpm -qa | sort before and after looking for packages that go *POOF*).
 
 Note also that rpm checks _EVERY_ return code from Berkeley DB, so
 you don't have to get too paranoid about mysterious internal crud.
 You should get an error msg as above for all anomolies.
 
 I'd also look carefully for something really odd, like rpm reading some
 fishy configuration (strace -e open should catch), or chattr(1) on a database 
 file,
 or an rpmdb on some non ext2/ext3 file system.
 
 hth
 
 73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Glibc %post

[Jeff Johnson]

On May 9, 2010, at 5:19 PM, Eric MSP Veith wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Hello list,
 
 is there a good and clean way of upgrading glibc on a running system via 
 RPM? Fedora, RedHat and SuSE ship a glibc_post_install.c file that seems 
 to take care of this, Slackware just does some linking/copying, like first 
 putting everything /lib/incoming, then ldconfig'ing links from /lib to 
 /lib/incoming, then copying it to /lib, then doing ldconfig again.
 
 The crucial part is always the actual swapping, i.e. re-setting the links. 

You mean running /sbin/ldconfig? yep.

 I'm wondering whether someone has to offer some insight on how to switch in 
 a good manner a C library? :-)
 

Not sure what switch in ... a C library means.

The issue upgrading glibc is that side-effects of upgrading a library
cannot be handled with /bin/sh scripting because glibc is a necessary
pre-requisite of all ELF executables. SO if you miss something upgrading
glibc _EVERYTHING_ dies.

Other C libraries are quite simple to upgrade -- the issue is entirely glibc 
itself.

This is the inner circle of dependency hell in RPM which contains ~10-13 
packages.

Note that the pugliness of glibc_install_post being statically linked and
used as a helper for glibc is hysterical nonsense:

A long time ago in a far and different place, NS libraries changed 
incompatibly,
thereby breaking remote access through SSH, and causing quite a furor 
about
glibc upgrades through RPM. Since NS libraries change like, say, once 
every 5+
years, you _REALLY_ needn't worry about the issue, nor is the 
glibc_install_post
statically linked helper _REALLY_ needed.

Disclaimer: I haven't looked recently, my barf bag overflowed when I last 
looked 2 years ago.

Meanwhile, since embedded lua has been around for years in RPM, and everything 
being
done (last I looked) by glibc_install_post could easily be done by embedded lua,
you might well look into replacing
%post -p /usr/sbin/glibc_post_upgrade.i686
with
%post -p lua
... whatever glibc_post_upgrade.i686 does but in Lua ...

That would benefit almost all RPM users, since the code in glibc_install_post
is exceptionally contorted and twisty and unreadable and hard.

I'll wager the same task that /usr/sbin/glibc_post_upgrade.i686 does can be done
like 10x fewer lines of lua than in C and voo-doo ASM. ANy takers?

Disclaimer:
I already know my answer: The house collects on 00 and 0 no matter 
what ;-)

73 de Jeff





 Thanks alot in advance,
   Eric
 
 
 P. S. For @rpm5, I'm still maintaining my todo list (i.e., rpm5 make check 
 on Arch Linux, more docs and web site twiddeling), but won't get to work on 
 it for at least two weaks, just to let you know. Work has been raining on me 
 like hell the last days, so it'll be stalled. Sorry.
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.9 (GNU/Linux)
 
 iEYEARECAAYFAkvnJt8ACgkQhS0drJ3goJKrNACeNWvyD4JLu3mDFHKyGIcSyVkA
 FGQAn3Z8d66EVKfCjdvjLd4d/nKOeTzG
 =dlTE
 -END PGP SIGNATURE-
 __
 RPM Package Managerhttp://rpm5.org
 User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Glibc %post

[Jeff Johnson]

On May 9, 2010, at 6:55 PM, Eric MSP Veith wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On Monday 10 May 2010, Jeff Johnson n3...@mac.com wrote:
 Well you _REALLY_ don't want to do any of that stuff while
 upgrading glibc reliably: do your dirty and move on, this _IS_
 the innermost circle of the dependency inferno.
 
 Yikes, no, not for the GLIBC package, sorry for the misunderstanding. This 
 question was more generally spoken.
 
 What I wanted to ask is: Consider I have some init file as Source42: 
 myfoo.init, which has stuff like %{_sbindir}/foodaemon or 
 %{my_custom_specfile_define} in it. Is there a way to design a lu 
 macro/script I can call from a spec file to do this kind of subsitution? In 
 a way of %{install_init_file %{SOURCE42}}?
 

So you want macros expanded generally within, say, %config file bodies?

Quite doable, its 20 line of code (including the pesky memory management).

RPM macros are quite good ate the templating mehcanism.

But you, the end luser, MUST ensure that RPM is configured correctly.

But if you want file content macro expanded, then experiment with, say
rpm -E `cat /etc/sysconfig/iptables`  /etc/sysconfig/iptables
(just to pick on some obscure %config file).

All depends on getting your head around templating rather than patching.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Glibc %post

[Jeff Johnson]

On May 9, 2010, at 8:18 PM, Eric MSP Veith wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On Monday 10 May 2010, Jeff Johnson n3...@mac.com wrote:
 So you want macros expanded generally within, say, %config file bodies?
 
 Nope, not generally. That would be overhead. Although it wouldn't hurt, what 
 I have in mind is more like a embedded lua script that does macro expension 
 in a single file where I want it to expand macros, among other things.
 
 Basically, what I'm asking is: Does rpm.expand() have (a) a size limit and 
 (b) the ability to expand things I define in the spec file?
 

Ah the macro expansion limit.

The limit is currently at 110K iirc. The buffer size is tunable in 
rpmio/macros.c,
the 110Kb is known to work for OpenPKG (which is using embedded lua 
extensively).

lua _IS_ a programming language, you should not be limited by whatever size
a macro expansion buffer is. If you do hit the limit, you should likely
change your approach to use, say, /tmp files! wotta concept!

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: newbie src.rpm question

[Jeff Johnson]

On May 8, 2010, at 2:25 PM, mike wrote:

 Sorry, I should have included the spec file.  Here it is:
 

snip

Hehe, no worries, I wrote that spec file back when I was
Drepper's elfutils packaging bitch @redhat.com. ;-)

(aside)
I can't say that elfutils.spec has improved with age,
the ELF cabal does like their nerdy complexity imho.

BTW, what is the robust patch supposed to do? I've
lost track of elfutils development issues ...

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm dependency checking via python

[Jeff Johnson]

On May 8, 2010, at 9:05 PM, юрка олейников wrote:

 i've tried to da as you told, but have some problem.
 i've added all RPMs from CentOS installation dvd DVD to transaction set (with 
 'i' parameter) and ts.check() succeeds.
 however the same operation on RHEL RPMs fails on ts.check() and a get a huge 
 list of rpm deps...
 

Please supply rpm version involved. I can't track with identifiers like
CentOS and RHEL and DVD.

Be careful with 'i' vs 'u' passed whaten adding elements.

I'd suggest ignoring 'i' and always doing 'u' because rpmlib behavior
is quite different between the two.

Note also that
'i' ==  -i,--install
'u' ==  -U,--upgrade
on the rpm CLI, which may help you narrow down to a specific reproducer,
with deterministic behavior. The rpm-python bindings are quite ancient,
with subtly different behaviors amongst different versions.

There's fewer changes on the rpm CLI.

There's also means to more specific debugging (that I can translate)
if you do rpm.setVerbosity() (I think that's the method, I don't use rpm-python 
daily) twice.

Call rpm.setVerbosity() twice  turns on the equivalent of -vv on the rpm CLI.

RPM is quite heavily insturmented, will supply details on almost every 
operation performed,
but one does need to turn on the diagnostics somehow.

hth

73 de Jeff

 
 2010/5/8 Jeff Johnson n3...@mac.com
 
 On May 8, 2010, at 10:26 AM, Jeff Johnson wrote:
 
 
  On May 8, 2010, at 9:15 AM, юрка олейников wrote:
 
  how do i check in python if rpmA has satisfied dependecies from rpmB if i 
  do not want to use rpmdb?
 
 
  If you don't want to use an rpmdb, then you cannot check added
  dependency assertions against installed packages. An rpmdb
  is in fact where the installed package metadata lives.
 
 
 The specific answer to your question (assuming you don't
 care whatsoever about installed package dependencies) is
 
 1) Get a transaction
 
 2) (counter intuitively) close the rpmdb using the ts.closedb() method.
 
 3) add headers to the transaction as if you were going to install.
 
 4) call the ts.check() method, and retrieve the dependency failures.
 
 What I believe you are going to find is that there are many
 failed installed dependency assertions that are normally satisfied
 by information contained in an rpmdb.
 
 To fix _THAT_ problem, just don't do step 2) let the transaction
 access an rpmdb as needed.
 
 hth
 
 73 de 
 Jeff__
 RPM Package Managerhttp://rpm5.org
 User Communication List rpm-users@rpm5.org
 
 
 
 -- 
 Be Secure, Stay Open - live w/ OpenBSD

Re: Performance Testing Size Limit RPM Package

[Jeff Johnson]

On Dec 22, 2009, at 6:43 AM, Manoj Palhade wrote:

 Dear RPM Team,
 
 I am doing Performance testing of RPM on My Target (QNX as an OS). I am 
 trying to install RPM Package of size 100 MB. This RPM Package contains zip 
 file of 100 MB. Time taken by RPM to install this 100 MB package is 3.10s; 
 which I believe is wrong for 100 MB rpm package. I haven’t seen Zip file got 
 copied after installation. I found following file at destination location 
 “Test100.zip;00fa”. I tried to remove this file but I am not able to 
 remove it.
 
 

All upgrade operations haev a 32 bit transaction id.

Files installed during a transaction are first created with the transaction
id suffixed and then renamed into place.

There's no reason I know of why you should not have been able to remove the 
file.

 I tried same Testing in VMWare Session where QNX is installed; it is working 
 file. I can see zip file of RPM Package getting copied.
  
 Is there any limitation on size of RPM Packages and File inside the RPM 
 Packages.

There are limits on both package and file size imposed by 32 bits used
to store the size. The 32bit integter is unsigned, so 4Gb should be the
limit. Internally, rpm uses a 64bit integer for file/package sizes, so
legacy compatibility when saving file/package sizes is the main
reason for the limitation. E.g. cpio cannot represent files 4Gb ...
 Another problem I am getting as follow.
 
 I am getting following error when I try to un-nstall RPM Packages
 
 error: cannot create %_repackage_dir /opt/sys/var/spool/repackage/88817
  

Some element in that path does not exist. Likely /opt/var/spool/repackages.

You can eith create the directory, or disable re-packaging.

To disable, undefine (or set to 0) this macro

#   If non-zero, all erasures will be automagically repackaged.
%_repackage_all_erasures1

 # rpm -e Test25-1.0-1
 error: cannot create %_repackage_dir /opt/sys/var/spool/repackage/1430
 
 # rpm -e Test20-1.0-1
 error: cannot create %_repackage_dir /opt/sys/var/spool/repackage/1445
 Last folder name (88817) varies each time. How to avoid this error. Is this 
 error is because of Fat16 file system.
 

Yes the last number is the transaction id, and will be different for every 
transaction.

(aside)
The transaction id is a time stamp, seconds since the epoch. You really should
set your clock correctly; other wise transaction id's based on system time will 
be
meaningless.

73 de Jeff

Re: error: unpacking of archive failed on file /: cpio: chown failed - No such file or directory

[Jeff Johnson]

On Dec 16, 2009, at 11:44 AM, Jeff Johnson wrote:
 
 Index: iosm.c
 ===
 RCS file: /v/rpm/cvs/rpm/rpmio/iosm.c,v
 retrieving revision 1.38.2.2
 diff -p -u -w -r1.38.2.2 iosm.c
 --- iosm.c22 May 2009 17:57:28 -  1.38.2.2
 +++ iosm.c16 Dec 2009 16:40:47 -
 @@ -2427,6 +2427,8 @@ if (!(fi-mapflags  IOSM_PAYLOAD_EXTRAC
   rpmlog(RPMLOG_DEBUG,  %8s (%s, %d, %d) %s\n, cur,
   iosm-path, (int)st-st_uid, (int)st-st_gid,
   (rc  0 ? strerror(errno) : ));
 + if (rc  0  (errno == EINVAL || errno == ENOSYS))
 + rc = 0;
   if (rc  0) rc = IOSMERR_CHOWN_FAILED;
   break;
  case IOSM_LCHOWN:
 

That should have been ENOENT not EINVAL, but filtering is dirt simple.

The errno's ENOENT and ENOSYS are highly unlikely to be encountered
in with usual chown(2) calls.

I'd rather _NOT_ carry this filtering in RPM because the filtering
is largely a hack-o-round for a deeper flaw in QNX. Masking the
error will only delay getting a real fix into QNX.

The problem appears limited to FAT16 file systems, which makes
it impossible to write a proper AutoFu build configuration test.

But if you insist, I can add the errno filtering under a
#if defined(__QNX__)

73 de Jeff

Re: error: unpacking of archive failed on file /: cpio: chown failed - No such file or directory

[Jeff Johnson]

On Dec 16, 2009, at 11:54 AM, Jeff Johnson wrote:

 
 
 The errno's ENOENT and ENOSYS are highly unlikely to be encountered
 in with usual chown(2) calls.
 
:
One last correction (after checking man 2 chown)

ENOENT _IS_ commonly returned by chown(2). But
in the very narrow usage case performing chown(2)
system calls while installing files, ENOENT is highly unlikely
to ever be encountered.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: high performance computing, HA and RPM5

[Jeff Johnson]

On Dec 7, 2009, at 9:37 AM, devzero2000 wrote:

 On Mon, Dec 7, 2009 at 2:55 PM, Jeff Johnson n3...@mac.com wrote:
 
 On Dec 7, 2009, at 7:47 AM, devzero2000 wrote:
 
 High performance computing systems are very popular for some time. The
 problems of Hign Avalibility computer systems are common in the same
 way.
 The question is how a package management system as rpm5 can address
 the problems of such environments. I have not found any reference to
 such issues, in general systems package management system, as rpm5.
 Overall this system are starting from the a simple assumption : a
 single system and a single db metadata (dpkg have not a real db
 however). But this assumption is wrong on a system of HPC: in general,
 the applications are installed in the absence of a true package but
 are installed manuallu on a file or network distributed systems: NFS,
 GFS2, Luster for example. The problem, from my point of view, is that
 applications are not installed using a package system like rpm5 but
 installed manually: anyone thinks at this point it is sufficient to
 create a virtual package with only requires for issue like update
 conflict or the like  and it is difficult to prove the opposite: Why
 should I install the same package separately on multiple nodes where
 the package is the same and it is installed on the same place (on a
 distributed or network filesystem). I have the opinion that a
 distributed system requires a rpm5 metadata distributed database and
 the fact that rpm5 includes a relational (or a sort of it in the
 latest incarnation of berkeley db) database system like the model is
 certainly an advantage - this what Iof a advocate  of the relational
 model as Chris Date tell about this issue, last time i have checked.
 On the pragmatic view, specifically, assuming the same (as patch
 version) 100 nodes should be possible to extend / var / lib / rpm /
 Packages  with a shared rpm5 Packages (extending _db_path for example
 ) on which should be able to act as a fragment of Packages (an Union
 of Packages if you like ) and if this it is unavailable, well no
 problem. The preceding are only a personal opinion. There are other
 opinions? I have perhaps missing something ?
 
 
 HPC is usually focussed on scaling, installing identical software
 on many nodes efficiently.
 
 Distributing system images with modest per-node customization tends to be
 simpler than per-node package management. Package management is useful for
 constructing the system images. But PM cannot compete with system images
 for installation scaling to multiple nodes.
 First of all, thanks for your reply. But i disagree on this point : it
 would be like saying that cloning is more  useful than using conga and
 puppet (or kickstart FWIW) and here I disagree.

Well let's decompose the above statements into pieces to identify where
we disagree.

Distributing system images ... tends to be simpler ... and cloning.
From a purely implementation POV, a package manager will always have 
more
overhead than blasting content onto physical media. The overhead 
introduced
by kernels and file systems and libraries and applications is
eliminated if physical images are distributed.

... modest per-node customization ... PM cannot compete and using 
conga/puppet/kickstart
package != configuration management is likely the crux of the 
disagreement.
I don't believe RPM is very good at configuration management, which
is better handled by kickstart or puppet or conga or Augeas. Most
attempts at CM in *.rpm are through scriptlets, with known deficiencies.
I would claim that scriptlets are the single largest cause of upgrade
failures today. Whether single largest or one of the largest is
hardly worth discussing.

So I suspect we differ in package vs. configuration management assumptions.

 
 Doing upgrades of multiple nodes is typically done by creating a new
 system image, and then undertaking a reinstallation of the new system
 image. This isn't as efficient as upgrading a package on a per-node basis
 because new system images will contain redundant already installed
 software. Its very hard to beat a reboot of a new system image located
 on a distributed file system for KISS efficiency.
 
 Tracking what system image is installed back to a specific PM database
 that describes the installed software within the system image could
 be done with a wrapper on rpm to choose 1-of-N rpmdb's to perform
 detailed queries re files in the system image. But a flat file manifest
 of what packages were installed in a system image is likely sufficient
 for most purposes as well.
 But THIS make it useless or worse, the role of a package managemement
 system, let it call call RPM5 or other.
 Are you sure ?

Not sure about anything. What I described is based on an assumption
that physical images produced by a package manager are what could
be distributed. What is THIS and why

Re: high performance computing, HA and RPM5

[Jeff Johnson]

On Dec 7, 2009, at 10:58 AM, Wichmann, Mats D wrote:

 
 
 Fascinating topic.  I think jbj's deliniation
 into package vs. config management is useful.
 Without going into a massive discussion, not that
 I'm necessarily competent to do so anyway, let me
 pick just one item and worry at it, and see if I
 can just irritate everyone :)
 
 
 Packages as containers for immutable files is where package management 
 works.
 The corollary is that mutable files, either through configuration/patch 
 management,
 or for files that aren't contained in packages, doesn't work very well with 
 RPM.
 
 
 On a shared-storage cluster - loosely used term, I mean
 any set up where there are lots of nodes and few
 corresponding humans to watch over things :) - you're
 going to have some local stuff that you want to keep
 small, and everything else gets pushed to the shared
 storage. I think there's a lot of agreement that single-system-
 image type solutions are a good way to go for the
 system part, stuff that always has to be there for a
 client to be useful.  But what is an application?  Is
 it part of system because it's been been put into a
 package and installed through the same package manager used
 to  handle the base system part?  Or is an application
 somehow different (this is, of course, an ancient argument
 with no clear answer)?  
 
 It you put a key application into shared storage because
 either you don't want to dedicate the space locally or
 just as likely want absolute control that the bits are
 the same for everyone, then the app is effctively mutable - 
 the bits might fit the definition of immutable from someone's
 perspective, but not from the perspective of a node, which 
 wouldn't be in control of the changes made (someone updates 
 the package from somewhere else).   So if you're using an image,
 does the image contain knowledge of accessing an app
 on the share, whatever it may be, or not?  Locally
 stored rpm metadata would seem to me to be a bad
 way to refer to bits that are under someone else's
 control. 
 
 
 Is there a place in this for rpm?  Or is this picture
 in the doesn't work very well part and we just stop
 thinking about it?

Perhaps my use of immutable brings in the wrong concepts. Clearly
file content can be altered at will as easily and files can
be copied around as well.

For a package manager to address configuration/patch management
meaningfully, the current concept of a file as a blob of plaintext
whose integrity is verified with a digest would need to be abandoned.

Instead, the sequence of operations performed on the file would
need to be managed, just like any VCS does, with other means
of integrity checking than using a file digest from signed metadata.

The traditional use of an archive to carry the plain text, with
package metadata carrying a digest (or signature, @rpm5.org
is perfectly prepared to validate clearsigned files through
a run-time dependency probe within the assertion checker,
and the run-time probes can be extended to non-package file content through
/etc/rpm/sysinfo/Requirename configurable assertions ... but I digress)
is what is at fault, nothing more.

Note that @rpm5.org already embeds Augeas for configuration management,
and I will add a VCS (likely subversion, libgit.a and libgit2.a just
aren't there yet) to handle configuration management.

Note also bsdiff/bspatch added last week for eventual embedding in RPM.
But there's lots more design work (essentially duplicating a VCS implementation
into RPM) that would be needed for patch management to become part of package
management.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm -qa performance with GPG key(s) installed in the DB

[Jeff Johnson]

On Dec 4, 2009, at 4:50 PM, Joshua Burns wrote:

 Hi Jeff, thanks for the information. 
 
 openssl is significantly faster and should make performance a non-issue. 
 
 I noticed in earlier posts that you were contemplating or had planned on 
 creating a macro for --usecrypto - does a macro for this option exist 
 currently in 5.1.9? 
 
 73 de Josh
 

;-)

Odd, the performance measurements I see on linux/x86 are
beecrypt  nss  openssl
using callgrind.

You mean a macro to select what crypto implementation to use persistently?

It wouldn't be too hard to add a macro. About all that is hard is
figgering a place to expand the macro and set the default
when rpmlib is initialized through bindings. That's likely
why I dinna bother with a macro ...

Here's the current build time crypto assignment (rpmio/rpmpgp.c):

/*...@unchecked@*/
pgpImplVecs_t * pgpImplVecs =
/* explicit selection (order DOES NOT matter here) */
#if defined(USE_CRYPTO_BEECRYPT)  defined(WITH_BEECRYPT)
rpmbcImplVecs;
#elif defined(USE_CRYPTO_GCRYPT)  defined(WITH_GCRYPT)
rpmgcImplVecs;
#elif defined(USE_CRYPTO_NSS)  defined(WITH_NSS)
rpmnssImplVecs;
#elif defined(USE_CRYPTO_OPENSSL)  defined(WITH_SSL)
rpmsslImplVecs;
/* implict selection (order DOES matter) */
#elif defined(WITH_BEECRYPT)
rpmbcImplVecs;
#elif defined(WITH_GCRYPT)
rpmgcImplVecs;
#elif defined(WITH_NSS)
rpmnssImplVecs;
#elif defined(WITH_SSL)
rpmsslImplVecs;
#else
#error INTERNAL ERROR: no suitable Cryptography library available
#endif

OTOH, does anyone but a nerd really care what crypto implementation is used? ;-)

BTW, rpm -qa is slowly getting faster and faster on HEAD ... no crypto needed.

73 de Jeff

Re: rpm -qa performance with GPG key(s) installed in the DB

[Jeff Johnson]

On Dec 4, 2009, at 5:07 PM, Jeff Johnson wrote:

 
 On Dec 4, 2009, at 4:50 PM, Joshua Burns wrote:
 
 Hi Jeff, thanks for the information. 
 
 openssl is significantly faster and should make performance a non-issue. 
 
 I noticed in earlier posts that you were contemplating or had planned on 
 creating a macro for --usecrypto - does a macro for this option exist 
 currently in 5.1.9? 
 
 73 de Josh
 
 

One last note re libgcrypt ...

I *think* (but I've forgot) that DSA w gcrypt is fine,
and I also believe that I fixed RSA w gcrypt about
a year ago.

So caveat emptor ...

73 de Jeff

Re: rpm -qa performance with GPG key(s) installed in the DB

[Jeff Johnson]

On Nov 30, 2009, at 5:17 PM, Joshua Burns wrote:

 Hi, 
 
 Just got rpm 5.1.9 compiled on Solaris, and have noticed that, after GPG keys 
 are installed in the RPM database, rpm -qa performance slows noticeably. 
 Cryptosignatures are a key element of our plans for packaging, so forgoing 
 them would be a pretty big issue. 
 
 Any thoughts? 
 


The terms key element  and forgoing seem to be at odds with each other.

I'm not sure whether you want fast crypto or no crypto.

So here's both answers, as well as the development answer:

FAST CRYPTO


rpm has _THREE_ crypto implementations (if built that way):
BeeCrypt
NSS
OpenSSL

They are selectable with
--usecrypto bc
--usecrypto nss
--usecrypto openssl

RPM is also insturmented with its own benchmarking using --stats.

Build, measure, use fastest.

For extra credit, try using callgrind. BeeCrypt is 10-15% faster.


NO CRYPTO
===

Otherwise, one can disable signature/digest checking persistently on rpm -qa 
using

   Verify digest/signature flags for various rpm modes:
#   0x30300 (_RPMVSF_NODIGESTS)--nohdrchk  if set, don't check 
digest(s)
#   0xc0c00 (_RPMVSF_NOSIGNATURES) --nosignature   if set, don't check 
signature(s)
#   0xf (_RPMVSF_NOPAYLOAD)--nolegacy  if set, check 
header+payload (if possible)
#   0x00f00 (_RPMVSF_NOHEADER) --nohdrchk  if set, don't check 
rpmdb headers
#
#   For example, the value 0xf0c00 (=0xf+0xc0c00) disables legacy
#   digest/signature checking, disables signature checking, but attempts
#   digest checking, also when retrieving headers from the database.
#
#   The checking overhead was ~11ms per header for digests/signatures on
#   a 600 Mhz Dell SMP server circa 1998.
#
#   Each header from the database is checked only when first encountered
#   for each database open.
#
#   Note: the %_vsflags_erase applies to --upgrade/--freshen modes as
#   well as --erase.
#
%__vsflags  0xf
%_vsflags_build %{__vsflags}
%_vsflags_erase %{__vsflags}
%_vsflags_install   %{__vsflags}
%_vsflags_query %{__vsflags}
%_vsflags_rebuilddb %{__vsflags}
%_vsflags_verify%{__vsflags}


NO CRYPTO NEEDED
==

Note that rpm on cvs HEAD no longer bothers with digest/signature checks
on rpmdb Headers. Its kinda pointless to verify memory that is PROT_READ
protected using mmap(2) (as on cvs HEAD).

The signature needs to be verified only when installing.

So far rpm -qa is merely 3x faster:
 $ /usr/bin/time rpm -qa  /dev/null
 0.00user 0.58system 0:00.66elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
 0inputs+0outputs (0major+63810minor)pagefaults 0swaps

And I expect faster yet when headerLoad() is eliminated, and rpm -qa reads 
installed
package names solely from an rpmdb table.

Note that Berkeley DB will do sha1 digest checking if one _REALLY_
needs that level of integrity checking on data elements.

73 de Jeff

Re: error: failed to open /etc/mnttab: No such file or directory

[Jeff Johnson]

On Nov 26, 2009, at 2:36 AM, Manoj Palhade wrote:



#ifndef MOUNTED

#define MOUNTED /etc/mnttab

#endif


I haven’t understood the purpose of this MOUNTED.



MOUNTED is the file path where mounted file systems are listed.


This MOUNTED is not used anywhere in RPM source code.



The size of mounted file systems is needed for RPM to do disk space  
accounting.
MOUNTED (or equivalent) contains the list of file systems to get the  
size from.




On my QNX if I create folder mnttab at /etc/ then no error message”  
appears while installing RPM Package



Thats one way to make to error messsage disappear.

Meanwhile yo need to look at how thos get the
size of a mounted file system on QNX for disk
space accounting to Just Work.

73 de Jeff

Re: regarding qf folder in /lib/rpm/

[Jeff Johnson]

On Nov 20, 2009, at 5:05 AM, Manoj Palhade wrote:

 Dear Jeff,
 
  Thanks for your quick reply.
 
  I want to install, un-install, modify RPM packages and necessary operations 
 related to dependent packages. I don't need advanced features of rpm.
 
  Please correct me if I am wrong with my understanding that I believe 
 rpmrepo.c is the C file use for rpm Repo (repository) which is used by 
 YUM to download updated.
 
  I don’t want to include this feature of maintaining repository, yum and Auto 
 update. So please let me know how to disable this option during complication 
 and/or during build;What changes I need to make in Make file and at which 
 location in Make File for not to create qf folder and it’s content.
 
 Thanks and Regards,
 
 Manoj Palhade
 
There is no disable during build, except by editing tools/Makefile and
removing the target rpmrepo.

Its just as easy to remove /usr/lib/rpm/bin/rpmrepo after building.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: regarding qf folder in /lib/rpm/

[Jeff Johnson]

On Nov 19, 2009, at 5:04 AM, Manoj Palhade wrote:

 Dear Team,
  
 I have installed RPM on QNX. Following is my finding.
 “qf” is the folder get created in /lib/rpm directory. I am not sure about the 
 purpose of this “qf” folder.
 /lib/rpm/qf
  
 Following are the files get created in “qf” folder. 
  
 deb_Packages,deb_Sources,deb_control,deb_md5sums,deb_postinst,deb_postrm,deb_preinst,deb_prerm,wnh_filelists_yaml,wnh_other_yaml,wnh_primary_yaml,yum_filelists_sqlite,yum_filelists_xml,yum_other_sqlite,yum_other_xml,yum_primary_sqlite,yum_primary_xml
  
 What is the purpose of these files? Is it mandatory to have these files 
 available all time? If it is not mandatory how can I make sure that these 
 files will not get created?
  
 
Those are qf template files that contain --queryformat arguments.

Traditionally, the --queryformats have been in /usr/lib/rpm/rpmpopt.

The --queryformat text was getting so large, and is shared
between executables, so separate files were created instead.

I can't answer mandatory nor even required without context.

If you remove the files, then certain
command line options will cease to work. The rpmrepo executable
will not function either.

You can edit scripts/Makefile if you need to make sure
the files will not be created. Just as easy to remove the
files after being created.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: regrading Berkeley DB

[Jeff Johnson]

On Nov 16, 2009, at 6:09 AM, Manoj Palhade wrote:

 Dear RPM Team,
  
 I want o remove Berkeley DB stuff from my rpm binaries. We want to use only 
 SQLite as database. What is the procedure to do it. Do i need to remove 
 Berkeley DB from source code or from rpm build setup i can remove it.
  

There are configure switches to build without Berkeley DB.

Note that sqlite in RPM works, but is not supported largely because
there's no known advantage to using sqlite.

The main flaw with sqlite is the SQL schema, which is not flexible
enough to be useful outside of RPM itself, and so the usual reasons
for using SQL onto a relational database for easy access simply
do not apply.

Note also that the schema used in a rpmdb is about to change
incompatibly, and only conversion tools, not legacy compatibility, will
be supplied. So far there is no work underway on sqlite for the changed schema.

73 de Jeff




smime.p7s
Description: S/MIME cryptographic signature

Re: defalut location of RPM database

[Jeff Johnson]

On Nov 16, 2009, at 6:11 AM, Manoj Palhade wrote:

 Dear RPM Team,
  
 I want to change default location of RPM Database . How can i do it.
  

Change the %_dbpath macro in /usr/lib/rpm/macros.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: regrading Berkeley DB

[Jeff Johnson]

On Nov 16, 2009, at 9:37 AM, Michael Baudisch wrote:

 Hello,
 I have a question regarding support for Sqlite.
 
 Note also that the schema used in a rpmdb is about to change
 incompatibly, and only conversion tools, not legacy compatibility, will
 be supplied. So far there is no work underway on sqlite for the changed
 schema.
 
 So this means that usage of Sqlite with RPM will no longer be supported? If 
 so: from which version will this incompatibiliy start?
 

Sqlite has _NEVER_ been supported if you read through RPM history
carefully. Sure works, and yes there are usage cases, supported
is something else entirely.

And one goal of the change is to rework an rpmdb so that SQL might become 
useful.

Look at the current rpmdb sqlite schema, try to devise some useful
SQL statement from an interactive /usr/bin/sqlite3, and I believe
you will discover how useless the existing sqlite3 schema actually is.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: regrading Berkeley DB

[Jeff Johnson]

On Nov 16, 2009, at 9:46 AM, Jeff Johnson wrote:

 
 
 And one goal of the change is to rework an rpmdb so that SQL might become 
 useful.
 

I should try to be more positive. Note that I haven't yet
started to write up publically
Transactionally Protected Package Management
which is driving the changes. The issue of supported
is largely a technicality, if you want sqlite3, well
I can try to accomodate. But I cannot support code
that I do not use, the reasoning there should be obvious.

Attached is a SQL schema that more-or-less describes
what will happen to a rpmdb. Note that attempting to represent
a non-SQL Berkeley DB database in SQL has some issues. E.g.
there's quite a few SQL dialects around, the one in the attachment
is what is implemented in db-4.8.24 db_sql(1).

Second, I haven't bothered to represent the many-to-many
relations in SQL accurately. In fact, the indices marked TABLE
are also secondary INDEX's with duplicates onto Packages.

Third, the primary record is still a blob, the fields represent
the one-to-one relations to be able to use a INDEX keyword
but are otherwise a fiction. There is no limit on any NUL
terminated string as used in an rpmdb. An open issue is whether the
NUL should be included (or not) in rpmdb string keys.

And the goal is to get Header blob's out of an rpmdb, leaving
behind a path (or URL) to a *.rpm file from which a Header can be
loaded.

If interested in RPM-sqlite3 (or any SQL), what is needed now
is to design the cascade that updates the secondary indices
when a package is added to the primary store.

That can be done (in sqlite3) with some trigger statements, there's
other schema syntax for other SQL implementations.

So the goal of changing is to be able to support SQL databases better,
with a reasonably well known SQL schema, not otherwise.

As for supported, RPM itself needs only a single database implementation,
and my criteria is solely performance, reliability is assumed.

I personally believe that Berkeley DB will run rings around any
relational database, but show me the numbers, and I will instantly
switch to any other database.

hth

73 de Jeff


bdb.sql
Description: Binary data


smime.p7s
Description: S/MIME cryptographic signature

Re: Conflicts vs Obsoletes

[Jeff Johnson]

On Nov 15, 2009, at 7:31 PM, Marc MERLIN wrote:



Yes, I can build dummy packages to try all this, but at the same  
time I want
to make sure I understand expected (not observed) behaviour as well  
as well

as what is recommended in case like this.



You likely want to build packages and test for careful work.

There's actually a Conflicts: bug that goes way way way back to  
rpm-3.0.2

with missing values that was reported about a month ago.

These 2 values in different packages should Conflict: and don't:

Conflicts: suspend-scripts  1.27-2mdv2007.1

and

Provides: suspend-scripts = 1.27

The issue is with the missing value for Release: in the Provides:
and the comparison behavior isn't correct for Conlficts:.

Your examples did not supply {Epoch,Version,Release} at all, and so  
may well

have odd/unexpected behavior.

I can point you at the line that needs to be changed if you want to fix
the issue. The code looks something like this:

   if (sense == 0) {
   sense = rpmvercmp(aV, bV);
   if (sense == 0  aR  *aR  bR  *bR)
   sense = rpmvercmp(aR, bR);
   }

and the ... aR  *aR  bR  *bR is what is b0rken. The fix
is to default missing values to  and actually call rpmvercmp()
with  values.

I can send a patch for rpm-4.0.2 if interested. Just too lazy to look  
atm ...


hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: Conflicts vs Obsoletes

[Jeff Johnson]

On Nov 15, 2009, at 8:09 PM, Eric MSP Veith wrote:




In that case, I put both Conflicts: bash.rpm / Obsoletes: bash.rpm
in newbash.rpm?


Depdends on what you express. Normally, Obsoletes suffices.



The fundamental design flaw with Obsoletes: is lack of persistence.

So if you undertake a old - new renaming, and the new package has
Obsoletes: old
then the old package is erased.

But nothing stops re-installing the old package.

Which is often why a Conflicts: is being added with Obsoletes:, for  
persistence.


I'm likely to make Obsoletes: persistent Real Soon Now. But that
won't change widely deployed RPM behavior.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: Regarding reliability of repackage and rollback options in RPM 5.x series

[Jeff Johnson]

The comments below most definitely do not apply to @rpm5.org code,
only to code produced by @rpm.org.

Repackaging is on by default always, and the --rollback framework
is still very much in place in @rpm5.org code.

As for unreliable, that is an opinion from developers who do not
(and have not) used --rollback as in @rpm5.org.

However, please note that something called
Transactionally Protected Package Management
(TPPM for short) is actively under development and will significantly  
change

how --rollback is implemented @rpm5.org over the next
couple of months.

So if you are looking to use --rollback then you may wish to consider
waiting for the description of TPPM.

73 de Jeff

On Nov 10, 2009, at 1:16 PM, manjunathan.pa...@wipro.com wrote:


Dear RPM5 team
 We are considering use of RPM 5.x series as the packaging tool for  
our system.
During evaluation of rpm we found the following information on   
features such as repackage and rollback are not reliable and  
should not used for generally/production use ?

Is this still the case in RPM 5.x series ?

---The release notes for RPM 4.6.0 contain  
this:--

 Removed features ¶

Support for repackage and rollback have been removed as they're seen  
too unreliable to be generally useful.
Internal network transports have been removed. The remote fetch  
capability is preserved by using an external, configurable helper  
(curl by default). For network related configuration such as proxy  
settings, environment variables like http_proxy are honored by curl,  
rever to helper documentation for futher information.
Solve database support has been removed as it has proven  
impractical, severely limited in functionality and hasn't been used  
by any major distro in years. Plans are to replace the suggestion  
mechanism by a more flexible plugin architecture at some point.
Support for automatic package relocation for IA-32 emulation layer  
on IA-64 has been removed.
  
---


Appreciate your expert advice on this.

Thanks  best regards,

Manjunathan Padua Y

Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any  
attachments to this message are intended for the exclusive use of  
the addressee(s) and may contain proprietary, confidential or  
privileged information. If you are not the intended recipient, you  
should not disseminate, distribute or copy this e-mail. Please  
notify the sender immediately and destroy all copies of this message  
and any attachments.


WARNING: Computer viruses can be transmitted via email. The  
recipient should check this email and any attachments for the  
presence of viruses. The company accepts no liability for any damage  
caused by any virus transmitted by this email.


www.wipro.com

Re: Creating patch rpms

[Jeff Johnson]

On Nov 7, 2009, at 2:01 PM, Derek Pressnall wrote:


I've got a packaging issue I'm trying to resolve.
Our application consists of a several hundred files (mostly input
forms and sql files).  Periodically the developers release patch sets
to add specific features or resolve issues, which consists of about a
about a dozen or so files.  These files are either in addition to, or
replacement of, files that are part of the original application.



OK. Please note that distributing patches, while a perfectly common
and useful methodolgy, is fundamentally at odds with RPM's original
design goals:
Virgin sources, with patches applied, built reproducibly.
So what RPM is/was designed for is a new package with patches applied  
during build,

not a base package, with patches applied on end-user machines.

But I can tell you what to do to distribute patches in a package to be  
applied to

a previously installed base package on end user machines.


Now normally with RPM, you would release a new package version and use
that to update the previous version.  The problem is that customers
each get a unique set of these patches (one customer may get patches
1, 17, and 25, while another will get patch 8, 17, and 31).  Now what
I'd like to do is install the main package rpm, then roll out an rpm
for each patch set (and use RPM's dependency tracking to force
dependent patches to be installed).  But the problem comes in that
many of the files included in the patch sets conflict with existing
files in the parent RPM, which would require the --force flag to be
used.  Furthermore, it would show the parent package as having corrupt
files (since the new files are owned by the patches).



OK, so some subset of patches needs to be applied on a per-customer  
basis.


Likely the easiest packaging is to have all patches in a single package,
updated whenever there are new patches. Likely just as easy is a package
per-patch, and have the customer choose packages/patches to apply.

Which model fits you best:
All patches distributed with configurable sub-set application.
or
	Each patch in a separate package, and a subset of all patches is  
installed,

(i.e. applied).


Is there any to package a set of files and tell RPM that it enhances
another package, so it would a) let you install it despite conflicts
(without --force), and b) keep the parent package  children packages
grouped together (so that when the parent package is upgraded, any
patches included with the new version get removed from the RPM
database)?



Enhances: gets into preferences are other complexities. You need/want
mechanism, not preferences.

The details of file conflicts can be addressed more carefully than  
using --force.


Upgrading the base package underneath a set of patches is likely  
impossible
with RPM. At that point, you are better off just building  
reproducibly, i.e.

patches are applied while building, not on the end-user system.

I can describe some approaches to distributing patches in packages once
I hear what your constraints are.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Creating patch rpms

[Jeff Johnson]

On Nov 7, 2009, at 5:33 PM, R P Herrold wrote:


On Sat, 7 Nov 2009, Jeff Johnson wrote:


OK. Please note that distributing patches, while a perfectly common
and useful methodolgy, is fundamentally at odds with RPM's original
design goals:



Virgin sources, with patches applied, built reproducibly.



From the crypt (rpm-list from years past):

On Wed, 15 Jan 2003, Jeff Johnson wrote:


FWIW, there are 2 big lies in rpm:
 1) reproducible builds.

#1 is true iff one knows how to set up a build environment.





I see you've omitted the other Big Lie in RPM.

The goal of
Transactionally Protected Package Management
should make rpm (even if not yet rpmbuild) an honest
program again ;-)

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Force removing rpmlib(PartialHardlinkSets) from an RPM

[Jeff Johnson]

On Oct 29, 2009, at 2:12 AM, Marc MERLIN wrote:


Very long story short, I'm building rpms with 4.4.2.1 on ubuntu
and need to install on RH 7.1 with rpm 4.0.2.

When you have stopped laughing, here's the problem :)
I need to tell rpm to never put a rpmlib(PartialHardlinkSets) in
the rpms it builds.



No laughter. I've heard older and odder problems with RPM.



When I add a hardlink in the rpm, rpm 4.4.2.1 helpfully adds a
rpmlib(PartialHardlinkSets) that is not relevant to my case and  
prevents
installing the rpm on RH 7.1 when otherwise said rpm and hardlinks  
therein,

work just fine.





Currently, the only way I found to fix this is:
perl -pi -e 's|rpmlib\(PartialHardlinkSets\)|rpmlib 
(CompressedFileNames)|' scripts-1.1.20091028-0.i386.rpm
perl -pi -e 's|8846d64e6f6f85e7d6613958acc0a9a1464ad2ef| 
93d6f9fdaea5908b6693fb3235320c43d70cc15d|'  
scripts-1.1.20091028-0.i386.rpm


command #1: replace property with another one of the same string  
length

command #2: hand fix the SHA1 signature



Now I'm laughing ;-)


I'm obviously hoping that there is a way to tell rpm 4.4.2.1 to just
not bother with rpmlib(PartialHardlinkSets)?
(yes, the rpm after I hack the property out, installs and works just  
fine in

my use case)



The issue is a bit deeper than just changing configuration since
there's a whole installer behavior of RPM itself that is being tracked.

The reliable fix is to to just add a patch to rpm-4.4.2.1 to not
add the tracking dependency.

You can try (its very much not the right thing to do, but you are
already pretty desperate for a fix if patching digests) adding
Provides: rpmlib(PartialHardlinkSets)
in the same package to stub-out the Requires:. I won't bore you
with the details of why its not the right thing to do. But if
you can mask the Requires: by adding a Provides: you will
likely be able to install.

Another approach to masking the Requires: would be to patch
the Provides: into rpm-4.0.2.

But if you can't rebuild either version of RPM, well, its rather
hard to retrofit forward compatibility without changing code.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Clarification of the way triggers do work

[Jeff Johnson]

On Oct 14, 2009, at 5:39 PM, Tobias Gerschner wrote:


Hi,

I have a couple of packages were the use of triggers does not produce
the expected result. Probably my understanding of the trigger logic is
wrong. Sorry I am still using rpm 4.4.5 in production and not rpm5 as
I want to .

I have a package foo, which contains

%triggerin -- bar
ln -sf /path/file/from/foo /path/directory/from/bar

My expectations are:


Here's the sequence diagram for when and where triggers fire:
  all-%pretrans
  ...
  any-%triggerprein (%triggerprein from other packages set off by new  
install)

  new-%triggerprein
  new-%pre  for new version of package being installed
  ...   (all new files are installed)
  new-%post for new version of package being installed

  any-%triggerin (%triggerin from other packages set off by new  
install)

  new-%triggerin
  old-%triggerun
  any-%triggerun (%triggerun from other packages set off by old  
uninstall)


  old-%preunfor old version of package being removed
  ...   (all old files are removed)
  old-%postun   for old version of package being removed

  old-%triggerpostun
  any-%triggerpostun (%triggerpostun from other packages set off by  
old un

install)
  ...
  all-%posttrans



1) Package foo is already installed. When package bar is being
installed the %triggerin scriptlet will be executed after the %post
scriptlet from package bar


When bar (the new package) is being installed, the already installed  
foo trigger

should fire at
any-triggerin
which is just after new-post (i.e. the %post from the new package  
bar ...).



2) Package bar is already installed. When package foo is being
installed the %triggerin scriptlet will be executed after the %post
scriptlet from package foo.



This appears to be just swapping foo - bar in what I just described.
If that isn't what you meant, ask again. Its crazy hard to try and
talk precisely about trigger behavior ;-)


Is this correct or is this wrong ? The triggers only need to be
executed once to set symlinks on upgrade no action needs to be taken.



Triggers will fire whenever the trigger condition is met. That may or  
may not

be only once on upgrade. Just make sure the trigger body is prepared
to create symlinks iff the symlink does not already exist. Running a
trigger isn't particularly expensive, and the trigger condition should
be tuned (by adding version comparison) to limit the trigger condition.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Clarification of the way triggers do work

[Jeff Johnson]

On Oct 14, 2009, at 10:12 PM, Tobias Gerschner wrote:

Triggers will fire whenever the trigger condition is met. That may  
or may

not
be only once on upgrade. Just make sure the trigger body is prepared
to create symlinks iff the symlink does not already exist. Running a
trigger isn't particularly expensive, and the trigger condition  
should
be tuned (by adding version comparison) to limit the trigger  
condition.


hth

73 de Jeff



Did you mean to say, that running triggers is an expensive operation ?
Why else would you tune to limit it's occurrence ?



Mostly just muttering about rpm devel on HEAD out loud ...

Triggers are potentially very costly because all added packages
get crossed against all installed packages 4 times while installing.

But the overhead is basically a database lookup, which almost always
finds nothing to do.

About a year ago, I generalized triggers to fire not just on package  
Name,

but also Provides, and directories and files and using glob patterns.

That _STILL_ is a tolerable overhead, looking up a key in a database
just isn't that expensive, and there just aren't that many triggers
in use.

The overhead of the new trigger implementation has been checked with  
callgrind.

The effects are visible, but tolerable.

However, watching the spewage from triggers hitting 4 indices 4 times  
for all files
directories and trigger tokens in packages is starting to drive me  
batty.


The short term fix will be to add a Bloom filter on the indices to  
avoid hitting an rpmdb
unnecessarily. Basically that avoids the debugging spewage, and saves  
my tired old eyes.


But the eventual fix will be to redesign the access to optimize away  
the overhead
entirely. But that will require larger changes, including a rpmdb  
schema change,
as well as consolidating the 4 rpmdb hits into a single pass operation  
to

pull out just those triggers that need to be run ...

None of the above has anything to do with the rpm that you are using,
nor is there significant performance degradation in rpm-5.1.7 and later
with the new trigger implementation.

The ability to trigger on not only package Name: but also
Provides: and directories and files and with patterns is a useful  
generalization, has
recently appeared on the OpenSUSE RPM summit roadmap (but is already  
implemented @rpm5.org).


But what I was really trying to point out to you is that triggers fire
when the trigger condition is met. If you write the trigger condition  
too loosely,
the triggers may fire where you are not expecting. You did say you  
expected triggers

to fire only once during upgrade ...

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Errors - API for rpm

[Jeff Johnson]

On Sep 18, 2009, at 5:36 AM, Manoj Palhade wrote:



Hi,

I am trying to implement sample program mentioned in “Maximum RPM”. My
understanding is that this content is not update with current version
of RPM source code.




The examples  in Maximum RPM won't compile with any version
of RPM released in the last 5-6 years. The examples were written in  
1997,

lots has changed since.


1.  In sample program I am getting following error.

undefined reference to `headerDump'
undefined reference to `headerFreeIterator'
undefined reference to `headerGetEntry'
undefined reference to `headerInitIterator'
undefined reference to `headerNextIterator'
undefined reference to `rpmFreeSignature'
undefined reference to `rpmReadPackageInfo'




If you post the source I can ttry to rewrite o the current @rpm5.org  
API.
2.  Also As mentioned in “Maximum RPM”; I included rpmlib.h but  
not

able to find header.h.




Yes. Most of what used to be in header.h has moved to rpmdb/rpmtag.h.

But rpmReadPackageFile() is likely what you are looking for.

73 de Jeff

Re: Attempting to compile rpm5 for RH Linux EL5

[Jeff Johnson]

On Sep 18, 2009, at 12:11 PM, Saravanan Shanmugham (sarvi) wrote:

Separating the symbol files and into their own package is definitely  
in

the plans.



Well, packaging detached debugging symbols (and source code so
that line numbers can be displayed) was done in RPM like 5 years
ago and is already widely deployed.

You really should look at what is implemented.

Note that from a RPM packaging POV, the existing -debuginfo scheme
is all very feeble and hacky and horendously fragile and obscure.

So don't take my comments as criticism, I'd *LOVE* to see a better
packaging framework for detached debugging symbols in RPM.

The question is the turnaround time between someone posting a  
traceback
on the web and a backend system translating the traceback with  
symbols.




If end-to-end turnaround efficiency is your goal, you need to
look carefully at how traceback information can be accurately
associated with the components involved. Extracting executables/ 
libraries

from *.rpm is just a very teensy part of a much larger problem.


One option is to have a filesystem where all the files have been
extracted and ready for translation. But can cost additional diskspace
which can add up in the long run.



That's basically what is in a -debuginfo package, files under
/usr/src/debug/* include source and detached debugging symbols.


If the RPMs are archived, the system would need to identify the RPMS
involved in traceback, download, extract them, do the translation,  
then

discard the extracted stuff.
This could takes some time to do, specially if you have to download  
the
whole RPM, and if you had to uncompress/untar each RPM in its  
entirity.


From RPM, tar.gz or CPIO, this can be expenseive. As I understand it  
you

need to unzip and untar the file, piping it to a filter to get the
file(s) you want.

XAR format maintains an index, that allows you to go to and extract a
specific file, and the individual files are compressed, as I  
understand

it, meaning the ones of interest can be extracted, which can be more
efficient.



Again, speed or overhead of archive extraction is hardly an issue when  
everything
necessary for debugging is _ALREADY_ extracted into a -debuginfo  
package.



At this point, I am just trying to understand/measure that cost
impact/tradeoff, and was looking to try these features.



Look closely at tools/debugedit.c and scripts/find-debuginfo.sh. Those
two files are what is used to automate producing -debuginfo packages.

I'd suggest grabbing some -debuginfo *.rpm files and examining. This
command makes it quite easy to see what is in a -debuginfo package

rpm -qp --yaml foo-debuginfo*.rpm

hth

73 de Jeff

Sarvi


-Original Message-
From: rpm-users-ow...@rpm5.org [mailto:rpm-users-ow...@rpm5.org] On
Behalf Of Jeff Johnson
Sent: Friday, September 18, 2009 5:34 AM
To: rpm-users@rpm5.org
Subject: Re: Attempting to compile rpm5 for RH Linux EL5


On Sep 18, 2009, at 2:04 AM, Saravanan Shanmugham (sarvi) wrote:


Thanks. I was able to get it compiled and installed.

I am proposing the use of RPM5 for package management internally. One
point of interest to me in RPM5 is the use of XAR format, and more
specifically the option of being able to extract specific files from
within the XAR archive without having to untar/extracting everything.
Atleast that's what the xar wiki page claims.



You can extract specific files from any archive format, including
cpio/tar/xar, rather easily.

We need a way to be able to extract specific executables and  
libraries



from a specific version of the build, from its archived RPMs, on
demand, to help decode a crash or traceback.



So you would seem to want debugging symbols. What is more commonly  
done

is to split the symbols from the executable/library and put into a
separate -debuginfo package.

GDB then loads the detached symbols.


So far, I haven't been able to figure out the RPM5 option or XAR
option that allows me to do extract a specific file or list of files.



For cpio payloads it starts with
rpm2cpio foo*.rpm | cpio -itv
to display the files in a payload. Then one adds logic (described in
man cpio) to select which files you want.


Is there such an option. If not how easy do you think it is to
implement such an option, if we can help.



Each archive format has a different selection process. But rpm2cpio is
used to seek to the start of the payload.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org


__
RPM Package Managerhttp

Re: Attempting to compile rpm5 for RH Linux EL5

[Jeff Johnson]

On Sep 17, 2009, at 8:12 PM, Saravanan Shanmugham (sarvi) wrote:



I got popt 1.15 from
http://www.sfr-fresh.com/linux/misc/popt-1.15.tar.gz
Compiled and installed it.
Now I get the following error during compilation

o -lm /usr/lib/libmagic.so -lbeecrypt -lz /usr/lib/libpopt.so - 
lpthread

-lrt -Wl,-rpath -Wl,/usr/local/lib
../lib/.libs/librpm.so: undefined reference to `poptReadConfigFiles'
../lib/.libs/librpm.so: undefined reference to `poptReadFile'
collect2: ld returned 1 exit status
make[3]: *** [rpm2cpio] Error 1
make[3]: Leaving directory `/operator/rpm-5.1.9/tools'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/operator/rpm-5.1.9/tools'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/operator/rpm-5.1.9'
make: *** [all] Error 2



The official popt-1.15.tar.gz is at
http://rpm5.org/files/popt/popt-1.15.tar.gz

I just download'd, unpackaged, ran configure, and looked for the  
symbols:

[...@fedora10 popt-1.15]$ nm .libs/libpopt.so
...
4fa0 T poptReadConfigFile
59c0 T poptReadConfigFiles
5660 T poptReadDefaultConfig
4dd0 T poptReadFile
so the symbols are there.



Is there a list of components/versions and where to get them, I need  
so

that I can install them and make sure compile install goes fine? I
notice that all the components like popt, xar all have copies in the
RPM5 CVS. Is this where we should pulling stuff from?



See the INSTALL file in the rpm tarball.

But yes, there's a slew of possible options for ./configure, many
more than most programs have.

Here's the options I use daily on Fedorable 11. For RPM development, I  
configure *everything*,

you likely want to pare this set down to something smaller and saner:

%configure \
--verbose \
--prefix=/usr \
--with-db=internal \
--with-db-tools-integrated \
--with-zlib=external \
--with-bzip2=external \
--with-xz=internal \
--with-file=internal \
--with-lua=internal \
--with-tcl=external \
--with-sqlite=external \
--with-syck=internal \
--with-readline=external \
--with-augeas=external \
--with-beecrypt=external \
--with-openssl=external \
--with-nss=external \
--with-gcrypt=external \
--with-expat=external \
--with-libtasn1=external \
--with-pakchois=external \
--with-gnutls=external \
--with-neon=external \
--with-pcre=internal \
--with-uuid=system \
--with-xar=internal \
--with-popt=external \
--with-keyutils=external \
--with-pthreads \
--with-libelf \
--with-selinux \
--with-sepol \
--with-semanage \
--with-squirrel=/usr/lib:/usr/include/squirrel \
--with-ficl=/usr/lib:/usr/include/ficl \
--with-js=internal \
--with-python \
--with-pythonembed=/usr/lib:/usr/include/python2.6 \
--with-perl \
--with-ruby=/usr/lib/ruby/1.8/i386-linux \
--with-build-extlibdep \
--with-build-maxextlibdep \
--with-valgrind \
--enable-build-pic \
--enable-build-versionscript \
--enable-build-warnings \
--enable-build-debug \
--enable-maintainer-mode



Also I am trying to compile RPM5 with XAR support. Additionally I want
the RPM tooling to be compiled and installed  at
/my/own-nfs/tooling/dir. I want this central install to be usable by  
all

the machines on the network, to mostly build RPMS in private workspace
and install those RPMs into private workspace. Installation of RPMs
should not happen to root (/). Is this doable with RPM5?



Passing --prefix=/my/own-nfs/tooling/dir should change the path
where rpm is installed.

Note that the rpmdb (normally in /var/lib/rpm) should be on local
disk, not on NFS.

Check this macro (normally in /usr/lib/rpm/macros)

#   The location of the rpm database file(s).
%_dbpath%{_var}/lib/rpm

You likely want to check all paths in the macros file.

But yes, all the rest is doable.

73 de Jeff


Sarvi


-Original Message-
From: rpm-users-ow...@rpm5.org [mailto:rpm-users-ow...@rpm5.org] On
Behalf Of Jeff Johnson
Sent: Thursday, September 17, 2009 11:13 AM
To: rpm-users@rpm5.org
Subject: Re: Attempting to compile rpm5 for RH Linux EL5


On Sep 17, 2009, at 1:50 PM, Saravanan Shanmugham (sarvi) wrote:


I am trying to compile and install RPM 5.1.9.tar.gz And am running
into the following errors. Is there any configure option I should be
trying.
Has this been fixed in a later release?

gcc -DHAVE_CONFIG_H -I. -I..  -I. -I.. -I../build -I../lib -I../lib
-I../rpmdb -I../rpmio -I../misc -I../db3 -I../db3 -I../pcre -I../pcre
-DRPM_OS_LINUX=020609  -g -O2 -D_GNU_SOURCE -D_REENTRANT -MT
rpmdigest.o -MD -MP -MF .deps/rpmdigest.Tpo -c -o rpmdigest.o
rpmdigest.c
rpmdigest.c:265: error: optionsTable[1].argInfo')
rpmdigest.c:267: error

Re: Attempting to compile rpm5 for RH Linux EL5

[Jeff Johnson]

RPM carries an internal version of xar largely
because most Linux distros don't carry or
maintain xar reliably.

Over time, rpm5.org has likely started to diverge from
upstream XAR in some complicated ways, particularly
with how XAR is configured. There's no intent to
diverge, just noone has bothered to verify how
closely internal - external xar agree, most of
the changes are needed to use automake (which
upstream XAR doesn't use).

Either getand install XAR externally from

XAR [5] optional  1.5.2   1.5.2   http://code.google.com/p/xar/

or try the internal version (which should be in the
released rpm tarball iirc).

hth

73 de Jeff
On Sep 17, 2009, at 8:44 PM, Saravanan Shanmugham (sarvi) wrote:


I pulled xar 1.5.2 from the RPM5, applied the xar 1.5.2.patch.
And tried compiling it.
/usr/bin/ranlib lib/libxar.a
sed -e s/@LIBXAR_SNAME@/libxar.a/ -e s/@LIBXAR_LNAME@/libxar.a.1/ -e
s/@LIBXAR_ANAME@/libxar.a/  lib/libxar.la.in  lib/libxar.la
gcc -Wall -g  -Iinclude -Iinclude -D_GNU_SOURCE -I/usr/include/libxml2
-c src/xar.c -o src/xar.o
gcc -Wall -g  -o src/xar src/xar.o -Wl,-rpath,/operator/xar-1.5.2/lib
-Llib  lib/libxar.a -lbz2 -lz -lcrypto -lacl  -L/usr/lib -lxml2 -lz
-lpthread -lm
lib/libxar.a(archive.static.o)(.text+0x2c5d): In function
`xar_unserialize':
lib/archive.c:1346: undefined reference to `xmlDictCleanup'
lib/libxar.a(archive.static.o)(.text+0x2cd4):lib/archive.c:1363:
undefined reference to `xmlDictCleanup'
lib/libxar.a(archive.static.o)(.text+0x2e45):lib/archive.c:1403:
undefined reference to `xmlDictCleanup'
lib/libxar.a(archive.static.o)(.text+0x2e6c):lib/archive.c:1411:
undefined reference to `xmlDictCleanup'
lib/libxar.a(archive.static.o)(.text+0x2e8d):lib/archive.c:1417:
undefined reference to `xmlDictCleanup'
collect2: ld returned 1 exit status
make: *** [src/xar] Error 1
rm src/xar.o
[opera...@nova-stuttgart xar-1.5.2]#

Sarvi

-Original Message-
From: rpm-users-ow...@rpm5.org [mailto:rpm-users-ow...@rpm5.org] On
Behalf Of Saravanan Shanmugham (sarvi)
Sent: Thursday, September 17, 2009 5:12 PM
To: rpm-users@rpm5.org
Subject: RE: Attempting to compile rpm5 for RH Linux EL5


I got popt 1.15 from
http://www.sfr-fresh.com/linux/misc/popt-1.15.tar.gz
Compiled and installed it.
Now I get the following error during compilation

o -lm /usr/lib/libmagic.so -lbeecrypt -lz /usr/lib/libpopt.so - 
lpthread

-lrt -Wl,-rpath -Wl,/usr/local/lib
../lib/.libs/librpm.so: undefined reference to `poptReadConfigFiles'
../lib/.libs/librpm.so: undefined reference to `poptReadFile'
collect2: ld returned 1 exit status
make[3]: *** [rpm2cpio] Error 1
make[3]: Leaving directory `/operator/rpm-5.1.9/tools'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/operator/rpm-5.1.9/tools'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/operator/rpm-5.1.9'
make: *** [all] Error 2


Is there a list of components/versions and where to get them, I need  
so

that I can install them and make sure compile install goes fine? I
notice that all the components like popt, xar all have copies in the
RPM5 CVS. Is this where we should pulling stuff from?

Also I am trying to compile RPM5 with XAR support. Additionally I want
the RPM tooling to be compiled and installed  at
/my/own-nfs/tooling/dir. I want this central install to be usable by  
all

the machines on the network, to mostly build RPMS in private workspace
and install those RPMs into private workspace. Installation of RPMs
should not happen to root (/). Is this doable with RPM5?

Sarvi


-Original Message-
From: rpm-users-ow...@rpm5.org [mailto:rpm-users-ow...@rpm5.org] On
Behalf Of Jeff Johnson
Sent: Thursday, September 17, 2009 11:13 AM
To: rpm-users@rpm5.org
Subject: Re: Attempting to compile rpm5 for RH Linux EL5


On Sep 17, 2009, at 1:50 PM, Saravanan Shanmugham (sarvi) wrote:


I am trying to compile and install RPM 5.1.9.tar.gz And am running
into the following errors. Is there any configure option I should be
trying.
Has this been fixed in a later release?

gcc -DHAVE_CONFIG_H -I. -I..  -I. -I.. -I../build -I../lib -I../lib
-I../rpmdb -I../rpmio -I../misc -I../db3 -I../db3 -I../pcre -I../pcre
-DRPM_OS_LINUX=020609  -g -O2 -D_GNU_SOURCE -D_REENTRANT -MT
rpmdigest.o -MD -MP -MF .deps/rpmdigest.Tpo -c -o rpmdigest.o
rpmdigest.c
rpmdigest.c:265: error: optionsTable[1].argInfo')
rpmdigest.c:267: error: initializer element is not constant
rpmdigest.c:267: error: (near initialization for POPT_ARG_ARGV'
undeclared here (not in a
function)BRrpmdigest.c:265: error: initializer element is not
constantBRrpmdigest.c:265: error: (near initialization for
optionsTable[1]')BRrpmdigest.c:270: error: initializer element is
not constantBR/FONT/DIV DIVFONT face=Arial
size=2/FONTnbsp;/DIV DIVFONT face=Arial
size=2/FONTnbsp;/DIV DIVSPAN class=156564317-17092009FONT
face=Arial size=2Sarvi/FONT/SPAN/DIV/BODY/HTML



You need popt-1.15 (or you need to patch out POPT_ARG_ARGV, which  
should

be in slightly older versions of RPM

Re: free() pointer error when using bindings to read specfile

[Jeff Johnson]

On Sep 13, 2009, at 11:08 AM, Eric MSP Veith wrote:

I'll see if I can reproduce the problem (I don't change much
with python/perl bindings).

The mechanism used by the python/perl bindings works
on cvs HEAD for me:

$ rpm -q --specsrpm --yaml libselinux.spec

and YAML is quite easy to load into either perl or python.

There's --xml if you don't like --yaml, and there is
--specsrpm instead of --specfile if you want tags from
the SRPM rather than the binary RPM's.



The perl version looks similar, but I'm using RPM::Spec-new 
($file) instead.


I've got no clue on where to look further here. rpmbuild e.g. works  
ok. Am I

misung the bindings?



The fundamental flaw is that the python (and perl) bindings lack
sufficient methods to be generally useful. E.g. there are no methods
to unpack and run a %prep scriptlet.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: free() pointer error when using bindings to read specfile

[Jeff Johnson]

On Sep 13, 2009, at 11:08 AM, Eric MSP Veith wrote:


Hello,

I'm trying to read a spec file using RPM's bindings. I've tried both  
rpm5-perl

and rpm5-python and both fail with the following error:

*** glibc detected *** python: free(): invalid next size (fast):  
0x08ab9040

***



This error is going to be RPM version specific. All memory is no
refcounted with mutexes, and malloc overhead is avoided by resusing
items from memory pools. Which is why I'm not able top reproduce on  
cvs HEAD.



This is what the Python script gives me, the error with Perl reads  
the same.

My code looks as follows:


#!/usr/bin/env python

import rpm
import os
import sys

if len(sys.argv) != 2:
   print Usage: %s SPECFILE %(sys.argv[0])
   exit(1)

ts_h = rpm.TransactionSet()
spec_h = ts_h.parseSpec(sys.argv[1])
print spec_h.check()




Meanwhile, this script exercises all the available python methods
---
#!/usr/bin/env python

import rpm
import os
import sys

if len(sys.argv) != 2:
   print Usage: %s SPECFILE %(sys.argv[0])
   exit(1)

ts_h = rpm.TransactionSet()
spec = ts_h.parseSpec(sys.argv[1])

print - sources:, spec.sources()
print - prep:, spec.prep()
print - build:, spec.build()
print - install:, spec.install()
print - clean:, spec.clean()
print - buildRoot:, spec.buildRoot()

print - check:, spec.check()


When run the above script a t.py I see:

$ python t.py /usr/src/rpm/SPECS/popt.spec
...
- clean: /bin/rm -rf '/var/tmp/popt-root'


- buildRoot: /var/tmp/popt-root
- check:
Traceback (most recent call last):
  File t.py, line 21, in module
print - check:, spec.check()
SystemError: error return without exception set

The error is because I ran the script on a popt.spec file
that does not have a %check section. Adding a try to catch
the exception would work. Arguably this should be in
spec-py.c, but the Spec methods are primitive and there's
a vanishingly small usage case so its hardly worth the effort imho.
But a copuple lines of patching in python/spec-py.c could
do whatever is desired. Private exception for better message?
a PyNONE return? What ... ?

static PyObject *
spec_get_check(specObject * s)
/*...@*/
{
Spec spec = specFromSpec(s);
return (spec != NULL  spec-check != NULL)
? Py_BuildValue(s, rpmiobStr(spec-check)) : NULL;
}


The perl version looks similar, but I'm using RPM::Spec-new 
($file) instead.




The perl version has unit tests that are passing afaik. I can look  
deeper

if you have a specific failure reproducer.

I still suggest using YAML (or XML if that's your poison). Loading  
YAML is
dirt simple and you end up with native objects (rather than custom  
crufty

getter/setter methods) ready for use.


hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Two small bugs in popt-1.15

[Jeff Johnson]

On Aug 27, 2009, at 6:21 PM, Charles Wilson wrote:


Typo in libpopt.vers, and extra ';' in popthelp.c.  The second error
only causes problems if you explicitly #undef POPT_WCHAR_HACK at the  
top

of the file -- which I had to do because my platform doesn't support
mbsrtowcs.



Fixes checked in.

BTW, you are unlikely to find joy with POPT_WCHAR_HACK undefined.
The issue has been there -- unfinished -- since popt-1.11.

Several patches were sent to fix --help alignment problems
were sent from someone (from Sun iirc) on the GNOME project
were sent. Unfortunately, the patches broken non-UTF8 alignment.

popt has flip-flopped several times on aligned --help displays
with mixed (and iconv() retrofitted) PO file encodings.

I have a reproducer for the non-UTF8 issues, but not for the
UTF8 issues requested by GNOME. I do not even know what PO
file illustrated the retro-fit fixing in popt, nor can
I read JA/KO.

So you can expect --help alignment issues with (iirc) UTF-8 encoded
PO files until I actually get a reproducer for whatever issue is
involved.

If you know anyone interested in solving the problem, well, so am I.

Meanwhile --help alignment hardly matters to popt option processing.

Thanks for the patches.

73 de Jeff


diff -urN old/popt-1.15/libpopt.vers new/popt-1.15/libpopt.vers
--- old/popt-1.15/libpopt.vers  2009-04-12 03:08:59.0 -0400
+++ new/popt-1.15/libpopt.vers  2009-08-27 18:18:27.06950 -0400
@@ -11,7 +11,7 @@
poptBadOption;
poptConfigFileToString;
poptDupArgv;
-poptFinit;
+poptFini;
poptFreeContext;
poptGetArg;
poptGetArgs;
diff -urN old/popt-1.15/popthelp.c new/popt-1.15/popthelp.c
--- old/popt-1.15/popthelp.c2009-04-12 14:14:38.0 -0400
+++ new/popt-1.15/popthelp.c2009-08-27 18:18:39.35350 -0400
@@ -144,7 +144,7 @@
n = mbsrtowcs (NULL, s, n, t);
#else
n = 0;
-for (; *s; s = POPT_next_char(s));
+for (; *s; s = POPT_next_char(s))
n++;
#endif


__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org


__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Fwd: error: cannot open Packages database in /var/lib/rpm

[Jeff Johnson]

Begin forwarded message:


From: Jeff Johnson n3...@mac.com
Date: May 28, 2009 7:17:46 AM EDT
To: spam.spam.spam.s...@free.fr
Subject: Re: error: cannot open Packages database in /var/lib/rpm


On May 28, 2009, at 4:35 AM, spam.spam.spam.s...@free.fr wrote:



Ok... So I need libpcre or I can use --nodeps (read the dependances  
is very

important! Maybe I should install libpcre).
I am on Slackware 12.2 and pcre-7.7 in installed. But I said to the  
configure

script : --without-pcre.
Hum, It's strange... I say --without-pcre and it try to use it  
during the

install of a RPM package.

Now watch what rpm say when I try with --nodeps :

#rpm --nodeps --macros /lib/rpm/macros -i  
8Kingdoms-1.1.0-6.fc9.i386.rpm
warning: 8Kingdoms-1.1.0-6.fc9.i386.rpm: Header V3 DSA signature:  
NOKEY, key ID

4ebfc273
error: unpacking of archive failed: cpio: Bad magic

I am sure I need to install CPIO now...
Oh my god, so much dependencies for RPM, my chroot system will be  
big...




In order to use *RE's within *.rpm tag content, a *RE dialect needed  
to be chosen.


Having both POSIX and PCRE within *.rpm packages would add an  
intolerable
Have it your own way! set of implementations that cannot be  
supported.


Consider what happens if a *.rpm included tags with PCRE patterns, and
the user has decided to build using --without-pcre. That package  
cannot

be installed.

The *RE patterns are already in use to express platform affinities,
install policies like Prefer i586 over i486 packages on i686  
platforms.


So PCRE was chosen as a *RE dialect for several reasons, including  
that

PCRE is a superset of POSIX *RE patterns, and PCRE is distributed
with RPM sources. The INSTALL document very clearly says:

...
PCREmandatory 7.0 7.9 http://www.pcre.org/
..

mandatory means exactly that: MANDATORY.

PCRE internal, (and Berkeley DB internal, the recommended means of
using Berkeley DB for many years), add no dependencies to RPM  
whatsoever.


You (or Slackware) are choosing the dependency complexity.


Well, If I understand, I need these two :
http://www.pcre.org/
http://www.gnu.org/software/cpio/



There is no need for cpio.

73 de Jeff


__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: error: cannot open Packages database in /var/lib/rpm

[Jeff Johnson]

I have written in the previous mail (see up):
'error: unpacking of archive failed: cpio: Bad magic'
This is an error I get. And that's the reason why I thought I need  
CPIO. But

CPIO isn't mandatory, so... Why this error?


(aside)
MANDATORY (of course) depends on a context, and there are
different contexts of reference.

wrto cpio, the usage case for cpio in RPM is payload extraction using  
rpm2cpio

(or rpm2cpio.sh) as in
rpm2cpio foo*.rpm | cpio -dim
RPM itself doesn't need that functionality, but your Slackware chroot  
might

benefit from being able to extract files from *.rpm packages without
using RPM.

Dunno what the failure is.

The magic check is attempted when the payload is first opened.

A failure is typically from truncated *.rpm input or (likelier
in your case) rpm being mis-built. Check what compressions
you have built into RPM.

Doing
rpm2cpio foo*.rpm | cpio -dim
is one easy test of the payload extraction.

There's also -vv --fsmdebug options that will display
diagnostic information regrading payload unpacking.
Likely --fsmdebug won't help with bad magic, all you
will see is that, indeed, the magic is bad.

There are some small toy *.rpm's that are in RPM cvs (
see the tests/ subdirectory) that can be used for simple
installs.

The tests/devtool-sanity-1.0-1.src.rpm package in particular
can be built and an install attempted to diagnose your problem.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Fwd: error: cannot open Packages database in /var/lib/rpm

[Jeff Johnson]

Begin forwarded message:


From: Jeff Johnson n3...@mac.com
Date: May 28, 2009 11:55:11 AM EDT
To: spam.spam.spam.s...@free.fr
Subject: Re: error: cannot open Packages database in /var/lib/rpm


On May 28, 2009, at 11:46 AM, spam.spam.spam.s...@free.fr wrote:




Ok so, you think I should check what compressions I have built into  
RPM. Do you
speak about the 8Kingdoms-1.1.0-6.fc9.i386.rpm RPM package or the  
rpm5 software?


The configure options I have chosen :

./configure --without-path-versioned --without-db-tools-integrated
--without-db-rpc --without-db-largefile --without-python --without- 
perl
--without-selinux --without-libelf --without-pthread --without- 
libintl-prefix
--without-libiconv-prefix --without-gnu-ld --without-apidocs -- 
without-pic
--disable-largefile --disable-rpath --disable-nls --disable-broken- 
chown
--disable-build-lafiles --disable-build-maxextlibdep --disable- 
build-extlibdep
--disable-build-intlibdep --disable-build-versionscript --disable- 
build-static

--disable-build-gcov --disable-build-pie --disable-build-pic
--disable-libtool-lock --disable-dependency-tracking --disable- 
maintainer-mode
--disable-build-warnings --disable-build-debug --with-popt -- 
without-zlib
--without-bzip2 --without-xz --with-beecrypt --without-nss -- 
without-openssl
--without-neon --without-file --with-db --without-sqlite --without- 
lua
--without-pcre -without-uuid --without-syck --without-xar --without- 
dmalloc

--without-efence --without-keyutils --prefix=/

How to know the compressions that have been built into RPM?


Well from your configure arguments I can tell:
--without-zlib  no gzip compression
--without-bzip2 no bzip2 compression
--without-xzno XZ compression
and so you've built a version of rpm that is incapable
of installing any packages (all *.rpm packages have some
form of compression)

At a minimum, I suggest adding
--with-zlib
if you want to actually build and install a *.rpm package.

Note also that disabling *everything* is a much harder
introduction to RPM than any other conceivable path.

73 de Jeff


__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: error: cannot open Packages database in /var/lib/rpm

[Jeff Johnson]

On May 28, 2009, at 1:51 PM, spam.spam.spam.s...@free.fr wrote:



# rpm --nodeps -i 8Kingdoms-1.1.0-6.fc9.i386.rpm
warning: 8Kingdoms-1.1.0-6.fc9.i386.rpm: Header V3 DSA signature:  
NOKEY, key ID

4ebfc273
error: failed to open /etc/mtab: No such file or directory
error: %post(8Kingdoms-1.1.0-6.fc9.i386) scriptlet failed, exit  
status 255




All that rpm knows about the script is the exit code: 255.

This will display the scripts:
rpm -qp --scripts 8Kingdoms-1.1.0-6.fc9.i386.rpm

And adding --nopost (or more crudely --noscripts) will
disable executing the script.

(my guess) In a chroot, with --nodeps, you likely don't
have necessary prerequisites installed for the %post
script to run.


So there is some errors as you can see but it in installed!
I see the files /usr/bin/8Kingdoms



Yes. RPM installs before %post is attempted.


nice wrok :)

The /etc/mtab error is normal... I am in a chrooted directory and no  
partition

table is available...

But the second error, I never seen it before :
error: %post(8Kingdoms-1.1.0-6.fc9.i386) scriptlet failed, exit  
status 255




Look in /var/tmp (or however you have configgered %_tmpdir)
and you will likely see the script that was attempted.

You will see something like
/var/tmp/rpm-tmp.123456
That is the scriptlet that was attempted.

Running that script outside of RPM in a sufficiently similar context
will fail in exactly the same way: exit status was 255.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: error: cannot open Packages database in /var/lib/rpm

[Jeff Johnson]

On May 28, 2009, at 3:09 PM, spam.spam.spam.s...@free.fr wrote:



Look in /var/tmp (or however you have configgered %_tmpdir)
and you will likely see the script that was attempted.

You will see something like
/var/tmp/rpm-tmp.123456
That is the scriptlet that was attempted.



...
1)I don't see this tmp file... My tmp/var/tmp/ folder is completly  
empty.

I try to find my tmpdir :
#rpm --showrc | grep tmpdir
-14: __dbi_other%{?_tmppath:tmpdir=%{_tmppath}} %{?__dbi_cdb}
-14: __dbi_sqlconfig_currentperms=0644 %{?_tmppath:tmpdir=% 
{_tmppath}}

-14: tmpdir %{_tmppath}
But I am not sure to understand. Where is it?



Run strace on rpm if necessary. RPM is expanding
the script into a file somewhere before executing, and
RPM does not remove failed (as in exit status = 255) scripts.

Look for open/execv lines in the strace spewage.

Note that %_tmpdir is the modern macro name for the traditional
configuration parameter %_tmppath which traditionally
has value /var/tmp. I cannot tell where your configuration
has chosen to map the path where scriptlets are written.

You can tell how you have configured RPM however. Examine
rpm --showrc
and adjust any discrepancies, particularly paths, to taste.

2)A strange thing is when I try to install a second time the RPM  
package :


rpm --nodeps -i 8Kingdoms-1.1.0-6.fc9.i386.rpm
warning: 8Kingdoms-1.1.0-6.fc9.i386.rpm: Header V3 DSA signature:  
NOKEY, key ID

4ebfc273
error: failed to open /etc/mtab: No such file or directory


(aside)
BTW, you can likely make this annoying message go away by doing
touch /path/to/chroot/etc/mtab
The file in the path just needs to exist somehow, doesn't even matter  
what is inside.




error: %post(8Kingdoms-1.1.0-6.fc9.i386) scriptlet failed, exit  
status 255

bash-4.0# rpm --nodeps -i 8Kingdoms-1.1.0-6.fc9.i386.rpm
warning: 8Kingdoms-1.1.0-6.fc9.i386.rpm: Header V3 DSA signature:  
NOKEY, key ID

4ebfc273
error: failed to open /etc/mtab: No such file or directory
error: ^(?:([^:-]+):)?([^:-]+)(?:-([^:-]+))?(?::([^:-]+))?$: regcomp  
failed:

Invalid preceding regular expression
Segmentation fault



You've lost mandatory PCRE libraries somehow. We did this analysis  
already ...


You can see now it absolutely wants I install libpcre (there is  
regularg
expression error because it dosen't find libpcre which is not  
installed for the

moment...). I tell --nodeps... But it wants to use libpcre.


Yes libpcre is mandatory. The easiest (imho) solution is configuring  
the build using


--with-pcre=internal



Why ONLY the second time I try to install, I see the error about  
regular

expression?


Dunno. You have lost mandatory -lpcre anytime you see the message:

error: ^(?:([^:-]+):)?([^:-]+)(?:-([^:-]+))?(?::([^:-]+))?$: regcomp  
failed:

Invalid preceding regular expression
Segmentation fault


That can happen for numerous reasons, including not controlling where  
libraries

are to be found.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Are rpm and rpm5 two different projects?

[Jeff Johnson]

On May 25, 2009, at 10:41 AM, spam.spam.spam.s...@free.fr wrote:


Hello,

Before I used 'rpm' project for my operating system : http://rpm.org/

Today, I find 'rpm5' project : http://rpm5.org/
I prefer it because the configure script in the sources allows more  
things to

disable...

The last release for 'rpm' is : 4.7.0
The last releade for 'rpm5' is : 5.1

Are rpm and rpm5 two different projects?



Yes, different projects with different goals.

The common source reference point was rpm-4.4.2 about 3-4 years ago,  
although

the code base @rpm5.org forked away from rpm.org years before that.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Can RPMs built for different distributions be used on Mac with this application

[Jeff Johnson]

On May 13, 2009, at 3:18 PM, Jerome BG wrote:

Sorry for the newbie question, but I would like to try an RPM  
install of PHP on my mac since I can't seem to get my own build to  
function properly. The builds I see available for php-devel (http://rpmfind.net/linux/rpm2html/search.php?query=php-devel 
) are for specific builds, and I would prefer not to have another  
nonfunctional copy of PHP on here from trying to install it with no  
chance of success.




Mixing *.rpm from linux - mac is a bit risky.

Might work for PHP, dunno.

If you install from *.rpm, erasing is dirt simple reliable doing
rpm -evv foo

But you can extract files from payload manually using rpm2cpio:
mkdir xxx
cd xxx
rpm2cpio foo*.rpm | cpio -dim

There's a shell script version called rpm2cpio.sh around as well.

hth

73 de Jeff

Re: RPM5 and YML-like Specfiles

[Jeff Johnson]

On May 10, 2009, at 3:40 PM, Anders F Björklund wrote:




If I understand the issue correctly, you *don't* want the indentation
stored in the package ? (like it is not being stored for other parts)



This is a problem with all mark-up, defining the boundaries for
invisible CDATA whitespace.

But specifically for indentation in %description, I could likely
left trim (and right trim) all the lines in %description
(and likely %changelog) in a matter of hours.

That's one way of clarifying what is CDATA and what is markup,
nuke it according to a rule.

With all the usual disabler and customization baggage to
Have ityour own way! that is invariably expected of rpm, sigh ...

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM5 and YML-like Specfiles

[Jeff Johnson]

On May 10, 2009, at 4:11 PM, Eric MSP Veith wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jeff,

I just got the impression YML was the way to go for you to have a  
grammar
for *.spec files. It seemed natural to me, since specs and YML  
already have
similar looks. I *don't* want any new customized Pimp-my-RPM  
features. I
just want to know how *you* want RPM spec files to look like, and  
want to

adopt that. (Even if you're going to choose XML, but I'm going to have
headaches then.)



On a more positive note:

The output of
rpm -qp --yaml *.src.rpm
has most of what I've been able to achieve with
a YAML representation for *.spec build recipes.

That content is pretty close to being sufficient
to drive a build, which achieves a split between
a proper parser and the build itself using
a source RPM header as an intermediate representation
of the data needed to drive a build.

Producing a SRPM much earlier, before the build
is actually started, would unsnarl the parsing
from the building. OTOH producing a SRPM earlier
is likely too big of a change for most rpmbuild
users to accomodate (the change is very not hard,
involves mebbe 10 lines of code that define the
mostly linear state machine implmented in rpmbuild).

If a build is driven by a SRPM header, many parsers,
YAML, XML, port(1), etc, could be attempted with
a common back-end build/packaging engine driven by the contents of
data carried in a SRPM header.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org