Re: A question about md5 being 00000000000000000000000000000000

[Jeff Johnson]

> On Oct 26, 2017, at 1:18 PM, zhang j  wrote:
> 
> Hi All,
> 
> I have a question about md5sum of a 0 sized file. The md5sum of a 0 sized 
> file built using rpm 5.4 on RHE 7 is . If do 
> 'rpm -qp --dump' of that rpm, the md5sum is shown 
> .
> 

You are correct that the MD5 of a zero length file is not zeroes:

$ md5sum < /dev/null
d41d8cd98f00b204e9800998ecf8427e  -

Are you sure this is rpm 5.4 (and not rpm4) on RHEL 7? There’s a fair amount of 
work in switching rpm4 -> rpm5 on RHEL7, congratulations if you succeeded!

Also: MD5 (for file digests) has largely been replaced with SHA256 everywhere 
in *.rpm packages.

> 
> But the real md5sum of that file is not . How 
> to build the correct md5sum in the rpm?
> 

I dimly remember someone objecting to the overhead of computing digests of 
empty files in *.rpm packages
like 8-10 years ago (but I could be mis-remembering).

That would have involved a patch to (likely) skip computing the digest for zero 
length files that (wrongly) 
returns the contents of a calloc’d buffer as .

73 de Jeff
> 
> Thanks,
> Juan

Re: pthread locking question

[Jeff Johnson]

> On Oct 15, 2017, at 4:06 AM, Arkadiusz Miśkiewicz  wrote:
> 
> On Sunday 15 of October 2017, Arkadiusz Miśkiewicz wrote:
>> Hi.
>> 
>> Is this lock supposed to be locked/unlocked before initialization?
> 
> Ok, this was missing
> 
> --- rpmio/rpmsq.c~  2017-10-15 09:34:23.0 +0200
> +++ rpmio/rpmsq.c   2017-10-15 10:05:39.859549557 +0200
> @@ -781,7 +781,9 @@
>(void) sigprocmask (SIG_SETMASK, , NULL);
> 
>/* Reset rpmsigTbl lock and refcnt. */
> +#ifndef PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP
>INIT_LOCK ();
> +#endif
> 
>(void) execve (argv[0], (char *const *) argv, environ);
>_exit (127);
> 

Thanks.

FWIW: The locking scheme (and macros) were borrowed from glibc ~2005 and not 
changed since … there appear to be some improvements/bugs/changes/fixes.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

RPM+MQTT in rpm-5.4.18 ...

[Jeff Johnson]

rpm-5.4.18 (soon) will embed the paho-mqtt client to do MQTT messaging.

I will likely do a snapshot of 5.4.18 as soon as I haul out some
MQTT debugging trash and repair portability damage.

TL;DR
Don't install paho-mqtt-1.1.0 and/or build rpm-5.4.18 --without-mqtt
and ignore the rest of this message.

Read further about RPM+MQTT in the attachment.

Discussions on  please.

73 de Jeff


MQTT.msg
Description: Binary data

Re: About rpm's SUPPORT_NOSIGNATURES

[Jeff Johnson]

On Jul 5, 2016, at 10:24 AM, Robert Yang wrote:

> Hi,
> 
> I'm using rpm 5.4.16, it seems that --nosignature has been disabled
> as system.h shows:
> #undef SUPPORT_NOSIGNATURES
> 

Yes.

YL;DR
Change SUPPORT_NOSIGNATURES back to whatever you wish.

> So that rpm --nosignature doesn't work, and when many processes(for example,
> 30) of "rpm -qp --nosignature file.rpm" are running, there might be race
> issues since they are trying to get locks for the db, and they may hang up.
> I've made a rough patch to make --nosignature work, they processes can
> run well without hang up. Here is the patch, and I will send it formally
> if you are fine with it.
> 

Um this isn't the right analysis.

First of all, all packages built by RPM5 have an included pubkey
that is used to verify the package signature produced by rpmbuild.

Second, even if you have clobbered the rpmbuild signature by
resigning *.rpm packages, there is exactly one retrieval per-pubkey
from either an rpmdb or (if configured) an hkp:// server.

Third, why are there 30+ rpm invocations running? That likely
should be fixed in whatever is triggering + rpm invocations. Meanwhile,
Berkeley DB should *easily* handle 30+ concurrent pubkey
retrievals. Of course opening an rpmdb is not exactly a lightweight
operation, but the solution there is performance related, change
whatever is invoking rpm 30+ times, likely by using the command
line arguments intelligently.

> 
> Subject: [PATCH] system.h/query.c: support nosignature
> 
> * Define SUPPORT_NOSIGNATURES to 1 in system.h
> * !QVA_ISSET(qva->qva_flags, SIGNATURE) -> QVA_ISSET(qva->qva_flags, 
> SIGNATURE),
>  otherwise, when use --nosignature would read database and verify
>  signature, this is not expected.
> 
> Upstream-Status: Pending
> 
> Signed-off-by: Robert Yang 
> ---
> lib/query.c | 6 +++---
> system.h| 4 ++--
> 2 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/query.c b/lib/query.c
> index 50a7453..b761d76 100644
> --- a/lib/query.c
> +++ b/lib/query.c
> @@ -954,19 +954,19 @@ JBJDEBUG((stderr, "--> %s(%p,%p,%p)\n", __FUNCTION__, 
> ts, qva, argv));
> vsflags = (rpmVSFlags) rpmExpandNumeric("%{?_vsflags_query}");
> vsflags = (rpmVSFlags) 0;  /* XXX FIXME: ignore default disablers. */
> #if defined(SUPPORT_NOSIGNATURES)
> -if (!QVA_ISSET(qva->qva_flags, DIGEST)) {
> +if (QVA_ISSET(qva->qva_flags, DIGEST)) {
>VSF_SET(vsflags, NOSHA1HEADER);
>VSF_SET(vsflags, NOMD5HEADER);
>VSF_SET(vsflags, NOSHA1);
>VSF_SET(vsflags, NOMD5);
> }
> -if (!QVA_ISSET(qva->qva_flags, SIGNATURE)) {
> +if (QVA_ISSET(qva->qva_flags, SIGNATURE)) {
>VSF_SET(vsflags, NODSAHEADER);
>VSF_SET(vsflags, NORSAHEADER);
>VSF_SET(vsflags, NODSA);
>VSF_SET(vsflags, NORSA);
> }
> -if (!QVA_ISSET(qva->qva_flags, HDRCHK)) {
> +if (QVA_ISSET(qva->qva_flags, HDRCHK)) {
>VSF_SET(vsflags, NOHDRCHK);
> }
> VSF_CLR(vsflags, NEEDPAYLOAD); /* XXX needed? */

Hmmm ... why was the above needed? Note that --verify flags are
the negation of --query flags, that might explain why you
are patching inverted logic into --query.

OTOH, its possible that I screwed the logic with triple negation and masking on
two modes of operation, wouldn't be the first time if so.

Meanwhile the code above has been "Known Good" for more than a decade,
so I'm not sure why this patch is needed.

> diff --git a/system.h b/system.h
> index 2ff8906..ad4619a 100644
> --- a/system.h
> +++ b/system.h
> @@ -787,9 +787,9 @@ static inline const char *rcsid(const char *p) {   
> #define SUPPORT_I18NSTRING_TYPE1
> 
> /**
> - * Eliminate signature/digest disablers.
> + * Signature/digest disablers.
>  */
> -#undef SUPPORT_NOSIGNATURES
> +#define SUPPORT_NOSIGNATURES 1
> 
> /**
>  * Permit ar(1) payloads. Disabled while rpmio/iosm.c is under development.
> 

Note that all the code marked with "SUPPORT_NOSIGNATURES" is targeted
for removal. RPM5 has been producing MANDATORY signed packages for
more than 5 years, so all packages produced by RPM5 SHOULD have
both verifiable signatures/pubkeys included for many years now.

The "fix" for @rpm.org packages (which may not be signed) is the obvious
rpm --addsign *.rpm
when/where needed.

Your alternative (of course) is to re-patch rpm to re-add --nosignatures as
you wish: I will be happy to send you the needed patch when I remove
all the SUPPORT_NOSIGNATURES code.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [Rpm-ecosystem] Using rpm db to track unneeded packages

[Jeff Johnson]

On Apr 1, 2016, at 9:03 AM, devzero2000 wrote:

> No bad idea
> 
> 


The semantics of "installed through dependencies" is
less useful than one might imagine. The intent is usually
to permit automatic  uninstall extensions when some
explicitly chosen package is uninstalled as in "unneeded".

The state machine in rpmlib (and transactional logs) cannot easily
accomodate, say, a subsidiary package that is "installed through
dependencies" that is then upgraded without the deliberately chosen
package being upgraded:.

Is that still a "installed through dependencies" package or not?
(i.e. is the magic marker persistent independent of
additional upgrades -- and changes in the package -- or not).

Sure one can invent semantics and define behavior explicitly,
just there are lots of side effects that what have to be undertaken
propagating the tag forward persistently. Consider what would be
needed with hot fixes like
rpm -e --nodeps --justdb
or
rpm --reinstall
to ensure that the tag could be maintained persistently and reliably.


Meanwhile all that RPM itself could do is provide the storage mechanism
with, say, a tag/state in a header in an rpmdb. RPM itself does
only what is specified, either from the CLI, or throuch libraries/bindings.

And if the storage is through an appended header tag, the behavior would not be
"persistent", and the side effects of propagating an appended tag through
additional upgrades to achieve "persistence".

Alternatively,  changing the rpmdb schema to add a different persistent
index tracking package state to save the "installed through dependencies"
state would likely not meet anyone's expectations any better than writing
the info that whatever the given depsolver wishes to have in some flat file.

hth

73 de Jeff

> -- Messaggio inoltrato --
> Da: "Pascal Terjan" 
> Data: 01/Apr/2016 14:12
> Oggetto: Re: [Rpm-ecosystem] Using rpm db to track unneeded packages
> A: , 
> Cc: 
> 
> On 1 April 2016 at 13:05, Tomas Chvatal  wrote:
> > Hello,
> >
> > I was wondering if it would be possible to extend rpmdb to contain
> > information about how package was pulled in the dependency graph.
> >
> > At this point we at openSUSE have some sort of solver trying to
> > magically do it in zypper and you guys at Fedora have it in dnf.
> >
> > I would more like to see it tracked in the rpm because that way we
> > could properly have all packages in the db, including the ones users
> > installed via rpm commands and could see if they are
> >  a) directly requested by user
> >  b) just dependency of something and thus eligible for removal
> 
> urpmi has its own tracking (a simple text file
> /var/lib/rpm/installed-through-deps.list) but I think that would be
> nice to have it in rpm directly so that it works whatever tool is used
> to install the packages
> 
> > I guess dnf/zypper would just flag them during install as True/False
> > depending if they are direct request or dependency and for rpm we would
> > always flag them as True for the solver as requested by user.
> >
> > Would something like this make sense?
> >
> > Cheers
> >
> > Tom
> > ___
> > Rpm-ecosystem mailing list
> > rpm-ecosys...@lists.rpm.org
> > http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> >
> ___
> Rpm-ecosystem mailing list
> rpm-ecosys...@lists.rpm.org
> http://lists.rpm.org/mailman/listinfo/rpm-ecosystem

rpm-5.4.16-0.20160321.src.rpm, take 3

[Jeff Johnson]

There is a 3rd snapshot release of rpm-5.4.16 now available at


http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT//rpm-5.4.16-0.20160321.src.rpm

This addresses the following issues:
1) mongoc.h needed #include  to rebuild with PLD configuration
2) build with or without in "system.h"
#define SUPPORT_I18NSTRING_TYPE 1

The tarball has
#undef SUPPORT_I18NSTRING
which will display the Summary/Description/Group tags ifrom the header n the C 
locale aways,
and otherwise eliminates the RPM_I18NSTRING_TYPE interface.

The tarball has been checked by building with a lightly modified PLD rpm.spec 
file from here:


http://git.pld-linux.org/gitweb.cgi?p=packages/rpm.git;a=tree;h=dev-5.4.16;hb=dev-5.4.16

and examining the built package metadata content using

rpm -qip *.rpm | more

and

 rpm -qp --yaml *.rpm | more

to ensure that Summary/Description/Group display (and were as expected) with or 
without
SUPPORT_I18NSTRING_TYPE

For PLD, you will need to
1) patch system.h to add
#define SUPPORT_I18NSTRING_TYPE 1
2) don't apply the patch
rpm-dont-copy-descriptive-tags.patch

hth

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: rpm-5.4.16 snapshot, take 2

[Jeff Johnson]

(aside)
I've added Reply-To: <rpm-devel@rpm5.org> which is likely the better forum,
and because I did not receive this msg for whatever reason:
rpm-5.4.16 snapshot, take 2

> i wish it would be possible to disable building cruft that is not used. 
> that is tests (as we do not run tests) and some weird and perhaps 
> experimental tools like b2sum. pld intention is not to do some build 
> coverage, but just get /bin/rpm compiled and packaged
Somehow I have to release one rpm-5.4.16 tarball, not one for every possible 
distro
purpose. The issue with missing va_start has nothing to do with "cruft", but 
rather
that librpmio is misbuilt on PLD somehow: ensuring that
#include 
happens will surely be the fix. Can you find which .o is not including 
?
Or make the actual failed build available so that I can diagnose?

> and the text fields still contain garbage:
As I tried to say below, PLD and rpm5.org are likely going to differ on 
removing the
RPM_I18NSTRING_TYPE =  9
data type from rpm headers.
I explicitly verified that Summary/Group/Description could be displayed
using both rpm-4.13.0 (as in Fedora 23), and rpm-5.4.16, in both
LANG=en_US.UTF-8
and
LANG=fr_FR
with the underlying switch disabling the change in system.h:
/**
 * Eliminate RPM_I18NSTRING_TYPE.
 */
#define SUPPORT_I18NSTRING_TYPE 1
(which is what PLD is likely going to use).
I will check the other value of SUPPORT_I18NSTRING_TYPE next.

hth

73 de Jeff



On Mar 18, 2016, at 3:49 AM, Jeff Johnson wrote:

> I have uploaded another snapshot release that addresses
> all the issues you have reported:
> 
>   rpm-5.4.16-0.20160318.src.rpm
> 
> The issue of garbled text is both locale and rpm version dependent
> and is _NOT_ reproducible with rpm-5.4.16, but is reproducible
> with rpm-4.13.0 when a locale is specified.
> 
> PLD and rpm5.org are likely to disagree on the issue of including
> I18N in spec files and in headers no matter what.
> 
> I have checked that rpm can be built using --without-openssl
> and/or --without-acl.
> 
> The linkage problem with rpmio/tgfs which uses shm_open(3) has to do
> with a missing -lrt which may be implicitly specified by some other
> configure option, perhaps OPENMP.
> 
> Meanwhile I have hardwired -lrt for tgfs ldflags. The other issues I had with
> tgfs (a test program for GridFS using MongoDB) had to do with 
> misconfiguration,
> not anything else. There is a need for an error message which I will add 
> shortly.
> 
> hth
> 
> 73 de Jeff
> 
> On Mar 15, 2016, at 6:14 PM, Elan Ruusamäe wrote:
> 
>> On 15.03.2016 22:27, Jeff Johnson wrote:
>>> There is a snapshot release of rpm-5.4.16 now available at
>>> 
>>>
>>> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm
>> it's text fields seem to contain garbage:
>> 
>> [~/rpm/packages/rpm(5.4.15) (dev-5.4.16)⚡] ➔ less 
>> rpm-5.4.16-0.20160315.src.rpm
>> Name: rpm  Relocations: (not relocatable)
>> Version : 5.4.16Vendor: (none)
>> Release : 0.20160315Build Date: T 15 märts 2016 
>> 21:13:03 EET
>> Install Date: (not installed)Build Host: hi.jbj.org
>> Group   : (^E^B  Source RPM: (none)
>> Size: 20861253     License: LGPLv2
>> Signature   : RSA/SHA1, T 15 märts 2016 21:13:03 EET, Key ID fd73f8a36edb4910
>> URL : http://rpm5.org
>> Summary : <88>^F^B
>> Architecture: x86_64
>> Description :
>> ^F^B
>> * K märts 16 2016 Jeff Johnson <j...@rpm5.org> -5.4.16
>> - improve uncoupling from rpmbuild configuration.
>> 
>> 
>> anyway, will push soon dev-5.4.16 branch in pld repo [1] for anyone wishing 
>> to test it
>> 
>> ps: tarball size increased 5mb:
>> 
>> [~/rpm/packages/rpm(5.4.16) (dev-5.4.16)⚡] ➔ l rpm-5.4.*gz
>> -rw-r- 1 glen users 15M 25. okt2014 rpm-5.4.15.tar.gz
>> -rw-r--r-- 1 glen users 20M 15. märts 21:04 rpm-5.4.16.tar.gz
>> 
>> 
>> [1] http://git.pld-linux.org/packages/rpm.git & 
>> https://github.com/pld-linux/rpm
>> 
>> -- 
>> glen
>> 
>> ___
>> pld-devel-en mailing list
>> pld-devel...@lists.pld-linux.org
>> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
> 
> ___
> pld-devel-en mailing list
> pld-devel...@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

rpm-5.4.16 snapshot

[Jeff Johnson]

There is a snapshot release of rpm-5.4.16 now available at

http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm

This is the first SRPM built by itself that is headed for release
in the next few weeks that is being provided as a public reference
point for integration and portability testing.

See the included INSTALL document for the build pre-requisite versions used.

From a distro POV, please note the following changes that are included
in the snapshot that will (at least) need to be considered when upgrading:

1) (recommended) rpm-5.4.16 uses BLAKE2bp for file digests.
  BLAKe2bp is a 256bit digest that is faster than SHA256 (and MD5)
  that will improve installation speeds.

  Details are here:
https://blake2.net

2) (recommended) rpm-5.4.16 uses libtomcrypt (rather than BeeCrypt).
  LibTomCrypt has support for ECDSA and is used by recent python and
  the linux kernel (iirc).

  Details are here:
https://github.com/libtom/libtomcrypt

3) (recommended) rpm-5.4.16 uses db-6.1.23 (not 6.1.26) with
  DB_MULTIVERSION and DB_TXN_SNAPSHOT.
  DB_TXN_SNAPSHOT avoids deadlocks with copy-on-write rather than
  locking semantics.

  The change is necessary to support nested transactional commits
  in rpm like
command transaction
   package transaction
  install transaction
  erase transaction
  without deadlocking on trigger lookups.

  Details about DB_MULTIVERSION and DB_TXN_SNAPSHOT can be found
  in the Oracle Berkeley DB documentation here:
http://docs.oracle.com/cd/E17076_04/html/index.html

As always, rpm can be configured to use any of ~120 digests, any of
BeeCrypt
NSS
Openssl
Libgcrypt
LibTomCrypt
and (most likely, unchecked) any version of Berkeley DB back to db-4.6.x.

Bug reports are requested at
https://launchpad.net/rpm

Patches and discussion are requested at


Enjoy!

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

EVR issues: set:versions, epoch-as-string, now twiddle-in-version

[Jeff Johnson]

OK, adoption of the ancient SuSE twiddle-in-version patch by
@rpm.org has now added yet a 3rd incompatibility that needs
to be addressed within what is commonly known as rpmvercmp

(aside)
The designator rpmvercmp is already horrendously misleading
because its the name of the comparison routine for each of the
V and R members of the tuple {E,V,R}. There are many many
many false starts at package version comparisons that naively
attempt
rc = rpmvercmp(E:V-R, whatever);
when what is actually needed is a loop of each member of the
tuple like
static char * EVRprecedence = EVR;
char ptype;
int i;

for (i = 0; (ptype = EVRprecedence[i]) != '\0; i++) {
switch (ptype) {
case 'E':   /* coerce and compare Epoch: digit strings */   break;
case 'V':
case 'R':
case 'D':   /* Distepoch/Disttag comparison is similar to V and R 
iirc */
rc = rpmvercmp(somestring, anotherstring);  break;
}
if (rc) break;
}

I've written out the algorithm so that the asymmetry of the full
comparison with the existing convention of Epoch: as digit string
can be more easily seen.

With 3 incompatibilities in existing EVR comparison looming,
I am tipped in favor of treating Eoch: more generally as a string, not
just a digit sequence, in order to
Handle all the incompatibilities (to disappoint every one equally) at 
the same time.

The issue of whether Epoch: SHOULD be treated as a [0-9]+: pattern is
merely a stricter policy convention on the general case. There are other
possible policy conventions with V and R that could and should be
imposed more strictly than
Neither V nor R can contain a '-' character.
which has always been seen as some sort of deficiency in RPM compared
to APT/DPKG.

I can also see some usage case if set is described as a namespace marker
for a type of comparison, particularly since '= is being overloaded as a
subset contained within semantic instead of rpmvercmp comparison.

The next most useful namespace marker is likely going to be deb (or
dpkg or borg to taste ;-) to at least attempt some means to also
write regression tests for the corner cases that are surely going to be
recognized soon. Because of pattern parsing ambiguity with ':' (which
has already been overused as a separator of Disttag/Distepoch), this
would mean that adding a deb item to the (being proposed here) EVR
name space based on a generalization of Epoch: as a string marker
for dpkg comparison (where Yet Another meaning for ':' is encountered).

An example is easier than words: Suppose that there is a EVR string like
0:1.2-3
that then needs a deb or rpm name space marker. The parser pattern
could/should/would then return a tuple like
{deb:0, 1.2, 3 }
or
{deb, 0:1.2, 3 }
Either parsing is deterministic and implementable. I point out the details
solely to indicate some of the implementation issues that will need
a conventionally defined answer.

So the generalization to epoch-as-string (with an enforcement pattern of [0-9]+
for the Luddites) use now
+1
from me.

And its time to open the discussion for other comments. Note that you likely
have 2-3 weeks to respond before I get around to attempting the epoch-as-string
(and twiddle-in-version and set:versions generation) implementations.

Other opinions?

(aside)
A simple +1, 0, or -1 reply suffices and will be tallied. Talking
about package manager version comparison in public is known to
have carcinogenic side effects inducing brain tumors.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Fwd: EVR issues: set:versions, epoch-as-string, now twiddle-in-version

[Jeff Johnson]

I should point out that writing the attached
message (and sending from the wrong e-mail address) has instantly
led to a different -- and perhaps more natural -- syntax like

Requires: set(libfoo.so.1) = whatever

for set:versions, and for the generalization (for writing strict regression 
tests,
its mostly useless in packaging because there is no mapping that specifies
how the mixed DEB - RPM version comparison might be done naturally)

Requires: deb(foo) = E:V-R
Requires: rpm(foo) = E:V-R

The precedent for foo(bar) name spacing in RPM dependencies with
the above syntax is already widely deployed although entirely
de facto.

Sure would be nice _NOT_ to have to consider Have it your own way!
competing syntaxes like
Requires: libfoo.so.1 = set:whatever
and
Requires: set(libfoo.so.1) = whatever
(and I'll ignore the gawdafulness of existing multilib dependencies like
Requires: libfoo.ao.1()(64bit)
coming along rapidly as a Newer! Better! Bestest! ABI standard now
that there is support in kernel = 3.2.

*shrug*

Its all implementable, just not very KISS when aesthetic/innate opinions get 
involved.

hth

73 de Jeff


Begin forwarded message:

 From: Jeff Johnson n3npq@gmail.com
 Subject: EVR issues: set:versions, epoch-as-string, now twiddle-in-version
 Date: April 23, 2012 10:16:28 AM EDT
 To: rpm-devel@rpm5.org
 
 OK, adoption of the ancient SuSE twiddle-in-version patch by
 @rpm.org has now added yet a 3rd incompatibility that needs
 to be addressed within what is commonly known as rpmvercmp
 
 (aside)
 The designator rpmvercmp is already horrendously misleading
 because its the name of the comparison routine for each of the
 V and R members of the tuple {E,V,R}. There are many many
 many false starts at package version comparisons that naively
 attempt
   rc = rpmvercmp(E:V-R, whatever);
 when what is actually needed is a loop of each member of the
 tuple like
static char * EVRprecedence = EVR;
char ptype;
int i;
 
for (i = 0; (ptype = EVRprecedence[i]) != '\0; i++) {
   switch (ptype) {
   case 'E':   /* coerce and compare Epoch: digit strings */   break;
   case 'V':
   case 'R':
   case 'D':   /* Distepoch/Disttag comparison is similar to V and R 
 iirc */
   rc = rpmvercmp(somestring, anotherstring);  break;
   }
   if (rc) break;
}
 
 I've written out the algorithm so that the asymmetry of the full
 comparison with the existing convention of Epoch: as digit string
 can be more easily seen.
 
 With 3 incompatibilities in existing EVR comparison looming,
 I am tipped in favor of treating Eoch: more generally as a string, not
 just a digit sequence, in order to
   Handle all the incompatibilities (to disappoint every one equally) at 
 the same time.
 
 The issue of whether Epoch: SHOULD be treated as a [0-9]+: pattern is
 merely a stricter policy convention on the general case. There are other
 possible policy conventions with V and R that could and should be
 imposed more strictly than
   Neither V nor R can contain a '-' character.
 which has always been seen as some sort of deficiency in RPM compared
 to APT/DPKG.
 
 I can also see some usage case if set is described as a namespace marker
 for a type of comparison, particularly since '= is being overloaded as a
 subset contained within semantic instead of rpmvercmp comparison.
 
 The next most useful namespace marker is likely going to be deb (or
 dpkg or borg to taste ;-) to at least attempt some means to also
 write regression tests for the corner cases that are surely going to be
 recognized soon. Because of pattern parsing ambiguity with ':' (which
 has already been overused as a separator of Disttag/Distepoch), this
 would mean that adding a deb item to the (being proposed here) EVR
 name space based on a generalization of Epoch: as a string marker
 for dpkg comparison (where Yet Another meaning for ':' is encountered).
 
 An example is easier than words: Suppose that there is a EVR string like
   0:1.2-3
 that then needs a deb or rpm name space marker. The parser pattern
 could/should/would then return a tuple like
   {deb:0, 1.2, 3 }
 or
   {deb, 0:1.2, 3 }
 Either parsing is deterministic and implementable. I point out the details
 solely to indicate some of the implementation issues that will need
 a conventionally defined answer.
 
 So the generalization to epoch-as-string (with an enforcement pattern of 
 [0-9]+
 for the Luddites) use now
   +1
 from me.
 
 And its time to open the discussion for other comments. Note that you likely
 have 2-3 weeks to respond before I get around to attempting the 
 epoch-as-string
 (and twiddle-in-version and set:versions generation) implementations.
 
 Other opinions?
 
 (aside)
 A simple +1, 0, or -1 reply suffices and will be tallied. Talking
 about package manager version comparison in public is known to
 have carcinogenic side effects inducing

Re: [CVS] RPM: rpm-5_4: rpm/ CHANGES rpm/lib/ rpmchecksig.c rpm/rpmio/ mac...

[Jeff Johnson]

Umm … whatever.

I don't mind living with this patch: but the precedent (if continued) will force
umask(2) wrapping of every system call that sets a file mode, and also
for library functions like mkstemp(3), if consistently applied everywhere.

That end-point -- controlling for umask(2) side-effects everywhere --
isn't the best approach to setting file modes deterministically in
order to meet expectations.

There are surely library calls that exist that need both +umask and -umask
because more than one file mode needs to be set.

hth

73 de Jeff

On Sep 30, 2011, at 8:43 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   30-Sep-2011 14:43:35
  Branch: rpm-5_4  Handle: 2011093012433202
 
  Modified files:   (Branch: rpm-5_4)
rpm CHANGES
rpm/lib rpmchecksig.c
rpm/rpmio   macro.c
 
  Log:
call umask before mkstemp.
 
The mkstemp() function generates a unique temporary file name from the 
 supplied template,
opens a file of that name using the O_EXCL flag (guaranteeing the current 
 process to be the only user)
and returns a file descriptor.
 
The POSIX specification does not say anything about file modes, so the 
 application should make sure its umask
is set appropriately before calling mkstemp
( ref. 
 https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/coding/781-BSI.html)
 
  Summary:
RevisionChanges Path
1.3501.2.174+8  -0  rpm/CHANGES
1.240.2.3   +3  -0  rpm/lib/rpmchecksig.c
2.249.2.5   +3  -0  rpm/rpmio/macro.c
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3501.2.173 -r1.3501.2.174 CHANGES
  --- rpm/CHANGES  24 Sep 2011 19:36:10 -  1.3501.2.173
  +++ rpm/CHANGES  30 Sep 2011 12:43:32 -  1.3501.2.174
  @@ -1,4 +1,12 @@
   5.4.2 - 5.4.3:
  +- devzero2000: call umask before mkstemp.
  +  The mkstemp() function generates a unique temporary file name from 
 the supplied template, 
  +  opens a file of that name using the O_EXCL flag (guaranteeing the 
 current process to be the only user)
  +  and returns a file descriptor.
  +
  +  The POSIX specification does not say anything about file modes, so 
 the application should make sure its umask 
  +  is set appropriately before calling mkstemp
  +  ( ref. 
 https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/coding/781-BSI.html)
   - jbj: js: port and compile against mosjs185 API. more todo++.
   - jbj: arm: add arm-peculier cpuinfo fields.
   - jbj: ruby: disable make check until -lpcreposix gets linked 
 correctly.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/lib/rpmchecksig.c
  
  $ cvs diff -u -r1.240.2.2 -r1.240.2.3 rpmchecksig.c
  --- rpm/lib/rpmchecksig.c5 Sep 2011 23:00:28 -   1.240.2.2
  +++ rpm/lib/rpmchecksig.c30 Sep 2011 12:43:33 -  1.240.2.3
  @@ -206,6 +206,7 @@
   rpmRC rpmrc = 0;
   int xx;
   int i;
  +mode_t mode;
 
   tmprpm[0] = '\0';
 
  @@ -417,7 +418,9 @@
   (void) stpcpy( stpcpy(tmprpm, fn), .XX);
 
   #if defined(HAVE_MKSTEMP)
  + mode = umask(0077);
   (void) close(mkstemp(tmprpm));
  +(void) umask(mode);
   #else
   (void) mktemp(tmprpm);
   #endif
  @@ .
  patch -p0 '@@ .'
  Index: rpm/rpmio/macro.c
  
  $ cvs diff -u -r2.249.2.4 -r2.249.2.5 macro.c
  --- rpm/rpmio/macro.c6 Sep 2011 15:03:54 -   2.249.2.4
  +++ rpm/rpmio/macro.c30 Sep 2011 12:43:34 -  2.249.2.5
  @@ -1237,6 +1237,7 @@
   char * buf = alloca(bufn);
   char *b = NULL, *be;
   int c;
  +mode_t mode;
 
   buf[0] = '\0';
   if (g != NULL) {
  @@ -1370,7 +1371,9 @@
   be++;
   /*@=globs@*/
   #if defined(HAVE_MKSTEMP)
  +mode = umask(0077);
   (void) close(mkstemp(b));
  +(void) umask(mode);
   #else
   (void) mktemp(b);
   #endif
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm-5_4: rpm/ CHANGES rpm/lib/ rpmchecksig.c rpm/rpmio/ mac...

[Jeff Johnson]

On Sep 30, 2011, at 9:12 AM, devzero2000 wrote:

 On Fri, Sep 30, 2011 at 2:53 PM, Jeff Johnson n3npq@gmail.com wrote:
 Umm … whatever.
 
 I don't mind living with this patch: but the precedent (if continued) will 
 force
 umask(2) wrapping of every system call that sets a file mode, and also
 for library functions like mkstemp(3), if consistently applied everywhere.
 
 No. It is only for function as mkstemp for now 
 https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/coding/781-BSI.html

Then add the pointer to the reason if solely limited to a (misguided and 
foolish imho, whatever)
security report. Citing the reason for adding umask(2) wrapping to mkstemp(3)
will limit the scope and inform the reason for the wrapping.

73 de Jeff

 Thanks
 Best Regards
  
 That end-point -- controlling for umask(2) side-effects everywhere --
 isn't the best approach to setting file modes deterministically in
 order to meet expectations.
 
 There are surely library calls that exist that need both +umask and -umask
 because more than one file mode needs to be set.
 
 hth
 
 73 de Jeff
 
 On Sep 30, 2011, at 8:43 AM, Pinto Elia wrote:
 
   RPM Package Manager, CVS Repository
   http://rpm5.org/cvs/
   
  
 
   Server: rpm5.org Name:   Pinto Elia
   Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
   Module: rpm  Date:   30-Sep-2011 14:43:35
   Branch: rpm-5_4  Handle: 2011093012433202
 
   Modified files:   (Branch: rpm-5_4)
 rpm CHANGES
 rpm/lib rpmchecksig.c
 rpm/rpmio   macro.c
 
   Log:
 call umask before mkstemp.
 
 The mkstemp() function generates a unique temporary file name from the 
  supplied template,
 opens a file of that name using the O_EXCL flag (guaranteeing the 
  current process to be the only user)
 and returns a file descriptor.
 
 The POSIX specification does not say anything about file modes, so the 
  application should make sure its umask
 is set appropriately before calling mkstemp
 ( ref. 
  https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/coding/781-BSI.html)
 
   Summary:
 RevisionChanges Path
 1.3501.2.174+8  -0  rpm/CHANGES
 1.240.2.3   +3  -0  rpm/lib/rpmchecksig.c
 2.249.2.5   +3  -0  rpm/rpmio/macro.c
   
  
 
   patch -p0 '@@ .'
   Index: rpm/CHANGES
   
  
   $ cvs diff -u -r1.3501.2.173 -r1.3501.2.174 CHANGES
   --- rpm/CHANGES  24 Sep 2011 19:36:10 -  1.3501.2.173
   +++ rpm/CHANGES  30 Sep 2011 12:43:32 -  1.3501.2.174
   @@ -1,4 +1,12 @@
5.4.2 - 5.4.3:
   +- devzero2000: call umask before mkstemp.
   +  The mkstemp() function generates a unique temporary file name from 
  the supplied template,
   +  opens a file of that name using the O_EXCL flag (guaranteeing the 
  current process to be the only user)
   +  and returns a file descriptor.
   +
   +  The POSIX specification does not say anything about file modes, so 
  the application should make sure its umask
   +  is set appropriately before calling mkstemp
   +  ( ref. 
  https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/coding/781-BSI.html)
- jbj: js: port and compile against mosjs185 API. more todo++.
- jbj: arm: add arm-peculier cpuinfo fields.
- jbj: ruby: disable make check until -lpcreposix gets linked 
  correctly.
   @@ .
   patch -p0 '@@ .'
   Index: rpm/lib/rpmchecksig.c
   
  
   $ cvs diff -u -r1.240.2.2 -r1.240.2.3 rpmchecksig.c
   --- rpm/lib/rpmchecksig.c5 Sep 2011 23:00:28 -   1.240.2.2
   +++ rpm/lib/rpmchecksig.c30 Sep 2011 12:43:33 -  1.240.2.3
   @@ -206,6 +206,7 @@
rpmRC rpmrc = 0;
int xx;
int i;
   +mode_t mode;
 
tmprpm[0] = '\0';
 
   @@ -417,7 +418,9 @@
(void) stpcpy( stpcpy(tmprpm, fn), .XX);
 
#if defined(HAVE_MKSTEMP)
   + mode = umask(0077);
(void) close(mkstemp(tmprpm));
   +(void) umask(mode);
#else
(void) mktemp(tmprpm);
#endif
   @@ .
   patch -p0 '@@ .'
   Index: rpm/rpmio/macro.c
   
  
   $ cvs diff -u -r2.249.2.4 -r2.249.2.5 macro.c
   --- rpm/rpmio/macro.c6 Sep 2011 15:03:54 -   2.249.2.4
   +++ rpm/rpmio/macro.c30 Sep 2011 12:43:34 -  2.249.2.5
   @@ -1237,6 +1237,7 @@
char * buf = alloca(bufn);
char *b = NULL, *be;
int c;
   +mode_t mode;
 
buf[0] = '\0';
if (g != NULL) {
   @@ -1370,7 +1371,9

Re: [CVS] RPM: rpm-5_4: rpm/ CHANGES rpm/lib/ rpmchecksig.c rpm/rpmio/ mac...

[Jeff Johnson]

On Sep 30, 2011, at 9:16 AM, Jeff Johnson wrote:

 
 On Sep 30, 2011, at 9:12 AM, devzero2000 wrote:
 
 On Fri, Sep 30, 2011 at 2:53 PM, Jeff Johnson n3npq@gmail.com wrote:
 Umm … whatever.
 
 I don't mind living with this patch: but the precedent (if continued) will 
 force
 umask(2) wrapping of every system call that sets a file mode, and also
 for library functions like mkstemp(3), if consistently applied everywhere.
 
 No. It is only for function as mkstemp for now 
 https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/coding/781-BSI.html
 

For pedantry mostly:

There's nothing at the cited link that even mentions the permissions
of the created file (and so its irrelevant whether umask is in use or not).

The exploit is largely a TOCTOU issue: A file created using tmp name(3)
might be substituted before being opened.

The recommended fix at the cited linkt is to use only the fdno,
and to permit mkstemp(3) to open O_EXCL to ensure that
the content attached to the fdno cannot be substituted.

What RPM does in the macro.c code you have patched, where there isn't
any means to pass back a fdno, is rely on +t to be set correctly on whatever
is in use as a temp directory. While relying on other state like +t  isn't as 
strong/strict
a test using only the fdno returned by mkstemp(3) opening a file with O_EXCL, 
the
simple sequence (ignoring error codes)
close(mkstemp(path))
is sufficient to ensure that a path has been created, owned by the current 
user, and
usually in /tmp where +t provides a guarantee that other users cannot 
substitute
the path with a symlink.

The other part of your patch is ancient hysteria that likely should use the
%{mkstemp:whateverXX}
macro primitive these days: the rpmchecksig code predates the implementation
of %{mkstemp:…} and was left as is to preserve the vslue of security audits.

The other place where temp file creation might occur that RPM uses consistently
since forever is in rpmTempFile(), where a different approach is taken to ensure
that the created temp file path cannot be substituted before being opened (the
mode of the file is checked after opening). That code has been audited many 
times
and has been explitly pointed out (by me) while audits were being done.

There's another mkstemp(3) binding in lua, and there's also the mkdtemp(3)
new-fangled analogue to mkstemp(3) that would need umask(2) wrapping
if you wish to control for not only the TOCTOU raciness, but also the created
tmp file mode.

The proper place to set permissions really SHOULD be a patch to mkstemp,
not in trying to control the implicit state in an application like RPM that 
might
be used by mkstemp.

hth

73 de Jeff

Re: script for BuildRequires to graphviz?

[Jeff Johnson]

On Sep 20, 2011, at 11:44 AM, Tim Mooney wrote:

 In regard to: Re: script for BuildRequires to graphviz?, Jeff Johnson said...:
 
 
 On Sep 15, 2011, at 1:23 PM, Jeff Johnson wrote:
 
 
 On Sep 15, 2011, at 12:43 PM, Tim Mooney wrote:
 
 
 All-
 
 Before I re-invent the (possibly simple) wheel, I have a vague
 recollection of seeing a script that can take a collection of spec files
 and build a directed graph from the BuildRequires, for graphing in
 something like graphviz.
 
 
 Yep. I can resurrect that tool (rpmgraph.c? I fergit) if interested.
 
 
 Dunno if you are using @rpm5.org or not. But there are two options
  --needswhat
  --whatneeds
 that look up towards the root of the dependency graph, or down into
 the sub-graph, displaying the packages found including all dependencies.
 
 I'm currently using rpm 5.1.9.  Thanks for the heads-up on those two
 options.  I wasn't aware of them.
 

Holler if you see issues with --needswhat/--whatneeds: I've looked at spot 
cases and have seen
what I expected at the time I implemented.

The real world is far more complex …

Upgrading to rpm-5.2.1 is mostly painless: the critical issue there
has to do with using PROT_READ to make memory blob's retrieved from
an rpmdb RO. There's a retrofit that dynamically rewrites a data
type in the blob for ancient packages that was attempted in order
to make the assertion:
All integers in RPMTAG_* are unsigned.
The rewrite for a single tag was needed to absolutely and unequivocally
remove RPM_CHAR_TYPE from older headers.

The way that pubkeys are wrapped and saved in an rpmdb is also slightly
different in rpm-5.2.1: ate worst, removing and re-importing pubkeys might
be needed.

 The man pages for rpmgraph are still part of the source from RPM, but
 rpmgraph.c appears to no longer be present.  I think it was rpmgraph
 that I was remembering.
 

rpmgraph.c is in CVS and could be resurrected (and improved) pretty easily:
the basics in RPM's API haven't meaningfully changed since forever.

Still: bindings are perhaps the better approach, python/perl (and JS and perhaps
ruby) have suffcient methods to construct a transaction, call 
rpmtsCheck/rpmtsOrder,
and printout *.dot markup suffcient to use graphviz and dotty.

 I don't think there's anything that you need to do, especially since
 most of the options are designed to work with information from the
 database and operate upon Requires, and I'm more interested in starting
 with a heap of spec files and generating the graph for BuildRequires (not
 Requires).
 
 It should be pretty easy to script what I need, so I think I'll just
 go that route.
 

I'll see if I can find Olivier Thauvin's very pretty (because of +coloring)
dependency graph generator. The original implementation was done with rpm-4.4.6.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: script for BuildRequires to graphviz?

[Jeff Johnson]

On Sep 15, 2011, at 12:43 PM, Tim Mooney wrote:

 
 All-
 
 Before I re-invent the (possibly simple) wheel, I have a vague
 recollection of seeing a script that can take a collection of spec files
 and build a directed graph from the BuildRequires, for graphing in
 something like graphviz.
 

Yep. I can resurrect that tool (rpmgraph.c? I fergit) if interested.

(aside)
Olivier Thauvin formerly @mandriva did a much nicer job in perl with colors etc.
I can likely dig that libnk out, and Olivier Thauvin can be found on IRC and/or
in Mageia. He also did the package database at sophie.zarb.org
now being reused by Mageia.

The trick is to avoid all the duplication involved with rpm dependencies.

WHat is needed is to choose the last node remaining as the parent
for a given package. One ends up with many fewer edges that
way, and also captures the essential dependency:
After the parent is installed, this package can be installed.

 I'm not finding it right now, and Google isn't giving me what I want
 either.
 
 I can do it myself and share if need be, but if there's prior art, I
 would appreciate a pointer to it.
 

I can probably help with RPM internals: the rpmte transaction element
still carries the index of the parent as defined above iirc.

But you're likely better off in any scripted language (that has a getter
for the index that I just allude to) than using rpmgraph.c, which
really was just a bunch of baggage around a fprintf(3) to emit the
*.dot line needed.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: script for BuildRequires to graphviz?

[Jeff Johnson]

On Sep 15, 2011, at 1:23 PM, Jeff Johnson wrote:

 
 On Sep 15, 2011, at 12:43 PM, Tim Mooney wrote:
 
 
 All-
 
 Before I re-invent the (possibly simple) wheel, I have a vague
 recollection of seeing a script that can take a collection of spec files
 and build a directed graph from the BuildRequires, for graphing in
 something like graphviz.
 
 
 Yep. I can resurrect that tool (rpmgraph.c? I fergit) if interested.
 

Dunno if you are using @rpm5.org or not. But there are two options
--needswhat
--whatneeds
that look up towards the root of the dependency graph, or down into
the sub-graph, displaying the packages found including all dependencies.

The parentdir/linkto dependencies are sure to be mysterious.

You would need to generate a solved and use --dbpath to point
at the solvedb rather than the default /var/lib/rpm, if you wish
to use --needswhat/--whatneeds as a standalone tool.

If I add a header tag extension to access the parent index,
then --queryformat templating is likely sufficient to have rpm
itself generate the *.dot format that you are seeking.

I'll knock that implementation out in the next week if interested.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/build/ parsePrep.c rpm/ configure.ac rpm/m...

[Jeff Johnson]

Could you push all these changes back to the rpm-5_4 branch
too please? That's where the buildbot's are running, and
I am active.

Otherwise these changes are gonna reside on HEAD until
a ROADMAP or participation exists, and that isn't likely
soon.

Thanks!

73 de Jeff

On Sep 6, 2011, at 10:41 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   06-Sep-2011 16:41:18
  Branch: HEAD Handle: 2011090614411701
 
  Modified files:
rpm CHANGES configure.ac
rpm/build   parsePrep.c
rpm/macros  macros.in
rpm/rpmio   macro.c rpmmacro.h
 
  Log:
Add lzip support
 
  Summary:
RevisionChanges Path
1.3674  +1  -0  rpm/CHANGES
2.137   +3  -0  rpm/build/parsePrep.c
2.501   +1  -0  rpm/configure.ac
1.61+2  -1  rpm/macros/macros.in
2.253   +6  -0  rpm/rpmio/macro.c
2.58+2  -1  rpm/rpmio/rpmmacro.h
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3673 -r1.3674 CHANGES
  --- rpm/CHANGES  6 Sep 2011 13:39:52 -   1.3673
  +++ rpm/CHANGES  6 Sep 2011 14:41:17 -   1.3674
  @@ -1,4 +1,5 @@
   HEAD:
  +- devzero2000: add lzip support 
   - devzero2000: add lrzip support 
 (blueprint 
 https://blueprints.launchpad.net/rpm/+spec/rpm5-use-lrzip-as-new-compression-program)
   - devzero2000: merge commit 31d807ffa8cdf91addf7c39fdf9abf6c61d77944
  @@ .
  patch -p0 '@@ .'
  Index: rpm/build/parsePrep.c
  
  $ cvs diff -u -r2.136 -r2.137 parsePrep.c
  --- rpm/build/parsePrep.c6 Sep 2011 13:39:52 -   2.136
  +++ rpm/build/parsePrep.c6 Sep 2011 14:41:17 -   2.137
  @@ -323,6 +323,9 @@
   case COMPRESSED_XZ:
   t = %{__xz} -dc;
   break;
  +case COMPRESSED_LZIP:
  +t = %{__lzip} -dc;
  +break;
   case COMPRESSED_LRZIP:
   t = %{__lrzip} -dqo-;
   break;
  @@ .
  patch -p0 '@@ .'
  Index: rpm/configure.ac
  
  $ cvs diff -u -r2.500 -r2.501 configure.ac
  --- rpm/configure.ac 6 Sep 2011 13:39:52 -   2.500
  +++ rpm/configure.ac 6 Sep 2011 14:41:17 -   2.501
  @@ -571,6 +571,7 @@
   AC_PATH_PROG(__ID, id, /usr/bin/id, $MYPATH)
   AC_PATH_PROG(__INSTALL, install, /usr/bin/install, $MYPATH)
   AC_PATH_PROG(__LRZIP, lrzip, /usr/bin/lrzip, $MYPATH)
  +AC_PATH_PROG(__LZIP, lzip, /usr/bin/lzip, $MYPATH)
   AC_PATH_PROG(__INSTALL_INFO, install-info, /sbin/install-info, $MYPATH)
   AC_PATH_PROG(__LDCONFIG, ldconfig, /sbin/ldconfig, $MYPATH)
   AC_PATH_PROG(__LN, ln, /bin/ln, $MYPATH)
  @@ .
  patch -p0 '@@ .'
  Index: rpm/macros/macros.in
  
  $ cvs diff -u -r1.60 -r1.61 macros.in
  --- rpm/macros/macros.in 6 Sep 2011 13:39:53 -   1.60
  +++ rpm/macros/macros.in 6 Sep 2011 14:41:18 -   1.61
  @@ -1,7 +1,7 @@
   #/*! \page config_macros Default configuration: @USRLIBRPM@/macros
   # \verbatim
   #
  -# $Id: macros.in,v 1.60 2011/09/06 13:39:53 devzero2000 Exp $
  +# $Id: macros.in,v 1.61 2011/09/06 14:41:18 devzero2000 Exp $
   #
   # This is a global RPM configuration file. All changes made here will
   # be lost when the rpm package is upgraded. Any per-system configuration
  @@ -91,6 +91,7 @@
   %__ldconfig @__LDCONFIG@
   %__ln   @__LN@
   %__ln_s @LN_S@
  +%__lzip @__LZIP@
   %__lrzip@__LRZIP@
   %__lzma @__LZMA@
   %__lzop @__LZOP@
  @@ .
  patch -p0 '@@ .'
  Index: rpm/rpmio/macro.c
  
  $ cvs diff -u -r2.252 -r2.253 macro.c
  --- rpm/rpmio/macro.c6 Sep 2011 13:39:53 -   2.252
  +++ rpm/rpmio/macro.c6 Sep 2011 14:41:18 -   2.253
  @@ -1356,6 +1356,9 @@
   case 7: /* COMPRESSED_LRZIP */
   sprintf(be, %%__lrzip -dqo- %s, b);
   break;
  +case 8: /* COMPRESSED_LZIP */
  +sprintf(be, %%__lzip -dc %s, b);
  +break;
   }
   b = be;
   } else if (STREQ(mkstemp, f, fn)) {
  @@ -3036,6 +3039,9 @@
   if (magic[0] == (unsigned char) 0xFD  magic[1] == 0x37  
 magic[2] == 0x7A
 magic[3] == 0x58  magic[4] == 0x5A  magic[5] == 

Re: [CVS] RPM: rpm/ CHANGES rpm/build/ parsePrep.c rpm/ configure.ac rpm/m...

[Jeff Johnson]

On Sep 6, 2011, at 11:05 AM, devzero2000 wrote:

 
 Done : lzip and lrzip support are in HEAD and in the 5_4 Branch. Tomorrow i 
 will
 search if there are some my patch that live only in HEAD and i will merge in 
 5_4 as well.
 Best Regards 
 

BTW, the buildbot master is now running at
http://harwich.jbj.org:8010
I can wire you up for access whenever.

Its also a single command to set up your own buildslave (and some
specific configuration on the buildbot master and in devtool.conf
to adjust to your specific platform and build options).

And rpm-5.4.2 was released quietly yesterday. rpm-5.4.3 might
be needed this week as well to start stabilizing linkages
with multiple versions of Berkeley DB external.

I will be targeting db-5.2.28 going forward with development.

hth

73 de Jeff

Re: [CVS] RPM: popt/ CHANGES popthelp.c system.h test1.c

[Jeff Johnson]

Please none of this format crap just to conform with oddball
platforms. There's literally no usage case for long long
in POPT: if you don't have long long and %lld, then error out the
routines or maintain the patch in MingW.

And there are like *THREE* different markups for UNUSED now,
none of which are very useful. Instead just do
x = x;
and then the variable is indeed used.




On Sep 5, 2011, at 7:26 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: popt Date:   05-Sep-2011 13:26:40
  Branch: HEAD Handle: 2011090511263900
 
  Modified files:
poptCHANGES popthelp.c system.h test1.c
 
  Log:
merge commit 8396019 from git pull request
https://github.com/devzero2000/POPT/pull/2
Work around missing format for long long in windows; fix warnings
by asenm (Matt Arsenault)
 
  Summary:
RevisionChanges Path
1.167   +4  -0  popt/CHANGES
1.104   +1  -1  popt/popthelp.c
1.24+22 -11 popt/system.h
1.53+2  -2  popt/test1.c
  
 
  patch -p0 '@@ .'
  Index: popt/CHANGES
  
  $ cvs diff -u -r1.166 -r1.167 CHANGES
  --- popt/CHANGES 5 Sep 2011 11:20:27 -   1.166
  +++ popt/CHANGES 5 Sep 2011 11:26:39 -   1.167
  @@ -1,4 +1,8 @@
   1.17 - 2.0:
  +- devzero2000: merge commit 8396019 from git pull request  
  +  https://github.com/devzero2000/POPT/pull/2 
  +  Work around missing format for long long in windows; fix warnings
  +  by asenm (Matt Arsenault)
   - devzero2000: merge  commit ea2978d from git pull request 
 https://github.com/devzero2000/POPT/pull/2 
 Fix build with MinGW 32/64 + MSVC by arsenm
  @@ .
  patch -p0 '@@ .'
  Index: popt/popthelp.c
  
  $ cvs diff -u -r1.103 -r1.104 popthelp.c
  --- popt/popthelp.c  16 Jun 2011 15:26:43 -  1.103
  +++ popt/popthelp.c  5 Sep 2011 11:26:39 -   1.104
  @@ -265,7 +265,7 @@
   le += sprintf(le, %ld, arg.longp[0]);
   break;
   case POPT_ARG_LONGLONG:
  -le += sprintf(le, %lld, arg.longlongp[0]);
  +le += sprintf(le, % LONG_LONG_FORMAT, arg.longlongp[0]);
   break;
   case POPT_ARG_FLOAT:
   {   double aDouble = (double) arg.floatp[0];
  @@ .
  patch -p0 '@@ .'
  Index: popt/system.h
  
  $ cvs diff -u -r1.23 -r1.24 system.h
  --- popt/system.h5 Sep 2011 11:20:27 -   1.23
  +++ popt/system.h5 Sep 2011 11:26:39 -   1.24
  @@ -7,7 +7,6 @@
   #endif
 
 
  -
   #if defined (__GLIBC__)  defined(__LCLINT__)
   /*@-declundef@*/
   /*@unchecked@*/
  @@ -54,10 +53,18 @@
   #include unistd.h
   #endif
 
  +
  +#if !defined(__GNUC__)  !defined(__attribute__)
  +#define __attribute__(x)
  +#endif
  +#define UNUSED(x) x __attribute__((__unused__))
  +
  +
   #ifdef _MSC_VER
   #  define inline __inline
   #endif
 
  +
   #if defined(_MSC_VER) || defined(__MINGW32__)
   #define _CRT_SECURE_NO_WARNINGS 1
   #include io.h
  @@ -73,6 +80,10 @@
   #define S_ISGID 0002
   #define S_ISVTX 0001
 
  +/* I haven't discovered a better way to work around these format
  +   specifier problems */
  +#define LONG_LONG_FORMAT I64d
  +
 
   /* CHECKME */
   #define S_IWGRP 0020
  @@ -112,17 +123,22 @@
   #define lseek _lseek
 
   /* Pretend to be root to replace these */
  -static inline int setuid(int x) { return 1; }
  +static inline int setuid(UNUSED(int x)) { return 1; }
 
   static inline int getuid(void) { return 0; }
 
  -static inline int seteuid(int x) { return 1; }
  +static inline int seteuid(UNUSED(int x)) { return 1; }
   static inline int geteuid(void) { return 0; }
 
  -static inline int setgid(int x) { return 1; }
  +static inline int setgid(UNUSED(int x)) { return 1; }
   static inline int getgid(void) { return 0; }
 
  -static inline int setegid(int x) { return 1; }
  +static inline int setegid(UNUSED(int x)) { return 1; }
  +
  +
  +#else
  +
  +#define LONG_LONG_FORMAT lld
 
   #endif /* defined(_MSC_VER) || defined(__MINGW32__) */
 
  @@ -186,12 +202,7 @@
   #define getenv(_s)  __secure_getenv(_s)
   #endif
 
  -#if !defined(__GNUC__)  !defined(__attribute__)
  -#define __attribute__(x)
  -#endif
  -#define UNUSED(x) x __attribute__((__unused__))
  -
   /* Include configmake.h autogenerated from Makefile.am */
  -
   #include configmake.h
   #include popt.h
  +
  @@ .
  patch -p0 '@@ .'
  Index: popt/test1.c
 

Re: [CVS] RPM: popt/ CHANGES popt.c system.h

[Jeff Johnson]

I'd rather see all the AutoCrap ripped out
then deal with dueling make processes.

POPT is insanely *by the book* portable and
doesn't need all this Have it your own way! stuff.

Particularly when the whole build process starts to diverge
into dueling builds.

73 de Jeff

On Sep 5, 2011, at 7:46 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: popt Date:   05-Sep-2011 13:46:28
  Branch: HEAD Handle: 2011090511462800
 
  Modified files:
poptCHANGES popt.c system.h
 
  Log:
devzero2000: merge commit fc8836e from git pull request
https://github.com/devzero2000/POPT/pull/2
Fix more MSVC build warnings
by asenm (Matt Arsenault). This is the last commit
of this patch series and pull request.
 
  Summary:
RevisionChanges Path
1.169   +5  -0  popt/CHANGES
1.160   +1  -0  popt/popt.c
1.25+11 -2  popt/system.h
  
 
  patch -p0 '@@ .'
  Index: popt/CHANGES
  
  $ cvs diff -u -r1.168 -r1.169 CHANGES
  --- popt/CHANGES 5 Sep 2011 11:35:53 -   1.168
  +++ popt/CHANGES 5 Sep 2011 11:46:28 -   1.169
  @@ -1,4 +1,9 @@
   1.17 - 2.0:
  +- devzero2000: merge commit fc8836e from git pull request  
  +  https://github.com/devzero2000/POPT/pull/2 
  +  Fix more MSVC build warnings
  +  by asenm (Matt Arsenault). This is the last commit
  +  of this patch series and pull request.
   - devzero2000: merge commit aee9f24 from git pull request  
 https://github.com/devzero2000/POPT/pull/2 
 Add makefile for mingw 
  @@ .
  patch -p0 '@@ .'
  Index: popt/popt.c
  
  $ cvs diff -u -r1.159 -r1.160 popt.c
  --- popt/popt.c  16 Jun 2011 15:26:43 -  1.159
  +++ popt/popt.c  5 Sep 2011 11:46:28 -   1.160
  @@ -31,6 +31,7 @@
   #define assert(_x)
   #endif
 
  +
   #ifdef  MYDEBUG
   /*@unchecked@*/
   int _popt_debug = 0;
  @@ .
  patch -p0 '@@ .'
  Index: popt/system.h
  
  $ cvs diff -u -r1.24 -r1.25 system.h
  --- popt/system.h5 Sep 2011 11:26:39 -   1.24
  +++ popt/system.h5 Sep 2011 11:46:28 -   1.25
  @@ -3,7 +3,15 @@
*/
 
   #ifdef HAVE_CONFIG_H
  -#include config.h
  +# ifndef _MSC_VER
  +#  include config.h
  +# else
  +#  include config_msvc.h
  +# endif /* _MSC_VER */
  +#endif /* HAVE_CONFIG_H */
  +
  +#if defined(_MSC_VER) || defined(__MINGW32__)
  +# define _CRT_SECURE_NO_WARNINGS 1
   #endif
 
 
  @@ -66,9 +74,9 @@
 
 
   #if defined(_MSC_VER) || defined(__MINGW32__)
  -#define _CRT_SECURE_NO_WARNINGS 1
   #include io.h
   #include malloc.h
  +#include process.h
   #define F_OK 0
   #define R_OK 4
   #define W_OK 2
  @@ -121,6 +129,7 @@
   #define open _open
   #define close _close
   #define lseek _lseek
  +#define execvp _execvp
 
   /* Pretend to be root to replace these */
   static inline int setuid(UNUSED(int x)) { return 1; }
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: popt/ CHANGES popthelp.c test1.c

[Jeff Johnson]

There's only a single (test1.c doesn't count) use for LONG_LONG_FORMAT:
$ grep LONG_LONG_FORMAT *.[ch]
popthelp.c: le += sprintf(le, % LONG_LONG_FORMAT, 
arg.longlongp[0]);
system.h:#define LONG_LONG_FORMAT I64d
system.h:#define LONG_LONG_FORMAT lld
test1.c:fprintf(stdout,  aLongLong: % LONG_LONG_FORMAT, 
aLongLong);

Just inline I64d … if you leave #define LONG_LONG_FORMAT in system.h,
someone will surely start to use in the future.

73 de Jeff




On Sep 5, 2011, at 8:14 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: popt Date:   05-Sep-2011 14:14:31
  Branch: HEAD Handle: 2011090512143000
 
  Modified files:
poptCHANGES popthelp.c test1.c
 
  Log:
redo commit 8396019
Useless to have format crap just for conforming
to oddball platform
(http://rpm5.org/community/rpm-devel/5166.html).
Keep the patch only inside MinGW.
 
  Summary:
RevisionChanges Path
1.170   +5  -0  popt/CHANGES
1.105   +4  -0  popt/popthelp.c
1.54+4  -0  popt/test1.c
  
 
  patch -p0 '@@ .'
  Index: popt/CHANGES
  
  $ cvs diff -u -r1.169 -r1.170 CHANGES
  --- popt/CHANGES 5 Sep 2011 11:46:28 -   1.169
  +++ popt/CHANGES 5 Sep 2011 12:14:30 -   1.170
  @@ -1,4 +1,9 @@
   1.17 - 2.0:
  +- devzero2000: redo commit 8396019 
  +  Useless to have format crap just for conforming
  +  to oddball platform 
  +  (http://rpm5.org/community/rpm-devel/5166.html).
  +  Keep the patch only inside MinGW.
   - devzero2000: merge commit fc8836e from git pull request  
 https://github.com/devzero2000/POPT/pull/2 
 Fix more MSVC build warnings
  @@ .
  patch -p0 '@@ .'
  Index: popt/popthelp.c
  
  $ cvs diff -u -r1.104 -r1.105 popthelp.c
  --- popt/popthelp.c  5 Sep 2011 11:26:39 -   1.104
  +++ popt/popthelp.c  5 Sep 2011 12:14:30 -   1.105
  @@ -265,7 +265,11 @@
   le += sprintf(le, %ld, arg.longp[0]);
   break;
   case POPT_ARG_LONGLONG:
  +#if defined(_MSC_VER) || defined(__MINGW32__)
   le += sprintf(le, % LONG_LONG_FORMAT, arg.longlongp[0]);
  +#else 
  +le += sprintf(le, %lld, arg.longlongp[0]);
  +#endif
   break;
   case POPT_ARG_FLOAT:
   {   double aDouble = (double) arg.floatp[0];
  @@ .
  patch -p0 '@@ .'
  Index: popt/test1.c
  
  $ cvs diff -u -r1.53 -r1.54 test1.c
  --- popt/test1.c 5 Sep 2011 11:26:39 -   1.53
  +++ popt/test1.c 5 Sep 2011 12:14:30 -   1.54
  @@ -334,7 +334,11 @@
   if (aLong != bLong)
   fprintf(stdout,  aLong: %ld, aLong);
   if (aLongLong != bLongLong)
  +#if defined(_MSC_VER) || defined(__MINGW32__)
   fprintf(stdout,  aLongLong: % LONG_LONG_FORMAT, aLongLong);
  +#else
  +fprintf(stdout,  aLongLong: %lld, aLongLong);
  +#endif
   /*@-realcompare@*/
   if (aFloat != bFloat)
   fprintf(stdout,  aFloat: %g, (double)aFloat);
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: popt/ CHANGES popt.c system.h

[Jeff Johnson]

On Sep 5, 2011, at 8:56 AM, devzero2000 wrote:

 On Mon, Sep 5, 2011 at 1:49 PM, Jeff Johnson n3...@mac.com wrote:
 I'd rather see all the AutoCrap ripped out
 then deal with dueling make processes.
 
 POPT is insanely *by the book* portable and
 doesn't need all this Have it your own way! stuff.
 Yes, ok. But there are some difference between platform anyway
 http://msdn.microsoft.com/en-us/library/8ef0s5kh%28v=vs.80%29.aspx
 Dunno if related but probably should be a good idea to try to introduce also 
 cmake as a possible
 build env for popt: better of a hardwired Makefile and better of autotool on 
 windows or mingw, apparently. 
 

I'm a minimalist: there's almost no reason why a simple Makefile
can't be done (except vendors are choosing to install popt
all over the place, with pkgconfig, and more …).

Its already been suggested that popt add gnu lib for portability here

http://rpm5.org/community/popt-devel/0122.html

Frigging FL/OSS advocacy suggesting a *huge* change to POPT
in order to use random(3) and glob(3) portably. POPT really doesn't
need any of long long, random(3) or glob(3): there simply isn't
a usage case.

I don't like cmake too much personally … *shrug* …

73 de Jeff

 Regards
 
 
 Particularly when the whole build process starts to diverge
 into dueling builds.
 
 73 de Jeff
 
 On Sep 5, 2011, at 7:46 AM, Pinto Elia wrote:
 
   RPM Package Manager, CVS Repository
   http://rpm5.org/cvs/
   
  
 
   Server: rpm5.org Name:   Pinto Elia
   Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
   Module: popt Date:   05-Sep-2011 13:46:28
   Branch: HEAD Handle: 2011090511462800
 
   Modified files:
 poptCHANGES popt.c system.h
 
   Log:
 devzero2000: merge commit fc8836e from git pull request
 https://github.com/devzero2000/POPT/pull/2
 Fix more MSVC build warnings
 by asenm (Matt Arsenault). This is the last commit
 of this patch series and pull request.
 
   Summary:
 RevisionChanges Path
 1.169   +5  -0  popt/CHANGES
 1.160   +1  -0  popt/popt.c
 1.25+11 -2  popt/system.h
   
  
 
   patch -p0 '@@ .'
   Index: popt/CHANGES
   
  
   $ cvs diff -u -r1.168 -r1.169 CHANGES
   --- popt/CHANGES 5 Sep 2011 11:35:53 -   1.168
   +++ popt/CHANGES 5 Sep 2011 11:46:28 -   1.169
   @@ -1,4 +1,9 @@
1.17 - 2.0:
   +- devzero2000: merge commit fc8836e from git pull request
   +  https://github.com/devzero2000/POPT/pull/2
   +  Fix more MSVC build warnings
   +  by asenm (Matt Arsenault). This is the last commit
   +  of this patch series and pull request.
- devzero2000: merge commit aee9f24 from git pull request
  https://github.com/devzero2000/POPT/pull/2
  Add makefile for mingw
   @@ .
   patch -p0 '@@ .'
   Index: popt/popt.c
   
  
   $ cvs diff -u -r1.159 -r1.160 popt.c
   --- popt/popt.c  16 Jun 2011 15:26:43 -  1.159
   +++ popt/popt.c  5 Sep 2011 11:46:28 -   1.160
   @@ -31,6 +31,7 @@
#define assert(_x)
#endif
 
   +
#ifdef  MYDEBUG
/*@unchecked@*/
int _popt_debug = 0;
   @@ .
   patch -p0 '@@ .'
   Index: popt/system.h
   
  
   $ cvs diff -u -r1.24 -r1.25 system.h
   --- popt/system.h5 Sep 2011 11:26:39 -   1.24
   +++ popt/system.h5 Sep 2011 11:46:28 -   1.25
   @@ -3,7 +3,15 @@
 */
 
#ifdef HAVE_CONFIG_H
   -#include config.h
   +# ifndef _MSC_VER
   +#  include config.h
   +# else
   +#  include config_msvc.h
   +# endif /* _MSC_VER */
   +#endif /* HAVE_CONFIG_H */
   +
   +#if defined(_MSC_VER) || defined(__MINGW32__)
   +# define _CRT_SECURE_NO_WARNINGS 1
#endif
 
 
   @@ -66,9 +74,9 @@
 
 
#if defined(_MSC_VER) || defined(__MINGW32__)
   -#define _CRT_SECURE_NO_WARNINGS 1
#include io.h
#include malloc.h
   +#include process.h
#define F_OK 0
#define R_OK 4
#define W_OK 2
   @@ -121,6 +129,7 @@
#define open _open
#define close _close
#define lseek _lseek
   +#define execvp _execvp
 
/* Pretend to be root to replace these */
static inline int setuid(UNUSED(int x)) { return 1; }
   @@ .
  __
  RPM Package Managerhttp://rpm5.org
  CVS Sources Repositoryrpm-...@rpm5.org
 
 __
 RPM Package Manager

Fwd: current status of ruby bindings in rpm5 (head) / latest stable ?

[Jeff Johnson]

Begin forwarded message:

 
 On Monday 22 August 2011, 12:57:35, Anders F Björklund wrote:
 Tobias Gerschner wrote:
 What's the state of ruby bindings in rpm5? The last discussion I 
 remember
 was centered around the absence of an usable API for ruby = 1.9.
 
 There are no ruby bindings in rpm5, only ruby embedding...
 (the old bindings were GPL, rather than LGPL like python)
 
 And Ruby upstream support for embedding is still very poor.
 
 Sorry for tuning in late. In current RPM-5.4 and RPM-HEAD there are Ruby 
 bindings which are not completeley defunct. Which means that one can fire up 
 a 
 Ruby 1.9 interpreter, do a require 'rpm' and e.g. parse and build a spec 
 file. That was my initial hacking goal.
 
 Yes, embedded Ruby is rather poor and a completeley different matter, for 
 many 
 reasons. One of it is that lack of MRI Ruby to cleanly cope with more than 
 one 
 interpreter instance at a time. Other reasons include the varity of Ruby 
 interpreters currently in existance. E.g., Rubinius offers a IMHO much 
 cleaner 
 API, but even if I chose to allow embedded ruby only with Rubyinius, what 
 about the bindings...?
 
 Please test the bindings code in current RPM-HEAD/-5.4.x. What's the actual 
 bug? What feature is missing?
 
 And it's also true that I never bothered to stringently persue Ruby bindings 
 with a well-defined ROADMAP etc. Sorry.
 
   Eric
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.9 (GNU/Linux)
 
 iQIcBAEBAgAGBQJOXWmnAAoJEIXD2TZ1epVBx2kQAJMvWzqkP/GYic2nMh/811vI
 c5iljvvRoU2b4eWxPSWWJoqpAhZx3n553FIicJZMdk47YPcXmxf94jZhpU4rQ8kr
 8do+18oW1HOQ9IO9nb4gyki3zDWHaP4F9vKfH09ltvdvHTO6s3nr4W11aXJi0DB+
 iEsxwfinES4frMBKQXom7ujMWxgzeotAE/ObFmC5GTtBc3Ndokofz1zoT+K6vR2o
 T9q7+V86KRUzA3XpTTx2uOloi2j42DWlSWR22esbeI4zB5lE+BaAox4wU0EeQcYW
 Vj0jGs8fA/1lQ7w1//Or3krs8ekhqvigbaTOgSNYCg4J7xQcmNgaajX8E1ILIbbo
 mnnw8bMUQDB97EUZ8iL9LoChXPfsUJcMOemATejZBBZmrjAZj6vvlG0reLiSTc7R
 WOV/la9+fOW0jWwd9t+X7fNDjDGzxX9UyQHq+7mRDP8ETh1v5WNUENItUkRV2fQP
 8RM+DK9t4PgfIjNZCr093qLWBy8uWcT/pGWnu2x3C+18qwzRrTC12334ucgxWO5b
 nmI/ChYZbM7lxNlfWN6CQAj3mibosHhSuudY9Amoi+ufa8n8WtFIVmTYw8fYtBbD
 eHBoRJqABhKKw6XCM2kGCY39jGmRVC8vbETUU9mQGaBpb05abGHdgejWCOFPMJKd
 AZF+yOmaPmHMtf+0tfl0
 =V/dC
 -END PGP SIGNATURE-

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm-5_3: rpm/ CHANGES rpm/build/ parseChangelog.c parseFile...

[Jeff Johnson]

Again we're working at cross purposes:

%clean was removed as part of spec file syntax in 2007.

And #ifdef RPM_VENDOR_MANDRIVA is going to be phased out …

*shrug*

73 de Jeff

On Aug 25, 2011, at 2:35 PM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   25-Aug-2011 20:35:20
  Branch: rpm-5_3  Handle: 2011082518351901
 
  Modified files:   (Branch: rpm-5_3)
rpm CHANGES
rpm/build   parseChangelog.c parseFiles.c
 
  Log:
really always invoke %clean at end.
 
  Summary:
RevisionChanges Path
1.3296.2.249+1  -0  rpm/CHANGES
2.50.4.2+0  -8  rpm/build/parseChangelog.c
2.36.4.3+0  -5  rpm/build/parseFiles.c
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3296.2.248 -r1.3296.2.249 CHANGES
  --- rpm/CHANGES  29 Jul 2011 04:27:49 -  1.3296.2.248
  +++ rpm/CHANGES  25 Aug 2011 18:35:19 -  1.3296.2.249
  @@ -1,4 +1,5 @@
   5.3.12 - 5.3.13
  +- proyvind: really always invoke %clean at end
   - proyvind: add sparc64v2 arch to %sparcx.
   - proyvind: try sanitize %sparc vs %sparcx inconsistencies, make %sparc
   all 32 bit sparc archs only, with %sparcx being all 32 bit archs AND
  @@ .
  patch -p0 '@@ .'
  Index: rpm/build/parseChangelog.c
  
  $ cvs diff -u -r2.50.4.1 -r2.50.4.2 parseChangelog.c
  --- rpm/build/parseChangelog.c   7 Jun 2011 16:34:10 -   2.50.4.1
  +++ rpm/build/parseChangelog.c   25 Aug 2011 18:35:20 -  2.50.4.2
  @@ -269,11 +269,7 @@
   /* There are no options to %changelog */
   if ((rc = readLine(spec, STRIP_COMMENTS))  0) {
   iob = rpmiobFree(iob);
  -#if defined(RPM_VENDOR_MANDRIVA)
  -return (spec-clean == NULL) ? PART_CLEAN : PART_NONE;
  -#else
   return PART_NONE;
  -#endif
   }
   if (rc != RPMRC_OK)
   return rc;
  @@ -285,11 +281,7 @@
   iob = rpmiobAppend(iob, spec-line, 0);
   line = _free(line);
   if ((rc = readLine(spec, STRIP_COMMENTS | STRIP_NOEXPAND))  0) {
  -#if defined(RPM_VENDOR_MANDRIVA)
  -nextPart = (spec-clean == NULL) ? PART_CLEAN : PART_NONE;
  -#else
   nextPart = PART_NONE;
  -#endif
   break;
   }
   if (rc != RPMRC_OK)
  @@ .
  patch -p0 '@@ .'
  Index: rpm/build/parseFiles.c
  
  $ cvs diff -u -r2.36.4.2 -r2.36.4.3 parseFiles.c
  --- rpm/build/parseFiles.c   25 Mar 2011 17:46:07 -  2.36.4.2
  +++ rpm/build/parseFiles.c   25 Aug 2011 18:35:20 -  2.36.4.3
  @@ -114,12 +114,7 @@
   }
   }
 
  -#if defined(RPM_VENDOR_MANDRIVA)
  -/* if no %clean section, add implicit */
  -rc = (nextPart == PART_NONE  spec-clean == NULL) ? PART_CLEAN : 
 nextPart;
  -#else
   rc = nextPart;
  -#endif
 
   exit:
   argv = _free(argv);
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: current status of ruby bindings in rpm5 (head) / latest stable ?

[Jeff Johnson]

On Aug 22, 2011, at 6:57 AM, Anders F Björklund wrote:

 Tobias Gerschner wrote:
 
 What's the state of ruby bindings in rpm5? The last discussion I remember 
 was centered around the absence of an usable API for ruby = 1.9.
 
 There are no ruby bindings in rpm5, only ruby embedding…

The difference between binding and embedding is largely moot.

There's a top-level executable that loads modules: the difference
is largely whether that executable is called /usr/bin/ruby or
/usr/bin/rpm or /usr/lib/rpm/bin/rpmruby.

 (the old bindings were GPL, rather than LGPL like python)
 

Yes: the *existing* (and largely orphaned/defunct afaik: the project
was seeking a new maintainer a couple years back and GPL instead
of LGPL prevented adoption by @rpm5.org) ruby bindings for rpm
are GPL. And there's no interest I know of (in fact the usual negative
interest in anything produced @rpm5.org because of Fork You! politics)
in the rather minor patch to use the existing ruby bindings with
@rpm5.org code.

 And Ruby upstream support for embedding is still very poor.
 

This isn't true. There are two issues here:
1) ruby 1.8.x != ruby 1.9.x incompatibilities
2) ruby-1.9.x embedding needs a separate persistent stack.

The support for embedding in ruby-1.8.x is fine: as good as any
other embedding (which usually isn't very good for any interpreted
language, perl/tcl/lua being the outstanding exceptions.)

The need in ruby-1.9.x is largely an architectural constraint.
What SHOULD (and did when I prototyped) work fine is to put the
ruby embedding on a separate thread with its own stack, and
use intra-thread communication to interpret ruby code at run-time.

But the architectures needed for embedding in ruby-1.8.x and ruby-1.9.x
are rather different, and Have it your own way! and the expectation
that ruby == ruby likely dooms the usefulness of pursuing ruby embedding
@rpm5.org.

But I definitely know what to do with both embeddings and bindings, and
with ruby 1.8.x and 1.9.x, and with rpm.org != rpm5.org API.

Lack of any significant development interest in rpm+ruby with a clearly
defined ROADMAP deliverable is all that stops doing whatever ...

 I'm very interested in seeing ruby-rpm bindings for rpm5 work for a project 
 I'm working on :  http://code.google.com/p/ubuild-linux/ .
 
 It would probably be easier to do this using python, those
 bindings are bad enough without suffering the non-standard ?
 

Heh: I pointed out a fairly significant flaw with
static FD_t fd;
in the python bindings yesterday. At some point its easier
to start from scratch in ruby than it its to pretend that
the rpm-python bindings are useful.

 The only project I know about using ruby-rpm is enhancerepo.
 

Yes: Duncan McVicar (and OpenSuSE) are most of the active interest
in rpm-ruby afaik. There *is* a great deal of interest in rpm-ruby
in Japan, but that is usually off my radar (but there is other interest
in rpm-ruby).

 I'm currently using rpm-5.2.xx and for all I know rpm-ruby bindings are 
 currently not functional.
 
 The patches for rpm5 were unfortunately rejected upstream.
 So only rpm.org is supported, similar to apt-rpm and yum.
 
 http://rubyforge.org/projects/ruby-rpm/
 

And I can fork ruby-rpm just like I've forked yum/createrepo @rpm5.org in
order to deliver a patch to use the existing ruby-rpm bindings with @rpm5.org
code whenever there is need or interest.

GPL != LGPL stops one and only one thing:
RPM from @rpm5.org is LGPL: no GPL code can be included
in rpm-X.Y.Z.tar.gz when released.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: current status of ruby bindings in rpm5 (head) / latest stable ?

[Jeff Johnson]

On Aug 22, 2011, at 10:11 AM, Anders F Björklund wrote:

 Jeff Johnson wrote:
 
 (the old bindings were GPL, rather than LGPL like python)
 
 
 Yes: the *existing* (and largely orphaned/defunct afaik: the project
 was seeking a new maintainer a couple years back and GPL instead
 of LGPL prevented adoption by @rpm5.org) ruby bindings for rpm
 are GPL. And there's no interest I know of (in fact the usual negative
 interest in anything produced @rpm5.org because of Fork You! politics)
 in the rather minor patch to use the existing ruby bindings with
 @rpm5.org code.
 
 Right, the current ruby-rpm doesn't even compile (missing header):
 https://gitorious.org/ruby-rpm/ruby-rpm/merge_requests/1 # st.h
 

Compile where? Without details, well, its all an open sores
random walk any way you look at the issue ...

 And Ruby upstream support for embedding is still very poor.
 
 
 This isn't true. There are two issues here:
  1) ruby 1.8.x != ruby 1.9.x incompatibilities
  2) ruby-1.9.x embedding needs a separate persistent stack.
 
 The support for embedding in ruby-1.8.x is fine: as good as any
 other embedding (which usually isn't very good for any interpreted
 language, perl/tcl/lua being the outstanding exceptions.)
 
 Anything including files like config.h and setting things
 like PACKAGE_NAME isn't really expecting to be embedded imho.
 

Yup. But upstream doesn't see this as an issue, so the
only remaining choice for patch delivery is
Fork ruby!
Forking is a helluva way to apply a patch … *shurg* whatever …

(aside)
I could likely fix the ruby-1.8.x - ruby-1.9.x quite simply
at the same time: all its takes is a helper thread for main()
that uses a different stack, a subtle but very not hard hack.

 But I definitely know what to do with both embeddings and bindings, and
 with ruby 1.8.x and 1.9.x, and with rpm.org != rpm5.org API.
 
 Lack of any significant development interest in rpm+ruby with a clearly
 defined ROADMAP deliverable is all that stops doing whatever ...
 
 The bindings should be fixable too, if needed. Were fixed for
 4.4.6, 4.4.8 and 5.0.0 already - could be done again for 5.2.x.
 
 I'm very interested in seeing ruby-rpm bindings for rpm5 work for a 
 project I'm working on :  http://code.google.com/p/ubuild-linux/ .
 
 It would probably be easier to do this using python, those
 bindings are bad enough without suffering the non-standard ?
 
 
 Heh: I pointed out a fairly significant flaw with
  static FD_t fd;
 in the python bindings yesterday. At some point its easier
 to start from scratch in ruby than it its to pretend that
 the rpm-python bindings are useful.
 
 True enough, just meant one would get some company being miserable ?

Ah yes, the social aspects of the lemming community herd …

 Thought all the bindings were in need of starting over from rpm lib.

As of August 12th, not only rpmbuild, but also all the bindings
are in the bit bucket imho. What that actually means is an open question ...

That shouldn't surprise anyone: any code that can't ever be meaningfully
fixed eventually rots. And there's lots of mileage on RPM code ...

 The patches for rpm5 were unfortunately rejected upstream.
 So only rpm.org is supported, similar to apt-rpm and yum.
 
 http://rubyforge.org/projects/ruby-rpm/
 
 
 And I can fork ruby-rpm just like I've forked yum/createrepo @rpm5.org in
 order to deliver a patch to use the existing ruby-rpm bindings with @rpm5.org
 code whenever there is need or interest.
 
 I have the old patches here somewhere, from a couple of years ago.
 Will push those to the repository, along with some darwin fixes.
 

Noted. Without a ROADMAP deliverable, its not clear where to
place the patch … short of forking ruby-rpm @rpm5.org, which is trivially
arranged any time there is interest …

 GPL != LGPL stops one and only one thing:
  RPM from @rpm5.org is LGPL: no GPL code can be included
  in rpm-X.Y.Z.tar.gz when released.
 
 Right, so it can't be included like the python and the perl…

… which likely won't be included in the rpm tar ball going forward …

 Then again, just because they can doesn't mean that they should.
 

Yup: the only reason for including python/* and perl/* and js/*
and ruby/* and tcl/* is to provide one-stop-shopping.

But there's so little interest in FL/OSS development that its time to
let the code float away … with embeddings rpm can target development
issues for all development language problems, not just whatever feebleness to do
rpm -qa
in Newer! Better! Bestest! ways has been devised in that particular scriptie 
dialect.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: current status of ruby bindings in rpm5 (head) / latest stable ?

[Jeff Johnson]

On Aug 22, 2011, at 12:07 PM, Anders F Björklund wrote:

 
 Yup: the only reason for including python/* and perl/* and js/*
 and ruby/* and tcl/* is to provide one-stop-shopping.
 
 The js/* is still the preferred/mandatory scripting, yes ?

Its preferred by me still yes. And with js-1.8.5 its
feasible to make it MANDATORY (though MANDATORY sounds like
I'm a dictator, when its really the reverse:
rpm cannot use any functionality that isn't
GUARANTEED to be present

 (with it being included in MongoDB and everything, that is)
 
 But there's so little interest in FL/OSS development that its time to
 let the code float away … with embeddings rpm can target development
 issues for all development language problems, not just whatever feebleness 
 to do
  rpm -qa
 in Newer! Better! Bestest! ways has been devised in that particular scriptie 
 dialect.
 
 A shared rpmlib binding might have worked - for a stable API…

Its not just the stable API: each and every binding has its
own special methods abstractions on same old same old.

 Then each script could canonicalize on top of the C functions.
 

Well there is SWIG that can be attempted off RPM converts to C++.

SWIG benefits from sub-classing is the reason: and all rpm objects
are essentially derived from a base class now that muteness are reliably
attached everywhere. Just its not C++! C++! C++! yet.

 Noting that the ruby-rpm links directly to popt and db. Sheesh.
 

In addition to -lrpmdb? Or instead of -lrpmdb? The second actually
makes some sense, and using /var/lib/rpm/DB_CONFIG to set tunables
makes it (almost, headerLoad()/headerGet() are still needed) almost
feasible to read an rpmdb without using RPM at all.

That is/was the intent choosing Berkeley DB ages ago ...

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm-5_3: rpm/rpmio/ rpmsq.c

[Jeff Johnson]

On Aug 6, 2011, at 4:13 PM, pinto.e...@gmail.com wrote:

 I am Sorry jbj, but perhaps could be better to reference the exact source of 
 the Bug number. Rpm5 is used by many distro these days, isn't it ? Best 
 regards 

Well only Mandriva has bugs in the ~60K range … but you are correct, I will do 
mdv#62262
in the future.

hth

73 de Jeff
 Messaggio originale
 Da: Jeff Johnson
 Inviato:  06/08/2011, 21:58 
 A: rpm-...@rpm5.org
 Oggetto: [CVS] RPM: rpm-5_3: rpm/rpmio/ rpmsq.c
 
 
  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Jeff Johnson
  Root:   /v/rpm/cvs   Email:  j...@rpm5.org
  Module: rpm  Date:   06-Aug-2011 21:58:59
  Branch: rpm-5_3  Handle: 2011080619585900
 
  Modified files:   (Branch: rpm-5_3)
rpm/rpmio   rpmsq.c
 
  Log:
- fix: backport gnome-terminal fix from head/rpm-5_4 (#62262).
 
  Summary:
RevisionChanges Path
1.42.4.1+1  -1  rpm/rpmio/rpmsq.c
  
 
  patch -p0 '@@ .'
  Index: rpm/rpmio/rpmsq.c
  
  $ cvs diff -u -r1.42 -r1.42.4.1 rpmsq.c
  --- rpm/rpmio/rpmsq.c7 Mar 2009 13:38:10 -   1.42
  +++ rpm/rpmio/rpmsq.c6 Aug 2011 19:58:59 -   1.42.4.1
  @@ -443,7 +443,7 @@
   int tblsignum = (signum = 0 ? signum : -signum);
   struct sigaction sa;
   rpmsig tbl;
  -int ret = -1;
  +int ret = (signum = 0 ? 1 : 0);
   int xx;
 
   xx = DO_LOCK ();
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: building rpm 5.4.1

[Jeff Johnson]

On Jul 31, 2011, at 2:58 PM, Robert Xu wrote:

 Hi all,
 
 So I finally got around to building rpm 5.4.1 for SuSE.
 I still have the patches (reformatted), but I am not using them for this 
 build.
 
 I'm getting this error: http://slexy.org/view/s2XO5ULWb4
 

The _globalI issue (assuming I have the correct 2180 line number)
is here:
#ifdef  WITH_SEMANAGE
...
rpmsm sm = (_globalI ? NULL
: rpmsmNew(_rpmsmStore, _rpmsmFlags));
so adding --without-semanage (or similar: check config.h
after configuring to see if WITH_SEMANAGE is defined; you can always just change
config.h after running configure if it gets in yer way).

The other error is innocuous and has to do with my C fetishism,
noting else:
xx = foo(…)
is a coding style I practice to note a return code that might need
checking in the future; having the return code assigned
to a value occasionally helps when using gdb or interpreting
core dumps (but I haven't had to do that in a long time, just
an ancient fetishism in my coding practices).

 RPM spec file (right now): http://slexy.org/view/s2tQBYc9i4
 

Checking … yep here's yer issue:
./configure --verbose --prefix=%{_prefix} \
…
--with-semanage \

Disabling semanage costs little: there are JS bindings that benefit,
and there is the Spook DSL (domain specific language) that I
will finish up for Halloween announcement if the Tresys security
trolls ever really get serious about distributing modular SELinux policy
in *.rpm packages. None of that is anywhere visible:

SHort answer:
You won't miss a thing if you change your rpm.spec to do
--without-semanage
instead.

hth

73 de Jeff
 
 
 -- 
 later, Robert Xu + rxu AT lincomlinux DOT org
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: building rpm 5.4.1

[Jeff Johnson]

On Jul 31, 2011, at 4:40 PM, Robert Xu wrote:

 
 SHort answer:
You won't miss a thing if you change your rpm.spec to do
--without-semanage
instead.
 
 
 Got further after this :)
 (I don't even know what SuSE uses semanage in RPM for anyway...)
 
 
 I've ended up here next...
 http://slexy.org/view/s2f7L0QS0T
 

This is the problem that everyone is encountering:
Berkeley DB is both actually and de facto MANDATORY to build rpm.

The issue is that
/usr/include/db51/db.h
and
/usr/lib*/libdb-5.1.so

MUST exist to build RPM from a tarball easily. Everyone
is expecting RPM to just transparently deal with whatever
Berkeley DB is installed and that is NOT what is implemented
at the moment.

Check your ./configure output (or generated config.h) to confirm whether
Berkeley DB was found … checking …

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I. -I.. -I../build -I../lib 
-I../lib -I../rpmdb -I../rpmio -I../misc -I../beecrypt/include 
-I../beecrypt/include -I../beecrypt -I../beecrypt -I../lua/local -I../lua/local 
-I../lua -I../lua -I../db -I../db -I../db/src -I../db/src -I../scripts 
-I../scripts -DRPM_OS_LINUX=03 -I/usr/local/include -DRPM_OS_LINUX=03 
-I/usr/local/include -fopenmp -fomit-frame-pointer -fmessage-length=0 -O2 -Wall 
-D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables 
-fasynchronous-unwind-tables -g -fno-strict-aliasing -ffunction-sections -g 
-D_GNU_SOURCE -D_REENTRANT -MT librpmdb_la-dbconfig.lo -MD -MP -MF 
.deps/librpmdb_la-dbconfig.Tpo -c dbconfig.c  -fPIC -DPIC -o 
.libs/librpmdb_la-dbconfig.o

I can tell from flags like
-I../db
that the RPM configuration has defaulted to internal and Berkeley DB
is no longer in the rpm-5.4.1 tarball.

The easiest path to fix your build failure is just to install db51 on SuSE. You 
can
find what I use by portably (translation: its not drop-in
ready, but the db51.spec can easily be adjusted to build db-5.1.25):

cvs -d :pserver:anonym...@rpm5.org:/v/rpm/cvs get distro/db51

You will need to add the db-5.1.25.tar.gz from Oracle and some copy
of db-1.85 from most any distro and then build/install a db51*.rpm.

All that's there is a Fedorable based db51.spec that does NOT install
/usr/include/db.h
and whatever other goofiness that system Berekely DB does. So the
db51 recipe SHOULD drop-in most everywhere whit no fuss or muss.

RPM doesn't NEED db-5.1.25, any version back to db-4.5.x can be used
either externally/internally (see archives here for ways to do that, or ask).

But there's so many versions of Berkeley DB around that I've
deliberately chosen db-5.1.25 in order to make my rpm release QA as simple
as possible by targeting one single version of Berkeley DB.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: building rpm 5.4.1

[Jeff Johnson]

On Jul 31, 2011, at 7:42 PM, Robert Xu wrote:

 
 that's... not supposed to happen, right?
 

Yes.

db3 is just a build directory with a configure wrapper (i.e. its not
an AutoFu ./configure just a stub to filter augments and reinvade
../db/dist/configure

Appears I nuked db3 (and db3/configure) back in rpm-5.3.4.

Appended is last known used configure. Do
mkdir db3
cp appended db3/configure

for the internal flags I just gave you.

Totally untested but what was working when I last did --with-db=internal.

hth

73 de Jeff


configure
Description: Binary data

Re: building rpm 5.4.1

[Jeff Johnson]

On Jul 31, 2011, at 7:43 PM, Robert Xu wrote:

 On Sun, Jul 31, 2011 at 19:42, Robert Xu rob...@gmail.com wrote:
 
 ++ searching location: internal
 -- skipping not existing local sub-directory: db3
 configure: error: unable to find internal Berkeley-DB library
 error: Bad exit status from /var/tmp/rpm-tmp.xXGIng (%build)
 
 that's... not supposed to happen, right?
 
 
 (what I mean by that is why is it looking for a db3 directory instead
 of a 'db' directory?)
 

Hysterical reasons mostly … the directory in which
to build Berkeley DB could be called anything.

The name db3 was chosen at the time because the Berkeley
DB in use was db-3.x.y.

If/when using cvs, one tries not to rename directories
because some manual scripting is almost certainly needed
to handle directory renaming issues.

SHort answer:
Easier to live with existing than to fix.

The name on the directory simply doesn't matter.

hth

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: building rpm 5.4.1

[Jeff Johnson]

On Jul 31, 2011, at 8:35 PM, Robert Xu wrote:

 
 
 Success!
 

Good.

 ++ searching location: internal
 -- skipping not existing local sub-directory: db3/sql
 configure: error: unable to find internal Berkeley-DB (+SQLite3) library
 error: Bad exit status from /var/tmp/rpm-tmp.7lmaLM (%build)
 

Ah sorry:

from memory this is
mkdir -p db3/sql
to fix

 ...
 
 (ps. I've never used CVS... so I'll take your word on its atrocities :P)
 

Java is an atrocity … cvs is just mildly senile … works fine if speak slowly …

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: building rpm 5.4.1

[Jeff Johnson]

On Jul 31, 2011, at 9:30 PM, Robert Xu wrote:

 On Sun, Jul 31, 2011 at 20:51, Jeff Johnson n3...@mac.com wrote:
 
 
 Ah sorry:
 
 from memory this is
mkdir -p db3/sql
 to fix
 
 
 works! thanks
 
 Java is an atrocity … cvs is just mildly senile … works fine if speak slowly 
 …
 
 
 hehe, I know. JRE is too slow, it's made by Oracle now (they shipped
 Java 7 with a bug that practically breaks the whole thing), and they
 teach it in my high school. XD
 
 Now I'm up to here...
 
 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I. -I.. -I../build
 -I../lib -I../lib -I../rpmdb -I../rpmio -I../misc
 -I../beecrypt/include -I../beecrypt/include -I../beecrypt
 -I../beecrypt -I../db3/sql -I../db3/sql -I../lua/local -I../lua/local
 -I../lua -I../lua -I../neon/src -I../neon/src -I../neon -I../neon
 -I../syck/lib -I../syck/lib -I../syck -I../syck -DRPM_OS_LINUX=03
 -I/usr/local/include -fopenmp -fomit-frame-pointer -fmessage-length=0
 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables
 -fasynchronous-unwind-tables -g -fno-strict-aliasing
 -ffunction-sections -g -D_GNU_SOURCE -D_REENTRANT -MT rpmsql.lo -MD
 -MP -MF .deps/rpmsql.Tpo -c rpmsql.c  -fPIC -DPIC -o .libs/rpmsql.o
 rpmsql.c:13:24: fatal error: db51/dbsql.h: No such file or directory

A symlink would fix that … so would a patch: dbsql.h is under db3 somewhere.

--without-dbsql makes the pain go away too.

Isn't Have it your own way! funner? I have indeed changed rpm to vote with
the majority opinion of external system Berkeley DB at some considerable
cost to my own sanity, as I'm doomed to support this complexity I don't believe 
in much.

73 de Jeff

 compilation terminated.
 make[4]: *** [rpmsql.lo] Error 1
 make[4]: Leaving directory `/usr/src/packages/BUILD/rpm-5.4.1/rpmio'
 make[3]: *** [all-recursive] Error 1
 make[3]: Leaving directory `/usr/src/packages/BUILD/rpm-5.4.1/rpmio'
 make[2]: *** [all] Error 2
 make[2]: Leaving directory `/usr/src/packages/BUILD/rpm-5.4.1/rpmio'
 make[1]: *** [all-recursive] Error 1
 make[1]: Leaving directory `/usr/src/packages/BUILD/rpm-5.4.1'
 make: *** [all] Error 2
 error: Bad exit status from /var/tmp/rpm-tmp.nuPM1T (%build)
 
 
 
 -- 
 later, Robert Xu + rxu AT lincomlinux DOT org
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: building rpm 5.4.1

[Jeff Johnson]

On Jul 31, 2011, at 10:05 PM, Robert Xu wrote:

 On Sun, Jul 31, 2011 at 21:54, Jeff Johnson n3...@mac.com wrote:
 
 A symlink would fix that … so would a patch: dbsql.h is under db3 somewhere.
 
 hm... I have db3/dbsql, but not db3/dbsql.h…
 

cd db3
make -n install  /tmp/foo
and look for dbsql.h. It should be there but is perhaps called
sqlite3.h in tree or something. It truly is sqlite3 in db-5.1.25.

 
 --without-dbsql makes the pain go away too.
 
 I wish, but I'll try to stick to SuSE's config params as long as possible
 so I don't have more pain later.
 

OK. Lemme take a look tomorrow …

 
 Isn't Have it your own way! funner? I have indeed changed rpm to vote with
 the majority opinion of external system Berkeley DB at some considerable
 cost to my own sanity, as I'm doomed to support this complexity I don't 
 believe in much.
 
 
 sure, of course it's better! /sarcasm
 

Seriously: Ite way too complicated to build rpm, gotta be fixed somehow.

73 de Jeff
 
 -- 
 later, Robert Xu + rxu AT lincomlinux DOT org
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: What should the contents of /etc/rpm/platform be ?

[Jeff Johnson]

On Jul 23, 2011, at 3:13 PM, Sriram Narayanan wrote:

 Thanks for your responses, everyone.
 

BTW here's a response to your smart-list question re arch scoring:

 Yes, I get this very output, but I don't know what it means. I need to
 study rpm internals and read the docs better!

The platform affinity patterns (See Tim Mooney's example) are used also to
set the archscore.

The arch score was original devised as a simple metric to, say,
Prefer *.i586.rpm packages on an i586 architecture.
when given all the usual *.{i386,i486,i586,i686}.rpm package choices.

The patterns in /etc/rpm/platform (except for the 1st line which is a string)
are used to determine the arch score, which is literally nothing more
that the line number in /etc/rpm/platform of the pattern that matches.

The arch score is just an integer with lower is better, the value has
always been arbitrary.

The preference metric implemented in arch score has always been a bit clunky.
Mostly its a design flaw:
Preference is multi-dimensional and cannot be reduced to a single 
integer.

(aside)
For the picky pedantics: almost anything can be mapped onto integers somehow,
but in the real world, preference has many dimensions and one can often end up
with conflicting preferences. Consider
I love pistachio ice cream.
I hate food that makes me fat.
and I defy you to propose a metric solution that leads to me eating chocolate
ice cream because that's all that is available. ;-)

I hope that helps to answer what arch score is about. Almost all issues can
be ignored as long as you are using only no arch and i86pc packages.
You (or rather smart) will attempt to use arch score to choose the right
kernel package, or perhaps some crypto package optimized for an architecture 
like
openssl, but otherwise arch score (and the patterns in /etc/rpm/platform) can
largely be ignored.

 I had to place i86pc-pc-Solaris in /etc/rpm/platform (I'm using a 32
 bit build for the time being).
 
 I also had to place /etc/rpm/macros and customize the following:
 %_arch  i86pc

The %_arch macro is/was used to set the canonical arch, like i386 on an 
i686 cpu.
Mostly useless these days (but there's still some distros that use, but
not for what it was intended for).

 %_build_arch  i86pc
 %_vendor   pc
 %_os Solaris
 

All of the above settings should be consistent with the CPU-VENDOR-OS-GNU
string (i.e. the first line in /etc/rpm/platform). The main reason why
Anders suggested that you use that file is so you can control for
certain values that if NOT specified, end up coming from uname(2)
with a translation mapping implemented in C whose values cannot be relied on,
particularly on non-linux platforms.

 With this, I now have the smart package manager working.
 

Congrats!

73 de Jeff

 -- Sriram
 
 On Sat, Jul 23, 2011 at 1:44 AM, Jeff Johnson n3...@mac.com wrote:
 
 On Jul 22, 2011, at 3:59 PM, Tim Mooney wrote:
 
 In regard to: What should the contents of /etc/rpm/platform be ?, Sriram...:
 
 sriram@ram-oi:/usr/src/rpm/RPMS/i86pc$ uname -a
 SunOS ram-oi 5.11 oi_147 i86pc i386 i86pc Solaris
 
 sriram@ram-oi:/usr/src/rpm/RPMS/i86pc$ ls /usr/src/rpm/RPMS/i86pc/
 autoconf-2.68-1.i86pc.rpm  cmake-2.8.5-1.i86pc.rpm  nginx-1.0.4-1.i86pc.rpm
 axel-2.4-1.i86pc.rpmgit-1.7.6-1.i86pc.rpmrepodata
 
 I build some packages already. (I now install from rpm packages only).
 
 rpm --eval %{_arch}-%{_vendor}-%{_os}%{?_gnu}
 i386-pc-solaris
 
 On my x86_64-sun-solaris2.10 workstation I have
 
 02:46 PM dogbert ~$cat /etc/rpm/platform x86_64-sun-solaris
 x86_64-sun-solaris2.10
 i686-sun-solaris
 i686-*-solaris
 i686-sun-solaris2.10
 x86_64-[^-]*-solaris.*
 noarch-[^-]*-.*
 [^-]*-[^-]*-noos
 
 
 Oooh nice! a worked example of platform affinity! Thank you!
 
 I always skip that explanation initially.
 
 BTW, there's negative affinity in /etc/rpm/platform patterns too if you 
 prefix a pattern with !
 it inverts the sense of the match.
 
 hth
 
 73 de Jeff
 Not all of those are relevant (or probably even valid...), but you get
 the idea.
 
 You may also want to consider overriding %_arch, %_vendor, and potentially
 other platform-related macros in your host-specific RPM macros (probably
 /etc/rpm/macros).
 
 Tim
 --
 Tim Mooney tim.moo...@ndsu.edu
 Enterprise Computing  Infrastructure  701-231-1076 (Voice)
 Room 242-J6, IACC Building 701-231-8541 (Fax)
 North Dakota State University, Fargo, ND 58105-5164
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm

Re: What should the contents of /etc/rpm/platform be ?

[Jeff Johnson]

On Jul 22, 2011, at 2:38 PM, Sriram Narayanan wrote:

 Hi:
 
 I'm facing a problem with the smart package manager downloading the
 createrepo based metadata, but not showing me any packages.
 
 Anders F Bjorklund on the smart package manager mailing list gave a
 few pointers, most of which I'd already applied.
 
 He also pointed out to me that I should have /etc/rpm/platform

Yes. You need something like the follow:ng 1-liner in /etc/rpm/platform
ix86-belenix-linux
You can add a 4th -gnu field to taste (RPM doesn't care what you wish to 
advocate)

Note that your choice of ix86, while perfectly okay, is a different
arch name than what most distros are using. You might run into some obscure
difficulties down the road if you choose ix86 instead of i586 or i686
which is more typical for linux distros using smart/yum/zypper/whatever.
 
 I don't have this with the make install that I carried out. What
 should the contents of this file be ?
 

There's too many variant names for arch like ix86 to
attempt automatically populating /etc/rpm/platform with
make install

On @redhat based systems using anaconda, its anaconda that
populates /etc/rpm/platform based off which kernel is chosen.
Anaconda gets /etc/rpm/platform wrong in many cases, but having
the installer populate /etc/rpm/platform based on what kernel
is chosen to boot is the right approach because that is the most
specific de facto info available.

 Here are various inputs from my side (sorry, it's a bit late for me
 and in my sleep I'm not able to think of anything much).
 
 sriram@ram-oi:/usr/src/rpm/RPMS/i86pc$ uname -a
 SunOS ram-oi 5.11 oi_147 i86pc i386 i86pc Solaris
 
 sriram@ram-oi:/usr/src/rpm/RPMS/i86pc$ ls /usr/src/rpm/RPMS/i86pc/
 autoconf-2.68-1.i86pc.rpm  cmake-2.8.5-1.i86pc.rpm  nginx-1.0.4-1.i86pc.rpm
 axel-2.4-1.i86pc.rpm git-1.7.6-1.i86pc.rpmrepodata
 
 I build some packages already. (I now install from rpm packages only).
 
 rpm --eval %{_arch}-%{_vendor}-%{_os}%{?_gnu}
 i386-pc-solaris
 

This string will work but the vendor field will be pc. Almost
nothing depends on the vendor field.

You can try various combinations and see where the values propagate with
rpm -v --showrc
Note that the -v MUST precede --showrc because --showrc exits from
CLI processing immediately: any further options/args will be ignored.

 $ python
 import rpm
 rpm.archscore(i86pc)
 0
 
 I placed i386-pc-solaris in /etc/rpm/platform, but that did not help.
 

Choose a CPU-VENDOR-OS-GNU string and put that in /etc/rpm/platform.

One easy way to guess that string conventionally is to run
./config.guess

hth

73 de Jeff
 Sriram
 -- 
 Belenix: www.belenix.org
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: What should the contents of /etc/rpm/platform be ?

[Jeff Johnson]

On Jul 22, 2011, at 3:59 PM, Tim Mooney wrote:

 In regard to: What should the contents of /etc/rpm/platform be ?, Sriram...:
 
 sriram@ram-oi:/usr/src/rpm/RPMS/i86pc$ uname -a
 SunOS ram-oi 5.11 oi_147 i86pc i386 i86pc Solaris
 
 sriram@ram-oi:/usr/src/rpm/RPMS/i86pc$ ls /usr/src/rpm/RPMS/i86pc/
 autoconf-2.68-1.i86pc.rpm  cmake-2.8.5-1.i86pc.rpm  nginx-1.0.4-1.i86pc.rpm
 axel-2.4-1.i86pc.rpmgit-1.7.6-1.i86pc.rpmrepodata
 
 I build some packages already. (I now install from rpm packages only).
 
 rpm --eval %{_arch}-%{_vendor}-%{_os}%{?_gnu}
 i386-pc-solaris
 
 On my x86_64-sun-solaris2.10 workstation I have
 
 02:46 PM dogbert ~$cat /etc/rpm/platform x86_64-sun-solaris
 x86_64-sun-solaris2.10
 i686-sun-solaris
 i686-*-solaris
 i686-sun-solaris2.10
 x86_64-[^-]*-solaris.*
 noarch-[^-]*-.*
 [^-]*-[^-]*-noos
 

Oooh nice! a worked example of platform affinity! Thank you!

I always skip that explanation initially.

BTW, there's negative affinity in /etc/rpm/platform patterns too if you prefix 
a pattern with !
it inverts the sense of the match.

hth

73 de Jeff
 Not all of those are relevant (or probably even valid...), but you get
 the idea.
 
 You may also want to consider overriding %_arch, %_vendor, and potentially
 other platform-related macros in your host-specific RPM macros (probably
 /etc/rpm/macros).
 
 Tim
 -- 
 Tim Mooney tim.moo...@ndsu.edu
 Enterprise Computing  Infrastructure  701-231-1076 (Voice)
 Room 242-J6, IACC Building 701-231-8541 (Fax)
 North Dakota State University, Fargo, ND 58105-5164
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: About Have it your own way!

[Jeff Johnson]

On Jul 16, 2011, at 8:47 AM, Sriram Narayanan wrote:

 Hi Jeff, 
 
 I've sometimes seen your responses posts where you talk about  Have it your 
 won way!.
 

There are many deep meanings of my use of that phrase that I'm
not going to be able to describe.

But some of the meanings you know quite well:

1) rpm is very very hard to build: there are dozens of options and
the options aren't simple switches and don't behave as
users expect. The options  never will behave properly because proper QA is 
impossible
because of combinatorial failure(s) of build options and platforms and versions 
and life is short.

2) In order to attempt to unify rpm development and functionality,
deviations -- even entirely unacceptable deviations with clear
and well described problems and issues -- were permitted @rpm5.org.

The problem there is that development is impossible when
vendors are just retrofitting whatever they want , usually
from ancient (5+ year old) patches retrofitted back into rpm
when the functionality has already been solved in other
(and imho better) ways.

So RPM development is all about legacy compatibility, where
issues from years ago don't get solved or improved but rather
get retrofitted again and again and again.

The personal problem I have with both 1) and 2) is this:

Who maintains the complexity of building and incompatible deviations?

That complexity and divergence can only be supported with patches
and usage.

What is happening (for years now) is that *I* am expected to develop
transparently: i.e. all possible combinations of previous versions
of RPM MUST work exactly as ever. In essence, nothing in RPM can
ever really be meaningfully changed.

Telling people what they want to hear is less effort than attempting to explain
other or better alternatives. And so I end up describing implementations that
I know are wrong/deficient and that I don't personally believe in or have 
already
implemented a better solution imho.

 Are you referring to distros wanting to be able to choose whether they will 
 provide rpm5 with its own copy of a library, or wish to force it to use the 
 distro-provided library ?
 

Presumably you are referring to external - internal Berkeley DB here …

… that is one (of many) examples of where what I recommended and chose
in RPM is continuously ignored everywhere. What I chose is what was
recommended by Sleepycat: go argue with Oracle, not me, if you have
some issue with what I just said.

When the quite predictable rpmdb issues occur, with building, or with 
incompatibilities
that are very well known with Berkeley DB (Berkeley DB is backward but not
forward compatible because time machines don't exist) occur, I am expected
to solve the issue, in public, usually with hostile and unhappy users 
complaining.

The above are some of the deeper meanings of the otherwise rather cheerful
Have it your own way!

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: About Have it your own way!

[Jeff Johnson]

On Jul 16, 2011, at 9:52 AM, Sriram Narayanan wrote:

 
 Telling people what they want to hear is less effort than attempting to 
 explain
 other or better alternatives. And so I end up describing implementations that
 I know are wrong/deficient and that I don't personally believe in or have 
 already
 implemented a better solution imho.
 
 
 This is a rather sad state of affairs.
 
 Here's a rather naive question:
 Assuming that all of the following are true:
 a. you get another version of rpm out (e.g. 5.5.x)

Till this month, RPM has been released monthly. And the only
reason that this month was skipped is that my development
environment is all torn up while preparing to upgrade
to Lion final (expected soon) and while getting infrastructure
in place to do arm board level development.

 b. that only guarantees that the spec files will be used
 c. the rpmdbDB be converted
 d. this rpm5.5 would be configured such that all your other
 recommendations are adhered to
 e. this rpm5.5 would have code such that all your recommended rpm5
 behavior is followed,
 

I think you may be misunderstanding some of my comments about
Have it your own way!
I have no desire that my recommendations be followed. Some of
the ideas I have I am personally deeply conflicted about and
only a try-and-see implementation can supply answers to
Is this the right engineering? Is it useful?
Some of those issues include using MongoDB, and attempting
a multi-threaded installer. I simply do not know my own
answer (I do know my own opinions) sufficiently well to
recommend anything.

But literally 2+ years of my life has ben _WASTED_ doing
support for Berkeley DB. I know the issues, and I know most
of the answers, and the support issues continue.

Berkeley DB was embedded in RPM way back when because -- at that time --
how to build RPM with Berkeley DB was the single largest support
problem.

And -- quite predictably -- now that Berekeley DB is again not distributed
with RPM, the issue(s) are once again the most significant support
issue with RPM. See the archives here and at launchpad.net/rpm bug reports.

 what issues with adoption do you see ?
 

As in world domination adoption for @rpm5.org?

Well go read LWN and OSnews … FL/OSS isn't exactly participatory for me.

 Are there behavior that people expect and continue to want to use
 despite there being other (better) ways to make it happen ?
 

The whole model of community development is reverting back
to vendor supplied software. And largely because people expect
to pay for what they use and vendors will efficiently
fill the supply side to satisfy the demand curve.

This is Economics 101 and capitalism in action, nothing more.

 Are you referring to distros wanting to be able to choose whether they will 
 provide rpm5 with its own copy of a library, or wish to force it to use the 
 distro-provided library ?
 
 
 Presumably you are referring to external - internal Berkeley DB here …
 
 … that is one (of many) examples of where what I recommended and chose
 in RPM is continuously ignored everywhere. What I chose is what was
 recommended by Sleepycat: go argue with Oracle, not me, if you have
 some issue with what I just said.
 
 
 I absolutely understand and support the embedding of BDB. On my box,
 for e.g., I have /usr/lib/rpmdb, and that's intended for rpm5 only.
 Anything else needed bdb should use whatever the system supplies, or
 embed its own.
 

Good. You should look at other usage cases, like openldap, nssdb, python,
perl, subversion, and consider whether your distro is more easily
upgraded if Berkeley DB is internalized in some of those other applications.

 In fact, once I complete a round trip of rpmbuild - createrepo -
 yum, I plan to embed everything else as well (e.g. file's magic,
 especially). I think something as core as a system's package manager
 should have the guarantee that users as well as developers of other
 packages as well as distro maintainers do not mess around with the
 dependencies it has.
 

The file(1) libmagic SHOULD be embedded if you wish to control carefully
for what ends up in *.rpm metadata: you need a private rpm-specific
version of what used to be in /etc/magic in order to have full control
of what strings end-up in *.rpm packages.

Whether the libmagic implementation is internal or external is less
important than being able to adjust the rules and strings that end up in
package metadata.

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/lib/ transaction.c

[Jeff Johnson]

On Jul 13, 2011, at 1:54 PM, Per Øyvind Karlsen wrote:

 
 
 Passing keys Somewhere Else Instead isn't needed either: rpmdbMireApply()
 is perfectly capable of being used intelligently, to perform a bullshit
 test that isn't necessary at all, however you wish.
 
 Would this be more acceptable? :)

This is fine with me but changes the meaning of identical
which is almost certainly not going to meet someone's expectations.

Using RPMTAG_HDRID instead of RPMTAG_NVRA is entirely consistent with a change
in another part of RPM (grep RPMTAG_HDRID) where it
was necessary (for Alt or PLD I fergit) to narrow
down identical to exactly the same package instead of
a package with the same name.

(aside)
There's a silly legacy compatible fallback to good old NEVRA
that likely just needs to be ripped out there. There are no
circumstances in which a fallback for legacy compatible
is meaningful any more, and everyone has had 5+ years to switch
and/or suggest better and none has: Time to haul out the trash …

… meanwhile if you back up a bit and look where  and how
this patch is being executed, its *insane* to apply a sanity check
of this sort through rpmtsRun().

The reason the test was done in rpmtsRun() originally is/was because this
is the earliest chance that RPM gets to see the _ENTIRE_
transaction fully formed. This sanity check pre-dates
anaconda and goes all the way back to RHL 6.0 when one
could justify an extra insurance sanity check.

But these days (10+ years later) its an API design failure:

If some script kiddie wants/needs training wheels from -lrpm for the feeble 
installer hacks,
well RPM should hand the training wheels (and diaper) to the script kiddie
on a non-critical code path, and let Darwinian evolution
punish the installer mutatations harshly, not pretend at errors or
features or anything else.

So back up a bit, look at the code path on which this bullshit
test is being performed, think a bit, and just rip the entire
test out.

73 de Jeff

 --- transaction.c   2011-07-12 11:50:34.229520174 +0200
 +++ /home/peroyvind/RPM/rpm/BUILD/rpm-5.3.12/lib/transaction.c
 2011-07-13 19:52:43.668223410 +0200
 @@ -1165,32 +1165,10 @@ rpmlog(RPMLOG_DEBUG, D_(sanity checking
if (!(rpmtsFilterFlags(ts)  RPMPROB_FILTER_REPLACEPKG)) {
ARGV_t keys = NULL;
int nkeys;
 -   xx = rpmdbMireApply(rpmtsGetRdb(ts), RPMTAG_NVRA,
 -   RPMMIRE_STRCMP, rpmteNEVRA(p), keys);
 +   xx = rpmdbMireApply(rpmtsGetRdb(ts), RPMTAG_HDRID,
 +   RPMMIRE_STRCMP, rpmteHdrid(p), keys);
nkeys = argvCount(keys);
 
 -#if defined(RPM_VENDOR_MANDRIVA)
 -   /* mdvbz: #63711
 -* workaround for distepoch never being added to RPMTAG_NVRA yet,
 -* leading to packages of same EVRA but with different distepoch
 -* being treated as the same package */
 -   if (nkeys  0) {
 -   int i;
 -   for (i = 0; i  nkeys; i++) {
 -   rpmmi mi = rpmtsInitIterator(ts, RPMTAG_NVRA, keys[i], 0);
 -   Header h;
 -   while ((h = rpmmiNext(mi)) != NULL) {
 -   HE_t he = memset(alloca(sizeof(*he)), 0, sizeof(*he));
 -   he-tag = RPMTAG_DISTEPOCH;
 -   xx = headerGet(h, he, 0);
 -   if (strcmp(he-p.str ? he-p.str : ,
 rpmteD(p) ? rpmteD(p) : ))
 -   nkeys--;
 -   he-p.ptr = _free(he-p.ptr);
 -   }
 -   mi = rpmmiFree(mi);
 -   }
 -   }
 -#endif
if (nkeys  0)
rpmpsAppend(ps, RPMPROB_PKG_INSTALLED,
rpmteNEVR(p), rpmteKey(p),
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/lib/ transaction.c

[Jeff Johnson]

On Jul 13, 2011, at 2:56 PM, Per Øyvind Karlsen wrote:

 
 
 So back up a bit, look at the code path on which this bullshit
 test is being performed, think a bit, and just rip the entire
 test out.
 Yeah, not disagreeing with you on that one.. :)
 

Good. Then haul out the trash please.

 What about the other checks done in rpmtsSanityCheck()?
 

More trash but trickier. See more or less the same sanity checks
in rpmtsAddInstallElement() in lib/depends.c. The modern (like last
5 years) test is on CPU-VENDOR-OS-GNU (aka CVOG) platform string
where the issue becomes mapping missing/unspecified VENDOR into
unknown in order to solve the remaining 0.1% of expectations
of RPM. But I digress …

… rip out all the tests. If the tests are really needed (as in diapers)
then a training wheel API needs to be devised on a transaction, akin
to exp-lain semantics inMongoDB and/or SQL. There's no purpose whatsoever
served by carrying this baggage around on the critical fast path
portion of RPM: it leads to an enormous amount of complexity that
leads to Yet More Ways that RPM does not succeed in installing packages
and no useful purpose for anyone with a 2W bulb for a brain.

 The same seems to apply to at least the RPMPROB_FILTER_OLDPACKAGE
 check as well..?

There's an immense amount of other trash that gets queued up as
soon as the stooped sanity check issues are dealte with.

E.g. rpmdbMireApply isn't useful anymore: RPM ACID is already
in place and in production. The flaw with rpmdbMireApply (which
is quite nicely general otherwise) is that its pig slow because it loads
every single header from an rpmdb in order to accomplish its pattern matching
against tags generally.

The bullshit test is one of two usage cases for rpmdbMireApply:
the other usage case is tied to this syntax:
rpm -qa 'arch=i[3456]86' 'version=1.2.3' 
'release=4.5.6.7.8.9.fedorabledisttagssuccketh'
which noone really needs or uses in its full rpmdbMireApply() generality.

Instead RPM ACID has way way faster *RE pattern matching like

rpm -q '^X11.*'

applied only to indices, avoiding headerLoad(), and the PM ACID implementation
 is already also on CLI erase paths (if you are truly that insanely brave)

rpm -e '^X11.*'

If one cannot figure out how to write a *RE against a NEVRA string, well

Use GOOG!

Short answer: rpmdbMireApply() is headed for the bit bucket as well. Not the
right answer even if it was a perfectly general/useful implementation.

 Or can rpmtsSanityCheck() be dropped entirely?
 

More trash (but I can't recall, look to see if there's anything useful there).

 Just skimming through code, about to leave the office, not the right moment
 to start think on my own.. ;)
 

Feel free to rip as needed. There's an immense amount of uselessness
that RPM carries around simply to avoid my wasting time on bikeshed
community discussions.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/lib/ transaction.c

[Jeff Johnson]

This isn't at all the right approach because
its doesn't scale.

You have changed a single high performing rpmdb
access that applies patterns to keys only, and added
a hack-o-round that then goes and loads multiple
headers repeatedly to try and adjust the set
returned by the original access to fit some desired
semantic goal.

You need to figure a better *RE pattern, not whack in
lots of gunky code to fix it somehow.

73 de Jeff


On Jul 12, 2011, at 6:39 AM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   12-Jul-2011 12:39:52
  Branch: HEAD Handle: 2011071210395200
 
  Modified files:
rpm CHANGES
rpm/lib transaction.c
 
  Log:
fix rpm treating packages of same NVRA, but different distepoch as
equal, preventing upgrades between them.
 
  Summary:
RevisionChanges Path
1.3661  +2  -0  rpm/CHANGES
1.433   +23 -0  rpm/lib/transaction.c
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3660 -r1.3661 CHANGES
  --- rpm/CHANGES  6 Jul 2011 12:55:29 -   1.3660
  +++ rpm/CHANGES  12 Jul 2011 10:39:52 -  1.3661
  @@ -1,4 +1,6 @@
   HEAD:
  +- proyvind: fix rpm treating packages of same NVRA, but different 
 distepoch
  +as equal, preventing upgrades between them.
   - proyvind: dbconvert: fix segfault when no root is specified.
   - proyvind: check-multiarch-files: remove unnecessary import of 
 MDK::Common,
   causing undesired dependency from perl dependency generator.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/lib/transaction.c
  
  $ cvs diff -u -r1.432 -r1.433 transaction.c
  --- rpm/lib/transaction.c1 Feb 2011 13:53:33 -   1.432
  +++ rpm/lib/transaction.c12 Jul 2011 10:39:52 -  1.433
  @@ -1130,6 +1130,29 @@
   xx = rpmdbMireApply(rpmtsGetRdb(ts), RPMTAG_NVRA,
   RPMMIRE_STRCMP, rpmteNEVRA(p), keys);
   nkeys = argvCount(keys);
  +
  +#if defined(RPM_VENDOR_MANDRIVA)
  +/* mdvbz: #63711
  + * workaround for distepoch never being added to RPMTAG_NVRA yet,
  + * leading to packages of same EVRA but with different distepoch
  + * being treated as the same package */
  +if (nkeys  0) {
  +int i;
  +for (i = 0; i  nkeys; i++) {
  +rpmmi mi = rpmtsInitIterator(ts, RPMTAG_NVRA, keys[i], 0);
  +Header h;
  +while ((h = rpmmiNext(mi)) != NULL) {
  +HE_t he = memset(alloca(sizeof(*he)), 0, sizeof(*he));
  +he-tag = RPMTAG_DISTEPOCH;
  +xx = headerGet(h, he, 0);
  +if (strcmp(he-p.str ? he-p.str : , rpmteD(p) ? 
 rpmteD(p) : ))
  +nkeys--;
  +he-p.ptr = _free(he-p.ptr);
  +}
  +mi = rpmmiFree(mi);
  +}
  +}
  +#endif
   if (nkeys  0)
   rpmpsAppend(ps, RPMPROB_PKG_INSTALLED,
   rpmteNEVR(p), rpmteKey(p),
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm/ CHANGES rpm/lib/ transaction.c

[Jeff Johnson]

On Jul 12, 2011, at 8:21 AM, Per Øyvind Karlsen wrote:

 2011/7/12 Jeff Johnson n3...@mac.com:
 This isn't at all the right approach because
 its doesn't scale.
 
 You have changed a single high performing rpmdb
 access that applies patterns to keys only, and added
 a hack-o-round that then goes and loads multiple
 headers repeatedly to try and adjust the set
 returned by the original access to fit some desired
 semantic goal.
 
 You need to figure a better *RE pattern, not whack in
 lots of gunky code to fix it somehow.
 Yes, I kinda suspected this certainly wasn't the most optimal
 way to solve this.. :|
 
 The problem basically is that it's doing comparision of RPMTAG_NVRA
 between the installed packaged and the package to be installed, which
 will give us problems when only distepoch differs since RPMTAG_NVRA
 still isn't consistent with %___NVRA.
 

There's a fundamental confusion here between identification
and versioning (as used in assertion checking).

The %___NVRA is there purely for identification. Whatever
elements you wish to add/delete in a chosen identification
string can be added at will. One can also undertake encoding
conversions, and attaching other data, from sources not
just derived from *.rpm metadata contents if you wish to
go to the trouble of adding a header tag/format extension
(which can be done without changing one line of RPM code
if necessary).

OTOH, versioning is hierarchical, and depends on a sequence
of comparison operations applied to well defined data, done
strictly in the same order, with choices for results when
missing (as in not present or empty, different meanings for
missing).

 ie. for:
 foo-1-1-mdv2011.0.x86_64 vs foo-1-1-mdv2012.0.x86_64,
 NVRA will be 'foo-1-1.x86_64' for both.
 
 So any good suggestions on performing the check on distepoch in addition
 without heavy drawbacks?

The very simplest solution is
Call rpmtsRun() with RPMPROB_FILTER_REPLACEPACKAGE set.
That is what the flag is implemented and intended for.

The next most simple fix starts to address a design flaw:
A sanity check on the elements in a transaction set
needs to be done somewhere else instead. Checking
in rpmtsRun() is far far far too late todo anything
but best effort fudging or bizarre FULLSTOP termination,
All of the checks, not just the *_REPLACEPKG check we happen to
be discussing, are total bullshit: if the check finds something
wrong, well, its some programmer's brain fart choosing what
packages to add to a transaction, not anything else.

But if you simply MUST fix *_REPLACEPG, then the right fix is to
change rpmdbMireApply() to not use the identification RPMTAG_NVRA
string but rather construct whatever tag tests one wishes however
one wants. Post processing fiddle ups (like you just checked in)
to correct a call that should have been done differently are silly band-aids.

 Checking distepoch of the package to be installed is easy and fast enough
 with rpmteD(p), but any chance of providing distepoch with the keys returned
 by rpmdbMireApply on RPMTAG_NVRA in some way..?
 

Um, not true: rpmteD is a simple getter, does no checking whatsoever,
so I have no idea how easy/fast applies to something that isn't being done.

Passing keys Somewhere Else Instead isn't needed either: rpmdbMireApply()
is perfectly capable of being used intelligently, to perform a bullshit
test that isn't necessary at all, however you wish.

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Distepoch identification (was Re: [CVS] RPM: rpm/ CHANGES rpm/lib/ transaction.c)

[Jeff Johnson]

Lemme split this into 2 threads to try to address
two twisted issues properly. The other thread will be
Transaction sanity checks

On Jul 12, 2011, at 8:48 AM, Jeff Johnson wrote:

 
 You need to figure a better *RE pattern, not whack in
 lots of gunky code to fix it somehow.
 Yes, I kinda suspected this certainly wasn't the most optimal
 way to solve this.. :|
 
 The problem basically is that it's doing comparision of RPMTAG_NVRA
 between the installed packaged and the package to be installed, which
 will give us problems when only distepoch differs since RPMTAG_NVRA
 still isn't consistent with %___NVRA.
 
 
 There's a fundamental confusion here between identification
 and versioning (as used in assertion checking).
 
 The %___NVRA is there purely for identification. Whatever
 elements you wish to add/delete in a chosen identification
 string can be added at will. One can also undertake encoding
 conversions, and attaching other data, from sources not
 just derived from *.rpm metadata contents if you wish to
 go to the trouble of adding a header tag/format extension
 (which can be done without changing one line of RPM code
 if necessary).
 
 OTOH, versioning is hierarchical, and depends on a sequence
 of comparison operations applied to well defined data, done
 strictly in the same order, with choices for results when
 missing (as in not present or empty, different meanings for
 missing).
 

Your Distepoch: (and Disttag) implementation is nearly perfect (even if unlikely
to be used outside of Mandriva derived packaging).

But there's several sick hacks in need of improvement. There's
nothing whatsoever wrong with the hacks, just its time to
refactor the code to the right place.

The hard hack is re-writing the *RE pattern applied to NVRA keys.
I can/will fix (by refactoring and writing a better *RE) that
properly, but identifying how the problem is to be fixed is the
point of this thread.

The one wartlet with Distepoch: is

How SHOULD a {N,E,V,R,D} tuple be serialized/concatenated in a string?

There is what is currently in Mandriva 2011 (I'll use popt for explicit 
example):

$ rpm -q libpopt0
libpopt0-1.16-2-mdv2011.0.i586

And there is also the legacy compatible (with %mkrel) concatenation that
would (not checked) look like this:

libpopt0-1.16-2mdv2011.0.i58

So the incompatibility is largely whether a '-' is added or not.

The incompatible '-' is added in %__NVRA (poifect!):

$ rpm --showrc | grep __NVRA
-14: ___NVRA
%%{NAME}-%%{VERSION}-%%{RELEASE}%%|DISTTAG?{-%%{DISTTAG}%%|DISTEPOCH? …
^ here is 
the incompatibility

The fundamental problem has to do with the one or two remaining 
non-identification
usage cases of RPMTAG_NVRA in RPM code.

The resolution will be to cleanly and _ENTIRELY_ split all uses of
%__NVRA and RPMTAG_NVRA from all versioning usage cases in RPM.

The subtlety (and the hacks) come from confusing the identification context
from the assertion versioning context. Yes it is _VERY_ confusing when its
just a string that needs to be sliced and diced differently depending on
whether its an identification (i.e. intended for human display) or an
assertion context.

The assertion context (and its what you originally agreed at least implicitly
back in 2009 when implemented) is in the pattern that splits the EVRD
string into a tuple for use by assertion version comparisons. You already know
(because I'm sure you've stared at the problems a zillion times) that the code 
is
almost exactly what you want implemented: i.e. you really haven't had to change
much in the EVRD tuple parsing (your other check-in today was dead-on).

The parsing hint for splitting an assertion string into a {E,V,R,D} tuple
is a ':' as described here:

# STEP 1: Match the string and capture regex parts
#  1  2   3 4
#  X :  X-X  :X
%evr_tuple_match  ^(?:([^:-]+):)?([^:-]+)(?:-([^:-]+))?(?::([^:-]+))?$

OTOH, the %___NVRA identification and RPMTAG_NVRA (which uses %___NVRA as a 
configurable
template) and the table /var/lib/rpm/Nvra (which is based on RPMTAG_NVRA and
configurable %___NVRA) are quite useful and important because its 
identification
and package names which everyone focusses on.

So there's still a few places where RPM uses RPMTAG_NVRA in the wrong context, 
one of
the places being what you just patched (I donna bother fixing properly there 
because
its my belief that the test is bullshot and needs to be eliminated, not fixed).

So ultimately we have a ':' and a '-' character as explicit tuple separators
and have dueling identification and assertion parsers.

There are 2 approaches to fixing:

1) (recommended) Rip out the incompatible '-' in %__NVRA. Yes I
know how many places that needs to be fiddled out @mandriva.

2) Change RPM to never use RPMTAG_NVRA

Re: Simple? query question

[Jeff Johnson]

On Jun 30, 2011, at 12:29 PM, Mark Hatle wrote:

 I'm working on multilib/multiarch support in the oe-core stuff.  And I've hit 
 an
 issue.  I'm trying to query for a package of a specific arch.
 
 rpm -q zlib
 
 returns:
 
 zlib-1.2.5-r0.x86_64
 
 if I do:
 
 rpm -q zlib-1.2.5 or zlib-1.2.5-r0 or zlib-1.2.5-r0.x86_64
 
 I get the same result..
 
 however, I thought I could do:
 
 rpm -q zlib.x86_64
 

That's a bug (and likely related to Distepoch: turn on
but might be RPM ACID related, dunno)

Meanwhile there's lots of ways to achieve that same query:

rpm -q zlib | grep x86_64
rpm -q '^zlib.*x86_64$'
rpm -qa zlib 'arch=x86_64'

 and it would also work, but that doesn't seem to be the case.  Is there any 
 way
 to do rpm -q zlib.x86_64 -- or do I need to do rpm -q zlib, and through is 
 some
 grep commands on the output?
 

DO what works.

FYI: The
rpm -q N.A
form of query is likely doomed in RPM. It was an expedient hack
when multilib was first deployed: I was given 4 hours to
devise, implement, test, and deliver a solution with mad manglers
watching my every key stroke.

 (The current case only has one instance of zlib, but eventually there could 
 be 2
 or 3 instances.)
 

Lemme see if I can fix this evening … tomorrow is 7/1 monthly release time.
No promises.

73 de Jeff

 Thanks!
 
 --Mark
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM DB, Berkley DB and journal files

[Jeff Johnson]

On Jun 29, 2011, at 8:45 AM, Tim Mooney wrote:

 In regard to: R: Re: RPM DB, Berkley DB and journal files, 
 pinto.elia@gmail...:
 
 Perhaps OT, but just a my random thought about this thread about  rpm5
 and berkley DB. most of which is discussed here is very similar, or
 equal sometime, on similar question on openldap. Apparently strange but
 not so quite if someone think carefully at the rpm5 rpm acid feature.
 Just for Remainder my self, and Who is interested, that the rpm acid
 docu, not so much at this moment, need also on a tuning guide. Sound
 like a plan ? :-)
 
 It's funny you mention OpenLDAP, since most of what I've learned about
 BDB in the past year has been because of the work I did to upgrade my
 site's primary and secondary LDAP servers to OpenLDAP 2.4.25.
 

(aside)
Hehe, I too am paying attention to OpenLDAP not rpmdb, because of Lion
recently.

 OpenLDAP, being long-running and highly performance-sensitive, demands a
 lot more tuning of BDB than RPM does and not all of the same
 considerations will apply, but learning something about BDB for one of
 those two packages will still help with tuning the other.
 

And almost certainly for both OpenLDAP and RPM:

If you don't understand what is in DB_CONFIG, you are
likely going to become a Berkeley DB Hater shortly.

Seriously: The default parameters in Berkeley DB -- while wisely chosen --
aren't anyplace close to being correct for even moderately large stores
like in both OpenLDAP and RPM. This is also quite true for larger
stores like SKS key dumps with 3Gb of pubkeys:
You MUST choose your parameters actively and adjust to usage case.

Meanwhile: Do you agree that increasing mmap size is the most important
performance related tunable? I've diddled most of the tunables at some
point or another using rpm --stats as a measurement, and mmap'd I?O
is/was the most important tunable (disabling sync cannot be done in production
imho).

73 de Jeff

 Tim
 -- 
 Tim Mooney tim.moo...@ndsu.edu
 Enterprise Computing  Infrastructure  701-231-1076 (Voice)
 Room 242-J6, IACC Building 701-231-8541 (Fax)
 North Dakota State University, Fargo, ND 58105-5164
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM DB, Berkley DB and journal files

[Jeff Johnson]

On Jun 29, 2011, at 9:59 AM, Tim Mooney wrote:

 In regard to: Re: RPM DB, Berkley DB and journal files, Jeff Johnson said...:
 
 Meanwhile: Do you agree that increasing mmap size is the most important
 performance related tunable?
 
 First, although I have indeed learned a lot about BDB from OpenLDAP, that
 still doesn't mean I know much.  ;-)  I went from knowing nothing about
 it to knowing just a little, and most of that specific to how OpenLDAP
 uses BDB.  You're still the expert, not me.
 

Well I'm not the expert by choice:
She LART's me for every picayune flaw in RPM+BDB …
and the experience has developed a fair amount of scar tissue
over the years …

 That said, if you're going to use mmap at all, then I trust you're correct
 about mmap size being the most important tunable.
 
 However, there has been considerable discussion on the OpenLDAP mailing
 list about how on certain platforms, using mmap for the BDB env is much,
 much slower than using System V IPC shared memory.  For example, on
 Solaris using shared memory is an order of magnitude faster than using
 mmap for the env:
 
   http://www.openldap.org/lists/openldap-technical/201103/msg00069.html
 
 Howard Chu has also stated that IPC shared memory is also beginning to
 be a performance win (at least for OpenLDAP) on recent Linux kernels too,
 though I don't believe it's the order of magnitude difference you see
 elsewhere:
 
 http://www.openldap.org/lists/openldap-technical/201104/msg00150.html
 
 I can find more references to the issue with some more digging, but
 basically, for at least how OpenLDAP uses BDB, System V IPC can be
 a big win.
 

Thank you for the current pointers. I cruise the openldap
lists every 6 months or so looking for real world experience
with Berkeley DB. I baby sit (as in run) 5-10 SKS key servers
for a similar purpose, seeing what issues there are in using
Berkeley DB on a large scale persistently …

BTW, I seem to recall an earlier post, also from Howard Chu,
about one of the tunables that led to ~11K transactions/second.
I can dig out the reference from a few years back if interested …
… all I recall atm is that I had to read and study the posting
for several days to do the same in RPM ACID, but have forgotten
the details even if not the (now ~2 year old) experience.

 That doesn't necessarily mean it's a good idea for RPM in general,
 or RPM on an embedded platform in particular.  It would get rid of
 most of the __db* files that the original bug report mention, though.  ;-)
 Possibly at the expense of making RPM not work at all on that platform.
 

Yes: The usage case of Berkeley DB in OpenLDAP and RPM is rather different.

OpenLDAP is transaction oriented with high thru-put as the primary metric.

RPM isn't a daemon, and MUST drop-in on bare iron and empty chroot's
and Just Work to the greatest extent possible.

These primary goals dictate the tuning and -- till RPM ACID -- Berkeley DB
performance simply wasn't an issue, and is still mostly irrelevant. What
RPM users expect is a cushy ride with no human intervention whatsoever,
and so KISS is more appropriate than better designed schema or anything else.

 I've diddled most of the tunables at some
 point or another using rpm --stats as a measurement, and mmap'd I?O
 is/was the most important tunable (disabling sync cannot be done in 
 production
 imho).
 
 Makes sense.  Using RAM for the db env is always going to be faster than
 using disk.
 

I haven't really looked at the SysV IPC option with Berkeley DB, mostly
because I know that most RPM users simply cannot cope with
man 1 ipcs
from the command line, and SysV IPC has always been a bit painful to
manage when something breaks.

My immediate question with IPC would be:

What happens when there are multiple IPC instances instantiated in a 
chroot?

There's almost certainly some corner case isolation issues because
IPC likely leaks information from the outer to the inner chroot
quite predictably. (Disclaimer: I haven't bothered to look, just …)

But the general principle of
Memory is faster than disk.
applies, but one needs to also consider the locking issues from
multiple contentious access to an rpmdb, and from both outside/inside
a chroot, or some well meaning claim of higher perfuming WILL
lead to corruption quite predictably. E.g. there are no provisions
I'm aware of (and arguably shouldn't be) for sharing a DBENV between
ELF32/ELF64 multilib environments. RPM users expect interoperability
to Just Work everywhere and always in spite of the rather daunting
engineering difficulties sharing a DBENV between architectures.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm-5_4: rpm/ CHANGES configure.ac rpm/tools/ dbconvert.c

[Jeff Johnson]

Nothing wrong with the patch per se but there's more
than testing for sys/endian.h that needs to be done.

There's breakage in colliding WORDS_BIGENDIAN et al, beecrypt
has internal m4 tests that set WORDS_BIGENDIAN, and there are platforms
that are missing endian.h entirely but need WORDS_BIGENDIAN defined
correctly.

It more a tedious issue than anything else. The proper fix (imho)
is to arrange for a run-time, not a compiled in AutoFu check, test for 
endianness.

The run-time test has been implemented 5 or 6 places in RPM code, and
isn't very hard to do (but does involve a union).

So the better fix is not to include endian.h anywhere.

73 de Jeff

On Jun 28, 2011, at 8:26 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   28-Jun-2011 14:26:11
  Branch: rpm-5_4  Handle: 2011062812261001
 
  Modified files:   (Branch: rpm-5_4)
rpm CHANGES configure.ac
rpm/tools   dbconvert.c
 
  Log:
merge from HEAD afb patch portability of endian
 
  Summary:
RevisionChanges Path
1.3501.2.144+1  -0  rpm/CHANGES
2.472.2.25  +2  -2  rpm/configure.ac
2.1.2.6 +10 -1  rpm/tools/dbconvert.c
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3501.2.143 -r1.3501.2.144 CHANGES
  --- rpm/CHANGES  28 Jun 2011 12:01:47 -  1.3501.2.143
  +++ rpm/CHANGES  28 Jun 2011 12:26:10 -  1.3501.2.144
  @@ -1,4 +1,5 @@
   5.4.1 - 5.4.2
  +- devzero2000: merge afb patch portability of endian
   - proyvind: check-multiarch-files: remove unnecessary import of 
 MDK::Common,
   causing undesired dependency from perl dependency generator.
   - devzero2000: add a simple usage command to devtool
  @@ .
  patch -p0 '@@ .'
  Index: rpm/configure.ac
  
  $ cvs diff -u -r2.472.2.24 -r2.472.2.25 configure.ac
  --- rpm/configure.ac 20 Jun 2011 12:33:27 -  2.472.2.24
  +++ rpm/configure.ac 28 Jun 2011 12:26:10 -  2.472.2.25
  @@ -672,8 +672,8 @@
   dnl # standard headers (generic)
   AC_CHECK_HEADERS([dnl
   fcntl.h getopt.h grp.h memory.h netdb.h pwd.h utime.h dnl
  -signal.h sys/ipc.h sys/socket.h sys/select.h sys/time.h dnl
  -sys/types.h sys/stdtypes.h sys/mman.h sys/resource.h dnl
  +signal.h sys/endian.h sys/ipc.h sys/socket.h sys/select.h dnl
  +sys/time.h sys/types.h sys/stdtypes.h sys/mman.h sys/resource.h dnl
   sys/utsname.h sys/prctl.h sys/wait.h netinet/in_systm.h dnl
   machine/types.h mntent.h sys/mnttab.h sys/systemcfg.h dnl
   sys/param.h sys/mount.h sys/mntctl.h sys/vmount.h dnl
  @@ .
  patch -p0 '@@ .'
  Index: rpm/tools/dbconvert.c
  
  $ cvs diff -u -r2.1.2.5 -r2.1.2.6 dbconvert.c
  --- rpm/tools/dbconvert.c2 Jun 2011 17:39:41 -   2.1.2.5
  +++ rpm/tools/dbconvert.c28 Jun 2011 12:26:11 -  2.1.2.6
  @@ -18,9 +18,18 @@
   #include rpmts.h
   #include rpmlog.h
 
  +#ifdef HAVE_SYS_ENDIAN_H
  +#include sys/endian.h
  +#endif
  +#ifdef __APPLE__
  +#include libkern/OSByteOrder.h
  +
  +#define htobe32(x) OSSwapHostToBigInt32(x)
  +#define htole32(x) OSSwapHostToLittleInt32(x)
  +#endif /* __APPLE__ */
   #if BYTE_ORDER == LITTLE_ENDIAN
   #define bswap32(x) htobe32(x)
  -#elif __BYTE_ORDER == BIG_ENDIAN
  +#elif BYTE_ORDER == BIG_ENDIAN
   #define bswap32(x) htole32(x)
   #endif
 
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM DB, Berkley DB and journal files

[Jeff Johnson]

On Jun 28, 2011, at 1:00 PM, Mark Hatle wrote:

 I was hoping someone can help me understand what is happening and how to 
 address
 it for our embedded system needs.
 
 For reference:
 
 http://bugzilla.pokylinux.org/show_bug.cgi?id=1174
 
 from the above, the complaint is:
 
 When using zypper/rpm install/removal package, database files and log files
 cost a lot of disk space and may cause target image out of space.
 

Sure: that is true for all caching: there's a trade-off between
performance benefit and caching cost. You need to find your
balance point (most users don't perceive the performance win).

There are tunables to limit the no. and both total and per-file sizes.

Most of the tuning related to __db* size is in DB_CONFIG:

set_cachesize  0 1048576 0

The last number is no. of files iirc. Note that I've _NEVER_
been able to capture a significant performance improvement
but changing cache size (but I haven't tried very hard). The
most significant performance gains come from having

set_mp_mmapsize 268435456

and no cache is needed whatsoever, all I/O is mmap'd straight
off the file system, if you can stand the memory mapping pressure
(the above is 256M w design heuristic of 25% of a nominal 1Gb host)

But the best possible answer for you is likely to move the __db* files
elsewhere, like /var/cache/rpm. The people who are complaining do not 
distinguish
that there are multiple types of items being stored, some important,
and some not, which just leads to Bloat! Bloat! Bloat! because
of the mixture, not anything else.

Note that moving forces re-thinking of permissions, and FHS, and
all the other community inputs and so I've chosen not to have that
meandering discussion that goes no place.

 The first time when we use RPM, a lot of database files will be created, 
 larger
 than 10MB.
 

I don't know what first time means here. Presumably the issue
is that __db* files are created and populated, exactly how
caching is supposed to work.

Note that there other uses than caching in __db* files, most notably
that locks are registered (and stale locks removed on next invocation),
and that __db* files are the means by which multiple processes
coordinate locking and I as in Isolation data visibility between
processes. I'd suggest just leaving __db* files in place is
safer/saner than the extreme measures that Berkeley DB has to undertake
to avoid interprocess races on the file system when instantiating a dbenv.
YUM e.g chose to open-and-close a dbenv on every data access, adding
a huge overhead and Yet More Raciness and was rather a disaster.

But the size of __db* is purely a caching related issue. Note also
that those files are likely sparse, so examine blocks allocated,
not reported EOF offset, before undertaking fixing what perhaps is
a non-existent perceptual problem.

Note that you have also missed log/* files that WILL be large, particularly
if you haven't enabled this flag to auto-remove unused (in the current
sense, logs are always relevant for historical serialization).

The flag to automate log clean up is
set_flags  db_log_autoremove   on

hth

73 de Jeff
 ---
 
 Each of the __db.* files created are large.  Nearly 10 MB each.  In addition 
 in
 the log directory are additional files that are also nearly 10 MB each.
 
 Are there any suggestions for changing the BerkleyDB configuration, either the
 config file or RPM settings/usages that would help me shrink down these sizes 
 --
 or potentially help me eliminate the creation of these files?
 
 I'm currently using rpm-5.4.0 from rpm-5.4.0-0.20101229.src.rpm.
 
 --Mark
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org

__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM DB, Berkley DB and journal files

[Jeff Johnson]

On Jun 28, 2011, at 1:35 PM, Per Øyvind Karlsen wrote:

 
 You probably want to enable automatic log removal, ie. look at
 Db_open() in URPM:
 http://svn.mandriva.com/cgi-bin/viewvc.cgi/soft/rpm/perl-URPM/trunk/URPM.xs?revision=272986view=markup
 
 For where it's actually gets enabled, look closer at line 3270.
 

There's no reason to implement in code what can be done
by simply editing a DB_CONFIG file.

Implementing in code creates mysterious undocumented side-effects
and endless perceptions and expectations that start like
RPM sux because …
that can _NEVER_ be fulfilled.

 This will make berkeley db remove all the log files that's not needed
 so that you'll
 only keep the last file under /var/log/rpm/log.
 

So will
set_flags  db_log_autoremove   on
and if you cannot bigger how to add DB_CONFIG to /var/lib/rpm where
needed, well, try
man 1 cp
and stare at how simple and obvious a file copy command actually is.

We most definitely disagree on what rpm SHOULD attempt here.

73 de Jeff__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: RPM DB, Berkley DB and journal files

[Jeff Johnson]

On Jun 28, 2011, at 5:09 PM, pinto.e...@gmail.com wrote:

 Perhaps OT, but just a my random thought about this thread about  rpm5 and 
 berkley DB. most of which is discussed here is very similar, or equal 
 sometime, on similar question on openldap. Apparently strange but not so 
 quite if someone think carefully at the rpm5 rpm acid feature. Just for 
 Remainder my self, and Who is interested, that the rpm acid docu, not so much 
 at this moment, need also on a tuning guide. Sound like a plan ? :-) Regards

Well its an OK plan but ..

… openldap has many more access patterns and a far more
varied schema than what RPM has.

There actually is database tuning involved, inserting
an index where needed to minimize a transactional
response time and optimize throughput. Its rather
nicely implemented in openldap imho (though I've
only looked at OpenDirectory logs on Lion recently, not code).

I've actually stated the only two Berkeley DB tunables of mny
measureable (by me) significance accessing an rpmdb:
Increase mmap'd I/o to be sufficiently large
and the other is file system related
Disable fsync
which instantly voids ACID behavioral guarantees.

Increased rpmdb performance can only be achieved (with current schema)
by removing duplicate/redundant accesses in RPM code, not by
adjusting Berkeley DB parameters. That's my experience anyways.

Tuning for embedded (this thread) is a whole different matter than tuning for 
performance.

If you wish to minimize resources, and less is _ALWAYS_ more, then
everything is discarded and ripped out until whatever external
constraints imposed on disk/memory footprint are achieved.

The focus is usually (in my experience) driven solely
by resource cost with very little thought to performance loss,
and so the ripping proceeds until resources fit or
one gets bored/tired and decides Memory is cheap! and gives up.

Its still unclear whether RPM ACID is a desirable feature or not.
Either way, I achieved what I set out to do, and there are significant
performance increases that were driven by my goal
RPM ACID must not be slower than concurrent access is/was.

73 de Jeff



__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org

Re: [CVS] RPM: rpm-5_3: rpm/ CHANGES cpu-os-macros.tar.gz

[Jeff Johnson]

The right fix here -- almost 4+ years after being phased out --
is to remove the cpu-os-macros.tar.gz tar ball in cvs.

There's no way that rpm itself could or should enumerate
the CPU enumeration faced with the proliferation of
architectures.

Nor has there ever been anything useful information in the CPU-OS/macros
file except %optflags, which is invariably both compiler and vendor specific.

Yes there is %_arch which Mandriva needs in CPU-OS/macros too.

73 de Jeff


On Jun 16, 2011, at 4:31 AM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   16-Jun-2011 10:31:32
  Branch: rpm-5_3  Handle: 2011061608313001
 
  Modified files:   (Branch: rpm-5_3)
rpm CHANGES cpu-os-macros.tar.gz
 
  Log:
fix broken %optflags for i686-linux macros
 
  Summary:
RevisionChanges Path
1.3296.2.239+1  -0  rpm/CHANGES
1.2.4.3 BLOBrpm/cpu-os-macros.tar.gz
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3296.2.238 -r1.3296.2.239 CHANGES
  --- rpm/CHANGES  7 Jun 2011 16:34:09 -   1.3296.2.238
  +++ rpm/CHANGES  16 Jun 2011 08:31:30 -  1.3296.2.239
  @@ -1,4 +1,5 @@
   5.3.12 - 5.3.13
  +- proyvind: fix broken %optflags for i686-linux macros.
   - proyvind: mandriva: fix implicit %clean not being run when %changelog 
 is
   present.
   - devzero2000: In fpLookupSubdir, data returned by hash should be of 
 type
  @@ .
  (cd rpm  \
   uudecode '@@ .'  \
   mv cpu-os-macros.tar.gz cpu-os-macros.tar.gz.orig  \
   xdelta patch cpu-os-macros.tar.gz.xdelta cpu-os-macros.tar.gz.orig 
 cpu-os-macros.tar.gz  \
   rm -f cpu-os-macros.tar.gz.orig cpu-os-macros.tar.gz.xdelta)
  Index: rpm/cpu-os-macros.tar.gz
  
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: Yocto, directory ownership, RPM deb packaging

[Jeff Johnson]

On Jun 10, 2011, at 11:25 PM, Mark Hatle wrote:

 
 I was wondering because we could optionally enable it as a form of QA.
 Otherwise we'll have to dump the data from the RPM packages and externally 
 run QA…
 

OK better QA is the goaal.

The choices are
QA by failing/correcting builds
QA by failing/correcting installs

To do build QA (which poky/yacto is well prepared for
with solvedbs that can be fed back to the build system)
What is the everything universe of package metadata?
to detect (your problem) multiple packages that contain a directory with
different permissions.

If there is a reference rpmdb available during build, then problems
can be detected or corrected: with permissions a principle of least/most
access isn't hard to implement with existing data.

There's the additional workflow issue determining when the reference
rpmdb is updated:
every successful build?
every successful image creation?
every successful release?
and the usually implementation language: python! perl! ruby! bash! cacophony.

A reference rpmdb is the right approach imho because all metadata is captured
and available as early as possible, where its rather easy to fix.

There is another approach that can be attempted with additional assertion 
checks like
No directory should have 700 permissions.
You can already do this by configuring additional scriptie to be run at end of 
%install.

Its unclear which is harder to maintain:
A reference rpmdb and implementations in RPM.
or
Opaque hooks that can be populated as needed to do whatever.

Since opaque hooks has been around since forever and still being discussed, 
well,
I'd try an everything rpmdb to ensure modes set correctly and (more 
importantly imho)
no file conflicts on path and no orphan directories.

For install QA its largely introducing Yet Another Failure Mode
for client installs, with all the usual
RPM sux because it won't install my packages
support issues. One could of course implement a correction
to directory permission problems based on least/most access,
but then another decade of discussion ensues on Newer! Better! Bestest!
so I'd rather _NOT_ see QA attempted during install's simply because
nooone wishes to write the scriptie for opaque hooking or wishes
responsibility for maintaining the everything solved so that
problems could be detected during build.

Yes poky/yacto doesn't use rpmbuild for building, but does use *.spec
templates for creating packages, which is where build QA would
occur, possibly inconveniently for your workflow.


 I plain and simply don't give a hoot unless a sound
 engineering reason is given for the change.
 
 Change for the sake of change isn't a sound reason, nor is consistency with
 the Debian Borg mind meld, nor is introducing Yet Another Way That RPM
 Doesn't Install Anything because of lack of vendor QA.
 
 Also, what is the impact to RPM if a lot of packages all own the same
 directories (such as /bin, /lib, /usr, etc..)?  [or alternatively have no
 packages own the directories and fix the owner/group/mode with some type of
 scripting]
 
 Lemme answer a question with a question:
 
  Why are you asking? Are you seeing impact or just enquiring?
 
 More inquiring at this point...  This all stems from two issues that I have to
 solve.  The first is pragmatic, we had a package that was incorrectly creating
 the /var/lib portion of a directory as 0700, it just happened it was being
 installed before the base-files package (which normally creates those
 directories as 0755...)  It caused a number of minor problems that took a 
 while
 to identify.
 

Hmmm … rpm -Va should have detected the permission change on the directory 
contained
in multiple packages easily.

The out-of-order indeterminacy (which is likely a nasty Heisenbug with no 
reproducer)
could be mitigated if you started relying on parent directory dependencies (but 
then
you have to pay attention to LOOP's to fully remove the indeterminacy).

 In the Debian world the folks I asked basically said this doesn't happen.. I
 asked how they were sure and they spouted some stuff that IMHO really means
 cause we treat it as a bug and fix it, well duh..
 

Yes the mind meld has very strong administratively based quality control and 
RPM doesn't.

 In the RPM world, I would have solved the issue different, I would have owned
 the directory and files within the /var/lib, but not /var/lib itself -- that
 would have been owned by the base-files package
 
 Owning all directories in every package so that
  rpm -qa | wc -l
 and
  rpm -qf / | wc -l
 are identical is what I think should be done on even days of the month.
 
 Currently I'm leaning toward having to have all packages own all files and
 directories, in order to match the deb behavior -- I don't see a reason why
 this isn't possible, just not what I'm used to doing.
 

It would not be hard to add rpmbuild 

Re: Yocto, directory ownership, RPM deb packaging

[Jeff Johnson]

On Jun 10, 2011, at 4:39 PM, Mark Hatle wrote:

 I've been trying to reconcile the behavior between rpm and deb packages in the
 way their directories are handled.
 
 In rpm, we can set the directories to own in various ways.  If there is a
 conflict between two packages owning the same directory -- first-in wins and
 that's where the permissions/owner/group are selected.  (correct me if I'm 
 wrong)
 

That's likely (but based on ancient memories of a performance speed-up
to avoid creating already existent directories ... not that mkdir is slow, just
that is what was originally implemented like 1997).

But that explains first rather than last. I could look at the code: so can you
rpm -Uvv --fsmdebug *.rpm
should spew every system call performed.


 In deb, directories are not owned by any package, they're just created as an
 artifact of the package installation.  All of the packages in the system are
 expected to have directories with the same permissions, owners, group or the
 package is broken.  I assume this means a similar first-in wins strategy as 
 well.
 
 Is there a way in RPM to change this strategy to create a true conflict 
 (install
 failure) to identify these situations?
 

There's no configurable way ... but changing opens a walloping amount
of pain because RPM will *again* have to justify its reliability if
change occurs.

I plain and simply don't give a hoot unless a sound
engineering reason is given for the change.

Change for the sake of change isn't a sound reason, nor is consistency with
the Debian Borg mind meld, nor is introducing Yet Another Way That RPM
Doesn't Install Anything because of lack of vendor QA.

 Also, what is the impact to RPM if a lot of packages all own the same
 directories (such as /bin, /lib, /usr, etc..)?  [or alternatively have no
 packages own the directories and fix the owner/group/mode with some type of
 scripting]

Lemme answer a question with a question:

Why are you asking? Are you seeing impact or just enquiring?

Owning all directories in every package so that
rpm -qa | wc -l
and
rpm -qf / | wc -l
are identical is what I think should be done on even days of the month.

On odd days of the month, I tend towards the schizoid alterantive POV, that
no directory should be in packages.

And on leap days I use dpkg ...

 
 Any help understanding RPM's behavior in these conditions performance or
 otherwise will help me figure out how to reconcile the behaviors.. thanks!
 

Everything that rpm does while installing is displayable with

rpm -Uvv --fsmdebug *.rpm

Any quirks aren't from the state machine, but rather from foolish
fiddle ups for unowned directory behavior w SELinux and other insanities.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: About ACL_FIRST_ENTRY, etc in rpm-5.3.11

[Jeff Johnson]

On Jun 9, 2011, at 11:24 AM, Sriram Narayanan wrote:

 
 The Solaris world doesn't have these ACL_TYPE definitions at
 /usr/include/sys/acl.h or elsewhere.
 

Hmmm ... will look.

 I presume that rpmacl is needed for ensuring that the user account has
 the right level of access, etc.
 
 Any guidance on how to resolve this ?
 

I'd just rip out rpmio/rpmacl.c by patching in

#if 0
...
#else
   return 0;
#endif

or whatever works. There's nothing on critically important code
paths that RPM needs/uses.

The rpmio/rpmacl.c was dragged in by *BSD find(1) iirc (and I looked
a bit at extending RPM to handle ACL's but I have no wish to trailblaze
into ACL management through *.rpm packaging. ACL's == configuration and
need to be done outside of packaging.)

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: A syntax (?) question

[Jeff Johnson]

On Jun 9, 2011, at 12:28 PM, Sriram Narayanan wrote:

 Another C n00b question:
 
 I faced this:
 
 libtool: link: gcc -shared  -fPIC -DPIC -Wl,-z -Wl,text -Wl,-h
 -Wl,librpmmisc-5.3.so -o .libs/librpmmisc-5.3.so  .libs/librpmmisc.o
 -R/workspace/rpm/rpm_5_3_11/beecrypt/.libs -R/usr/sfw/lib
 -R/workspace/rpm/rpm_5_3_11/neon/src/.libs
 -R/workspace/rpm/rpm_5_3_11/pcre/.libs
 -R/workspace/rpm/rpm_5_3_11/popt/.libs -R/workspace/altopt/lib
 -R/usr/sfw/lib -L/workspace/rpm/rpm_5_3_11/pcre/.libs -L../beecrypt
 -L../neon/src -L../neon -L../pcre -L../popt -L/workspace/altopt/lib
 -L/usr/gnu/lib -L/lib
 /workspace/rpm/rpm_5_3_11/beecrypt/.libs/libbeecrypt.so
 -L/usr/sfw/lib/gcc/i386-pc-solaris2.11/3.4.3/../../.. -L/usr/sfw/lib
 /usr/sfw/lib/libstdc++.so
 /workspace/rpm/rpm_5_3_11/neon/src/.libs/libneon.so
 /workspace/rpm/rpm_5_3_11/pcre/.libs/libpcreposix.so
 /workspace/rpm/rpm_5_3_11/pcre/.libs/libpcre.so
 /workspace/rpm/rpm_5_3_11/popt/.libs/libpopt.so -lpython2.4
 -lpython2.6 -lexpat -lcrypto -lbz2 -lz -lpthread -lsocket -lnsl -lelf
 -lgcrypt -lgpg-error -lc  -O2 -Wl,--version-script=./librpmmisc.vers
 ld: fatal: unrecognized option '--'
 

Ah this is loader maps in use so that RPM doesn't accidentally
re-export an entire library (like Berkeley DB or BeeCrypt or neon or ...)
that creates some accidental symbol collisions.

Again the easiest fix is just to disable loader maps (I'm forced in
development to carry maximum features to ensure no bit rot ...)

That can be done with a configure option ... lemme find (untested) ...

$ ./configure --help
...
  --enable-build-versionscript
  enable/disable use of GNU ld(1) version script
  (default is system dependent)

So --disable-build-versionscript SHOULD (again untested by me) the loader
map whose syntax is apparently causing problems with the Solaris linker.


 I then edited misc/Makefile
 replacing - am__append_1 = -Wl,--version-script=$(srcdir)/librpmmisc.vers
 with - am__append_1 = -Wl --version-script=$(srcdir)/librpmmisc.vers
 
 I replaced the comma with a space.
 

Likely its not just s/,/ /. You would need to look at the ld(1)
man page to find out how to pass a symbol map, and there's likely
going to be other deep dark chocolate issues almost instantly.

RPM won't behave any differently with/without loader maps. The
issue is solely to avoid accidental symbol pollution Just In Case.

The hard implementation is/was using loader maps, other issues such as changing
symbol names if/when there is an issue can be addressed as found.

No issues been reported for years ...

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: How do I get the rpm build to see the external db and dbsql files ?

[Jeff Johnson]

On Jun 9, 2011, at 2:59 PM, Sriram Narayanan wrote:

 I have bdb 5.1.19 installed at /workspace/altopt/ (with sub folders
 being include, bin, lib, etc).

Note that there is a ABI breakage between db-5.1.19 - db-5.1.25. You
might want to start using db-5.1.25 to avoid later pain.
\
ABI breakage == you MUST recompile everything linked against
db-5.1.19 in order to move to db-5.1.25 or you WILL segfault.

 /workspace/altopt/include contains db.h and dbsql.h
 
 Here's a snippet of what I use to autogen and configure the rpm build process:
 /bin/sh ./autogen.sh
 ./configure CPPFLAGS=-I/workspace/altopt/include -I/usr/include
 LDFLAGS=-L/workspace/altopt/lib -L/usr/gnu/lib -L/lib \
  --prefix=/workspace/altopt \
  --enable-shared \
  --with-db=external \
  --with-dbsql=external \
  --without-db-tools-integrated \
  --with-zlib \
  --with-bzip2 \
  --with-xz \
 
 
 And yet I see that ./configure looks for a subfolder db51 and tries to
 use that. This is resulting in the build not finding the DB_SEQUENCE
 data structure, resulting in a build failure.
 
 What else should I set such that the external DB is picked up ?
 

Yes. There are two places (iirc) that
#include db51/db.h
are hard wired.

Create a symlink on some include path to where ever you built Berkeley DB
is one easy way to avoid.

The other is to patch rpm (grep db51 */*.[ch])
to do something else instead.

Yet another is to untar db-5.1.X into the top level directory, do
mv db-5.1.x db
and build --with-db=internal (there's also --with-dbsql=internal needed).

hth

73 de Jeff

The hard wired path isn't likely to be changed because its hugely
complicated to test all possible ways of installing Berkeley DB.

If anything,
 -- Sriram
 ==
 Belenix: www.belenix.org
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: How do I get the rpm build to see the external db and dbsql files ?

[Jeff Johnson]

On Jun 9, 2011, at 3:30 PM, Sriram Narayanan wrote:

 On Fri, Jun 10, 2011 at 12:37 AM, Jeff Johnson n3...@mac.com wrote:
 
 On Jun 9, 2011, at 2:59 PM, Sriram Narayanan wrote:
 
 I have bdb 5.1.19 installed at /workspace/altopt/ (with sub folders
 being include, bin, lib, etc).
 
 Note that there is a ABI breakage between db-5.1.19 - db-5.1.25. You
 might want to start using db-5.1.25 to avoid later pain.
 \
 ABI breakage == you MUST recompile everything linked against
 db-5.1.19 in order to move to db-5.1.25 or you WILL segfault.
 
 
 Ok, I can definitely move to 5.1.25. However, the INSTALL document
 mentions 5.1.19, you may want to correct that.
 

todo++

 /workspace/altopt/include contains db.h and dbsql.h
 
 Here's a snippet of what I use to autogen and configure the rpm build 
 process:
 /bin/sh ./autogen.sh
 ./configure CPPFLAGS=-I/workspace/altopt/include -I/usr/include
 LDFLAGS=-L/workspace/altopt/lib -L/usr/gnu/lib -L/lib \
  --prefix=/workspace/altopt \
  --enable-shared \
  --with-db=external \
  --with-dbsql=external \
  --without-db-tools-integrated \
  --with-zlib \
  --with-bzip2 \
  --with-xz \
 
 
 And yet I see that ./configure looks for a subfolder db51 and tries to
 use that. This is resulting in the build not finding the DB_SEQUENCE
 data structure, resulting in a build failure.
 
 What else should I set such that the external DB is picked up ?
 
 
 Yes. There are two places (iirc) that
#include db51/db.h
 are hard wired.
 
 Create a symlink on some include path to where ever you built Berkeley DB
 is one easy way to avoid.
 
 The other is to patch rpm (grep db51 */*.[ch])
 to do something else instead.
 
 Yet another is to untar db-5.1.X into the top level directory, do
mv db-5.1.x db
 and build --with-db=internal (there's also --with-dbsql=internal needed).
 
 
 The directory structure that the rpmbuild expects (at least db51/db.h)
 is different from what db-5.1.19 provides.
 

Yes. If built internally, then the build ends up with ./db3/db.h being included.

You *might* need
ln -s db3 db51 in the to[p level directory.

 What is your folder structure for bdb ?
 
 Also, which version are you testing with ?
 

I use this db51.spec and packaging (or some modest variant thereof).

I test against db-5.1.25 (but can/have used db-5.1.19, I switched
when Mandriva cooker swithced).

falmouth:tmp jbj$ cvs -d :pserver:anonym...@rpm5.org:/v/rpm/cvs get distro/db51
U distro/db51/db-1.85-errno.patch
U distro/db51/db-4.5.20-jni-include-dir.patch
U distro/db51/db-4.6.21-1.85-compat.patch
U distro/db51/db51.spec
U distro/db51/patch.1.1
U distro/db51/patch.1.2
U distro/db51/patch.1.3
U distro/db51/patch.1.4

Add the upstream db-5.1.25 and you should be good to go.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/ CHANGES configure.ac rpm/macros/ macros.in

[Jeff Johnson]

The AutoFu is perfectly okay, but there's deeper design problems here.

First of, there's additional configuration that needs to be automated.

There are 2 usage cases, one for files, the other for package scriptlets,
for bash --rpm-requires

1) for files
The files have only %_foo_requires, there is no %_foo_provides. The
reason there is that
Requires: executable(whatever)
is a run-time probe modeled on which(1) (and uglix PATH envvar):

The executable(whatever) dependency is TRUE iff
a) the executable is found on PATH (or is absolute path)
b) access(2) claims the object on path is executable.

I.e. The semantic for executable(whatever) is defined as a run-time
condition, not as a dependency assertion string match from package metadata.

The enabling for this extractor follows the usual conventions for disabling
by undefining or setting to %{nil}. The sad reality is that almost all 
distros
I'm aware of are choosing opt-in behavior akin to Waiting for Godot 
The implementation has been done since forever.

2) for package scriptlets
The automated scriptlet dependencies also have no provides, again because
executable(whatever) is a run-time probe that checks a condition not
an essertion string.

The enabling for automating scriptlet dependencies is OR'ing in 0x2 to

%_use_internal_dependency_generator

(0x1 enables ELF extraction and disables %_find_requires for ELF, 0x4 was
used to prepare LSB packages until lack of interest from LSB in their
own chosen format prevented finishing the implementation).

Both of the above configuration changes need to be tied into AutoFu if you
choose to automate from configure.ac (or no problem has really been automated 
or solved).

I personally think that it isn't worth adding AutoFu for complex features that
are routinely disabled (or rendered as default opt-in which is de facto 
disabled).

hth

73 de Jeff



On Jun 7, 2011, at 8:35 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   07-Jun-2011 14:35:30
  Branch: HEAD Handle: 2011060712352900
 
  Modified files:
rpm CHANGES configure.ac
rpm/macros  macros.in
 
  Log:
define the scriplet_requires macro
as /bin/bash --rpm-requires only if bash have the
required patch.
 
Submitted by: Elia Pinto
 
  Summary:
RevisionChanges Path
1.3652  +3  -0  rpm/CHANGES
2.497   +16 -0  rpm/configure.ac
1.57+2  -2  rpm/macros/macros.in
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3651 -r1.3652 CHANGES
  --- rpm/CHANGES  27 May 2011 13:47:29 -  1.3651
  +++ rpm/CHANGES  7 Jun 2011 12:35:29 -   1.3652
  @@ -8,6 +8,9 @@
   - afb: yaml: use !!timestamp for timestamps
 
   5.4.0 - 5.4.1:
  +- devzero2000: define the scriplet_requires macro
  +  as /bin/bash --rpm-requires only if bash have the
  +  required patch. 
   - devzero2000: In fpLookupSubdir, data returned by hash should be of 
 type
 struct rpmffi_s ** instead of struct rpmffi_s * to avoid
 segfault. Patch by Qing He.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/configure.ac
  
  $ cvs diff -u -r2.496 -r2.497 configure.ac
  --- rpm/configure.ac 26 May 2011 14:26:26 -  2.496
  +++ rpm/configure.ac 7 Jun 2011 12:35:29 -   2.497
  @@ -390,6 +390,22 @@
 AC_DEFINE([RPMVERCMP_DIGITS_BEAT_ALPHA],[1],[Permit in RPM version 
 comparison that digits beat alpha])
 ])
 
  +dnl # Check if bash have the --rpm-requires patch
  +dnl # If so define the scriptlet_requires output variable 
  +dnl # to bash --rpm-requires  
  +dnl # and BASH_RPM_REQUIRES to the bash absolute path  
  +AC_CACHE_CHECK([for bash if supports --rpm-requires], 
 [ac_cv_path_BASH_RPM_REQUIRES],
  +  [AC_PATH_PROGS_FEATURE_CHECK([BASH_RPM_REQUIRES], [bash],
  +[[ bash_rpm_requires_out=`echo ls | $ac_path_BASH_RPM_REQUIRES 
 --rpm-requires | grep executable(ls)`
  +  test x$bash_rpm_requires_out != x \
  +   ac_cv_path_BASH_RPM_REQUIRES=$ac_path_BASH_RPM_REQUIRES 
 scriptlet_requires=$ac_path_BASH_RPM_REQUIRES --rpm-requires 
  +]],
  +[[   scriptlet_requires=%{nil} ]], [$PATH] 
  +)])
  +AS_IF([test x$ac_cv_path_BASH_RPM_REQUIRES != x],
  +  [AC_SUBST([BASH_RPM_REQUIRES],[$ac_cv_path_BASH_RPM_REQUIRES])],
  +  

Re: [CVS] RPM: rpm-5_4: rpm/ CHANGES configure.ac rpm/macros/ macros.in

[Jeff Johnson]

Please note that this also just broke every
build in devtool.conf (because an additional
build argument, with path to a bash enabled
with --rpm-requires is added.

There's a point at which increasingly precise
AutoFu and Have it your own way! configuration
only increases the difficulty of developing RPM
and using continous intergration on multiple platforms.

The simpler approach is choosing MANDATORY behavior,
either NEVER or ALWAYS doing automated scriptlet
dependency extraxtion, and (if ALWAYS my preference)
then distributing rpmbuild with a patched version
of bash that has --rpm-requires availaible in
/usr/lib/rrpm/bin/bash

I WILL go to distributing bash with rpmbuild becuase
its an order of magnitude less work than attempting
to keep up with the endless
Have it your own way!
AutoFu under continuous integration buildbots on
multiple platforms (what I've been wasting my time on
for more than 2 years now).

hth

73 de Jeff

On Jun 7, 2011, at 10:01 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   07-Jun-2011 16:01:52
  Branch: rpm-5_4  Handle: 2011060714015001
 
  Modified files:   (Branch: rpm-5_4)
rpm CHANGES configure.ac
rpm/macros  macros.in
 
  Log:
define the scriplet_requires macro
as /bin/bash --rpm-requires only if bash have the
required patch.
 
Submitted by: Elia Pinto
 
  Summary:
RevisionChanges Path
1.3501.2.136+3  -0  rpm/CHANGES
2.472.2.22  +16 -0  rpm/configure.ac
1.39.2.13   +2  -2  rpm/macros/macros.in
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3501.2.135 -r1.3501.2.136 CHANGES
  --- rpm/CHANGES  7 Jun 2011 13:11:42 -   1.3501.2.135
  +++ rpm/CHANGES  7 Jun 2011 14:01:50 -   1.3501.2.136
  @@ -1,4 +1,7 @@
   5.4.1 - 5.4.2
  +- devzero2000: define the scriplet_requires macro
  +  as /bin/bash --rpm-requires only if bash have the
  +  required patch. 
   - proyvind: don't pick up python version from python egg, rely on path
   only in stead to avoid incorrect version dependencies if any other
   egg metadata for other versions is found.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/configure.ac
  
  $ cvs diff -u -r2.472.2.21 -r2.472.2.22 configure.ac
  --- rpm/configure.ac 2 Jun 2011 21:15:21 -   2.472.2.21
  +++ rpm/configure.ac 7 Jun 2011 14:01:51 -   2.472.2.22
  @@ -390,6 +390,22 @@
 AC_DEFINE([RPMVERCMP_DIGITS_BEAT_ALPHA],[1],[Permit in RPM version 
 comparison that digits beat alpha])
 ])
 
  +dnl # Check if bash have the --rpm-requires patch
  +dnl # If so define the scriptlet_requires output variable 
  +dnl # to bash --rpm-requires  
  +dnl # and BASH_RPM_REQUIRES to the bash absolute path  
  +AC_CACHE_CHECK([for bash if supports --rpm-requires], 
 [ac_cv_path_BASH_RPM_REQUIRES],
  +  [AC_PATH_PROGS_FEATURE_CHECK([BASH_RPM_REQUIRES], [bash],
  +[[ bash_rpm_requires_out=`echo ls | $ac_path_BASH_RPM_REQUIRES 
 --rpm-requires | grep executable(ls)`
  +  test x$bash_rpm_requires_out != x \
  +   ac_cv_path_BASH_RPM_REQUIRES=$ac_path_BASH_RPM_REQUIRES 
 scriptlet_requires=$ac_path_BASH_RPM_REQUIRES --rpm-requires 
  +]],
  +[[   scriptlet_requires=%{nil} ]], [$PATH] 
  +)])
  +AS_IF([test x$ac_cv_path_BASH_RPM_REQUIRES != x],
  +  [AC_SUBST([BASH_RPM_REQUIRES],[$ac_cv_path_BASH_RPM_REQUIRES])],
  +  [AC_SUBST([BASH_RPM_REQUIRES],[:])])
  +AC_SUBST([scriptlet_requires])
   dnl # FIXME: partly Linux-specific only!?
   dnl # (but at least it usually doesn't harm on other platforms)
   CFLAGS=$CFLAGS -D_GNU_SOURCE -D_REENTRANT
  @@ .
  patch -p0 '@@ .'
  Index: rpm/macros/macros.in
  
  $ cvs diff -u -r1.39.2.12 -r1.39.2.13 macros.in
  --- rpm/macros/macros.in 11 Apr 2011 00:34:48 -  1.39.2.12
  +++ rpm/macros/macros.in 7 Jun 2011 14:01:51 -   1.39.2.13
  @@ -1,7 +1,7 @@
   #/*! \page config_macros Default configuration: @USRLIBRPM@/macros
   # \verbatim
   #
  -# $Id: macros.in,v 1.39.2.12 2011/04/11 00:34:48 jbj Exp $
  +# $Id: macros.in,v 1.39.2.13 2011/06/07 14:01:51 devzero2000 Exp $
   #
   # This is a global RPM configuration file. All changes made here will
   # be lost when the rpm package is upgraded. Any per-system configuration
  @@ -925,7 +925,7 @@
   # helpers are also used by %{_rpmhome}/rpmdeps 

Re: [CVS] RPM: rpm/js/ rpmaug-js.c rpmbc-js.c rpmbf-js.c rpmcudf-js.c rpmd...

[Jeff Johnson]

 
 patch -p0 '@@ .'
 Index: rpm/js/rpmaug-js.c
 
 $ cvs diff -u -r1.12 -r1.13 rpmaug-js.c
 --- rpm/js/rpmaug-js.c   2 Apr 2011 02:01:53 -   1.12
 +++ rpm/js/rpmaug-js.c   3 Jun 2011 16:32:15 -   1.13
 @@ -29,8 +29,14 @@
  /* XXX does aug_defnode() need binding? */
  /* XXX unclear whether aug.defvar(foo, bar) or aug.foo = bar is 
 better */
  static JSBool
 -rpmaug_defvar(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval 
 *rval)
 +rpmaug_defvar(JSContext *cx, uintN argc, jsval *vp)

There are instant portability issues here.

  {
 +jsval *argv = JS_ARGV(cx , vp);
 +JSObject *obj = JS_NewObjectForConstructor(cx , vp);
 +if(!obj) {
 +JS_ReportError(cx , Failed to create 'this' object);
 +return JS_FALSE;
 +}

This needs to be a macro.

  void * ptr = JS_GetInstancePrivate(cx, obj, rpmaugClass, NULL);
  rpmaug aug = ptr;
  JSBool ok = JS_FALSE;
 @@ -49,7 +55,7 @@
  case 1: /* success */
  /* XXX return NAME or EXPR on success?  or bool for success/failure? */
  /* XXX hmmm, bool and string mixed returns. */
 -*rval = STRING_TO_JSVAL(JS_NewStringCopyZ(cx, _name));
 +JS_SET_RVAL(cx, vp, STRING_TO_JSVAL(JS_NewStringCopyZ(cx, _name)));
  break;

And the style was chosen for portability reasons. There are _SERIOUS_
issues targeting any single -ljs (even if JS 1.8.5 is the current known best).

And the rest is mostly ditto ditto ditto afaict (not looked).

The real problem that needs to be solved _FIRST_ isn't in the code,
but rather with the modularization, and until you attempt GPSEE
and its module loading you plain and simply will _NOT_ see the
real issues in need of solving.

And even before attempting the FIRST issue, I ask
Do you have any idea -- except from de facto Compiles: ship it! --
that any of your changes actually function?
If you haven't run any of the fairly complete test scripts, well, all you've
done is just make a whopping big mess by voiding out any ability to diff 
sources.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/js/ rpmaug-js.c rpmbc-js.c rpmbf-js.c rpmcudf-js.c rpmd...

[Jeff Johnson]

On Jun 3, 2011, at 12:44 PM, Per Øyvind Karlsen wrote:

 2011/6/3 Jeff Johnson n3...@mac.com:
 Talk to me first *PLEASE* ... nothing at all wrong with what you
 are doing _EXCEPT_ you aren't talking.
 Yeah, I just realised that I left it quite broken earlier and figured that
 I'd might as well just go ahead fixing it..
 

So let's try and get some reasonable plan in place first 

Now that you more or less know the code and what JS modules have
been written, do this

Choose _ONE_ module.

and rip out (by commenting out in Makefile.am) every other module.

I'd suggest something like rpmbf-js.c because the Bloom filter methods
are easy to understand, and its the ctors/dtors and getter/setter and
calls that need to be worked through first. There are other equally
simple modules, but the design issues start to get complex in a hurry,
because JS comes with ABSOLUTELY NO BATTERIES INCLUDED. I.e. no octet
data type, no I/O, no system calls, no NOTHING. And it is NOT obvious
how to retrofit bog-standard binding functionality into JS both
portably and successfully WHATSOEVER.

Please do _NOT_ choose the Berkeley DB JS bindings because that's
way way way too hard to hack on initially.

After choosing _ONE_ module, you need to get a module runner in place so that 
you can
actually run what is tscripts/Bf.js to exercise whatever you are doing.

You have two choices:

1) target JS 1.8.5 directly and punt on GPSEE module loading.

2) go get GPSEE - JS 1.8.5 built and fucntional an punt on modules.


The 2) is the critical path forward to RPM+GPSEE+JS, but 1) is perfectly
okay if you want to dink with JS.

The script runner for 2) is gsr.c/ngsr.c, and what is needed is careful
thought about how ELF symbols from multiple loaded modules start to work
(which is going to be a bit complex, whatever you *think* needs doing
is likely wrong. That is based on my deirect primary experience, several
attempts to solve the issues have been tried already.)

The script runner for 1) is tjs.c, which basically substitutes linking
for module loading, so that one can focus solely on module development,
not design issues and paths and common serevr-js political standards
and such.

Find an older copy of tjs.c in CVS, edit the table that looks like this

/*@unchecked@*/ /*@observer@*/
static struct rpmjsClassTable_s classTable[] = {
{ Aug,rpmjs_InitAugClass,  25 },
{ Bc, rpmjs_InitBcClass,   40 },  /* todo++ */
{ Bf, rpmjs_InitBfClass,   26 },

{ Cudf,   rpmjs_InitCudfClass, -50 }, /* todo++ */

{ Db, rpmjs_InitDbClass,   -45 },
{ Dbc,rpmjs_InitDbcClass,  -46 },
...

and either delete everything but the Bf (or whatever you have chosen)
module, or negate the 3rd number (to disable testing).

Lather runse repeat intul either ngsr/tjs actually runs the test cases
for rpmbf-js.c (or whatever you have chosen) in tscripts/Bf.js

DO NOT PASS GO, DO NOT COLLECT $200, UNTIL YOU HAVE A TEST HARNESS IN PLACE.

please ;-)

Sound like a plan?

73 de Jeff

 For the actual changes, I seem to have only partly ported some of the
 code to the newer API..(!)
 I'm not sure what might've happened, my guess is that I probably had
 both the libraries and the headers of both versions installed, then it
 must've included them both and managed to build and link it using
 some headers from newer version together with the older or
 something..
 
 
 There's a a couple issues remaining (ie. JS_PushArguments() being
 removed from API) and some memleaks due to API changes that
 still needs to be fixed, I've left those with a #warning in the code
 for now..
 
 --
 Regards,
 Per Øyvind
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: annoying yet?

[Jeff Johnson]

On May 31, 2011, at 3:26 PM, Robert Xu wrote:

 Hi all (again),
 
 This... patch is confusing me. Especially since it changes between
 rpm4.8 and rpm4.9, I do not know where to look:
 
 https://github.com/devzero2000/RPM/commit/1d6e4fcd667472dc9d252194b9a12a390aa32abb
 

There's not really much happening in this patch.

The hardest part of a build is getting the %files manifest correct.

All the patch does is try to _ALSO_ do the checkfiles()
sanity check so that _ALL_ the errors can be dealt with,
rather than having to go through the tedium of
lather ... rinse ... repeat

So its mostly a NICETOHAVE, and rather minor tweak, rather
than anything essential.

Yes there's often internal rearrangements of code in rpmbuild
that are difficult to understand applying patches but are otherwise
mostly cosmetic.

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: annoying yet?

[Jeff Johnson]

On May 31, 2011, at 4:14 PM, Robert Xu wrote:

 On Tue, May 31, 2011 at 16:01, Jeff Johnson n3...@mac.com wrote:
 
 There's not really much happening in this patch.
 
 The hardest part of a build is getting the %files manifest correct.
 
 All the patch does is try to _ALSO_ do the checkfiles()
 sanity check so that _ALL_ the errors can be dealt with,
 rather than having to go through the tedium of
lather ... rinse ... repeat
 
 So its mostly a NICETOHAVE, and rather minor tweak, rather
 than anything essential.
 
 That would be easier for anyone, so I'm going to try and figure out
 this patch...
 

The best effort to report all errors is usually (but not in this case)
already implemented @rpm5.org. The reason why the patch doesn't apply
directly is that there's already better/saner error checks than just
running the script find if there are files in %{buildroot} that
aren't in %files.


 
 Yes there's often internal rearrangements of code in rpmbuild
 that are difficult to understand applying patches but are otherwise
 mostly cosmetic.
 
 Hmm... so at files.c line 2896, this would be ok?
 
   if ((*t != '\0')  (*t != '\n')) {
   rc = (_unpackaged_files_terminate_build) ? 1 : 0;
   rpmlog((rc ? RPMLOG_ERR : RPMLOG_WARNING),
   _(Installed (but unpackaged) file(s) found:\n%s), t);
   }
 +else
 +checkUnpackagedFiles(spec);
 

Not quite (but you are in the right place for the change).

Adding your patch would end up recursing (by calling the same routine).

What would be needed is to find every occurrence of goto error
in the checkUnpackagedFiles() routine and to save the global error
code while continuing through the 3 checks that are implemented.

I could likely type out a patch in 15 minutes (but I'm not atm
in any position to test and to get the change checked in on multiple branches
largely because the monthly release from @rpm5.org happens today/tomorrow).

But holler at me if I forget to do the change next month.

No matter what: The patch is truly a minor NICETOHAVE, not anything super 
important.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: ugly suse build.diff

[Jeff Johnson]

On May 29, 2011, at 5:41 PM, Robert Xu wrote:

 Hi all,
 
 Would like some words of wisdom ( or such ) on this patch:
 https://github.com/devzero2000/RPM/commit/cfdfc6e358c47d0b6fef2fe4f99d67ca7922ffed
 

Most of this patch is attempting to automate suse identification
and add default packaging policyconfiguration.

SInce thepolicy is invariably re-configured through other means,
and because the vendor identifier has already been added
(still needs to be resolved whether opensuse or suse is correct),
most of the patch can be simply ignored.

You can run rpm --showrc later and diff against some reference
implementation to see what might not be compatible any time.

hth

73 de Jeff


smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/rpmdb/ hdrfmt.c

[Jeff Johnson]

The claim mapping is not ordered  actually isn't true.

Header tags _ARE_ ordered, just the collation isn't
what no brainer conversions into native data types
are implementing.

But it hardly matters with spewage, fewer tokens to ask
about KISS simplicity trumps everything else in FL/OSS.

73 de Jeff

On May 27, 2011, at 2:29 AM, Anders F. Björklund wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Anders F. Björklund
  Root:   /v/rpm/cvs   Email:  a...@rpm5.org
  Module: rpm  Date:   27-May-2011 08:29:15
  Branch: HEAD Handle: 2011052706291500
 
  Modified files:
rpm/rpmdb   hdrfmt.c
 
  Log:
yaml: mapping is not ordered (= sequence of singles)
 
  Summary:
RevisionChanges Path
1.167   +1  -1  rpm/rpmdb/hdrfmt.c
  
 
  patch -p0 '@@ .'
  Index: rpm/rpmdb/hdrfmt.c
  
  $ cvs diff -u -r1.166 -r1.167 hdrfmt.c
  --- rpm/rpmdb/hdrfmt.c   26 May 2011 17:51:55 -  1.166
  +++ rpm/rpmdb/hdrfmt.c   27 May 2011 06:29:15 -  1.167
  @@ -474,7 +474,7 @@
   /*@unchecked@*/ /*@observer@*/ 
   static const struct spew_s _yaml_spew = {
   .spew_name  = yaml,
  -.spew_init  = - !!omap\n,
  +.spew_init  = -\n,
   .spew_fini  = \n,
   .spew_strlen= yamlstrlen,
   .spew_strcpy= yamlstrcpy
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/rpmdb/ hdrfmt.c

[Jeff Johnson]

On May 27, 2011, at 8:53 AM, Anders F Björklund wrote:

 Jeff Johnson wrote:
 
 The claim mapping is not ordered  actually isn't true.
 
 Header tags _ARE_ ordered, just the collation isn't
 what no brainer conversions into native data types
 are implementing.
 
 I was referring to the actual YAML output...
 
 Currently it is outputting:

And the collation is integer numeric based on tag numbers.

 ac0c428101b6ea299e5d42b7c242c91c059b02c4:
  Name:rpm
  Arch:i586
  Version: 5.3.9
  Epoch:   1
  Release: 0.20110330.6
  ...
 
 If this was to be an ordered mapping, it needs:
 ac0c428101b6ea299e5d42b7c242c91c059b02c4: !!omap
  - Name:rpm
  - Arch:i586
  - Version: 5.3.9
  - Epoch:   1
  - Release: 0.20110330.6
  ...
 
 So the YAML parser was failing at input ?
 

There's a s superficial and a deep answer here.

SOmehow it needs to be indicated that header speawage
is _NOT_ random, but rather carefully sorted in many
ways.

The LSB packaging standard totally blew it with respect to
tag ordering.

And the spewage -- if not carefully controlled -- will be useless
for RPM itself, whose task is to import/export through speawage
into a header blob.

 But it hardly matters with spewage, fewer tokens to ask
 about KISS simplicity trumps everything else in FL/OSS.
 
 For the YAML and JSON formats, it's easier unordered.

Easier for whom? Lusers who don't undertsand what
canoniocally represented plaintext actually means?

Or why sorted data can be accessed in logN not linear time?

 And I think the XML would need a DTD, to do ordering ?
 

No idea what XML needs. I do know from rpmrepo that
its _IMPOSSIBLE_ to be bit for bit compatible because
tag data is being run through a python dict which
_DESTROYS_ the ordering of the original data.

 i.e. XML does ordering now, but I think a parser is
 free to reorder the elements without invalidating ?
 

Please note that I'm disagreeing with your patch whatsoever.

But somehow and somewhere it needs to be hinted to all
the spewage suckers that there are most definitely
performance and interoperability wins by establishing
a sorted and canonical ordering on the spewage items.

Yes I know how to use qsort(3) wherever needed. I'm enetrested
in proper spewage specification on which it becomes feasible
to define digests/signatures and simplify interoperability
and implementations. And most definitely I'm not holding my breath
waiting for FedEx to ship me a pony ...

73 de Jeff
 --anders
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/rpmdb/ hdrfmt.c

[Jeff Johnson]

On May 27, 2011, at 9:36 AM, Anders F Björklund wrote:

 Jeff Johnson:
 
 Header tags _ARE_ ordered, just the collation isn't
 what no brainer conversions into native data types
 are implementing.
 
 And the collation is integer numeric based on tag numbers.
 
 Okay. Does this tag ordering need to be preserved ?

All depends on context.

*IF* there is a canonical representation in the spewage (basically
a defined, not defacto, order) that standard plaintext digests
and signatires can be retrofitted to secure the spewage and
achieve interoperality. The alternative de facto approach is
defacto createrepo toolchain lock-in.

 If so, we should change the --yaml and --json now.
 

Its too early to finalize CLI options until the goals
interoperability through stricter spewage definitions
are well understood. Basically all I'm saying is the
same as DER and BER (and PER and ...) encodings for
spewage where one doesn't have the luxury of ASN.1 standards
when dealing with spewage. See the lengths that XML-SEC
has to specify what is essentially a retrofitted canonical
definition of plaintext on which signatures can be defined.

But yes, --yaml and --json would need to change as the usage
case and goals are more clearly understood.

 There's a s superficial and a deep answer here.
 
 SOmehow it needs to be indicated that header speawage
 is _NOT_ random, but rather carefully sorted in many
 ways.
 
 All the sequences/arrays maintain their sort order,
 but the mappings/objects do not (they're unordered)
 

Yes. but its implementation defined for YAML iirc, and
bindings are free to interpret !!omap however they wish.
So a mapping might be a sorted array instead of a hash
table, depending on implementation.

Meanwhile these are largely (imho) moot technically obscure discussions
that can only meaningfully be answered by looking  at the real world
of usage cases and implementations.

 The LSB packaging standard totally blew it with respect to
 tag ordering.
 
 And the spewage -- if not carefully controlled -- will be useless
 for RPM itself, whose task is to import/export through speawage
 into a header blob.
 
 If spewage is to preserve tag ordering, then the
 currently used markup/schema needs to be changed.
 
 From:
 {
  Tag1: Value1,
  Tag2: Value2,
  Tag3: Value3
 }
 
 To:
 [
  { Tag1: Value1 },
  { Tag2: Value2 },
  { Tag3: Value3 }
 ]
 
 This will make it slightly trickier to handle,
 but it will preserve the order of the keys/tags.
 

Yes, a specification that ALSO defines the ordering starts
to become pretty complex. See XML-SEC.

 But it hardly matters with spewage, fewer tokens to ask
 about KISS simplicity trumps everything else in FL/OSS.
 
 For the YAML and JSON formats, it's easier unordered.
 
 Easier for whom? Lusers who don't undertsand what
 canoniocally represented plaintext actually means?
 
 Or why sorted data can be accessed in logN not linear time?
 
 Easier formats, i.e. not needing nested structures ?
 

You are correct that RPM metadata doesn't need all the generality
provided by various spewage formats.

 But if it's needed, it's needed. I thought it wasn't.
 

The specific usage case that I see short-term is Poky/Ycto.

Instead of using *.spec templating, YAML (or XML or JSON) would
be used as a better (than *.spec) templating for driving
packaging (i.e. just producing *.rpm from a build not performed
by rpmbuild).

The risk there is that almost instantly not just Poky/Yacto will be attempting
to produce *.rpm packages from markup and so I worry up front about
issues like
How SHOULD the ordering criteria be hinted?
Its not OPTIONAL: *.rpm data has all sorts of implicit
constraints, and you will NOT be happy just typing up
some markup and feeding that spewage to a backend that
attempts to produce *.rpm package from %{buildroot} and markup.

 And I think the XML would need a DTD, to do ordering ?
 
 
 No idea what XML needs. I do know from rpmrepo that
 its _IMPOSSIBLE_ to be bit for bit compatible because
 tag data is being run through a python dict which
 _DESTROYS_ the ordering of the original data.
 
 Right. The same goes for using a mongo document iirc ?
 

No. A python dict is a hash, and the loss of sort order comes
from walking hash buckets serially.

A document structured MongoDB has the ability to add an
ordering key that a python dict (as used in createrepo) does not.

But yes explicit means to preserve order WILL need to be undertaken
to simplify generating header blob's (which is also not
the general, but rather the de facto first and most common
container representation in use by RPM where order _IS_ important).

 i.e. XML does ordering now, but I think a parser is
 free to reorder the elements without invalidating ?
 
 
 Please note that I'm disagreeing with your patch whatsoever.
 
 But somehow and somewhere it needs to be hinted to all
 the spewage suckers that there are most definitely
 performance and interoperability wins by establishing
 a sorted

Re: [CVS] RPM: rpm/ CHANGES rpm/rpmdb/ fprint.c

[Jeff Johnson]

Just FYI:

These are deep internal interfaces which should NOT
be relied on within RPM.

I.e. if you're segfaulting here you need an RPM upgrade, not a patch.
The fingerprinting code is NOT a supportable API, and IS being
actively refactored to be replaced and eliminated.

This code path is explicitly exercised as part of monthly
releases, and the fact that the segfault wasn't seen indicates
a data driven obscure code path that SHOULD be looked at
carefully in the packaging, not in RPM.

Short answer: You can expect further breakage.

73 de Jeff


On May 27, 2011, at 9:47 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   27-May-2011 15:47:30
  Branch: HEAD Handle: 2011052713472901
 
  Modified files:
rpm CHANGES
rpm/rpmdb   fprint.c
 
  Log:
In fpLookupSubdir, data returned by hash should be of type
struct rpmffi_s ** instead of struct rpmffi_s * to avoid
segfault. Patch by Qing He.
 
  Summary:
RevisionChanges Path
1.3651  +3  -0  rpm/CHANGES
1.41+3  -3  rpm/rpmdb/fprint.c
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3650 -r1.3651 CHANGES
  --- rpm/CHANGES  27 May 2011 07:19:54 -  1.3650
  +++ rpm/CHANGES  27 May 2011 13:47:29 -  1.3651
  @@ -8,6 +8,9 @@
   - afb: yaml: use !!timestamp for timestamps
 
   5.4.0 - 5.4.1:
  +- devzero2000: In fpLookupSubdir, data returned by hash should be of 
 type
  +  struct rpmffi_s ** instead of struct rpmffi_s * to avoid
  +  segfault. Patch by Qing He.
   - devzero2000: the elfutils --reloc-debug-sections is useful
 mostly for LKM and not in general. Better to enable
 when needed/useful and not systemwide.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/rpmdb/fprint.c
  
  $ cvs diff -u -r1.40 -r1.41 fprint.c
  --- rpm/rpmdb/fprint.c   6 Mar 2010 16:20:46 -   1.40
  +++ rpm/rpmdb/fprint.c   27 May 2011 13:47:30 -  1.41
  @@ -333,7 +333,7 @@
   *te = '\0';
 
   while (te  se) {
  -struct rpmffi_s * recs;
  +struct rpmffi_s ** recs;
   int numRecs;
   int i;
 
  @@ -346,8 +346,8 @@
   const char * link;
   int fx;
 
  -fx = recs[i].fileno;
  -fi =  recs[i].p-fi;
  +fx = recs[i]-fileno;
  +fi =  recs[i]-p-fi;
   flink = fi-flinks[fx];
   if (!(flink  *flink != '\0'))
   continue;
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm-5_3: rpm/rpmdb/ rpmdb.c

[Jeff Johnson]

Nice!

Still needs to be done somewhere else though for performance.
Nothing at all wrong with this patch, just SHOULD be done
deeper in dbiFindMatches() for highest performing, most general, etc etc.

73 de Jeff

On May 27, 2011, at 10:50 AM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   27-May-2011 16:50:27
  Branch: rpm-5_3  Handle: 2011052714502700
 
  Modified files:   (Branch: rpm-5_3)
rpm/rpmdb   rpmdb.c
 
  Log:
come up with a better hack for querying nvra with disttag/distepoch
that should be rather generic
 
  Summary:
RevisionChanges Path
1.386.2.12  +80 -12 rpm/rpmdb/rpmdb.c
  
 
  patch -p0 '@@ .'
  Index: rpm/rpmdb/rpmdb.c
  
  $ cvs diff -u -r1.386.2.11 -r1.386.2.12 rpmdb.c
  --- rpm/rpmdb/rpmdb.c24 Mar 2011 16:51:17 -  1.386.2.11
  +++ rpm/rpmdb/rpmdb.c27 May 2011 14:50:27 -  1.386.2.12
  @@ -2446,24 +2446,92 @@
   /* XXX Special case #4: gather primary keys with patterns. */
   rpmRC rc;
 
  +rc = dbiFindMatches(dbi, keyp, set);
   #if defined(RPM_VENDOR_MANDRIVA)
   /*
  - * ugly hack to workaround disttag/distepoch pattern matching issue to 
 buy some
  + * Hack to workaround disttag/distepoch pattern matching issue to buy 
 some
* time to come up with better pattern fix..
  + * One size should fit all now.. ;)
  + *
  + * This patch will try match NVR first, then for all matches returned,
  + * it will match disttag, distepoch  arch individually.
*/
  -const char *tmp = strstr(keyp, -mdv2011.0);
  -if(tmp) {
  -const char *origkeyp = keyp;
  -size_t klen = strlen(keyp);
  -keyp = alloca(klen);
  -memset((void*)keyp, klen, 0);
  -klen = tmp-origkeyp+1;
  -snprintf((char*)keyp, klen, %s, origkeyp);
  -if(strlen(tmp)  sizeof(-mdv2011.0)-1)
  -stpcpy((char*)keyp+(klen-1), tmp[sizeof(-mdv2011.0)-1]);
  +
  +/* We'll only try this if query fails */
  +if(!rc  ((const char*)keyp)[0] != '^'  tag == RPMTAG_NVRA 
  +(set == NULL || set-count  1)) {
  +size_t i;
  +char *tmp = (char*)keyp;
  +
  +/* If pattern has less than three '-', it can't contain disttag, so
  + * no point in trying */
  +for (i = 0; (tmp = strchr(tmp, '-')); i++, tmp++);
  +if (i = 3) {
  +dbiIndex pdbi;
  +DBC *pdbc;
  +const char *origkeyp = keyp;
  +size_t klen = strlen(keyp)+1;
  +size_t size = 0;
  +int xx;
  +
  +keyp = alloca(klen);
  +stpcpy((char*)keyp, origkeyp);
  +tmp = strrchr(keyp, '-');
  +*tmp = '\0';
  +rc = dbiFindMatches(dbi, keyp, set);
  +
  +pdbi = dbiOpen(db, RPMDBI_PACKAGES, 0);
  +xx = dbiCopen(pdbi, dbiTxnid(pdbi), pdbc, 0);
  +
  +for(i = 0; i  set-count; i++) {
  +DBT k = DBT_INIT;
  +DBT v = DBT_INIT;
  +Header h;
  +uint32_t offset = _hton_ui(set-recs[i].hdrNum);
  +rpmTag checkTags[] =
  +{ RPMTAG_DISTTAG, RPMTAG_DISTEPOCH, RPMTAG_ARCH };
  +int j;
  +
  +memset(k, 0, sizeof(k));
  +memset(v, 0, sizeof(v));
  +k.data = offset;
  +k.size = sizeof(offset);
  +
  +xx = dbiGet(dbi, pdbc, k, v, DB_SET);
  +h = headerLoad(v.data);
  +tmp = (char*)((size_t)keyp + strlen(keyp) + 1);
  +
  +for (j = 0; j  
 (int)(sizeof(checkTags)/sizeof(checkTags[0])) 
  +*tmp != '\0'; j++) {
  +he-tag = checkTags[j];
  +if(headerGet(h, he, 0)) {
  +size_t len = strlen(he-p.str);
  +
  +if (he-tag == RPMTAG_ARCH  *tmp == '.')
  +tmp++;
  +
  +if(!strncmp(he-p.str, tmp, len))
  +tmp += len;
  +_free(he-p.ptr);
  +}
  +}
  +if(j  *tmp  == '\0') {
  +set-recs[size].hdrNum = set-recs[i].hdrNum;
  +set-recs[size].tagNum = set-recs[i].tagNum;
  +size++;
  +}
  +
  +h = headerFree(h);
  +}
  +  

Re: [CVS] RPM: rpm/ CHANGES rpm/rpmdb/ fprint.c

[Jeff Johnson]

On May 27, 2011, at 11:04 AM, devzero2000 wrote:

 On Fri, May 27, 2011 at 4:53 PM, Jeff Johnson n3...@mac.com wrote:
 Just FYI:
 
These are deep internal interfaces which should NOT
be relied on within RPM.
 
 I.e. if you're segfaulting here you need an RPM upgrade, not a patch.
 The fingerprinting code is NOT a supportable API, and IS being
 actively refactored to be replaced and eliminated.
 
 I had imagined, in fact, I was not sure. 
 This code path is explicitly exercised as part of monthly
 releases, and the fact that the segfault wasn't seen indicates
 a data driven obscure code path that SHOULD be looked at
 carefully in the packaging, not in RPM.
 Ok, for now revert and will ask the POLKY (I think) developer the test case.

Nah, no reason to revert a patch for an RPM segfault whatsoever ;-)

But RPM is a canary in the mine shaft here. Something else in the packaging
needs to be looked at as well, and the fact that the segfault wasn't caught
by continuous integration needs to be examined and fixed too.

Likely gonna be a tricky reproducer ... perhaps the easiest
approach would be to just grab a pile of poky/yacto packages and
try de facto testing. I'm sure its gonna be hard to write a specific
test case.


hth

73 de Jeff

 
 Short answer: You can expect further breakage.
 
 
 73 de Jeff
 
 
 On May 27, 2011, at 9:47 AM, Pinto Elia wrote:
 
   RPM Package Manager, CVS Repository
   http://rpm5.org/cvs/
   
  
 
   Server: rpm5.org Name:   Pinto Elia
   Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
   Module: rpm  Date:   27-May-2011 15:47:30
   Branch: HEAD Handle: 2011052713472901
 
   Modified files:
 rpm CHANGES
 rpm/rpmdb   fprint.c
 
   Log:
 In fpLookupSubdir, data returned by hash should be of type
 struct rpmffi_s ** instead of struct rpmffi_s * to avoid
 segfault. Patch by Qing He.
 
   Summary:
 RevisionChanges Path
 1.3651  +3  -0  rpm/CHANGES
 1.41+3  -3  rpm/rpmdb/fprint.c
   
  
 
   patch -p0 '@@ .'
   Index: rpm/CHANGES
   
  
   $ cvs diff -u -r1.3650 -r1.3651 CHANGES
   --- rpm/CHANGES  27 May 2011 07:19:54 -  1.3650
   +++ rpm/CHANGES  27 May 2011 13:47:29 -  1.3651
   @@ -8,6 +8,9 @@
- afb: yaml: use !!timestamp for timestamps
 
5.4.0 - 5.4.1:
   +- devzero2000: In fpLookupSubdir, data returned by hash should be of 
  type
   +  struct rpmffi_s ** instead of struct rpmffi_s * to avoid
   +  segfault. Patch by Qing He.
- devzero2000: the elfutils --reloc-debug-sections is useful
  mostly for LKM and not in general. Better to enable
  when needed/useful and not systemwide.
   @@ .
   patch -p0 '@@ .'
   Index: rpm/rpmdb/fprint.c
   
  
   $ cvs diff -u -r1.40 -r1.41 fprint.c
   --- rpm/rpmdb/fprint.c   6 Mar 2010 16:20:46 -   1.40
   +++ rpm/rpmdb/fprint.c   27 May 2011 13:47:30 -  1.41
   @@ -333,7 +333,7 @@
*te = '\0';
 
while (te  se) {
   -struct rpmffi_s * recs;
   +struct rpmffi_s ** recs;
int numRecs;
int i;
 
   @@ -346,8 +346,8 @@
const char * link;
int fx;
 
   -fx = recs[i].fileno;
   -fi =  recs[i].p-fi;
   +fx = recs[i]-fileno;
   +fi =  recs[i]-p-fi;
flink = fi-flinks[fx];
if (!(flink  *flink != '\0'))
continue;
   @@ .
  __
  RPM Package Managerhttp://rpm5.org
  CVS Sources Repositoryrpm-...@rpm5.org
 
 



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm-5_3: rpm/rpmdb/ rpmdb.c

[Jeff Johnson]

On May 27, 2011, at 11:05 AM, Per Øyvind Karlsen wrote:

 2011/5/27 Jeff Johnson n3...@mac.com:
 Nice!
 \o/
 
 Still needs to be done somewhere else though for performance.
 Nothing at all wrong with this patch, just SHOULD be done
 deeper in dbiFindMatches() for highest performing, most general, etc etc.
 Yupp, there's still a lot room for improvement, I've tried to minimize
 the use of the patch as much as possible as is though, but of course it's
 not really very optimal several queries gets done using it..
 
 Berkeley db is still a bit arcane to me, and whenever I start looking at
 wressling with it again after a couple of months not messing with,
 I spend way too much time having to refresh my memory again.. :|
 

Well its not Berkeley DB but rather the pattern matching code for
partial key retrievals that is _VERY_ tricky.

 Oh well, this one at least works now and should finally put the last
 real issue with this migration for cooker to rest, one less thing to
 keep concerning myself about.. :p
 

Yep. There's other issues such as generalizing the pattern matching
partial key retrievals that need to be attempted to. ATM the main
usage cases of Name/Nvra retrievals are wired up (and so the code
works). But you surely remember what happened when the same code
was permitted on Providename lookup's ... i.e. it the entire
access that needs to be looked at, not the partial key retrieval,
nor Berkeley DB, to ensure nothing breaks.

I.e. its the global context that needs to be examined carefully
and methodically ...

 Might just try optimize it a bit further later on.. :)
 

Hint: get the test cases and the callgrind baseline in place first.

After that its just wild hacking until you get the desired functionality 
implemented.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: so now that we've established weakdeps on rpm5...

[Jeff Johnson]

On May 25, 2011, at 11:05 AM, Robert Xu wrote:

 On Wed, May 25, 2011 at 03:48, devzero2000 pinto.e...@gmail.com wrote:
 FWIW, i have forgotten to tell that the SUSE rpm spec of  rpm 4.9.0 122.1
 contain
 
 PreReq: %insserv_prereq %fillup_prereq permissions
 
 
 I would hope that Requires still accept macros, right?
 

Macros are expanded whereever they are found with all the usual behaviors.

WHat likely isn't obvious is that macro expansions can be postponed
to the install, not just during the build. This is
PreReq: %%insserv_prereq
(note the escaping '%%') and the install, not the build, macros are
used to expand.

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: so now that we've established weakdeps on rpm5...

[Jeff Johnson]

On May 24, 2011, at 7:02 AM, Klaus Kaempf wrote:

 * Jeff Johnson n3...@mac.com [May 24. 2011 12:33]:
 
 And this information is dynamic and should come from an online service
 and not be put into the package, where its 'frozen'
 
 
 This is of course exactly the opposite of what you said here
 
   ... By putting it into
 the package, you get a clearly defined place, clear semantics, and
 packagers don't have to edit multiple places.
 
 Which is it: putting it into the package or dynamic?
 
 
 The information _should_ be dynamic(ally) loaded but there's no
 _current_ solution to achieve this. So we choose the
 not-so-optimal-but-better-than-nothing solution of putting it into the
 package.
 

Yes repo-metadata is impossible to change.

hint: @rpm5.org mas a mongo-c-drive included and server-side infrastructure
is being deployed to start developing schema and tools.

_currently_ yes. Choose what you wish.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: so now that we've established weakdeps on rpm5...

[Jeff Johnson]

On May 24, 2011, at 7:06 AM, Klaus Kaempf wrote:

 
 LOL - anyways, I'm the wrong person to ask. This should go to the
 opensuse community.
 

My cat knows more about RPM than community does. And just miaows in reply ...

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/ CHANGES devtool.conf

[Jeff Johnson]

There's a better and simpler fix (for beecrypt) by not including
the C++ stubs in a -lbeecrypt.

The fix is here (beecrypt/Makefile.am) commenting out cppglue.cxx:

libbeecrypt_la_SOURCES = aes.c base64.c beecrypt.c blockmode.c blockpad.c 
blowfish.c dhies.c dldp.c dlkp.c dlpk.c dlsvdp-dh.c dsa.c elgamal.c 
endianness.c entropy.c fips186.c hmac.c hmacmd5.c hmacsha1.c hmacsha224.c 
hmacsha256.c md4.c md5.c hmacsha384.c hmacsha512.c memchunk.c mp.c mpbarrett.c 
mpnumber.c mpprime.c mtprng.c pkcs1.c pkcs12.c ripemd128.c ripemd160.c 
ripemd256.c ripemd320.c rsa.c rsakp.c rsapk.c sha1.c sha224.c sha256.c sha384.c 
sha512.c sha2k32.c sha2k64.c timestamp.c # cppglue.cxx

73 de Jeff


On May 24, 2011, at 6:39 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   24-May-2011 12:39:12
  Branch: HEAD Handle: 2011052410391101
 
  Modified files:
rpm CHANGES devtool.conf
 
  Log:
added gcc-c++ in devtool.conf for fedora (required by beecrypt)
 
  Summary:
RevisionChanges Path
1.3647  +1  -0  rpm/CHANGES
2.380   +1  -0  rpm/devtool.conf
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3646 -r1.3647 CHANGES
  --- rpm/CHANGES  23 May 2011 13:58:40 -  1.3646
  +++ rpm/CHANGES  24 May 2011 10:39:11 -  1.3647
  @@ -1,4 +1,5 @@
   5.4.0 - 5.4.1:
  +- devzero2000: added gcc-c++ in devtool.conf for fedora (required by 
 beecrypt)
   - devzero2000: added fedora per platform macro from redhat-rpm-config
   - devzero2000: added suse to the per vendor change configure.ac enabler
   - devzero2000: added suse.in macros file from rpm-4.9.0-122
  @@ .
  patch -p0 '@@ .'
  Index: rpm/devtool.conf
  
  $ cvs diff -u -r2.379 -r2.380 devtool.conf
  --- rpm/devtool.conf 12 May 2011 07:39:26 -  2.379
  +++ rpm/devtool.conf 24 May 2011 10:39:12 -  2.380
  @@ -1697,6 +1697,7 @@
   cvs \
   libtool \
   gcc \
  +gcc-c++ \
   gettext \
   make \
   pkgconfig \
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: so now that we've established weakdeps on rpm5...

[Jeff Johnson]

On May 24, 2011, at 11:05 AM, Robert Xu wrote:

 On Tue, May 24, 2011 at 06:42, Jeff Johnson n3...@mac.com wrote:
 
 
 PreReq: was made equivalkent to Requires: in 2001 or so.
 
 
 Isn't there like Requires(pre)? Or is that still the same?
 

Requires(pre) is entirely unrelated to PreReq:.

The (pre) is a context marker that indicates that this
requirement is needed solely for installation.

There are two usage case:
1) dependencies needed only for an install need not be
registered in a rpmdb after the install has occurred.
2) by differentiating an install from an erase context,
certain dependency loops can be avoided. I.e. for installation
(and thw 1st part of upgrade) there's no need to look at
the erase context dependencies.

hth

73 de Jeff
 
 -- 
 later, Robert Xu
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: weak deps already done, right?

[Jeff Johnson]

On May 23, 2011, at 3:23 AM, Klaus Kaempf wrote:

 * Jeff Johnson n3...@mac.com [May 22. 2011 22:04]:
 
 I see no reason for anything _EXCEPT_ the usual PRCO Gang Of Four:
  Provides:
  Requires:
  Conflicts:
  Obsoletes:
 
 
 From a pure RPM pov, I fully agree.
 

Well I realize that SuSE does'n consider RPM a package manager
(not listed on your web site way back when).

 However, from a user/packager pov, additional dependencies are useful.
 

And the discussion is about the mechanics of patching and the engineering
involved in building @rpm5.org on SuSE for a user who wants to do that.

Additional dependencies like Recommends: aren't rally useful for building
@rpm5.org.

 In SUSE (not only SLES10, but all versions of SUSE since SLES10) these
 are used to e.g. support the following use cases:
 
 1.
 The VLC packager wants a 'best effort' experience for the VideoLan
 movie player by pulling in as many codecs as possible during install.
 However, VLC fails gracefully if a codec is not available, so all the
 codecs are not strictly required and VLC installation should not fail
 if a codec is missing.
 To achieve this behaviour in the VLC package, 'recommends' are used to
 pull-in codec package which are installable
 

used is not the only the engineering solution.

Since the preference is implemented outside of rpm itself, you are essentially
saying that rpm MUST do what SuSE wants because SuSE chose Recommends: and that
not doing what SuSE has done is depriving users of the experience of watching 
movies.



 
 2.
 When looking at a product at the Amazon website, users are pointed to
 other products with customer who looked at this also looked at that.
 This is useful information, esp. when you have a huge amount of
 products and putting them all into one list is not practical.
 This is where 'suggests' is helpful. The packager can add 'hints' to
 the package, to point to other useful and interesting packages.
 

Sure. And Amazon has a patent on their one-click shopping experience.
You may wish to reconsider your one-click download-and-install of
RPM package.

again, Suggests: isn't the only engineering, and your reference isn't
the needed hard semantic for how Suggests: AHOULD be implemented, just
yet another opinion.

I should have RPM redirect Suggests: to Amazon so that users can shop for
similar package items? I should send packagers who choose to hint that they
might like to be paid for their efforts to Amazon too? Neither of those 
semantics
is out of line with your suggestion.

 
 3.
 You found a new game package which might also interest your kids.
 However, there are only foreign translations available for this
 package and your kid doesn't understand a word.
 After a couple of weeks, your package update application shows a new
 translation package, you install it and your kid is finally happy.
 The translation package has a supplements dependency to the original
 game thereby enabling the package update application to detect and
 install it.
 

Again no hard semantic that is implementable. What if my kid wants a pony
instead of a game on alternate days of the week because of a bipolar condition?

 
 There are many more uses, esp. when it comes to language and hardware
 support where SUSE makes use of weak dependencies to improve the user
 experience.
 

I'm glad you have improved your SuSE users experience.

 Of course, the information encoded in weak dependencies could be
 stored differently or even in a different place. By putting it into
 the package, you get a clearly defined place, clear semantics, and
 packagers don't have to edit multiple places.
 

I guess you've clarified my feeble objections.

So go patch @rpm5.org code and help your SuSE user here do the port he's 
attempting.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: so now that we've established weakdeps on rpm5...

[Jeff Johnson]

On May 23, 2011, at 5:47 PM, Robert Xu wrote:

 Hi,
 
 So now that it's clarified that RPMSENSE_MISSINGOK works... (I can
 throw out this patch!
 https://github.com/devzero2000/RPM/commit/21ee982d1655c3d8528ed4a32e821aec775ebe14)
 

Yes you shouldn't need that patch with @rpm5.org.

 So... now how could I revise these two patches?
 https://github.com/devzero2000/RPM/commit/2a1443ea095ab3cb87fb593f459f299365e7919c

This patch shouldn't hurt nor help much (its specific to --queryformat).

The basic structure of --singleSprintf isn't radically different. Punt
this one to me if you want and I'll figger out how/where the patch
should be merged. At a first approximation you can likely run
without the patch (but that's just a guess based on where --qf
tends to be used).


 https://github.com/devzero2000/RPM/commit/225db4c7033e014f826fc50ab997f596882c3312

This patch is needed if you want to build packages like SuSE
does, with weak dependencies in explicit tag data. I can carry
this @rpm5.org under
#ifdef RPM_VENDOR_OPENSUSE
if there is interest.

That was basically the point of the exchange with Klaus: there may
be a better and simpler way of adding weak dependencies using
the arbitrary tags I mentioned. With arbitrary tags, the strings
aren't syntax checked quite as carefully (there are *RE patterns that can be 
used
if the syntax were limited to a single condition per tag) and
so a modest parser woule be needed to split the string into
a {N,EVR,F} tripe. Since EVR already needs parsing, that isn't a huge deal.

The alternative form (which is being pursued @rpm.org with OrderedBy: is to
continue with {N,EVR,F} arrays of tuples, which is essentially what
this patch is doing for Suggests: et al.

Without consensus on what the proper representation of weak dependency
tags I'm reluctant to lead the innovation. RPMSENSE_MISSINGOK is a single
bit used as a hint attribute on existing PRCO dependencies.

Arbitrary tags are quite general extensions to header metadata. But sooner
or later the RFE comes in for RPM to actually implement some desired semantic
with tag values.

And its not very hard to re-use the existing rpmds object as a container for
other triples. The problem here is that lots of additional API's are needed
because the representation leads to Yet More Tags which -- once deployed --
will have to be continued likely forever (and it isn't at all clear to me that
weak dependencies are solving a Real World problem even if there are a
small number of known usage cases that can be stated as Klaus has done).

That's where all of the above has been for 4+ years (7+ years if you go back
to when RPMSENSE_MISSINGOK was originally proposed, and weak dependencies
were mostly implemented a few months later). Its kinda hard to
get excited about 7 year old implementations that haven't ever really
been used or deployed widely (yes SuSE has, noone else I'm aware of).

hth

73 de Jeff
 
 -- 
 later, Robert Xu
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: weak deps already done, right?

[Jeff Johnson]

On May 22, 2011, at 2:31 PM, Robert Xu wrote:

 Hi,
 
 Weak Dependencies like Suggests, Recommends, Enhances, Supplements,
 and Requires(missingok) are implemented already into RPM5, right?
 Just double-checking.
 

The flags and tags are reserved but again, there's sharp disagreement
about the need for weak dependencies ala SuSE.

My disagreement is that you can add a strength parameter,
and implement using an integer, where strongest maps
onto an integer comparison.

But who and how are the integers assigned? If done
with external configuration, then rpm installs
become indeterminate. If done statically in an external
file that is shipped with rpm and never changed, then
why is there a need for an external table?

I also dislie (and see no need) for Yet More Types of Dependencies.

At the time weak dependencies were added, there was concern
for SLES 10 compatibility (still using downrev rpm at the time).

So why is there a need to architect weak dependencies for
an essentially mainteneance only hysterical enterprise distro
shipped by a single vendor?

None of the weak dependencies are implemented anywhere outside
of SuSE afaik, including rpm.org.

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: weak deps already done, right?

[Jeff Johnson]

On May 22, 2011, at 3:48 PM, Robert Xu wrote:

 On Sun, May 22, 2011 at 15:06, Jeff Johnson n3...@mac.com wrote:
 
 The flags and tags are reserved but again, there's sharp disagreement
 about the need for weak dependencies ala SuSE.
 
 
 At the time weak dependencies were added, there was concern
 for SLES 10 compatibility (still using downrev rpm at the time).
 
 
 So it is in rpm5 for the time being...
 
 
 My disagreement is that you can add a strength parameter,
 and implement using an integer, where strongest maps
 onto an integer comparison.
 
 But who and how are the integers assigned? If done
 with external configuration, then rpm installs
 become indeterminate. If done statically in an external
 file that is shipped with rpm and never changed, then
 why is there a need for an external table?
 
 Well, why would a strength table be needed? I see weak
 dependencies as a just added functionality item and
 strong dependencies as a required definitely.
 
 I'm not really sure I understood you correctly on that.
 
 btw openSUSE explains it in their old wiki... they should
 move it to the new wiki:
 
 http://old-en.opensuse.org/Software_management/Dependencies
 

There's nothing I heard when weak depends wad first implemented
years ago that isn't on that wiki.

I see no reason for anything _EXCEPT_ the usual PRCO Gang Of Four:
Provides:
Requires:
Conflicts:
Obsoletes:

SuSE (to be backwards compatible with SLES10) chose to do additional tags rather
than change anything.

The weak part is basically best effort: if it fails ignore.

A best effort leads to indeterminism during installs/upgrades/erases,
and the additional tags forces all code using metadata from *.rpm packages
to be changed, creating instant legacy compatibility issues everywhere.

 
 I also dislie (and see no need) for Yet More Types of Dependencies.
 
 Well, that's true.
 
 So why is there a need to architect weak dependencies for
 an essentially mainteneance only hysterical enterprise distro
 shipped by a single vendor?
 
 None of the weak dependencies are implemented anywhere outside
 of SuSE afaik, including rpm.org.
 
 
 IIRC Mandriva also uses weak dependencies to a degree
 with Suggests and stuff. Might be wrong.
 

Nope. Mandriva uses RPMSENSE_MISSINGOK, which was the 1st attempt
at add Suggests: to RPM. SuSE weak dependencies directly followed
as a consequence of RPMSENSE_MISSINGOK.

ANd like 6 years later there is _STILL_ no consensus nor adoption
of any of Suggests: et al in any version of RPM.

There really isn't any need for Suggests et al, or there WOULD
be a widespread deployment of an implementation by now.

I personally think that preferences need to be targeted. All
this would/should/coulda/kinda/dowhatimean that is implied
by naive intrinsic meanings to the words siggests and enhances
and recommends hasn't either a strong enough semantic or an
industrial strength implementation to handle preference years
and years later.

You can of course make up whetever semantic one wishes, implement it, and then 
claim
It Just Works!
on some blog. 

73 de Jeff
 
 -- 
 later, Robert Xu
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: weak deps already done, right?

[Jeff Johnson]

On May 22, 2011, at 3:50 PM, Robert Xu wrote:

 On Sun, May 22, 2011 at 15:44, pinto.e...@gmail.com
 pinto.e...@gmail.com wrote:
 In general this is a difficult question to answer, for my knowledge. There 
 is not clear semantic for the weak deps, in other package management system 
 the interpretation is into the deps solver that decide what to do ( aptitude 
 vs apt vs dselect iirc  ) . Suse have some patch for rpm.org about this 
 issue last time i have cheched for rpm 4.8.
 
 There's always heated discussion about this.
 

The heat isn't from the discussion, but rather the insane vanity of
Signed-off By: Britney Sphears

 Anyway i think a better question could be : is it zypper rpm5 compatible ? I 
 don't know the answer now, never tried. But i am pretty sure that @rpm5.org 
 could be have interest to answer. Best Regards.
 
 I emailed bero from arklinux (knowing that he is working on getting
 libzypp/libsatsolver/zypper to work with rpm5) and I'm waiting for a
 response. But from what I hear, he's been terribly busy these days.
 

Yes, and zypper is used in ark (largely because apt-rpm wasn't
maintained nor getting the job done).

The Poky/Yacto distribution has also used zypper+rpm5, largely based
on some of bero's work with a few other fiddle ups.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: when not using internal generator...

[Jeff Johnson]

On May 22, 2011, at 10:05 PM, Robert Xu wrote:

 Hi all,
 
 SuSE doesn't like using an internal generator. Figures.
 

SuSE and Mandriva just gotta be different yes.

The difference is at the product level with multilib, not in how
packages are built. The internal ELF generator is known reliable
for years, just SuSE and Mandriva can never figger that out.

The core issue to watch out for is -- if not using the internal generator --
that the rpmds data gets sorted somehow. That is/was one of the reasons
for the implementation (not multilib).

But if y'all want linear behavior instead of logN with bsearch,
well vendors can run as SLOWER and S-L-O-W-E-R as they wish, not my
problem mon.

 Can %_perl_provides and %_perl_requires, and %_python_provides ...
 etc... be omitted?
 Because AFAICT, SuSE just uses regular find-provides and find-requires.
 

All depends on what is implemented in find-provides/find-requires. What's
in rpm5.org sources hasn't been looked at (by me) for 6+ years now, caveat 
emptor.

You CAN add a sort there ... someone tell Mandriva please.

Yes %_perl* et al can be ignored if find-provides/find-requires are written 
correctly.

Note that there is /usr/lib/rpm/bin/rpmdeps --provides,-P and --requires,-R that
can be dropped into find_provides and find_requires too. That's basically
what Poky/Yacto is doing, building without using *.spec files (yes *.spec files
are used for templating the packaging but not for building).

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: when not using internal generator...

[Jeff Johnson]

On May 22, 2011, at 10:32 PM, Robert Xu wrote:

 
 
 On May 22, 2011, at 22:19, Jeff Johnson n3...@mac.com wrote:
 
 
 On May 22, 2011, at 10:05 PM, Robert Xu wrote:
 
 Hi all,
 
 SuSE doesn't like using an internal generator. Figures.
 
 
 SuSE and Mandriva just gotta be different yes.
 
 The difference is at the product level with multilib, not in how
 packages are built. The internal ELF generator is known reliable
 for years, just SuSE and Mandriva can never figger that out.
 
 The core issue to watch out for is -- if not using the internal generator --
 that the rpmds data gets sorted somehow. That is/was one of the reasons
 for the implementation (not multilib).
 
 But if y'all want linear behavior instead of logN with bsearch,
 well vendors can run as SLOWER and S-L-O-W-E-R as they wish, not my
 problem mon.
 
 Actually, with rpm 4.9 they switched to the internal generator:
 
 --
 
 Hi Packagers,
 
 I've spent the last week to upgrade rpm from 4.8.0 to 4.9.0.
 The package was submitted to Factory yesterday and is already
 checked in.
 

Only took 6 years ... *shrug*

 Besides some bug fixes and an update to a newer BerkeleyDB
 library rpm-4.9.0 contains plugin architecture for dependency
 generation. In older rpms, the internal dependency generator
 was pretty much hardcoded in C, so we always used the old
 external one to generate dependencies. With rpm-4.9.0, the
 internal generator has become flexible enough so that we
 can use it.
 

The term hardcoded is misused here ...

 This means for you, that rpm will no longer use the %__find_provides
 and %__find_requires macros. Some packages redefined those
 macros to be able to filter the generated dependencies.
 This will no longer work in rpm-4.9.0. Instead, support for
 dependency filtering was added to rpm. You can now use
 
%define %__requires_exclude_from ^/foo.*bar
 
 to tell rpm to ignore files matching the regexp when generating
 requires (the $RPM_BUILD_ROOT is already stripped).
 

... and this was added from PLD 2+ years ago. The dialect of *RE
had to be established first, and its likely PCRE != POSIX RE's for extra
funnerliness.

 You can use
 
%define %__requires_exclude libfoo.*
 
 to remove a generated dependency. The same is possible for
 %__provides_ and %__supplements_.
 

Don't need no steenking supplements. This is OBS only.

 As the generator definitions are now pluggable, I'll migrate them
 from the rpm package into the corresponding packages in the next
 days, i.e. the mono plugin definition into the mono package.
 

And pluggable is misused here.

 Have fun with rpm-4.9.0,
  Michael.
 
 --
 
 I'll need to take a look at recent commits in rpm to see what has been done 
 since I last checked out and if I need to patch some more.
 
 And I don't know about RPM5's internal generator, so...
 

The internal generator can be seen outside of rpm by doing things like

find /bin | /usr/lib/rpm/bin/rpmdeps --provides

and similarly for --requires.

 
 Can %_perl_provides and %_perl_requires, and %_python_provides ...
 etc... be omitted?
 Because AFAICT, SuSE just uses regular find-provides and find-requires.
 
 
 All depends on what is implemented in find-provides/find-requires. What's
 in rpm5.org sources hasn't been looked at (by me) for 6+ years now, caveat 
 emptor.
 
 You CAN add a sort there ... someone tell Mandriva please.
 
 Per Oyvind's job ;)
 
 
 Yes %_perl* et al can be ignored if find-provides/find-requires are written 
 correctly.
 
 To me it looks ok, but again see above with regard to rpm 4.9 in which I 
 might have to go nuts.
 

The divergence is mostly in configgery. A glob is used to gather files
that configure 6 or so parameters per type.

Otherwise its the same basic mechanism, just now exploded into gazillions
of teensy config files to match on paths and suffices.

I personaly don't believe in paths because FHS keeps changing,
and I don't believe in mime-type suffixes because they are too
flimsy, and so never bothered to explode out the configuration.

*Yawn* -- likely gotta keep up with the Joneses and mow the lawn and trim the 
weeds.

todo++

 
hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: when not using internal generator...

[Jeff Johnson]

On May 22, 2011, at 10:52 PM, Jeff Johnson wrote:

 
 Only took 6 years ... *shrug*
 

The important thing to look for here is whether dependcies are sorted:

rpm -qp --requires foo*.rpm

with package built by rpm-4.9.0. If not sorted, the job isn't finished yet.

hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: split formats.c?

[Jeff Johnson]

On May 21, 2011, at 8:02 PM, Robert Xu wrote:

 On Sat, May 21, 2011 at 19:55, Per Øyvind Karlsen pkarl...@rpm5.org wrote:
 2011/5/22 Robert Xu rob...@gmail.com:
 Hi all,
 
 I've been going through opensuse patches with quilt (sorry Jeff, I
 couldn't wait to build RPM5, but didn't want to throw away whatever
 SuSE does completely)...
 I ran across localetag.diff, and then ran through the rpm5 repository,
 only to discover that rpm.org split formats.c into formats.c and
 tagext.c.
 
 
 oops, it's tagexts.c
 
 Now with this patch, I'm wondering - what should I do with it?
 Provide us with the patch, and it'll be easier for us to comment on.. ;)
 (I'm way too lazy to even consider tracking it down myself.. :p)
 
 
 fine, fine ;)
 
 build.o.o: 
 https://build.opensuse.org/package/view_file?file=localetag.diffpackage=rpmproject=openSUSE%3AFactorysrcmd5=fb1d950eb1be36c83419ecc039e31256
 
 formats.c @ rpm5.org: http://rpm5.org/cvs/fileview?f=rpm/lib/formats.c
 
 log of tagexts.c, including the creation, @ rpm.org:
 http://rpm.org/gitweb?p=rpm.git;a=history;f=lib/tagexts.c;h=aca3209a6869f453b567cc6ed4e431306cf67a5c;hb=HEAD
 

You shouldn't need to touch anything in queryformats.

Yes there is significant divergence between rpm.org - @rpm5.org,
both architecturally and in the feature set.

E.g. @rpm5.org treats header extension tags just like regular tags
in headerGet().

(aside)
A header extension tag is essentially a transform on tag data.
Tag data is exactly what is in a header literally.

By treating header tag extensions just like header tags then
headerGet() (the usual way of retrieving tags) can be used also
for the transforms. Otherwise there's a whole different headerSprintf()
method that MUST be used in order to see/use header extension tags.

If there's any specific header tag extension you need/want, ask.

As you can tell from the tagexts.c git log comments, rpm.org spends
a whole lotta time ripping out working code for unused reasons,
rather than really ever adding anything, so I suspect you don't
need anything new because there isn't anything new at rpm.org.

hth

73 de Jeff
 -- 
 later, Robert Xu
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: split formats.c?

[Jeff Johnson]

On May 21, 2011, at 6:56 PM, Robert Xu wrote:

 Hi all,
 
 I've been going through opensuse patches with quilt (sorry Jeff, I
 couldn't wait to build RPM5, but didn't want to throw away whatever
 SuSE does completely)...

I have a build on OpenSUSE sitting idle in a VM somewhere anytime you want.

 I ran across localetag.diff, and then ran through the rpm5 repository,
 only to discover that rpm.org split formats.c into formats.c and
 tagext.c.
 
 Now with this patch, I'm wondering - what should I do with it?
 

Ignore the localetag.diff at first approximation. If its the
patch I think it is, SuSE is attempting a conversion from the
encoding included in the package to a different encoding the
hard way.

These days there's a routine called iconv(3) that did not exist 6+ years
ago when SuSE did the patch.

I would claim that the patch is not the best thing to do: transforming 
encodings on
the fly (which is why the SuSE patch was only partially integrated in RPM
way back when). This point is debatable and arguable and subtle. My experience
doing something similar in POPT has been a nightmare to try to fix.

Caveat:
I may be misremembering the patch.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/ CHANGES rpm/scripts/ find-debuginfo.sh

[Jeff Johnson]

Nice!

You beat me to the fix ;-)

I was staring at just how gawd awful find-debuginfo.sh really is.

That script is really really twisty and in need of writing in C, not shell.

73 de Jeff

On May 18, 2011, at 9:32 AM, Pinto Elia wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Pinto Elia
  Root:   /v/rpm/cvs   Email:  devzero2...@rpm5.org
  Module: rpm  Date:   18-May-2011 15:32:50
  Branch: HEAD Handle: 2011051813324901
 
  Modified files:
rpm CHANGES
rpm/scripts find-debuginfo.sh
 
  Log:
Sort file list for debuginfo extraction.
(inspired by Robert Schiele 
 http://lists.rpm.org/pipermail/rpm-list/2011-May/000894.html)
 
Submitted by: Elia Pinto
 
  Summary:
RevisionChanges Path
1.3642  +2  -0  rpm/CHANGES
1.15+7  -7  rpm/scripts/find-debuginfo.sh
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3641 -r1.3642 CHANGES
  --- rpm/CHANGES  16 May 2011 00:33:13 -  1.3641
  +++ rpm/CHANGES  18 May 2011 13:32:49 -  1.3642
  @@ -1,4 +1,6 @@
   5.4.0 - 5.4.1:
  +- devzero2000: Sort file list for debuginfo extraction. 
  +  (inspired by Robert Schiele 
 http://lists.rpm.org/pipermail/rpm-list/2011-May/000894.html)
   - proyvind: update doxygen input filename paths.
   - proyvind: fix assertion error when trying to install packages without
   required permissions.
  @@ .
  patch -p0 '@@ .'
  Index: rpm/scripts/find-debuginfo.sh
  
  $ cvs diff -u -r1.14 -r1.15 find-debuginfo.sh
  --- rpm/scripts/find-debuginfo.sh10 Aug 2010 12:18:52 -  1.14
  +++ rpm/scripts/find-debuginfo.sh18 May 2011 13:32:50 -  1.15
  @@ -176,7 +176,7 @@
   # Strip ELF binaries
   find $RPM_BUILD_ROOT ! -path ${debugdir}/*.debug -type f \
\( -perm -0100 -or -perm -0010 -or -perm -0001 \) \
  - -print |
  + -print | LANG=C sort |
   file -N -f - | sed -n -e 's/^\(.*\):[   ]*.*ELF.*, not stripped/\1/p' |
   xargs --no-run-if-empty stat -c '%h %D_%i %n' |
   while read nlinks inum f; do
  @@ -232,7 +232,7 @@
 
   # For each symlink whose target has a .debug file,
   # make a .debug symlink to that file.
  -find $RPM_BUILD_ROOT ! -path ${debugdir}/* -type l -print |
  +find $RPM_BUILD_ROOT ! -path ${debugdir}/* -type l -print | LANG=C sort |
   while read f
   do
 t=$(readlink -m $f).debug
  @@ -249,19 +249,19 @@
 LC_ALL=C sort -z -u $SOURCEFILE | egrep -v -z 
 '(internal|built-in)$' |
 (cd $RPM_BUILD_DIR; cpio -pd0mL ${RPM_BUILD_ROOT}/usr/src/debug)
 # stupid cpio creates new directories in mode 0700, fixup
  -  find ${RPM_BUILD_ROOT}/usr/src/debug -type d -print0 |
  +  find ${RPM_BUILD_ROOT}/usr/src/debug -type d -print0 | LANG=C sort |
 xargs --no-run-if-empty -0 chmod a+rx
   fi
 
   if [ -d ${RPM_BUILD_ROOT}/usr/lib -o -d ${RPM_BUILD_ROOT}/usr/src ]; 
 then
 ((nout  0)) ||
 test ! -d ${RPM_BUILD_ROOT}/usr/lib ||
  -  (cd ${RPM_BUILD_ROOT}/usr/lib; find debug -type d) |
  +  (cd ${RPM_BUILD_ROOT}/usr/lib; find debug -type d) | LANG=C sort |
 sed 's,^,%dir /usr/lib/,'  $LISTFILE
 
 (cd ${RPM_BUILD_ROOT}/usr
  -   test ! -d lib/debug || find lib/debug ! -type d
  -   test ! -d src/debug || find src/debug -mindepth 1 -maxdepth 1
  +   test ! -d lib/debug || find lib/debug ! -type d | LANG=C sort
  +   test ! -d src/debug || find src/debug -mindepth 1 -maxdepth 1 | LANG=C 
 sort
 ) | sed 's,^,/usr/,'  $LISTFILE
   fi
 
  @@ -321,7 +321,7 @@
   done
   if ((nout  0)); then
 # Now add the right %dir lines to each output list.
  -  (cd ${RPM_BUILD_ROOT}; find usr/lib/debug -type d) |
  +  (cd ${RPM_BUILD_ROOT}; find usr/lib/debug -type d) | LANG=C sort
 sed 's#^.*$#\\@^//@{h;s@^.*$@%dir /@p;g;}#' |
 LC_ALL=C sort -ur  ${LISTFILE}.dirs.sed
 i=0
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm-5_4: rpm/tools/ Makefile.am

[Jeff Johnson]

The right fix is likely to add
$(RPMDB_LDADD_COMMON)

Note that I rewrote dbconver.c somewhat somewhere.
I forget whethere I checked the changes in at all, but I most certainly
did NOT check it into the rpm-5_3 branch because of the unknown
production risk there when deployed into Cooker mirrors.

73 de Jeff

On May 16, 2011, at 4:19 AM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   16-May-2011 10:19:18
  Branch: rpm-5_4  Handle: 2011051608191800
 
  Modified files:   (Branch: rpm-5_4)
rpm/tools   Makefile.am
 
  Log:
fix linkage of dbconvert  install it by default
 
  Summary:
RevisionChanges Path
2.170.2.4   +2  -1  rpm/tools/Makefile.am
  
 
  patch -p0 '@@ .'
  Index: rpm/tools/Makefile.am
  
  $ cvs diff -u -r2.170.2.3 -r2.170.2.4 Makefile.am
  --- rpm/tools/Makefile.am9 May 2011 04:06:34 -   2.170.2.3
  +++ rpm/tools/Makefile.am16 May 2011 08:19:18 -  2.170.2.4
  @@ -57,6 +57,7 @@
   pkgbin_PROGRAMS =   \
   @WITH_AUGEAS_AUGTOOL@ chroot cp @WITH_CUDF_CUDFTOOL@ find mtree \
   @WITH_SEMANAGE_SEMODULE@ wget \
  +dbconvert \
   rpmcache rpmdigest rpmrepo rpmspecdump \
   rpmcmp rpmdeps sqlite3 @WITH_KEYUTILS_RPMKEY@ @WITH_LIBELF_DEBUGEDIT@
   dist_man_MANS = rpmgrep.1
  @@ -80,7 +81,7 @@
 
   dbconvert_SOURCES = dbconvert.c
   dbconvert_LDFLAGS = @LDFLAGS_STATIC@ $(LDFLAGS)
  -dbconvert_LDADD =   $(RPMIO_LDADD_COMMON)
  +dbconvert_LDADD =   $(RPM_LDADD_COMMON)
 
   debugedit_SOURCES = debugedit.c hashtab.c
   debugedit_LDFLAGS = @LDFLAGS_STATIC@ $(LDFLAGS)
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/ CHANGES rpm/rpmio/ iosm.c

[Jeff Johnson]

FYI: This code (i.e. 3 1-line changes) was responsible for a 10% performance
increase in wall-clock install time.

You've also had personabl experience why Fsync (close byt) isn't/wans't
the right implementation (was requested by Russell Coker for xfs hardening).

Study carefully, MADV_DONTNEED isn't where one expects to find
I/O performance improvements though its perfectly
obvious once you sift through the gory details.

hth

73 de Jeff


On May 14, 2011, at 4:36 PM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   14-May-2011 22:36:14
  Branch: HEAD Handle: 2011051420361400
 
  Modified files:
rpm CHANGES
rpm/rpmio   iosm.c
 
  Log:
fix assertion error when trying to install packages without required
permissions.
 
  Summary:
RevisionChanges Path
1.3640  +2  -0  rpm/CHANGES
1.44+2  -1  rpm/rpmio/iosm.c
  
 
  patch -p0 '@@ .'
  Index: rpm/CHANGES
  
  $ cvs diff -u -r1.3639 -r1.3640 CHANGES
  --- rpm/CHANGES  12 May 2011 10:01:48 -  1.3639
  +++ rpm/CHANGES  14 May 2011 20:36:14 -  1.3640
  @@ -1,4 +1,6 @@
   5.4.0 - 5.4.1:
  +- proyvind: fix assertion error when trying to install packages without
  +required permissions.
   - devzero2000: begin to put the preliminary check for SUSE
   - afb: yaml: fix changelog entry markup
   - afb: hdrfmt.c: add ISO-8601 date format
  @@ .
  patch -p0 '@@ .'
  Index: rpm/rpmio/iosm.c
  
  $ cvs diff -u -r1.43 -r1.44 iosm.c
  --- rpm/rpmio/iosm.c 29 Sep 2010 14:54:30 -  1.43
  +++ rpm/rpmio/iosm.c 14 May 2011 20:36:14 -  1.44
  @@ -2630,7 +2630,8 @@
   rc = IOSMERR_OPEN_FAILED;
   }
   #if defined(POSIX_FADV_DONTNEED)
  -(void) Fadvise(iosm-wfd, 0, 0, POSIX_FADV_DONTNEED);
  +else
  +(void) Fadvise(iosm-wfd, 0, 0, POSIX_FADV_DONTNEED);
   #endif
   if (iosm-debug  (stage  IOSM_SYSCALL))
   rpmlog(RPMLOG_DEBUG,  %8s (%s, \w\) wfd %p wrbuf %p\n, cur,
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/rpmdb/ hdrfmt.c

[Jeff Johnson]

Nothing wrong with this patch whatsoever.

But do note that there is a common API and naming
in hdrfmt.c prepared for a major trash hauling
one of these days.

The additional integer argument breaks the rule and
it literally does not matter until all the trash gets hauled.

The --queryformat code -- while quite general -- was never
intended for (but is handling) the spewage load that --qf is being
repurposed for. Can't be helped, can clean up the trash any time
its clearer what spewage is _REALLY_ needed, and a commitment to UUID
and RPMTAG_HDRID and RPMTAG_PKGID is a huge step in the right direction.

just fyi

73 de Jeff

On May 13, 2011, at 7:59 AM, Anders F. Björklund wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Anders F. Björklund
  Root:   /v/rpm/cvs   Email:  a...@rpm5.org
  Module: rpm  Date:   13-May-2011 13:59:38
  Branch: HEAD Handle: 2011051311593800
 
  Modified files:
rpm/rpmdb   hdrfmt.c
 
  Log:
ISO timestamps are UTC in yaml, make it explicit
 
  Summary:
RevisionChanges Path
1.159   +8  -5  rpm/rpmdb/hdrfmt.c
  
 
  patch -p0 '@@ .'
  Index: rpm/rpmdb/hdrfmt.c
  
  $ cvs diff -u -r1.158 -r1.159 hdrfmt.c
  --- rpm/rpmdb/hdrfmt.c   12 May 2011 08:54:47 -  1.158
  +++ rpm/rpmdb/hdrfmt.c   13 May 2011 11:59:38 -  1.159
  @@ -195,7 +195,7 @@
* @return  formatted string
*/
   static char * realDateFormat(HE_t he, /*@unused@*/ /*@null@*/ const char ** 
 av,
  -const char * strftimeFormat)
  +const char * strftimeFormat, int utc)
   /*@*/
   {
   char * val;
  @@ -208,7 +208,10 @@
 
   /* this is important if sizeof(rpmuint64_t) ! sizeof(time_t) */
   {   time_t dateint = he-p.ui64p[0];
  -tstruct = localtime(dateint);
  +if (utc)
  +tstruct = gmtime(dateint);
  +else
  +tstruct = localtime(dateint);
   }
   buf[0] = '\0';
   if (tstruct)
  @@ -229,7 +232,7 @@
   static char * dateFormat(HE_t he, /*@null@*/ const char ** av)
   /*@*/
   {
  -return realDateFormat(he, av, _(%c));
  +return realDateFormat(he, av, _(%c), 0);
   }
 
   /**
  @@ -241,7 +244,7 @@
   static char * dayFormat(HE_t he, /*@null@*/ const char ** av)
   /*@*/
   {
  -return realDateFormat(he, av, _(%a %b %d %Y));
  +return realDateFormat(he, av, _(%a %b %d %Y), 0);
   }
 
   /**
  @@ -253,7 +256,7 @@
   static char * isodateFormat(HE_t he, /*@null@*/ const char ** av)
   /*@*/
   {
  -return realDateFormat(he, av, _(%Y-%m-%dT%H:%M:%S));
  +return realDateFormat(he, av, %Y-%m-%dT%H:%M:%SZ, 1);
   }
 
   /**
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/rpmdb/ hdrfmt.c

[Jeff Johnson]

Should the date in %changelog also be changed?

Or leave bad enuf alone for status quo ante compatibility?

BTW, RPM got reamed years ago because %changelog isn't standard, been
on my todo++ since forever.

73 de Jeff

On May 12, 2011, at 2:34 AM, Anders F. Björklund wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Anders F. Björklund
  Root:   /v/rpm/cvs   Email:  a...@rpm5.org
  Module: rpm  Date:   12-May-2011 08:34:44
  Branch: HEAD Handle: 2011051206344400
 
  Modified files:
rpm/rpmdb   hdrfmt.c
 
  Log:
add ISO-8601 date format
 
  Summary:
RevisionChanges Path
1.156   +14 -0  rpm/rpmdb/hdrfmt.c
  
 
  patch -p0 '@@ .'
  Index: rpm/rpmdb/hdrfmt.c
  
  $ cvs diff -u -r1.155 -r1.156 hdrfmt.c
  --- rpm/rpmdb/hdrfmt.c   20 Apr 2011 06:09:42 -  1.155
  +++ rpm/rpmdb/hdrfmt.c   12 May 2011 06:34:44 -  1.156
  @@ -245,6 +245,18 @@
   }
 
   /**
  + * Return isodate (ISO-8601) formatted data.
  + * @param hetag container
  + * @param avparameter list (or NULL)
  + * @return  formatted string
  + */
  +static char * isodateFormat(HE_t he, /*@null@*/ const char ** av)
  +/*@*/
  +{
  +return realDateFormat(he, av, _(%Y-%m-%dT%H:%M:%S));
  +}
  +
  +/**
* Return shell escape formatted data.
* @param hetag container
* @param avparameter list (or NULL)
  @@ -315,6 +327,8 @@
   { .fmtFunction = dateFormat } },
   { HEADER_EXT_FORMAT, day,
   { .fmtFunction = dayFormat } },
  +{ HEADER_EXT_FORMAT, isodate,
  +{ .fmtFunction = isodateFormat } },
   { HEADER_EXT_FORMAT, shescape,
   { .fmtFunction = shescapeFormat } },
   { HEADER_EXT_LAST, NULL, { NULL } }
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/scripts/ wnh_other_yaml wnh_primary_yaml

[Jeff Johnson]

Last month I was adding types for dates in JSON.

I have mixed feelings for adding types to metadata.

The easy types like integers and time/date just
make it harder to code because of additional special
cases (instead of essentiallt treatin all integers
as network order uint32_t and leave the interpretation
unspecified. Does anyone not now how to convert uglix time(2)?

The useful types are tuples like EVRD and that isn't
usually handled by types but rather collections of some sort.

I chose to add yime/date types to --json largely because I'm bored.

SHould types like time/data be used when available, thereby
becoming standard and imposing a context of interpreteation?

Or should what are essentially uglix stamps be left as integers
to make it as easy to write parsers?

I can tell which way you are headed from the check-in ... same
direction I'm headed with --json.

73 de Jeff

On May 12, 2011, at 2:36 AM, Anders F. Björklund wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Anders F. Björklund
  Root:   /v/rpm/cvs   Email:  a...@rpm5.org
  Module: rpm  Date:   12-May-2011 08:36:53
  Branch: HEAD Handle: 2011051206365201
 
  Modified files:
rpm/scripts wnh_other_yaml wnh_primary_yaml
 
  Log:
yaml: use cvs -q diff -utimestamp for timestamps
 
  Summary:
RevisionChanges Path
1.4 +1  -1  rpm/scripts/wnh_other_yaml
1.4 +2  -2  rpm/scripts/wnh_primary_yaml
  
 
  patch -p0 '@@ .'
  Index: rpm/scripts/wnh_other_yaml
  
  $ cvs diff -u -r1.3 -r1.4 wnh_other_yaml
  --- rpm/scripts/wnh_other_yaml   12 May 2011 06:34:03 -  1.3
  +++ rpm/scripts/wnh_other_yaml   12 May 2011 06:36:53 -  1.4
  @@ -12,7 +12,7 @@
   [\
   \n   -\
   \nAuthor: %{CHANGELOGNAME}\
  -\nDate:   %{CHANGELOGTIME:date}\
  +\nDate:   !!timestamp '%{CHANGELOGTIME:isodate}'\
   \n%{CHANGELOGTEXT:yaml}\
   ]\
   }|\
  @@ .
  patch -p0 '@@ .'
  Index: rpm/scripts/wnh_primary_yaml
  
  $ cvs diff -u -r1.3 -r1.4 wnh_primary_yaml
  --- rpm/scripts/wnh_primary_yaml 12 May 2011 06:34:02 -  1.3
  +++ rpm/scripts/wnh_primary_yaml 12 May 2011 06:36:52 -  1.4
  @@ -11,8 +11,8 @@
   \n  Description: %{DESCRIPTION:yaml}\
   \n  Packager:%|PACKAGER?{%{PACKAGER:yaml}}:{~}|\
   \n  Url: %|URL?{%{URL:yaml}}:{~}|\
  -\n  Filetime:%{PACKAGETIME:date}\
  -\n  Buildtime:   %{BUILDTIME:date}\
  +\n  Filetime:!!timestamp '%{PACKAGETIME:isodate}'\
  +\n  Buildtime:   !!timestamp '%{BUILDTIME:isodate}'\
   \n  Packagesize: %{PACKAGESIZE}\
   \n  Size:%{SIZE}\
   \n  Archivesize: %{ARCHIVESIZE}\
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/ devtool.conf

[Jeff Johnson]

On May 12, 2011, at 3:35 AM, Anders F Björklund wrote:

 Jeff Johnson wrote:
 
 No idea whether it works, but at least it seems to build...
 
 
 ... the point being that libtsan1 is headed for the bit bucket
 anyways, and openssl is so so so much easier to use than gnutls
 (but only by a hair's breadth) and the One True Crypto: NSS is actually
 okay compared to the other ancient dinosaur (CDMA? CDMS? never can remember)
 that is around for time os yore ...
 
 ... so holler if you wish me to dump libtasn1 sooner rather than later.
 
 One other problem is that the tools are using $(OPENMP_CFLAGS)
 but not $(OPENMP_LIBS), which leads to missing symbols at runtime:
 
 dyld: lazy symbol binding failed: Symbol not found: _GOMP_parallel_start
 
 One workaround is --disable-openmp, but for it to work it also needs
 to include the libraries which are compiler-dependent (gcc: libgomp)
 

--disable-openmp needs -lgomp? That's broken ...

 Beecrypt does this, but on Darwin each library (like rpmio) also must...
 


Hmmm ... beecrypt was the model for what is in rpm. If it
isn't exactly same behavior than something is screwed.

The problem on Mac OS X is LLVM != gcc and (at the time I looked) there wasn't 
any
clarity or guidance of what to do. I tried both compilers, and GCC was much 
more pain
than not using OPENMP. This was perhaps 18-24 months ago.

Given a choice between using LLVM (and clang) vs getting tied
to a second class compiler like GCC on Mac OS X in order to use GOMP,
well OPENMP doesn't have a whole lot of benefit for either
beecrypt or rpm atm.

 --anders
 
 PS. Something like:
 AC_OPENMP
 if test .`$CC --version 21 | grep 'GCC'` != .; then
  if test $OPENMP_CFLAGS != ; then
AC_SUBST(OPENMP_LIBS,-lgomp)
  fi
 fi
 

There were SUPPOSED to be useful AutoFu macros and I tried to use
exactly by the book.

If the above makes it easier for what you are doing, go fer it!


73 de jeff
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/rpmdb/ hdrfmt.c

[Jeff Johnson]

On May 12, 2011, at 10:06 AM, Anders F Björklund wrote:

 Jeff Johnson wrote:
 
 Should the date in %changelog also be changed?
 
 You mean for the .spec input ? Wouldn't that invalidate
 all spec files, unless it's made optional (either/or) ?
 

Yep, incompatible (invalidate seems to hint that ther is an
agenda which doesn't exist) is why never done.

 Well, I live in a civilized country which uses ISO dates...
 But there's plenty of middle-endian date formats in use.
 
 Or leave bad enuf alone for status quo ante compatibility?
 
 The big problem with the timestamps isn't the order, but
 the lack of a timezone. Usually meaning localtime, but...
 

Timezone really isn't too hard to solve in spewage:

Everything in UTC always.

and convert to local timezone however/whenever you please.

 For the YAML timestamp, I think a missing timezone _might_
 be interpreted as UTC which means the day could be off by one.
 

The UTC always enforcement can be done outside of YAML. And
there's little lossage in the format choosing to always add UTC.

Slicker would be to make UTC always a rule (it is since forever
for all RPM timestamps, anything not in UTC is a bug), and reduce
the markup redundancy, as well as the eye scratchiness, but ...

 BTW, RPM got reamed years ago because %changelog isn't standard, been
 on my todo++ since forever.
 
 The author fields are also horribly abused since forever,
 but seperating out the release at this point might be hard ?
 

Yep. The added version is exploiting a bug in the RFC-822 e-mail parser.

 But for the timestamp I was just going with canonical,
 seems like if you use a space it means 1 date + 1 time.
 

While there's all sorts of peripheral issues with date/time.

I was more asking

Should spewage attempt typing (like date/time) or just stick with integers?

Lots of the peripheral issues disappear if its just an integer written
into spewage.

OTOH, integers are not very informative when read or grep'ed.

I still don't have my own clear answer, and both your changes
to YAML, and my changes to JSON are headed towards adding
date/time types in the spewage, which is going to open
up all the usual peripheral issues.

Either integers/types works. I 'spose that explicit times is
easier to perceive as a feature ...

73 de Jeff


 --anders
 
 __
 RPM Package Managerhttp://rpm5.org
 Developer Communication Listrpm-devel@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

What threading model for RPM? (was Re: [CVS] RPM: rpm/ devtool.conf)

[Jeff Johnson]

On May 12, 2011, at 10:47 AM, Anders F Björklund wrote:

 Jeff Johnson wrote:
 
 One other problem is that the tools are using $(OPENMP_CFLAGS)
 but not $(OPENMP_LIBS), which leads to missing symbols at runtime:
 
 dyld: lazy symbol binding failed: Symbol not found: _GOMP_parallel_start
 
 One workaround is --disable-openmp, but for it to work it also needs
 to include the libraries which are compiler-dependent (gcc: libgomp)
 
 
 --disable-openmp needs -lgomp? That's broken ...
 
 I meant the workaround to the missing symbol (with OpenMP)
 is to disable it, otherwise it'll default to on if present.
 

Yes: opt-in and opt-out isn't correct in the AutoFu by policy.
Likely easy to flip to opt-in if the spiffy (and useless)
by-the-book AutoFu is ripped out.

The more general ROADMAP issue is
OPENMP or free-threading as a model?
OPENMP is so easy to do one misses that its all of RPM code
would need to be refactored to continue the pretense of easy.


I suppose I should open up
How should RPM become multi-threaded?

Its silly that RPM can only use a single processor
and package management is accused of S-L-O-O-W
all the time.

73 de Jeff




smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/ devtool.conf

[Jeff Johnson]

I'll snip the libtasn1 dependency out if it causes pain ...

... its an implicit dependency from
rpm - neon - gnutls - libtasn1
which can be flipped into --with-openssl in a flash
now that --with-neon=internal is in place.

Note neon 0.29.5 - 0.29.6 looms on the todo++ list ...

... no hurry no worry gotta get UL and OM and momonga and serentos
drilled green at
http://cooker.rpm5.org:8010
first.

cvs import is actually quite pleasant compared to git. *shrug*

todo++

73 de Jeff

On May 11, 2011, at 11:59 AM, Anders F. Björklund wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Anders F. Björklund
  Root:   /v/rpm/cvs   Email:  a...@rpm5.org
  Module: rpm  Date:   11-May-2011 17:59:22
  Branch: HEAD Handle: 2011051115592200
 
  Modified files:
rpm devtool.conf
 
  Log:
mac/bsd: document mandatory libtasn1 requirement
 
  Summary:
RevisionChanges Path
2.378   +4  -0  rpm/devtool.conf
  
 
  patch -p0 '@@ .'
  Index: rpm/devtool.conf
  
  $ cvs diff -u -r2.377 -r2.378 devtool.conf
  --- rpm/devtool.conf 7 May 2011 23:30:48 -   2.377
  +++ rpm/devtool.conf 11 May 2011 15:59:22 -  2.378
  @@ -2625,6 +2625,7 @@
   #   sqlite3
   #   popt
   #   beecrypt
  +#   libtasn1
   #   expat
   #   neon
   #   zlib
  @@ -2663,6 +2664,7 @@
   --with-sqlite \
   --with-popt=external \
   --with-beecrypt=external \
  +--with-libtasn1=external \
   --with-neon=external \
   --with-xar=external \
   --with-xz=external \
  @@ -2696,6 +2698,7 @@
   #   databases/sqlite3   EXTENSION=on
   #   devel/popt  = 1.15
   #   security/beecrypt   = 4.2.1
  +#   security/libtasn1
   #   www/neon29
   #   sysutils/file
   #   archivers/xar
  @@ -2727,6 +2730,7 @@
   --with-db-tools-integrated \
   --with-popt=external \
   --with-beecrypt=external \
  +--with-libtasn1=external \
   --with-neon=external \
   --with-zlib=external \
   --with-file=external \
  @@ .
 __
 RPM Package Managerhttp://rpm5.org
 CVS Sources Repositoryrpm-...@rpm5.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/ devtool.conf

[Jeff Johnson]

On May 11, 2011, at 12:16 PM, Anders F Björklund wrote:

 
 No idea whether it works, but at least it seems to build...
 

From my investigations with TSS and TCG and TPM and MTM
which led me to a decision to attempt a ASN.1 embedding not
only into RPM libraries but also into the *.rpm/*.wdj format 
itself ...

... I was quite disgusted at just how feebly deficient actually is ...

... like many in the FL/OSS worls I'd assumed that there was a high quality,
AutoFu library that could be used anytime I wished ...

s
/sarcasm
... the sad reality of libtasn1 is that an integer is
assumed to be 32/64 bits. w00t! Let's do RSA-64! That is so
so so so secure! 
/sarcasm

... the point being that libtsan1 is headed for the bit bucket
anyways, and openssl is so so so much easier to use than gnutls
(but only by a hair's breadth) and the One True Crypto: NSS is actually
okay compared to the other ancient dinosaur (CDMA? CDMS? never can remember)
that is around for time os yore ...

... so holler if you wish me to dump libtasn1 sooner rather than later.

No problem.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: [CVS] RPM: rpm/rpmdb/ rpmdb.h

[Jeff Johnson]

This isn't your code. Leave it be.

WTF does C++ typecasting have to do with anything?

Do we go through a repeat of last week? Again?

73 de Jeff

On May 10, 2011, at 9:38 PM, Per Øyvind Karlsen wrote:

  RPM Package Manager, CVS Repository
  http://rpm5.org/cvs/
  
 
  Server: rpm5.org Name:   Per Øyvind Karlsen
  Root:   /v/rpm/cvs   Email:  pkarl...@rpm5.org
  Module: rpm  Date:   11-May-2011 03:38:43
  Branch: HEAD Handle: 2011051101384300
 
  Modified files:
rpm/rpmdb   rpmdb.h
 
  Log:
do explicit typecasting for c++ compatibility
 
  Summary:
RevisionChanges Path
1.143   +18 -18 rpm/rpmdb/rpmdb.h
  
 
  patch -p0 '@@ .'
  Index: rpm/rpmdb/rpmdb.h
  
  $ cvs diff -u -r1.142 -r1.143 rpmdb.h
  --- rpm/rpmdb/rpmdb.h9 Dec 2010 20:50:05 -   1.142
  +++ rpm/rpmdb/rpmdb.h11 May 2011 01:38:43 -  1.143
  @@ -860,7 +860,7 @@
   /*@*/
   {
   rpmdb rpmdb = (dbi ? dbi-dbi_rpmdb : NULL);
  -DB_TXN * _txn = (rpmdb ? rpmdb-db_txn : NULL);
  +DB_TXN * _txn = (DB_TXN*)(rpmdb ? rpmdb-db_txn : NULL);
   return _txn;
   }
 
  @@ -870,7 +870,7 @@
   /*@unused@*/ static inline
   int rpmlkId(rpmdb rpmdb, uint32_t *_idp)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
   int rc = (rpmdb-_dbi[0]-dbi_eflags  0x080)
   ? dbenv-lock_id(dbenv, _idp) : ENOTSUP;
   if (_rpmdb_debug)
  @@ -881,7 +881,7 @@
   /*@unused@*/ static inline
   int rpmlkIdFree(rpmdb rpmdb, uint32_t _id)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
   int rc = (rpmdb-_dbi[0]-dbi_eflags  0x080)
   ? dbenv-lock_id_free(dbenv, _id) : ENOTSUP;
   if (_rpmdb_debug)
  @@ -892,11 +892,11 @@
   /*@unused@*/ static inline
   int rpmlkGet(rpmdb rpmdb, DBT * _object, uint32_t _lockmode, void * _lock)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
   uint32_t _locker = 0x12344321;
   uint32_t _flags = 0;
   int rc = (rpmdb-_dbi[0]-dbi_eflags  0x080)
  -? dbenv-lock_get(dbenv, _locker, _flags, _object, _lockmode, _lock)
  +? dbenv-lock_get(dbenv, _locker, _flags, _object, 
 (db_lockmode_t)_lockmode, (DB_LOCK*)_lock)
   : ENOTSUP;
   if (_rpmdb_debug)
   fprintf(stderr, -- %s(%p,0x%x,0x%x,%p,0x%x,%p) rc %d\n, 
 dbenv-lock_get, dbenv, _locker, _flags, _object, _lockmode, _lock, rc);
  @@ -906,9 +906,9 @@
   /*@unused@*/ static inline
   int rpmlkPut(rpmdb rpmdb, void * _lock)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
   int rc = (rpmdb-_dbi[0]-dbi_eflags  0x080)
  -? dbenv-lock_put(dbenv, _lock)
  +? dbenv-lock_put(dbenv, (DB_LOCK*)_lock)
   : ENOTSUP;
   if (_rpmdb_debug)
   fprintf(stderr, -- %s(%p,%p) rc %d\n, dbenv-lock_put, dbenv, _lock, 
 rc);
  @@ -918,7 +918,7 @@
   /*@unused@*/ static inline
   int rpmlgcOpen(rpmdb rpmdb)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
   DB_LOGC * _logc = NULL;
   uint32_t _flags = 0;
   int rc = (rpmdb-_dbi[0]-dbi_eflags  0x100)
  @@ -954,7 +954,7 @@
   /*@unused@*/ static inline
   int rpmlgcFile(rpmdb rpmdb, const DB_LSN * _lsn, char * name, size_t len)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
   int rc = dbenv-log_file(dbenv, _lsn, name, len);
   if (_rpmdb_debug)
   fprintf(stderr, -- %s(%p,%p,%p[%u]) \%s\ rc %d\n, dbenv-log_file, 
 dbenv, _lsn, name, (unsigned)len, name, rc);
  @@ -964,7 +964,7 @@
   /*@unused@*/ static inline
   int rpmlgcFlush(rpmdb rpmdb, const DB_LSN * _lsn)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
   int rc = dbenv-log_flush(dbenv, _lsn);
   if (_rpmdb_debug)
   fprintf(stderr, -- %s(%p,%p) rc %d\n, dbenv-log_flush, dbenv, _lsn, 
 rc);
  @@ -974,8 +974,8 @@
   /*@unused@*/ static inline
   int rpmlgcPrintf(rpmdb rpmdb, const char * fmt, void *_A1, void *_A2, void 
 *_A3, void *_A4, void *_A5)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  -DB_TXN * _txnid = rpmdb-db_txn;
  +DB_ENV * dbenv = (DB_ENV*)rpmdb-db_dbenv;
  +DB_TXN * _txnid = (DB_TXN*)rpmdb-db_txn;
   int rc = dbenv-log_printf(dbenv, _txnid, fmt, _A1, _A2, _A3, _A4, _A5);
   if (_rpmdb_debug)
   fprintf(stderr, -- %s(%p,%p,\%s\, ...) rc %d\n, dbenv-log_printf, 
 dbenv, _txnid, fmt, rc);
  @@ -985,7 +985,7 @@
   /*@unused@*/ static inline
   int rpmlgcPut(rpmdb rpmdb, DB_LSN * _lsn, const DBT * data, uint32_t flags)
   {
  -DB_ENV * dbenv = rpmdb-db_dbenv;
  +DB_ENV * dbenv =