Re: [gentoo-portage-dev] [PATCH] Manifest2 reloaded

2006-03-16 Thread tvali
Just in case ...what i have to do to test those results


...ps. should i send it here if i have a working c++ class for forking
-- it evolved from that python thought here, which evolved into
general interest to those pipes and interacting with other apps in my
case (which is, as i have understood, important in unix-like
operating systems) ..therefore i encapsulate it into some generic c++
class to do piping and add some error checking, which would give
simple way to use scripting languages, too. i have still not installed
that c++ ide i like, but anyway, Kate is not so bad :) I have it
almost ready but it seems that i have to do some work now for a while
...do i send it here, too, when done or noone needs such thing
anymore? it just runs some command and gives a simple way to send
messages to its stdin and read its stdout so that interacting with
things like python could be simple, too.

2006/3/16, Brian Harring [EMAIL PROTECTED]:
 On Wed, Mar 15, 2006 at 11:14:04PM -0800, Donnie Berkholz wrote:
  Brian Harring wrote:
   python -m timeit -s 's=asdf*400;s+=fdsa.ebuild' 
   's.endswith(.ebuild)'
   100 loops, best of 3: 0.88 usec per loop
 
   python -m timeit -s 's=asdf*400;s+=fdsa.ebuild' 's[-7:] == .ebuild'
   100 loops, best of 3: 0.564 usec per loop
 
   Use endswith
 
   oddly, worth noting that startswith differs in this behaviour...
   python -m timeit -s 's=asdf*400;s+=fdsa.ebuild' 's[:7] == .ebuild'
   100 loops, best of 3: 0.592 usec per loop
 
   python -m timeit -s 's=asdf*400;s+=fdsa.ebuild' 
   's.startswith(.ebuild)'
   100 loops, best of 3: 0.842 usec per loop
 
  Um, those both read the same way to me. You just switched the ordering
  around, so the (starts|ends)with is on the bottom instead of the top,
  but both times (starts|ends)with is longer.

 This is why crack is bad, mm'kay.

 /me lights the pipe and goes back to his corner.

 Pardon, just did a quick test and screwed the results ;)
 ~harring






--
tvali
(e-mail: [EMAIL PROTECTED]; msn: [EMAIL PROTECTED];
icq: 317-492-912)

Ühe eesti internetifirma lehel kohtasin tsitaati:
If you don't do it excellently, dont do it at all. Because if it's not
excellent, it won't be profitable or fun, and if you're not in
business for fun or profit, what the hell are you doing here?
Robert Townsend

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] Wildcards in package.keywords

2006-03-16 Thread Brian Harring
On Thu, Mar 16, 2006 at 11:57:06AM +0100, Simon Stelling wrote:
 Pingveno wrote:
 kde-base/* ~x86
 
 or, to apply it to a single version, this:
 
 =kde-base/*-3.5.1 ~x86
 
 Regular expressions would, of course, work too. They might be a little
 bit of overkill, though.
 
 Bug 57153, was RESOLVED LATER
 
 http://bugs.gentoo.org/show_bug.cgi?id=57153

If anyone is bored, I'd be interested in an implementation of that for 
pkgcore (formerly bcportage, formerly saviour, formerly savior, 
formerly the portage rewrite)...

Implementation of it would require modification to 
pkgcore/config/domain.py , namely changing the package_keywords 
splitter to use 

from pkgcore.restrictions import values, packages
from pkgcore.package.atom import VersionMatch
packages.BooleanRestriction(
   packages.PackageRestriction(category, values.StrExactmatch(cat)),
   Versionmatch(ver_component, rev_component)
)

instead of just passing the str to atom, and change the DictBased 
filter used for keywords adjustment.

Bit wordy, but it's actually minor to do- 20 lines or so, just not top 
priority for me (eg I'll get it someday down the line if I remember) :)

~harring


pgpPclLxHWDYE.pgp
Description: PGP signature


[gentoo-portage-dev] Portage sqlite support

2006-03-16 Thread tvali
Brian told that portage already has sqlite support.

Where i can find it or how i can turn it on? Is there any bugs or
other reasons to not use it? I would like to support it's code as this
is nr. 1 priority for me right now :)

--
tvali
(e-mail: [EMAIL PROTECTED]; msn: [EMAIL PROTECTED];
icq: 317-492-912)

I like net more than life, cause if i do something wrong, then people
in net will tell me that i do, so that i can fix it -- people in life
will tell others that i do.

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Paul de Vrieze
On Wednesday 15 March 2006 16:13, Gustavo Sverzut Barbieri wrote:
 Hello,

 There is any provision for binary dependency on Gentoo/Portage? The
 way it works now is quite messy with things like revdep-rebuild.

 I have an idea to solve this problem: after software is build, you
 check which files it links (ldd binaries libraries) and check the used
 against installed packages. If a library is not provided by  an
 installed package we could have a policy to inform user or just abort
 installation.

 Also, if we implement these dependencies in rpm-generator, we could
 just generate RPM packages and install it in the RPM-DB and let it
 handle these kind of things.

 With these in place emerge would handle the build stage (where it
 excels), but rpm would handle the binary installation and dependencies
 (where it excels).

Solving this is not trivial. Basically suppose application X depends on 
sys-libs/db-4.*

This can be resolved by differently slotted libraries. We need to record 
which one was actually used (not easy by itself, but more an issue of the 
ebuild itself, if more than 1 candidate is available). But suppose that 
we know that the application was compiled to use db-4.3.29. We must then 
know that it is ok to replace the db-4.3.29 package with 4.3.30, but that 
it isn't ok to replace it by 4.3.28 or 4.4.20.

To make this things worse, the above example assumes that within a slot, 
the libraries are binary compatible. There are examples of libraries that 
are not. And what about a library whose interface is dependent on a third 
library: B uses A, C uses B, but B exports A. So B is dependent on A, and 
the binary package of C must record that B was compiled with A.

In short, welcome to binary package hell. This is the reason that binary 
distributions must use versions. Even debian. It is just very very hard 
to fix these kinds of indirect dependencies.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpqRcIBYHhbT.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread tvali
 To make this things worse, the above example assumes that within a slot,
 the libraries are binary compatible. There are examples of libraries that
 are not. And what about a library whose interface is dependent on a third
 library: B uses A, C uses B, but B exports A. So B is dependent on A, and
 the binary package of C must record that B was compiled with A.

 In short, welcome to binary package hell. This is the reason that binary
 distributions must use versions. Even debian. It is just very very hard
 to fix these kinds of indirect dependencies.

I think that there are many reasons, why such kind of important thing
should not be automatically generated. Starting from the fact that one
thing is dependency in binary, another thing is knowledge if this
combination actually works, has no bugs, should not be masked.

Anyway, if such binary dependancyes should ever be used, then it would
be best to:
1. register version number of interfaces
2. register interface groups

I mean:
If some lib supports commands turnon() and turnoff(), but one version
supports additionally command int checkifturnedon() and another
supports char checkifturnedon(), then there should be:
* Interface group, which states only that there are turnon and turnoff
functions -- as many apps use only most standard and minimalistic set
of commands from one lib, there is no need to know if some advanced
features are 0.9.80 or 0.9.64 compatible.
* Two interface versions, 0.9.80a and 0.9.80b, for example

All those interfaces and groups could be file links --
lib_interfacegroup_interface1.a would link to libinterface1.a.

If there is 1 library with 2 sets of minimal functionality, then it's
questionable, how to implement both (there would be linker error in
some cases) -- which makes this conception weaker.

Anyway, when standard interfaces are used and linked, that would help a bit.

Anyway -- why should i check dependencies *after* building of a big
pack, maybe hours of building, when i can check it without even
downloading it, when there is nice portage tree? And, i think that
making it binary would allow too much bad style -- there are imho
things, which should not be automated without very-very careful
thinking even if one can only win with automating them in theory and
portage tree is most definitely one of them.

 Paul

 --
 Paul de Vrieze
 Gentoo Developer
 Mail: [EMAIL PROTECTED]
 Homepage: http://www.devrieze.net





--
tvali
(e-mail: [EMAIL PROTECTED]; msn: [EMAIL PROTECTED];
icq: 317-492-912)

I like net more than life, cause if i do something wrong, then people
in net will tell me that i do, so that i can fix it -- people in life
will tell others that i do.

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Brian Harring
On Thu, Mar 16, 2006 at 02:58:00PM +0100, Paul de Vrieze wrote:
 On Wednesday 15 March 2006 16:13, Gustavo Sverzut Barbieri wrote:
  Hello,
 
  There is any provision for binary dependency on Gentoo/Portage? The
  way it works now is quite messy with things like revdep-rebuild.

 Solving this is not trivial. Basically suppose application X depends on 
 sys-libs/db-4.*
 
 This can be resolved by differently slotted libraries. We need to record 
 which one was actually used (not easy by itself, but more an issue of the 
 ebuild itself, if more than 1 candidate is available). But suppose that 
 we know that the application was compiled to use db-4.3.29. We must then 
 know that it is ok to replace the db-4.3.29 package with 4.3.30, but that 
 it isn't ok to replace it by 4.3.28 or 4.4.20.
 
 To make this things worse, the above example assumes that within a slot, 
 the libraries are binary compatible. There are examples of libraries that 
 are not. And what about a library whose interface is dependent on a third 
 library: B uses A, C uses B, but B exports A. So B is dependent on A, and 
 the binary package of C must record that B was compiled with A.
 
 In short, welcome to binary package hell. This is the reason that binary 
 distributions must use versions. Even debian. It is just very very hard 
 to fix these kinds of indirect dependencies.

You're forgetting we're equally screwed when upstream (*cough* 
openssl) goes and changes the soname while preserving abi.

Old solution to it was to add another metadata key to ebuilds, 
bincompat that is compared within slotting to see if a rebuild of 
rdepend consumers needs to occur.

fex...
alsa-lib-0.90
slot=0
bincompat=0

alsa-lib-1.0
slot=0
bincompat=1

The pkg manager does it's little dance to figure out what should be 
installed where (and what versions can coexist), while doing it if 
it's going to replace a pkg and bincompat differs, pull in the subset 
of the state graph at that point (including vdb) that rdeps on that 
package/slot and force a rebuild of them.

As per the norm, requires a smart resolver; for c++ would expect 
cycles to occur where the only solution is to pull in libstdc++ (fex) 
to sidestep horkage while doing the rebuilds...

Also note that (like all general solutions) it's not going to be able 
to cover all insane stuff; if the api breaks in transitioning 
bincompat is going to cause a minor explossion that is only resolvable 
by packages specifying exact dependencies (read: version ranges).
~harring


pgpFDZuONXCnW.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Paul de Vrieze
On Thursday 16 March 2006 15:18, tvali wrote:
  To make this things worse, the above example assumes that within a
  slot, the libraries are binary compatible. There are examples of
  libraries that are not. And what about a library whose interface is
  dependent on a third library: B uses A, C uses B, but B exports A. So
  B is dependent on A, and the binary package of C must record that B
  was compiled with A.
 
  In short, welcome to binary package hell. This is the reason that
  binary distributions must use versions. Even debian. It is just very
  very hard to fix these kinds of indirect dependencies.

 I think that there are many reasons, why such kind of important thing
 should not be automatically generated. Starting from the fact that one
 thing is dependency in binary, another thing is knowledge if this
 combination actually works, has no bugs, should not be masked.

 Anyway, if such binary dependancyes should ever be used, then it would
 be best to:
 1. register version number of interfaces
 2. register interface groups

 I mean:
 If some lib supports commands turnon() and turnoff(), but one version
 supports additionally command int checkifturnedon() and another
 supports char checkifturnedon(), then there should be:
 * Interface group, which states only that there are turnon and turnoff
 functions -- as many apps use only most standard and minimalistic set
 of commands from one lib, there is no need to know if some advanced
 features are 0.9.80 or 0.9.64 compatible.
 * Two interface versions, 0.9.80a and 0.9.80b, for example

While theoretically one can't even know that the semantics are correct, we 
know from experience that hell will not break through. Downgrading 
libraries is always problematic though. It's not desirable enough to try 
to support it.


 All those interfaces and groups could be file links --
 lib_interfacegroup_interface1.a would link to libinterface1.a.

Looking at a sub-package level is going to make things way too complex. I 
don't think that doing so is going to help gentoo/portage in any way.

 If there is 1 library with 2 sets of minimal functionality, then it's
 questionable, how to implement both (there would be linker error in
 some cases) -- which makes this conception weaker.

 Anyway, when standard interfaces are used and linked, that would help a
 bit.

IMHO we need to know from every binary or installed package which 
package version and slot was used to satisfy it. We also need to know 
which of those dependencies must be exported to packages which use this 
one.

All this must also take useflags into account ;-) of course.


 Anyway -- why should i check dependencies *after* building of a big
 pack, maybe hours of building, when i can check it without even
 downloading it, when there is nice portage tree? And, i think that
 making it binary would allow too much bad style -- there are imho
 things, which should not be automated without very-very careful
 thinking even if one can only win with automating them in theory and
 portage tree is most definitely one of them.

The difference between binary packages and tracking what is installed with 
which relations are very small. Basically once compiled, the candidates 
that may resolve a dependency are fewer. While a BINSLOT variable could 
be an option (Although I'm not sure whether this would be needed), we 
still need to know the minimal version of the package within the slot.

It's not about checking dependencies after building. It's about seeing 
whether one should rebuild package B because it depends on package A that 
has been rebuilt. Perhaps package A is slotted, so I don't need to 
rebuilt. But then when I want to remove the old version, portage should 
realise that while the source dependencies are still met, the binary 
dependencies have not been met, so a rebuild is needed.

Revdep-rebuild is not going to solve this. It is a hack because it does 
not work on portage/package level. It tries to understand the elf 
(executable) format and as a result it tries to find libraries and 
executables (can't find all though, if they are installed at the wrong 
place) that depend on a particular library. Then it tries to find the 
package that the dependant library/executable comes from. As however 
libraries may have unresolved symbols, some libraries (e.g. libapr-util) 
do not actually state their dependencies.

Further revdep-rebuild does not work with downgrades, or with scripts or 
other things that are not in the elf format. Finally revdep-rebuild needs 
to scour the filesystem too much.

Recording binary dependencies can be used to resolve this. It is however 
not easy to do.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgp9RhC4YUuuN.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Paul de Vrieze
On Thursday 16 March 2006 15:24, Brian Harring wrote:
I would have called bincompat BINSLOT, but the idea is the same.

 As per the norm, requires a smart resolver; for c++ would expect
 cycles to occur where the only solution is to pull in libstdc++ (fex)
 to sidestep horkage while doing the rebuilds...

Don't start about c++. Suppose you have a library A that is build against 
libstdc++ 5, It contains a class B with a toString method that returns an 
std::string instance. Next we have application C that extends class B in 
class D. It (also) redefines the toString method. This application is 
binary and we don't have (or don't want to use) the source. It was 
however linked against libstdc++ 6.

Question:
Which std::string should class D return in it's toString method? The one 
from libstdc++5 or from libstdc++6 ?

(And for a moment disregard the fact that actually the string classes are 
binary compatible so it doesn't matter).


-- answer follows after some lines ---













Answer:
There is no right answer. We have hit a situation whose solution is 
undefined. In reality this is solved by the fact that the dynamic 
linker does not look at which library a particular symbol (read name) is 
from.

With linker symbol versions, what happens is that the types are silently 
cast. This however breaks runtime type information. We can namely now get 
the situation where two std::string instances do not actually have the 
same runtime type. This breaks casting. Fun isn't it.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpTksQAd6Qop.pgp
Description: PGP signature