Re: [gentoo-portage-dev] DB and binary dependency

2006-03-26 Thread Paul de Vrieze
On Friday 24 March 2006 15:24, tvali wrote:
  Unfortunately, your wrong. This only makes sure that you have the
  right slots to put your squares, triangles and circles in. It does
  not say that b(int,int) from the first lib actually does the same
  thing as b(int,int) from the second library.

 Sorry i thought about it for a moment, then left out later as broken
 design cases didnt seems so important. Versioning of distinct
 functions would be good, anyway not automatic.

In many cases these things are not broken design. My example is, to make 
it clear. Unfortunately at the moment you do automatic dependencies, you 
MUST be correct between 99.9% and 100%. These cases are too common to 
disregard. Changing semantics are common enough. Sometimes the old 
semantics are just broken.

  While the above example is clearly broken design, this does happen
  enough in actual libraries in way more subtle ways. And that is
  disregarding the fact that the linux/elf ABI does not include
  argument lists in symbol linking. As such b(int,char) is
  indistinguishable from b(int,int). To overcome this C++ uses name
  mangling which creates names based on the signature.

 Ok, assuming broken design it's harder. Anyway, how would binary deps
 solve this? This seems to be humans work in all cases.

We don't look at what packages do. We take a default heuristic, and let 
humans give the correct answer when the default is wrong. 

 Anyway, as .h files have become as well as standard, you may show in
 code as well as in bin, which functions are needed.

Except that the way header files are used is not standard. You know there 
is a reason for the way gcc handles precompiled header files. That is 
because of the way they work. Header files can have unpredictable 
interactions. Including A then B can be really really different from 
including B then A. While in most cases it isn't, it is in too many 
cases.

 Is there any advantage of bin dep over code dep is a question for me
 ..some here suggest that there is, i think that everything can be done
 as well with code deps.

bin dep is not about how to get the dependencies. Binary dependencies 
specify what the dependencies have become after compilation. Binary 
dependencies basically strengthen the runtime dependencies, by taking the 
concrete dependencies used into account.

With binary dependencies comes the need to verify them. This is because 
too many ebuilds do not use the version of a multislotted library that is 
expected. As such verifying the dependencies of a binary package gives 
the opportunity for developers to signal and fix this.


   I'm actually sure that this all can be calculated up from
   sourcecode and bindep would be after that a check if cpu didnt
   calculate something wrong :P Another question, how difficult it is
   and is it worth the time.
 
  Perhaps you should read up your knowledge of the C language. After
  you found that the C language is a mess, try C++, it makes things
  worse. After that's finished take a look at solving this problem for
  ALL languages.

 I actually know c++, but dont know c. Anyway, what i mean is that if
 you have .h files, you may make up automatic check if interfaces fit
 each other. I have to think about if c++ is a mess :)

C++ is every bit as much a mess as C as it is a superset. The problem is 
actually partly in the C language and partly in the preprocessor. First 
the preprocessor is fragile, and can depend on arbitrary command line 
options (Defines). The language problem is that C/C++ has no notion of 
modules. file a.c can define a function as int a(char) and b.c can have 
its prototype as int a(wchar). The compiler will not complain about 
this, AND with the right defines it IS actually the same.

The fundamental problem is that the compiler only knows about symbols that 
may be imported from other object files. It does not look at the actual 
object to look at the prototypes of the symbols defined there (nor do .o 
files outputted by gcc contain this information). Instead header files 
can be used to share this information. Often they do this in a broken way 
and errors similar to the above exist.


  Automatic source analysis for dependency calculation is a dead end.
  Even if you manage to find the proper interfaces (oops, the package
  had it's own gl.h instead of using the system one), you don't know
  anything about the semantics of those interfaces. Two things with the
  same name may very well have very different behaviours.

 I think it should be solved with:
 // system's header
 #include ...
 // own header
 #include ...

It's not your code, you can't change it. And including two headers which 
define the same symbols can be disastrous.

 Ok -- i agree that there are several problems, which have to be solved
 before taking this topic up again and that they are possibly
 unsolvable in many cases and possibly too much work in many others.

The thing is,

with reasonable effort you are probably 

Re: [gentoo-portage-dev] DB and binary dependency

2006-03-24 Thread tvali
On 24/03/06, Paul de Vrieze [EMAIL PROTECTED] wrote:
 On Thursday 23 March 2006 21:35, tvali wrote:
 
  BINSLOT is a new word for me.

 Ok BINSLOT is normally slot. However in some cases packages are in the
 same slot, but not binary compatible (like their libraries having a
 different SONAME e.g. openssl-0.9.6 and 0.9.7 (iirc)).

 Calculating after build is only for validating configurations. It allows
 one to see whether one can merge binary package A on the current system

Ok, i'm starting to fully understand it :)

So it is something what could be helpful even on other linux systems
than gentoo?

 Header files come from packages that should be mentioned in the
 dependencies.

Yes i'm here thinking about my code-deps :)

 Calculating dependencies automatically is diabolically hard. The fact that
 we have a tree that largely works shows that predicting them works well.
 The reason is that the dependencies are statically defined by the source
 of the package. At those parts where people make mistakes, or the build
 systems are broken, occur the dependencies could be verified by automatic
 tools that check most dependencies. Doing that for native code is easiest
 done by disecting the libraries and executables. For header files,
 inclusion parsing is probably a good approach. But then look at what is
 installed, not what was started with. The reason being that at what is
 installed is no longer dynamic based on a configure script, but totally
 static. The only disadvantage being that this only checks the current
 combination of useflags+context. Many checks by many users would however
 give a good test coverage.

If i can understand you now, then i would think that you didnt
understand me fully. I will write my script, then..

 Not really, with a static library A and a static library B that uses the
 static A, this dependency is only runtime when the interface of B exposes
 the headers of A. If not, one could unmerge both the static library and
 the headers and one could still compile against the library.

I meant headers as .h or .hpp files, which contain function headers.
They're needed, but they are in many cases, included in pack which
uses them, even if that means putting some .h's into all libs.

 Ok, let me clarify. With a binary dependency I mean the actual
 requirements of a package once it has been compiled. In general one could
 say that given a depend atom sys-libs/db-4.* could be statisfied by
 ebuilds in 5 slots: db-4.{0,1,2,3,4}. The sourcecode does not care which
 one is used. However, code compiled agaist db-4.0.14 will not run against
 any libdb-4.{1,2,3,4}.*. This is because these should libraries live in
 different BINSLOT's (in other words, they are incompatible). As these
 libraries have different SLOT's they can coexist though. For a binary
 package therefore it is necessary to record the version of db it was
 actually compiled against. As this might be very strict I also want to
 say that any package within the same BINSLOT with a higher version than
 the one compiled against also works (as it does).

Cant understand it. Do the *headers* change so much through versions?
Is there no backward-compability in so many cases?

 Paul

 --
 Paul de Vrieze
 Gentoo Developer
 Mail: [EMAIL PROTECTED]
 Homepage: http://www.devrieze.net





--
tvali

Measuring programming progress by lines of code is like measuring
aircraft building progress by weight. -Bill Gates

For every complex problem there is an answer that is clear, simple,
and wrong. -H L Mencken

Theory is when you know something, but it doesn't work. Practice is
when something works, but you don't know why. Programmers combine
theory and practice: Nothing works and they don't know why.

http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-24 Thread Brian Harring
On Fri, Mar 24, 2006 at 01:40:01PM +0200, tvali wrote:
 On 24/03/06, Paul de Vrieze [EMAIL PROTECTED] wrote:
  On Thursday 23 March 2006 21:38, Gustavo Sverzut Barbieri wrote:
   Cons:
- it's not the final solution to the problem, as said, interfaces
   would be better... but interfaces would demand much more effort and
   not being automatically generated, would be async and probably
   incorrect at some point
 
  Interfaces, while nice, would overcomplicate things. They are also not
  needed as we have depend atoms.
 
  Paul
 
 Interface can be made somewhat automatically checkable.
Checking the interfaces/symbols sucks however, because you can only do 
it _after_ you've built whatever you're building (packages do adjust 
the defines/typedefs/structs dependant on configure/build options).

As I stated earlier, bincompat (not binslot paul :P) is the route to 
go- it gives you up front information so a resolver can plan out what 
has to be rebuilt automatically.

~harring



pgpVQWlcq5D0D.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-24 Thread tvali
On 24/03/06, Brian Harring [EMAIL PROTECTED] wrote:
 Checking the interfaces/symbols sucks however, because you can only do
 it _after_ you've built whatever you're building (packages do adjust
 the defines/typedefs/structs dependant on configure/build options).

 As I stated earlier, bincompat (not binslot paul :P) is the route to
 go- it gives you up front information so a resolver can plan out what
 has to be rebuilt automatically.

no-no-no i'm not talking about binary and i will make another thread
when i go further with my own topic :) to use those interfaces, u dont
need to compile anything.

 ~harring

--
tvali

Measuring programming progress by lines of code is like measuring
aircraft building progress by weight. -Bill Gates

For every complex problem there is an answer that is clear, simple,
and wrong. -H L Mencken

Theory is when you know something, but it doesn't work. Practice is
when something works, but you don't know why. Programmers combine
theory and practice: Nothing works and they don't know why.

http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-24 Thread Paul de Vrieze
On Friday 24 March 2006 12:40, tvali wrote:

 Interface can be made somewhat automatically checkable.

 For example:
 void a(int);
 void b(int, int);
 void b(int, char);

 Is compatible with:
 void a(int);
 void b(int, int);

Unfortunately, your wrong. This only makes sure that you have the right 
slots to put your squares, triangles and circles in. It does not say that 
b(int,int) from the first lib actually does the same thing as b(int,int) 
from the second library.

To make it worse, suppose that header file 1 defines the following macro:
#define c(x,y) { int A = x; int B = y; if (AB) { b(A,B); } else 
{ b(B,A); }

And header file 2 defines the following macro
#define c(x,y) { int A = x; int B = y; if (AB) { b(A,B); } else 
{ b(B,A); }

And the semantics of B are such that one is supposed to use c(x,y) to call 
b, and b1 accordingly expects the first argument to be smaller, and b2 
expects the second one to be smaller. On the source level, this macro 
makes it transparent. The libraries, although they may seem different, 
are incompatible. While they implement the same source interface, they do 
NOT implement the same binary interface. On a binary (read compiled) 
level, they can not be exchanged for eachother.

While the above example is clearly broken design, this does happen enough 
in actual libraries in way more subtle ways. And that is disregarding the 
fact that the linux/elf ABI does not include argument lists in symbol 
linking. As such b(int,char) is indistinguishable from b(int,int). To 
overcome this C++ uses name mangling which creates names based on the 
signature.

 And this can be checked automatically - real header must have all
 headings that interface describes. Microsoft has actually done good
 work in C# on that topic -- first place where i saw something like
 that after thinking about it myself :) This MS version is feature of a
 language, of course, not a feature of compiler. Anyway, that might
 give an idea.

There is a reason that MS implements it on the language level, in C#. It 
is impossible to do while keeping the (admittedly extremely poor) 
semantics of C and C++. Unfortunately the linux ABI (Application Binary 
Interface) is based on C. As such one has to take C into account.

 I'm actually sure that this all can be calculated up from sourcecode
 and bindep would be after that a check if cpu didnt calculate
 something wrong :P Another question, how difficult it is and is it
 worth the time.

Perhaps you should read up your knowledge of the C language. After you 
found that the C language is a mess, try C++, it makes things worse. 
After that's finished take a look at solving this problem for ALL 
languages.

Automatic source analysis for dependency calculation is a dead end. Even 
if you manage to find the proper interfaces (oops, the package had it's 
own gl.h instead of using the system one), you don't know anything about 
the semantics of those interfaces. Two things with the same name may very 
well have very different behaviours.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgp9iE5mTKGuU.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-24 Thread Paul de Vrieze
On Friday 24 March 2006 13:10, Brian Harring wrote:
 As I stated earlier, bincompat (not binslot paul :P) is the route to

If you want to call it bincompat, I'd have to insist to make it 
BINCOMPAT ;-).

 go- it gives you up front information so a resolver can plan out what
 has to be rebuilt automatically.

An argument I forgot to mention indeed.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpeje4vSUL4a.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-24 Thread tvali
On 24/03/06, Paul de Vrieze [EMAIL PROTECTED] wrote:
 On Friday 24 March 2006 12:40, tvali wrote:
 
  Interface can be made somewhat automatically checkable.
 
  For example:
  void a(int);
  void b(int, int);
  void b(int, char);
 
  Is compatible with:
  void a(int);
  void b(int, int);
 
 Unfortunately, your wrong. This only makes sure that you have the right
 slots to put your squares, triangles and circles in. It does not say that
 b(int,int) from the first lib actually does the same thing as b(int,int)
 from the second library.

Sorry i thought about it for a moment, then left out later as broken
design cases didnt seems so important. Versioning of distinct
functions would be good, anyway not automatic.

 To make it worse, suppose that header file 1 defines the following macro:
 #define c(x,y) { int A = x; int B = y; if (AB) { b(A,B); } else
 { b(B,A); }

 And header file 2 defines the following macro
 #define c(x,y) { int A = x; int B = y; if (AB) { b(A,B); } else
 { b(B,A); }

 And the semantics of B are such that one is supposed to use c(x,y) to call
 b, and b1 accordingly expects the first argument to be smaller, and b2
 expects the second one to be smaller. On the source level, this macro
 makes it transparent. The libraries, although they may seem different,
 are incompatible. While they implement the same source interface, they do
 NOT implement the same binary interface. On a binary (read compiled)
 level, they can not be exchanged for eachother.

 While the above example is clearly broken design, this does happen enough
 in actual libraries in way more subtle ways. And that is disregarding the
 fact that the linux/elf ABI does not include argument lists in symbol
 linking. As such b(int,char) is indistinguishable from b(int,int). To
 overcome this C++ uses name mangling which creates names based on the
 signature.

Ok, assuming broken design it's harder. Anyway, how would binary deps
solve this? This seems to be humans work in all cases.

  And this can be checked automatically - real header must have all
  headings that interface describes. Microsoft has actually done good
  work in C# on that topic -- first place where i saw something like
  that after thinking about it myself :) This MS version is feature of a
  language, of course, not a feature of compiler. Anyway, that might
  give an idea.

 There is a reason that MS implements it on the language level, in C#. It
 is impossible to do while keeping the (admittedly extremely poor)
 semantics of C and C++. Unfortunately the linux ABI (Application Binary
 Interface) is based on C. As such one has to take C into account.

Thats true that C# has much more clearness in it's language about what is what.

Anyway, as .h files have become as well as standard, you may show in
code as well as in bin, which functions are needed.

Is there any advantage of bin dep over code dep is a question for me
..some here suggest that there is, i think that everything can be done
as well with code deps.

  I'm actually sure that this all can be calculated up from sourcecode
  and bindep would be after that a check if cpu didnt calculate
  something wrong :P Another question, how difficult it is and is it
  worth the time.

 Perhaps you should read up your knowledge of the C language. After you
 found that the C language is a mess, try C++, it makes things worse.
 After that's finished take a look at solving this problem for ALL
 languages.

I actually know c++, but dont know c. Anyway, what i mean is that if
you have .h files, you may make up automatic check if interfaces fit
each other. I have to think about if c++ is a mess :)

 Automatic source analysis for dependency calculation is a dead end. Even
 if you manage to find the proper interfaces (oops, the package had it's
 own gl.h instead of using the system one), you don't know anything about
 the semantics of those interfaces. Two things with the same name may very
 well have very different behaviours.

I think it should be solved with:
// system's header
#include ...
// own header
#include ...

But, yes, not in all cases.

Ok -- i agree that there are several problems, which have to be solved
before taking this topic up again and that they are possibly
unsolvable in many cases and possibly too much work in many others.

 Paul

 --
 Paul de Vrieze
 Gentoo Developer
 Mail: [EMAIL PROTECTED]
 Homepage: http://www.devrieze.net





--
tvali

If I had eight hours to chop down a tree, I'd spend six sharpening my
axe. -Abraham Lincoln

http://www.kuro5hin.org/story/2005/1/28/32622/4244

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread tvali
Can someone tell me, which portage python commands should be used or
which kind of file created, if i'm going to test this idea? -- in
beginning, i would like to just add simple deps - are ebuilds the only
place to change and is there any clear doc of them [as i wouldnt like
to go through them all to be sure that there is no rarely used things i
dont know]?

Also, where i can find relations between useflags and c/c++ #defines?-- tvaliMeasuring programming progress by lines of code is like measuring aircraft building progress by weight. -Bill Gates
For every complex problem there is an answer that is clear, simple, and wrong. -H L MenckenTheory
is when you know something, but it doesn't work. Practice is when
something works, but you don't know why. Programmers combine theory and
practice: Nothing works and they don't know why.http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread Alec Warner

tvali wrote:
Can someone tell me, which portage python commands should be used or 
which kind of file created, if i'm going to test this idea? -- in 
beginning, i would like to just add simple deps - are ebuilds the only 
place to change and is there any clear doc of them [as i wouldnt like to 
go through them all to be sure that there is no rarely used things i 
dont know]?


Also, where i can find relations between useflags and c/c++ #defines?


USE flags technically have anything to do with C++ defines; they can 
technically enable/disable any optional support ( docs, plugins, etc.. )
Usually USE flags interact with a packages build system (configure or 
whatnot)


Dependencies are only in DEPEND, RDEPEND,PDEPEND.




--
tvali

Measuring programming progress by lines of code is like measuring 
aircraft building progress by weight. -Bill Gates


For every complex problem there is an answer that is clear, simple, and 
wrong. -H L Mencken


Theory is when you know something, but it doesn't work. Practice is when 
something works, but you don't know why. Programmers combine theory and 
practice: Nothing works and they don't know why.


http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/


--
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread tvali
On 23/03/06, Alec Warner [EMAIL PROTECTED] wrote:
 tvali wrote:
  Can someone tell me, which portage python commands should be used or
  which kind of file created, if i'm going to test this idea? -- in
  beginning, i would like to just add simple deps - are ebuilds the only
  place to change and is there any clear doc of them [as i wouldnt like to
  go through them all to be sure that there is no rarely used things i
  dont know]?
 
  Also, where i can find relations between useflags and c/c++ #defines?

 USE flags technically have anything to do with C++ defines; they can
 technically enable/disable any optional support ( docs, plugins, etc.. )
 Usually USE flags interact with a packages build system (configure or
 whatnot)


In c++ code, use flags must definitely have something to do with
#defines. Otherwise they couldnt do what they do -- so
uflags=buildsys=defines.

Configure, for example, will define set of #defines.

Ok, your answer at least shows that this relation is not as simple and
clear as i would like it :)

Where i could see some useflag actually interacting with build? Where
i see that line which makes it so that if i define gtk, gtk support
will be added to makefile?

--
tvali

Measuring programming progress by lines of code is like measuring
aircraft building progress by weight. -Bill Gates

For every complex problem there is an answer that is clear, simple,
and wrong. -H L Mencken

Theory is when you know something, but it doesn't work. Practice is
when something works, but you don't know why. Programmers combine
theory and practice: Nothing works and they don't know why.

http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread tvali
>From Paul de Vrieze:The semantics that make up the relationships between useflags and the actual
source as goes out of the preprocessor is very complicated. Probably theeasiest way to find it out is to try each permutation and somehow hook intogcc/g++ to get the result of that choice.And that's only for C and C++ sources.

Ok, thanks :)

So this interaction is one more thing to get simpler? I'm starting to
think that i should seek for some very-very small part of portage to
develop, because size of things [amount of work], which i already think
i should improve in some way, is getting to somewhere near infinity :)
...ok, near what infinity is for me, not theoretical infinity ;)

What is the simplest way to get one portage running in some virtual
space where i could install and uninstall without actually changing my
system? Using chroot in the same way as when installing gentoo is good?

-- tvaliMeasuring programming progress by lines of code is like measuring aircraft building progress by weight. -Bill GatesFor every complex problem there is an answer that is clear, simple, and wrong. -H L Mencken
Theory
is when you know something, but it doesn't work. Practice is when
something works, but you don't know why. Programmers combine theory and
practice: Nothing works and they don't know why.http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread Paul de Vrieze
On Thursday 23 March 2006 20:53, tvali wrote:
 So this interaction is one more thing to get simpler? I'm starting to think
 that i should seek for some very-very small part of portage to develop,
 because size of things [amount of work], which i already think i should
 improve in some way, is getting to somewhere near infinity :) ...ok, near
 what infinity is for me, not theoretical infinity ;)

 What is the simplest way to get one portage running in some virtual space
 where i could install and uninstall without actually changing my system?
 Using chroot in the same way as when installing gentoo is good?

Certainly,

chroot combined with lvm snapshots would be the easiest way.

If you want to focus on binary packages, you might want to start with not 
doing it automatically, but using some crude heuristics. You can make it 
configurable for when the heuristics don't work.

Basically what is the difference between a binary package and the source 
package is the dependencies.

You can regard a dependency as a restriction on the configurations in which 
the package can be installed. For binary packages only runtime dependencies 
are relevant (well in gentoo anyway, we don't have initial configuration 
deps). Binary packages do however further restrict the configurations from 
the runtime dependencies. These restrictions originate from the configuration 
when compiling the package.

Then I have some good and bad news. The good news:
- Binary dependencies are as necessary for binary packages, as for validating
  a current configuration (set of installed packages).
The bad:
- For proper binary dependencies the depend expression possibilities must be
  extended. Useflag dependencies would be at least usefull. BINSLOT would be
  almost required. We already saw that SLOT can't always be used for BINSLOT.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpC1Bf8otT4G.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread tvali
On 23/03/06, Paul de Vrieze [EMAIL PROTECTED] wrote:
Certainly,chroot combined with lvm snapshots would be the easiest way.If you want to focus on binary packages, you might want to start with notdoing it automatically, but using some crude heuristics. You can make it
configurable for when the heuristics don't work.Basically what is the difference between a binary package and the sourcepackage is the dependencies.You can regard a dependency as a restriction on the configurations in which
the package can be installed. For binary packages only runtime dependenciesare relevant (well in gentoo anyway, we don't have initial configurationdeps). Binary packages do however further restrict the configurations from
the runtime dependencies. These restrictions originate from the configurationwhen compiling the package.Then I have some good and bad news. The good news:- Binary dependencies are as necessary for binary packages, as for validating
a current configuration (set of installed packages).The bad:- For proper binary dependencies the depend _expression_ possibilities must beextended. Useflag dependencies would be at least usefull. BINSLOT would be
almost required. We already saw that SLOT can't always be used for BINSLOT.
BINSLOT is a new word for me.

I dont see any meaning in calculating dependencies after build. There would be too much problems:
* If any of packages you depend on, are masked, you get an error after build, but should get it before
* In many cases, not only libs, but also headers are included from
another pack [kernel headers, for example, as i have seen from error
messages]
* You cant pre-calculate dependancies

My interest, for sure, is code dependancies. At first i see a clear way
to get it work for now, secondly i think it can be used for bin deps,
also.

In cases, where both headers and libs are included from other pack,
which is same on both cases, this is totally irrelevant for deps if
this other pack is statically or dynamically used.

In cases, where headers are in pack to be compiled and .so is in
another package, i havent yet checked, how to understand if dependancy
is binary -- i cant say for sure that this does not include some work
from user.

Ok, i better write few lines of code now as i know where i should start
(or at least i have one part of code, which could be discussed here, in
front of my minds-eye).

Thanks. You will understand me better when something works :)
Paul--Paul de VriezeGentoo DeveloperMail: 
[EMAIL PROTECTED]Homepage: http://www.devrieze.net-- tvaliMeasuring programming progress by lines of code is like measuring aircraft building progress by weight. -Bill Gates
For every complex problem there is an answer that is clear, simple, and wrong. -H L MenckenTheory
is when you know something, but it doesn't work. Practice is when
something works, but you don't know why. Programmers combine theory and
practice: Nothing works and they don't know why.http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread Gustavo Sverzut Barbieri
Ok... this discussion is missing my initial point that is how to
provide binary dependency and avoid many crashes we have now with
almost no effort.

My initial proposal was to, after compile and before install is done,
we should parse linker information and check for each library it
depends, which package-version provides it and then mark the just
compiled package runtime-dependent on them. If the dependency was not
installed by portage, then issue a warning/error and take some action
(ask user, abort or ignore)

Pros:
 - it would avoid many crashes we have now because we remove packages
that others depend on.
 - it would avoid many crashes we have now because we update packages
that others depend on, but are not listed correctly in ebuilds.
 - it would avoid dependency errors caused by missing
--disable-feature with autoconf-like packages

Cons:
 - it's not the final solution to the problem, as said, interfaces
would be better... but interfaces would demand much more effort and
not being automatically generated, would be async and probably
incorrect at some point


--
Gustavo Sverzut Barbieri
--
Jabber: [EMAIL PROTECTED]
   MSN: [EMAIL PROTECTED]
  ICQ#: 17249123
 Skype: gsbarbieri
Mobile: +55 (81) 9927 0010
 Phone:  +1 (347) 624 6296; [EMAIL PROTECTED]
   GPG: 0xB640E1A2 @ wwwkeys.pgp.net

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread tvali
On 23/03/06, Gustavo Sverzut Barbieri [EMAIL PROTECTED] wrote:
Ok... this discussion is missing my initial point that is how toprovide binary dependency and avoid many crashes we have now withalmost no effort.
Paul was not missing it ;)

Part of his message was for me, part was for you.

I have got my job now so i leave for a while :)
--Gustavo Sverzut Barbieri--Jabber: 
[EMAIL PROTECTED] MSN: [EMAIL PROTECTED]ICQ#: 17249123 Skype: gsbarbieriMobile: +55 (81) 9927 0010 Phone:+1 (347) 624 6296; 
[EMAIL PROTECTED] GPG: 0xB640E1A2 @ wwwkeys.pgp.net--gentoo-portage-dev@gentoo.org
 mailing list-- tvaliMeasuring programming progress by lines of code is like measuring aircraft building progress by weight. -Bill GatesFor every complex problem there is an answer that is clear, simple, and wrong. -H L Mencken
Theory
is when you know something, but it doesn't work. Practice is when
something works, but you don't know why. Programmers combine theory and
practice: Nothing works and they don't know why.http://www.eskimo.com/~hottub/software/programming_quotes.html
http://www.softwarequotes.com/


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-23 Thread solar
This thread keeps going and going and it's a subject thats already been 
covered... So I'll just Re you here. 
Search the archives here for RRDEPEND, LDEPEND

As soon as I can figure out a way in python to do fast lookups of libs 
it will be integrated. I can do it really really fast in c but 
I don't think everybody here is keen on installing the portage-utils.

-peace


On Wed, 2006-03-15 at 12:13 -0300, Gustavo Sverzut Barbieri wrote:
 Hello,
 
 There is any provision for binary dependency on Gentoo/Portage? The
 way it works now is quite messy with things like revdep-rebuild.
 
 I have an idea to solve this problem: after software is build, you
 check which files it links (ldd binaries libraries) and check the used
 against installed packages. If a library is not provided by  an
 installed package we could have a policy to inform user or just abort
 installation.
 
 Also, if we implement these dependencies in rpm-generator, we could
 just generate RPM packages and install it in the RPM-DB and let it
 handle these kind of things.
 
 With these in place emerge would handle the build stage (where it
 excels), but rpm would handle the binary installation and dependencies
 (where it excels).
 
 --
 Gustavo Sverzut Barbieri
 --
 Mobile: +55 (81) 9927 0010
  Phone:  +1 (347) 624 6296; [EMAIL PROTECTED]
 Jabber: [EMAIL PROTECTED]
   ICQ#: 17249123
MSN: [EMAIL PROTECTED]
GPG: 0xB640E1A2 @ wwwkeys.pgp.net
 
-- 
solar [EMAIL PROTECTED]
Gentoo Linux

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-22 Thread Gustavo Sverzut Barbieri
On 3/20/06, tvali [EMAIL PROTECTED] wrote:
 2006/3/20, Gustavo Sverzut Barbieri [EMAIL PROTECTED]:
 
  I do think you're overcomplicating things where you shouldn't.
 
  Declaring stuff manually will always break, and to ensure a safe
  system, it's better to use compiler information.

 In all cases, dependancy should be based on interfaces, not code.

 All packages may:
 * Provide an interface
 * Use an interface

 Depending on useflags, OS and other compile options, it differs, which
 interfaces are provided and used.

 This is, abstractly, what portage does with interfaces.

 If portage uses some interface, it may need it's header files when
 building. It may also need another lib for static build. This means
 that binary check is not possible in all cases.

 Now, the problem is:
 * How to get an information about a package, which specifies exactly,
 which interface is needed. How to get it before building in case when
 this interface is needed to be emerged before compilation [before
 linking everything together, at least]. Which is a form of this
 information and what could be read out from that?
 * How to get information about which interfaces are provided by which
 packages *not yet emerged* -- by their current use flags(?). This
 means that it must be possible to know, which interfaces are provided
 by packages, without first building it -- and the form given by binary
 check must be the same as the form of descriptor used by this package
 check.

 So, how to get correct provider together with correct client?

Ok, I agree with you that this would be the perfect solution, but it
would demand too much effort to have this right.

I'm not proposing the final-perfect solution, just something better
than we have now. It would not cover every case, but would cover most
cases in a satisfactory way.


--
Gustavo Sverzut Barbieri
--
Jabber: [EMAIL PROTECTED]
   MSN: [EMAIL PROTECTED]
  ICQ#: 17249123
 Skype: gsbarbieri
Mobile: +55 (81) 9927 0010
 Phone:  +1 (347) 624 6296; [EMAIL PROTECTED]
   GPG: 0xB640E1A2 @ wwwkeys.pgp.net

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-22 Thread tvali
I was thinking about it, too, and found something i do like maybe more.
It would be not binary, but code dependancy. This is limited to
specific languages, then, but after all, there may also be different
binary dependencies, too [for example, you may depend on fonts 
images from another package].

Ok, my thought is like that. Lets imagine a simple c file:

#ifdef usepackagex
#include packagex.h
#endif

As it is simple to see, this file could be used to calculate non-binary
dependency, inluding useflags. And there is a plus -- tool, which just
reads those #ifdef's and #includes, could easily understand, which
flags make up which dependancy, without building package again and
again with different useflags. PS. i dont mean X by packagex, but
rather a random pack. I write here as if #define always includes
useflag name - i may be wrong, but it doesnt make a huge difference.
Sure, there are #ifdef's in code, which should not read, so we must
have an array for legal useflags - but this is also not important in my
explanation.

Lets add some custom comment syntax (line numbers are given for explanation):
1. #ifdef usepackagex
2. /* Portage: packagex.h = abc-def/packagex = 1.0.2 */
3. /* Portage: use fonts/customfontpackage */
4. #include packagex.h
5. #endif

Now, what a parser does:

Start parse: setup empty array UseFlags = []

Line 1: #ifdef usepackagex
Add useflag name into useflags array:
UseFlags = ['usepackagex']
Line 2: /* Portage: packagex.h = abc-def/packagex = 1.0.2 */
This line maybe outside of any #ifdef's as well, but in our small application there is no external conf.c file ;)
Dictionary 'Links' will contain a notice that whenever packagex is
included, it must be at least version 1.0.2 of package abc-def/packagex.
Line 3: /* Portage: use fonts/customfontpackage */
Dependancy on fonts/customfontpackage, therefore add to dependancies:
[['usepackagex'], 'fonts/customfontpackage']
Line 4: #include packagex.h
Dependancy on fonts/customfontpackage, therefore add to dependancies:

[['usepackagex'], 'abc-def/packagex = 1.0.2']
Line 5: #endif
Remove 'usepackagex' from useflags:
UseFlags = []
2006/3/22, Gustavo Sverzut Barbieri [EMAIL PROTECTED]:
On 3/20/06, tvali [EMAIL PROTECTED] wrote: 2006/3/20, Gustavo Sverzut Barbieri [EMAIL PROTECTED]:   I do think you're overcomplicating things where you shouldn't.
   Declaring stuff manually will always break, and to ensure a safe  system, it's better to use compiler information. In all cases, dependancy should be based on interfaces, not code.
 All packages may: * Provide an interface * Use an interface Depending on useflags, OS and other compile options, it differs, which interfaces are provided and used.
 This is, abstractly, what portage does with interfaces. If portage uses some interface, it may need it's header files when building. It may also need another lib for static build. This means
 that binary check is not possible in all cases. Now, the problem is: * How to get an information about a package, which specifies exactly, which interface is needed. How to get it before building in case when
 this interface is needed to be emerged before compilation [before linking everything together, at least]. Which is a form of this information and what could be read out from that? * How to get information about which interfaces are provided by which
 packages *not yet emerged* -- by their current use flags(?). This means that it must be possible to know, which interfaces are provided by packages, without first building it -- and the form given by binary
 check must be the same as the form of descriptor used by this package check. So, how to get correct provider together with correct client?Ok, I agree with you that this would be the perfect solution, but it
would demand too much effort to have this right.I'm not proposing the final-perfect solution, just something betterthan we have now. It would not cover every case, but would cover mostcases in a satisfactory way.
--Gustavo Sverzut Barbieri--Jabber: [EMAIL PROTECTED] MSN: [EMAIL PROTECTED]
ICQ#: 17249123 Skype: gsbarbieriMobile: +55 (81) 9927 0010 Phone:+1 (347) 624 6296; [EMAIL PROTECTED] GPG: 0xB640E1A2 @ 
wwwkeys.pgp.net--gentoo-portage-dev@gentoo.org mailing list-- tvaliFrom a programmer's point of view, the user is a peripheral that types when you issue a read request.-P. Williams
If
you think your management doesn't know what it's doing or that your
organisation turns out low-quality software crap that embarrasses you,
then leave.-Ed YourdonWe all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. -Larry Wall[
http://www.softwarequotes.com/ ] -
http://www.softwarequotes.com/ShowQuotes.asp?ID=544Name=Borenstein,_Nathaniel_S.
-
http://www.softwarequotes.com/ShowQuotes.asp?ID=571Name=Boehm,_Barry


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-22 Thread tvali
I hope you did read my previous mail of code dependancy -- if not, it's
at the end of message. This here is not so much overthought thing, but
a good starting point, maybe, if code deps are used.

Anyway, i give here the same idea in more complete form and good syntax
(good, because it could be added to compilers, for example):

/* As compiler may give warnings or errors otherwise, portage code parses defines portage-providers. */
#ifdef runasdaemon

#ifdef portage-providers
#provide interface customdaemon
/* if provided *as*, then does not actually provide anything before include clause */
#use interface daemonheaders 1.0.2 as dhead.h
/* when test.h is included, depends on dev-apps/test at least version 1.0.2 */
#provider-atom test set package=dev-apps/test
#provider-atom test set version=1.0.2
/* require testflag to be set */
#provider-atom test need useflag testflag Hello world!
/* require testflag2 to be unset */
#provider-atom test hate useflag testflag2 Testflag2 bad :(
/* have some good taste in warnings */
#provider-atom test warn useflag testflag3 Test warning!
#provider-atom test ask useflag testflag4 Testflag4 good :)
#use atom test as test.h
#provider-atom otheratom set package=dev-apps/otherpackage
/* if runasdaemon, otheratom will be required */
#use atom otheratom
#endif

#ifdef something

/* if runasdaemon and something, daemonheaders 1.0.2 is needed */
#include dhead.h
/* if runasdaemon and something, atom test is needed */
#include test.h

/* atoms could be provided as well; interface is a simple atom with few params; just test.h could be provided, also: */
#ifdef portage-providers
#provide header welcome.h
#use header someheader.h
#endif

#endif

#endif2006/3/22, tvali [EMAIL PROTECTED]:
I was thinking about it, too, and found something i do like maybe more.
It would be not binary, but code dependancy. This is limited to
specific languages, then, but after all, there may also be different
binary dependencies, too [for example, you may depend on fonts 
images from another package].

Ok, my thought is like that. Lets imagine a simple c file:

#ifdef usepackagex
#include packagex.h
#endif

As it is simple to see, this file could be used to calculate non-binary
dependency, inluding useflags. And there is a plus -- tool, which just
reads those #ifdef's and #includes, could easily understand, which
flags make up which dependancy, without building package again and
again with different useflags. PS. i dont mean X by packagex, but
rather a random pack. I write here as if #define always includes
useflag name - i may be wrong, but it doesnt make a huge difference.
Sure, there are #ifdef's in code, which should not read, so we must
have an array for legal useflags - but this is also not important in my
explanation.

Lets add some custom comment syntax (line numbers are given for explanation):
1. #ifdef usepackagex
2. /* Portage: packagex.h = abc-def/packagex = 1.0.2 */
3. /* Portage: use fonts/customfontpackage */
4. #include packagex.h
5. #endif

Now, what a parser does:

Start parse: setup empty array UseFlags = []

Line 1: #ifdef usepackagex
Add useflag name into useflags array:
UseFlags = ['usepackagex']
Line 2: /* Portage: packagex.h = abc-def/packagex = 1.0.2 */
This line maybe outside of any #ifdef's as well, but in our small application there is no external conf.c file ;)
Dictionary 'Links' will contain a notice that whenever packagex is
included, it must be at least version 1.0.2 of package abc-def/packagex.
Line 3: /* Portage: use fonts/customfontpackage */
Dependancy on fonts/customfontpackage, therefore add to dependancies:
[['usepackagex'], 'fonts/customfontpackage']
Line 4: #include packagex.h
Dependancy on fonts/customfontpackage, therefore add to dependancies:

[['usepackagex'], 'abc-def/packagex = 1.0.2']
Line 5: #endif
Remove 'usepackagex' from useflags:
UseFlags = []
-- tvaliFrom a programmer's point of view, the user is a peripheral that types when you issue a read request.-P. WilliamsIf
you think your management doesn't know what it's doing or that your
organisation turns out low-quality software crap that embarrasses you,
then leave.-Ed YourdonWe all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. -Larry Wall[
http://www.softwarequotes.com/ ] -
http://www.softwarequotes.com/ShowQuotes.asp?ID=544Name=Borenstein,_Nathaniel_S.
-
http://www.softwarequotes.com/ShowQuotes.asp?ID=571Name=Boehm,_Barry


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-22 Thread Patrick Lauer
On Wed, 2006-03-22 at 19:40 +0200, tvali wrote:
 I was thinking about it, too, and found something i do like maybe
 more. It would be not binary, but code dependancy. This is limited to
 specific languages, then, but after all, there may also be different
 binary dependencies, too [for example, you may depend on fonts 
 images from another package].
 
 Ok, my thought is like that. Lets imagine a simple c file:
 
 #ifdef usepackagex
 #include packagex.h
 #endif
 
 As it is simple to see, this file could be used to calculate
 non-binary dependency, inluding useflags. And there is a plus -- tool,
 which just reads those #ifdef's and #includes, could easily
 understand, which flags make up which dependancy, without building
 package again and again with different useflags. 

There are a few reasons why this won't work :-)

First: What if I have assembler? python? perl?
Your example is limited to the C preprocessor.

Second: You'll have to get this depend information applied by upstream
unless you want to patch every file ... and as upstream I'd laugh at
anyone proposing that

Third: Since there is no easy way of generating this automatically
you'll have to replicate every bit of dependency information that is in
the ebuilds. That will be much work, you won't keep ebuilds synchronized
to the included dep info ...

So in short, interesting concept, but I don't see how it can work and
how it is an improvement over what we have now. Please don't reinvent
the wheel ;-)


Patrick 
-- 
Stand still, and let the rest of the universe move


signature.asc
Description: This is a digitally signed message part


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-22 Thread tvali
2006/3/22, Patrick Lauer [EMAIL PROTECTED]:
There are a few reasons why this won't work :-)First: What if I have assembler? python? perl?Your example is limited to the C preprocessor.
Yes that i did tell there that it's limited to c already :) but bin
dep, after all, is limited to lib dependencies and probably just .so
files -- which means probably not python (as it's too dynamic to
programmatically understand, where and which libs are used), not perl
(as it doesnt have binaries). So i dont see my version being much worse
than bin.

If you have an assembler, then everything depends on which one you
have. If that's internal assembler of c, everything works fine ;) If
that is some macro-assembler, then there is a bit different syntax from
c, but other code is basically the same (parsing needed information out
from assembler code is not too hard).

If that's python, then you cant anymore make it so simple -- there is
nothing like #define in python, as it's dynamic. In python, you have
several ways -- adding dependencies in classical way; adding a *.c
file, which talks about dependancies or making some python script,
which is able to tell on what it depends -- the last way has the same
weakness with bin dep that it should be downloaded before.

About perl, i dont know enough to tell, how it could be used there, but
anyway, as it is scripting language, i think that smth similar to py
applies.

I think that there could be simple dependency builder for packages,
which will be compiled, but not for dynamic languages. There is no
simple way in dynamic languages for knowing, which libs will be used,
even if you know the useflags or have them built into pyc files. As
code deps are only my thought about repairing the biggest bottleneck in
bin-dep - that you cant say anything about deps of a random combination
of flags without rebuilding with excactly that combination -, i am not
trying to make it usable in dynamic/scripting languages.
Second: You'll have to get this depend information applied by upstreamunless you want to patch every file ... and as upstream I'd laugh at
anyone proposing that
This is actually simple build. I just told about method, which would
allow you to include dependancy metadata directly in c code. It could
be compiled into current portage dependancy data as well.
Third: Since there is no easy way of generating this automaticallyyou'll have to replicate every bit of dependency information that is in
the ebuilds. That will be much work, you won't keep ebuilds synchronizedto the included dep info ...
Data in ebuilds should be generated from that data.
So in short, interesting concept, but I don't see how it can work andhow it is an improvement over what we have now. Please don't reinvent
the wheel ;-)
Yes, if it comes to wheels, i better try to invent a better wheel, not reinvent an old wheel ;)

PS. in my last example, i dont show how useflags are related to
#defines. That's because i guess that it must be already described
somewhere and have not to be in c code :) I hope it is so ;)
Patrick--Stand still, and let the rest of the universe move-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2.2-ecc0.1.6 (GNU/Linux)iD8DBQBEIZTiqER3hOUoZM4RAkEtAJ913WnthMPjRZrUGQCgYnOw9Xcu7ACeO1DhVKWKYb6XcDuugBJSSBFMgc0==xD2e-END PGP SIGNATURE-
-- tvaliFrom a programmer's point of view, the user is a peripheral that types when you issue a read request.-P. WilliamsIf
you think your management doesn't know what it's doing or that your
organisation turns out low-quality software crap that embarrasses you,
then leave.-Ed YourdonWe all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. -Larry Wall[
http://www.softwarequotes.com/ ] -
http://www.softwarequotes.com/ShowQuotes.asp?ID=544Name=Borenstein,_Nathaniel_S.
-
http://www.softwarequotes.com/ShowQuotes.asp?ID=571Name=Boehm,_Barry


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-22 Thread tvali
As an addition to code deps discussion.

I didnt understand exactly, why bin deps were supposed to be better
than what we have now, as i am not yet exactly sure what we have :)

Anyway, i see one basic plus of code deps. It's that you may have huge
number of codelines, all containing #defines and #ifdefs. You may know
that whenever it uses ?.h header, it needs library ? -- but
when you need ?.h may be somewhat complex case. There may be,
for example, 40 different .h files of your own, which include ?.h file
-- and each of them may be included only if corresponding useflag is
set. In such case, it's easier to describe, which package you need when
?.h is used than to write all those if's twise in code and some
other place, which make sure if you need that ? pack or not.

I havent done such thing in reality, so i dont know, how big problem it
is for a programmer (how much you have that situation i described in
real life), but i guess that this is the problem what binary deps were
supposed to solve.
-- tvaliFrom a programmer's point of view, the user is a peripheral that types when you issue a read request.-P. WilliamsIf
you think your management doesn't know what it's doing or that your
organisation turns out low-quality software crap that embarrasses you,
then leave.-Ed YourdonWe all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. -Larry Wall[
http://www.softwarequotes.com/ ] -
http://www.softwarequotes.com/ShowQuotes.asp?ID=544Name=Borenstein,_Nathaniel_S.
-
http://www.softwarequotes.com/ShowQuotes.asp?ID=571Name=Boehm,_Barry


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-20 Thread Gustavo Sverzut Barbieri
I'm replying to this one, but I've read the whole thread...


On 3/16/06, Paul de Vrieze [EMAIL PROTECTED] wrote:
 On Wednesday 15 March 2006 16:13, Gustavo Sverzut Barbieri wrote:
  Hello,
 
  There is any provision for binary dependency on Gentoo/Portage? The
  way it works now is quite messy with things like revdep-rebuild.
 
  I have an idea to solve this problem: after software is build, you
  check which files it links (ldd binaries libraries) and check the used
  against installed packages. If a library is not provided by  an
  installed package we could have a policy to inform user or just abort
  installation.
 
  Also, if we implement these dependencies in rpm-generator, we could
  just generate RPM packages and install it in the RPM-DB and let it
  handle these kind of things.
 
  With these in place emerge would handle the build stage (where it
  excels), but rpm would handle the binary installation and dependencies
  (where it excels).

 Solving this is not trivial. Basically suppose application X depends on
 sys-libs/db-4.*

 This can be resolved by differently slotted libraries. We need to record
 which one was actually used (not easy by itself, but more an issue of the
 ebuild itself, if more than 1 candidate is available). But suppose that
 we know that the application was compiled to use db-4.3.29. We must then
 know that it is ok to replace the db-4.3.29 package with 4.3.30, but that
 it isn't ok to replace it by 4.3.28 or 4.4.20.

 To make this things worse, the above example assumes that within a slot,
 the libraries are binary compatible. There are examples of libraries that
 are not. And what about a library whose interface is dependent on a third
 library: B uses A, C uses B, but B exports A. So B is dependent on A, and
 the binary package of C must record that B was compiled with A.

 In short, welcome to binary package hell. This is the reason that binary
 distributions must use versions. Even debian. It is just very very hard
 to fix these kinds of indirect dependencies.

I do think you're overcomplicating things where you shouldn't.

Declaring stuff manually will always break, and to ensure a safe
system, it's better to use compiler information.

So I wouldn't mind fixing it to one package instead of a slot.

I mean, if user compiled software X-1.0 and it depends on library
Y-2.0, provided at the moment of X-1.0 compilation by package Z-3.0,
then make X-1.0 depend on Z-3.0.

If you were to remove Z-3.0 or upgrade it or even add other option by
means of USE, have X-1.0 to be recompiled too... you could be even
more correct to make it check CFLAGS too.

Of course this correctness could piss users, then you could have a
--nodeps or something like that to avoid this and use the old
behaviour.

--
Gustavo Sverzut Barbieri
--
Jabber: [EMAIL PROTECTED]
   MSN: [EMAIL PROTECTED]
  ICQ#: 17249123
 Skype: gsbarbieri
Mobile: +55 (81) 9927 0010
 Phone:  +1 (347) 624 6296; [EMAIL PROTECTED]
   GPG: 0xB640E1A2 @ wwwkeys.pgp.net

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Paul de Vrieze
On Wednesday 15 March 2006 16:13, Gustavo Sverzut Barbieri wrote:
 Hello,

 There is any provision for binary dependency on Gentoo/Portage? The
 way it works now is quite messy with things like revdep-rebuild.

 I have an idea to solve this problem: after software is build, you
 check which files it links (ldd binaries libraries) and check the used
 against installed packages. If a library is not provided by  an
 installed package we could have a policy to inform user or just abort
 installation.

 Also, if we implement these dependencies in rpm-generator, we could
 just generate RPM packages and install it in the RPM-DB and let it
 handle these kind of things.

 With these in place emerge would handle the build stage (where it
 excels), but rpm would handle the binary installation and dependencies
 (where it excels).

Solving this is not trivial. Basically suppose application X depends on 
sys-libs/db-4.*

This can be resolved by differently slotted libraries. We need to record 
which one was actually used (not easy by itself, but more an issue of the 
ebuild itself, if more than 1 candidate is available). But suppose that 
we know that the application was compiled to use db-4.3.29. We must then 
know that it is ok to replace the db-4.3.29 package with 4.3.30, but that 
it isn't ok to replace it by 4.3.28 or 4.4.20.

To make this things worse, the above example assumes that within a slot, 
the libraries are binary compatible. There are examples of libraries that 
are not. And what about a library whose interface is dependent on a third 
library: B uses A, C uses B, but B exports A. So B is dependent on A, and 
the binary package of C must record that B was compiled with A.

In short, welcome to binary package hell. This is the reason that binary 
distributions must use versions. Even debian. It is just very very hard 
to fix these kinds of indirect dependencies.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpqRcIBYHhbT.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread tvali
 To make this things worse, the above example assumes that within a slot,
 the libraries are binary compatible. There are examples of libraries that
 are not. And what about a library whose interface is dependent on a third
 library: B uses A, C uses B, but B exports A. So B is dependent on A, and
 the binary package of C must record that B was compiled with A.

 In short, welcome to binary package hell. This is the reason that binary
 distributions must use versions. Even debian. It is just very very hard
 to fix these kinds of indirect dependencies.

I think that there are many reasons, why such kind of important thing
should not be automatically generated. Starting from the fact that one
thing is dependency in binary, another thing is knowledge if this
combination actually works, has no bugs, should not be masked.

Anyway, if such binary dependancyes should ever be used, then it would
be best to:
1. register version number of interfaces
2. register interface groups

I mean:
If some lib supports commands turnon() and turnoff(), but one version
supports additionally command int checkifturnedon() and another
supports char checkifturnedon(), then there should be:
* Interface group, which states only that there are turnon and turnoff
functions -- as many apps use only most standard and minimalistic set
of commands from one lib, there is no need to know if some advanced
features are 0.9.80 or 0.9.64 compatible.
* Two interface versions, 0.9.80a and 0.9.80b, for example

All those interfaces and groups could be file links --
lib_interfacegroup_interface1.a would link to libinterface1.a.

If there is 1 library with 2 sets of minimal functionality, then it's
questionable, how to implement both (there would be linker error in
some cases) -- which makes this conception weaker.

Anyway, when standard interfaces are used and linked, that would help a bit.

Anyway -- why should i check dependencies *after* building of a big
pack, maybe hours of building, when i can check it without even
downloading it, when there is nice portage tree? And, i think that
making it binary would allow too much bad style -- there are imho
things, which should not be automated without very-very careful
thinking even if one can only win with automating them in theory and
portage tree is most definitely one of them.

 Paul

 --
 Paul de Vrieze
 Gentoo Developer
 Mail: [EMAIL PROTECTED]
 Homepage: http://www.devrieze.net





--
tvali
(e-mail: [EMAIL PROTECTED]; msn: [EMAIL PROTECTED];
icq: 317-492-912)

I like net more than life, cause if i do something wrong, then people
in net will tell me that i do, so that i can fix it -- people in life
will tell others that i do.

-- 
gentoo-portage-dev@gentoo.org mailing list



Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Brian Harring
On Thu, Mar 16, 2006 at 02:58:00PM +0100, Paul de Vrieze wrote:
 On Wednesday 15 March 2006 16:13, Gustavo Sverzut Barbieri wrote:
  Hello,
 
  There is any provision for binary dependency on Gentoo/Portage? The
  way it works now is quite messy with things like revdep-rebuild.

 Solving this is not trivial. Basically suppose application X depends on 
 sys-libs/db-4.*
 
 This can be resolved by differently slotted libraries. We need to record 
 which one was actually used (not easy by itself, but more an issue of the 
 ebuild itself, if more than 1 candidate is available). But suppose that 
 we know that the application was compiled to use db-4.3.29. We must then 
 know that it is ok to replace the db-4.3.29 package with 4.3.30, but that 
 it isn't ok to replace it by 4.3.28 or 4.4.20.
 
 To make this things worse, the above example assumes that within a slot, 
 the libraries are binary compatible. There are examples of libraries that 
 are not. And what about a library whose interface is dependent on a third 
 library: B uses A, C uses B, but B exports A. So B is dependent on A, and 
 the binary package of C must record that B was compiled with A.
 
 In short, welcome to binary package hell. This is the reason that binary 
 distributions must use versions. Even debian. It is just very very hard 
 to fix these kinds of indirect dependencies.

You're forgetting we're equally screwed when upstream (*cough* 
openssl) goes and changes the soname while preserving abi.

Old solution to it was to add another metadata key to ebuilds, 
bincompat that is compared within slotting to see if a rebuild of 
rdepend consumers needs to occur.

fex...
alsa-lib-0.90
slot=0
bincompat=0

alsa-lib-1.0
slot=0
bincompat=1

The pkg manager does it's little dance to figure out what should be 
installed where (and what versions can coexist), while doing it if 
it's going to replace a pkg and bincompat differs, pull in the subset 
of the state graph at that point (including vdb) that rdeps on that 
package/slot and force a rebuild of them.

As per the norm, requires a smart resolver; for c++ would expect 
cycles to occur where the only solution is to pull in libstdc++ (fex) 
to sidestep horkage while doing the rebuilds...

Also note that (like all general solutions) it's not going to be able 
to cover all insane stuff; if the api breaks in transitioning 
bincompat is going to cause a minor explossion that is only resolvable 
by packages specifying exact dependencies (read: version ranges).
~harring


pgpFDZuONXCnW.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Paul de Vrieze
On Thursday 16 March 2006 15:18, tvali wrote:
  To make this things worse, the above example assumes that within a
  slot, the libraries are binary compatible. There are examples of
  libraries that are not. And what about a library whose interface is
  dependent on a third library: B uses A, C uses B, but B exports A. So
  B is dependent on A, and the binary package of C must record that B
  was compiled with A.
 
  In short, welcome to binary package hell. This is the reason that
  binary distributions must use versions. Even debian. It is just very
  very hard to fix these kinds of indirect dependencies.

 I think that there are many reasons, why such kind of important thing
 should not be automatically generated. Starting from the fact that one
 thing is dependency in binary, another thing is knowledge if this
 combination actually works, has no bugs, should not be masked.

 Anyway, if such binary dependancyes should ever be used, then it would
 be best to:
 1. register version number of interfaces
 2. register interface groups

 I mean:
 If some lib supports commands turnon() and turnoff(), but one version
 supports additionally command int checkifturnedon() and another
 supports char checkifturnedon(), then there should be:
 * Interface group, which states only that there are turnon and turnoff
 functions -- as many apps use only most standard and minimalistic set
 of commands from one lib, there is no need to know if some advanced
 features are 0.9.80 or 0.9.64 compatible.
 * Two interface versions, 0.9.80a and 0.9.80b, for example

While theoretically one can't even know that the semantics are correct, we 
know from experience that hell will not break through. Downgrading 
libraries is always problematic though. It's not desirable enough to try 
to support it.


 All those interfaces and groups could be file links --
 lib_interfacegroup_interface1.a would link to libinterface1.a.

Looking at a sub-package level is going to make things way too complex. I 
don't think that doing so is going to help gentoo/portage in any way.

 If there is 1 library with 2 sets of minimal functionality, then it's
 questionable, how to implement both (there would be linker error in
 some cases) -- which makes this conception weaker.

 Anyway, when standard interfaces are used and linked, that would help a
 bit.

IMHO we need to know from every binary or installed package which 
package version and slot was used to satisfy it. We also need to know 
which of those dependencies must be exported to packages which use this 
one.

All this must also take useflags into account ;-) of course.


 Anyway -- why should i check dependencies *after* building of a big
 pack, maybe hours of building, when i can check it without even
 downloading it, when there is nice portage tree? And, i think that
 making it binary would allow too much bad style -- there are imho
 things, which should not be automated without very-very careful
 thinking even if one can only win with automating them in theory and
 portage tree is most definitely one of them.

The difference between binary packages and tracking what is installed with 
which relations are very small. Basically once compiled, the candidates 
that may resolve a dependency are fewer. While a BINSLOT variable could 
be an option (Although I'm not sure whether this would be needed), we 
still need to know the minimal version of the package within the slot.

It's not about checking dependencies after building. It's about seeing 
whether one should rebuild package B because it depends on package A that 
has been rebuilt. Perhaps package A is slotted, so I don't need to 
rebuilt. But then when I want to remove the old version, portage should 
realise that while the source dependencies are still met, the binary 
dependencies have not been met, so a rebuild is needed.

Revdep-rebuild is not going to solve this. It is a hack because it does 
not work on portage/package level. It tries to understand the elf 
(executable) format and as a result it tries to find libraries and 
executables (can't find all though, if they are installed at the wrong 
place) that depend on a particular library. Then it tries to find the 
package that the dependant library/executable comes from. As however 
libraries may have unresolved symbols, some libraries (e.g. libapr-util) 
do not actually state their dependencies.

Further revdep-rebuild does not work with downgrades, or with scripts or 
other things that are not in the elf format. Finally revdep-rebuild needs 
to scour the filesystem too much.

Recording binary dependencies can be used to resolve this. It is however 
not easy to do.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgp9RhC4YUuuN.pgp
Description: PGP signature


Re: [gentoo-portage-dev] DB and binary dependency

2006-03-16 Thread Paul de Vrieze
On Thursday 16 March 2006 15:24, Brian Harring wrote:
I would have called bincompat BINSLOT, but the idea is the same.

 As per the norm, requires a smart resolver; for c++ would expect
 cycles to occur where the only solution is to pull in libstdc++ (fex)
 to sidestep horkage while doing the rebuilds...

Don't start about c++. Suppose you have a library A that is build against 
libstdc++ 5, It contains a class B with a toString method that returns an 
std::string instance. Next we have application C that extends class B in 
class D. It (also) redefines the toString method. This application is 
binary and we don't have (or don't want to use) the source. It was 
however linked against libstdc++ 6.

Question:
Which std::string should class D return in it's toString method? The one 
from libstdc++5 or from libstdc++6 ?

(And for a moment disregard the fact that actually the string classes are 
binary compatible so it doesn't matter).


-- answer follows after some lines ---













Answer:
There is no right answer. We have hit a situation whose solution is 
undefined. In reality this is solved by the fact that the dynamic 
linker does not look at which library a particular symbol (read name) is 
from.

With linker symbol versions, what happens is that the types are silently 
cast. This however breaks runtime type information. We can namely now get 
the situation where two std::string instances do not actually have the 
same runtime type. This breaks casting. Fun isn't it.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpTksQAd6Qop.pgp
Description: PGP signature