Re: [gentoo-dev] SDLMame maintainer with Gnome setup wanted

2008-07-26 Thread Christian Birchinger
On Fri, Jul 25, 2008 at 11:07:10PM +0200, Luca Barbato wrote:
 Christian Birchinger wrote:
 Hello
 Anyone interested in maintaining further SDLMame updates?
 Beginning with 0.126 it requires GConf to get a font setting
 for it's now mandatory debugger.
 I use a plain XFCE setup and don't really want to install
 stuff like Orbit and GConf etc.
 My patch to have GConf optional was rejected by upstream:
 https://bugs.gentoo.org/show_bug.cgi?id=232233
 http://www.bannister.org/forums/ubbthreads.php?ubb=showflatNumber=42698page=3

 seemed that it would be acceptable from what I read, _maybe_ I'd check if 
 would be possible use fontconfig directly.


Well, for me personaly a hardcoded Monospace 10 fallback is
perfecly fine, because i have a proper fontconfig setup with an
acceptable aliases list and also proper DPI settings on all my
machines. This lets me use exactly the same font config (in
fact the same desktop setup) on all my machines, regardless
of the display/monitor size.

But no matter how wrong i think it is, i usualy respect the
upstreams wishes.

Christian



Re: [gentoo-dev] SDLMame maintainer with Gnome setup wanted

2008-07-26 Thread Luca Barbato

Christian Birchinger wrote:

But no matter how wrong i think it is, i usualy respect the
upstreams wishes.


If upstream is wrong I think we should help them...

lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero




Re: [gentoo-dev] SDLMame maintainer with Gnome setup wanted

2008-07-26 Thread Olivier Galibert
On Sat, Jul 26, 2008 at 03:13:36PM +0200, Luca Barbato wrote:
 Christian Birchinger wrote:
 But no matter how wrong i think it is, i usualy respect the
 upstreams wishes.
 
 If upstream is wrong I think we should help them...

Upstream thinks it's a bad idea not to give the user any possibility
of changing the font.  Upstream does not want to handle questions
about that if the gentoo ebuild is crippled by default in that aspect.

  OG.



[gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Ryan Hill
On Thu, 24 Jul 2008 18:36:28 +0200
Arfrever Frehtes Taifersar Arahesis [EMAIL PROTECTED] wrote:

 I would like to suggest new policy stating that packages should
 respect LDFLAGS. Small amount of packages which ignore LDFLAGS should
 be patched to respect them. Such patches are usually small and easy
 to write.
 
 This policy is required to allow QA team to fix packages which ignore
 LDFLAGS.

Um, this already is the policy.  We've always fixed bug reports about
LDFLAGS being ignored.


-- 
gcc-porting,  by design, by neglect
treecleaner,  for a fact or just for effect
wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Arfrever Frehtes Taifersar Arahesis
2008-07-26 18:06:12 Ryan Hill napisał(a):
 On Thu, 24 Jul 2008 18:36:28 +0200
 Arfrever Frehtes Taifersar Arahesis [EMAIL PROTECTED] wrote:
 
  I would like to suggest new policy stating that packages should
  respect LDFLAGS. Small amount of packages which ignore LDFLAGS should
  be patched to respect them. Such patches are usually small and easy
  to write.
  
  This policy is required to allow QA team to fix packages which ignore
  LDFLAGS.
 
 Um, this already is the policy.  We've always fixed bug reports about
 LDFLAGS being ignored.

Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that this policy
doesn't exist. I understand that bug reports about LDFLAGS being ignored are
usually fixed, so I ask for the formal enacting of this policy.

-- 
Arfrever Frehtes Taifersar Arahesis


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Arfrever Frehtes Taifersar Arahesis
2008-07-26 02:45:57 Ciaran McCreesh napisał(a):
 On Sat, 26 Jul 2008 00:15:03 + (UTC)
 Duncan [EMAIL PROTECTED] wrote:
  In particular, --as-needed makes a HUGE very practical difference.
  It may or may not be the wrong answer to the problem in theory, but
  lacking anything even close to as workable right now, that alone is
  IMO reason enough to work to get LDFLAGS honored.  I appreciate the
  difference it made here every time I run revdep-rebuild!
 
 Ignoring CFLAGS on some archs results in code that is either an order
 of magnitude slower or just plain won't run. Ignoring LDFLAGS means on
 those rare occasions when libraries aren't slotted properly you have to
 rebuild a few more things.
 
 Rather a large difference in impact there...

Respecting LDFLAGS provides also some some degree of optimization.
Potential benefits of LDFLAGS are sufficient to fix packages which ignore
LDFLAGS. The difference in impact is irrelevant, because even bugs without
any impact can be filed and should be fixed.

-- 
Arfrever Frehtes Taifersar Arahesis


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Ciaran McCreesh
On Sat, 26 Jul 2008 18:54:20 +0200
Arfrever Frehtes Taifersar Arahesis [EMAIL PROTECTED] wrote:
 Respecting LDFLAGS provides also some some degree of optimization.

It's a *very* small degree, and certainly nowhere near on the scale of
the difference made by CFLAGS on some archs.

If CFLAGS only made the kind of difference that it made on x86 on other
archs, packages ignoring CFLAGS wouldn't be considered a big deal. It's
only because ignoring CFLAGS results in either certain types of
performance-critical code being *a factor of ten* slower or binaries
that won't run that it matters.

(As to why you get that factor of ten: gcc on sparc builds v7 code
unless you tell it not to. v7 has no hardware integer multiply or
divide, and doing it manually takes something like twenty clock cycles.
v9 and later, which is what nearly everyone uses, has hardware integer
multiply and divide, and can have several scheduled at the same time.
For crypto, this matters a hell of a lot.)

(And the breakage? Some archs need to be told to use IEEE floating
point via a -m CFLAG, or they'll generate code using the wrong ABI.)

 Potential benefits of LDFLAGS are sufficient to fix packages which
 ignore LDFLAGS. The difference in impact is irrelevant, because even
 bugs without any impact can be filed and should be fixed.

Of all the things people could be doing, making packages honour LDFLAGS
is an extremely minor issue at best. Whilst there's nothing wrong with
making the changes, it's not exactly the most productive use of limited
resources...

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Donnie Berkholz
On 18:37 Sat 26 Jul , Arfrever Frehtes Taifersar Arahesis wrote:
 Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that this policy
 doesn't exist. I understand that bug reports about LDFLAGS being ignored are
 usually fixed, so I ask for the formal enacting of this policy.

Why are you asking us? He's the QA lead, you should be talking with the 
QA team about this.

-- 
Thanks,
Donnie

Donnie Berkholz
Developer, Gentoo Linux
Blog: http://dberkholz.wordpress.com


pgp4SH3OZY8nf.pgp
Description: PGP signature


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Arfrever Frehtes Taifersar Arahesis
2008-07-26 21:35:08 Donnie Berkholz napisał(a):
 On 18:37 Sat 26 Jul , Arfrever Frehtes Taifersar Arahesis wrote:
  Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that this 
  policy
  doesn't exist. I understand that bug reports about LDFLAGS being ignored are
  usually fixed, so I ask for the formal enacting of this policy.
 
 Why are you asking us? He's the QA lead, you should be talking with the 
 QA team about this.

Halcy0n said that this policy required prior discussion on this mailing list.

-- 
Arfrever Frehtes Taifersar Arahesis


signature.asc
Description: This is a digitally signed message part.


[gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Ryan Hill
On Sat, 26 Jul 2008 18:37:06 +0200
Arfrever Frehtes Taifersar Arahesis [EMAIL PROTECTED] wrote:

 Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that
 this policy doesn't exist. I understand that bug reports about
 LDFLAGS being ignored are usually fixed, so I ask for the formal
 enacting of this policy.

Fair enough then. ;)


-- 
gcc-porting,  by design, by neglect
treecleaner,  for a fact or just for effect
wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662


signature.asc
Description: PGP signature


[gentoo-dev] split Qt 4.4 dependencies

2008-07-26 Thread Carsten Lohrke
Since it is time to get Qt 4.4 into testing, here some information how to get 
the dependencies in the ebuilds you maintain, right.

Beforehand: Relying on best_version() or the broken qt4_min_version() stuff 
from qt4.eclass is not fine.


- Migrating existing ebuilds requires a dependency like 

|| ( ( splitQt1 .. splitQtN )  x11-libs/qt-4.4:4 )

For x11-libs/qt any dependency atom _not_ including version 4.4 is fine. It is 
a requirement¹ to list the split Qt ebuilds first.

- When Qt 4.4 is in testing, new ebuilds to be added to the tree may not 
include x11-libs/qt. Depend on the split ebuilds only.

- For all ebuilds which are not tested or supposed to work with split Qt 4.4 
ebuilds change the dependency to x11-libs/qt-4.4:4 or a comparable 
dependency atom, please.
 

The split Qt ebuilds are:

x11-libs/qt-assistant
x11-libs/qt-core
x11-libs/qt-dbus
x11-libs/qt-demo
x11-libs/qt-embedded
x11-libs/qt-gui
x11-libs/qt-opengl
x11-libs/qt-phonon
x11-libs/qt-qt3support
x11-libs/qt-script
x11-libs/qt-sql
x11-libs/qt-svg
x11-libs/qt-test
x11-libs/qt-webkit
x11-libs/qt-xmlpatterns


Carsten


[1] https://bugs.gentoo.org/show_bug.cgi?id=232246


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Gilles Dartiguelongue
Le samedi 26 juillet 2008 à 21:39 +0200, Arfrever Frehtes Taifersar
Arahesis a écrit :
 2008-07-26 21:35:08 Donnie Berkholz napisał(a):
  On 18:37 Sat 26 Jul , Arfrever Frehtes Taifersar Arahesis wrote:
   Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that this 
   policy
   doesn't exist. I understand that bug reports about LDFLAGS being ignored 
   are
   usually fixed, so I ask for the formal enacting of this policy.
  
  Why are you asking us? He's the QA lead, you should be talking with the 
  QA team about this.
 
 Halcy0n said that this policy required prior discussion on this mailing list.
 

like Ciaran said, fixing packages ignoring LDFLAGS isn't going to be top
priority because it is far less important than fixing packages ignoring
CFLAGS or doing pre-stripping. If making this a policy can make you
happy, I see no problem with that. It won't make bugs fix themselves
either way.

-- 
Gilles Dartiguelongue [EMAIL PROTECTED]
Gentoo


signature.asc
Description: Ceci est une partie de message	numériquement signée


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Carsten Lohrke
On Samstag, 26. Juli 2008, Donnie Berkholz wrote:
 Why are you asking us? He's the QA lead, you should be talking with the
 QA team about this.

Such issues are not up to a self chosen group, but are topic for this list.


Carsten



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Carsten Lohrke
On Samstag, 26. Juli 2008, Arfrever Frehtes Taifersar Arahesis wrote:
  Um, this already is the policy.  We've always fixed bug reports about
  LDFLAGS being ignored.

 Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that this
 policy doesn't exist. I understand that bug reports about LDFLAGS being
 ignored are usually fixed, so I ask for the formal enacting of this policy.

Afaik it has always been the way that *sane* LDFLAGS are to be respected, but 
exceptions exist of course and it's up to the maintainer to mangle or clear 
your LDFLAGS, if deemed necessary. I'd like to know, why Mark asked to bring 
this question up here. Shouldn't this be common sense!?


Carsten


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Arfrever Frehtes Taifersar Arahesis
2008-07-26 23:43:53 Gilles Dartiguelongue napisał(a):
 Le samedi 26 juillet 2008 à 21:39 +0200, Arfrever Frehtes Taifersar
 Arahesis a écrit :
  2008-07-26 21:35:08 Donnie Berkholz napisał(a):
   On 18:37 Sat 26 Jul , Arfrever Frehtes Taifersar Arahesis wrote:
Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that this 
policy
doesn't exist. I understand that bug reports about LDFLAGS being 
ignored are
usually fixed, so I ask for the formal enacting of this policy.
   
   Why are you asking us? He's the QA lead, you should be talking with the 
   QA team about this.
  
  Halcy0n said that this policy required prior discussion on this mailing 
  list.
  
 
 like Ciaran said, fixing packages ignoring LDFLAGS isn't going to be top
 priority

I never suggested that it should be top priority :).

 If making this a policy can make you happy, I see no problem with that.
 It won't make bugs fix themselves either way.

It will at least allow QA team to fix such bugs where patches are already
available.

-- 
Arfrever Frehtes Taifersar Arahesis


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Arfrever Frehtes Taifersar Arahesis
2008-07-27 00:00:55 Carsten Lohrke napisał(a):
 On Samstag, 26. Juli 2008, Arfrever Frehtes Taifersar Arahesis wrote:
   Um, this already is the policy.  We've always fixed bug reports about
   LDFLAGS being ignored.
 
  Mark Loeser (Halcy0n) (QA project leader) said on 2008-07-24 that this
  policy doesn't exist. I understand that bug reports about LDFLAGS being
  ignored are usually fixed, so I ask for the formal enacting of this policy.
 
 Afaik it has always been the way that *sane* LDFLAGS are to be respected, but 
 exceptions exist of course and it's up to the maintainer to mangle or clear 
 your LDFLAGS, if deemed necessary. I'd like to know, why Mark asked to bring 
 this question up here. Shouldn't this be common sense!?

+1.

-- 
Arfrever Frehtes Taifersar Arahesis


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Ciaran McCreesh
On Sun, 27 Jul 2008 00:00:55 +0200
Carsten Lohrke [EMAIL PROTECTED] wrote:
 Afaik it has always been the way that *sane* LDFLAGS are to be
 respected, but exceptions exist of course and it's up to the
 maintainer to mangle or clear your LDFLAGS, if deemed necessary. I'd
 like to know, why Mark asked to bring this question up here.
 Shouldn't this be common sense!?

The way it is currently: Packages ignoring CFLAGS without a *very* good
reason (and 'upstream thinks they know better' is rarely a very good
reason, especially when upstream supposedly knowing better leads to v7
builds on v9 systems) need to be fixed. Packages ignoring LDFLAGS can
be fixed if the maintainer feels like it, but there's no requirement to
do so and filing bugs about it is frowned upon.

Until recently, LDFLAGS have been put in the anyone using these is a
ricer category. Unfortunately, the misguided promotion of 'as-needed'
despite its massive design flaws has lead people to think that setting
LDFLAGS is in some way useful or cool. I expect next someone will try
to find a way to force 'ASFLAGS' onto everyone...

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature


[gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Nikos Chantziaras

Ciaran McCreesh wrote:

On Sun, 27 Jul 2008 00:00:55 +0200
Carsten Lohrke [EMAIL PROTECTED] wrote:

Afaik it has always been the way that *sane* LDFLAGS are to be
respected, but exceptions exist of course and it's up to the
maintainer to mangle or clear your LDFLAGS, if deemed necessary. I'd
like to know, why Mark asked to bring this question up here.
Shouldn't this be common sense!?


The way it is currently: Packages ignoring CFLAGS without a *very* good
reason (and 'upstream thinks they know better' is rarely a very good
reason, especially when upstream supposedly knowing better leads to v7
builds on v9 systems) need to be fixed. Packages ignoring LDFLAGS can
be fixed if the maintainer feels like it, but there's no requirement to
do so and filing bugs about it is frowned upon.

Until recently, LDFLAGS have been put in the anyone using these is a
ricer category. Unfortunately, the misguided promotion of 'as-needed'
despite its massive design flaws has lead people to think that setting
LDFLAGS is in some way useful or cool. I expect next someone will try
to find a way to force 'ASFLAGS' onto everyone...


This is totally irrelevant though.  If I have --as-needed in my LDFLAGS 
(I do) I still consider it a bug if a package does not honor it.  So 
what I'm doing is fixing the ebuild (*if* the ebuild does not mention a 
reason of not honoring LDFLAGS of course) and submit it in bugzilla.  I 
don't know if the maintainers are getting annoying by this.  They 
shouldn't.  If some LDFLAGS turn out to break a package in some way 
doesn't mean that it's OK for the package to ignore LDFLAGS altogether. 
 If I have CFLAGS=-O99 -fsuper-mega-fast-math 
-enable-leet-broken-experimental-optimize doesn't mean the package 
should ignore CFLAGS :P


(As for --as-needed, it's the same as -O3 in CFLAGS; if a package turns 
out to break, an ebuild *could* explicitly filter out -O3, but that's 
not really a priority.  In the end, if the user chooses ricer-flags and 
breaks his system, he can blame himself.  If he explicitly wants to 
shoot himself in the foot, the ebuild should allow him to do so.  Any 
effort spent to protect the ricers from themselves is *wasted* effort 
better spent somewhere else.


Not that I have ever seen a package that breaks with --as-needed though. 
 Of course that's just me.)





Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Gilles Dartiguelongue
Le dimanche 27 juillet 2008 à 02:12 +0300, Nikos Chantziaras a écrit :
 Not that I have ever seen a package that breaks with --as-needed though. 
   Of course that's just me.)

ahah ! now I have an example for you, nemiver. It seems it does the
module loading thingy that was brought up in the relevant thread. It
breaks with --as-needed as expected.

-- 
Gilles Dartiguelongue [EMAIL PROTECTED]
Gentoo


signature.asc
Description: Ceci est une partie de message	numériquement signée


[gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Ryan Hill
On Sun, 27 Jul 2008 02:12:13 +0300
Nikos Chantziaras [EMAIL PROTECTED] wrote:

 Not that I have ever seen a package that breaks with --as-needed
 though. Of course that's just me.

Well, then, behold:
http://tinyurl.com/5jvkm9

Now you have.  Enjoy. :)


-- 
gcc-porting,  by design, by neglect
treecleaner,  for a fact or just for effect
wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662


signature.asc
Description: PGP signature


[gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Nikos Chantziaras

Ryan Hill wrote:

On Sun, 27 Jul 2008 02:12:13 +0300
Nikos Chantziaras [EMAIL PROTECTED] wrote:


Not that I have ever seen a package that breaks with --as-needed
though. Of course that's just me.


Well, then, behold:
http://tinyurl.com/5jvkm9

Now you have.  Enjoy. :)


Then I must be lucky.  I actually don't use ricer-flags (I'm on -O2 
-march=native -pipe for CFLAGS.)  --as-needed is simply just too good 
for revdep-rebuild in order not to use it.  I figured that when actually 
something does break with it (nothing yet on my system) it would show 
immediately with a build or runtime error (good) rather than running 
with silent bugs (bad) and possibly corrupting things; build error or 
immediate crash.  I hope I'm not assuming too much here :P





Re: [gentoo-dev] Re: [RFC] New policy: LDFLAGS should be respected

2008-07-26 Thread Jeremy Olexa

Arfrever Frehtes Taifersar Arahesis wrote:

It will at least allow QA team to fix such bugs where patches are already
available.


So, if bugs are being fixed why is there a need to fix something that 
isn't broken with regards to a policy _needed_ to enforce this action? 
Are bugs being ignored or RESOLVED, WONTFIX?


Hypothetically, if there was such a policy and there was 100's of bugs 
filed with patches (maintainers should handle the patches anyway) this 
isn't any different than the present lack of policy. Let us also pretend 
that there were 100's of bugs filed that had no such patches 
available..would the QA team have the manpower to generate patches to 
fix this and apply them or would the bugs just rot in bugzilla and not 
achieve anything?


-Jeremy





[gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel

2008-07-26 Thread Duncan
So I'm running the 2.2-rcs and have been seeing blogs about the new 
parallel merge capacities...  Having a dual-dual-core Opteron and having 
run multiple merges manually for some time, this is VERY welcome news. 
=8^)

So after upgrading to -rc3 I set EMERGE_DEFAULT_OPTS to include

--jobs=10 --keep-going --load-average=15

and tried a few small merges (the rest of the world update).  It worked 
nicely.

So then, as I had adjusted LDFLAGS recently and hadn't yet done a full 
world remerge, I decided to try the BIG test, emerge --emptytree --world, 
with some 674 packages.

For the first 100 or so packages, it worked quite well.  However, about 
there, maybe package 120 or so, so about 20% of the way thru, it reverted 
to doing them one-at-a-time again.  I'm now on package #279 and it's 
still doing them only one at a time, and this has included stuff like the 
xorg-docs (IIRC) package, that really shouldn't be pre-deps for stuff 
that has come since, so /something/ else should have been trying to 
merge, as long as load-average stays low, as it has much of the time (I 
have MAKEOPTS=-j -l20 so it's not going to be low all the time).

Is this a known issue still being worked on, perhaps due to the limits of 
the package dependency and scheduling resolution such that it can't 
handle a full world remerge and defaults back to one-at-a-time after it 
reaches the extent of its mapping, or is this a new bug?

Also, a little monitoring utility that could be run in another terminal 
and just list and update all the currently merging packages, and any that 
had failed to merge, so I could take a look at them while the system is 
still working on the rest, would be quite useful.  I tried watch ls -d 
$PORTAGE_TMPDIR/*/* and it starts to work, but of course starts to error 
in a few seconds when the first package completes since the *s are 
resolved initially.  With a bit of work I'm sure can get something simple 
working, but it'd be nice if there were a pre-made utility to do it.  
Maybe there is and I just don't know about it yet?  =8^)

Finally... I was rather confused the first time at just one job an 
install took a bit, as that's apparently not counted as running, so it 
appeared nothing was going on for a bit.  Maybe an installing count as 
well would be useful... and prevent that confusion.

Thanks, guys.  It's already fun playing with! =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




Re: [gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel

2008-07-26 Thread Andrew Gaffney

Duncan wrote:

--jobs=10 --keep-going --load-average=15


For a dual-dual-core setup, a load average of 4.0 is fully loaded. Anything 
higher than that and you're just causing jobs to queue up unnecessarily and your 
system to thrash.



have MAKEOPTS=-j -l20 so it's not going to be low all the time).


Same thing here. Also, why would you specify different --load-average values in 
these two places?


--
Andrew Gaffney http://dev.gentoo.org/~agaffney/
Gentoo Linux Developer Catalyst/Installer + x86 release coordinator



[gentoo-portage-dev] Re: portage-2.2-rc3 parallel merges quit being parallel

2008-07-26 Thread Duncan
Andrew Gaffney [EMAIL PROTECTED] posted [EMAIL PROTECTED],
excerpted below, on  Sat, 26 Jul 2008 16:56:20 -0500:

 Duncan wrote:
 --jobs=10 --keep-going --load-average=15
 
 For a dual-dual-core setup, a load average of 4.0 is fully loaded.
 Anything higher than that and you're just causing jobs to queue up
 unnecessarily and your system to thrash.

Not really.  The highest system load-average possible is the one-minute 
load-average, right?  From my experience there are times when it just 
sits there doing nothing.  No I/O, CPU graphs low, but load average still 
high so it won't start any more jobs.

I see gains at times up to ~4 jobs per core (tho it's arguably possible 
they're counteracted above say, 3/core, by extra shuffling, I won't argue 
that and haven't checked that closely, I just don't like to see blank 
spots in the CPU utilization that aren't accounted for by I/O).  I just 
boosted it to five so I could do the below

 have MAKEOPTS=-j -l20 so it's not going to be low all the time).
 
 Same thing here. Also, why would you specify different --load-average
 values in these two places?

The idea here is to create a differential, so it doesn't start new builds 
when a single build can adequately parallelize.  I'm building in tmpfs, 
which is (limited quantity, 8 gigs, but...) memory, and if a single 
emerge is keeping the system sufficiently busy, no reason to add a new 
one to the pile.  However, when a single merge isn't keeping the system 
busy, /then/ add more.  The differential at least in theory should favour 
single packages when they can provide sufficient parallelization. 

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




Re: [gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel

2008-07-26 Thread Zac Medico
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Duncan wrote:
 For the first 100 or so packages, it worked quite well.  However, about 
 there, maybe package 120 or so, so about 20% of the way thru, it reverted 
 to doing them one-at-a-time again.  I'm now on package #279 and it's 
 still doing them only one at a time, and this has included stuff like the 
 xorg-docs (IIRC) package, that really shouldn't be pre-deps for stuff 
 that has come since, so /something/ else should have been trying to 
 merge, as long as load-average stays low, as it has much of the time (I 
 have MAKEOPTS=-j -l20 so it's not going to be low all the time).
 
 Is this a known issue still being worked on, perhaps due to the limits of 
 the package dependency and scheduling resolution such that it can't 
 handle a full world remerge and defaults back to one-at-a-time after it 
 reaches the extent of its mapping, or is this a new bug?

The current algorithm is intentionally as conservative as possible
in the sense that it will not build given package in parallel if
there are any packages in it's subgraph of deep dependencies
scheduled to be merged. We can add one or more options to control
the criteria for choosing packages. Those options will modify the
behavior of Scheduler._choose_pkg().

There are a couple of reasons for the current conservative
behavior. In many cases the conservative behavior is beneficial
(avoids breakage) in order to ensure that a package's subgraph of
deep dependencies is up to date before building the package itself.
In addition, the conservative behavior lessens the need to be
concerned about invariance requirements like those discussed in
bug 232990 [1].

 Also, a little monitoring utility that could be run in another terminal 
 and just list and update all the currently merging packages, and any that 
 had failed to merge, so I could take a look at them while the system is 
 still working on the rest, would be quite useful.  I tried watch ls -d 
 $PORTAGE_TMPDIR/*/* and it starts to work, but of course starts to error 
 in a few seconds when the first package completes since the *s are 
 resolved initially.  With a bit of work I'm sure can get something simple 
 working, but it'd be nice if there were a pre-made utility to do it.  
 Maybe there is and I just don't know about it yet?  =8^)

I'm not aware of any tool like that yet. Such a tool should probably
 use the hidden lock file located in the parent directory of a build
directory in order to detect an active build. In the future I plan
to have a daemon process to allow cooperation between multiple
emerge invocations, for things like bug 177311 [2]. Once that's
implemented, there will be a way to query the daemon for information
about running builds.

 Finally... I was rather confused the first time at just one job an 
 install took a bit, as that's apparently not counted as running, so it 
 appeared nothing was going on for a bit.  Maybe an installing count as 
 well would be useful... and prevent that confusion.

There used to be a merges count in the status display but somebody
thought it was confusing (darkside/Jeremy Olexa) and I decided that
it wasn't interesting enough to be worthy of it's display space so I
removed it. I guess we can add it back if there's space and demand
for it. Maybe it should only be shown when the job count drops to zero?

Zac

[1] http://bugs.gentoo.org/show_bug.cgi?id=232990
[2] http://bugs.gentoo.org/show_bug.cgi?id=177311
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkiLupEACgkQ/ejvha5XGaP8KQCffKnpIiplEioyP4JOlD8HGD21
Q2QAnAroP5voKc8zyXcCFArPxrYjsec3
=3epU
-END PGP SIGNATURE-



Re: [gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel

2008-07-26 Thread Marius Mauch
On Sat, 26 Jul 2008 16:56:20 -0500
Andrew Gaffney [EMAIL PROTECTED] wrote:

 Duncan wrote:
  --jobs=10 --keep-going --load-average=15
 
 For a dual-dual-core setup, a load average of 4.0 is fully loaded.

Only in ideal cases, when you have long-running processes
hammering the cpu and little or no context switches. This isn't the case
with builds, the actual compile processes that need cpu time usually
terminate very quickly, which increases the load average.
I did some benchmarking a while ago with different combinations of -j
and -l in MAKEOPTS, using the kernel as testcase, and IIRC the fastest
builds were around -l6.0 (on a dual-core system) and high (or
unlimited) values for -j (sidenote here: some ebuilds like openoffice
parse MAKEOPTS to get parameters for their own build systems, but only
recognize/support -j and ignore -l, so one should still be careful with
the value for -j).

Marius



Re: [gentoo-portage-dev] [RFC/PATCH v2] New objects cpv, pv and version to be used instead of raw strings.

2008-07-26 Thread Zac Medico
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Ali Polatel wrote:
 Thanks for the comments.

I haven't been able to finish reviewing your patch yet but I just
want you to know that it's still in my queue.

Zac
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkiL5MwACgkQ/ejvha5XGaMSzACgyuAEQHMNFOjiubmTQhyuZzFa
EusAoM97a0BCfByh/gEU6Sj6X6kx9MCt
=vJa6
-END PGP SIGNATURE-