Re: [gentoo-user] Portage load control

2023-05-12 Thread Jack
On 5/12/23 20:08, Peter Humphrey wrote: On Saturday, 13 May 2023 00:53:49 BST Mark Knecht wrote: Anyway, I had a couple of thoughts: 1) If it's really a bug then as others have said report it up the chain and hope for a fix. https://bugs.gentoo.org/905933 2) If I wanted to solve the

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Saturday, 13 May 2023 00:53:49 BST Mark Knecht wrote: >Anyway, I had a couple of thoughts: > > 1) If it's really a bug then as others have said report it up the > chain and hope for a fix. https://bugs.gentoo.org/905933 > 2) If I wanted to solve the problem today(ish) then I'd build > a

Re: [gentoo-user] Portage load control

2023-05-12 Thread Mark Knecht
On Fri, May 12, 2023 at 10:42 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 17:58:46 BST Jack wrote: > > > Again, --load-average tells emerge whether it can start a new > > job/package, but has no control over how high the load will get based > > on the already started jobs. If emerge

Re: [gentoo-user] Portage load control

2023-05-12 Thread Mark Knecht
On Fri, May 12, 2023 at 9:59 AM Jack wrote: > > On 2023.05.12 12:23, Mark Knecht wrote: > [snip .] > >One interesting point is that the first Gentoo page I found to > > look at the emerge man page shows LOAD as the value provided > > to the --load-average option, but nowhere does it

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 17:58:46 BST Jack wrote: > Again, --load-average tells emerge whether it can start a new > job/package, but has no control over how high the load will get based > on the already started jobs. If emerge starts new jobs when the load > is over that specified by

Re: [gentoo-user] Portage load control

2023-05-12 Thread Jack
On 2023.05.12 12:23, Mark Knecht wrote: [snip .] One interesting point is that the first Gentoo page I found to look at the emerge man page shows LOAD as the value provided to the --load-average option, but nowhere does it specify anything other than it's a floating point value: I suspect

Re: [gentoo-user] Portage load control

2023-05-12 Thread Mark Knecht
On Fri, May 12, 2023 at 9:08 AM Jack wrote: > > > -j 1 > > -j1 --load-average=40 > > -j1 --load-aveeage=40.0 > > -j1 --load-average=4.0 > > -j1 --load-average=0.4 > > -j10 --load-average=0.4 > > > > etc., and see what happens? > --load-average controls whether or not emerge starts another >

Re: [gentoo-user] Portage load control

2023-05-12 Thread Jack
On 2023.05.12 11:27, Mark Knecht wrote: On Fri, May 12, 2023 at 7:27 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 15:13:08 BST Mark Knecht wrote: > > > My opinion: load-average probably works, but we are misunderstanding > > the documentation. > > That's what bothers me the most -

Re: [gentoo-user] Portage load control

2023-05-12 Thread Mark Knecht
On Fri, May 12, 2023 at 7:27 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 15:13:08 BST Mark Knecht wrote: > > > My opinion: load-average probably works, but we are misunderstanding > > the documentation. > > That's what bothers me the most - that I have a mental block somewhere. :( > > --

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 15:13:08 BST Mark Knecht wrote: > My opinion: load-average probably works, but we are misunderstanding > the documentation. That's what bothers me the most - that I have a mental block somewhere. :( -- Regards, Peter.

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 15:06:21 BST Michael Cook wrote: > You can read /usr/share/portage/config/make.conf.example for an > explanation. All children processes will use that. I can run portage and > play games on the same system with my settings. That example says nothing about any of the emerge

Re: [gentoo-user] Portage load control

2023-05-12 Thread Mark Knecht
On Fri, May 12, 2023 at 6:46 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 00:08:03 BST Mark Knecht wrote: > > On Thu, May 11, 2023 at 3:07 PM Peter Humphrey > > > > wrote: > > > On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: > > > > > > > > The ''problem' is this can easily hit

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 14:37:13 BST Jack wrote: > I still see two separate issues. First, you are saying that emerge > still launches new jobs when the load is over what is set with > --load-average. A possible way to test this directly is to run or > create some job that pushed the load

Re: [gentoo-user] Portage load control

2023-05-12 Thread Michael Cook
On 5/12/23 09:46, Peter Humphrey wrote: On Friday, 12 May 2023 00:08:03 BST Mark Knecht wrote: On Thu, May 11, 2023 at 3:07 PM Peter Humphrey wrote: On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: The ''problem' is this can easily hit 100% of the cores you have in the machine if

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 00:08:03 BST Mark Knecht wrote: > On Thu, May 11, 2023 at 3:07 PM Peter Humphrey > > wrote: > > On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: > > > > > The ''problem' is this can easily hit 100% of the cores you have in the > > > machine if not sensibly set.

Re: [gentoo-user] Portage load control

2023-05-12 Thread Jack
On 5/12/23 09:16, Peter Humphrey wrote: On Friday, 12 May 2023 11:09:37 BST Arve Barsnes wrote: On Fri, 12 May 2023 at 10:34, Peter Humphrey wrote: I have said several times that portage is ignoring that setting. I have it at 40, yet portage kicks off more packages at 72, and continues

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 11:09:37 BST Arve Barsnes wrote: > On Fri, 12 May 2023 at 10:34, Peter Humphrey wrote: > > I have said several times that portage is ignoring that setting. I have it > > at 40, yet portage kicks off more packages at 72, and continues doing so > > for extended periods - at

Re: [gentoo-user] Portage load control

2023-05-12 Thread Arve Barsnes
On Fri, 12 May 2023 at 10:34, Peter Humphrey wrote: > On Friday, 12 May 2023 01:38:52 BST Jack wrote: > > The --load-average to emerge itself just tells it not to start a new job > > if the load is above the setting. If there are several large jobs, but > > all start with single threaded

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 09:34:27 BST I wrote: > > The --load-average in MAKEOPTS gets passed to make, and controls how > > many processes make starts. If that is set, and the load is still too > > high, the problem is in make not in emerge. Also, that setting will > > have no effect if the

Re: [gentoo-user] Portage load control

2023-05-12 Thread Peter Humphrey
On Friday, 12 May 2023 01:38:52 BST Jack wrote: > Sorry if I'm repeating myself, but as I see it, there are two different > --load-average settings to consider. I'd have to go back to the > beginning of the thread to confirm you are setting both of them. I am also going to repeat myself. > The

Re: [gentoo-user] Portage load control

2023-05-11 Thread Michael Cook
On 5/11/23 23:23, Eldon wrote: On Thu, May 11, 2023 at 11:07:04PM +0100, Peter Humphrey wrote: Once again, --load-average is being ignored. Why is it there? Surely, it must be to mitigate the worst effects of that N*K, but it isn't doing so. Take all of the following with a grain of salt and

Re: [gentoo-user] Portage load control

2023-05-11 Thread Jack
On 5/11/23 18:07, Peter Humphrey wrote: On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: I'm sure you get this but I'm pointing toward the EMERGE_DEFAULT_OPTS portage variable which, according to it's page that "defines entries to be appended to the emerge command line." I suspect they

Re: [gentoo-user] Portage load control

2023-05-11 Thread Mark Knecht
On Thu, May 11, 2023 at 3:07 PM Peter Humphrey wrote: > > On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: > > The ''problem' is this can easily hit 100% of the cores you have in the > > machine if not sensibly set. (You choose what's 'sensible') > > Once again, --load-average is being

Re: [gentoo-user] Portage load control

2023-05-11 Thread Peter Humphrey
On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: > I'm sure you get this but I'm pointing toward the EMERGE_DEFAULT_OPTS > portage variable which, according to it's page that "defines entries to be > appended to the emerge command line." I suspect they are appended, but > that doesn't

Re: [gentoo-user] Portage load control

2023-05-11 Thread Mark Knecht
On Thu, May 11, 2023 at 9:03 AM Peter Humphrey wrote: > > On Thursday, 11 May 2023 15:58:20 BST Mark Knecht wrote: > > Going further, this page states: > > > > "The load average value is the same as displayed by top or uptime, and for > > an N-core system, a load average of N.0 would be a 100%

Re: [gentoo-user] Portage load control

2023-05-11 Thread Peter Humphrey
On Thursday, 11 May 2023 15:58:20 BST Mark Knecht wrote: > Going further, this page states: > > "The load average value is the same as displayed by top or uptime, and for > an N-core system, a load average of N.0 would be a 100% load. Another rule > of thumb here is to set X.Y=N*0.9 which will

Re: [gentoo-user] Portage load control

2023-05-11 Thread Peter Humphrey
On Thursday, 11 May 2023 14:45:26 BST Mark Knecht wrote: > OK, this is a bit of a weird thing for me to ask you to try but this page > on emerge: > > https://wiki.gentoo.org/wiki/EMERGE_DEFAULT_OPTS > > says pretty clearly that "--load-average X.Y" should be a floating point > number so try it

Re: [gentoo-user] Portage load control

2023-05-11 Thread Mark Knecht
Going further, this page states: "The load average value is the same as displayed by top or uptime, and for an N-core system, a load average of N.0 would be a 100% load. Another rule of thumb here is to set X.Y=N*0.9 which will limit the load to 90%, thus maintaining system responsiveness." So,

Re: [gentoo-user] Portage load control

2023-05-11 Thread Mark Knecht
On Thu, May 11, 2023 at 6:34 AM Peter Humphrey wrote: > > On Monday, 8 May 2023 11:20:45 BST Neil Bothwick wrote: > > > Maybe you should take this to bgo where it can be flagged for the portage > > devs to look at, just keep us posted on the outcome. > > So far, I've just been asked whether I

Re: [gentoo-user] Portage load control

2023-05-11 Thread Peter Humphrey
On Monday, 8 May 2023 11:20:45 BST Neil Bothwick wrote: > Maybe you should take this to bgo where it can be flagged for the portage > devs to look at, just keep us posted on the outcome. So far, I've just been asked whether I expected something different, to which I replied "Why is

Re: [gentoo-user] Portage load control

2023-05-08 Thread Peter Humphrey
On Monday, 8 May 2023 11:20:45 BST Neil Bothwick wrote: > Maybe you should take this to bgo where it can be flagged for the portage > devs to look at, just keep us posted on the outcome. https://bugs.gentoo.org/905933 -- Regards, Peter.

Re: [gentoo-user] Portage load control

2023-05-08 Thread Neil Bothwick
On Sun, 07 May 2023 17:00:16 +0100, Peter Humphrey wrote: > Everybody keeps explaining how the system is supposed to work. I know > all that, as I said last time. > > The problem is that PORTAGE IS NOT DOING WHAT IT'S SUPPOSED TO. I see the same at times. I realise that portage can only look at

Re: [gentoo-user] Portage load control

2023-05-07 Thread Walter Dnes
On Sat, May 06, 2023 at 12:50:08PM +0100, Peter Humphrey wrote > Second, the two pages contribute actively to the confusion between the emerge > jobs submitted in parallel by portage and the concurrent tasks that may be > launched by each of those. > > The test: > > I ran 'emerge -e @world'

Re: [gentoo-user] Portage load control

2023-05-07 Thread Peter Humphrey
On Sunday, 7 May 2023 15:52:08 BST Michael wrote: > As I understand it and have so far confirmed on my systems, the --jobs > directive explained on the emerge man page, places a limit of how many > different non-dependent packages will be emerged in parallel at any time, by > any single emerge

Re: [gentoo-user] Portage load control

2023-05-07 Thread Michael
On Sunday, 7 May 2023 11:27:14 BST Peter Humphrey wrote: > On Saturday, 6 May 2023 19:18:25 BST Jack wrote: > > I hope I'm not preaching to the choir, and I have NOT reread the > > various man pages, but the different options you mention (and some you > > don't) apply to different parts of the

Re: [gentoo-user] Portage load control

2023-05-07 Thread Peter Humphrey
On Saturday, 6 May 2023 19:18:25 BST Jack wrote: > Minor point - are you sure ccache isn't affecting your results? Pretty sure - it isn't installed here. :) > I hope I'm not preaching to the choir, and I have NOT reread the > various man pages, but the different options you mention (and some

Re: [gentoo-user] Portage load control

2023-05-06 Thread Jack
On 2023.05.06 07:50, Peter Humphrey wrote: Hello list, I still don't know how this works. I ran a test over the last two days, and the result does not accord with 'man make.conf' nor 'man 1 make'. First, 'man make.conf' does not state that --load-average, if set, will override --jobs,

[gentoo-user] Portage load control

2023-05-06 Thread Peter Humphrey
Hello list, I still don't know how this works. I ran a test over the last two days, and the result does not accord with 'man make.conf' nor 'man 1 make'. First, 'man make.conf' does not state that --load-average, if set, will override --jobs, as it clearly does. Second, the two pages