Re: [gentoo-user] Portage load control
On 5/12/23 20:08, Peter Humphrey wrote: On Saturday, 13 May 2023 00:53:49 BST Mark Knecht wrote: Anyway, I had a couple of thoughts: 1) If it's really a bug then as others have said report it up the chain and hope for a fix. https://bugs.gentoo.org/905933 2) If I wanted to solve the problem today(ish) then I'd build a Gentoo VM in Virtualbox, dedicate some number of cores to it, build everything with binary packages and probably run an NFS server in the VM which I mount in the host machine. I then update the host machine from the binary packages and Virtualbox manages to never use more cores than I give it. That fix is more or less guaranteed to work. Sounds like a lot of work. :( A new thought on an easier test. With -j any higher than 1, doesn't emerge put out a fairly constant stream of how many out of how many jobs are complete, how many are currently running, and the load average? If it launches new jobs when it's own display of load average is above what you set, that should be pretty compelling to the developers. 3) As a question for the far more knowledgeable system folks I'd ask "Can this problem be solved by cgroups?" If I have a cgroup with 10 processors in it, can I start emerge in the host environment and then just transfer the emerge process ID to a cgroup that I've set up for this purpose? Isn't that what cgroups is supposed to be used for? Interesting idea, that. Anyway, just thoughts. All grist to the mill...
Re: [gentoo-user] Portage load control
On Saturday, 13 May 2023 00:53:49 BST Mark Knecht wrote: >Anyway, I had a couple of thoughts: > > 1) If it's really a bug then as others have said report it up the > chain and hope for a fix. https://bugs.gentoo.org/905933 > 2) If I wanted to solve the problem today(ish) then I'd build > a Gentoo VM in Virtualbox, dedicate some number of cores > to it, build everything with binary packages and probably > run an NFS server in the VM which I mount in the host > machine. I then update the host machine from the binary > packages and Virtualbox manages to never use more cores > than I give it. That fix is more or less guaranteed to work. Sounds like a lot of work. :( > 3) As a question for the far more knowledgeable system > folks I'd ask "Can this problem be solved by cgroups?" If > I have a cgroup with 10 processors in it, can I start emerge > in the host environment and then just transfer the emerge > process ID to a cgroup that I've set up for this purpose? > Isn't that what cgroups is supposed to be used for? Interesting idea, that. > Anyway, just thoughts. All grist to the mill... -- Regards, Peter.
Re: [gentoo-user] Portage load control
On Fri, May 12, 2023 at 10:42 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 17:58:46 BST Jack wrote: > > > Again, --load-average tells emerge whether it can start a new > > job/package, but has no control over how high the load will get based > > on the already started jobs. If emerge starts new jobs when the load > > is over that specified by --load-average, that does smell like a bug in > > emerge. > > Hooray! > Peter, I agree with Jack's response, but the keyword & potential issue is all based around that one word - "If". The way I see this is unless you have tracked down realtime what processes are running and where the CPU usage is going, and can further be sure that it's a process emerge itself started, then we don't really know what is causing the problem. My concern is what happens if emerge is honoring --load-average but you're seeing system usage created by some tool emerge called that doesn't understand --jobs and emerge doesn't know about at that level? Think some Rust code getting built by a rust compiler, or some deep make system. Anyway, I had a couple of thoughts: 1) If it's really a bug then as others have said report it up the chain and hope for a fix. 2) If I wanted to solve the problem today(ish) then I'd build a Gentoo VM in Virtualbox, dedicate some number of cores to it, build everything with binary packages and probably run an NFS server in the VM which I mount in the host machine. I then update the host machine from the binary packages and Virtualbox manages to never use more cores than I give it. That fix is more or less guaranteed to work. 3) As a question for the far more knowledgeable system folks I'd ask "Can this problem be solved by cgroups?" If I have a cgroup with 10 processors in it, can I start emerge in the host environment and then just transfer the emerge process ID to a cgroup that I've set up for this purpose? Isn't that what cgroups is supposed to be used for? Anyway, just thoughts. Good luck, Mark
Re: [gentoo-user] Portage load control
On Fri, May 12, 2023 at 9:59 AM Jack wrote: > > On 2023.05.12 12:23, Mark Knecht wrote: > [snip .] > >One interesting point is that the first Gentoo page I found to > > look at the emerge man page shows LOAD as the value provided > > to the --load-average option, but nowhere does it specify anything > > other than it's a floating point value: > I suspect the specification of floating point implies that it CAN take > digits after the decimal point, but not that they are required, > although that should be easy enough to test. > > > > https://dev.gentoo.org/~zmedico/portage/doc/man/emerge.1.html > > > > For clarification reading other sites, my understanding is that a > > load average value of 1 in the top application is meant to > > represent 1 CPU core operating at 100%. Assuming that's > > true, then on Peter's 24 core machine, with LOAD=40, he's > > telling emerge it's ok to use more cores than his machine has. > > > > Is that consistent with your (or others) understanding? > Close, but not quite. (See > https://en.wikipedia.org/wiki/Load_(computing) for more details.) I > think your understanding will match any observations, but I see the > definition as different. I understand the load (instantaneous, not > average) is the number of processed in the "r" state, i.e., running or > waiting for a CPU slice. That excludes any process explicitly sleeping > or waiting for IO. Since it can change so quickly, the point load is > not very useful, so it is more commonly presented as a value averaged > over a period of time. Top shows 1, 5, and 15 minute averages. > > Again, --load-average tells emerge whether it can start a new > job/package, but has no control over how high the load will get based > on the already started jobs. If emerge starts new jobs when the load > is over that specified by --load-average, that does smell like a bug in > emerge. > > > > > I think the mistake is one of those easy to make ones where > > the human things 40% (hence 40) and the machine things > > 40% (hence 0.4) > > > > Cheers, > > Mark > > OK, I find that all reasonable. One point about the Wikipedia description for anyone following who may not actually read it is that the average is accomplished with an exponential moving average and therefore is not, by definition, linear over time. As a little experiment that anyone can run I'll include a little AI generated batch file people can use to actually see more of what's going on in top, htop and btop. Note on my system the CPU affinity didn't work and I don't care to debug it. However this loops continuously until you hit ctrl-C. If you watch CPU load you'll see it climb quickly at first and then more slowly until you get up to 1.0. It will go a little higher (1.03 in my case) which is likely the CPU load from the programs monitoring the system and other background junk. None the less, 1 core running continuously generates as load of 1 after some period of time. As with all code on the Internet I take no responsibility for any damage caused my this code and neither does Google's Bard. #!/bin/bash # This batch program loops until you hit Ctrl-C. # Get the current processor affinity. affinity=$(cat /proc/self/cpuset) # Set the processor affinity to a single core. echo $affinity | sudo tee /proc/self/cpuset while true; do # Do nothing. : done # Reset the processor affinity to the default. echo "" | sudo tee /proc/self/cpuset
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 17:58:46 BST Jack wrote: > Again, --load-average tells emerge whether it can start a new > job/package, but has no control over how high the load will get based > on the already started jobs. If emerge starts new jobs when the load > is over that specified by --load-average, that does smell like a bug in > emerge. Hooray! :) -- Regards, Peter.
Re: [gentoo-user] Portage load control
On 2023.05.12 12:23, Mark Knecht wrote: [snip .] One interesting point is that the first Gentoo page I found to look at the emerge man page shows LOAD as the value provided to the --load-average option, but nowhere does it specify anything other than it's a floating point value: I suspect the specification of floating point implies that it CAN take digits after the decimal point, but not that they are required, although that should be easy enough to test. https://dev.gentoo.org/~zmedico/portage/doc/man/emerge.1.html For clarification reading other sites, my understanding is that a load average value of 1 in the top application is meant to represent 1 CPU core operating at 100%. Assuming that's true, then on Peter's 24 core machine, with LOAD=40, he's telling emerge it's ok to use more cores than his machine has. Is that consistent with your (or others) understanding? Close, but not quite. (See https://en.wikipedia.org/wiki/Load_(computing) for more details.) I think your understanding will match any observations, but I see the definition as different. I understand the load (instantaneous, not average) is the number of processed in the "r" state, i.e., running or waiting for a CPU slice. That excludes any process explicitly sleeping or waiting for IO. Since it can change so quickly, the point load is not very useful, so it is more commonly presented as a value averaged over a period of time. Top shows 1, 5, and 15 minute averages. Again, --load-average tells emerge whether it can start a new job/package, but has no control over how high the load will get based on the already started jobs. If emerge starts new jobs when the load is over that specified by --load-average, that does smell like a bug in emerge. I think the mistake is one of those easy to make ones where the human things 40% (hence 40) and the machine things 40% (hence 0.4) Cheers, Mark
Re: [gentoo-user] Portage load control
On Fri, May 12, 2023 at 9:08 AM Jack wrote: > > > -j 1 > > -j1 --load-average=40 > > -j1 --load-aveeage=40.0 > > -j1 --load-average=4.0 > > -j1 --load-average=0.4 > > -j10 --load-average=0.4 > > > > etc., and see what happens? > --load-average controls whether or not emerge starts another > job/package, so testing by emerging a single package will not actually > test this. That's why I suggested running some application to get the > load up to 10 (arbitrary number) and then emerging a larger number of > small packages. If --load-average is set to anything less than the > actual load, it should only launch one package at a time. Having that > simple example to add to the bug would give the developers an easy way > to test. > > I think the fact that Peter's actual load went over 70 is because each > individual job/package had no limit on the number of parallel compiles > make could kick off. There is likely no bug there. The real problem > (as Peter keeps pointing out) is that with the load that high, emerge > still starts additional jobs. Jack, I totally agree, as long as nothing is broken, but yeah, the list I provided was more meant to engender ideas for Peter. One interesting point is that the first Gentoo page I found to look at the emerge man page shows LOAD as the value provided to the --load-average option, but nowhere does it specify anything other than it's a floating point value: https://dev.gentoo.org/~zmedico/portage/doc/man/emerge.1.html For clarification reading other sites, my understanding is that a load average value of 1 in the top application is meant to represent 1 CPU core operating at 100%. Assuming that's true, then on Peter's 24 core machine, with LOAD=40, he's telling emerge it's ok to use more cores than his machine has. Is that consistent with your (or others) understanding? I think the mistake is one of those easy to make ones where the human things 40% (hence 40) and the machine things 40% (hence 0.4) Cheers, Mark
Re: [gentoo-user] Portage load control
On 2023.05.12 11:27, Mark Knecht wrote: On Fri, May 12, 2023 at 7:27 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 15:13:08 BST Mark Knecht wrote: > > > My opinion: load-average probably works, but we are misunderstanding > > the documentation. > > That's what bothers me the most - that I have a mental block somewhere. :( > > -- > Regards, > Peter. Just for clarity, how are you measuring 'load average'? Just looking at what is reported in top or something else that takes stats? So if it's either a documentation issue, or an understanding the documentation issue, possibly set up a 'design of experiments' set of tests? For instance: 1) Pick 1 semi-large package that spawns a few extra jobs to get built 2) Remove the binaries from your system 3) Ensure all the source is prefetched 4) Build the package with no options measuring load-average Repeat 2 - 4 using a few different options: -j 1 -j1 --load-average=40 -j1 --load-aveeage=40.0 -j1 --load-average=4.0 -j1 --load-average=0.4 -j10 --load-average=0.4 etc., and see what happens? --load-average controls whether or not emerge starts another job/package, so testing by emerging a single package will not actually test this. That's why I suggested running some application to get the load up to 10 (arbitrary number) and then emerging a larger number of small packages. If --load-average is set to anything less than the actual load, it should only launch one package at a time. Having that simple example to add to the bug would give the developers an easy way to test. I think the fact that Peter's actual load went over 70 is because each individual job/package had no limit on the number of parallel compiles make could kick off. There is likely no bug there. The real problem (as Peter keeps pointing out) is that with the load that high, emerge still starts additional jobs.
Re: [gentoo-user] Portage load control
On Fri, May 12, 2023 at 7:27 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 15:13:08 BST Mark Knecht wrote: > > > My opinion: load-average probably works, but we are misunderstanding > > the documentation. > > That's what bothers me the most - that I have a mental block somewhere. :( > > -- > Regards, > Peter. Just for clarity, how are you measuring 'load average'? Just looking at what is reported in top or something else that takes stats? So if it's either a documentation issue, or an understanding the documentation issue, possibly set up a 'design of experiments' set of tests? For instance: 1) Pick 1 semi-large package that spawns a few extra jobs to get built 2) Remove the binaries from your system 3) Ensure all the source is prefetched 4) Build the package with no options measuring load-average Repeat 2 - 4 using a few different options: -j 1 -j1 --load-average=40 -j1 --load-aveeage=40.0 -j1 --load-average=4.0 -j1 --load-average=0.4 -j10 --load-average=0.4 etc., and see what happens?
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 15:13:08 BST Mark Knecht wrote: > My opinion: load-average probably works, but we are misunderstanding > the documentation. That's what bothers me the most - that I have a mental block somewhere. :( -- Regards, Peter.
Re: [gentoo-user] Apache and systemd problem
>
> Yes, you need to configure the apache files to specify IP address to
> listen
> to, webroot, etc. for each host you want to serve files from.
>
> Ok, it's done
>
>
> Fix this by adding the "start" subcommand:
>
> ExecStart=/usr/sbin/apache2ctl start
>
>
> ExecStart=/usr/sbin/apache2ctl start does not work.
Nothing happens and system's waiting for something indefinitely ... (file
attached)
The same with ExecStart=/usr/sbin/apache2ctl stop.
So, i found a very very ugly way to make it working.
I wrote a micro bash shell script : /usr/bin/op_apache
!/bin/bash
case ${1} in
"start")
apache2ctl
;;
"stop")
#the ugly way... Don't repeat !
killall apache2
;;
esac
And i launch the script in apache2.service
#Description=The Apache HTTP Server
After=network.target remote-fs.target nss-lookup.target
#Documentation=https://httpd.apache.org/docs/2.4/
[Service]
Type=forking
ExecStart=/usr/bin/op_apache start
ExecStop=/usr/bin/op_apache stop
[Install]
WantedBy=multi-user.target
So, waiting for better, it works !
--
Jacques
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 15:06:21 BST Michael Cook wrote: > You can read /usr/share/portage/config/make.conf.example for an > explanation. All children processes will use that. I can run portage and > play games on the same system with my settings. That example says nothing about any of the emerge default options. I have no problem with system responsiveness (except, recently, running BOINC). -- Regards, Peter.
Re: [gentoo-user] Portage load control
On Fri, May 12, 2023 at 6:46 AM Peter Humphrey wrote: > > On Friday, 12 May 2023 00:08:03 BST Mark Knecht wrote: > > On Thu, May 11, 2023 at 3:07 PM Peter Humphrey > > > > wrote: > > > On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: > > > > > > > > The ''problem' is this can easily hit 100% of the cores you have in the > > > > machine if not sensibly set. (You choose what's 'sensible') > > > > > > Once again, --load-average is being ignored. Why is it there? Surely, it > > > must be to mitigate the worst effects of that N*K, but it isn't doing so. > > > > From your description, yeah, it's weird, but possibly it's managing it over > > (for instance) over much longer time frames or something like that. > > > > Or possibly it just doesn't work. > > That's it, I'm sure. > > > Or possibly whoever wrote the man page misunderstood. > > Load-average has been around for a long time. > > > Poking around a bit this morning I took the path at the bottom of the > > link I gave you to the Portage niceness page. It says scheduling policy > > control started with portage-3.0.35 which on paper sounds sort of recent. > > Possibly a bug crept in, but I was curious as to what you have for > > PORTAGE_SCHEDULING_POLICY, if any, and whether you need to enable some > > sort of scheduling to get this under control? > > > > https://wiki.gentoo.org/wiki/Portage_niceness > > I have no PORTAGE_SCHEDULING_POLICY, or not that I can find. It seems to me > that such a policy is to do with the running of portage in the OS, rather than > how it launches its own emerge jobs. Is that right? > > > Anyway, I feel for ya. Peter, My point about PORTAGE_SCHEDULING_POLICY is that it *might* have an effect on what you're seeing, not that it *would* have an effect. If there's one thing I most distrust about the Open Source world, it's documentation... WRT the floating point issue, the Gentoo Catalyst page https://wiki.gentoo.org/wiki/Catalyst in the "Jobs and load average" section states: FILE /etc/catalyst/catalyst.confcatalyst.conf # Integral value passed to emerge as the parameter to --jobs and is used to # define MAKEOPTS during the target build. jobs = 4 # Floating-point value passed to emerge as the parameter to --load-average and # is used to define MAKEOPTS during the target build. # load-average = 4.0 so once again, the use of floating point is documented as (you choose) either important or required. My opinion: load-average probably works, but we are misunderstanding the documentation. Note that the example using 4.0 is a pretty load number. Good luck, Mark
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 14:37:13 BST Jack wrote: > I still see two separate issues. First, you are saying that emerge > still launches new jobs when the load is over what is set with > --load-average. A possible way to test this directly is to run or > create some job that pushed the load average to over some number, say > 5. I have tested it, directly, with emerge. I reported what happened at the start of this thread. To recap: I was running 'emerge -e @world'; no extras, no ifs, no buts. Make.conf had EMERGE_DEFAULT_OPTS="--jobs --load-average=40 ... ; MAKEOPTS was not specified. As the six-hour job proceeded and portage was working with larger packages in the plasma group, the load average rose to "72 75 75", clearly much higher than the 40 I'd specified and continuing over at least 15 minutes. Yet portage was still starting more emerge jobs to keep the load that high. --->8 > The second issues is whether MAKEOPTS --load-average is actually getting > passed to each job and whether make is then observing that limit. > Whether this is the case or not is independent of the first issue. I > suppose this could be tested without even involving emerge. Given you > observed an actual load of 72 (do I remember correctly?) with both > --load-averages set significantly below this, you could test, as long as > you have a single compile which is busy enough. I had no MAKEOPTS, so I assume -j took the default value of num_cores=24. I don't know what the -l default is. Here's the rub, though: whatever values were taken by -j, -l or --jobs, the value of --load-average should not have been exceeded so grossly and persistently. I'd like to thank everyone who's offered ideas and suggestions, but I'm just going to have to wait for the outcome of the bug I reported. -- Regards, Peter.
Re: [gentoo-user] Portage load control
On 5/12/23 09:46, Peter Humphrey wrote: On Friday, 12 May 2023 00:08:03 BST Mark Knecht wrote: On Thu, May 11, 2023 at 3:07 PM Peter Humphrey wrote: On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: The ''problem' is this can easily hit 100% of the cores you have in the machine if not sensibly set. (You choose what's 'sensible') Once again, --load-average is being ignored. Why is it there? Surely, it must be to mitigate the worst effects of that N*K, but it isn't doing so. From your description, yeah, it's weird, but possibly it's managing it over (for instance) over much longer time frames or something like that. Or possibly it just doesn't work. That's it, I'm sure. Or possibly whoever wrote the man page misunderstood. Load-average has been around for a long time. Poking around a bit this morning I took the path at the bottom of the link I gave you to the Portage niceness page. It says scheduling policy control started with portage-3.0.35 which on paper sounds sort of recent. Possibly a bug crept in, but I was curious as to what you have for PORTAGE_SCHEDULING_POLICY, if any, and whether you need to enable some sort of scheduling to get this under control? https://wiki.gentoo.org/wiki/Portage_niceness I have no PORTAGE_SCHEDULING_POLICY, or not that I can find. It seems to me that such a policy is to do with the running of portage in the OS, rather than how it launches its own emerge jobs. Is that right? Anyway, I feel for ya. :) You can read /usr/share/portage/config/make.conf.example for an explanation. All children processes will use that. I can run portage and play games on the same system with my settings.
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 00:08:03 BST Mark Knecht wrote: > On Thu, May 11, 2023 at 3:07 PM Peter Humphrey > > wrote: > > On Thursday, 11 May 2023 17:18:17 BST Mark Knecht wrote: > > > > > The ''problem' is this can easily hit 100% of the cores you have in the > > > machine if not sensibly set. (You choose what's 'sensible') > > > > Once again, --load-average is being ignored. Why is it there? Surely, it > > must be to mitigate the worst effects of that N*K, but it isn't doing so. > > From your description, yeah, it's weird, but possibly it's managing it over > (for instance) over much longer time frames or something like that. > > Or possibly it just doesn't work. That's it, I'm sure. > Or possibly whoever wrote the man page misunderstood. Load-average has been around for a long time. > Poking around a bit this morning I took the path at the bottom of the > link I gave you to the Portage niceness page. It says scheduling policy > control started with portage-3.0.35 which on paper sounds sort of recent. > Possibly a bug crept in, but I was curious as to what you have for > PORTAGE_SCHEDULING_POLICY, if any, and whether you need to enable some > sort of scheduling to get this under control? > > https://wiki.gentoo.org/wiki/Portage_niceness I have no PORTAGE_SCHEDULING_POLICY, or not that I can find. It seems to me that such a policy is to do with the running of portage in the OS, rather than how it launches its own emerge jobs. Is that right? > Anyway, I feel for ya. :) -- Regards, Peter.
Re: [gentoo-user] Portage load control
On 5/12/23 09:16, Peter Humphrey wrote: On Friday, 12 May 2023 11:09:37 BST Arve Barsnes wrote: On Fri, 12 May 2023 at 10:34, Peter Humphrey wrote: I have said several times that portage is ignoring that setting. I have it at 40, yet portage kicks off more packages at 72, and continues doing so for extended periods - at least 15 minutes. But are you sure that it is actually ignored? It was said in an earlier message from Mark that the value was related to number of cores, where your 24 cores at 100% average load would translate to a value of --load-average 24.0. That would put your value of 40 at 166% average load? What load are you actually trying to limit it to? If you want 40% load, that should apparently be --load-average 9.6. I'm reading man make.conf, which makes quite clear that --load-average limits the number of portage packages to be emerged, so as to avoid excess load. Simple. Either --load-average is designed to do as the man page says, but it doesn't work and should be fixed, or it should be removed, being useless and misleading. We can't have an option that limits load, but can be ignored at portage's whim. I haven't had a reply to my question in the bug report yesterday: "Why is --load-average=40 being ignored?" but it is perhaps early days yet. A possibility has just occurred to me: it seems to me that the use of a floating-point number for load average is a recent development, at least I've never seen it before and I've always used a plain integer. Could portage be skipping over it when it doesn't find a decimal point? That would be easier to fix than a wholesale failure of function. I still see two separate issues. First, you are saying that emerge still launches new jobs when the load is over what is set with --load-average. A possible way to test this directly is to run or create some job that pushed the load average to over some number, say 5. (It doesn't have to be high, just predictable, although a higher load would make a more obvious result.) Then start an emerge of two or more packages with --load-average=3. It should start the first job, and should then not start another until the load is below 3 or the first job has finished. You can try with both 3 and 3.0. If the second job does get started, this is an easy to run, concrete test you can post to the bug. The second issues is whether MAKEOPTS --load-average is actually getting passed to each job and whether make is then observing that limit. Whether this is the case or not is independent of the first issue. I suppose this could be tested without even involving emerge. Given you observed an actual load of 72 (do I remember correctly?) with both --load-averages set significantly below this, you could test, as long as you have a single compile which is busy enough. Another possible test would be (for example) to set emerge's --load-average to 2 or 2.0 and MAKEOPT's --load-average=10.
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 11:09:37 BST Arve Barsnes wrote: > On Fri, 12 May 2023 at 10:34, Peter Humphrey wrote: > > I have said several times that portage is ignoring that setting. I have it > > at 40, yet portage kicks off more packages at 72, and continues doing so > > for extended periods - at least 15 minutes. > > But are you sure that it is actually ignored? It was said in an > earlier message from Mark that the value was related to number of > cores, where your 24 cores at 100% average load would translate to a > value of --load-average 24.0. That would put your value of 40 at 166% > average load? What load are you actually trying to limit it to? If you > want 40% load, that should apparently be --load-average 9.6. I'm reading man make.conf, which makes quite clear that --load-average limits the number of portage packages to be emerged, so as to avoid excess load. Simple. Either --load-average is designed to do as the man page says, but it doesn't work and should be fixed, or it should be removed, being useless and misleading. We can't have an option that limits load, but can be ignored at portage's whim. I haven't had a reply to my question in the bug report yesterday: "Why is --load-average=40 being ignored?" but it is perhaps early days yet. A possibility has just occurred to me: it seems to me that the use of a floating-point number for load average is a recent development, at least I've never seen it before and I've always used a plain integer. Could portage be skipping over it when it doesn't find a decimal point? That would be easier to fix than a wholesale failure of function. -- Regards, Peter.
Re: [gentoo-user] Apache and systemd problem
On Friday, 12 May 2023 10:41:32 BST Jacques Montier wrote: > > After you are able to start it manually, you can edit your systemd service > > file accordingly. > > GOOD NEWS ! i can start apache2ctl manually by #/usr/bin/apache2ctl > I get the warning message : > AH00558: apache2: Could not reliably determine the server's fully qualified > domain name, using 127.0.0.1. Set the 'ServerName' directive globally to > suppress this message Yes, you need to configure the apache files to specify IP address to listen to, webroot, etc. for each host you want to serve files from. > But, apache2 is working as i stay localhost. > > So, by editing minimal apache2.service : > > [Unit] > #Description=The Apache HTTP Server > After=network.target remote-fs.target nss-lookup.target > #Documentation=https://httpd.apache.org/docs/2.4/ > > [Service] > #Type=forking > #Environment=APACHE_STARTED_BY_SYSTEMD=true > ExecStart=/usr/sbin/apache2ctl Fix this by adding the "start" subcommand: ExecStart=/usr/sbin/apache2ctl start signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Portage load control
On Fri, 12 May 2023 at 10:34, Peter Humphrey wrote: > On Friday, 12 May 2023 01:38:52 BST Jack wrote: > > The --load-average to emerge itself just tells it not to start a new job > > if the load is above the setting. If there are several large jobs, but > > all start with single threaded configuration activity such as > > ./configure or cmake, multiple jobs can clearly get started before the > > load average starts climbing. > > I have said several times that portage is ignoring that setting. I have it at > 40, yet portage kicks off more packages at 72, and continues doing so for > extended periods - at least 15 minutes. But are you sure that it is actually ignored? It was said in an earlier message from Mark that the value was related to number of cores, where your 24 cores at 100% average load would translate to a value of --load-average 24.0. That would put your value of 40 at 166% average load? What load are you actually trying to limit it to? If you want 40% load, that should apparently be --load-average 9.6. Regards, Arve
Re: [gentoo-user] Apache and systemd problem
> > After you are able to start it manually, you can edit your systemd service > file accordingly. GOOD NEWS ! i can start apache2ctl manually by #/usr/bin/apache2ctl I get the warning message : AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message But, apache2 is working as i stay localhost. So, by editing minimal apache2.service : [Unit] #Description=The Apache HTTP Server After=network.target remote-fs.target nss-lookup.target #Documentation=https://httpd.apache.org/docs/2.4/ [Service] #Type=forking #Environment=APACHE_STARTED_BY_SYSTEMD=true ExecStart=/usr/sbin/apache2ctl ExecStop=/usr/sbin/apache2ctl graceful-stop ExecReload=/usr/sbin/apache2ctl graceful KillMode=mixed #PrivateTmp=true Restart=on-abort TimeoutStartSec=5 [Install] WantedBy=multi-user.target #systemctl start apache2 works, but #systemctl stop apache2 fails and is waiting for something So i can't reboot as apache2 is not stopped... :-( Not perfect, but it is going... -- Jacques
Re: [gentoo-user] Apache and systemd problem
Le ven. 12 mai 2023 à 10:40, Alexandru N. Barloiu a écrit :
> How can you not miss it when it's specified in the ebuild?
>
> [root@ruja:~]# grep service
> /usr/portage/www-servers/apache/apache-2.4.55-r1.ebuild
> # Then apache2.4.service can be used and systemd support controlled
> systemd_newunit "${FILESDIR}/apache2.4-hardened.service"
> "apache2.service"
>
> I have the line in the ebuild, but no apache2.service is installed when
emerging apache.
No 00_systemd.conf in /etc/apache2/modules.d
A problem about missing uses flags ?
Here are mine
www-servers/apache-2.4.55-r1:2::gentoo USE="gdbm ssl suexec-caps systemd
-debug -doc -ldap (-selinux) (-split-usr) -static -suexec -suexec-syslog
-threads" APACHE2_MODULES="actions alias auth_basic authn_anon authn_core
authn_dbm authn_file authz_core authz_dbm authz_groupfile authz_host
authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate
dir env expires ext_filter file_cache filter headers http2 include info
log_config logio mime mime_magic negotiation rewrite setenvif socache_shmcb
speling status unique_id unixd userdir usertrack vhost_alias -access_compat
-asis -auth_digest -auth_form -authn_dbd -authn_socache -authz_dbd -brotli
-cache_disk -cache_socache -cern_meta -charset_lite -dbd -dumpio -ident
-imagemap -lbmethod_bybusyness -lbmethod_byrequests -lbmethod_bytraffic
-lbmethod_heartbeat -log_forensic (-lua) -macro -md -proxy -proxy_ajp
-proxy_balancer -proxy_connect -proxy_fcgi -proxy_ftp -proxy_hcheck
-proxy_html -proxy_http -proxy_http2 -proxy_scgi -proxy_uwsgi
-proxy_wstunnel -ratelimit -remoteip -reqtimeout -session -session_cookie
-session_crypto -session_dbd -slotmem_shm -socache_memcache -substitute
-version -watchdog -xml2enc" APACHE2_MPMS="-event -prefork -worker"
LUA_SINGLE_TARGET="lua5-1 -lua5-3 -lua5-4"
Jacques
>
>
> thanks in advance.
>
> Jacques
>
>>
>>
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 09:34:27 BST I wrote: > > The --load-average in MAKEOPTS gets passed to make, and controls how > > many processes make starts. If that is set, and the load is still too > > high, the problem is in make not in emerge. Also, that setting will > > have no effect if the package uses ninja or something else instead of > > make. Ninja does have a -l setting for load average, but I don't know > > if emerge passes any MAKEOPTS to ninja. That might be an interesting > > enhancement request. > > I didn't know about that setting, and I can't see it in the man pages, so > you can be sure it isn't set here. You don't mean --jobs, do you? Please ignore that. I must go for some coffee. -- Regards, Peter.
Re: [gentoo-user] Apache and systemd problem
How can you not miss it when it's specified in the ebuild?
[root@ruja:~]# grep service
/usr/portage/www-servers/apache/apache-2.4.55-r1.ebuild
# Then apache2.4.service can be used and systemd support controlled
systemd_newunit "${FILESDIR}/apache2.4-hardened.service"
"apache2.service"
On 5/12/2023 11:29 AM, Jacques Montier wrote:
/
/
Le ven. 12 mai 2023 à 02:54, Alexandru N. Barloiu a écrit :
first of all, gentoo does install with a service file:
[root@noela:~]# equery f apache | grep systemd | grep service
/lib/systemd/system/apache2.service
Hi Alexandru,
Gentoo did not install any apache2.service.
I had to manually edit one and those found on the net don't work.
Have you got one i could use ?
thanks in advance.
Jacques
Re: [gentoo-user] Portage load control
On Friday, 12 May 2023 01:38:52 BST Jack wrote: > Sorry if I'm repeating myself, but as I see it, there are two different > --load-average settings to consider. I'd have to go back to the > beginning of the thread to confirm you are setting both of them. I am also going to repeat myself. > The --load-average to emerge itself just tells it not to start a new job > if the load is above the setting. If there are several large jobs, but > all start with single threaded configuration activity such as > ./configure or cmake, multiple jobs can clearly get started before the > load average starts climbing. I have said several times that portage is ignoring that setting. I have it at 40, yet portage kicks off more packages at 72, and continues doing so for extended periods - at least 15 minutes. > The --load-average in MAKEOPTS gets passed to make, and controls how > many processes make starts. If that is set, and the load is still too > high, the problem is in make not in emerge. Also, that setting will > have no effect if the package uses ninja or something else instead of > make. Ninja does have a -l setting for load average, but I don't know > if emerge passes any MAKEOPTS to ninja. That might be an interesting > enhancement request. I didn't know about that setting, and I can't see it in the man pages, so you can be sure it isn't set here. You don't mean --jobs, do you? -- Regards, Peter.
Re: [gentoo-user] Apache and systemd problem
Le ven. 12 mai 2023 à 02:54, Alexandru N. Barloiu a écrit : > first of all, gentoo does install with a service file: > > [root@noela:~]# equery f apache | grep systemd | grep service > /lib/systemd/system/apache2.service > > Hi Alexandru, Gentoo did not install any apache2.service. I had to manually edit one and those found on the net don't work. Have you got one i could use ? thanks in advance. Jacques > >
