Re: [gentoo-user] how to prevent ebuild from checking for available space
On 2024.03.22 16:22, Neil Bothwick wrote:
On Fri, 22 Mar 2024 12:57:34 -0400, Jack wrote:
> In this case, the offending package is dev-lang/rust, but this has
> happened to me previously with other packages that require a lot of
> space and time to build.
>
> The build fails for any reason. Since it has already progressed
over
> half way through the build, I would like to continue the build
instead
> of starting from scratch. I fix whatever caused the issue. In this
> particular case, it failed for out of space, so I added another G to
> the ramdisk mounted at /var/tmp/portageg. I issue "ebuild
> path/to/package.ebuild compile" but it fails immediately because
there
> isn't enough free space. The problem is that it's checking for
enough
> free space to do the entire build, not just to continue from where
it
> left off.
Why not add more to the ramdisk, assuming it is a tmpfs. If it needs
more
than your physical memory, it will use swap, but that won't happen
because you only need the extra space.
I generally avoid this problem by having large packages use a
PORTAGE_TMPDIR on a spinning disk. It may be slightly slower, but that
doesn't really matter on a long compile, not nearly as much as having
to
restart it!
>
> I've done some searching, and found one old forum post which
suggests
> to just temporarily remove the space check from the ebuild.
>
> In this case, that seems to be line 236 of
> /usr/portage/dev-lang/rust/rust-1.75.0-r1.ebuild, which is
> "CHECKREQS_DISK_BUILD=${M}M check-reqs_pkg_${EBUILD_PHASE}" the last
> line withing pre_build_checks(). However, if I comment out that
line,
> and run "ebuild path/to/ebuild manifest" another attempt to compile
> still gives the same error about not enough space.
Have you tried setting M to a smaller value immediately before that
line?
It seems the problem is that the enviroment file in the temp dir of the
build area is sourced when you run ebuild/emerge. (It's among the
first output when you run ebuild.) Since that file was created based
on the state of the ebuild when the first emerge/ebuild was run,
editing the ebuild file doesn't affect it. I ended up finding a place
to unset CHECKREQS_FAILED, and the compile is now continuing.
Re: [gentoo-user] how to prevent ebuild from checking for available space
On 2024.03.22 16:22, Neil Bothwick wrote:
On Fri, 22 Mar 2024 12:57:34 -0400, Jack wrote:
> In this case, the offending package is dev-lang/rust, but this has
> happened to me previously with other packages that require a lot of
> space and time to build.
>
> The build fails for any reason. Since it has already progressed
over
> half way through the build, I would like to continue the build
instead
> of starting from scratch. I fix whatever caused the issue. In this
> particular case, it failed for out of space, so I added another G to
> the ramdisk mounted at /var/tmp/portageg. I issue "ebuild
> path/to/package.ebuild compile" but it fails immediately because
there
> isn't enough free space. The problem is that it's checking for
enough
> free space to do the entire build, not just to continue from where
it
> left off.
Why not add more to the ramdisk, assuming it is a tmpfs. If it needs
more
than your physical memory, it will use swap, but that won't happen
because you only need the extra space.
That's actually what I did. The problem is not how to get enough
space, it's how to resume the emerge instead of starting over, once I
have added the space.
It was initially set to 14G (out of 32G RAM) and I added 2G. I suppose
I can add another 14G, but that would only leave 4G for the system
itself. I'm not sure how well that would work, but I suppose it's
worth a try. The worst that happens is I reboot and start the emerge
from scratch.
I generally avoid this problem by having large packages use a
PORTAGE_TMPDIR on a spinning disk. It may be slightly slower, but that
doesn't really matter on a long compile, not nearly as much as having
to
restart it!
>
> I've done some searching, and found one old forum post which
suggests
> to just temporarily remove the space check from the ebuild.
>
> In this case, that seems to be line 236 of
> /usr/portage/dev-lang/rust/rust-1.75.0-r1.ebuild, which is
> "CHECKREQS_DISK_BUILD=${M}M check-reqs_pkg_${EBUILD_PHASE}" the last
> line withing pre_build_checks(). However, if I comment out that
line,
> and run "ebuild path/to/ebuild manifest" another attempt to compile
> still gives the same error about not enough space.
Have you tried setting M to a smaller value immediately before that
line?
I tried that first, and when that didn't work, I tried commenting out
the line I thought was calling the check. I'll try again, to be sure
it was a reasonable value for M.
--
Neil Bothwick
Religious error: (A)tone, (R)epent, (I)mmolate?
Re: [gentoo-user] how to prevent ebuild from checking for available space
On Fri, 22 Mar 2024 12:57:34 -0400, Jack wrote:
> In this case, the offending package is dev-lang/rust, but this has
> happened to me previously with other packages that require a lot of
> space and time to build.
>
> The build fails for any reason. Since it has already progressed over
> half way through the build, I would like to continue the build instead
> of starting from scratch. I fix whatever caused the issue. In this
> particular case, it failed for out of space, so I added another G to
> the ramdisk mounted at /var/tmp/portageg. I issue "ebuild
> path/to/package.ebuild compile" but it fails immediately because there
> isn't enough free space. The problem is that it's checking for enough
> free space to do the entire build, not just to continue from where it
> left off.
Why not add more to the ramdisk, assuming it is a tmpfs. If it needs more
than your physical memory, it will use swap, but that won't happen
because you only need the extra space.
I generally avoid this problem by having large packages use a
PORTAGE_TMPDIR on a spinning disk. It may be slightly slower, but that
doesn't really matter on a long compile, not nearly as much as having to
restart it!
>
> I've done some searching, and found one old forum post which suggests
> to just temporarily remove the space check from the ebuild.
>
> In this case, that seems to be line 236 of
> /usr/portage/dev-lang/rust/rust-1.75.0-r1.ebuild, which is
> "CHECKREQS_DISK_BUILD=${M}M check-reqs_pkg_${EBUILD_PHASE}" the last
> line withing pre_build_checks(). However, if I comment out that line,
> and run "ebuild path/to/ebuild manifest" another attempt to compile
> still gives the same error about not enough space.
Have you tried setting M to a smaller value immediately before that line?
--
Neil Bothwick
Religious error: (A)tone, (R)epent, (I)mmolate?
pgpW4vyxIVRbF.pgp
Description: OpenPGP digital signature
Re: [gentoo-user] Terminal emulator to replace Konsole
On 2024.03.22 16:01, Dale wrote: Howdy, I've been using Konsole, part of KDE, for command line stuff ever since I started using Linux. Linux is all I've ever used. No windoze. ;-) While Konsole is good enough for almost everything, there is one feature I wish it had. The ability to edit with the mouse. I don't know of a way to make it do this. The only way I know of to edit a command, left arrow to what you want to edit and change it. I'd like to find one where I can use the mouse to place the cursor and edit from there. Even maybe highlight and replace. As far as I know, Konsole doesn't have that ability. I looked in x11-terms and there is a few options, I think. I tried looking at home pages and such but none of them mention a feature like this but it may have it. I was wondering if anyone knows of a terminal emulator that allows the mouse to place the cursor to edit parts or whole sections of a command. Some of my commands are really long and it seems the part I want to edit is always at the beginning. :/ Hoping for some ideas. I don't remember how I initially set it up, but I can use emacs commands (not sure if it was related to konsole or bash) so I can use up and down arrows or Ctl-p/Ctl-n to move back and forth in previous commands, then Ctl-a to go to the beginning of the line, etc. I suspect you can get vim style keys to do the same, if that's your preference. Sorry, but I have no idea how to tell if any terminal emulator is truly a gui, where clicking on a character in the middle of the window actually sets the cursor at that point, to be recognized by both the terminal and the shell you are running.
[gentoo-user] Terminal emulator to replace Konsole
Howdy, I've been using Konsole, part of KDE, for command line stuff ever since I started using Linux. Linux is all I've ever used. No windoze. ;-) While Konsole is good enough for almost everything, there is one feature I wish it had. The ability to edit with the mouse. I don't know of a way to make it do this. The only way I know of to edit a command, left arrow to what you want to edit and change it. I'd like to find one where I can use the mouse to place the cursor and edit from there. Even maybe highlight and replace. As far as I know, Konsole doesn't have that ability. I looked in x11-terms and there is a few options, I think. I tried looking at home pages and such but none of them mention a feature like this but it may have it. I was wondering if anyone knows of a terminal emulator that allows the mouse to place the cursor to edit parts or whole sections of a command. Some of my commands are really long and it seems the part I want to edit is always at the beginning. :/ Hoping for some ideas. Thanks. Dale :-) :-)
Re: [gentoo-user] How do I zap a specific area of a gnumeric spreadsheet page?
On Fri, Mar 22, 2024 at 05:52:04PM +1100, Paul Colquhoun wrote
> Bash can do patern substitution in variable references.
>
> Replace accum3=$(( ${accum3} + ${dataarray[3]} ))
> with accum3=$(( ${accum3} + ${dataarray[3]/'.'/0} ))
>
> and similarly with the other lines and any array value of '.' will
> be replaced with a '0'
Replacing "." with a zero is not the problem. I already do this at
the top of the script with "sed"...
sed "s/\"\.\"/0/g" region_hospital_icu_covid_data.csv >
region_hospital_icu_covid_datax.csv
The problem is that a certain range of valid-looking zero sums in the
summary output is actually invalid and has to be zapped. I was
originally hoping to wipe that range with a macro after importing it
into gnumeric. "Plan B" is to tweak the way that a certain date range
gets written out. Note the difference starting at 2023-09-09...
2023-09-06,28,10,568,31,12,3,2
2023-09-07,30,6,594,33,8,3,2
2023-09-08,33,7,600,31,11,-2,4
2023-09-09,0,0,628,0,0,0,0
2023-09-10,0,0,657,0,0,0,0
2023-09-11,0,0,625,0,0,0,0
2023-09-06,28,10,568,31,12,3,2
2023-09-07,30,6,594,33,8,3,2
2023-09-08,33,7,600,31,11,-2,4
2023-09-09,,,628,0,0,0,0
2023-09-10,,,657,0,0,0,0
2023-09-11,,,625,0,0,0,0
Gnumeric imports it OK with nothing in the cells that have missing data
in the source. While this isn't the way I had originally intended, it
works, which is what matters. In the revised script, note the filter...
if [ "${prevdate}" \< "2023-09-09" ] || [ "${prevdate}" \> "2023-10-20" ]
One advantage of -MM-DD date format is that I can do straight
string comparisons. Here's the revised script...
==
# Strip out missing "." that screw up the script
sed "s/\"\.\"/0/g" region_hospital_icu_covid_data.csv >
region_hospital_icu_covid_datax.csv
dos2unix -n region_hospital_icu_covid_datax.csv
region_hospital_icu_covid_datay.csv
#
# tail skips headers at beginning of file
# sed deletes Row_ID, and strips out quotes
# Output goes to file /dev/shm/temp0.txt
tail -n +2 region_hospital_icu_covid_datay.csv | sed "s/\"//g" | sort >
/dev/shm/temp0.txt
#
## Set up IFS for easier parsing
oldifs="${IFS}"
IFS=","
#
# Initialize previous line's date to enter loop smoothly
# expando to read first line
dataline=$( head -1 /dev/shm/temp0.txt )
dataarray=(${dataline})
prevdate="${dataarray[0]}"
#
# Zero out accumulators to enter loop smoothly
accum2=0
accum3=0
accum4=0
accum5=0
accum6=0
accum7=0
accum8=0
#
# Remove previous hospsum.csv and open a new one for writing
rm -rf hospsum.csv
exec 3>hospsum.csv
#
# Write header line to output file
echo
"date,icu_current_covid,icu_current_covid_vented,hospitalizations,icu_crci_total,icu_crci_total_vented,icu_former_covid,icu_former_covid_vented"
>&3
#
# Main loop
# Read the data from one line in /dev/shm/temp0.txt
while read
do
dataarray=(${REPLY})
if [ "${dataarray[0]}" = "${prevdate}" ]; then
#
# If this line's date is same as previous line's date, add amounts to
accumulators.
accum2=$(( ${accum2} + ${dataarray[2]} ))
accum3=$(( ${accum3} + ${dataarray[3]} ))
accum4=$(( ${accum4} + ${dataarray[4]} ))
accum5=$(( ${accum5} + ${dataarray[5]} ))
accum6=$(( ${accum6} + ${dataarray[6]} ))
accum7=$(( ${accum7} + ${dataarray[7]} ))
accum8=$(( ${accum8} + ${dataarray[8]} ))
else
#
# If this line's date has changed, output to hospsum.csv, update prevdate,
# and update accumulators. ***IMPORTANT*** "echo" TO hospsum.csv MUST BE
# EXECUTED BEFORE UPDATING ACCUMULATORS AND prevdate***
#
# Data *NOT* in range 2023-09-09 ... 2023-10-20 is written out in full.
# Data in that range gets null data for ${accum2} and ${accum3}
if [ "${prevdate}" \< "2023-09-09" ] || [ "${prevdate}" \> "2023-10-20" ]
then
echo
"${prevdate},${accum2},${accum3},${accum4},${accum5},${accum6},${accum7},${accum8}"
>&3
else
echo "${prevdate},,,${accum4},${accum5},${accum6},${accum7},${accum8}"
>&3
fi
prevdate="${dataarray[0]}"
accum2=${dataarray[2]}
accum3=${dataarray[3]}
accum4=${dataarray[4]}
accum5=${dataarray[5]}
accum6=${dataarray[6]}
accum7=${dataarray[7]}
accum8=${dataarray[8]}
fi
done
[gentoo-user] how to prevent ebuild from checking for available space
In this case, the offending package is dev-lang/rust, but this has
happened to me previously with other packages that require a lot of
space and time to build.
The build fails for any reason. Since it has already progressed over
half way through the build, I would like to continue the build instead
of starting from scratch. I fix whatever caused the issue. In this
particular case, it failed for out of space, so I added another G to the
ramdisk mounted at /var/tmp/portageg. I issue "ebuild
path/to/package.ebuild compile" but it fails immediately because there
isn't enough free space. The problem is that it's checking for enough
free space to do the entire build, not just to continue from where it
left off.
I've done some searching, and found one old forum post which suggests to
just temporarily remove the space check from the ebuild.
In this case, that seems to be line 236 of
/usr/portage/dev-lang/rust/rust-1.75.0-r1.ebuild, which is
"CHECKREQS_DISK_BUILD=${M}M check-reqs_pkg_${EBUILD_PHASE}" the last
line withing pre_build_checks(). However, if I comment out that line,
and run "ebuild path/to/ebuild manifest" another attempt to compile
still gives the same error about not enough space.
Am I commenting the wrong line? Have I missed something about where
this check is atually done? Is it actually possible to do what I'm
trying to do?
Thanks for any pointers or suggestions.
Jack
Re: [gentoo-user] How do I zap a specific area of a gnumeric spreadsheet page?
On Friday, March 22, 2024 3:20:08 P.M. AEDT Walter Dnes wrote:
> On Thu, Mar 21, 2024 at 11:02:21PM +0100, Frank Steinmetzger wrote
>
> > Why not make the alteration one step before -- in the CSV? There are
> > CSV abstraction tools like `q`, which gives you a SQL-like interface
> > to a csv file. Or you could write a quick transformer in python,
> > if you know the language a bit.
>
> I wrote a quick transformer in bash, because I know the language a
> bit . There are six "hospital regions" in Ontario. The CSV file
> does all data for one "hospital region" April 1, 2020 to present,
> followed by same for next hospital region, etc, etc, for all 6 regions.
> My first step is to run dos2unix and sort by date, ending up with 6
> consecutive lines per day, sorted by date. Then a "while" loop reads
> through the input CSV file. Accumulators are added up for all 6 lines
> and a line is written out for each date. Because I'm doing bash
> arithmetic, the data *MUST* be valid numbers, not ".". So I *MUST* set
> missing data to somthing like zero. Adding "." to a number causes bash
> to error out with a CSV file like so...
>
> "2023-09-05","CENTRAL",5,2,152,6,3,1,1
> "2023-09-06","CENTRAL",5,2,136,6,3,1,1
> "2023-09-07","CENTRAL",7,1,158,8,2,1,1
> "2023-09-08","CENTRAL",7,3,154,6,3,-1,0
> "2023-09-09","CENTRAL",".",".",169,".",".",".","."
> "2023-09-10","CENTRAL",".",".",169,".",".",".","."
> "2023-09-11","CENTRAL",".",".",155,".",".",".","."
> "2023-09-12","CENTRAL",".",".",147,".",".",".","."
>
> The magic incantation for CSV files is to set IFS to a comma like so...
>
> oldifs="${IFS}"
> IFS=","
>
> This allows parsing CSV files like so...
>
> while read
> do
>dataarray=(${REPLY})
>
> ...and, kaboom, you have a fully populated array from reading one line
> of a CSV file. The following "transformer" is my "parsehospicu" script
> that summarizes the data to "hospsum.csv". Note: I've deleted the
> leading "shebang slash bin slash bash" line because the Gentoo mailing
> list software doesn't seem to like "executable emails".
>
> I suppose I could have two different versions of the...
>
> echo "${prevdate}...
>
> ...line inside of an if/then/else/fi construct. It would depend on the
> date being inside the "missing data range".
>
> ===
>
> # Strip out missing "." that screw up the script
> sed "s/\"\.\"/0/g" region_hospital_icu_covid_data.csv >
> region_hospital_icu_covid_datax.csv dos2unix -n
> region_hospital_icu_covid_datax.csv region_hospital_icu_covid_datay.csv #
> # tail skips headers at beginning of file
> # sed deletes Row_ID, and strips out quotes
> # Output goes to file /dev/shm/temp0.txt
> tail -n +2 region_hospital_icu_covid_datay.csv | sed "s/\"//g" | sort >
> /dev/shm/temp0.txt #
> ## Set up IFS for easier parsing
> oldifs="${IFS}"
> IFS=","
> #
> # Initialize previous line's date to enter loop smoothly
> # expando to read first line
> dataline=$( head -1 /dev/shm/temp0.txt )
> dataarray=(${dataline})
> prevdate="${dataarray[0]}"
> #
> # Zero out accumulators to enter loop smoothly
> accum2=0
> accum3=0
> accum4=0
> accum5=0
> accum6=0
> accum7=0
> accum8=0
> #
> # Remove previous hospsum.csv and open a new one for writing
> rm -rf hospsum.csv
> exec 3>hospsum.csv
> #
> # Write header line to output file
> echo
> "date,icu_current_covid,icu_current_covid_vented,hospitalizations,icu_crci_
> total,icu_crci_total_vented,icu_former_covid,icu_former_covid_vented" >&3 #
> # Main loop
> # Read the data from one line in /dev/shm/temp0.txt
> while read
> do
>dataarray=(${REPLY})
>if [ "${dataarray[0]}" = "${prevdate}" ]; then
> #
> # If this line's date is same as previous line's date, add amounts to
> accumulators. accum2=$(( ${accum2} + ${dataarray[2]} ))
> accum3=$(( ${accum3} + ${dataarray[3]} ))
> accum4=$(( ${accum4} + ${dataarray[4]} ))
> accum5=$(( ${accum5} + ${dataarray[5]} ))
> accum6=$(( ${accum6} + ${dataarray[6]} ))
> accum7=$(( ${accum7} + ${dataarray[7]} ))
> accum8=$(( ${accum8} + ${dataarray[8]} ))
>else
> #
> # If this line's date has changed, output to hospsum.csv, update prevdate,
> # and update accumulators. ***IMPORTANT*** "echo" TO hospsum.csv MUST BE
> # EXECUTED BEFORE UPDATING ACCUMULATORS AND prevdate***
> echo
> "${prevdate},${accum2},${accum3},${accum4},${accum5},${accum6},${accum7},${
> accum8}" >&3 prevdate="${dataarray[0]}"
> accum2=${dataarray[2]}
> accum3=${dataarray[3]}
> accum4=${dataarray[4]}
> accum5=${dataarray[5]}
> accum6=${dataarray[6]}
> accum7=${dataarray[7]}
> accum8=${dataarray[8]}
>fi
> done #
> # Close file hospsum.csv
> exec 3<&-
> IFS="${oldifs}"
Bash can do patern substitution in variable references.
Replace accum3=$(( ${accum3} + ${dataarray[3]} ))
with accum3=$(( ${accum3} + ${dataarray[3]/'.'/0} ))
and similarly with the other lines and any array
