[gentoo-user] Re: tmp on tmpfs

[Martin Vaeth]

Andrew Savchenko  wrote:
> For similar needs I found zswap the most suitable, it's so much
> better than zram:

This sounds like one is an alternative to the other.
This is not the case. It can even make sense to use them together.
For instance, the swap device necessarily required for zswap
can be a zram device. Whether this is advantegous depends on your
usage pattern and swappiness value.

[gentoo-user] Re: tmp on tmpfs

[Kai Krakow]

Am Wed, 24 May 2017 12:30:36 -0700
schrieb Rich Freeman :

> On Wed, May 24, 2017 at 11:34 AM, Ian Zimmerman 
> wrote:
> > On 2017-05-24 08:00, Kai Krakow wrote:
> >  
> >> Unix semantics suggest that /tmp is not expected to survive reboots
> >> anyways (in contrast, /var/tmp is expected to survive reboots), so
> >> tmpfs is a logical consequence to use for /tmp.  
> >
> > /tmp is wiped by the bootmisc init job anyway.
> >  
> 
> In general I haven't found anything that is bothered by /var/tmp being
> lost on reboot, but obviously that is something you need to be
> prepared for if you put it on tmpfs.
> 
> One thing that wasn't mentioned is that having /tmp in tmpfs might
> also have security benefits depending on what is stored there, since
> it won't be written to disk.  If you have a filesystem on tmpfs and
> your swap is encrypted (which you should consider setting up since it
> is essentially "free") then /tmp also becomes a useful dumping ground
> for stuff that is decrypted for temporary processing.  For example, if
> you keep your passwords in a gpg-encrypted file you could copy it to
> /tmp, decrypt it there, do what you need to, and then delete it.  That
> wouldn't leave any recoverable traces of the file.

Interesting point... How much performance impact does encrypted swap
have? I don't mean any benchmark numbers but real life experience from
your perspective when the system experiences memory pressure?

> There are lots of guides about encrypted swap.  It is the sort of
> thing that is convenient to set up since there is no value in
> preserving a swap file across reboots, so you can just generate a
> random key on each boot.  I suspect that would break down if you're
> using hibernation / suspend to disk.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: tmp on tmpfs

[Kai Krakow]

Am Wed, 24 May 2017 11:34:20 -0700
schrieb Ian Zimmerman :

> On 2017-05-24 08:00, Kai Krakow wrote:
> 
> > While I have no benchmarks and use the systemd default of tmpfs for
> > /tmp, I also put /var/tmp/portage on tmpfs, automounted through
> > systemd so it is cleaned up when no longer used (by unmounting).
> > 
> > What can I say? It works so much faster: Building packages is a lot
> > faster most of the time, even if you'd expect gcc uses a lot of
> > memory.
> > 
> > Well, why might that be? First, tmpfs is backed by swap space, that
> > means, you need a swap partition of course. Swap is a lot simpler
> > than file systems, so swapping out unused temporary files is fast
> > and is a good thing. Also, unused memory sitting around may be
> > swapped out early. Why would you want inactive memory resident? So
> > this is also a good thing. Portage can use memory much more
> > efficient by this.
> > 
> > Applying this reasoning over to /tmp should no explain why it works
> > so well and why you may want it.
> > 
> > BTW: I also use zswap, so tmpfs sits in front of a compressed
> > write-back cache before being written out to swap compressed. This
> > should generally be much more efficient (performance-wise) than
> > putting /tmp on zram.
> > 
> > I configured tmpfs for portage to use up to 30GB of space, which is
> > almost twice the RAM I have. And it works because tmpfs is not
> > required to be resident all the time: Inactive parts will be swapped
> > out. The kernel handles this much similar to the page cache, with
> > the difference that your files aren't backed by your normal file
> > system but by swap.  And swap has a lot lower IO overhead.
> > 
> > Overall, having less IO overhead (and less head movement for portage
> > builds) is a very very efficient thing to do. GCC constantly needs
> > all sorts of files from your installation (libs for linking, header
> > files, etc), and writes a lot of transient files which are needed
> > once later and then discarded. There's no point in putting it on a
> > non-transient file system.
> > 
> > I use the following measures to get more performance out of this
> > setup:
> > 
> >   * I have three swap partitions spread across three HDDs
> >   * I have a lot of swap space (60 GB) to have space for tmpfs
> >   * I have bcache in front of my HDD filesystem
> >   * I have a relatively big SSD dedicated to bcache
> > 
> > My best recommendation is to separate swap and filesystem devices.
> > While I didn't do it that way, I still separate them through bcache
> > and thus decouple fs access and swap access although they are on the
> > same physical devices. My bcache is big enough that most accesses
> > would go to the SSD only. I enabled write-back to have that effect
> > also for write access.
> > 
> > If you cannot physically split swap from fs, a tmpfs setup for
> > portage may not be recommended (except you have a lot of memory,
> > like 16GB or above). But YMMV.
> > 
> > Still, I recommend it for /tmp, especially if your system is on
> > SSD.  
> 
> All interesting points, and you convinced me to at least give tmpfs a
> try on the desktop.
> 
> My laptop is different, though.  It doesn't have that much RAM by
> comparison (4G) and it _only_ has a SSD.  Builds have been slow :(  I
> am afraid to mess with it lest I increase the wear on the SSD.

You still may want to test /var/tmp/portage as tmpfs for small
packages... Or manually call:

# sudo PORTAGE_TMPDIR=/path/to/tmpfs emerge -1a small-package

For big packages, I suggest to nfs mount some storage from your desktop.
It probably will still be slow (maybe a little bit slower) but should
be much better for your SSD lifetime.


> > Unix semantics suggest that /tmp is not expected to survive reboots
> > anyways (in contrast, /var/tmp is expected to survive reboots), so
> > tmpfs is a logical consequence to use for /tmp.  
> 
> /tmp is wiped by the bootmisc init job anyway.

That's why such jobs exist, and why usually /tmp is wiped completely
while /var/tmp is wiped based on atime/mtime...


-- 
Regards,
Kai

Replies to list-only preferred.

Re: [gentoo-user] ...doubled updates?

[Dale]

tu...@posteo.de wrote:
> Hi,
>
> this morning I got this updates to be done:
>
 Emerging (8 of 10) sys-apps/collectl-4.1.3::gentoo
 Emerging (9 of 10) dev-libs/nss-3.30.2-r1::gentoo
 Installing (8 of 10) sys-apps/collectl-4.1.3::gentoo
 Installing (9 of 10) dev-libs/nss-3.30.2-r1::gentoo
>
> Position 8 and 9 are doubled.
> Why?
>
> Thanks for any info in advance!
> Cheers
> Meino
>
>
>
>

I put them a little closer together to make the difference more
obvious.  When I first saw it, I thought you were right.  Took a second
look to go, OH.  One is emerging, other is installing.  I guess that is
what you are talking about??

What I don't see, maybe it is outside what you posted, the installation
of package 2 or 10.  Well, did it??

Dale

:-)  :-) 

Re: [gentoo-user] ...doubled updates?

[Sam Jorna (wraeth)]

On 25/05/17 12:36, tu...@posteo.de wrote:
> Emerging (8 of 10) sys-apps/collectl-4.1.3::gentoo
> Emerging (9 of 10) dev-libs/nss-3.30.2-r1::gentoo
> Installing (8 of 10) sys-apps/collectl-4.1.3::gentoo
> Installing (9 of 10) dev-libs/nss-3.30.2-r1::gentoo

Looks like one is Emerging, the other is Installing. This is normal
behaviour.

-- 
Sam Jorna (wraeth)
GnuPG ID: D6180C26



signature.asc
Description: OpenPGP digital signature

[gentoo-user] ...doubled updates?

[tuxic]

Hi,

this morning I got this updates to be done:

>>> Verifying ebuild manifests
>>> Emerging (1 of 10) sys-apps/keyutils-1.5.10::gentoo
>>> Emerging (2 of 10) sys-apps/diffutils-3.6::gentoo
>>> Emerging (3 of 10) kde-frameworks/kcoreaddons-5.34.0-r1::gentoo
>>> Emerging (4 of 10) app-text/poppler-0.55.0::gentoo
>>> Emerging (5 of 10) dev-libs/boost-1.63.0::gentoo
>>> Emerging (6 of 10) dev-vcs/git-2.13.0::gentoo
>>> Installing (1 of 10) sys-apps/keyutils-1.5.10::gentoo
>>> Installing (3 of 10) kde-frameworks/kcoreaddons-5.34.0-r1::gentoo
>>> Installing (6 of 10) dev-vcs/git-2.13.0::gentoo
>>> Installing (4 of 10) app-text/poppler-0.55.0::gentoo
>>> Installing (5 of 10) dev-libs/boost-1.63.0::gentoo
>>> Emerging (7 of 10) media-plugins/alsa-plugins-1.1.4-r1::gentoo
>>> Emerging (8 of 10) sys-apps/collectl-4.1.3::gentoo
>>> Emerging (9 of 10) dev-libs/nss-3.30.2-r1::gentoo
>>> Emerging (10 of 10) sci-electronics/kicad-4.0.6::gentoo
>>> Installing (8 of 10) sys-apps/collectl-4.1.3::gentoo
>>> Installing (7 of 10) media-plugins/alsa-plugins-1.1.4-r1::gentoo
>>> Installing (9 of 10) dev-libs/nss-3.30.2-r1::gentoo


Position 8 and 9 are doubled.
Why?

Thanks for any info in advance!
Cheers
Meino

[gentoo-user] Re: Which update broke VMWare?

[Nikos Chantziaras]

On 05/25/2017 02:51 AM, Adam Carter wrote:
I just confirmed i need both +bundled-libs and unset 
VMWARE_USE_SHIPPED_LIBS to get it to work.


Thanks! That works indeed. You just saved half my bacon :-)

Re: [gentoo-user] General ODF error

[Dale]

the...@sys-concept.com wrote:
>
> I only get this error when try to open the file with Gnumeric.
> LibraOffice is reading the file/spreadsheet OK, though slow.
>
> When I hit OK I can Gnumeric can read the file OK.
> I went through the spreadsheet (deleted some) but still getting same
> error with Gnumeric.

Just a thought.  Does this happen with only one file or any file created
the same way?  If it's just one file, could it be that the file is
somehow corrupted?  Maybe export then import it and save it as a new
name and test? 

Just a thought.

Dale

:-)  :-) 

Re: [gentoo-user] Which update broke VMWare?

[John Covici]

On Wed, 24 May 2017 18:39:28 -0400,
Adam Carter wrote:
> 
> [1  ]
> [2  ]
> On Thu, May 25, 2017 at 4:52 AM, Nikos Chantziaras  wrote:
> 
>  VMWare Workstation stopped working on ~amd64. And I don't know which of the 
> updated packages broke it. Before I start reverting all emerges I made in the 
> last 20 days or so (it's a BIG list according to qlop), has someone already
>  figured out what broke it?
> 
>  When I say "stopped working", I mean nothing happens:
> 
>  $ vmware
>  $
> 
>  (No output. Doesn't start.)
> 
>  If I enable the "bundled-libs" USE flag, I get:
> 
>  $ vmware
>  Loop on signal 11
> 
>  $
> 
>  And nothing happens after that error message.
> 
> There's two hacks i use to make workstation run on ~amd64;
> cp /lib64/libz.so.1 to /opt/vmware/lib/vmware/lib/libz.so.1/
> 
> And 
> unset VMWARE_USE_SHIPPED_LIBS && vmware &
> 
> I am under the impression that i needed +bundled-libs with the unset 
> VMWARE_USE_SHIPPED_LIBS.but that seems counter intuitive so would be worth 
> checking.
> 

I tried this maybe 6 months ago under kernel 4.4.39 and I got no joy.
I wonder if I should try it again, or should I try to install their
bundle?

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici
 cov...@ccs.covici.com

Re: [gentoo-user] Which update broke VMWare?

[Adam Carter]

I just confirmed i need both +bundled-libs and unset
VMWARE_USE_SHIPPED_LIBS to get it to work.

Re: [gentoo-user] broadcom-sta for my wireless card

[allan gottlieb]

On Wed, May 24 2017, Neil Bothwick wrote:

> On Wed, 24 May 2017 12:38:29 -0700, Daniel Frey wrote:
>
>> You can use `eselect kernel set` to change this symlink. In my case,
>> using `eselect kernel set 2` will change it to 4.9.16. In your case,
>> list the installed kernels and set it to the new kernel and `emerge -a
>> @module-rebuild` and you'll be good to go.
>
> Or you can set the symlink USE flag for your kernel package and it will
> automatically set the symlink to the most recently installed kernel.

Thank you all.
allan

Re: [gentoo-user] Which update broke VMWare?

[Adam Carter]

>
> There's two hacks i use to make workstation run on ~amd64;
> cp /lib64/libz.so.1 to /opt/vmware/lib/vmware/lib/libz.so.1/
>
> And
> unset VMWARE_USE_SHIPPED_LIBS && vmware &
>
> I am under the impression that i needed +bundled-libs with the unset
> VMWARE_USE_SHIPPED_LIBS.but that seems counter intuitive so would be worth
> checking.
>
>
> Oh - libz hack no longer required as its been fixed in overlay
https://bugs.gentoo.org/show_bug.cgi?id=616958

Re: [gentoo-user] Re: tmp on tmpfs

[Rich Freeman]

On Wed, May 24, 2017 at 2:16 PM, Andrew Savchenko  wrote:
>
> Apparently it is pointless to encrypt swap if unencrypted
> hibernation image is used, because all memory is accessible through
> that image (and even if it is deleted later, it can be restored
> from hdd and in some cases from ssd).
>

Yeah, that was my main concern with an approach like that.  I imagine
you could use a non-random key and enter it on each boot and restore
from the encrypted swap, though I haven't actually used hibernation on
linux so I'd have to look into how to make that work.  I imagine with
an initramfs it should be possible.

-- 
Rich

Re: [gentoo-user] Which update broke VMWare?

[Adam Carter]

On Thu, May 25, 2017 at 4:52 AM, Nikos Chantziaras  wrote:

> VMWare Workstation stopped working on ~amd64. And I don't know which of
> the updated packages broke it. Before I start reverting all emerges I made
> in the last 20 days or so (it's a BIG list according to qlop), has someone
> already figured out what broke it?
>
> When I say "stopped working", I mean nothing happens:
>
>   $ vmware
>   $
>
> (No output. Doesn't start.)
>
> If I enable the "bundled-libs" USE flag, I get:
>
>   $ vmware
>   Loop on signal 11
>
>   $
>
> And nothing happens after that error message.
>

There's two hacks i use to make workstation run on ~amd64;
cp /lib64/libz.so.1 to /opt/vmware/lib/vmware/lib/libz.so.1/

And
unset VMWARE_USE_SHIPPED_LIBS && vmware &

I am under the impression that i needed +bundled-libs with the unset
VMWARE_USE_SHIPPED_LIBS.but that seems counter intuitive so would be worth
checking.

Re: [gentoo-user] Re: tmp on tmpfs

[Andrew Savchenko]

On Wed, 24 May 2017 12:30:36 -0700 Rich Freeman wrote:
> On Wed, May 24, 2017 at 11:34 AM, Ian Zimmerman  wrote:
> > On 2017-05-24 08:00, Kai Krakow wrote:
> >
> >> Unix semantics suggest that /tmp is not expected to survive reboots
> >> anyways (in contrast, /var/tmp is expected to survive reboots), so
> >> tmpfs is a logical consequence to use for /tmp.
> > 
> > /tmp is wiped by the bootmisc init job anyway.
> >
> 
> In general I haven't found anything that is bothered by /var/tmp being
> lost on reboot, but obviously that is something you need to be
> prepared for if you put it on tmpfs.
> 
> One thing that wasn't mentioned is that having /tmp in tmpfs might
> also have security benefits depending on what is stored there, since
> it won't be written to disk.  If you have a filesystem on tmpfs and
> your swap is encrypted (which you should consider setting up since it
> is essentially "free") then /tmp also becomes a useful dumping ground
> for stuff that is decrypted for temporary processing.  For example, if
> you keep your passwords in a gpg-encrypted file you could copy it to
> /tmp, decrypt it there, do what you need to, and then delete it.  That
> wouldn't leave any recoverable traces of the file.
> 
> There are lots of guides about encrypted swap.  It is the sort of
> thing that is convenient to set up since there is no value in
> preserving a swap file across reboots, so you can just generate a
> random key on each boot.  I suspect that would break down if you're
> using hibernation / suspend to disk.

It is easy to use both encrypted swap and encrypted hibernation
image (I do this on my laptop). Just before s2disk call disable swap
completely, then create empty unencrypted swap and run s2disk
(swappiness may be disabled to protect from accidental write of
unencrypted data before fresh swap creation and s2disk call).

Afterwards s2disk may be used to create encrypted memory image and
store it in the swap partition. On resume just reverse actions.

Apparently it is pointless to encrypt swap if unencrypted
hibernation image is used, because all memory is accessible through
that image (and even if it is deleted later, it can be restored
from hdd and in some cases from ssd).

Best regards,
Andrew Savchenko


pgpHK8m7gAlj4.pgp
Description: PGP signature

Re: [gentoo-user] broadcom-sta for my wireless card

[Neil Bothwick]

On Wed, 24 May 2017 12:38:29 -0700, Daniel Frey wrote:

> You can use `eselect kernel set` to change this symlink. In my case,
> using `eselect kernel set 2` will change it to 4.9.16. In your case,
> list the installed kernels and set it to the new kernel and `emerge -a
> @module-rebuild` and you'll be good to go.

Or you can set the symlink USE flag for your kernel package and it will
automatically set the symlink to the most recently installed kernel.


-- 
Neil Bothwick

The careful application of terror is also a form of communication.


pgp6Og6L4ZK3f.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] broadcom-sta for my wireless card

[Daniel Frey]

On 05/24/2017 11:46 AM, allan gottlieb wrote:
> On Wed, May 24 2017, Daniel Frey wrote:
> 
>> On 05/24/2017 11:10 AM, allan gottlieb wrote:
>>> My older laptop needs broadcom-sta.  Back when I bought and setup the
>>> laptop (kernel 3.18.12), I emerged broadcom-sta and wireless worked.
>>>
>>> Now I am upgrading to kernel 4.9.16.  The kernel boots but no wireless.
>>> In /lib/modules/3.18.12-gentoo-3 I have net/wireless/wl.ko.
>>> In /lib/modules/4.9.16-gentoo-3 there is no net.
>>>
>>> Am I supposed to remerge broadcom-sta?  If so how do I indicate the
>>> kernel version for /lib/modules?
>>>Do I set the the /usr/src/linux symlink?
>>>Do I run 4.9.16 during the emerge?
>>>Something else?
>>>
>>> thanks,
>>> allan
>>>
>>
>> Sounds like it installs kernel modules, so yes, after a new kernel build
>> you need to re-emerge all packages that install modules. Easiest way to
>> emerge all packages that install kernel modules is:
>>
>> `emerge -a @module-rebuild`
>>
>> Dan
> 
> Thank you dan, but how do I tell the build which /lib/modules/* to use.
> I gave two guesses above.
> 
> allan
> 

Use `eselect kernel list` : it will show which /lib/modules it will go
to (as an example here's mine):

$ eselect kernel list
Available kernel symlink targets:
  [1]   linux-4.1.37-gentoo *
  [2]   linux-4.9.16-gentoo

If I merge a package with kernel modules, it will currently go to
lib/modules/4.1.37-gentoo, which is what my kernel symlink is set to.
You can use `eselect kernel set` to change this symlink. In my case,
using `eselect kernel set 2` will change it to 4.9.16. In your case,
list the installed kernels and set it to the new kernel and `emerge -a
@module-rebuild` and you'll be good to go.

Dan

Re: [gentoo-user] General ODF error

[R0b0t1]

On Wed, May 24, 2017 at 12:50 PM,   wrote:
>
> On 05/21/2017 05:07 PM, R0b0t1 wrote:
>> On Sat, May 20, 2017 at 4:37 PM,   wrote:
>>> I've a spreadsheet file (ODS) with several sheets in it but when I try
>>> to open it, it take long time to load
>>>
>>> When I try to open it with Gnumeric I get a warning:
>>>
>>> General ODF error
>>> Unable to parse
>>> '((#REF!;#REF!))'
>>> ('All entries in the set must be references')
>>> Unable to parse
>>> '(((#REF!;#REF!);#REF!'
>>> ('All entries in the set must be references')
>>> Unable to parse
>>> '(((#REF!;#REF!);#REF!'
>>> ('All entries in the set must be references')
>>>
>>> I think it must be due to these references.
>>> How to locate/look in spreadsheets for these references
>>>
>>> --
>>> Thelma
>>
>> The ODF formats are XML-based (or can be exported to XML), and should
>> be human readable, though it will not be fun. See
>> https://help.libreoffice.org/Common/XML_File_Formats.
>>
>> It looks like the strings which do not parse are literally what you
>> have pasted into your message - why are there so many mismatched close
>> parenthesis? #REF means the given cell [range] can't be resolved. Did
>> the file become truncated at some point?
>
> I only get this error when try to open the file with Gnumeric.
> LibraOffice is reading the file/spreadsheet OK, though slow.
>
> When I hit OK I can Gnumeric can read the file OK.
> I went through the spreadsheet (deleted some) but still getting same
> error with Gnumeric.

Unfortunately I don't know much about the internals of either so my
recommendation is to use LibreOffice Calc as long as you can, and try
recreating the file from scratch.

Re: [gentoo-user] Re: tmp on tmpfs

[Rich Freeman]

On Wed, May 24, 2017 at 11:34 AM, Ian Zimmerman  wrote:
> On 2017-05-24 08:00, Kai Krakow wrote:
>
>> Unix semantics suggest that /tmp is not expected to survive reboots
>> anyways (in contrast, /var/tmp is expected to survive reboots), so
>> tmpfs is a logical consequence to use for /tmp.
>
> /tmp is wiped by the bootmisc init job anyway.
>

In general I haven't found anything that is bothered by /var/tmp being
lost on reboot, but obviously that is something you need to be
prepared for if you put it on tmpfs.

One thing that wasn't mentioned is that having /tmp in tmpfs might
also have security benefits depending on what is stored there, since
it won't be written to disk.  If you have a filesystem on tmpfs and
your swap is encrypted (which you should consider setting up since it
is essentially "free") then /tmp also becomes a useful dumping ground
for stuff that is decrypted for temporary processing.  For example, if
you keep your passwords in a gpg-encrypted file you could copy it to
/tmp, decrypt it there, do what you need to, and then delete it.  That
wouldn't leave any recoverable traces of the file.

There are lots of guides about encrypted swap.  It is the sort of
thing that is convenient to set up since there is no value in
preserving a swap file across reboots, so you can just generate a
random key on each boot.  I suspect that would break down if you're
using hibernation / suspend to disk.

-- 
Rich

Re: [gentoo-user] broadcom-sta for my wireless card

[Alan McKinnon]

On 24/05/2017 20:46, allan gottlieb wrote:
> On Wed, May 24 2017, Daniel Frey wrote:
> 
>> On 05/24/2017 11:10 AM, allan gottlieb wrote:
>>> My older laptop needs broadcom-sta.  Back when I bought and setup the
>>> laptop (kernel 3.18.12), I emerged broadcom-sta and wireless worked.
>>>
>>> Now I am upgrading to kernel 4.9.16.  The kernel boots but no wireless.
>>> In /lib/modules/3.18.12-gentoo-3 I have net/wireless/wl.ko.
>>> In /lib/modules/4.9.16-gentoo-3 there is no net.
>>>
>>> Am I supposed to remerge broadcom-sta?  If so how do I indicate the
>>> kernel version for /lib/modules?
>>>Do I set the the /usr/src/linux symlink?
>>>Do I run 4.9.16 during the emerge?
>>>Something else?
>>>
>>> thanks,
>>> allan
>>>
>>
>> Sounds like it installs kernel modules, so yes, after a new kernel build
>> you need to re-emerge all packages that install modules. Easiest way to
>> emerge all packages that install kernel modules is:
>>
>> `emerge -a @module-rebuild`
>>
>> Dan
> 
> Thank you dan, but how do I tell the build which /lib/modules/* to use.
> I gave two guesses above.
> 
> allan
> 


it installs according to the /usr/src/linux symlink



-- 
Alan McKinnon
alan.mckin...@gmail.com

[gentoo-user] Which update broke VMWare?

[Nikos Chantziaras]

VMWare Workstation stopped working on ~amd64. And I don't know which of 
the updated packages broke it. Before I start reverting all emerges I 
made in the last 20 days or so (it's a BIG list according to qlop), has 
someone already figured out what broke it?


When I say "stopped working", I mean nothing happens:

  $ vmware
  $

(No output. Doesn't start.)

If I enable the "bundled-libs" USE flag, I get:

  $ vmware
  Loop on signal 11

  $

And nothing happens after that error message.

Re: [gentoo-user] broadcom-sta for my wireless card

[allan gottlieb]

On Wed, May 24 2017, Daniel Frey wrote:

> On 05/24/2017 11:10 AM, allan gottlieb wrote:
>> My older laptop needs broadcom-sta.  Back when I bought and setup the
>> laptop (kernel 3.18.12), I emerged broadcom-sta and wireless worked.
>> 
>> Now I am upgrading to kernel 4.9.16.  The kernel boots but no wireless.
>> In /lib/modules/3.18.12-gentoo-3 I have net/wireless/wl.ko.
>> In /lib/modules/4.9.16-gentoo-3 there is no net.
>> 
>> Am I supposed to remerge broadcom-sta?  If so how do I indicate the
>> kernel version for /lib/modules?
>>Do I set the the /usr/src/linux symlink?
>>Do I run 4.9.16 during the emerge?
>>Something else?
>> 
>> thanks,
>> allan
>> 
>
> Sounds like it installs kernel modules, so yes, after a new kernel build
> you need to re-emerge all packages that install modules. Easiest way to
> emerge all packages that install kernel modules is:
>
> `emerge -a @module-rebuild`
>
> Dan

Thank you dan, but how do I tell the build which /lib/modules/* to use.
I gave two guesses above.

allan

[gentoo-user] Re: tmp on tmpfs

[Nikos Chantziaras]

On 05/24/2017 08:16 AM, Ian Zimmerman wrote:

So what are gentoo users' opinions on this matter of faith?

I have long been in the camp that thinks tmpfs for /tmp has no
advantages (and may have disadvantages) over a normal filesystem like
ext3, because the files there are normally so small that they will stay
in the page cache 100% of the time.

But I see that tmpfs is the default with systemd.  Surely they have a
good reason for this? :)


Their reason is described here:

  https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems

It seems that they consider it an important *default* to have /tmp exist 
even if nothing else exists yet during boot-up.


Normally I wouldn't care too much whether /tmp is tmpfs or not. The only 
cases where I do care, is when unpacking a huge tarball with contents I 
didn't intend to keep around. But I stopped doing that in /tmp anyway. I 
have my own ~/tmp for that now. When using /tmp for that, you need to rm 
-rf what you don't need anymore, since it eats up RAM.


Another case is when I download something big that I intend to install 
(*.bin installers) or unpack into a final location on disk. In those 
cases, /tmp on tmpfs helps since it lowers disk fragmentation: you 
download it to RAM, then install to disk.

[gentoo-user] Re: tmp on tmpfs

[Ian Zimmerman]

On 2017-05-24 08:00, Kai Krakow wrote:

> While I have no benchmarks and use the systemd default of tmpfs for
> /tmp, I also put /var/tmp/portage on tmpfs, automounted through
> systemd so it is cleaned up when no longer used (by unmounting).
> 
> What can I say? It works so much faster: Building packages is a lot
> faster most of the time, even if you'd expect gcc uses a lot of
> memory.
> 
> Well, why might that be? First, tmpfs is backed by swap space, that
> means, you need a swap partition of course. Swap is a lot simpler than
> file systems, so swapping out unused temporary files is fast and is a
> good thing. Also, unused memory sitting around may be swapped out
> early. Why would you want inactive memory resident? So this is also a
> good thing. Portage can use memory much more efficient by this.
> 
> Applying this reasoning over to /tmp should no explain why it works so
> well and why you may want it.
> 
> BTW: I also use zswap, so tmpfs sits in front of a compressed
> write-back cache before being written out to swap compressed. This
> should generally be much more efficient (performance-wise) than
> putting /tmp on zram.
> 
> I configured tmpfs for portage to use up to 30GB of space, which is
> almost twice the RAM I have. And it works because tmpfs is not
> required to be resident all the time: Inactive parts will be swapped
> out. The kernel handles this much similar to the page cache, with the
> difference that your files aren't backed by your normal file system
> but by swap.  And swap has a lot lower IO overhead.
> 
> Overall, having less IO overhead (and less head movement for portage
> builds) is a very very efficient thing to do. GCC constantly needs all
> sorts of files from your installation (libs for linking, header files,
> etc), and writes a lot of transient files which are needed once later
> and then discarded. There's no point in putting it on a non-transient
> file system.
> 
> I use the following measures to get more performance out of this
> setup:
> 
>   * I have three swap partitions spread across three HDDs
>   * I have a lot of swap space (60 GB) to have space for tmpfs
>   * I have bcache in front of my HDD filesystem
>   * I have a relatively big SSD dedicated to bcache
> 
> My best recommendation is to separate swap and filesystem devices.
> While I didn't do it that way, I still separate them through bcache
> and thus decouple fs access and swap access although they are on the
> same physical devices. My bcache is big enough that most accesses
> would go to the SSD only. I enabled write-back to have that effect
> also for write access.
> 
> If you cannot physically split swap from fs, a tmpfs setup for portage
> may not be recommended (except you have a lot of memory, like 16GB or
> above). But YMMV.
> 
> Still, I recommend it for /tmp, especially if your system is on SSD.

All interesting points, and you convinced me to at least give tmpfs a
try on the desktop.

My laptop is different, though.  It doesn't have that much RAM by
comparison (4G) and it _only_ has a SSD.  Builds have been slow :(  I am
afraid to mess with it lest I increase the wear on the SSD.

> Unix semantics suggest that /tmp is not expected to survive reboots
> anyways (in contrast, /var/tmp is expected to survive reboots), so
> tmpfs is a logical consequence to use for /tmp.

/tmp is wiped by the bootmisc init job anyway.

-- 
Please *no* private Cc: on mailing lists and newsgroups
Personal signed mail: please _encrypt_ and sign
Don't clear-text sign:
http://primate.net/~itz/blog/the-problem-with-gpg-signatures.html

Re: [gentoo-user] broadcom-sta for my wireless card

[Daniel Frey]

On 05/24/2017 11:10 AM, allan gottlieb wrote:
> My older laptop needs broadcom-sta.  Back when I bought and setup the
> laptop (kernel 3.18.12), I emerged broadcom-sta and wireless worked.
> 
> Now I am upgrading to kernel 4.9.16.  The kernel boots but no wireless.
> In /lib/modules/3.18.12-gentoo-3 I have net/wireless/wl.ko.
> In /lib/modules/4.9.16-gentoo-3 there is no net.
> 
> Am I supposed to remerge broadcom-sta?  If so how do I indicate the
> kernel version for /lib/modules?
>Do I set the the /usr/src/linux symlink?
>Do I run 4.9.16 during the emerge?
>Something else?
> 
> thanks,
> allan
> 

Sounds like it installs kernel modules, so yes, after a new kernel build
you need to re-emerge all packages that install modules. Easiest way to
emerge all packages that install kernel modules is:

`emerge -a @module-rebuild`

Dan

[gentoo-user] broadcom-sta for my wireless card

[allan gottlieb]

My older laptop needs broadcom-sta.  Back when I bought and setup the
laptop (kernel 3.18.12), I emerged broadcom-sta and wireless worked.

Now I am upgrading to kernel 4.9.16.  The kernel boots but no wireless.
In /lib/modules/3.18.12-gentoo-3 I have net/wireless/wl.ko.
In /lib/modules/4.9.16-gentoo-3 there is no net.

Am I supposed to remerge broadcom-sta?  If so how do I indicate the
kernel version for /lib/modules?
   Do I set the the /usr/src/linux symlink?
   Do I run 4.9.16 during the emerge?
   Something else?

thanks,
allan

Re: [gentoo-user] General ODF error

[thelma]

On 05/21/2017 05:07 PM, R0b0t1 wrote:
> On Sat, May 20, 2017 at 4:37 PM,   wrote:
>> I've a spreadsheet file (ODS) with several sheets in it but when I try
>> to open it, it take long time to load
>>
>> When I try to open it with Gnumeric I get a warning:
>>
>> General ODF error
>> Unable to parse
>> '((#REF!;#REF!))'
>> ('All entries in the set must be references')
>> Unable to parse
>> '(((#REF!;#REF!);#REF!'
>> ('All entries in the set must be references')
>> Unable to parse
>> '(((#REF!;#REF!);#REF!'
>> ('All entries in the set must be references')
>>
>> I think it must be due to these references.
>> How to locate/look in spreadsheets for these references
>>
>> --
>> Thelma
> 
> The ODF formats are XML-based (or can be exported to XML), and should
> be human readable, though it will not be fun. See
> https://help.libreoffice.org/Common/XML_File_Formats.
> 
> It looks like the strings which do not parse are literally what you
> have pasted into your message - why are there so many mismatched close
> parenthesis? #REF means the given cell [range] can't be resolved. Did
> the file become truncated at some point?

I only get this error when try to open the file with Gnumeric.
LibraOffice is reading the file/spreadsheet OK, though slow.

When I hit OK I can Gnumeric can read the file OK.
I went through the spreadsheet (deleted some) but still getting same
error with Gnumeric.

[gentoo-user] Re: tmp on tmpfs

[Kai Krakow]

Am Wed, 24 May 2017 08:00:33 +0200
schrieb Kai Krakow :

> Am Wed, 24 May 2017 07:34:34 +0200
> schrieb gentoo-u...@c-14.de:
> 
> > On 17-05-23 at 22:16, Ian Zimmerman wrote:  
> > > So what are gentoo users' opinions on this matter of faith?
> > I use an ext4 partition backed by zram. Gives me ~3x compression on
> > the things I normally have lying around there (plain text files) and
> > ensures that anything I throw there (or programs throw there) gets
> > cleaned up on reboot.
> >   
> > > I have long been in the camp that thinks tmpfs for /tmp has no
> > > advantages (and may have disadvantages) over a normal filesystem
> > > like ext3, because the files there are normally so small that they
> > > will stay in the page cache 100% of the time.
> > I've never actually benchmarked this. Most of the things I notice
> > that tend to end up there are temporary build files generated during
> > configure stages or temporary log files used by various programs
> > (clang static analyzer). Even if the entire file stays in the page
> > cache, it'll still generate IO overhead and extra seeks that might
> > slow down the rest of your system (unless your /tmp is on a
> > different hard drive) which on spinning rust will cause slowdowns
> > while on an ssd it'll eat away at your writes (which you may or may
> > not have to worry about).
> >   
> > > But I see that tmpfs is the default with systemd.  Surely they
> > > have a good reason for this? :)
> > Or someone decided they liked the idea and made it the default and
> > nobody ever complained (or if they did were told to just change it
> > on their system). 
> > 
> > Either way, it'd be nice if someone actually benchmarked this.  
> 
> While I have no benchmarks and use the systemd default of tmpfs
> for /tmp, I also put /var/tmp/portage on tmpfs, automounted through
> systemd so it is cleaned up when no longer used (by unmounting).
> 
> What can I say? It works so much faster: Building packages is a lot
> faster most of the time, even if you'd expect gcc uses a lot of
> memory.
> 
> Well, why might that be? First, tmpfs is backed by swap space, that
> means, you need a swap partition of course.

To get in line with Rich Freeman: I didn't want to imply that zswap
only works with swap, neither that tmpfs only works with swap. Both
work without. But if you want to put some serious amount of data into
tmpfs, you need swap as a backing device sooner or later.

> Swap is a lot simpler than
> file systems, so swapping out unused temporary files is fast and is a
> good thing. Also, unused memory sitting around may be swapped out
> early. Why would you want inactive memory resident? So this is also a
> good thing. Portage can use memory much more efficient by this.
> 
> Applying this reasoning over to /tmp should no explain why it works so
> well and why you may want it.
> 
> BTW: I also use zswap, so tmpfs sits in front of a compressed
> write-back cache before being written out to swap compressed. This
> should generally be much more efficient (performance-wise) than
> putting /tmp on zram.
> 
> I configured tmpfs for portage to use up to 30GB of space, which is
> almost twice the RAM I have. And it works because tmpfs is not
> required to be resident all the time: Inactive parts will be swapped
> out. The kernel handles this much similar to the page cache, with the
> difference that your files aren't backed by your normal file system
> but by swap. And swap has a lot lower IO overhead.
> 
> Overall, having less IO overhead (and less head movement for portage
> builds) is a very very efficient thing to do. GCC constantly needs all
> sorts of files from your installation (libs for linking, header files,
> etc), and writes a lot of transient files which are needed once later
> and then discarded. There's no point in putting it on a non-transient
> file system.
> 
> I use the following measures to get more performance out of this
> setup:
> 
>   * I have three swap partitions spread across three HDDs
>   * I have a lot of swap space (60 GB) to have space for tmpfs
>   * I have bcache in front of my HDD filesystem
>   * I have a relatively big SSD dedicated to bcache
> 
> My best recommendation is to separate swap and filesystem devices.
> While I didn't do it that way, I still separate them through bcache
> and thus decouple fs access and swap access although they are on the
> same physical devices. My bcache is big enough that most accesses
> would go to the SSD only. I enabled write-back to have that effect
> also for write access.
> 
> If you cannot physically split swap from fs, a tmpfs setup for portage
> may not be recommended (except you have a lot of memory, like 16GB or
> above). But YMMV.
> 
> Still, I recommend it for /tmp, especially if your system is on SSD.
> Unix semantics suggest that /tmp is not expected to survive reboots
> anyways (in contrast, /var/tmp is expected to survive reboots), so
> tmpfs is a logical consequence 

Re: [gentoo-user] tmp on tmpfs

[R0b0t1]

On Wed, May 24, 2017 at 12:16 AM, Ian Zimmerman  wrote:
> So what are gentoo users' opinions on this matter of faith?
>

On Wed, May 24, 2017 at 12:34 AM,   wrote:
> Either way, it'd be nice if someone actually benchmarked this.
>

I don't have exhaustive benchmarks but moving PORTAGE_TMPDIR to a
tmpfs makes builds at least an order of magnitude faster. For general
usage with /tmp you may or may not notice, but the lack of normal IO
overhead can only make it faster.

Re: [gentoo-user] etiquette on bgo

[allan gottlieb]

On Wed, May 24 2017, allan gottlieb wrote:

> I am an emacs user patiently awaiting the stabilization of version 25.1
> on x86.  I understand that I can install the testing version.
>
> On 10 feb  Ulrich Müller (gentoo-dev) wrote in bug 608192 (a
> stabilization bug)
>Arch teams, please proceed
>
> On 23 feb  arm ppc ppc64 went stable
> On 
>
> Since then nothing 

Please ignore I didn't mean to send.
Sorry for the noise.
allan

[gentoo-user] etiquette on bgo

[allan gottlieb]

I am an emacs user patiently awaiting the stabilization of version 25.1
on x86.  I understand that I can install the testing version.

On 10 feb  Ulrich Müller (gentoo-dev) wrote in bug 608192 (a
stabilization bug)
   Arch teams, please proceed

On 23 feb  arm ppc ppc64 went stable
On 

Since then nothing 

Re: [gentoo-user] tmp on tmpfs

[Andrew Savchenko]

Hi,

On Wed, 24 May 2017 05:34:09 -0400 Rich Freeman wrote:
> On Wed, May 24, 2017 at 1:16 AM, Ian Zimmerman  wrote:
> >
> > I have long been in the camp that thinks tmpfs for /tmp has no
> > advantages (and may have disadvantages) over a normal filesystem like
> > ext3, because the files there are normally so small that they will stay
> > in the page cache 100% of the time.
> >
> 
> The file being in the page cache only speeds up reads of the file.  On
> a conventional filesystem the file will still be forced to be
> committed to disk within 30 seconds, or whatever you've set your max
> writeback delay to.  That means guaranteed disk write IO.  If the
> drive is mostly idle it will have no impact on performance, but if the
> disk is fairly busy then it will, especially for spinning disks.  For
> an SSD /tmp would be a source of erase cycles (which also have
> performance implications, but there it is more of a wear issue).  When
> the file is removed that would also generate write IO.
> 
> The flip side is that on most systems /tmp probably doesn't get THAT much IO.
> 
> On Gentoo doing your builds in tmpfs definitely has a large
> performance impact, because there are a lot of files created during
> the build process that are sizable but which don't end up getting
> installed (object files mostly).  Plus you have the extraction of the
> source itself.  For a typical build that is many MB of data being
> extracted and then deleted after maybe a minute, which is a lot of
> useless IO, especially when the actual install is probably creating a
> fairly sizable IO queue on its own.
> 
> To avoid a reply, I'll also note that tmpfs does NOT require swap to
> work.  It does of course require plenty of memory, and as with any
> situation where lots of memory is required swap may be useful, but it
> is not a requirement.
> 
> Others have mentioned zram.  I've used it, but unless something has
> changed one of its limitations is that it can't give up memory.  That
> is less of an issue if you're using swap since it can be swapped out
> if idle.  However, if you're not using swap then you're potentially
> giving up a chunk of RAM to do it, though less RAM than a tmpfs if it
> is full most of the time (which I doubt is typically the case).
 
For similar needs I found zswap the most suitable, it's so much
better than zram:

- smaller CPU overhead: not every i/o is being compressed, e.g. if
there is sill enough RAM available it is used without compression
overhead as usual, but if memory is not enough, swapped out pages
are being compressed instead of swapping out to disk;

- no size limitation: if zswap pool is full, data is being pulled
to swap, the same happens with non-compressible pages;

- pool size and compression type can be dynamically adjusted, I
prefer z3fold.

So I have normal tmpfs on /tmp (and /var/tmp on hosts with lots or
RAM), but both tmpfs and running daemons/apps can benefit from
compressed memory for rarely used pages while enjoing full RAM
speed for frequently accessed ones.

Best regards,
Andrew Savchenko


pgpr8SXNUHm1t.pgp
Description: PGP signature

Re: [gentoo-user] tmp on tmpfs

[Rich Freeman]

On Wed, May 24, 2017 at 5:43 AM,   wrote:
> On 17-05-24 at 05:34, Rich Freeman wrote:
> [..]
>> Others have mentioned zram.  I've used it, but unless something has
>> changed one of its limitations is that it can't give up memory.  That
>> is less of an issue if you're using swap since it can be swapped out
>> if idle.  However, if you're not using swap then you're potentially
>> giving up a chunk of RAM to do it, though less RAM than a tmpfs if it
>> is full most of the time (which I doubt is typically the case).
> Seems to work fine here (with kernels newer than the late 3.x when I started 
> using zram):
>

Thanks.  Useful to know.  Perhaps something changed.  Then again, I
don't think I actually tested it so it could have also been
misinformation somewhere.

-- 
Rich

Re: [gentoo-user] tmp on tmpfs

[gentoo-user]

On 17-05-24 at 05:34, Rich Freeman wrote:
[..]
> Others have mentioned zram.  I've used it, but unless something has
> changed one of its limitations is that it can't give up memory.  That
> is less of an issue if you're using swap since it can be swapped out
> if idle.  However, if you're not using swap then you're potentially
> giving up a chunk of RAM to do it, though less RAM than a tmpfs if it
> is full most of the time (which I doubt is typically the case).
Seems to work fine here (with kernels newer than the late 3.x when I started 
using zram):

radiocarbon:~% dd if=/dev/urandom of=/tmp/foo
^C3405370+0 records in
3405370+0 records out
1743549440 bytes (1.7 GB, 1.6 GiB) copied, 10.8268 s, 161 MB/s
dd if=/dev/urandom of=/tmp/foo  8 MiB 10.853 (user: 0.339, kernel: 10.482)
radiocarbon:~% zramctl
NAME   ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lz4 2G4K   64B4K   8 [SWAP]
/dev/zram1 lz4 3G  1.6G  1.6G  1.6G   8 /tmp
radiocarbon:~% free -m
  totalusedfree  shared  buff/cache   available
Mem:   79203096  61 22847634487
Swap:  2047   02047
radiocarbon:~% rm /tmp/foo
radiocarbon:~% zramctl
NAME   ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lz4 2G4K   64B4K   8 [SWAP]
/dev/zram1 lz4 3G  3.9M1M  1.3M   8 /tmp
radiocarbon:~% free -m
  totalusedfree  shared  buff/cache   available
Mem:   792014123458 22930496171
Swap:  2047   02047

-- 
Simon Thelen

Re: [gentoo-user] tmp on tmpfs

[Rich Freeman]

On Wed, May 24, 2017 at 1:16 AM, Ian Zimmerman  wrote:
>
> I have long been in the camp that thinks tmpfs for /tmp has no
> advantages (and may have disadvantages) over a normal filesystem like
> ext3, because the files there are normally so small that they will stay
> in the page cache 100% of the time.
>

The file being in the page cache only speeds up reads of the file.  On
a conventional filesystem the file will still be forced to be
committed to disk within 30 seconds, or whatever you've set your max
writeback delay to.  That means guaranteed disk write IO.  If the
drive is mostly idle it will have no impact on performance, but if the
disk is fairly busy then it will, especially for spinning disks.  For
an SSD /tmp would be a source of erase cycles (which also have
performance implications, but there it is more of a wear issue).  When
the file is removed that would also generate write IO.

The flip side is that on most systems /tmp probably doesn't get THAT much IO.

On Gentoo doing your builds in tmpfs definitely has a large
performance impact, because there are a lot of files created during
the build process that are sizable but which don't end up getting
installed (object files mostly).  Plus you have the extraction of the
source itself.  For a typical build that is many MB of data being
extracted and then deleted after maybe a minute, which is a lot of
useless IO, especially when the actual install is probably creating a
fairly sizable IO queue on its own.

To avoid a reply, I'll also note that tmpfs does NOT require swap to
work.  It does of course require plenty of memory, and as with any
situation where lots of memory is required swap may be useful, but it
is not a requirement.

Others have mentioned zram.  I've used it, but unless something has
changed one of its limitations is that it can't give up memory.  That
is less of an issue if you're using swap since it can be swapped out
if idle.  However, if you're not using swap then you're potentially
giving up a chunk of RAM to do it, though less RAM than a tmpfs if it
is full most of the time (which I doubt is typically the case).

-- 
Rich

Re: [gentoo-user] Re: Re: Qt-4.8.7 bug

[Peter Humphrey]

On Tuesday 23 May 2017 23:16:48 Frank Steinmetzger wrote:
> On Mon, May 22, 2017 at 09:49:01AM +0200, Jörg Schaible wrote:
> > Peter Humphrey wrote:
> > 
> > [snip]
> > 
> > well, this does not seem to be the complete truth. When I switched to
> > gcc
> > 5.x I did a revdep-rebuild for anything that was compiled against
> > libstdc++.so.6 just like the according news entry was recommending. And
> > I am quite sure that those Qt plugins were part of my 515 recompiled
> > packages.
> > 
> > Nevertheless, my KDE 4 apps were broken after the update to Qt 4.8.7.
> > Rebuilding anything that was using libQtCore.so.4 solved it, but I fail
> > to see how this is related to the gcc update two weeks ago.
> 
> I, too, was affected by this. I did the libstdc++ rebuild after upgrading
> gcc (some 550 packages) a while back and now I was hit by the Qt problem,
> so another rebuild of 500 packages with --changed-deps world.
> 
> 
> Once finished, it left me with a new problem:
> KDE doesn’t find my beloved terminus font anymore, both on my PC and my
> laptop. It does not show up in any font selection dialog. The same goes
> for GTK applications such as gimp (GTK2) and firefox (GTK3). No Terminus
> anywhere.
> 
> Does that ring a bell with anyone?

Not with me, no, but on looking at System Settings to see, I found all the 
icons missing. And the selected single-click-to-open setting was ignored - 
not everywhere, just in System Settings.

Remerging kde-plasma/systemsettings-5.8.6 hasn't helped.

-- 
Regards
Peter

Re: [gentoo-user] Re: Qt-4.8.7 bug

[Frank Steinmetzger]

On Tue, May 23, 2017 at 09:59:20PM -0700, Ian Zimmerman wrote:
> On 2017-05-23 23:16, Frank Steinmetzger wrote:
>
> > KDE doesn’t find my beloved terminus font anymore, both on my PC and
> > my laptop.  It does not show up in any font selection dialog. The same
> > goes for GTK applications such as gimp (GTK2) and firefox (GTK3). No
> > Terminus anywhere.
> >
> > Does that ring a bell with anyone?
>
> Yes:
>
> https://bugs.gentoo.org/show_bug.cgi?id=618918
>
> I seem to scent a faint Poettering type hormone in the air around
> freetype :-(

Ah, thank you very very much. I’ll be offline for the next few days and it’s
nice to know a solution. Coulda thunk myself to look at bgo.

-- 
Gruß | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

The boss is a human just like everyone else, he just doesn’t know.


signature.asc
Description: Digital signature

Re: [gentoo-user] tmp on tmpfs

[Andrew Tselischev]

On Tue, May 23, 2017 at 10:16:56PM -0700, Ian Zimmerman wrote:
> So what are gentoo users' opinions on this matter of faith?
> 
> I have long been in the camp that thinks tmpfs for /tmp has no
> advantages (and may have disadvantages) over a normal filesystem like
> ext3, because the files there are normally so small that they will stay
> in the page cache 100% of the time.
> 
> But I see that tmpfs is the default with systemd.  Surely they have a
> good reason for this? :)

for most purposes, it avoids thrashing your storage media with useless i/o.
if your purposes are unusual, by all means change it back.

[gentoo-user] Re: tmp on tmpfs

[Kai Krakow]

Am Wed, 24 May 2017 07:34:34 +0200
schrieb gentoo-u...@c-14.de:

> On 17-05-23 at 22:16, Ian Zimmerman wrote:
> > So what are gentoo users' opinions on this matter of faith?  
> I use an ext4 partition backed by zram. Gives me ~3x compression on
> the things I normally have lying around there (plain text files) and
> ensures that anything I throw there (or programs throw there) gets
> cleaned up on reboot.
> 
> > I have long been in the camp that thinks tmpfs for /tmp has no
> > advantages (and may have disadvantages) over a normal filesystem
> > like ext3, because the files there are normally so small that they
> > will stay in the page cache 100% of the time.  
> I've never actually benchmarked this. Most of the things I notice that
> tend to end up there are temporary build files generated during
> configure stages or temporary log files used by various programs
> (clang static analyzer). Even if the entire file stays in the page
> cache, it'll still generate IO overhead and extra seeks that might
> slow down the rest of your system (unless your /tmp is on a different
> hard drive) which on spinning rust will cause slowdowns while on an
> ssd it'll eat away at your writes (which you may or may not have to
> worry about).
> 
> > But I see that tmpfs is the default with systemd.  Surely they have
> > a good reason for this? :)  
> Or someone decided they liked the idea and made it the default and
> nobody ever complained (or if they did were told to just change it on
> their system). 
> 
> Either way, it'd be nice if someone actually benchmarked this.

While I have no benchmarks and use the systemd default of tmpfs
for /tmp, I also put /var/tmp/portage on tmpfs, automounted through
systemd so it is cleaned up when no longer used (by unmounting).

What can I say? It works so much faster: Building packages is a lot
faster most of the time, even if you'd expect gcc uses a lot of memory.

Well, why might that be? First, tmpfs is backed by swap space, that
means, you need a swap partition of course. Swap is a lot simpler than
file systems, so swapping out unused temporary files is fast and is a
good thing. Also, unused memory sitting around may be swapped out
early. Why would you want inactive memory resident? So this is also a
good thing. Portage can use memory much more efficient by this.

Applying this reasoning over to /tmp should no explain why it works so
well and why you may want it.

BTW: I also use zswap, so tmpfs sits in front of a compressed
write-back cache before being written out to swap compressed. This
should generally be much more efficient (performance-wise) than putting
/tmp on zram.

I configured tmpfs for portage to use up to 30GB of space, which is
almost twice the RAM I have. And it works because tmpfs is not required
to be resident all the time: Inactive parts will be swapped out. The
kernel handles this much similar to the page cache, with the difference
that your files aren't backed by your normal file system but by swap.
And swap has a lot lower IO overhead.

Overall, having less IO overhead (and less head movement for portage
builds) is a very very efficient thing to do. GCC constantly needs all
sorts of files from your installation (libs for linking, header files,
etc), and writes a lot of transient files which are needed once later
and then discarded. There's no point in putting it on a non-transient
file system.

I use the following measures to get more performance out of this setup:

  * I have three swap partitions spread across three HDDs
  * I have a lot of swap space (60 GB) to have space for tmpfs
  * I have bcache in front of my HDD filesystem
  * I have a relatively big SSD dedicated to bcache

My best recommendation is to separate swap and filesystem devices.
While I didn't do it that way, I still separate them through bcache
and thus decouple fs access and swap access although they are on the
same physical devices. My bcache is big enough that most accesses would
go to the SSD only. I enabled write-back to have that effect also for
write access.

If you cannot physically split swap from fs, a tmpfs setup for portage
may not be recommended (except you have a lot of memory, like 16GB or
above). But YMMV.

Still, I recommend it for /tmp, especially if your system is on SSD.
Unix semantics suggest that /tmp is not expected to survive reboots
anyways (in contrast, /var/tmp is expected to survive reboots), so
tmpfs is a logical consequence to use for /tmp.


-- 
Regards,
Kai

Replies to list-only preferred.