Re: [gentoo-user] New computer and Gentoo
On Thursday 21 Jul 2011 03:29:17 Adam Carter wrote: amd64 means any x86 64bit platform, so Intel too. march=native is good if you're not using distcc, or if you're only using distcc on core2 boxes. Otherwise be specific. I recommend using the 64 bit profile (amd64) for = GCC 4.3 which shows - march=core2. This is what I use here with multilib and had no problems. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] New computer and Gentoo
On Thu, 2011-07-21 at 06:26 +0100, Mick wrote: On Thursday 21 Jul 2011 03:29:17 Adam Carter wrote: amd64 means any x86 64bit platform, so Intel too. march=native is good if you're not using distcc, or if you're only using distcc on core2 boxes. Otherwise be specific. I recommend using the 64 bit profile (amd64) for = GCC 4.3 which shows - march=core2. This is what I use here with multilib and had no problems. Ive just stumbled on something weird with march=native: At some point I had march=prescott on a core2 E4600 running 32bit - worked well. Changed to march=native and did some upgrades with a few odd things like asterisk segfaulting in a glibc library afterwards, and some things not building. Then to add confusion, I changed to an pentium Duo E6600 (flies!) and added another stick of ram. More odd things happening such as reiserfs oopsing on shutdown. Last night the penny dropped and I looked the new processor up and changed to march=core2 and have mostly corrected (recompiled) the damage. So not sure about march=native now as it is only what was built with native thats been problematic. With 20-20 hindsight it was perhaps predictable ... BillK
Re: [gentoo-user] Using KVM, what about clocks?
Am 2011-07-21 02:41, schrieb Albert Hopkins: On Wednesday, July 20 at 23:43 (+0200), Stefan G. Weichinger said: [...] Are there any recommended kernel-config-settings for a performant and non-drifting KVM-server? Well, KVM_CLOCK obviously: KVM_CLOCK bool KVM paravirtualized clock select PARAVIRT select PARAVIRT_CLOCK Turning on this option will allow you to run a paravirtualized clock when running over the KVM hypervisor. Instead of relying on a PIT (or probably other) emulation by the underlying device model, the host provides the guest with timing infrastructure such as time of day, and system time I wasn't sure if this option was for the server/host or for using it in the kernel of the guest-vm (like the virtio-drivers ...). Thanks for telling me! Stefan
Re: [gentoo-user] Re: Can't find reiser4 patch for kernel-2.6.39
On 07/20/2011 07:33 PM, Volker Armin Hemmann wrote: On Wednesday 20 July 2011 18:36:39 Michael Orlitzky wrote: On 07/20/2011 07:53 AM, Nikos Chantziaras wrote: On 07/18/2011 11:45 PM, Bill Longman wrote: On 07/18/2011 06:50 AM, Volker Armin Hemmann wrote: On Monday 18 July 2011 14:30:28 Stroller wrote: On 18 July 2011, at 12:18, Mick wrote: Is it a matter of waiting a bit longer? Yes, I think he'll be eligible for parole beginning 2023. please refrain yourself from idiotic remarks like this. Everyone *knows* he's got full internet access, Stroller.sheesh. File a bug report and the warden will pass it his way. Given that he invented his own file system, I wonder why the prison bars are able to keep him locked up... If using a filesystem written by a murderer is wrong, I don't want to read/write! so you guys decided to jump on Stroller's bandwagon. Please, continue. Helps a lot when to decide which threads to ignore. Just trying to lighten the mood, don't take it the wrong way.
Re: [gentoo-user] Re: evdev broken?
On Wed, 20 Jul 2011 18:34:03 -0400, Michael Orlitzky wrote: I've always wondered why, if portage knows that has to be done, can't portage just go ahead and do it? Now that we have a set to do this, I see no reason why this could not be an option, enabled by a USE flag. The last time I complained about this, someone sent me here: https://bugs.gentoo.org/show_bug.cgi?id=192319 That seems to be discussing ABI changes. The X drivers situation is different. Also, that issue has largely been resolved, in as much as ABI changes don't break things like they used to, with @preserved-rebuild. -- Neil Bothwick Programming Language: (n.) a shorthand way of describing a series of bugs to a computer or a programmer. signature.asc Description: PGP signature
RE: [gentoo-user] New computer and Gentoo
-original message- Subject: Re: [gentoo-user] New computer and Gentoo From: Bill Kenworthy bi...@iinet.net.au Date: 2011-07-21 12:54 On Thu, 2011-07-21 at 06:26 +0100, Mick wrote: On Thursday 21 Jul 2011 03:29:17 Adam Carter wrote: amd64 means any x86 64bit platform, so Intel too. march=native is good if you're not using distcc, or if you're only using distcc on core2 boxes. Otherwise be specific. I recommend using the 64 bit profile (amd64) for = GCC 4.3 which shows - march=core2. This is what I use here with multilib and had no problems. Ive just stumbled on something weird with march=native: At some point I had march=prescott on a core2 E4600 running 32bit - worked well. Changed to march=native and did some upgrades with a few odd things like asterisk segfaulting in a glibc library afterwards, and some things not building. Then to add confusion, I changed to an pentium Duo E6600 (flies!) and added another stick of ram. More odd things happening such as reiserfs oopsing on shutdown. Last night the penny dropped and I looked the new processor up and changed to march=core2 and have mostly corrected (recompiled) the damage. So not sure about march=native now as it is only what was built with native thats been problematic. With 20-20 hindsight it was perhaps predictable ... IMO you're not supposed to compile part of the system with -march=something and the rest with -march=native. The instructions (and optimizations) emitted by -march=native might not be compatible with your previous -march. I had deployed more than 10 Gentoo servers using -march=native without any problem. CMIIW Rgds, -- FdS Pandu E Poluan ~ IT Optimizer ~ Sent from Nokia E72-1
Re: [gentoo-user] New computer and Gentoo
Am 21.07.2011 10:57, schrieb Pandu Poluan: -original message- Subject: Re: [gentoo-user] New computer and Gentoo From: Bill Kenworthy bi...@iinet.net.au Date: 2011-07-21 12:54 On Thu, 2011-07-21 at 06:26 +0100, Mick wrote: [...] Ive just stumbled on something weird with march=native: At some point I had march=prescott on a core2 E4600 running 32bit - worked well. Changed to march=native and did some upgrades with a few odd things like asterisk segfaulting in a glibc library afterwards, and some things not building. Then to add confusion, I changed to an pentium Duo E6600 (flies!) and added another stick of ram. More odd things happening such as reiserfs oopsing on shutdown. Last night the penny dropped and I looked the new processor up and changed to march=core2 and have mostly corrected (recompiled) the damage. So not sure about march=native now as it is only what was built with native thats been problematic. With 20-20 hindsight it was perhaps predictable ... IMO you're not supposed to compile part of the system with -march=something and the rest with -march=native. The instructions (and optimizations) emitted by -march=native might not be compatible with your previous -march. I'd like to see a reference for this claim. -march=native doesn't do more than set -march=core2 and some other optimizations for cache size etc. This should be no more troublesome than mixing code compiled with different specific -march settings. When you look at binary distributions (and especially precompiled packages from the developer instead of the distribution), this is pretty much normal. The compiler is not allowed to change the external interfaces of functions for optimization purposes (see [1]). Besides this, I can only think of alignment problems ([2]) but even this should be handled correctly by the compiler. Everything else is a compiler bug that should be reported. [1] http://en.wikipedia.org/wiki/Calling_convention [2] http://en.wikipedia.org/wiki/Data_alignment Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] New computer and Gentoo
On Thu, Jul 21, 2011 at 16:21, Florian Philipp li...@binarywings.net wrote: Am 21.07.2011 10:57, schrieb Pandu Poluan: -original message- Subject: Re: [gentoo-user] New computer and Gentoo From: Bill Kenworthy bi...@iinet.net.au Date: 2011-07-21 12:54 On Thu, 2011-07-21 at 06:26 +0100, Mick wrote: [...] Ive just stumbled on something weird with march=native: At some point I had march=prescott on a core2 E4600 running 32bit - worked well. Changed to march=native and did some upgrades with a few odd things like asterisk segfaulting in a glibc library afterwards, and some things not building. Then to add confusion, I changed to an pentium Duo E6600 (flies!) and added another stick of ram. More odd things happening such as reiserfs oopsing on shutdown. Last night the penny dropped and I looked the new processor up and changed to march=core2 and have mostly corrected (recompiled) the damage. So not sure about march=native now as it is only what was built with native thats been problematic. With 20-20 hindsight it was perhaps predictable ... IMO you're not supposed to compile part of the system with -march=something and the rest with -march=native. The instructions (and optimizations) emitted by -march=native might not be compatible with your previous -march. I'd like to see a reference for this claim. -march=native doesn't do more than set -march=core2 and some other optimizations for cache size etc. This should be no more troublesome than mixing code compiled with different specific -march settings. When you look at binary distributions (and especially precompiled packages from the developer instead of the distribution), this is pretty much normal. The compiler is not allowed to change the external interfaces of functions for optimization purposes (see [1]). Besides this, I can only think of alignment problems ([2]) but even this should be handled correctly by the compiler. Everything else is a compiler bug that should be reported. [1] http://en.wikipedia.org/wiki/Calling_convention [2] http://en.wikipedia.org/wiki/Data_alignment Hmmm... You do have a point. So, gcc's -march option *should* maintain ABI compatibility. There *are* other switches that can impact compatibility [1]. What I can't be sure of, whether the -march option (improperly) activates one or more of those switches when set to -march=native [1] http://gcc.gnu.org/onlinedocs/gcc/i386-and-x86_002d64-Options.html Rgds, -- Pandu E Poluan ~ IT Optimizer ~ Blog : http://pepoluan.tumblr.com • Linked-In : http://id.linkedin.com/in/pepoluan
Re: [gentoo-user] New computer and Gentoo
Am 21.07.2011 11:21, schrieb Florian Philipp: Am 21.07.2011 10:57, schrieb Pandu Poluan: -original message- Subject: Re: [gentoo-user] New computer and Gentoo From: Bill Kenworthy bi...@iinet.net.au Date: 2011-07-21 12:54 On Thu, 2011-07-21 at 06:26 +0100, Mick wrote: [...] Ive just stumbled on something weird with march=native: At some point I had march=prescott on a core2 E4600 running 32bit - worked well. Changed to march=native and did some upgrades with a few odd things like asterisk segfaulting in a glibc library afterwards, and some things not building. Then to add confusion, I changed to an pentium Duo E6600 (flies!) and added another stick of ram. More odd things happening such as reiserfs oopsing on shutdown. Last night the penny dropped and I looked the new processor up and changed to march=core2 and have mostly corrected (recompiled) the damage. So not sure about march=native now as it is only what was built with native thats been problematic. With 20-20 hindsight it was perhaps predictable ... IMO you're not supposed to compile part of the system with -march=something and the rest with -march=native. The instructions (and optimizations) emitted by -march=native might not be compatible with your previous -march. I'd like to see a reference for this claim. -march=native doesn't do more than set -march=core2 and some other optimizations for cache size etc. This should be no more troublesome than mixing code compiled with different specific -march settings. When you look at binary distributions (and especially precompiled packages from the developer instead of the distribution), this is pretty much normal. The compiler is not allowed to change the external interfaces of functions for optimization purposes (see [1]). Besides this, I can only think of alignment problems ([2]) but even this should be handled correctly by the compiler. Everything else is a compiler bug that should be reported. [1] http://en.wikipedia.org/wiki/Calling_convention [2] http://en.wikipedia.org/wiki/Data_alignment Regards, Florian Philipp I've just checked it by comparing the output of `gcc -v -Q -O2 -march=native test.c` with `gcc -v -Q -O2 -march=core2 test.c`: -march=native is resolved to: -march=core2 -mcx16 -msahf -mpopcnt -msse4.2 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=256 -mtune=core2 I also see that the -march=core2 version has the following options enabled: -mno-sse4 whereas -march=native enabled: -mpopcnt -msse4 -msse4.1 -msse4.2 -mssse3 and lower SSE versions were enabled in both cases. Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
[gentoo-user] Upgrade query
A little advice please? I am about to build a new box going from athlon dual core to phenom six core. Including new sata drives and motherboard. I was going to clone all my partitions and the re emerged all packages with march native Firstly would you reccommend cloning and if so what is best technology? Second is a complete reinstall a better option or safer? Sent from my BlackBerry® smartphone on O2
Re: [gentoo-user] Upgrade query
On Thursday, July 21 at 10:01 (+), j...@jdm.myzen.co.uk said: A little advice please? I am about to build a new box going from athlon dual core to phenom six core. Including new sata drives and motherboard. I was going to clone all my partitions and the re emerged all packages with march native Firstly would you reccommend cloning and if so what is best technology? When I move to a different machine, I just * boot into a live cd * back up all the partitions with rsync (or use tar or similar if you need compression) to an external (USB) drive. * boot new machine into livcd * repartion, copy backed up files * install bootloader (and reconfigure/build kernel if necessary) If both source and target are on the same network you can probably also get away with rsync'ing over the LAN instead of using an external drive. As for what technology is best, they are not going to make a whole lot of difference, IMO. I find rsync/cp easier to work with (you can manipulate files before copying them to the new box). tar is more efficient if you need compression. dd, would be the least efficient in my opinion, because it's going to clone the entire partition, including unused blocks, when you're really only concerned about the files. Tools like partimage, etc. can clone a partition smartly but I tend to use those tools less often as I'm really only concerned about the files, not the partitions. Unless your source and target partitions are going to have the exact same geometry, I don't see the benefit if cloning partitions. Just my 2¢ -a
Re: [gentoo-user] Upgrade query
Am 21.07.2011 12:01, schrieb j...@jdm.myzen.co.uk: A little advice please? I am about to build a new box going from athlon dual core to phenom six core. Including new sata drives and motherboard. I was going to clone all my partitions and the re emerged all packages with march native Firstly would you reccommend cloning and if so what is best technology? Second is a complete reinstall a better option or safer? Cloning works fairly well. Especially for architectures that are so close to each other as Athlon X2 and Phenom. I've done it for an upgrade from one of the original Athlon 64 to a Phenom just a few months ago. Just make sure your kernel contains all necessary drivers. Also, udev will usually detect your network interfaces as new interfaces and give them different numbers (eth1 instead of eth0 and so on). Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
[gentoo-user] Kernel panics and more info
Here we go again. New thread, same problem. I'm compiling info over a period of time here so bear with me. Info alert: root@fireball / # emerge --info firefox Portage 2.2.0_alpha45 (default/linux/amd64/10.0/desktop/kde, gcc-4.5.2, glibc-2.13-r4, 2.6.39-gentoo-r2 x86_64) = System Settings = System uname: Linux-2.6.39-gentoo-r2-x86_64-AMD_Phenom-tm-_II_X4_955_Processor-with-gentoo-2.0.2 Timestamp of tree: Sun, 17 Jul 2011 01:45:01 + app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.1-r1, 3.1.3-r1 dev-util/cmake: 2.8.4-r1 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.0.2 sys-apps/openrc: 0.8.3-r1 sys-apps/sandbox: 2.4 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.20.1-r1 sys-devel/gcc:4.4.5, 4.5.2 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool:2.2.10 sys-devel/make: 3.82 sys-kernel/linux-headers: 2.6.36.1 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo Installed sets: ACCEPT_KEYWORDS=amd64 ACCEPT_LICENSE=* CBUILD=x86_64-pc-linux-gnu CFLAGS=-march=native -O2 -pipe CHOST=x86_64-pc-linux-gnu CONFIG_PROTECT=/etc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/hsqldb CONFIG_PROTECT_MASK=/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo CXXFLAGS=-march=native -O2 -pipe DISTDIR=/usr/portage/distfiles EMERGE_DEFAULT_OPTS=--with-bdeps y --backtrack=30 FEATURES=assume-digests binpkg-logs buildpkg distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch FFLAGS= GENTOO_MIRRORS=http://mirror.mcs.anl.gov/pub/gentoo/ http://gentoo.chem.wisc.edu/gentoo/ http://mirrors.cs.wmich.edu/gentoo http://www.cyberuse.com/gentoo/ http://mirror.datapipe.net/gentoo http://gentoo.mirrors.easynews.com/linux/gentoo/ http://chi-10g-1-mirror.fastsoft.net/pub/linux/gentoo/gentoo-distfiles/ http://www.gtlib.gatech.edu/pub/gentoo http://gentoo.mirrors.hoobly.com/ http://distro.ibiblio.org/pub/linux/distributions/gentoo/ http://lug.mtu.edu/gentoo/ http://gentoo.netnitco.net http://gentoo.osuosl.org/ http://gentoo.mirrors.pair.com/ http://mirrors.rit.edu/gentoo/ http://mirror.iawnet.sandia.gov/gentoo/ http://gentoo.llarian.net/ http://gentoo.mirrors.tds.net/gentoo; LANG=en_US LC_ALL=en_US.UTF8 LDFLAGS=-Wl,-O1 -Wl,--as-needed LINGUAS=en_US en MAKEOPTS=-j6 PKGDIR=/usr/portage/packages PORTAGE_CONFIGROOT=/ PORTAGE_RSYNC_OPTS=--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages PORTAGE_TMPDIR=/var/tmp PORTDIR=/usr/portage PORTDIR_OVERLAY= SYNC=rsync://rsync21.us.gentoo.org/gentoo-portage USE=3dnow 3dnowext X aac acpi alsa amd64 aml auto-hinter automount avahi bash-completion berkdb bzip2 cairo cdda cddb cdr chroot cleartype cli clucene consolekit corefonts cracklib cups curl cxx dbus declarative dri dvd dvdr emboss encode escreen esd exif fam ffmpeg firefox flac fortran gdbm gdu gif gimp gkrellm gnutls gphoto2 gpm gtk hbci hddtemp iconv ipv6 java javascript jbig jpeg jpeg2k justify kde kipi lcms libnotify libwww logrotate loop-aes mad mdnsresponder-compat melt mmx mng modules mp3 mp4 mpeg mplayer mudflap multilib mysql ncurses nls nptl nptlonly nsplugin offensive ofx ogg opengl openmp pam pango parport pcre pdf perl phonon plasma png policykit ppds ppp pppd python qt3 qt3support qt4 readline sasl sdl seamonkey semantic-desktop session sift smp spell sse sse2 ssl startup-notification svg sysfs syslog tcl tcpd threads tiff tk truetype type1 udev unicode usb vcd vorbis webkit wma wmf x264 xcb xcomposite xinerama xml xorg xscreensaver xv xvid yahoo zeroconf zlib ALSA_CARDS=hda-intel ALSA_PCM_PLUGINS=adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol APACHE2_MODULES=actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias CALLIGRA_FEATURES=braindump flow karbon kexi kpresenter krita tables words CAMERAS=canon ptp2 COLLECTD_PLUGINS=df interface irq load memory rrdtool swap syslog ELIBC=glibc GPSD_PROTOCOLS=ashtech
Re: [gentoo-user] Upgrade query
Thanks. Rsync sounds like a good option as I can boot pc with old hard disks installed. I assume that rsync works ok with ntfs? Jdm Sent from my BlackBerry® smartphone on O2 -Original Message- From: Albert Hopkins mar...@letterboxes.org Date: Thu, 21 Jul 2011 06:18:53 To: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Upgrade query On Thursday, July 21 at 10:01 (+), j...@jdm.myzen.co.uk said: A little advice please? I am about to build a new box going from athlon dual core to phenom six core. Including new sata drives and motherboard. I was going to clone all my partitions and the re emerged all packages with march native Firstly would you reccommend cloning and if so what is best technology? When I move to a different machine, I just * boot into a live cd * back up all the partitions with rsync (or use tar or similar if you need compression) to an external (USB) drive. * boot new machine into livcd * repartion, copy backed up files * install bootloader (and reconfigure/build kernel if necessary) If both source and target are on the same network you can probably also get away with rsync'ing over the LAN instead of using an external drive. As for what technology is best, they are not going to make a whole lot of difference, IMO. I find rsync/cp easier to work with (you can manipulate files before copying them to the new box). tar is more efficient if you need compression. dd, would be the least efficient in my opinion, because it's going to clone the entire partition, including unused blocks, when you're really only concerned about the files. Tools like partimage, etc. can clone a partition smartly but I tend to use those tools less often as I'm really only concerned about the files, not the partitions. Unless your source and target partitions are going to have the exact same geometry, I don't see the benefit if cloning partitions. Just my 2¢ -a
Re: [gentoo-user] Kernel panics and more info
Dale writes: Here we go again. New thread, same problem. I'm compiling info over a period of time here so bear with me. Info alert: [...] Right now, just look over my info, see if you see anything insane there and if not, recommend something I can use besides flash. Maybe some third party thing that works reasonably well. If youtube works, I should be good to go since the sites I go to are using the same thing, some even source youtube. I'd also try other video drivers, like nouveau or nv. I think you did not do this yet, sorry if I just overlooked it. They may not work as well as the nvidia-drivers for you, but this way you can rule out the video drivers, or confirm it has something to do with them. I still suspect they are the cause, and there is an obscure bug that only Firefox triggers. Oh, and have you tried firefox-bin already? Wonko
Re: [gentoo-user] Upgrade query
j...@jdm.myzen.co.uk writes: A little advice please? I am about to build a new box going from athlon dual core to phenom six core. Including new sata drives and motherboard. I was going to clone all my partitions and the re emerged all packages with march native Firstly would you reccommend cloning and if so what is best technology? I'd plug the new system drive into the old PC and clone with dd. You can even make the partitions larger and use resize2fs (in case of ext3/4) to enlarge the space after cloning. Once caveat: Your new SATA drives are probably using a block size of 4K instead of 512 bytes, internally. To the OS they still look like they have 512 bytes. But when the partitions are not aligned to 4K boundaries (and I think this is still the case when using fdisk), there is a big performance loss. I also would make sure the whole drive is being written to once, in order to detect bad blocks. badblocks -sw device will do this. Second is a complete reinstall a better option or safer? I'd say just clone it. Remove your /etc/udev/rules.d/*-persistent-*.rules files, so your network interfaces will not be renamed from eth0 to eth1. If you change your CFLAGS, emerge -e world. I'm not sure if you better emerge -e system before (once or even twice) so the toolchain is already compiled with the new settings, maybe someone else will say something about this. Wonko
[gentoo-user] high load from x11-terms/terminator
Greets, I use x11-terms/terminator most of the time and over the last few days I noticed that these processes generate a high load on my CPUs. The processes also seem to hang around even after I close the terminator-windows! Rebuilding the pkg (and gnome-terminal as well, just in case) has not yet helped ... it's ~amd64 here, everything up to date so far ... Gnome 2.32 btw Anyone else noticed this? Any idea what to look at? I already did revdep-rebuild, emerge -avuDN world ... a reboot etc Stefan
Re: [gentoo-user] Kernel panics and more info
Alex Schuster wrote: Dale writes: Here we go again. New thread, same problem. I'm compiling info over a period of time here so bear with me. Info alert: [...] Right now, just look over my info, see if you see anything insane there and if not, recommend something I can use besides flash. Maybe some third party thing that works reasonably well. If youtube works, I should be good to go since the sites I go to are using the same thing, some even source youtube. I'd also try other video drivers, like nouveau or nv. I think you did not do this yet, sorry if I just overlooked it. They may not work as well as the nvidia-drivers for you, but this way you can rule out the video drivers, or confirm it has something to do with them. I still suspect they are the cause, and there is an obscure bug that only Firefox triggers. Oh, and have you tried firefox-bin already? Wonko I have not been able to get the nv drivers to work. It has been so long since I had to use them, it appears I have forgot how to use them. I'm not sure I have ever used them since I been using Gentoo. I found the link on the xorg site. It wasn't very useful for me. I do have xf86-video-nv installed tho. I thought that was it. However, when I change the driver in xorg.conf to nv, no more GUI. That always worked in the past. Can you give me a light bulb moment here? ;-) I may be missing something obvious. As for Firefox-bin, I'm not sure that would help Seamonkey. I could try it but not sure how that would help. Seamonkey would still crash. Now that I have the same tool I was using in Firefox, I'll most likely get rid of Firefox. The download helper was the only reason I was using Firefox. Thanks. Dale :-) :-)
Re: [gentoo-user] Upgrade query
On Thursday, July 21 at 11:10 (+), j...@jdm.myzen.co.uk said: Well, depends on your definition of works. AFAIK linux does not expose the NFTS permission system fully, because they are very different and there is no 1:1 mapping between them. So while the *data* may be copied over, the permissions will likely only be copied as far as how the linux filesystem layer sees them, and may not be preserved 100%. There are NTFS tools afaik though, (ntfsclone). Since I don't actually use NTFS for anything, someone else may better be able to assist.
Re: [gentoo-user] xfce (window manager)
- Original Message -From: András Csányi Date: Wednesday, July 20, 2011 4:50 amSubject: [gentoo-user] xfce (window manager)To: gentoo-user Dear All, I would like to ask the community anybody has experienced any problems regarding xfce? My problem is that after update the window decorators (window border, header the buttons on the left side [close, minimize, etc]) are missing. I'm sorry I'm not sure the window decorators definition is the proper for them. I've deleted my xfce settings to start a session with default settings without any success. Anybody has any idea or suggestion what should I do? Thanks in advance! András -- - - -- Csanyi Andras (Sayusi Ando) -- http://sayusi.hu -- http://facebook.com/andras.csanyi -- Trust in God and keep your gunpowder dry! - Cromwell Not sure if this will fix the problem, but . . . Is XDG set correctly? Mine is set for gnome, as follows.$ cd$ grep -i xdg .*.bash_profile:# XDG_MENU_PREFIX needed for alacarte menu editor..bash_profile:export XDG_MENU_PREFIX=gnome-Yours should be XDG_MENU_PREFIX=xfce4-Also check your /etc/xdg directory.
Re: [gentoo-user] Kernel panics and more info
Dale writes: Alex Schuster wrote: I'd also try other video drivers, like nouveau or nv. I think you did not do this yet, sorry if I just overlooked it. They may not work as well as the nvidia-drivers for you, but this way you can rule out the video drivers, or confirm it has something to do with them. I still suspect they are the cause, and there is an obscure bug that only Firefox triggers. Oh, and have you tried firefox-bin already? I have not been able to get the nv drivers to work. It has been so long since I had to use them, it appears I have forgot how to use them. I'm not sure I have ever used them since I been using Gentoo. I found the link on the xorg site. It wasn't very useful for me. I do have xf86-video-nv installed tho. I thought that was it. However, when I change the driver in xorg.conf to nv, no more GUI. That always worked in the past. Can you give me a light bulb moment here? ;-) I may be missing something obvious. Sorry, no, I had an NVidia card years ago, now I'm an ATI user. When I had trouble with the nvidia-drivers (I often had, with every kernel update I feared it would happen again) I simply replaced the nvidia by nv in the Drivers section, and all was fine, except for OpenGL speed. As for Firefox-bin, I'm not sure that would help Seamonkey. I could try it but not sure how that would help. Seamonkey would still crash. Yes, but if firefox-bin would not crash, it might indicate a compiler problem on your side, or something. Just trying to narrow things. And there's also seamonkey-bin. But my guess is those will also crash. Wonko
Re: [gentoo-user] Kernel panics and more info
On Thursday 21 July 2011 05:54:32 Dale wrote: I been working on gathering information for this for a while. I just tried something else. I use the download helper plugin in Firefox to download videos. Also, it crashes when I am downloading videos but that is about all I use Firefox for. I had a light bulb moment and decided to see if the same plugin was available for Seamonkey. It was available so I installed it. It works the same as in Firefox. So, I closed as much stuff as I could and started downloading a couple good size videos. After a few minutes, you guessed it, kernel panic followed by a reboot. To make sure it was not a fluke or something, I repeated the process and got the same result. Dale, I would suspect the download-helper-plugin as that is common in both the Firefox and Seamonkey crashes. Can you provide a link to the plugin you use to allow others to test this to see if it works for them? -- Joost
Re: [gentoo-user] Kernel panics and more info
On Thursday 21 July 2011 05:54:32 Dale wrote: Right now, just look over my info, see if you see anything insane there and if not, recommend something I can use besides flash. Maybe some third party thing that works reasonably well. If youtube works, I should be good to go since the sites I go to are using the same thing, some even source youtube. About this bit. Can you play youtube videos without using that download-helper-plugin? Or is your connection too slow to be able to watch directly without downloading first? As an alternative to using the download-helper-plugin, you could try the following: $ eix youtube-dl * net-misc/youtube-dl Available versions: ~2009.05.23[1] 2010.01.19[1] 2010.10.03 ~2010.10.24 ~2010.11.19 2010.12.09 ~2011.01.30 Homepage:http://rg3.github.com/youtube-dl/ Description: A small command-line program to download videos from YouTube. Hope this helps, Joost
[gentoo-user] Re: Kernel panics and more info
The 21/07/11, Dale wrote: I have not been able to get the nv drivers to work. It has been so long since I had to use them, it appears I have forgot how to use them. I'm not sure I have ever used them since I been using Gentoo. Try VESA. As for Firefox-bin, I'm not sure that would help Seamonkey. I could try it but not sure how that would help. Seamonkey would still crash. Now that I have the same tool I was using in Firefox, I'll most likely get rid of Firefox. The download helper was the only reason I was using Firefox. A book writer would say: when my system crash, I'm always using my text editor; so my editor makes the system crash. I'm not telling the root cause you suspect is not the real cause but that it is NOT likely to be the real cause in the first place. -- Nicolas Sebrecht
Re: [gentoo-user] New computer and Gentoo
On Thu, 2011-07-21 at 11:21 +0200, Florian Philipp wrote: Am 21.07.2011 10:57, schrieb Pandu Poluan: -original message- Subject: Re: [gentoo-user] New computer and Gentoo From: Bill Kenworthy bi...@iinet.net.au Date: 2011-07-21 12:54 On Thu, 2011-07-21 at 06:26 +0100, Mick wrote: [...] Ive just stumbled on something weird with march=native: I'd like to see a reference for this claim. -march=native doesn't do more than set -march=core2 and some other optimizations for cache size etc. This should be no more troublesome than mixing code compiled with unfortunately this is now my main machine so I cant fiddle too much with it! It would mean going back to the E4600 and comparing march=prescott, march=native then fitting the E6600 and checking march=native and march=core2. What I cant find is a reference to how it works out what native is? - lookup-table, checking the flags in /proc/cpuinfo or what? Ive now rolled back (that is recompiled) the majority of packages so now I can keep working while it does an emerge -e world. I would have thought the two intel processors would be close enough that it would be just a performance hit and not segfaults, but the machine is now working reliably so thats proof enough for me. What I am having difficulty with is that packages compiled with native should have been a closer match to the cpu so why was it those packages (asterisk, glibc and some random others cause problems. BillK
Re: [gentoo-user] Upgrade query
On Thursday 21 July 2011 12:20:30 Florian Philipp wrote:
Also, udev will usually detect your network interfaces as new interfaces
and give them different numbers (eth1 instead of eth0 and so on).
You can solve this by deleting the respective entries in the udev config:
**
$ cat /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib64/udev/write_net_rules
# program run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single line.
# PCI device 0x1234:0x1234 (eth-device)
SUBSYSTEM==net, DRIVERS==?*, ATTR{address}==00:11:22:33:44:55,
KERNEL==eth*, NAME=eth0
***
Removing this file or just the entries should force udev to reuse the network-
device-names.
--
Joost
Re: [gentoo-user] high load from x11-terms/terminator
On Thu, Jul 21, 2011 at 7:23 AM, Stefan G. Weichinger li...@xunil.at wrote: Greets, I use x11-terms/terminator most of the time and over the last few days I noticed that these processes generate a high load on my CPUs. The processes also seem to hang around even after I close the terminator-windows! Rebuilding the pkg (and gnome-terminal as well, just in case) has not yet helped ... it's ~amd64 here, everything up to date so far ... Gnome 2.32 btw Anyone else noticed this? Any idea what to look at? I already did revdep-rebuild, emerge -avuDN world ... a reboot etc That sounds a *lot* like a software bug. Particularly, some supporting thread gets stuck in an infinite loop (or some wake event or some such never gets reset), but the main thread continues. Main thread dies when the app is closed, but the supporting thread continues. I'd suggest trying older versions until you find what the last version was that didn't have the bug, and then file a bug report so the developer can get it fixed. -- :wq
Re: [gentoo-user] New computer and Gentoo
Am 21.07.2011 15:10, schrieb William Kenworthy: On Thu, 2011-07-21 at 11:21 +0200, Florian Philipp wrote: Am 21.07.2011 10:57, schrieb Pandu Poluan: -original message- Subject: Re: [gentoo-user] New computer and Gentoo From: Bill Kenworthy bi...@iinet.net.au Date: 2011-07-21 12:54 On Thu, 2011-07-21 at 06:26 +0100, Mick wrote: [...] Ive just stumbled on something weird with march=native: I'd like to see a reference for this claim. -march=native doesn't do more than set -march=core2 and some other optimizations for cache size etc. This should be no more troublesome than mixing code compiled with unfortunately this is now my main machine so I cant fiddle too much with it! It would mean going back to the E4600 and comparing march=prescott, march=native then fitting the E6600 and checking march=native and march=core2. What I cant find is a reference to how it works out what native is? - lookup-table, checking the flags in /proc/cpuinfo or what? [...] Use the source, Luke. Take a look at the gcc sources in your distfiles directory. It is in the gcc package, file ./gcc/config/i386/driver-i386.c (found via grep for \native and then grep for host_detect_local_cpu). It is surprisingly understandable. Most of the detection is done by introspecting CPUID. Most importantly, the CPU family is deducted from the found capabilities, not vice versa. Therefore there is no chance that a CPU that claims to be, let's say, a Core2, but lacks some capabilities, ends up being classified as a Core2 (unless, of course, if the GCC guys get their switch-case logic wrong). Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Kernel panics and more info
On Thu, 21 Jul 2011 06:41:31 -0500, Dale wrote: I have not been able to get the nv drivers to work. It has been so long since I had to use them, it appears I have forgot how to use them. I'm not sure I have ever used them since I been using Gentoo. I found the link on the xorg site. It wasn't very useful for me. I do have xf86-video-nv installed tho. I thought that was it. However, when I change the driver in xorg.conf to nv, no more GUI. Instead of editing xorg.conf, move it out of the way and let X figure things out for itself. You could also try the VESA drivers, slow but reliable. -- Neil Bothwick Top Oxymorons Number 2: Exact estimate signature.asc Description: PGP signature
Re: [gentoo-user] high load from x11-terms/terminator
On 21.07.2011 13:23, Stefan G. Weichinger wrote: Greets, I use x11-terms/terminator most of the time and over the last few days I noticed that these processes generate a high load on my CPUs. The processes also seem to hang around even after I close the terminator-windows! Rebuilding the pkg (and gnome-terminal as well, just in case) has not yet helped ... it's ~amd64 here, everything up to date so far ... Gnome 2.32 btw Anyone else noticed this? Any idea what to look at? I already did revdep-rebuild, emerge -avuDN world ... a reboot etc Stefan If you have the x11-drivers/nvidia-drivers-275.19 installed, your issue may be related to https://bugs.gentoo.org/show_bug.cgi?id=375615 poncho
Re: [gentoo-user] Kernel panics and more info
Neil Bothwick wrote: On Thu, 21 Jul 2011 06:41:31 -0500, Dale wrote: I have not been able to get the nv drivers to work. It has been so long since I had to use them, it appears I have forgot how to use them. I'm not sure I have ever used them since I been using Gentoo. I found the link on the xorg site. It wasn't very useful for me. I do have xf86-video-nv installed tho. I thought that was it. However, when I change the driver in xorg.conf to nv, no more GUI. Instead of editing xorg.conf, move it out of the way and let X figure things out for itself. You could also try the VESA drivers, slow but reliable. OK. How do I do the VESA drivers? Honestly, the only trouble I can recall out of nvidia was upgrading the kernel then rebooting and realizing I forgot to rebuild against the new kernel. I don't recall every having anything like this. The biggest GUI problem I can recall was hal and xorg. Let's not go down that road. dale starts to steam Is VESA a kernel option or some package I need to install? I really want to figure this out but not just for me. Surely I'm not the only one this has happened to. Dale :-) :-)
Re: [gentoo-user] Kernel panics and more info
Joost Roeleveld wrote: On Thursday 21 July 2011 05:54:32 Dale wrote: I been working on gathering information for this for a while. I just tried something else. I use the download helper plugin in Firefox to download videos. Also, it crashes when I am downloading videos but that is about all I use Firefox for. I had a light bulb moment and decided to see if the same plugin was available for Seamonkey. It was available so I installed it. It works the same as in Firefox. So, I closed as much stuff as I could and started downloading a couple good size videos. After a few minutes, you guessed it, kernel panic followed by a reboot. To make sure it was not a fluke or something, I repeated the process and got the same result. Dale, I would suspect the download-helper-plugin as that is common in both the Firefox and Seamonkey crashes. Can you provide a link to the plugin you use to allow others to test this to see if it works for them? I'm not sure you are right but it is possible. Here is a linky: https://addons.mozilla.org/en-Us/seamonkey/addon/video-downloadhelper/ It's pretty straight forward. I can get a failure by going to youtube and starting the download of two or three fairly long movies. I usually middle click and open the video in a new tab. Then start the download and close the tab. I usually walk off or watch TV while it downloads. I have the slow DSL here so it takes a bit to download but it usually crashes in a minute or two. Firefox crashes faster but Seamonkey only takes a couple minutes or something. If anyone tests this, let me know if it messes up. I'm on amd64 multilib too. That may have some effect. Thanks. Dale :-) :-)
Re: [gentoo-user] mysqld invoked oom-killer
On 7/20/2011 6:29 PM, Michael Mol wrote: Also, run a caching proxy if at all possible. That made the single biggest difference for my server. Other useful things: * Set the MaxRequestsPerChild to something like 450. That's pretty low. You'd barely get your application parsed, cached, and load some data before you'd have to recycle the child process. Most people set it around 1. Large enough to be useful, but still deal with any minor memory leaks. kashani
Re: [gentoo-user] Kernel panics and more info
Joost Roeleveld wrote: On Thursday 21 July 2011 05:54:32 Dale wrote: Right now, just look over my info, see if you see anything insane there and if not, recommend something I can use besides flash. Maybe some third party thing that works reasonably well. If youtube works, I should be good to go since the sites I go to are using the same thing, some even source youtube. About this bit. Can you play youtube videos without using that download-helper-plugin? Or is your connection too slow to be able to watch directly without downloading first? As an alternative to using the download-helper-plugin, you could try the following: $ eix youtube-dl * net-misc/youtube-dl Available versions: ~2009.05.23[1] 2010.01.19[1] 2010.10.03 ~2010.10.24 ~2010.11.19 2010.12.09 ~2011.01.30 Homepage:http://rg3.github.com/youtube-dl/ Description: A small command-line program to download videos from YouTube. Hope this helps, Joost My DSL is just a hair to slow unless it is a low res video. I get about 768Kb down and I swear they make it so it will stop just enough to annoy the crap out of a person so you will upgrade to a faster package. o_O Grr. Anyway, I just download it first then play it with smplayer. No stopping and all that then. If I don't like it, dump and move on. I did find that tool once but I'm not real sure how it works. I sort of browsed around the pages a bit but just fell back to the plugin. May end up using it tho. I will try just watching it in a bit and see if it crashes. I can start it then hit pause and resume after it gets a real good start. That could lead to something. ;-) Thanks. Dale :-) :-)
Re: [gentoo-user] Re: evdev broken?
On 07/21/2011 04:57 AM, Neil Bothwick wrote: On Wed, 20 Jul 2011 18:34:03 -0400, Michael Orlitzky wrote: I've always wondered why, if portage knows that has to be done, can't portage just go ahead and do it? Now that we have a set to do this, I see no reason why this could not be an option, enabled by a USE flag. The last time I complained about this, someone sent me here: https://bugs.gentoo.org/show_bug.cgi?id=192319 That seems to be discussing ABI changes. The X drivers situation is different. Also, that issue has largely been resolved, in as much as ABI changes don't break things like they used to, with @preserved-rebuild. Huh? If implemented, that would allow FORCEREBUILD=x11-drivers/xf86-input* on major Xorg upgrades.
Re: [gentoo-user] mysqld invoked oom-killer
I ran into an out of memory problem. The first mention of it in the kernel log is mysqld invoked oom-killer. I haven't run into this before. I do have a swap partition but I don't activate it based on something I read previously that I later found out was wrong so I suppose I should activate it. Is fstab the way to do that? I have a commented line in there for swap. Can anyone tell how much swap this is: /dev/sda2 80325 1140614 530145 82 Linux swap / Solaris If it's something like 512MB, that may not have prevented me from running out of memory since I have 4GB RAM. Is there any way to find out if there was a memory leak or other problem that should be investigated? To activate swap, put a line in fstab like so: /dev/sda2 none swap sw 0 0 However, you do not want to use it. it is not the life-saver some howto authors on the internet claim it to be. When a linux machine hits swap, it does so very aggressively, there is nothing nice about it at all. The entire machine slows to a painstaking crawl for easily a minute at a time while the kernel writes pages out to disk, and disk is thousands of times slower than RAM. It gets so bad that you can't even run a shell properly to try and see what's going on and kill the actual memory hog. My personal rule of thumb: if you hit swap, the bad thing has already gone very very south, usually to the point where you can't do much about it and it's already too late. Besides, that bastard deomon spawn of satan called the oom-killer is likely about to kick in and REALLY make your day. Anyone else notice how oom-killer seems to be hard coded to zap the most inconvenient process of all?. What you need to be doing is monitor your memory usage during normal conditions and deal with issues before they become problems. Hi Alan, I think it was your advice I took a long time ago when I stopped installing new machines with a swap partition and disabled it on my already-installed machines. Some time later, others on this list caught wind of what I'd done and told me I was an idiot. Is there a consensus on this? If the drawbacks and advantages of using swap cancel each other out, I won't use it. - Grant
Re: [gentoo-user] Re: evdev broken?
On Thu, 21 Jul 2011 12:13:09 -0400, Michael Orlitzky wrote: That seems to be discussing ABI changes. The X drivers situation is different. Also, that issue has largely been resolved, in as much as ABI changes don't break things like they used to, with @preserved-rebuild. Huh? @preserved-rebuild largely prevents ABI changes breaking the system by keeping old libraries around until packages have been rebuilt. It's the other way round since the depending packages are affected, whereas here it is the dependencies. If implemented, that would allow FORCEREBUILD=x11-drivers/xf86-input* on major Xorg upgrades. Or even @x11-module-rebuild -- Neil Bothwick Got kleptomania? Be sure to take something for it. signature.asc Description: PGP signature
Re: [gentoo-user] mysqld invoked oom-killer
I ran into an out of memory problem. The first mention of it in the kernel log is mysqld invoked oom-killer. I haven't run into this before. I do have a swap partition but I don't activate it based on something I read previously that I later found out was wrong so I suppose I should activate it. Is fstab the way to do that? I have a commented line in there for swap. ... Switching from prefork to threads and vice versa can be very difficult depending on which modules and libraries your site uses. It is not on the list of things you should try first. Or second. Maybe 37th. I wouldn't expect adding swap to do much in this case. Your site gets hit hard, Mysql is a bit slow, Apache processes start stacking up, the system starts swapping, disk is really slow compared to RAM, and everything grinds to a complete halt possibly locking the machine up. The easiest thing to try is to turn off keepalives so child processes aren't hanging around keeping connections up. Also lower the number of Apache children to 8 * number of processors or a minimum of 32. Test a bit. Turning off keep alive can cause problems for Flash based uploaders to your site and code that expect the connection to stay up. For most sites this shouldn't matter. Next I'd look at tuning your Mysql config. If you've never touched my.cnf, by default it's set to use 64MB IIRC. You may need to raise this to get better performance. key_buffer and innodb_buffer_pool_size are the only two I'd modify without knowing more. I use the default MyISAM tables and it looks like there are three key_buffer definitions in my.cnf. One under [mysqld] is 16M, one under [isamchk] is 20M, and one under [myisamchk] is 20M. All defaults. Should I increase them all to 64M? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
I ran into an out of memory problem. The first mention of it in the kernel log is mysqld invoked oom-killer. I haven't run into this before. I do have a swap partition but I don't activate it based on something I read previously that I later found out was wrong so I suppose I should activate it. Is fstab the way to do that? I have a commented line in there for swap. ... Also, run a caching proxy if at all possible. That made the single biggest difference for my server. Other useful things: * Set the MaxRequestsPerChild to something like 450. As part of their caching, things like mod_php will grow the process size a bit as the apache process gets old in the tooth. Setting MaxRequestsPerChild lower causes the process to expire and be replaced sooner. On my server, I see apache processes consume about 60MB towards the end, and then cycle back and consume about 22MB. Default is 1 so 450 might be too low. * On my server, I have MinSpareServers at 10, and MaxSpareServers at 12. I handle spikes pretty well, and free the memory quickly. I use 20 and 40 for Min and Max respectively. * If you're using PHP, set memory_limit in php.ini as low as your applications can survive. I'm assuming you're running on a VPS or similar. At 512MB of RAM with a web server and database server, you need to keep things very tight. I have 4GB RAM. 512MB is the size of my currently unused swap partition. - Grant
Re: [gentoo-user] mysqld invoked oom-killer
On Thu, Jul 21, 2011 at 11:39 AM, Grant emailgr...@gmail.com wrote: Hi Alan, I think it was your advice I took a long time ago when I stopped installing new machines with a swap partition and disabled it on my already-installed machines. Some time later, others on this list caught wind of what I'd done and told me I was an idiot. Is there a consensus on this? If the drawbacks and advantages of using swap cancel each other out, I won't use it. I think it's basically like this: No swap = If you run out of memory, OOM-killer starts killing things randomly and stuff breaks. With Swap = System does not run out of memory, so things don't die, but it runs poetntially much slower during that period of high memory usage depending on your disk speed and how heavily it is leaning on swap at that moment (if it is actively trying to use more data in RAM than you physically have RAM for, it's a total slowdown disaster). If it's a case of run-away memory usage, it'll run out of swap, too, anyway, so having swap in that case only delays the OOM-killer. I think if you have 4GB of RAM you shouldn't need any swap under normal circumstances. I have a gentoo box with just 256MB of RAM that's running web server (apache + php), mail server (postfix + dovecot), and database (mariadb), and it works fine if i disable swap. I do normally have swap enabled on it, though, because emerging sometimes uses a lot of RAM.
Re: [gentoo-user] mysqld invoked oom-killer
Hi Alan, I think it was your advice I took a long time ago when I stopped installing new machines with a swap partition and disabled it on my already-installed machines. Some time later, others on this list caught wind of what I'd done and told me I was an idiot. Is there a consensus on this? If the drawbacks and advantages of using swap cancel each other out, I won't use it. I think it's basically like this: No swap = If you run out of memory, OOM-killer starts killing things randomly and stuff breaks. With Swap = System does not run out of memory, so things don't die, but it runs poetntially much slower during that period of high memory usage depending on your disk speed and how heavily it is leaning on swap at that moment (if it is actively trying to use more data in RAM than you physically have RAM for, it's a total slowdown disaster). If it's a case of run-away memory usage, it'll run out of swap, too, anyway, so having swap in that case only delays the OOM-killer. I think if you have 4GB of RAM you shouldn't need any swap under normal circumstances. I have a gentoo box with just 256MB of RAM that's running web server (apache + php), mail server (postfix + dovecot), and database (mariadb), and it works fine if i disable swap. I do normally have swap enabled on it, though, because emerging sometimes uses a lot of RAM. Thanks Paul. I'm leaning toward leaving swap disabled. So I'm sure I have the concept right, is adding a 1GB swap partition functionally identical to adding 1GB RAM with regard to the potential for out-of-memory conditions? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
On Thu, Jul 21, 2011 at 12:06 PM, Grant emailgr...@gmail.com wrote: Thanks Paul. I'm leaning toward leaving swap disabled. So I'm sure I have the concept right, is adding a 1GB swap partition functionally identical to adding 1GB RAM with regard to the potential for out-of-memory conditions? Yep.
Re: [gentoo-user] mysqld invoked oom-killer
I ran into an out of memory problem. The first mention of it in the kernel log is mysqld invoked oom-killer. I haven't run into this before. I do have a swap partition but I don't activate it based on something I read previously that I later found out was wrong so I suppose I should activate it. Is fstab the way to do that? I have a commented line in there for swap. ... If you're running any other servers that utilize MySQL like Apache or something, check its access logs to see if you had an abnormal number of connections. Bruteforce hacking or some kind of flooding/DOS attack might cause it to use more memory than it ordinarily would. I don't know why I didn't check the apache2 error log before, but I got the following entry 2 seconds before the server became unresponsive: [error] server reached MaxClients setting, consider raising the MaxClients setting I use the default 256 for MaxClients. This confirms the server was brought down by too many child processes consuming too much memory. Looking back at the access_log, it's clear this condition was caused by the single IP which requested one of my pages about 300 times over the course of 1 minute. This caused my entire server to lock up for hours until I rebooted it. I hesitate to reduce MaxClients from 256. I think my server should be able to handle it since it's the default. So I need to prevent my apache2 child processes from consuming so much memory? apache2 was restarted about an hour before the lockup so it had a pretty fresh start. I do use mod_perl which is a memory hog from what I understand. Do I just need more RAM? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
* Paul Hartman paul.hartman+gen...@gmail.com [110721 12:33]: [..] I think if you have 4GB of RAM you shouldn't need any swap under normal circumstances. I have a gentoo box with just 256MB of RAM that's running web server (apache + php), mail server (postfix + dovecot), and database (mariadb), and it works fine if i disable swap. I do normally have swap enabled on it, though, because emerging sometimes uses a lot of RAM. Indeed, I have a server with 256MB of memory and it couldn't emerge gcc when I had swap disabled. However, it typically runs fine with no swap. It's only running bind basically.
Re: [gentoo-user] mysqld invoked oom-killer
Thanks Paul. I'm leaning toward leaving swap disabled. So I'm sure I have the concept right, is adding a 1GB swap partition functionally identical to adding 1GB RAM with regard to the potential for out-of-memory conditions? Yep. It sounds like adding physical RAM is better than enabling swap in every way. I'll stay in the anti-swap camp. - Grant
Re: [gentoo-user] mysqld invoked oom-killer
[..] I think if you have 4GB of RAM you shouldn't need any swap under normal circumstances. I have a gentoo box with just 256MB of RAM that's running web server (apache + php), mail server (postfix + dovecot), and database (mariadb), and it works fine if i disable swap. I do normally have swap enabled on it, though, because emerging sometimes uses a lot of RAM. Indeed, I have a server with 256MB of memory and it couldn't emerge gcc when I had swap disabled. I can't compile openoffice with -j1 on 3GB RAM and no swap. - Grant
Re: [gentoo-user] mysqld invoked oom-killer
On 7/21/2011 9:53 AM, Grant wrote: Next I'd look at tuning your Mysql config. If you've never touched my.cnf, by default it's set to use 64MB IIRC. You may need to raise this to get better performance. key_buffer and innodb_buffer_pool_size are the only two I'd modify without knowing more. I use the default MyISAM tables and it looks like there are three key_buffer definitions in my.cnf. One under [mysqld] is 16M, one under [isamchk] is 20M, and one under [myisamchk] is 20M. All defaults. Should I increase them all to 64M? You can, but [mysqld] is the only one that matters for normal production. Depends on the size of your data and tables, but 64M is fine to start. If you've got a few GB in your databases I'd go with 256-512M or as high as you think you can get away with. Any reason you're still using MyISAM tables? Innodb is almost as fast or much much faster than MyISAM in nearly every way these days. kashani
Re: [gentoo-user] Upgrade query
On Thursday 21 July 2011 10:01:10 j...@jdm.myzen.co.uk wrote: A little advice please? I am about to build a new box going from athlon dual core to phenom six core. Including new sata drives and motherboard. I was going to clone all my partitions and the re emerged all packages with march native Firstly would you reccommend cloning and if so what is best technology? Second is a complete reinstall a better option or safer? are you using ACLs? if not a good old cp -auv is sufficient. But.. why clone the system? This is a good chance to get rid of cruft and forgotten packages. A clean installation (and copied /etc) might not be a bad choice. -- #163933
Re: [gentoo-user] mysqld invoked oom-killer
On 7/21/2011 10:22 AM, Grant wrote: I ran into an out of memory problem. The first mention of it in the kernel log is mysqld invoked oom-killer. I haven't run into this before. I do have a swap partition but I don't activate it based on something I read previously that I later found out was wrong so I suppose I should activate it. Is fstab the way to do that? I have a commented line in there for swap. ... If you're running any other servers that utilize MySQL like Apache or something, check its access logs to see if you had an abnormal number of connections. Bruteforce hacking or some kind of flooding/DOS attack might cause it to use more memory than it ordinarily would. I don't know why I didn't check the apache2 error log before, but I got the following entry 2 seconds before the server became unresponsive: [error] server reached MaxClients setting, consider raising the MaxClients setting I use the default 256 for MaxClients. This confirms the server was brought down by too many child processes consuming too much memory. Looking back at the access_log, it's clear this condition was caused by the single IP which requested one of my pages about 300 times over the course of 1 minute. This caused my entire server to lock up for hours until I rebooted it. I hesitate to reduce MaxClients from 256. I think my server should be able to handle it since it's the default. So I need to prevent my apache2 child processes from consuming so much memory? apache2 was restarted about an hour before the lockup so it had a pretty fresh start. I do use mod_perl which is a memory hog from what I understand. Do I just need more RAM? Most people do not think about this correctly. Can your server run 1 Apache processes? No, not enough resources. 1000? No, same problem. 256? I'd say no based on this thread. If you're not going to set it at 1 why try to keep it at 256? Next image a grocery store with 256 checkout lanes, but only four cashiers. Four cashiers trying to run that many lanes is actually slower than having only four lanes. However 32 lanes could faster than 4. People can have their groceries setup, baggers aren't getting in the way, etc. The analogy breaks down a bit, but you get the point. There is no performance gain in configuring for concurrency your hardware and software can not support. kashani
[gentoo-user] Can't kill Firefox!
Whatever Dale had on his machine must have infected mine! LOL! A 32bit x86 box with KDE4.6, running firefox-3.6.17 and xulrunner-1.9.2.17 after a few hours and loads of tabs (sometimes up to 15 or so) eventually hangs X. I can switch to a console and login as the user running the X session, then try to kill the bloody thing with killall, kill -15 pid and kill -9 pid. It won't barge. Mind you I haven't tried this as root. How can this be? -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] high load from x11-terms/terminator
Am 21.07.2011 16:21, schrieb Poncho: If you have the x11-drivers/nvidia-drivers-275.19 installed, your issue may be related to https://bugs.gentoo.org/show_bug.cgi?id=375615 Thanks for the pointer, I downgraded the drivers, looks better so far! Stefan
Re: [gentoo-user] Can't kill Firefox!
Mick wrote: Whatever Dale had on his machine must have infected mine! LOL! A 32bit x86 box with KDE4.6, running firefox-3.6.17 and xulrunner-1.9.2.17 after a few hours and loads of tabs (sometimes up to 15 or so) eventually hangs X. I can switch to a console and login as the user running the X session, then try to kill the bloody thing with killall, kill -15pid and kill -9pid. It won't barge. Mind you I haven't tried this as root. How can this be? I would try to kill it as root. The -9 option should work. That hasn't failed me yet. I always run kill commands as root and DOUBLE check the PID after typing it in. I regret to find out that this is not just my problem. Now we got to figure this out. It even affects x86 huh. This is interesting. Dale :-) :-)
Re: [gentoo-user] mysqld invoked oom-killer
On Thu, Jul 21, 2011 at 11:56 AM, kashani kashani-l...@badapple.net wrote: On 7/20/2011 6:29 PM, Michael Mol wrote: Also, run a caching proxy if at all possible. That made the single biggest difference for my server. Other useful things: * Set the MaxRequestsPerChild to something like 450. That's pretty low. You'd barely get your application parsed, cached, and load some data before you'd have to recycle the child process. Most people set it around 1. Large enough to be useful, but still deal with any minor memory leaks. Depends on your application. I had to set it low because the application wouldn't fit in a 540MB VPS, otherwise. I've since bumped up to a 2GB VPS, so I can probably afford Really, a caching proxy is the first, best thing, if it's not already in use. Let the thread carry on... -- :wq
Re: [gentoo-user] Kernel panics and more info
On Thu, 21 Jul 2011 10:46:55 -0500, Dale wrote: You could also try the VESA drivers, slow but reliable. OK. How do I do the VESA drivers? Honestly, the only trouble I can recall out of nvidia was upgrading the kernel then rebooting and realizing I forgot to rebuild against the new kernel. I don't recall every having anything like this. The biggest GUI problem I can recall was hal and xorg. Let's not go down that road. dale starts to steam Is VESA a kernel option or some package I need to install? It's the standard video driver, x11-drivers/xf86-video-vesa -- Neil Bothwick God created the world in six days. On the seventh day he also decided to create England... just to try out his Practical Joke Weather Machine. signature.asc Description: PGP signature
[gentoo-user] Re: Can't find reiser4 patch for kernel-2.6.39
On 07/21/2011 11:38 AM, Michael Orlitzky wrote: On 07/20/2011 07:33 PM, Volker Armin Hemmann wrote: Is it a matter of waiting a bit longer? Yes, I think he'll be eligible for parole beginning 2023. please refrain yourself from idiotic remarks like this. Everyone *knows* he's got full internet access, Stroller.sheesh. File a bug report and the warden will pass it his way. Given that he invented his own file system, I wonder why the prison bars are able to keep him locked up... If using a filesystem written by a murderer is wrong, I don't want to read/write! so you guys decided to jump on Stroller's bandwagon. Please, continue. Helps a lot when to decide which threads to ignore. Just trying to lighten the mood, don't take it the wrong way. So... why didn't he partition his wife if he didn't want her to be found?
Re: [gentoo-user] Kernel panics and more info
Neil Bothwick wrote: On Thu, 21 Jul 2011 10:46:55 -0500, Dale wrote: You could also try the VESA drivers, slow but reliable. OK. How do I do the VESA drivers? Honestly, the only trouble I can recall out of nvidia was upgrading the kernel then rebooting and realizing I forgot to rebuild against the new kernel. I don't recall every having anything like this. The biggest GUI problem I can recall was hal and xorg. Let's not go down that road. dale starts to steam Is VESA a kernel option or some package I need to install? It's the standard video driver, x11-drivers/xf86-video-vesa And I change nvidia to vesa or do I need to unmerge nvidia first? Also, are these done as modules like nvidia is? Hmmm, if I remove xorg.conf, how does it know which driver to use? The more I find out, the more questions I have. That's normal tho. lol Dale :-) :-)
Re: [gentoo-user] Kernel panics and more info
Dale asks: Neil Bothwick wrote: On Thu, 21 Jul 2011 06:41:31 -0500, Dale wrote: I have not been able to get the nv drivers to work. It has been so long since I had to use them, it appears I have forgot how to use them. I'm not sure I have ever used them since I been using Gentoo. I found the link on the xorg site. It wasn't very useful for me. I do have xf86-video-nv installed tho. I thought that was it. However, when I change the driver in xorg.conf to nv, no more GUI. Instead of editing xorg.conf, move it out of the way and let X figure things out for itself. You could also try the VESA drivers, slow but reliable. OK. How do I do the VESA drivers? Add 'vesa' to your VIDEO_CARDS variable in make.conf, then emerge -DautvN xorg-drivers. And change the Driver in xorg.conf from 'nvidia' to 'vesa'. Wonko
Re: [gentoo-user] Kernel panics and more info
On Thu, 21 Jul 2011 14:14:11 -0500, Dale wrote: It's the standard video driver, x11-drivers/xf86-video-vesa And I change nvidia to vesa or do I need to unmerge nvidia first? If you keep xorg.conf, change it to use vesa. Also, are these done as modules like nvidia is? Hmmm, if I remove xorg.conf, how does it know which driver to use? Hardware detection. If you don't use third party drivers, you can usually do without an xorg.conf. -- Neil Bothwick ASCII stupid question... get a stupid ANSI! signature.asc Description: PGP signature
Re: [gentoo-user] Kernel panics and more info
Alex Schuster wrote: Dale asks: Neil Bothwick wrote: On Thu, 21 Jul 2011 06:41:31 -0500, Dale wrote: I have not been able to get the nv drivers to work. It has been so long since I had to use them, it appears I have forgot how to use them. I'm not sure I have ever used them since I been using Gentoo. I found the link on the xorg site. It wasn't very useful for me. I do have xf86-video-nv installed tho. I thought that was it. However, when I change the driver in xorg.conf to nv, no more GUI. Instead of editing xorg.conf, move it out of the way and let X figure things out for itself. You could also try the VESA drivers, slow but reliable. OK. How do I do the VESA drivers? Add 'vesa' to your VIDEO_CARDS variable in make.conf, then emerge -DautvN xorg-drivers. And change the Driver in xorg.conf from 'nvidia' to 'vesa'. Wonko Thanks. I remembered after hitting reply to set it in make.conf. I got some done anyway. I'll report back what blows up. O_O Dale :-) :-)
Re: [gentoo-user] Kernel panics and more info
Dale wrote: Thanks. I remembered after hitting reply to set it in make.conf. I got some done anyway. I'll report back what blows up. O_O Dale :-) :-) OoooK. That didn't work to well. Using VESA, the screen was ALL messed up. It was mostly garbage to say it lightly. I also tried the nv driver again, all I got was a blinking cursor. I don't think it even tried to do anything. So, I can't see to do anything with VESA and nv appears to have not had any smoke to begin with. Can I shoot it now? Dale :-) :-)
Re: [gentoo-user] mysqld invoked oom-killer
On 7/21/2011 11:55 AM, Michael Mol wrote: On Thu, Jul 21, 2011 at 11:56 AM, kashanikashani-l...@badapple.net wrote: On 7/20/2011 6:29 PM, Michael Mol wrote: Also, run a caching proxy if at all possible. That made the single biggest difference for my server. Other useful things: * Set the MaxRequestsPerChild to something like 450. That's pretty low. You'd barely get your application parsed, cached, and load some data before you'd have to recycle the child process. Most people set it around 1. Large enough to be useful, but still deal with any minor memory leaks. Depends on your application. I had to set it low because the application wouldn't fit in a 540MB VPS, otherwise. I've since bumped up to a 2GB VPS, so I can probably afford Really, a caching proxy is the first, best thing, if it's not already in use. Let the thread carry on... Hey if it worked, but I think the thrash would be expensive in a normal system where you've got a sensible amount of RAM. I do like the reverse proxy idea. Turn Apache into an application server on localhost and let the reverse proxy deal with the Internet. If you picked the right proxy multiple requests could be collapsed, static files could be served directly, etc. kashani
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday 21 July 2011 09:39:52 Grant did opine thusly: My personal rule of thumb: if you hit swap, the bad thing has already gone very very south, usually to the point where you can't do much about it and it's already too late. Besides, that bastard deomon spawn of satan called the oom-killer is likely about to kick in and REALLY make your day. Anyone else notice how oom-killer seems to be hard coded to zap the most inconvenient process of all?. What you need to be doing is monitor your memory usage during normal conditions and deal with issues before they become problems. Hi Alan, I think it was your advice I took a long time ago when I stopped installing new machines with a swap partition and disabled it on my already-installed machines. Some time later, others on this list caught wind of what I'd done and told me I was an idiot. Is there a consensus on this? If the drawbacks and advantages of using swap cancel each other out, I won't use it. I would strongly advise you to make your own measurements and heed your own counsel. I can only speak from my own experience, and I may well be speaking a whole load of codswallop. Or I may be right and the opposing view is wrong. Who's to tell? My own experience with backing swap has been almost uniformly bad, especially on machines running Apache and MySQL due to the massive performance hit it invariably causes. I see memory as a finite resource - you only have so much of it, so use it wisely and stay away from using all of it up. The oom killer is also a point of contention. The algorithm is designed to try and detect the best pid to kill in order to keep the machine up, but there is no measurement for least important process. So instead it has to infer it from time last used, time running and various other bits. These assumptions can never be 100% right. I believe the best solution to running out of memory is well-written apps that degrade or die gracefully when they hit out of memory conditions. I always thought Apache was rather good at this, it would simply kill of the child process and deliver an error. You seem to have found a way round this :-) kashani's advice seems reasonable, tune your machine to suit it's load. We've established that the problem was a client hitting your webserver 300 times in a minute. That is a DOS, so the solution would be to find a way to configure Apache to detect abuse like this and not deliver the page. But back to swap. I have two cases where it is rather useful. This notebook uses swap for image storage when hibernating, and my Sybase database servers at work use swap well, retrieving data pages from swap is faster than searching through the database indexes for where they are, simply because the kernel alredy knows exactly where the swapped data is on disk. But this is a niche circumstance and in no way representative of a typical Linux machine's behaviour. Summary: Do your own tests, make your own conclusions and vigorously defend them. Sorry for the complete lack of a definitive answer, we are victims of TheRealWorldOutThere(tm) -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday 21 July 2011 10:30:21 Grant did opine thusly: [..] I think if you have 4GB of RAM you shouldn't need any swap under normal circumstances. I have a gentoo box with just 256MB of RAM that's running web server (apache + php), mail server (postfix + dovecot), and database (mariadb), and it works fine if i disable swap. I do normally have swap enabled on it, though, because emerging sometimes uses a lot of RAM. Indeed, I have a server with 256MB of memory and it couldn't emerge gcc when I had swap disabled. I can't compile openoffice with -j1 on 3GB RAM and no swap. Sounds like a case for a swap partition that can be activated when you need it for big emerges. I hit the same thing with firefox-5 oddly enough. As for OOo, long ago I figured the pain wasn't worth the gain so now I use the -bin packages. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday 21 July 2011 10:27:58 Grant did opine thusly: Thanks Paul. I'm leaning toward leaving swap disabled. So I'm sure I have the concept right, is adding a 1GB swap partition functionally identical to adding 1GB RAM with regard to the potential for out-of-memory conditions? Yep. It sounds like adding physical RAM is better than enabling swap in every way. I'll stay in the anti-swap camp. To throw a spanner in my own works: Some kernels *really* want at least some swap, even if it's just a little bit. IIRC it fits the role of a bit of wiggle room for when RAM is full. It came up the last time we all discussed this topic, but I no longer have those mails. If you can determine the appropriate amount, I'd recommend a small swap *file* if you decide to go this route. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Re: Can't find reiser4 patch for kernel-2.6.39
On 07/21/2011 11:58 AM, Nikos Chantziaras wrote: Just trying to lighten the mood, don't take it the wrong way. So... why didn't he partition his wife if he didn't want her to be found? I think his fragment size was too large.
Re: [gentoo-user] mysqld invoked oom-killer
Next I'd look at tuning your Mysql config. If you've never touched my.cnf, by default it's set to use 64MB IIRC. You may need to raise this to get better performance. key_buffer and innodb_buffer_pool_size are the only two I'd modify without knowing more. I use the default MyISAM tables and it looks like there are three key_buffer definitions in my.cnf. One under [mysqld] is 16M, one under [isamchk] is 20M, and one under [myisamchk] is 20M. All defaults. Should I increase them all to 64M? You can, but [mysqld] is the only one that matters for normal production. Depends on the size of your data and tables, but 64M is fine to start. If you've got a few GB in your databases I'd go with 256-512M or as high as you think you can get away with. Any reason you're still using MyISAM tables? Innodb is almost as fast or much much faster than MyISAM in nearly every way these days. Just because it's the default. I can imagine there's more to switching than flipping a bit and I haven't gotten around to it yet. I've been meaning to do it. - Grant
Re: [gentoo-user] mysqld invoked oom-killer
I ran into an out of memory problem. The first mention of it in the kernel log is mysqld invoked oom-killer. I haven't run into this before. I do have a swap partition but I don't activate it based on something I read previously that I later found out was wrong so I suppose I should activate it. Is fstab the way to do that? I have a commented line in there for swap. ... If you're running any other servers that utilize MySQL like Apache or something, check its access logs to see if you had an abnormal number of connections. Bruteforce hacking or some kind of flooding/DOS attack might cause it to use more memory than it ordinarily would. I don't know why I didn't check the apache2 error log before, but I got the following entry 2 seconds before the server became unresponsive: [error] server reached MaxClients setting, consider raising the MaxClients setting I use the default 256 for MaxClients. This confirms the server was brought down by too many child processes consuming too much memory. Looking back at the access_log, it's clear this condition was caused by the single IP which requested one of my pages about 300 times over the course of 1 minute. This caused my entire server to lock up for hours until I rebooted it. I hesitate to reduce MaxClients from 256. I think my server should be able to handle it since it's the default. So I need to prevent my apache2 child processes from consuming so much memory? apache2 was restarted about an hour before the lockup so it had a pretty fresh start. I do use mod_perl which is a memory hog from what I understand. Do I just need more RAM? Most people do not think about this correctly. Can your server run 1 Apache processes? No, not enough resources. 1000? No, same problem. 256? I'd say no based on this thread. If you're not going to set it at 1 why try to keep it at 256? This is illuminating, thank you. I wonder why the default is set at 256 though. Do my apache2 processes use an unusual amount of memory at 25M-67M? Even at 25M each, you would need 6.4GB of memory for 256 processes. Is an apache2 restart the only thing that prevents apache2 processes from growing in memory usage indefinitely? - Grant Next image a grocery store with 256 checkout lanes, but only four cashiers. Four cashiers trying to run that many lanes is actually slower than having only four lanes. However 32 lanes could faster than 4. People can have their groceries setup, baggers aren't getting in the way, etc. The analogy breaks down a bit, but you get the point. There is no performance gain in configuring for concurrency your hardware and software can not support. kashani
Re: [gentoo-user] mysqld invoked oom-killer
Also, run a caching proxy if at all possible. That made the single biggest difference for my server. Other useful things: * Set the MaxRequestsPerChild to something like 450. That's pretty low. You'd barely get your application parsed, cached, and load some data before you'd have to recycle the child process. Most people set it around 1. Large enough to be useful, but still deal with any minor memory leaks. Depends on your application. I had to set it low because the application wouldn't fit in a 540MB VPS, otherwise. I've since bumped up to a 2GB VPS, so I can probably afford Really, a caching proxy is the first, best thing, if it's not already in use. Let the thread carry on... A cache makes sense, but I hesitate to establish a new layer. How is the setup and maintenance? I found this which is at least not super-easy: http://www.apachetutor.org/admin/reverseproxies Would I be able to make a change to an image, immediately refresh, and see the change? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
Next I'd look at tuning your Mysql config. If you've never touched my.cnf, by default it's set to use 64MB IIRC. You may need to raise this to get better performance. key_buffer and innodb_buffer_pool_size are the only two I'd modify without knowing more. I use the default MyISAM tables and it looks like there are three key_buffer definitions in my.cnf. One under [mysqld] is 16M, one under [isamchk] is 20M, and one under [myisamchk] is 20M. All defaults. Should I increase them all to 64M? You can, but [mysqld] is the only one that matters for normal production. Depends on the size of your data and tables, but 64M is fine to start. If you've got a few GB in your databases I'd go with 256-512M or as high as you think you can get away with. Any reason you're still using MyISAM tables? Innodb is almost as fast or much much faster than MyISAM in nearly every way these days. Can multiple processes be utilized for mysql like they are for apache2? Perhaps not since it's a database? - Grant
Re: [gentoo-user] Kernel panics and more info
On Thu, 21 Jul 2011 15:18:34 -0500, Dale wrote about Re: [gentoo-user] Kernel panics and more info: OoooK. That didn't work to well. Using VESA, the screen was ALL messed up. It was mostly garbage to say it lightly. I also tried the nv driver again, all I got was a blinking cursor. I don't think it even tried to do anything. When you switch away from the proprietary drivers you need to do an eselect opengl to switch the 3D rendering to use Mesa (X.Org 's library). Thus, to get the nv driver working: Ensure there is no frame buffer driver loaded by the kernel; ** Update xorg.conf to have the nv driver loaded in a Device section; Update xorg.conf to have the nv Device related to a Screen section; eselect opengl set 2 (or whatever number for Mesa); /etc/init.d/xdm restart (or reboot, or whatever). ** This is very important! The nv driver does not like any other driver blowing on the same trumpet at the same time -- it's unhygienic. I am currently running the nv driver on this box using a very elderly GeForce2 MX-400 GPU. It runs rather well and isn't discernably slower than the proprietary driver for most workloads. So, I can't see to do anything with VESA and nv appears to have not had any smoke to begin with. You should be getting error messages in /var/log/Xorg.0.log if things are going wrong. Can I shoot it now? Up to you. It wouldn't be legal in this country, as we aren't allowed to own guns. -- Regards, Dave [RLU #314465] *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* dwn...@ntlworld.com (David W Noon) *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* signature.asc Description: PGP signature
[gentoo-user] Re: Kernel panics and more info
On 07/21/2011 01:18 PM, Dale wrote: Dale wrote: Using VESA, the screen was ALL messed up. It was mostly garbage to say it lightly. I also tried the nv driver again, all I got was a blinking cursor. I don't think it even tried to do anything. If you have your opengl set to nvidia, you might try changing it back to xorg-x11. eselect opengl list
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday, July 21 at 10:27 (-0700), Grant said: It sounds like adding physical RAM is better than enabling swap in every way. I'll stay in the anti-swap camp. I don't see why it has to be one way *or* the other... Yes more RAM is always going to be better than more swap, RAM is just way faster than disk, however byte-per-byte, disk is cheaper. The whole reason why we have swap.. back in the olden days, some programs needed more RAM than perhaps the system provided. Some of these program were written with this in mind, and actually handled this by manually writing some of it's data to disk, then freeing that data from RAM, doing something, then when it needed the disk data, reading it back into RAM (after having freed the previous data). This is of course cumbersome. Enter virtual memory operating systems, which basically treat fast memory (RAM) and slow memory (disk) as one flat memory pool. Then the program it all looks like memory, and the OS does the paging in and out to disk. By now you would think oh, but if I just had one system that had a more RAM than i would ever use simultaneously, then I don't need swap, right? Well, not exactly, because modern operationg systems also do something called filesystem caching. What this does is, recently, and often used (parts of) files on the filesystem are kept into fast RAM, so when a program needs that data it can be fetched from cached RAM instead of hitting slower disk. Ok, that's nice, but what does that have to do with swap? Well, not only does Linux keep track of wIt sounds like adding physical RAM is better than enabling swap in every way. I'll stay in the anti-swap camp.hat files are used often, it also keeps track of what pages of virtual memory are *not* used often. Say you started some program a long time ago, or some program launches at boot time, but that program sleeps and doesn't do anything for a long time. Now ordinarily that program would just sit there taking up RAM. Now you are running some other programs, and these programs are very actively hitting the disk. Now Linux would love to use more RAM for caching those disk hits, but that program you haven't touched in hours is taking up RAM doing nothing. That's where swap comes in. Linux would like to take that sleeping process and swap some of its pages out to disk, so it can use the freed RAM for more cache, and therefore speed up the programs that are actually being used. Then, if the barely used process ever does wake up, Linux can expire some cache and put it back into RAM. In this case swap *is* good because it's making more efficient use of RAM by swapping out seldom-used processes and using that RAM to cache often-accessed files. So a healthy combination of swap and RAM *can* be a good thing. If, however, you have so much RAM that you can run every program you'd ever run simultaneously with every file you'd ever access cached in RAM then I wouldn't worry about swap ;) -a
[gentoo-user] Re: Can't kill Firefox!
On 07/21/2011 11:38 AM, Mick wrote: Whatever Dale had on his machine must have infected mine! LOL! A 32bit x86 box with KDE4.6, running firefox-3.6.17 and xulrunner-1.9.2.17 after a few hours and loads of tabs (sometimes up to 15 or so) eventually hangs X. I can switch to a console and login as the user running the X session, then try to kill the bloody thing with killall, kill -15 pid and kill -9 pid. It won't barge. Mind you I haven't tried this as root. I wonder if the machine is busy swapping because RAM is full. Run top and see how much swap is used or if some process is using 100% of CPU.
[gentoo-user] Re: New computer and Gentoo
On 07/20/2011 10:54 PM, Bill Kenworthy wrote: So not sure about march=native now as it is only what was built with native thats been problematic. Makes me wonder if gcc and glibc need to be recompiled with arch=native before rebuilding the rest of the system?
Re: [gentoo-user] mysqld invoked oom-killer
On 7/21/2011 2:50 PM, Grant wrote: Any reason you're still using MyISAM tables? Innodb is almost as fast or much much faster than MyISAM in nearly every way these days. Can multiple processes be utilized for mysql like they are for apache2? Perhaps not since it's a database? Mysql is multithreaded and spawns a thread for each connection. Try a ps -efL and you should see a number of Mysql threads. However that is part of the problem with MyISAM. It throws a giant table lock blocking all other threads until the SQL statement is complete. Innodb uses row locks which allows the other threads to use the table. As far as moving to Innodb tables it's actually easy, but with a number of caveats. I'd lower your Apache max clients, tweak my.cnf, and runs some load tests before getting deep into Mysql. When you're ready I'd go about this way. 1. Make backups first. 2. See if you have any full text fields. Tables with full text fields will have to remain MyISAM. 3. Dump your database out to text. If it's not a huge amount of data I'd just vi it and change the ENGINE to Innodb. Then import the whole thing as a new database. If you have a lot of data, I'd dump the schema with -d edit, import schema, then dump your data with no create statements and finally import the data into the new database. 4. Point your staging code to the new database and test 5. Plan a maintenance window to do all the above and take the site offline while you reimport the data to be Innodb 6. take the RAM you gave to key_buffer and give it to innodb. Storage engines do not share buffers in Mysql. You can alter tables in place, but it locks them for the duration. If you site is small and low traffic you could get away with it, but testing with a copy of your site database is better. kashani
Re: [gentoo-user] Re: Can't find reiser4 patch for kernel-2.6.39
On Thu, 21 Jul 2011 14:13:11 -0700, Bill Longman wrote: Just trying to lighten the mood, don't take it the wrong way. So... why didn't he partition his wife if he didn't want her to be found? I think his fragment size was too large. I thought she disappeared without a trace, but that sounds more like XFS... -- Neil Bothwick Top Oxymorons Number 36: Alone together signature.asc Description: PGP signature
Re: [gentoo-user] Re: Kernel panics and more info
walt wrote: On 07/21/2011 01:18 PM, Dale wrote: Dale wrote: Using VESA, the screen was ALL messed up. It was mostly garbage to say it lightly. I also tried the nv driver again, all I got was a blinking cursor. I don't think it even tried to do anything. If you have your opengl set to nvidia, you might try changing it back to xorg-x11. eselect opengl list Thanks. I'm going to go shoot myself in the foot now. lol Let me go test this again. BRB Dale :-) :-)
Re: [gentoo-user] mysqld invoked oom-killer
On 07/21/2011 04:49 PM, Alan McKinnon wrote: On Thursday 21 July 2011 10:27:58 Grant did opine thusly: Thanks Paul. I'm leaning toward leaving swap disabled. So I'm sure I have the concept right, is adding a 1GB swap partition functionally identical to adding 1GB RAM with regard to the potential for out-of-memory conditions? Yep. It sounds like adding physical RAM is better than enabling swap in every way. I'll stay in the anti-swap camp. To throw a spanner in my own works: Some kernels *really* want at least some swap, even if it's just a little bit. IIRC it fits the role of a bit of wiggle room for when RAM is full. I was waiting for this =) Alan's previous advice (basically, everything should fit in RAM these days) is only true in a world where the VM doesn't occasionally make stupid decisions. In real life...
Re: [gentoo-user] mysqld invoked oom-killer
... I would strongly advise you to make your own measurements and heed your own counsel. I can only speak from my own experience, and I may well be speaking a whole load of codswallop. Or I may be right and the opposing view is wrong. Who's to tell? My own experience with backing swap has been almost uniformly bad, especially on machines running Apache and MySQL due to the massive performance hit it invariably causes. I see memory as a finite resource - you only have so much of it, so use it wisely and stay away from using all of it up. The oom killer is also a point of contention. The algorithm is designed to try and detect the best pid to kill in order to keep the machine up, but there is no measurement for least important process. So instead it has to infer it from time last used, time running and various other bits. These assumptions can never be 100% right. I believe the best solution to running out of memory is well-written apps that degrade or die gracefully when they hit out of memory conditions. I always thought Apache was rather good at this, it would simply kill of the child process and deliver an error. You seem to have found a way round this :-) I'd like to know more about that if anyone has info on it. apache2 is supposed to detect when the system runs out of memory and proceed to kill child processes? It failed to do so on my system which then proceeded to lock up. kashani's advice seems reasonable, tune your machine to suit it's load. We've established that the problem was a client hitting your webserver 300 times in a minute. That is a DOS, so the solution would be to find a way to configure Apache to detect abuse like this and not deliver the page. So any apache system that hasn't been configured with a special security module can be locked up by refreshing a page a suitable number of times within a suitable amount of time? But back to swap. I have two cases where it is rather useful. This notebook uses swap for image storage when hibernating, and my Sybase database servers at work use swap well, retrieving data pages from swap is faster than searching through the database indexes for where they are, simply because the kernel alredy knows exactly where the swapped data is on disk. But this is a niche circumstance and in no way representative of a typical Linux machine's behaviour. In situations like the above, would you prefer a swap file to a swap partition? It sounds better to me. Summary: Do your own tests, make your own conclusions and vigorously defend them. Sorry for the complete lack of a definitive answer, we are victims of TheRealWorldOutThere(tm) I can handle that. :) - Grant
Re: [gentoo-user] Re: Can't find reiser4 patch for kernel-2.6.39
On 07/21/2011 02:58 PM, Nikos Chantziaras wrote: On 07/21/2011 11:38 AM, Michael Orlitzky wrote: On 07/20/2011 07:33 PM, Volker Armin Hemmann wrote: Is it a matter of waiting a bit longer? Yes, I think he'll be eligible for parole beginning 2023. please refrain yourself from idiotic remarks like this. Everyone *knows* he's got full internet access, Stroller.sheesh. File a bug report and the warden will pass it his way. Given that he invented his own file system, I wonder why the prison bars are able to keep him locked up... If using a filesystem written by a murderer is wrong, I don't want to read/write! so you guys decided to jump on Stroller's bandwagon. Please, continue. Helps a lot when to decide which threads to ignore. Just trying to lighten the mood, don't take it the wrong way. So... why didn't he partition his wife if he didn't want her to be found? Did Hans keep a journal? If so, maybe they can use it to recover his lost wife...
Re: [gentoo-user] mysqld invoked oom-killer
[..] I think if you have 4GB of RAM you shouldn't need any swap under normal circumstances. I have a gentoo box with just 256MB of RAM that's running web server (apache + php), mail server (postfix + dovecot), and database (mariadb), and it works fine if i disable swap. I do normally have swap enabled on it, though, because emerging sometimes uses a lot of RAM. Indeed, I have a server with 256MB of memory and it couldn't emerge gcc when I had swap disabled. I can't compile openoffice with -j1 on 3GB RAM and no swap. Sounds like a case for a swap partition that can be activated when you need it for big emerges. I hit the same thing with firefox-5 oddly enough. Swap file just as good? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday 21 July 2011 16:27:03 Grant did opine thusly: [..] I think if you have 4GB of RAM you shouldn't need any swap under normal circumstances. I have a gentoo box with just 256MB of RAM that's running web server (apache + php), mail server (postfix + dovecot), and database (mariadb), and it works fine if i disable swap. I do normally have swap enabled on it, though, because emerging sometimes uses a lot of RAM. Indeed, I have a server with 256MB of memory and it couldn't emerge gcc when I had swap disabled. I can't compile openoffice with -j1 on 3GB RAM and no swap. Sounds like a case for a swap partition that can be activated when you need it for big emerges. I hit the same thing with firefox-5 oddly enough. Swap file just as good? Yeah. If the docs are true, then swap files and partitions give similar performance these days. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday 21 July 2011 19:19:07 Michael Orlitzky did opine thusly: On 07/21/2011 04:49 PM, Alan McKinnon wrote: On Thursday 21 July 2011 10:27:58 Grant did opine thusly: Thanks Paul. I'm leaning toward leaving swap disabled. So I'm sure I have the concept right, is adding a 1GB swap partition functionally identical to adding 1GB RAM with regard to the potential for out-of-memory conditions? Yep. It sounds like adding physical RAM is better than enabling swap in every way. I'll stay in the anti-swap camp. To throw a spanner in my own works: Some kernels *really* want at least some swap, even if it's just a little bit. IIRC it fits the role of a bit of wiggle room for when RAM is full. I was waiting for this =) Alan's previous advice (basically, everything should fit in RAM these days) is only true in a world where the VM doesn't occasionally make stupid decisions. In real life... :-) You caught me out. As I was typing all these posts today I was having these haunting thoughts about RealLife -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Re: Kernel panics and more info
Dale wrote: walt wrote: On 07/21/2011 01:18 PM, Dale wrote: Dale wrote: Using VESA, the screen was ALL messed up. It was mostly garbage to say it lightly. I also tried the nv driver again, all I got was a blinking cursor. I don't think it even tried to do anything. If you have your opengl set to nvidia, you might try changing it back to xorg-x11. eselect opengl list Thanks. I'm going to go shoot myself in the foot now. lol Let me go test this again. BRB Dale :-) :-) I have now. Same thing tho. I tried both vesa and nv. I only have this to select from tho. root@fireball / # eselect opengl list Available OpenGL implementations: [1] nvidia * [2] xorg-x11 root@fireball / # I set it to xorg's but still the same. I even tried logging into KDE but it was a mess. Maybe I can just find me a different download tool for Seamonkey. See if that works any better. Sure would like to nail this down tho. Dale :-) :-)
Re: [gentoo-user] mysqld invoked oom-killer
It sounds like adding physical RAM is better than enabling swap in every way. I'll stay in the anti-swap camp. I don't see why it has to be one way *or* the other... Yes more RAM is always going to be better than more swap, RAM is just way faster than disk, however byte-per-byte, disk is cheaper. The whole reason why we have swap.. back in the olden days, some programs needed more RAM than perhaps the system provided. Some of these program were written with this in mind, and actually handled this by manually writing some of it's data to disk, then freeing that data from RAM, doing something, then when it needed the disk data, reading it back into RAM (after having freed the previous data). This is of course cumbersome. Enter virtual memory operating systems, which basically treat fast memory (RAM) and slow memory (disk) as one flat memory pool. Then the program it all looks like memory, and the OS does the paging in and out to disk. By now you would think oh, but if I just had one system that had a more RAM than i would ever use simultaneously, then I don't need swap, right? Well, not exactly, because modern operationg systems also do something called filesystem caching. What this does is, recently, and often used (parts of) files on the filesystem are kept into fast RAM, so when a program needs that data it can be fetched from cached RAM instead of hitting slower disk. Ok, that's nice, but what does that have to do with swap? Well, not only does Linux keep track of what files are used often, it also keeps track of what pages of virtual memory are *not* used often. Say you started some program a long time ago, or some program launches at boot time, but that program sleeps and doesn't do anything for a long time. Now ordinarily that program would just sit there taking up RAM. Now you are running some other programs, and these programs are very actively hitting the disk. Now Linux would love to use more RAM for caching those disk hits, but that program you haven't touched in hours is taking up RAM doing nothing. That's where swap comes in. Linux would like to take that sleeping process and swap some of its pages out to disk, so it can use the freed RAM for more cache, and therefore speed up the programs that are actually being used. Then, if the barely used process ever does wake up, Linux can expire some cache and put it back into RAM. In this case swap *is* good because it's making more efficient use of RAM by swapping out seldom-used processes and using that RAM to cache often-accessed files. So a healthy combination of swap and RAM *can* be a good thing. If, however, you have so much RAM that you can run every program you'd ever run simultaneously with every file you'd ever access cached in RAM then I wouldn't worry about swap ;) So swap isn't treated exactly like RAM. It actually has special handling in Linux which makes it beneficial to have on almost any Linux system? According to Alan, things get very bad when a Linux system hits swap. How can behavior like this be beneficial: When a linux machine hits swap, it does so very aggressively, there is nothing nice about it at all. The entire machine slows to a painstaking crawl for easily a minute at a time while the kernel writes pages out to disk, and disk is thousands of times slower than RAM. It gets so bad that you can't even run a shell properly to try and see what's going on and kill the actual memory hog. Also, aren't you likely to wear out your hard disk sooner using swap? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
Any reason you're still using MyISAM tables? Innodb is almost as fast or much much faster than MyISAM in nearly every way these days. Can multiple processes be utilized for mysql like they are for apache2? Perhaps not since it's a database? Mysql is multithreaded and spawns a thread for each connection. Try a ps -efL and you should see a number of Mysql threads. However that is part of the problem with MyISAM. It throws a giant table lock blocking all other threads until the SQL statement is complete. Innodb uses row locks which allows the other threads to use the table. As far as moving to Innodb tables it's actually easy, but with a number of caveats. I'd lower your Apache max clients, tweak my.cnf, and runs some load tests before getting deep into Mysql. When you're ready I'd go about this way. apache MaxClients has been lowered to 50 which is a shame because I have 30+ separate images on each of my pages and that number can not be reduced. This means I may not be able to serve more than 1 full page at a time. 1. Make backups first. 2. See if you have any full text fields. Tables with full text fields will have to remain MyISAM. Many of my tables have one or more fields defined as TEXT out of laziness. Should I instead come up with an appropriate char(N) declaration for each? Can N go as high as necessary? 3. Dump your database out to text. If it's not a huge amount of data I'd just vi it and change the ENGINE to Innodb. Then import the whole thing as a new database. If you have a lot of data, I'd dump the schema with -d edit, import schema, then dump your data with no create statements and finally import the data into the new database. 4. Point your staging code to the new database and test 5. Plan a maintenance window to do all the above and take the site offline while you reimport the data to be Innodb 6. take the RAM you gave to key_buffer and give it to innodb. Storage engines do not share buffers in Mysql. OK, just leave key_buffer at the default 16M? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
On 7/21/2011 4:53 PM, Grant wrote: So swap isn't treated exactly like RAM. It actually has special handling in Linux which makes it beneficial to have on almost any Linux system? According to Alan, things get very bad when a Linux system hits swap. How can behavior like this be beneficial: When a linux machine hits swap, it does so very aggressively, there is nothing nice about it at all. The entire machine slows to a painstaking crawl for easily a minute at a time while the kernel writes pages out to disk, and disk is thousands of times slower than RAM. It gets so bad that you can't even run a shell properly to try and see what's going on and kill the actual memory hog. Also, aren't you likely to wear out your hard disk sooner using swap? 1. swap is good. Unless you have a good reason, leave it there. You do not have a good reason to remove it and neither does anyone else. 2. Don't use the swap that you have. It's slow. It is not a replacement for RAM. 3. If you use a little bit of swap, 100-200MB, that's fine. It's also a sign you need more RAM. 4. If you're using all your RAM and a couple of GB of swap, you're screwed. Avoid this. 5. Swap that you never write to or read from never needs to hit the drives. If you're worried about drive wear, turn off logging. kashani
Re: [gentoo-user] mysqld invoked oom-killer
On 7/21/2011 5:14 PM, Grant wrote: Any reason you're still using MyISAM tables? Innodb is almost as fast or much much faster than MyISAM in nearly every way these days. Can multiple processes be utilized for mysql like they are for apache2? Perhaps not since it's a database? Mysql is multithreaded and spawns a thread for each connection. Try a ps -efL and you should see a number of Mysql threads. However that is part of the problem with MyISAM. It throws a giant table lock blocking all other threads until the SQL statement is complete. Innodb uses row locks which allows the other threads to use the table. As far as moving to Innodb tables it's actually easy, but with a number of caveats. I'd lower your Apache max clients, tweak my.cnf, and runs some load tests before getting deep into Mysql. When you're ready I'd go about this way. apache MaxClients has been lowered to 50 which is a shame because I have 30+ separate images on each of my pages and that number can not be reduced. This means I may not be able to serve more than 1 full page at a time. This is wrong. 1. Make backups first. 2. See if you have any full text fields. Tables with full text fields will have to remain MyISAM. Many of my tables have one or more fields defined as TEXT out of laziness. Should I instead come up with an appropriate char(N) declaration for each? Can N go as high as necessary? TEXT fields don't matter, FULL TEXT indexes do. Sorry my mistake. OK, just leave key_buffer at the default 16M? No. Make key_buffer 256M and then restart Mysql or update it from the commandline. You're starving Mysql for resources. Fix this first. Then you can mess around with tables and engines. kashani
Re: [gentoo-user] mysqld invoked oom-killer
So swap isn't treated exactly like RAM. It actually has special handling in Linux which makes it beneficial to have on almost any Linux system? According to Alan, things get very bad when a Linux system hits swap. How can behavior like this be beneficial: When a linux machine hits swap, it does so very aggressively, there is nothing nice about it at all. The entire machine slows to a painstaking crawl for easily a minute at a time while the kernel writes pages out to disk, and disk is thousands of times slower than RAM. It gets so bad that you can't even run a shell properly to try and see what's going on and kill the actual memory hog. Also, aren't you likely to wear out your hard disk sooner using swap? 1. swap is good. Unless you have a good reason, leave it there. You do not have a good reason to remove it and neither does anyone else. 2. Don't use the swap that you have. It's slow. It is not a replacement for RAM. 3. If you use a little bit of swap, 100-200MB, that's fine. It's also a sign you need more RAM. 4. If you're using all your RAM and a couple of GB of swap, you're screwed. Avoid this. 5. Swap that you never write to or read from never needs to hit the drives. If you're worried about drive wear, turn off logging. kashani OK, how about I enable a 512MB swap file and keep an eye on it. As long as I'm not using more than 200MB, I'm not suffering from disk swap slowdown, right? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
apache MaxClients has been lowered to 50 which is a shame because I have 30+ separate images on each of my pages and that number can not be reduced. This means I may not be able to serve more than 1 full page at a time. This is wrong. Agreed. From TFM; The MaxClients directive sets the limit on the number of simultaneous requests that will be served and i'd say when they say requests, they're talking about TCP sessions. So in the old days of HTTP/1.0 you'd be right, and if you'd turned off pipelining (KeepAlives) you'd be right. The default for MaxKeepAliveRequests is 100, so no problems downloading the 30+ objects within a single session, assuming you have KeepAlive on.
Re: [gentoo-user] mysqld invoked oom-killer
Any reason you're still using MyISAM tables? Innodb is almost as fast or much much faster than MyISAM in nearly every way these days. Can multiple processes be utilized for mysql like they are for apache2? Perhaps not since it's a database? Mysql is multithreaded and spawns a thread for each connection. Try a ps -efL and you should see a number of Mysql threads. However that is part of the problem with MyISAM. It throws a giant table lock blocking all other threads until the SQL statement is complete. Innodb uses row locks which allows the other threads to use the table. As far as moving to Innodb tables it's actually easy, but with a number of caveats. I'd lower your Apache max clients, tweak my.cnf, and runs some load tests before getting deep into Mysql. When you're ready I'd go about this way. apache MaxClients has been lowered to 50 which is a shame because I have 30+ separate images on each of my pages and that number can not be reduced. This means I may not be able to serve more than 1 full page at a time. This is wrong. MaxClients is defined as the limit on the number of simultaneous requests that will be served. If each of my pages requires 30 requests in order to be fully served, I don't think I'll be able to fully serve more than one page at a time. 1. Make backups first. 2. See if you have any full text fields. Tables with full text fields will have to remain MyISAM. Many of my tables have one or more fields defined as TEXT out of laziness. Should I instead come up with an appropriate char(N) declaration for each? Can N go as high as necessary? TEXT fields don't matter, FULL TEXT indexes do. Sorry my mistake. I don't have any FULL TEXT. Since TEXT is alright, I will be making the switch to InnoDB very soon. OK, just leave key_buffer at the default 16M? No. Make key_buffer 256M and then restart Mysql or update it from the commandline. You're starving Mysql for resources. Fix this first. Then you can mess around with tables and engines. Yep, I increased key_buffer when you told me to previously. - Grant
Re: [gentoo-user] mysqld invoked oom-killer
OK, how about I enable a 512MB swap file and keep an eye on it. As long as I'm not using more than 200MB, I'm not suffering from disk swap slowdown, right? Its more how much i/o rather than the size. If you have a bunch of stuff swapped out, but it hardly ever needs to be swapped in, the impact will be low. Keep an eye on the use with vmstat; adam@rix ~ $ vmstat 5 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa 0 0 56700 351244 79564 20784800 3 3 117 1 0 99 0 0 0 56700 351244 79564 20784800 0 8 52 27 0 0 100 0 0 0 56700 351244 79564 20784800 0 0 45 14 0 0 100 0 0 0 56700 351244 79564 20784800 0 0 47 17 0 0 100 0 from the man page; Swap si: Amount of memory swapped in from disk (/s). so: Amount of memory swapped to disk (/s).
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday, July 21 at 16:53 (-0700), Grant said: So swap isn't treated exactly like RAM. It actually has special handling in Linux which makes it beneficial to have on almost any Linux system? According to Alan, things get very bad when a Linux system hits swap. How can behavior like this be beneficial: When a linux machine hits swap, it does so very aggressively, there is nothing nice about it at all. The entire machine slows to a painstaking crawl for easily a minute at a time while the kernel writes pages out to disk, and disk is thousands of times slower than RAM. This is not entirely true. There's regular swapping and there is thrashing. Thrashing is indicative of a memory-starved system, i.e. when many processes are trying to access memory, but there just isn't enough and the system is frantically swapping in/out. I'm talking about your normal day-to-day swapping that you probably don't even notice. It gets so bad that you can't even run a shell properly to try and see what's going on and kill the actual memory hog. Again, that is thrashing. I'm talking about normal swappage. Dont throw the baby out with the bath water. Also, aren't you likely to wear out your hard disk sooner using swap? Is this coming from someone who uses Gentoo linux, which is constantly downloading/compiling/linking object files? Syslog and other loggers writing everything under the sun to a log file. Backups, journal writes, database transactions, etc. Compare how many disk transactions take place during your normal Gentoo usage versus a few megabytes here/there being swapped in/out. Again, I'm talking about regular swapping, not oh my god I has no RAM and my hard drive won't stop Even so, we're talking about modern drives here. This isn't the 1960s.
Re: [gentoo-user] mysqld invoked oom-killer
On Friday, July 22 at 10:56 (+1000), Adam Carter said: Its more how much i/o rather than the size. If you have a bunch of stuff swapped out, but it hardly ever needs to be swapped in, the impact will be low. Keep an eye on the use with vmstat; adam@rix ~ $ vmstat 5 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa 0 0 56700 351244 79564 20784800 3 3 117 1 0 99 0 0 0 56700 351244 79564 20784800 0 8 52 27 0 0 100 0 0 0 56700 351244 79564 20784800 0 0 45 14 0 0 100 0 0 0 56700 351244 79564 20784800 0 0 47 17 0 0 100 0 from the man page; Swap si: Amount of memory swapped in from disk (/s). so: Amount of memory swapped to disk (/s). Exactly! My system is the same way. Right now I've got a 4GB system that's using 708MB swap. But vmstat isn't showing any swap activity. Why? Because some processes that I'm not aware about because I'm obviously not using, got swapped out a long time ago, and Linux is using that reclaimed RAM to compile chromium ;) If/when I need part of that 708MB becomes active, Linux will swap it back in in one short burst that I doubt that I'll even notice.
Re: [gentoo-user] mysqld invoked oom-killer
apache MaxClients has been lowered to 50 which is a shame because I have 30+ separate images on each of my pages and that number can not be reduced. This means I may not be able to serve more than 1 full page at a time. This is wrong. Agreed. From TFM; The MaxClients directive sets the limit on the number of simultaneous requests that will be served and i'd say when they say requests, they're talking about TCP sessions. So in the old days of HTTP/1.0 you'd be right, and if you'd turned off pipelining (KeepAlives) you'd be right. The default for MaxKeepAliveRequests is 100, so no problems downloading the 30+ objects within a single session, assuming you have KeepAlive on. I'm trying to figure out the maximum number of apache2 processes that could run simultaneously according to my config so I don't run out of memory again. I have KeepAlive on, but I can see in the log that a different pid serves each file associated with a particular page request. Doesn't that mean a different apache2 process is serving each file and I need one process for each file served at any particular moment? How does KeepAlive relate to the number of running apache2 processes? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
Its more how much i/o rather than the size. If you have a bunch of stuff swapped out, but it hardly ever needs to be swapped in, the impact will be low. Keep an eye on the use with vmstat; adam@rix ~ $ vmstat 5 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 56700 351244 79564 207848 0 0 3 3 11 7 1 0 99 0 0 0 56700 351244 79564 207848 0 0 0 8 52 27 0 0 100 0 0 0 56700 351244 79564 207848 0 0 0 0 45 14 0 0 100 0 0 0 56700 351244 79564 207848 0 0 0 0 47 17 0 0 100 0 from the man page; Swap si: Amount of memory swapped in from disk (/s). so: Amount of memory swapped to disk (/s). Exactly! My system is the same way. Right now I've got a 4GB system that's using 708MB swap. But vmstat isn't showing any swap activity. Why? Because some processes that I'm not aware about because I'm obviously not using, got swapped out a long time ago, and Linux is using that reclaimed RAM to compile chromium ;) If/when I need part of that 708MB becomes active, Linux will swap it back in in one short burst that I doubt that I'll even notice. Then why not have a really big swap file? If swap is useful as a second layer of caching behind RAM, why doesn't everyone with some extra hard drive space have a 100GB swap file? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
I'm trying to figure out the maximum number of apache2 processes that could run simultaneously according to my config so I don't run out of memory again. I have KeepAlive on, but I can see in the log that a different pid serves each file associated with a particular page request. Ok, I didnt not expect that, and am now questioning my own understanding. Perhaps each process can only handle a single get at once, so if you're pipelining you have to use multiple processes to support the request. How did you determine each file was served from a different PID? What browser did you use to test? FWIW Firefox only sends 6 gets per persistent session. Doesn't that mean a different apache2 process is serving each file and I need one process for each file served at any particular moment? Certainly sounds like it. How does KeepAlive relate to the number of running apache2 processes? Not directly related AFAIK.
Re: [gentoo-user] mysqld invoked oom-killer
So swap isn't treated exactly like RAM. It actually has special handling in Linux which makes it beneficial to have on almost any Linux system? According to Alan, things get very bad when a Linux system hits swap. How can behavior like this be beneficial: When a linux machine hits swap, it does so very aggressively, there is nothing nice about it at all. The entire machine slows to a painstaking crawl for easily a minute at a time while the kernel writes pages out to disk, and disk is thousands of times slower than RAM. This is not entirely true. There's regular swapping and there is thrashing. Thrashing is indicative of a memory-starved system, i.e. when many processes are trying to access memory, but there just isn't enough and the system is frantically swapping in/out. I'm talking about your normal day-to-day swapping that you probably don't even notice. It gets so bad that you can't even run a shell properly to try and see what's going on and kill the actual memory hog. Again, that is thrashing. I'm talking about normal swappage. Dont throw the baby out with the bath water. Also, aren't you likely to wear out your hard disk sooner using swap? Is this coming from someone who uses Gentoo linux, which is constantly downloading/compiling/linking object files? Syslog and other loggers writing everything under the sun to a log file. Backups, journal writes, database transactions, etc. Compare how many disk transactions take place during your normal Gentoo usage versus a few megabytes here/there being swapped in/out. Again, I'm talking about regular swapping, not oh my god I has no RAM and my hard drive won't stop Even so, we're talking about modern drives here. This isn't the 1960s. If I understand correctly, an out-of-memory condition that would lock up a system without swap, will cause it to thrash with swap. A remote system of mine was locked up for many hours due to running out of memory without swap. If I had enabled swap, the system would have thrashed for those hours? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
I'm trying to figure out the maximum number of apache2 processes that could run simultaneously according to my config so I don't run out of memory again. I have KeepAlive on, but I can see in the log that a different pid serves each file associated with a particular page request. Ok, I didnt not expect that, and am now questioning my own understanding. Perhaps each process can only handle a single get at once, so if you're pipelining you have to use multiple processes to support the request. How did you determine each file was served from a different PID? What browser did you use to test? FWIW Firefox only sends 6 gets per persistent session. My apologies. I assumed the seemingly random numbers that appear at the end of my access_log entries were PIDs but they are in fact the Size of response in bytes. I'm sure your understanding of KeepAlive is correct. So with KeepAlive on, the same apache2 process serves the page itself and all associated files? - Grant
Re: [gentoo-user] mysqld invoked oom-killer
On Thursday, July 21 at 18:29 (-0700), Grant said: Then why not have a really big swap file? If swap is useful as a second layer of caching behind RAM, why doesn't everyone with some extra hard drive space have a 100GB swap file? You've not understood what I said, I think. Swap is not useful as filesystem cache. Swap is as efficient (probably a little less) than the files on the disk. It's RAM that's efficient as filesystem cache. Where swap comes in is the kernel can swap out pages from stale processes, and reclaim the RAM as filesystem cache. Think of it this way: You have a house with an attic. Now the attic is not as efficient as say, the middle of your living room. You have a Christmas tree, but you only use that Christmas tree maybe once a year. Now it's much more efficient to keep that Christmas tree in the attic for 11 months of the year and use that reclaimed space in your living room for.. say a coffee table. Then, when you need that Christmas tree in December, you pull it out of the attic and maybe put the coffee table up in the attic for a month. The Christmas tree represents a process that's just sitting out there doing not much half the time, but taking up space. The space in your living room is RAM, and the space in your attic is swap. The coffee table is filesystem cache.
