[gentoo-user] emerge dev-python/python-dateutil-2.1 failed (compile phase)
I'm at a loss, as to how to solve this problem. Any advice would be greatly appreciated # emerge --info '=dev-python/python-dateutil-2.1' Portage 2.1.10.65 (default/linux/amd64/10.0/desktop, gcc-4.5.3, glibc-2.14.1-r3, 3.3.8-gentoo x86_64) = System Settings = System uname: Linux-3.3.8-gentoo-x86_64-Intel-R-_Core-TM-2_Duo_CPU_P8600_@_2.40GHz-with-gentoo-2.1 Timestamp of tree: Fri, 20 Jul 2012 16:15:01 + app-shells/bash: 4.2_p20 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.3-r2, 3.2.3 dev-util/cmake: 2.8.7-r5 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.9.8.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc:4.5.3-r2 sys-devel/gcc-config: 1.6 sys-devel/libtool:2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 3.4-r1 (virtual/os-headers) sys-libs/glibc: 2.14.1-r3 Repositories: gentoo ACCEPT_KEYWORDS=amd64 ACCEPT_LICENSE=* -@EULA CBUILD=x86_64-pc-linux-gnu CFLAGS=-march=core2 -O2 -pipe -msse4.1 CHOST=x86_64-pc-linux-gnu CONFIG_PROTECT=/etc /usr/share/gnupg/qualified.txt CONFIG_PROTECT_MASK=/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c CXXFLAGS=-march=core2 -O2 -pipe -msse4.1 DISTDIR=/usr/portage/distfiles FCFLAGS=-O2 -pipe FEATURES=assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv FFLAGS=-O2 -pipe GENTOO_MIRRORS=http://ftp.swin.edu.au/gentoo http://gentoo.channelx.biz/ http://gentoo.gg3.net/ http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.jaist.ac.jp/pub/Linux/Gentoo/; LANG=en_US.UTF-8 LDFLAGS=-Wl,-O1 -Wl,--as-needed LINGUAS=en MAKEOPTS=-j3 PKGDIR=/usr/portage/packages PORTAGE_CONFIGROOT=/ PORTAGE_RSYNC_OPTS=--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages PORTAGE_TMPDIR=/var/tmp PORTDIR=/usr/portage PORTDIR_OVERLAY= SYNC=rsync://rsync.au.gentoo.org/gentoo-portage USE=X a52 aac acl acpi alsa amd64 audiofile berkdb bluetooth branding bzip2 cairo cdda cddb cdr cli consolekit cracklib crypt css cups cxx dbus dga directfb djvu dri dts dvd dvdr emboss encode enscript exif fam fbcon ffmpeg fftw firefox flac fortran gdbm gif gpm gtk hddtemp iconv ipv6 jack joystick jpeg ladspa lame lash lcms ldap libnotify libsamplerate lm_sensors mad matroska mmx mng modules mp3 mp4 mpeg mplayer mudflap multilib musicbrainz ncurses networkmanager nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds pppd qt3support qt4 readline sasl scanner sdl session smp sound spell sse sse2 sse3 ssl startup-notification svg tcpd theora tiff truetype udev udisks unicode upower usb v4l vim vorbis wifi wxwidgets x264 xcb xcomposite xml xorg xscreensaver xv xvid xvmc zlib ALSA_CARDS=hda-intel ALSA_PCM_PLUGINS=adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol APACHE2_MODULES=actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias CALLIGRA_FEATURES=kexi words flow plan sheets stage tables krita karbon braindump CAMERAS=ptp2 COLLECTD_PLUGINS=df interface irq load memory rrdtool swap syslog ELIBC=glibc GPSD_PROTOCOLS=ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx GRUB_PLATFORMS=efi-64 INPUT_DEVICES=keyboard mouse synaptics evdev KERNEL=linux LCD_DEVICES=bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text LIBREOFFICE_EXTENSIONS=presenter-console presenter-minimizer LINGUAS=en PHP_TARGETS=php5-3 PYTHON_TARGETS=python3_2 python2_7 RUBY_TARGETS=ruby18 ruby19 USERLAND=GNU VIDEO_CARDS=nouveau v4l XTABLES_ADDONS=quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos
Re: [gentoo-user] emerge dev-python/python-dateutil-2.1 failed (compile phase)
On 08/14/2012 02:49 AM, Cinder wrote: I'm at a loss, as to how to solve this problem. Any advice would be greatly appreciated # emerge --info '=dev-python/python-dateutil-2.1' Portage 2.1.10.65 (default/linux/amd64/10.0/desktop, gcc-4.5.3, glibc-2.14.1-r3, 3.3.8-gentoo x86_64) = System Settings = System uname: Linux-3.3.8-gentoo-x86_64-Intel-R-_Core-TM-2_Duo_CPU_P8600_@_2.40GHz-with-gentoo-2.1 Timestamp of tree: Fri, 20 Jul 2012 16:15:01 + app-shells/bash: 4.2_p20 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.3-r2, 3.2.3 dev-util/cmake: 2.8.7-r5 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.9.8.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc:4.5.3-r2 sys-devel/gcc-config: 1.6 sys-devel/libtool:2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 3.4-r1 (virtual/os-headers) sys-libs/glibc: 2.14.1-r3 Repositories: gentoo ACCEPT_KEYWORDS=amd64 ACCEPT_LICENSE=* -@EULA CBUILD=x86_64-pc-linux-gnu CFLAGS=-march=core2 -O2 -pipe -msse4.1 CHOST=x86_64-pc-linux-gnu CONFIG_PROTECT=/etc /usr/share/gnupg/qualified.txt CONFIG_PROTECT_MASK=/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c CXXFLAGS=-march=core2 -O2 -pipe -msse4.1 DISTDIR=/usr/portage/distfiles FCFLAGS=-O2 -pipe FEATURES=assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv FFLAGS=-O2 -pipe GENTOO_MIRRORS=http://ftp.swin.edu.au/gentoo http://gentoo.channelx.biz/ http://gentoo.gg3.net/ http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.jaist.ac.jp/pub/Linux/Gentoo/; LANG=en_US.UTF-8 LDFLAGS=-Wl,-O1 -Wl,--as-needed LINGUAS=en MAKEOPTS=-j3 PKGDIR=/usr/portage/packages PORTAGE_CONFIGROOT=/ PORTAGE_RSYNC_OPTS=--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages PORTAGE_TMPDIR=/var/tmp PORTDIR=/usr/portage PORTDIR_OVERLAY= SYNC=rsync://rsync.au.gentoo.org/gentoo-portage USE=X a52 aac acl acpi alsa amd64 audiofile berkdb bluetooth branding bzip2 cairo cdda cddb cdr cli consolekit cracklib crypt css cups cxx dbus dga directfb djvu dri dts dvd dvdr emboss encode enscript exif fam fbcon ffmpeg fftw firefox flac fortran gdbm gif gpm gtk hddtemp iconv ipv6 jack joystick jpeg ladspa lame lash lcms ldap libnotify libsamplerate lm_sensors mad matroska mmx mng modules mp3 mp4 mpeg mplayer mudflap multilib musicbrainz ncurses networkmanager nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds pppd qt3support qt4 readline sasl scanner sdl session smp sound spell sse sse2 sse3 ssl startup-notification svg tcpd theora tiff truetype udev udisks unicode upower usb v4l vim vorbis wifi wxwidgets x264 xcb xcomposite xml xorg xscreensaver xv xvid xvmc zlib ALSA_CARDS=hda-intel ALSA_PCM_PLUGINS=adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol APACHE2_MODULES=actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias CALLIGRA_FEATURES=kexi words flow plan sheets stage tables krita karbon braindump CAMERAS=ptp2 COLLECTD_PLUGINS=df interface irq load memory rrdtool swap syslog ELIBC=glibc GPSD_PROTOCOLS=ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx GRUB_PLATFORMS=efi-64 INPUT_DEVICES=keyboard mouse synaptics evdev KERNEL=linux LCD_DEVICES=bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text LIBREOFFICE_EXTENSIONS=presenter-console presenter-minimizer LINGUAS=en PHP_TARGETS=php5-3 PYTHON_TARGETS=python3_2 python2_7 RUBY_TARGETS=ruby18 ruby19 USERLAND=GNU VIDEO_CARDS=nouveau v4l XTABLES_ADDONS=quota2 psd pknock lscan
Re: [OT] Re: [gentoo-user] Cannot see Grub menu
On Tue, 14 Aug 2012 01:16:05 +0100, Peter Humphrey wrote: ...unlike grocers' apostrophe's, which crop up everywhere and are far more grating for me. Agreed, except that I think you mean greengrocers'. Both are valid. Greengrocers' is the more common, grocers' is shorter. When you are paid by the word, the difference is important :) I also find that commas seem to be thrown at random into a piece of prose in the apparent hope that a few will land where they might do some good. I know what you mean, but that is more a matter of style than rules. I have been criticised by editors for using too many and too few. Even Penrose is sometimes guilty of that. And don't start me on the egregious Oxford comma. I wouldn't dare. Nor on the German insistence on separating the verb from the object with a comma, as though the action could proceed without something to act on. Different language, different rules. Put another way, I don't speak German so I don't care :) -- Neil Bothwick Cross a tagline and a tribble? You get a full HD... signature.asc Description: PGP signature
Re: [gentoo-user][SOLVED]emerge dev-python/python-dateutil-2.1 failed (compile phase)
Thank you kindly Alex, running python-updater did the trick. --- i.am.the.mem...@gmail.com wrote: From: Alex i.am.the.mem...@gmail.com To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] emerge dev-python/python-dateutil-2.1 failed (compile phase) Date: Tue, 14 Aug 2012 03:22:40 -0400 On 08/14/2012 02:49 AM, Cinder wrote: I'm at a loss, as to how to solve this problem. Any advice would be greatly appreciated # emerge --info '=dev-python/python-dateutil-2.1' Portage 2.1.10.65 (default/linux/amd64/10.0/desktop, gcc-4.5.3, glibc-2.14.1-r3, 3.3.8-gentoo x86_64) = System Settings = System uname: Linux-3.3.8-gentoo-x86_64-Intel-R-_Core-TM-2_Duo_CPU_P8600_@_2.40GHz-with-gentoo-2.1 Timestamp of tree: Fri, 20 Jul 2012 16:15:01 + app-shells/bash: 4.2_p20 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.3-r2, 3.2.3 dev-util/cmake: 2.8.7-r5 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.9.8.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc:4.5.3-r2 sys-devel/gcc-config: 1.6 sys-devel/libtool:2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 3.4-r1 (virtual/os-headers) sys-libs/glibc: 2.14.1-r3 Repositories: gentoo ACCEPT_KEYWORDS=amd64 ACCEPT_LICENSE=* -@EULA CBUILD=x86_64-pc-linux-gnu CFLAGS=-march=core2 -O2 -pipe -msse4.1 CHOST=x86_64-pc-linux-gnu CONFIG_PROTECT=/etc /usr/share/gnupg/qualified.txt CONFIG_PROTECT_MASK=/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c CXXFLAGS=-march=core2 -O2 -pipe -msse4.1 DISTDIR=/usr/portage/distfiles FCFLAGS=-O2 -pipe FEATURES=assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv FFLAGS=-O2 -pipe GENTOO_MIRRORS=http://ftp.swin.edu.au/gentoo http://gentoo.channelx.biz/ http://gentoo.gg3.net/ http://ftp.iij.ad.jp/pub/linux/gentoo/ http://ftp.jaist.ac.jp/pub/Linux/Gentoo/; LANG=en_US.UTF-8 LDFLAGS=-Wl,-O1 -Wl,--as-needed LINGUAS=en MAKEOPTS=-j3 PKGDIR=/usr/portage/packages PORTAGE_CONFIGROOT=/ PORTAGE_RSYNC_OPTS=--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages PORTAGE_TMPDIR=/var/tmp PORTDIR=/usr/portage PORTDIR_OVERLAY= SYNC=rsync://rsync.au.gentoo.org/gentoo-portage USE=X a52 aac acl acpi alsa amd64 audiofile berkdb bluetooth branding bzip2 cairo cdda cddb cdr cli consolekit cracklib crypt css cups cxx dbus dga directfb djvu dri dts dvd dvdr emboss encode enscript exif fam fbcon ffmpeg fftw firefox flac fortran gdbm gif gpm gtk hddtemp iconv ipv6 jack joystick jpeg ladspa lame lash lcms ldap libnotify libsamplerate lm_sensors mad matroska mmx mng modules mp3 mp4 mpeg mplayer mudflap multilib musicbrainz ncurses networkmanager nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds pppd qt3support qt4 readline sasl scanner sdl session smp sound spell sse sse2 sse3 ssl startup-notification svg tcpd theora tiff truetype udev udisks unicode upower usb v4l vim vorbis wifi wxwidgets x264 xcb xcomposite xml xorg xscreensaver xv xvid xvmc zlib ALSA_CARDS=hda-intel ALSA_PCM_PLUGINS=adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol APACHE2_MODULES=actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias CALLIGRA_FEATURES=kexi words flow plan sheets stage tables krita karbon braindump CAMERAS=ptp2 COLLECTD_PLUGINS=df interface irq load memory rrdtool swap syslog ELIBC=glibc GPSD_PROTOCOLS=ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx GRUB_PLATFORMS=efi-64 INPUT_DEVICES=keyboard mouse synaptics evdev KERNEL=linux LCD_DEVICES=bayrad
Re: [gentoo-user][SOLVED]emerge dev-python/python-dateutil-2.1 failed (compile phase)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 14.08.2012 09:55, Cinder wrote:
SNIP
Unpacking source... Unpacking python-dateutil-2.1.tar.gz to
/var/tmp/portage/dev-python/python-dateutil-2.1/work Source
unpacked in
/var/tmp/portage/dev-python/python-dateutil-2.1/work
Preparing source in
/var/tmp/portage/dev-python/python-dateutil-2.1/work/python-dateutil-2.1
...
* Applying python-dateutil-2.1-open-utf-8.patch ...
[ ok ]
Source prepared. Configuring source in
/var/tmp/portage/dev-python/python-dateutil-2.1/work/python-dateutil-2.1
... Source configured. Compiling source in
/var/tmp/portage/dev-python/python-dateutil-2.1/work/python-dateutil-2.1
...
* Building of dev-python/python-dateutil-2.1 with CPython 2.7...
python2.7 setup.py build -b build-2.7 Traceback (most recent
call last): File setup.py, line 8, in module from setuptools
import setup ImportError: No module named setuptools * ERROR:
dev-python/python-dateutil-2.1 failed (compile phase): *
Building failed with CPython 2.7 in distutils_building() function
* * Call stack: * ebuild.sh, line 85: Called src_compile
* environment, line 5022: Called distutils_src_compile *
environment, line 1185: Called python_execute_function
'distutils_building' * environment, line 3406: Called die *
The specific snippet of code: * die
${failure_message}; *
SNIP
Hi,
since the error is ImportError: No module named setuptools you
should make sure, that dev-python/setuptools is installed (for the
version of python you are using).
If it's installed you could try to run python-updater.
WKR
Hinnerk
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJQKgiPAAoJEJwwOFaNFkYcHykIAItcWoVhxfhpK350whS81hdl
UwjMxJQaHEIcSnwct6bhdueFIRT36PiXiNPir+X7w4Bs1lk2ZXUHH88k62H7wSNF
EGsBcXJl1hAhqVLOu06Vc6IHp/gpsB6PPHceFqQ7tl7jWMDvVVhmEvfQCg6st1Js
yuqwQFfwHrSAifByJS5jwclPo7H3HptMtIFw0MnFg3/JkYJGG1rv6avhjK+zTf71
1dxIIJCroGaGz+s5M8XYpXhcOr7B1U+rWPQImh2rjJM+wML2KdMEH9SzDqQJioKh
XoaVSpG2fz4dXahv1FFJ/Qjq4VRV0UfKQIlMD1Qglk98j0mSly9mlJR+S/p4WaY=
=1x0r
-END PGP SIGNATURE-
Re: [gentoo-user][SOLVED]emerge dev-python/python-dateutil-2.1 failed (compile phase)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 14.08.2012 10:13, Hinnerk van Bruinehsen wrote:
On 14.08.2012 09:55, Cinder wrote: SNIP
Unpacking source... Unpacking python-dateutil-2.1.tar.gz
to /var/tmp/portage/dev-python/python-dateutil-2.1/work
Source unpacked in
/var/tmp/portage/dev-python/python-dateutil-2.1/work
Preparing source in
/var/tmp/portage/dev-python/python-dateutil-2.1/work/python-dateutil-2.1
...
* Applying python-dateutil-2.1-open-utf-8.patch ... [ ok ]
Source prepared. Configuring source in
/var/tmp/portage/dev-python/python-dateutil-2.1/work/python-dateutil-2.1
... Source configured. Compiling source in
/var/tmp/portage/dev-python/python-dateutil-2.1/work/python-dateutil-2.1
...
* Building of dev-python/python-dateutil-2.1 with CPython
2.7... python2.7 setup.py build -b build-2.7 Traceback (most
recent call last): File setup.py, line 8, in module from
setuptools import setup ImportError: No module named setuptools
* ERROR: dev-python/python-dateutil-2.1 failed (compile phase):
* Building failed with CPython 2.7 in distutils_building()
function * * Call stack: * ebuild.sh, line 85: Called
src_compile * environment, line 5022: Called
distutils_src_compile * environment, line 1185: Called
python_execute_function 'distutils_building' * environment,
line 3406: Called die * The specific snippet of code: *
die ${failure_message}; *
SNIP
Hi,
since the error is ImportError: No module named setuptools you
should make sure, that dev-python/setuptools is installed (for the
version of python you are using). If it's installed you could try
to run python-updater.
WKR Hinnerk
I read your question on my mobile phone about an hour ago and didn't
read again before posting.
Sorry for spamming the list!
Hinnerk
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJQKgmaAAoJEJwwOFaNFkYcgnwH/0Jtv6LDrgX/kSKzmScHnL6A
5SVo6V7lmItSQrxmKox0DDdhDSoppo+n6xiWQig9YCFX2rbNadrW/hT/eqRRb13p
iVUArA7wVuZPSQMLJgp2BlEDz8LJRieruzRj9lZgeMP6rZPsIzNAVW/NwGDqFH3z
JYGMH5OrVCH6JGN1p5nRzXO5D7K3MddiMJPKbPctMPOJdvU8XN1Q5EVsZnrilG2x
8Rb8WreGAjTTa1f0yttdTbQ8Lb5yjqGMJ6fZsb/fbXAwZoNhIwsnPwW4/SiSY34z
CXN42t7pI3rc5mNM9PSDY9b5Fo2bAHoEVnTP/PFEtf110H9qrdBnZB6m3AApRzM=
=R0r5
-END PGP SIGNATURE-
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On 13.08.2012 16:53, Michael Hampicke wrote: 2012/8/13 Daniel Troeder dan...@admin-box.com 3rd thought: purging old files with find? your cache system should have some kind of DB that holds that information. 3: Well, it's a 3rd party application that - in theory - should take care of removing old files. Sadly, it does not work as it's supposed to be, While time passes the number of orphans grow :( There is also the possibility to write a really small daemon (less than 50 lines of C) that registers with inotify for the entire fs and journals the file activity to a sqlite-db. A simple sql-query from a cron/bash script will then give you all the files to delete with paths. It will probably be less work to write the daemon than to do 40 fs-benchmarks - and the result will be the most efficient.
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On Tue, 14 Aug 2012 10:21:54 +0200, Daniel Troeder wrote: There is also the possibility to write a really small daemon (less than 50 lines of C) that registers with inotify for the entire fs and journals the file activity to a sqlite-db. sys-process/incron ? -- Neil Bothwick A friend of mine sent me a postcard with a satellite photo of the entire planet on it, and on the back he wrote, Wish you were here. signature.asc Description: PGP signature
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 14.08.2012 11:46, schrieb Neil Bothwick: On Tue, 14 Aug 2012 10:21:54 +0200, Daniel Troeder wrote: There is also the possibility to write a really small daemon (less than 50 lines of C) that registers with inotify for the entire fs and journals the file activity to a sqlite-db. sys-process/incron ? I think in order to make it work, you have to increase the number of file descriptors available to inotify. See /proc/sys/fs/inotify/max_user_watches Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On 14.08.2012 11:46, Neil Bothwick wrote: On Tue, 14 Aug 2012 10:21:54 +0200, Daniel Troeder wrote: There is also the possibility to write a really small daemon (less than 50 lines of C) that registers with inotify for the entire fs and journals the file activity to a sqlite-db. sys-process/incron ? Uh... didn't know that one! ... very interesting :) Have you used it? How does it perform if there are lots of modifications going on? Does it have a throttle against fork bombing? must-read-myself-a-little. A incron line # sqlite3 /file.sql 'INSERT filename, date INTO table' would be inefficient, because it spawn lots of processes, but it would be very nice to simply test out the idea. Then a # sqlite3 /file.sql 'SELECT filename FROM table SORTBY date date-30days' or something to get the files older than 30 days, and voilá :)
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 13.08.2012 20:18, schrieb Michael Hampicke:
Am 13.08.2012 19:14, schrieb Florian Philipp:
Am 13.08.2012 16:52, schrieb Michael Mol:
On Mon, Aug 13, 2012 at 10:42 AM, Michael Hampicke
mgehampi...@gmail.com mailto:mgehampi...@gmail.com wrote:
Have you indexed your ext4 partition?
# tune2fs -O dir_index /dev/your_partition
# e2fsck -D /dev/your_partition
Hi, the dir_index is active. I guess that's why delete operations
take as long as they take (index has to be updated every time)
1) Scan for files to remove
2) disable index
3) Remove files
4) enable index
?
--
:wq
Other things to think about:
1. Play around with data=journal/writeback/ordered. IIRC, data=journal
actually used to improve performance depending on the workload as it
delays random IO in favor of sequential IO (when updating the journal).
2. Increase the journal size.
3. Take a look at `man 1 chattr`. Especially the 'T' attribute. Of
course this only helps after re-allocating everything.
4. Try parallelizing. Ext4 requires relatively few locks nowadays (since
2.6.39 IIRC). For example:
find $TOP_DIR -mindepth 1 -maxdepth 1 -print0 | \
xargs -0 -n 1 -r -P 4 -I '{}' find '{}' -type f
5. Use a separate device for the journal.
6. Temporarily deactivate the journal with tune2fs similar to MM's idea.
Regards,
Florian Philipp
Trying out different journals-/options was already on my list, but the
manpage on chattr regarding the T attribute is an interesting read.
Definitely worth trying.
Parallelizing multiple finds was something I already did, but the only
thing that increased was the IO wait :) But now having read all the
suggestions in this thread, I might try it again.
Separate device for the journal is a good idea, but not possible atm
(machine is abroad in a data center)
Something else I just remembered. I guess it doesn't help you with your
current problem but it might come in handy when working with such large
cache dirs: I once wrote a script that sorts files by their starting
physical block. This improved reading them quite a bit (2 minutes
instead of 11 minutes for copying the portage tree).
It's a terrible clutch, will probably fail when passing FS boundaries or
a thousand other oddities and requires root for some very scary
programs. I never had the time to finish an improved C version. Anyway,
maybe it helps you:
#!/bin/bash
#
# Example below copies /usr/portage/* to /tmp/portage.
# Replace /usr/portage with the input directory.
# Replace `cpio` with whatever does the actual work. Input is a
# \0-delimited file list.
#
FIFO=/tmp/$(uuidgen).fifo
mkfifo $FIFO
find /usr/portage -type f -fprintf $FIFO 'bmap %i 0\n' -print0 |
tr '\n\0' '\0\n' |
paste (
debugfs -f $FIFO /dev/mapper/vg-portage |
grep -E '^[[:digit:]]+'
) - |
sort -k 1,1n |
cut -f 2- |
tr '\n\0' '\0\n' |
cpio -p0 --make-directories /tmp/portage/
unlink $FIFO
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-user] Comparison between 32 bit and 64 bit
On Mon, Aug 13, 2012 at 6:18 PM, Frank Steinmetzger war...@gmx.de wrote: What a pity though -- you just don't get 1400x1050 laptops anymore these days (or any 4:3 laptops for that matter). I also have a 1400x1050 (15-inch screen) laptop and I think this resolution and screen size are hitting the sweet-spot for a 4:3 monitor of that size...
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 14.08.2012 15:54, schrieb Daniel Troeder:
On 14.08.2012 11:46, Neil Bothwick wrote:
On Tue, 14 Aug 2012 10:21:54 +0200, Daniel Troeder wrote:
There is also the possibility to write a really small daemon (less than
50 lines of C) that registers with inotify for the entire fs and
journals the file activity to a sqlite-db.
sys-process/incron ?
Uh... didn't know that one! ... very interesting :)
Have you used it?
How does it perform if there are lots of modifications going on?
Does it have a throttle against fork bombing?
must-read-myself-a-little.
A incron line
# sqlite3 /file.sql 'INSERT filename, date INTO table'
would be inefficient, because it spawn lots of processes, but it would
be very nice to simply test out the idea. Then a
# sqlite3 /file.sql 'SELECT filename FROM table SORTBY date date-30days'
or something to get the files older than 30 days, and voilá :)
Maybe inotifywait is better for this kind of batch job.
Collecting events:
inotifywait -rm -e CREATE,DELETE --timefmt '%s' --format \
$(printf '%%T\t%%e\t%%w%%f') /tmp events.tbl
# the printf is there because inotifywait's format does not
# recognize common escapes like \t
# Output format:
# Seconds since epoch \t CREATE/DELETE \t file name \n
Filtering events:
sort --stable -k3 events.tbl |
awk '
function update() {
line=$0; exists= $2==DELETE ? 0 : 1; file=$3
}
NR==1{ update(); next }
{ if($3!=file exists==1){ print line } update() }'
# Sorts by file name while preserving temporal order.
# Uses awk to suppress output of files that have been deleted.
# Output: Last CREATE event for each existing file
Retrieving files created 30+ days ago:
awk -v newest=$(date -d -5seconds +%s) '
$1newest{ nextfile }
{ print $3 }'
Remarks:
The awk scripts need some improvement if you have to handle whitespaces
in filenames but with the input format, it should be able to work with
everything except newlines.
Inotifywait itself is utterly useless when dealing with newlines in file
names unless you want to put some serious effort into sanitizing the output.
Regards,
Florian Philipp
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 14.08.2012 17:09, schrieb Florian Philipp:
Retrieving files created 30+ days ago:
awk -v newest=$(date -d -5seconds +%s) '
$1newest{ nextfile }
{ print $3 }'
s/-5seconds/-30days/
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On 08/14/2012 04:07:39 AM, Adam Carter wrote: I think btrfs probably is meant to provide a lot of the modern features like reiser4 or xfs Unfortunately btrfs is still generally slower than ext4 for example. Checkout http://openbenchmarking.org/, eg http://openbenchmarking.org/s/ext4%20btrfs The OS will use any spare RAM for disk caching, so if there's not much else running on that box, most of your content will be served from RAM. It may be that whatever fs you choose wont make that much of a difference anyways. If one can run a recent kernel (3.5.x) btrfs seems quite stable (It's used by some distribution and Oracle for real work) Most benchmark don't use compression since other FS can't use it. But that's unfair. With compression, one needs to read much less data (my /usr partition has less than 50% of an ext4 partition, savings with the root partition are even higher). I'm using the mount options compress=lzo,noacl,noatime,autodefrag,space_cache which require a recent kernel. I'd give it a try. Helmut.
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On Aug 14, 2012 11:42 PM, Helmut Jarausch jarau...@igpm.rwth-aachen.de wrote: On 08/14/2012 04:07:39 AM, Adam Carter wrote: I think btrfs probably is meant to provide a lot of the modern features like reiser4 or xfs Unfortunately btrfs is still generally slower than ext4 for example. Checkout http://openbenchmarking.org/, eg http://openbenchmarking.org/s/ext4%20btrfs The OS will use any spare RAM for disk caching, so if there's not much else running on that box, most of your content will be served from RAM. It may be that whatever fs you choose wont make that much of a difference anyways. If one can run a recent kernel (3.5.x) btrfs seems quite stable (It's used by some distribution and Oracle for real work) Most benchmark don't use compression since other FS can't use it. But that's unfair. With compression, one needs to read much less data (my /usr partition has less than 50% of an ext4 partition, savings with the root partition are even higher). I'm using the mount options compress=lzo,noacl,noatime,autodefrag,space_cache which require a recent kernel. I'd give it a try. Helmut. Are the support tools for btrfs (fsck, defrag, etc.) already complete? If so, I certainly would like to take it out for a spin... Rgds,
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Sure, but wouldn't compression make write operations slower? And isn't he looking for performance? On Aug 14, 2012 1:14 PM, Pandu Poluan pa...@poluan.info wrote: On Aug 14, 2012 11:42 PM, Helmut Jarausch jarau...@igpm.rwth-aachen.de wrote: On 08/14/2012 04:07:39 AM, Adam Carter wrote: I think btrfs probably is meant to provide a lot of the modern features like reiser4 or xfs Unfortunately btrfs is still generally slower than ext4 for example. Checkout http://openbenchmarking.org/, eg http://openbenchmarking.org/s/ext4%20btrfs The OS will use any spare RAM for disk caching, so if there's not much else running on that box, most of your content will be served from RAM. It may be that whatever fs you choose wont make that much of a difference anyways. If one can run a recent kernel (3.5.x) btrfs seems quite stable (It's used by some distribution and Oracle for real work) Most benchmark don't use compression since other FS can't use it. But that's unfair. With compression, one needs to read much less data (my /usr partition has less than 50% of an ext4 partition, savings with the root partition are even higher). I'm using the mount options compress=lzo,noacl,noatime,autodefrag,space_cache which require a recent kernel. I'd give it a try. Helmut. Are the support tools for btrfs (fsck, defrag, etc.) already complete? If so, I certainly would like to take it out for a spin... Rgds,
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am Dienstag, 14. August 2012, 13:21:35 schrieb Jason Weisberger: Sure, but wouldn't compression make write operations slower? And isn't he looking for performance? not really. As long as the CPU can compress faster than the disk can write stuff. More interessting: is btrfs trying to be smart - only compressing compressible stuff? -- #163933
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 14.08.2012 16:00, schrieb Florian Philipp:
Am 13.08.2012 20:18, schrieb Michael Hampicke:
Am 13.08.2012 19:14, schrieb Florian Philipp:
Am 13.08.2012 16:52, schrieb Michael Mol:
On Mon, Aug 13, 2012 at 10:42 AM, Michael Hampicke
mgehampi...@gmail.com mailto:mgehampi...@gmail.com wrote:
Have you indexed your ext4 partition?
# tune2fs -O dir_index /dev/your_partition
# e2fsck -D /dev/your_partition
Hi, the dir_index is active. I guess that's why delete operations
take as long as they take (index has to be updated every time)
1) Scan for files to remove
2) disable index
3) Remove files
4) enable index
?
--
:wq
Other things to think about:
1. Play around with data=journal/writeback/ordered. IIRC, data=journal
actually used to improve performance depending on the workload as it
delays random IO in favor of sequential IO (when updating the journal).
2. Increase the journal size.
3. Take a look at `man 1 chattr`. Especially the 'T' attribute. Of
course this only helps after re-allocating everything.
4. Try parallelizing. Ext4 requires relatively few locks nowadays (since
2.6.39 IIRC). For example:
find $TOP_DIR -mindepth 1 -maxdepth 1 -print0 | \
xargs -0 -n 1 -r -P 4 -I '{}' find '{}' -type f
5. Use a separate device for the journal.
6. Temporarily deactivate the journal with tune2fs similar to MM's idea.
Regards,
Florian Philipp
Trying out different journals-/options was already on my list, but the
manpage on chattr regarding the T attribute is an interesting read.
Definitely worth trying.
Parallelizing multiple finds was something I already did, but the only
thing that increased was the IO wait :) But now having read all the
suggestions in this thread, I might try it again.
Separate device for the journal is a good idea, but not possible atm
(machine is abroad in a data center)
Something else I just remembered. I guess it doesn't help you with your
current problem but it might come in handy when working with such large
cache dirs: I once wrote a script that sorts files by their starting
physical block. This improved reading them quite a bit (2 minutes
instead of 11 minutes for copying the portage tree).
It's a terrible clutch, will probably fail when passing FS boundaries or
a thousand other oddities and requires root for some very scary
programs. I never had the time to finish an improved C version. Anyway,
maybe it helps you:
#!/bin/bash
#
# Example below copies /usr/portage/* to /tmp/portage.
# Replace /usr/portage with the input directory.
# Replace `cpio` with whatever does the actual work. Input is a
# \0-delimited file list.
#
FIFO=/tmp/$(uuidgen).fifo
mkfifo $FIFO
find /usr/portage -type f -fprintf $FIFO 'bmap %i 0\n' -print0 |
tr '\n\0' '\0\n' |
paste (
debugfs -f $FIFO /dev/mapper/vg-portage |
grep -E '^[[:digit:]]+'
) - |
sort -k 1,1n |
cut -f 2- |
tr '\n\0' '\0\n' |
cpio -p0 --make-directories /tmp/portage/
unlink $FIFO
No, I don't think that's practicable with the number of files in my
setup. To be honest, currently I am quite happy with the performance of
btrfs. Running through the directory tree only takes 1/10th of the time
it took with ext4, and deletes are pretty fast as well. I'm sure there's
still room for more improvement, but right now it's much better than it
was before.
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am Mittwoch, 15. August 2012, 00:05:40 schrieb Pandu Poluan: Are the support tools for btrfs (fsck, defrag, etc.) already complete? no -- #163933
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 14.08.2012 10:21, schrieb Daniel Troeder: On 13.08.2012 16:53, Michael Hampicke wrote: 2012/8/13 Daniel Troeder dan...@admin-box.com 3rd thought: purging old files with find? your cache system should have some kind of DB that holds that information. 3: Well, it's a 3rd party application that - in theory - should take care of removing old files. Sadly, it does not work as it's supposed to be, While time passes the number of orphans grow :( There is also the possibility to write a really small daemon (less than 50 lines of C) that registers with inotify for the entire fs and journals the file activity to a sqlite-db. A simple sql-query from a cron/bash script will then give you all the files to delete with paths. It will probably be less work to write the daemon than to do 40 fs-benchmarks - and the result will be the most efficient. That is an interesting idea, but I have never used inotify on such a huge file base, I am not sure what impact that has in terms of cpu cycles being used. But I am going to try this on some snowy winter weekend :)
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 14.08.2012 19:21, schrieb Jason Weisberger: Sure, but wouldn't compression make write operations slower? And isn't he looking for performance? On Aug 14, 2012 1:14 PM, Pandu Poluan pa...@poluan.info wrote: On Aug 14, 2012 11:42 PM, Helmut Jarausch jarau...@igpm.rwth-aachen.de wrote: On 08/14/2012 04:07:39 AM, Adam Carter wrote: I think btrfs probably is meant to provide a lot of the modern features like reiser4 or xfs Unfortunately btrfs is still generally slower than ext4 for example. Checkout http://openbenchmarking.org/, eg http://openbenchmarking.org/s/ext4%20btrfs The OS will use any spare RAM for disk caching, so if there's not much else running on that box, most of your content will be served from RAM. It may be that whatever fs you choose wont make that much of a difference anyways. If one can run a recent kernel (3.5.x) btrfs seems quite stable (It's used by some distribution and Oracle for real work) Most benchmark don't use compression since other FS can't use it. But that's unfair. With compression, one needs to read much less data (my /usr partition has less than 50% of an ext4 partition, savings with the root partition are even higher). I'm using the mount options compress=lzo,noacl,noatime,autodefrag,space_cache which require a recent kernel. I'd give it a try. Helmut. Are the support tools for btrfs (fsck, defrag, etc.) already complete? If so, I certainly would like to take it out for a spin... Rgds, I have enough cpu power at hand for compression, I guess that should not be the issue. But the cache dir mostly consists of prescaled jpeg images, so compressing them again would not give me any benefits, speed- or size-wise.
Re: [gentoo-user] Fast file system for cache directory with lot's of files
Am 14.08.2012 19:42, schrieb Volker Armin Hemmann: Am Dienstag, 14. August 2012, 13:21:35 schrieb Jason Weisberger: Sure, but wouldn't compression make write operations slower? And isn't he looking for performance? not really. As long as the CPU can compress faster than the disk can write stuff. More interessting: is btrfs trying to be smart - only compressing compressible stuff? It does do that, but letting btrfs check if the files are already compressed, if you know, that they are compressed, is a waste of cpu cycles :)
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On Tue, Aug 14, 2012 at 12:05 PM, Pandu Poluan pa...@poluan.info wrote: On Aug 14, 2012 11:42 PM, Helmut Jarausch jarau...@igpm.rwth-aachen.de wrote: On 08/14/2012 04:07:39 AM, Adam Carter wrote: I think btrfs probably is meant to provide a lot of the modern features like reiser4 or xfs Unfortunately btrfs is still generally slower than ext4 for example. Checkout http://openbenchmarking.org/, eg http://openbenchmarking.org/s/ext4%20btrfs The OS will use any spare RAM for disk caching, so if there's not much else running on that box, most of your content will be served from RAM. It may be that whatever fs you choose wont make that much of a difference anyways. If one can run a recent kernel (3.5.x) btrfs seems quite stable (It's used by some distribution and Oracle for real work) Most benchmark don't use compression since other FS can't use it. But that's unfair. With compression, one needs to read much less data (my /usr partition has less than 50% of an ext4 partition, savings with the root partition are even higher). I'm using the mount options compress=lzo,noacl,noatime,autodefrag,space_cache which require a recent kernel. I'd give it a try. Helmut. Are the support tools for btrfs (fsck, defrag, etc.) already complete? Do they exist? Yes (sys-fs/btrfs-progs). Are they complete? Probably not...
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On Tue, Aug 14, 2012 at 12:50 PM, Michael Hampicke gentoo-u...@hadt.biz wrote: Am 14.08.2012 19:42, schrieb Volker Armin Hemmann: Am Dienstag, 14. August 2012, 13:21:35 schrieb Jason Weisberger: Sure, but wouldn't compression make write operations slower? And isn't he looking for performance? not really. As long as the CPU can compress faster than the disk can write stuff. More interessting: is btrfs trying to be smart - only compressing compressible stuff? It does do that, but letting btrfs check if the files are already compressed, if you know, that they are compressed, is a waste of cpu cycles :) Also look into the difference between compress and compress-force[0]. I wonder how much overhead checking whether or not to compress a file costs. I use mount options similar to Helmut and get great results: defaults,autodefrag,space_cache,compress=lzo,subvol=@,relatime But most of my data is compressible. Compression makes such a huge difference, it surprises me. Apparently on this Ubuntu system it automatically makes use of all files on / as a subvolume in @. Interesting. Anyway, btrfs-progs does include basic fsck now but I wouldn't use it for anything serious[1]. [0] https://btrfs.wiki.kernel.org/index.php/Mount_options [1] https://btrfs.wiki.kernel.org/index.php/Btrfsck
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On Tue, Aug 14, 2012 at 3:55 PM, Alecks Gates aleck...@gmail.com wrote: On Tue, Aug 14, 2012 at 12:50 PM, Michael Hampicke gentoo-u...@hadt.biz wrote: Am 14.08.2012 19:42, schrieb Volker Armin Hemmann: Am Dienstag, 14. August 2012, 13:21:35 schrieb Jason Weisberger: Sure, but wouldn't compression make write operations slower? And isn't he looking for performance? not really. As long as the CPU can compress faster than the disk can write stuff. More interessting: is btrfs trying to be smart - only compressing compressible stuff? It does do that, but letting btrfs check if the files are already compressed, if you know, that they are compressed, is a waste of cpu cycles :) Also look into the difference between compress and compress-force[0]. I wonder how much overhead checking whether or not to compress a file costs. I use mount options similar to Helmut and get great results: defaults,autodefrag,space_cache,compress=lzo,subvol=@,relatime But most of my data is compressible. Compression makes such a huge difference, it surprises me. Apparently on this Ubuntu system it automatically makes use of all files on / as a subvolume in @. Interesting. Huge difference, how? Could we see some bonnie++ comparisons between the various configurations we've discussed for ext4 and btrfs? Depending on the results, it might be getting time for me to take the plunge myself. -- :wq
Re: [gentoo-user] Fast file system for cache directory with lot's of files
On Tue, Aug 14, 2012 at 3:17 PM, Michael Mol mike...@gmail.com wrote: On Tue, Aug 14, 2012 at 3:55 PM, Alecks Gates aleck...@gmail.com wrote: On Tue, Aug 14, 2012 at 12:50 PM, Michael Hampicke gentoo-u...@hadt.biz wrote: Am 14.08.2012 19:42, schrieb Volker Armin Hemmann: Am Dienstag, 14. August 2012, 13:21:35 schrieb Jason Weisberger: Sure, but wouldn't compression make write operations slower? And isn't he looking for performance? not really. As long as the CPU can compress faster than the disk can write stuff. More interessting: is btrfs trying to be smart - only compressing compressible stuff? It does do that, but letting btrfs check if the files are already compressed, if you know, that they are compressed, is a waste of cpu cycles :) Also look into the difference between compress and compress-force[0]. I wonder how much overhead checking whether or not to compress a file costs. I use mount options similar to Helmut and get great results: defaults,autodefrag,space_cache,compress=lzo,subvol=@,relatime But most of my data is compressible. Compression makes such a huge difference, it surprises me. Apparently on this Ubuntu system it automatically makes use of all files on / as a subvolume in @. Interesting. Huge difference, how? Could we see some bonnie++ comparisons between the various configurations we've discussed for ext4 and btrfs? Depending on the results, it might be getting time for me to take the plunge myself. -- :wq Check out some of the benchmarks on Phoronix[0]. It's definitely not a win-win scenario, but it seems to be great at random writes and compiling. And a lot of those wins are without compress=lzo enabled, so it only gets better. I'm not going to say it's the absolute best out there (because it isn't, of course), but it's at least worth checking into. I'm using a standard 2.5 HDD like in this[1] so perhaps that's why I see the results. [0] http://www.phoronix.com/scan.php?page=searchq=Btrfs [1] http://www.phoronix.com/scan.php?page=articleitem=btrfs_old_linux31
Re: [gentoo-user] new installation (ssd, new udev, grub2)
Hi guys, after quick read about ssd, I have a couple of question: 1. My friend have new server with a ssd installed. He plan to RHEL 5.7 (I don't know why he choose this) on it. On redhat website, it say something like this: However, if the device does not export topology information, Red Hat recommends that the first partition be created at a 1MB boundary. What does it mean by 1MB boundary? Does it mean he have to create 1MB free space in front or he have to create a 1MB partition in front of his actual partition(s)? 2. Is it possible to combine TRIM support and ext3 partition (AFAIK, RHEL 5.7 haven't support ext4)? *i hope this is not count as hijacking On 8/14/12, Alan McKinnon alan.mckin...@gmail.com wrote: On Mon, 13 Aug 2012 11:55:31 -0400 Michael Mol mike...@gmail.com wrote: On Mon, Aug 13, 2012 at 11:47 AM, Alan McKinnon alan.mckin...@gmail.comwrote: On Mon, 13 Aug 2012 08:17:23 -0400 Michael Mol mike...@gmail.com wrote: On Mon, Aug 13, 2012 at 4:06 AM, Neil Bothwick n...@digimed.co.uk wrote: On Sun, 12 Aug 2012 14:11:37 -0400, Allan Gottlieb wrote: I have one of those. But I decided to stick with traditional DOS partitioning style and grub instead of GPT and grub2. I am leaning toward traditional partitioning, but with grub2. Do those two not mix well? GRUB2 works fine with MBR partition tables. But if you're starting from scratch, you may as well use GPT and get rid of the legacy MBR limitations and fragility. I'm not dissing GPT...but what's fragile about MBR? it's 30 years old, only 4 primary partitions, only 16 extended partitions, it's got that weird DOS boot flag thing, it all has to fit in one sector. I had to fix a mispartitioned disk over the weekend, this really should have been a simple mv-type operation, but because all 4 primary partitions were in use I had to disable swap and use it as a leap-frog area. It felt like I was playing 15 pieces with the disk. That's fragile - not that the disk breaks, but that it breaks my ability to set the thing up easily. Basically, mbr was built to cater for the needs of DOS-3. In the meantime, 1982 called and they want their last 30 years back. Just because we can hack workarounds into it to get it to function doesn't mean we should continue to use it. You misunderstand me. I wasn't arguing that GPT wasn't perhaps more elegant than MBR and dos partitions. I wanted to know what was _fragile_ about MBR. Completely different things. I did answer (somewhat obliquely). mbr as a single isolated unit is not especially fragile; very little software is and bits don't magically rot It's the system into which the sysadmin inserts mbr that is fragile. The whole system is fragile like an egg is fragile - it can't withstand much manhandling or moving of stuff around before some mistake wreaks everything, and that is mostly due to mbr's limits. It's not semantic nitpicking here, if the system as a unit becomes fragile as a result of part X, then the system is still fragile. -- Alan McKinnon alan.mckin...@gmail.com -- Salam, J.Marcos S. Sent from X1™
