Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-28 Thread Siju George
On Sat, Feb 23, 2013 at 1:59 AM, Miod Vallat m...@online.fr wrote: Please, please, please, can someone port ZFS, just to end this endless thread...? Please someone port HAMMER instead. We are only interested in free software, with no strings attached. YAY!!!

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Tomas Bodzar
On Thu, Feb 21, 2013 at 11:50 PM, Jeremie Le Hen jere...@le-hen.org wrote: On Thu, Feb 21, 2013 at 05:15:35PM -0500, Bryan Horstmann-Allen wrote: I apologize this is off-topic, but I'm somewhat close to the illumos project and would like to correct a few things. [...things corrected...]

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Peter Hessler
On 2013 Feb 22 (Fri) at 09:27:59 +0100 (+0100), Tomas Bodzar wrote: :On Thu, Feb 21, 2013 at 11:50 PM, Jeremie Le Hen jere...@le-hen.org wrote: : On Thu, Feb 21, 2013 at 05:15:35PM -0500, Bryan Horstmann-Allen wrote: : I apologize this is off-topic, but I'm somewhat close to the illumos project :

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Jiri B
On Fri, Feb 22, 2013 at 03:29:21AM +0100, Juan Francisco Cantero Hurtado wrote: OpenBSD doesn't have support for loadable kernel modules or FUSE, so OpenBSD should include the code inside of the kernel. This is a big difference with FreeBSD/NetBSD/Linux.

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Martin Schröder
2013/2/22 Juan Francisco Cantero Hurtado i...@juanfra.info: Here in the BSD world, we have HAMMER, a good alternative with a license compatible and a reasonable requirements. Here in the OpenBSD world we don't have HAMMER. Best Martin

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Jiri B
On Fri, Feb 22, 2013 at 04:22:51AM -0500, Jiri B wrote: On Fri, Feb 22, 2013 at 03:29:21AM +0100, Juan Francisco Cantero Hurtado wrote: OpenBSD doesn't have support for loadable kernel modules or FUSE, so OpenBSD should include the code inside of the kernel. This is a big difference with

Re: Millions of files in /var/www inode / out of space issue.

2013-02-22 Thread Paolo Aglialoro
The source was available, but it relies on Sun/Oracle patents. The CDDL license it was provided under allows use of those patents, but only subject to certain conditions, and there are indemnification clauses that some projects cannot agree to. Does this mean that freebsd, netbsd, maczfs, zfs

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Eric Furman
but Martin Schröder is not a developer. So what is his word worth??? I don't know and neither does Martin Schröder. On Fri, Feb 22, 2013, at 04:23 AM, Martin Schröder wrote: 2013/2/22 Juan Francisco Cantero Hurtado i...@juanfra.info: Here in the BSD world, we have HAMMER, a good alternative

Re: Millions of files in /var/www inode / out of space issue.

2013-02-22 Thread Eric Furman
YES, unless they signed NDA. Which I can tell you they did. On Fri, Feb 22, 2013, at 05:44 AM, Paolo Aglialoro wrote: The source was available, but it relies on Sun/Oracle patents. The CDDL license it was provided under allows use of those patents, but only subject to certain conditions, and

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Martin Schröder
2013/2/22 Eric Furman ericfur...@fastmail.net: but Martin Schröder is not a developer. So what is his word worth??? http://www.openbsd.org/faq/faq8.html#Journaling Now go and fuck yourself.

Re: Millions of files in /var/www inode / out of space issue.

2013-02-22 Thread Janne Johansson
2013/2/22 Paolo Aglialoro paol...@gmail.com: The source was available, but it relies on Sun/Oracle patents. The CDDL license it was provided under allows use of those patents, but only subject to certain conditions, and there are indemnification clauses that some projects cannot agree to.

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Peter Hessler
Please take personal insults off list. We are not interested. -- Don't cook tonight -- starve a rat today!

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Eric Furman
That proves nothing. Until your name is on this list; http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/ports/geo/openbsd-developers/files/OpenBSD YOU ARE NOT A DEVELOPER. FUCK YOU! On Fri, Feb 22, 2013, at 06:29 AM, Martin Schröder wrote: 2013/2/22 Eric Furman ericfur...@fastmail.net: but

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Juan Francisco Cantero Hurtado
On Fri, Feb 22, 2013 at 04:22:51AM -0500, Jiri B wrote: On Fri, Feb 22, 2013 at 03:29:21AM +0100, Juan Francisco Cantero Hurtado wrote: OpenBSD doesn't have support for loadable kernel modules or FUSE, so OpenBSD should include the code inside of the kernel. This is a big difference with

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Janne Johansson
OpenBSD doesn't believe much in them, and doesn't need lkms for all basic usage. It just didn't get disabled in case someone (like OpenAFS users) wants it. It doesn't work on all platforms, either 2013/2/22 Juan Francisco Cantero Hurtado i...@juanfra.info: On Fri, Feb 22, 2013 at 04:22:51AM

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Ted Unangst
On Fri, Feb 22, 2013 at 06:42, Eric Furman wrote: Until your name is on this list; http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/ports/geo/openbsd-developers/files/OpenBSD YOU ARE NOT A DEVELOPER. Now that's hysterical.

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Brian Callahan
On 2/22/2013 8:02 AM, Ted Unangst wrote: On Fri, Feb 22, 2013 at 06:42, Eric Furman wrote: Until your name is on this list; http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/ports/geo/openbsd-developers/files/OpenBSD YOU ARE NOT A DEVELOPER. I'm making this into a shirt. ~Brian

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Mike Jackson
Yeah, can I order one? This thread is hilariously funny! I even managed to get an entire car full of people laughing by reading it to them from my mobile. Long live OpenBSD and long live ZFS - I love you both! On 22 Feb 2013, at 16:18, Brian Callahan bcal...@devio.us wrote: On 2/22/2013

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread bofh
On Fri, Feb 22, 2013 at 3:27 AM, Tomas Bodzar tomas.bod...@gmail.com wrote: What's much more funny is that Oracle is paying for training and support to Joyent to be able to offer at least some level of support in ZFS for its own customers :D http://www.youtube.com/watch?v=-zRN7XLCRhc Funny

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Dustin Fechner
Please, please, please, can someone port ZFS, just to end this endless thread...?

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Miod Vallat
Please, please, please, can someone port ZFS, just to end this endless thread...? Please someone port HAMMER instead. We are only interested in free software, with no strings attached.

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-22 Thread Peter Hessler
No, just end the thread. -- There's an old proverb that says just about whatever you want it to.

Re: Millions of files in /var/www inode / out of space issue.

2013-02-21 Thread Stuart Henderson
On 2013-02-20, Keith ke...@scott-land.net wrote: Hi, thanks for the info. Yesterday I did a backup, format, restore of the /var/www partition although to be honest I wasn't really sure what i was doing with regards to the newfs command. I tried running newfs -iwith different values and

Re: Millions of files in /var/www inode / out of space issue.

2013-02-21 Thread Stuart Henderson
On 2013-02-20, Matthias Appel appel.matth...@gmail.com wrote: *ZFS was open source (FSF would say free) until Oracle acquired Sun The source was available, but it relies on Sun/Oracle patents. The CDDL license it was provided under allows use of those patents, but only subject to certain

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Jeremie Le Hen
On Wed, Feb 20, 2013 at 12:32:02AM +0100, Matthias Appel wrote: Yupp, I think, that's (beside the CDDL part of ZFS) it the major turn-off in any kind of productive enviroment. At the moment I don't know how FreeBSD handles the ZFS development, but maintaining a not-really-fully-ZFS

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Jeremie Le Hen
On Thu, Feb 21, 2013 at 05:15:35PM -0500, Bryan Horstmann-Allen wrote: I apologize this is off-topic, but I'm somewhat close to the illumos project and would like to correct a few things. [...things corrected...] Well, thank you very much for correcting me and providing us high quality

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Bryan Horstmann-Allen
I apologize this is off-topic, but I'm somewhat close to the illumos project and would like to correct a few things. +-- | On 2013-02-21 22:12:45, Jeremie Le Hen wrote: | | So, long story short, I do not see any option

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Theo de Raadt
There is zero reason not to have ZFS in a free system. Consider its inclusion in FreeBSD. Just because FreeBSD decided to compromise in regards to ZFS, does not mean everyone else has to as well. They could include all sorts of other code with similar licenses, yet there they often stand firm.

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Juan Francisco Cantero Hurtado
On Thu, Feb 21, 2013 at 05:15:35PM -0500, Bryan Horstmann-Allen wrote: I apologize this is off-topic, but I'm somewhat close to the illumos project and would like to correct a few things. +-- | On 2013-02-21

Re: Millions of files in /var/www inode / out of space issue.

2013-02-21 Thread Juan Francisco Cantero Hurtado
On Tue, Feb 19, 2013 at 07:41:11AM -0500, Nick Holland wrote: On 02/19/13 05:47, MJ wrote: Which app are you running that is generating millions of tiny files in a single directory? Regardless, in this case OpenBSD is not the right tool for the job. You need either FreeBSD or a Solaris

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Andres Perera
On Thu, Feb 21, 2013 at 9:59 PM, Juan Francisco Cantero Hurtado i...@juanfra.info wrote: OpenBSD doesn't have support for loadable kernel modules or FUSE, so OpenBSD should include the code inside of the kernel. This is a big difference with FreeBSD/NetBSD/Linux. lkm(4) is outdated with wrong

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Juan Francisco Cantero Hurtado
On Thu, Feb 21, 2013 at 10:54:58PM -0430, Andres Perera wrote: On Thu, Feb 21, 2013 at 9:59 PM, Juan Francisco Cantero Hurtado i...@juanfra.info wrote: OpenBSD doesn't have support for loadable kernel modules or FUSE, so OpenBSD should include the code inside of the kernel. This is a big

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Rod Whitworth
On Thu, 21 Feb 2013 22:54:58 -0430, Andres Perera wrote: On Thu, Feb 21, 2013 at 9:59 PM, Juan Francisco Cantero Hurtado i...@juanfra.info wrote: OpenBSD doesn't have support for loadable kernel modules or FUSE, so OpenBSD should include the code inside of the kernel. This is a big difference

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Philip Guenther
On Thu, Feb 21, 2013 at 8:29 PM, Rod Whitworth glis...@witworx.com wrote: On Thu, 21 Feb 2013 22:54:58 -0430, Andres Perera wrote: ... lkm(4) is outdated with wrong information about a feature no longer present? From cvsweb:src/lkm/ap/Attic/README Revision 1.3 Mon Feb 24 22:30:12 2003 UTC

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Eric Furman
On Thu, Feb 21, 2013, at 11:43 PM, Philip Guenther wrote: On Thu, Feb 21, 2013 at 8:29 PM, Rod Whitworth glis...@witworx.com wrote: On Thu, 21 Feb 2013 22:54:58 -0430, Andres Perera wrote: ... lkm(4) is outdated with wrong information about a feature no longer present? From

Re: Precisions on ZFS (was: Millions of files in /var/www inode / out of space issue.)

2013-02-21 Thread Antoine Verheijen
On 2013-02-21, at 11:21 PM, Eric Furman wrote: On Thu, Feb 21, 2013, at 11:43 PM, Philip Guenther wrote: On Thu, Feb 21, 2013 at 8:29 PM, Rod Whitworth glis...@witworx.com wrote: On Thu, 21 Feb 2013 22:54:58 -0430, Andres Perera wrote: ... lkm(4) is outdated with wrong information about a

Re: Millions of files in /var/www inode / out of space issue.

2013-02-20 Thread Jiri B
On Wed, Feb 20, 2013 at 12:32:02AM +0100, Matthias Appel wrote: And by talking of ZFS, why not consider ext3/4,reiser,xfs,jfs,ntfs,whatever-fs to be ported to OpenBSD? Where are the diffs? For example real improvement would be FAT/NTFS speed on OpenBSD, as it is much much slower than on Linux.

Re: Millions of files in /var/www inode / out of space issue.

2013-02-20 Thread Jérémie Courrèges-Anglas
Jiri B ji...@devio.us writes: On Wed, Feb 20, 2013 at 12:32:02AM +0100, Matthias Appel wrote: And by talking of ZFS, why not consider ext3/4,reiser,xfs,jfs,ntfs,whatever-fs to be ported to OpenBSD? Where are the diffs? For example real improvement would be FAT/NTFS speed on OpenBSD, as it

Re: Millions of files in /var/www inode / out of space issue.

2013-02-20 Thread Luca Ferrari
I suspect the application described here should not use a filesystem, probably a database will be better for the aim. However, assuming it is not possible to fix/change the application behavior, I guess using several filesystems/mount points will help. While ZFS (and many others) will be good at

Re: Millions of files in /var/www inode / out of space issue.

2013-02-20 Thread Matthias Appel
Am 20.02.2013 09:21, schrieb Jiri B: On Wed, Feb 20, 2013 at 12:32:02AM +0100, Matthias Appel wrote: And by talking of ZFS, why not consider ext3/4,reiser,xfs,jfs,ntfs,whatever-fs to be ported to OpenBSD? Where are the diffs? For example real improvement would be FAT/NTFS speed on OpenBSD, as

Re: Millions of files in /var/www inode / out of space issue.

2013-02-20 Thread Keith
On 20/02/2013 07:36, Jan Stary wrote: On Tue, Feb 19, 2013 at 00:35, Keith wrote: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? newfs defaults to -f 2k and -b 16k which is fine

Re: Millions of files in /var/www inode / out of space issue.

2013-02-20 Thread Jan Stary
On Feb 20 20:58:49, ke...@scott-land.net wrote: On 20/02/2013 07:36, Jan Stary wrote: On Tue, Feb 19, 2013 at 00:35, Keith wrote: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ?

Re: Millions of files in /var/www inode / out of space issue.

2013-02-20 Thread Bryan
On Wed, Feb 20, 2013 at 4:07 PM, Jan Stary h...@stare.cz wrote: I tried running newfs -iwith different values and settled on newfs -i 1 /var/www as it seemed at the time to makes the make the most inodes and that was just based on how much output was generated while newfs was running.

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Otto Moerbeek
On Tue, Feb 19, 2013 at 08:42:01AM +0100, Janne Johansson wrote: 2013/2/19 Keith ke...@scott-land.net: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? Since you probably

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Otto Moerbeek
On Tue, Feb 19, 2013 at 09:09:49AM +0100, Otto Moerbeek wrote: On Tue, Feb 19, 2013 at 08:42:01AM +0100, Janne Johansson wrote: 2013/2/19 Keith ke...@scott-land.net: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread MJ
Which app are you running that is generating millions of tiny files in a single directory? Regardless, in this case OpenBSD is not the right tool for the job. You need either FreeBSD or a Solaris variant to handle this problem because you need ZFS. What limits does ZFS have?

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Paolo Aglialoro
Or you could just use ZFS, XFS, whateverFS in a separate unix/linux box and go NFS on it, simulating a true external storage appliance :) On Tue, Feb 19, 2013 at 11:47 AM, MJ m...@sci.fi wrote: Which app are you running that is generating millions of tiny files in a single directory?

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Rafal Bisingier
Hi, Or you could fix your application, to not do stupid things (like generating millions of files in a single directory) in the first place... ;-) On 2013-02-19 at 12:10 CET Paolo Aglialoro paol...@gmail.com wrote: Or you could just use ZFS, XFS, whateverFS in a separate unix/linux box and go

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Keith
On 19/02/2013 10:47, MJ wrote: Which app are you running that is generating millions of tiny files in a single directory? Regardless, in this case OpenBSD is not the right tool for the job. You need either FreeBSD or a Solaris variant to handle this problem because you need ZFS. What

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Wayne Oliver
On 19 Feb 2013, at 1:40 PM, Rafal Bisingier wrote: Hi, Or you could fix your application, to not do stupid things (like generating millions of files in a single directory) in the first place... ;-) +1 On 2013-02-19 at 12:10 CET Paolo Aglialoro paol...@gmail.com wrote: Or you

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Nick Holland
On 02/19/13 05:47, MJ wrote: Which app are you running that is generating millions of tiny files in a single directory? Regardless, in this case OpenBSD is not the right tool for the job. You need either FreeBSD or a Solaris variant to handle this problem because you need ZFS. What

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Andres Perera
On Tue, Feb 19, 2013 at 8:11 AM, Nick Holland n...@holland-consulting.net wrote: I use ZFS, and have a few ZFS systems in production, and what it does is pretty amazing, but mostly in the sense of the gigabytes of RAM it consumes for basic operation (and unexplained file system wedging). I've

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Eric S Pulley
On 02/19/13 05:47, MJ wrote: Which app are you running that is generating millions of tiny files in a single directory? Regardless, in this case OpenBSD is not the right tool for the job. You need either FreeBSD or a Solaris variant to handle this problem because you need ZFS. What limits

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Matthias Appel
Am 19.02.2013 18:01, schrieb Eric S Pulley: [snip] I feel anyone expecting to run any of the recently hatched filesystem on 10+ year old hardware falls into the design flaw category you mention. As for needing to turn nobs to get it to work properly this is not necessary if you use a modern

Re: Millions of files in /var/www inode / out of space issue.

2013-02-19 Thread Jan Stary
On Tue, Feb 19, 2013 at 00:35, Keith wrote: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? newfs defaults to -f 2k and -b 16k which is fine if you know in advance you will

Millions of files in /var/www inode / out of space issue.

2013-02-18 Thread Keith
Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? The problem is that my server has the default disk layout as I didn't expect to have millions of files (I though they would be

Re: Millions of files in /var/www inode / out of space issue.

2013-02-18 Thread Zé Loff
On Tue, Feb 19, 2013 at 12:35:31AM +, Keith wrote: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? The problem is that my server has the default disk layout as I didn't

Re: Millions of files in /var/www inode / out of space issue.

2013-02-18 Thread Ted Unangst
On Tue, Feb 19, 2013 at 00:35, Keith wrote: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? Yes, newfs -i with a smaller number. Note that the number of inodes is highly

Re: Millions of files in /var/www inode / out of space issue.

2013-02-18 Thread Janne Johansson
2013/2/19 Keith ke...@scott-land.net: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? Since you probably aren't going to have 50G/2k number of files in a single dir, then you'd be