Re: portage tree (Was: Re: reiser4 status (correction))
Thanks Christian. You can go ahead and add something to our wiki pointing to it if you would like. This might help tide people over until the repacker ships. Hans
Re: future r4 maintenance question
Hello David, Saturday, July 22, 2006, 10:16:48 PM, you wrote: > There's another note to kernel developers -- if Reiser5, 6, and 7 are > implemented as suites of plugins on top of Reiser4, then the Reiser4 > code will be maintained for a very long time. Kind of like ext2 vs > ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted with > additional mount options. This the exact information I was hoping to get. I think it's important for people to realize it. Thanks. > # Make a playlist of all files by Hendrix, mp3 or otherwise: > for i in `find`; do > if [ `cat "$i/.../song-info/artist"` == 'Jimi Hendrix' ]; then > echo "../$i" >> playlists/hendrix.m3u; > fi; > done I've read about this :-) Are plugins for such a thing entirely/mostly kernel-space? I mean, the power of FUSE is being in user-space... If there was an API to write a id3 plugin it propably be more beneficial. Well, it's plugins. If someone writes a kernelspace id3 plugin, someone else could write a kernelspace plugin with a userspace supplied configuration. -- Best regards, Maciej
Re: reiser4 status (correction)
Mike Benoit wrote: code. Even just compressing that small portion though I could probably save between 5-10GB. The difference is though I can do a df before, and a df after, and I can instantly see I got my moneys worth. Same with encryption. In the case of encryption, it's also got competition. There are two FUSE filesystems that do crypto, and there's cryptoloop/dm-crypt, which you need anyway for encrypted swap. True, it's nowhere near as nice, but it is functional. You would think one of these would be easier to develop to where it's useful. And if you're encrypting it, you're already accepting a performance hit. So again, the Reiser4 advantage here is all speed. http://parallel.vub.ac.be/~johan/compFUSEd/index.php?option=polls&task=results&pollid=31 Of course they are, or they wouldn't use FUSE. But if that's really true, why wouldn't a FUSE driver help you? Most of those FUSE file systems you linked to scare me. This is from the APFS web page: I have lost all my data! How do I get it back? From the backup, obviously." To be fair, that's what you get from any FS. But how hard would it be to make a FUSE filesystem work properly? How hard would it be to get a Reiser4 plugin to work properly? FUSE is great, but can it even come close to matching the performance of in-kernel file systems? Performance again! Not only that, but if you want to compress a directory you have to go through about a 12 step process of moving the files, setting up a mount point, and moving the files back. You listed 3 steps. Besides, I don't see anything stopping you from modifying one of these to selectively compress things, the way Reiser4 would. So, put your entire FS under a FUSE system, then configure which directories you want compressed. Again, the drawback is huge gobs of performance. It just seems natural to use the repacker if you're going to use any Reiser4-based replacement for these. You could, and we did charge by the megabyte, only after they exceeded the limit of their package. However web hosting is a fiercely competitive market, so we were constantly adjusting our packages to include more disk space, more bandwidth, more features for the same amount, or lower prices, just to compete. The way many "shared hosting" companies work is each server is waaay over sold, at least in the disk Ah, so I guess the advantage of a more expensive, dedicated box is that you know no one's overselling it? Personally, I get a little sick of the insane amounts of overselling that happen -- you just know it's going to come back and bite you in the ass. This is happening right now with bandwidth, and overselling is pretty much solely responsible for the whole Net Neutrality mess -- it would be a complete non-issue if "5 megabits to your house!" was backed up by an actual 5 megabits reserved for me, or if they sold them as "2-5 megabits", where 2 is guaranteed, and 5 is what you get when no one else is using it. I'm a bit skeptical of my local ISP's Fiber To The Home initiative, because I can't imagine they have enough spare upstream bandwidth just lying around. I mean, if you actually have enough bandwidth, you don't want or need your ISP to do QoS or prioritizing for you -- you can just do it yourself. Personally, ssh packets would be of a much higher priority to me than anything else... Thats pretty much the only way you make money with $10-20/month packages. Yeah, I know, hard for an honest guy to compete... I don't doubt the benefits of the repacker, but from a business perspective the repacker is something that runs transparently in the background, once you install it, things magically speed up then you never hear from it again as it does its job. Out of sight, out of mind. Whereas the compression/encryption plugin are always in your face, every time you run df, or enter a passphrase to gain access to your files, you know its there and working. Its something you'll tell your friends about. After you first run it, the repacker just fades away in to the background and you forget about it. Ah. This might explain the success of things like Ruby On Rails. Once it's done, the insane amount of CPU required to run a high-level interpreted language (versus even something semi-interpreted like perl) is out of sight, out of mind. But when you first set it up, the savings in development time are immediately obvious, in your face. Although I would want to find a way to avoid typing a passphrase every time -- maybe keep the key on a USB keychain. Passphrases would just annoy users. Unless, of course, you could tie it to logon -- assuming you can actually change the passphrase later... Charging only for commercial use of the repacker/compression/encryption plugin would be a great middle ground. Good, I'm glad it wasn't a completely insane idea.
Re: future r4 maintenance question
Maciej Sołtysiak wrote: I Understand. This is good news. Hm, do you think that reiser4's pluggability is enough to have this single kernel tree (fs/reiser4) for a longer period away with the "future vision" and other "future *" stuff you've written about ? eg. I remember reading about very granular security ACLs like restricting a certain line in a file (like /etc/passwd) I think the granular security ACLs were always intended to be a v4 thing (with plugins). I believe the Future Vision stuff is about even more namespace unification. I seem to remember v5 was semantic changes and v6 was a semi-distributed FS, but I could have that backwards... Anyway, there are definitely plans for things which are non-trivially beyond the plugin capabilities of Reiser4, though they will almost certainly use Reiser4 as a starting point. There's another note to kernel developers -- if Reiser5, 6, and 7 are implemented as suites of plugins on top of Reiser4, then the Reiser4 code will be maintained for a very long time. Kind of like ext2 vs ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted with additional mount options. There is definitely a lot that can be done to move Reiser4 (as it is today) closer to the Reiser4 whitepaper on the homepage. ACLs are one thing, files as a directory are another. The idea of v4 is to do away with many cases where a separate namespace is created for no good reason -- for instance, where is the data in an id3 tag? It's inside an mp3 file, and you can only get to it with tools written for id3 tags of mp3 files. The Reiser4 concept is to allow things like that to exist, but not require programs to know about libid3 or whatever. Want to know what the artist of a particular file is? foo.mp3/.../id3/artist Or maybe a more generic way: foo.mp3/.../song-info/artist That way, you could have tools which don't even have to know if the file has an id3 tag, or something entirely new, or if the metadata is being stored outside the main file. It'd be entirely possible to allow that file to be treated as a separate file entirely by the plugin, rather than something derived from foo.mp3. The advantages don't seem immediately obvious until you consider that the program which does this doesn't have to even know that it's dealing with song metadata. Consider some of the one-line shell scripts possible: # Change the artist name for all songs in the directory: for i in *; do echo 'Jimi Hendrix' > foo.mp3/.../song-info/artist; done # Make a playlist of all files by Hendrix, mp3 or otherwise: for i in `find`; do if [ `cat "$i/.../song-info/artist"` == 'Jimi Hendrix' ]; then echo "../$i" >> playlists/hendrix.m3u; fi; done # Copy all files needed by said playlist to a USB device: cp playlist/hendrix.m3u/.../files/* /mnt/usb cp playlist/hendrix.m3u /mnt/usb I'm sure others can think of much more interesting examples. All of that is planned for v4, eventually. It's very pluggable. Well, I think it is. I don't work here...
Re: reiser4 status (correction)
Hello Andreas, Saturday, July 22, 2006, 1:06:54 AM, you wrote: > On 17:45 Fri 21 Jul , David Masover wrote: >> Question, then: Can the ext2 defrag work on a raw ext3 partition, without >> having to convert it first? > Dunno, but I don't think so I tried that once, back in 2002 i think. defrag seemed doing things fine, but thrashed the filesystem very badly. The good thing is, that I was able to fsck it and recover most of my files. However it is not something you should try. -- Best regards, Maciej
Re: future r4 maintenance question
Hello Mike, Saturday, July 22, 2006, 8:53:06 PM, you wrote: > I don't understand the concerns here? Kernel v2.4 is essentially in > "bug-fix only" mode right now, yet I don't see many people complaining > that the kernel developers have abandoned it. I personally think its > crazy to expect major features be back ported to old versions. > For as long as I've been alive programmers have released a major version > of their software, and only maintained it with minor version bug fixes > while they work on the next major version to introduce features to it. > Hans himself has pretty much followed that same path with ReiserFS V3, > and now V4. There just happened to be a several year gap in between. Well, that's completely clear. However, reiserfs has had a different situation: during the bugfix mode, important functionality was beeing added to other filesystems: extended attribute support, data=ordered, data=journaled. Reiserfs v3 had noone but Chris Mason (outside of namesys) to do it. The issue is that reiser4 users should never again feel abandoned like they did with reiserfs v3. Well, actually a lot of people did not feel it or they hadn't known about the state of things. Note that back then reiser4 was basically vapourware if I may say so. Concluding, I am not here to point anything bad, I am looking adoption-stoppers for reiser4. BTW, have Mark Shuttleworth and Hans had a talk about support/sponsorship/ubuntu endorsement/anything? (i know it may be absolutely private stuff, sorry for asking if this is the case) -- Best regards, Maciej
Re: reiser4 status (correction)
On Sat, 2006-07-22 at 07:34 -0500, David Masover wrote: > The compression will probably mostly be about speed. Remember, if > we're > talking about people who want to see tangible, visceral results, > we're > probably also talking about end-users. And trust me, the vast > majority > of most of my data (as an end-user) is not very compressible. > Sure, mine too. Between the many gigs of MP3s and movies I have stored on my HD, only about 10-20GB is the OS, Email, and documents/source code. Even just compressing that small portion though I could probably save between 5-10GB. The difference is though I can do a df before, and a df after, and I can instantly see I got my moneys worth. Same with encryption. With the repacker it is much more difficult (for average users) to time how long a program takes to load some file (or launch), before the repacker and after. Especially since caching comes in to play. Also, according to this little poll on one of the compressed FUSE sites you linked to, more people are looking to compression for space saving, then for speed: http://parallel.vub.ac.be/~johan/compFUSEd/index.php?option=polls&task=results&pollid=31 > > No, mostly we're talking about things like office documents, the > majority of which fit in less than a gigabyte, and multimedia (music, > movies, games) which will gain very little from compression. If > anything, the benefit would be mostly in compressing software. > > > less tangible like fragmentation percentages and minor I/O throughput > > improvements. I used to work at a large, world wide web hosting company > > and I could see making a case to management for purchasing Reiser4 > > compression would be pretty easy for our shared servers. Instantly > > freeing up large amounts of disk space (where .html/.php files were the > > vast majority) would save huge amounts of money on disk drives, > > especially since most of the servers used RAID1 and adding new drives > > was a huge pain in the neck. Making a case to purchase a repacker would > > be much, much more difficult. > > Hmm, the problem is, if storage space is really the big deal, it's been > done before, and some of these efforts are still usable and free: > > http://parallel.vub.ac.be/~johan/compFUSEd/ > http://www.miio.net/fusecompress/ > http://north.one.pl/~kazik/pub/LZOlayer/ > http://apfs.humorgraficojr.com/apfs_ingles.html > > And while we're on the topic, here's an FS that does unpacking of > archives, probably about the same way we imagined it in Reiser4 > pseudofiles/magic: > > http://www.nongnu.org/unpackfs/ > > But regardless, as far as I can tell, the only real, tangible benefit of > using Reiser4 compression instead of one of those four FUSE filesystems > is speed. Reiser4 would compress/decompress when actually hitting the > disk, not just the FS, and it would also probably use in-kernel > compression, rather than calling out to userspace on every FS operation. > > But you see, if you're talking about speed, 10% is a respectably big > improvement, so I could see selling them on a repacker at the same time. > Most of those FUSE file systems you linked to scare me. This is from the APFS web page: "Permissions do not work! Do not worry, it is not your fault, permissions are not implemented yet. It is like all files had rwxrwxrwx mode. I have lost all my data! How do I get it back? From the backup, obviously." FUSE is great, but can it even come close to matching the performance of in-kernel file systems? Not only that, but if you want to compress a directory you have to go through about a 12 step process of moving the files, setting up a mount point, and moving the files back. Will Reiser4 not allow us to mount with compression enabled then enable/disable compression on a per file/per directory basis? > Maybe bundles are a good idea... > Maybe there should be a Reiser4 Whitepaper Value Pack, once everything > on the whitepaper is done? > > > See, customers who used lots of CPU were easy to up-sell to a dedicated > > server because page load times were tangible and if they didn't move we > > would be forced to shut them off. However customers who used gobs of > > disk space were much more difficult to up-sell to dedicated servers > > because it didn't affect themselves or other customers in a tangible > > way. They wouldn't notice any difference by moving to a much more > > expensive dedicated server. > > Sounds like more a marketing problem than a technical one. Couldn't you > just charge more on the virtual server? Or start charging by the megabyte? You could, and we did charge by the megabyte, only after they exceeded the limit of their package. However web hosting is a fiercely competitive market, so we were constantly adjusting our packages to include more disk space, more bandwidth, more features for the same amount, or lower prices, just to compete. The way many "shared hosting" companies work is each server is waaay over sold, at l
Re: future r4 maintenance question
Hello Hans, Saturday, July 22, 2006, 8:03:28 PM, you wrote: > We are going to give changing the paradigm a try. The difference > between 4.1-beta and 4.0 is that different plugins are the default, and > the experimental code is in the plugins you see when mounting with the > mount option 4.1-beta. Let's see if it works in practice. I Understand. This is good news. Hm, do you think that reiser4's pluggability is enough to have this single kernel tree (fs/reiser4) for a longer period of time. I mean, can you predict a need of spawning something like reiser5 in the forseeable future or would fs/reiser4 + plugins be enough to do away with the "future vision" and other "future *" stuff you've written about ? eg. I remember reading about very granular security ACLs like restricting a certain line in a file (like /etc/passwd) -- Best regards, Maciej
Re: future r4 maintenance question
Mike Benoit wrote: Just like many other applications that support plugins/extensions/addons ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised if Reiser4 doesn't spawn a large community of people completely independent of Namesys that create plugins from the useless, to the amazingly useful for years to come. You would hope. The problem is, most of the ideas we have for amazing/useless/practical plugins have already been done as specialized filesystems in FUSE. What's more, FUSE doesn't usually require root privileges to run, causing some people to do such useless things as support for various filesystems already in the kernel (so you can loop-mount filesystems without using the loop device or having root privileges). Although, there is one other thing to consider -- NTFS in the kernel has stagnated for something like 10 years without write support, while userspace tools like ntfsck and resize_ntfs, using libntfs, have actually gotten relatively stable at dealing with all kinds of ntfs oddities. For awhile, we had captive-ntfs (also done in FUSE), but more recently, someone wrote a FUSE driver for libntfs, making the first reasonably stable fully featured read/write NTFS driver for Linux. It's also reasonably fast, sometimes faster than ext2/3, I'm told. The big advantage of that is, FUSE is easier to deal with than the kernel, and userland daemons can crash, but if something in the kernel crashes, there goes your OS. Also, userland just has more options. Some FUSE drivers are written in Perl, others in Ruby. I realize there are some things that are really better done in the kernel, like crypto/compression support, and Reiser4 will support things that FUSE currently doesn't do very well. But if we're to pull any filesystem hackers away from FUSE, we need to provide as good or better an API, both in-kernel, and for userspace daemons.
Re: future r4 maintenance question
Pysiak Satriani wrote: Hello, one of the concerns of LKML folks about r4 is that namesys after finishing their work with r4 will "rush off" to reiser5 leaving r4 as "stable" in a "bugfix-mode only" But a question arises. Will namesys keep r4 in a bugfix-only mode (as with reiserfs v3) or will it have resources planned not only for bugfixes but for updates too ? As I understand it, r4 is currently not finished, especially comparing the whitepaper with reality. So, while the features that are finished are mostly stable, I can't imagine them rushing off to v5 without finishing the features originally planned for v4. Examples would be pseudofiles, custom plugins (per-file and per-directory, also), a repacker, fsync optimization, a crypto/compression plugin... Ultimately, this may happen, but by then, I expect v4 would be fairly stable and complete. Besides, bugfix-only does imply bugfixes, which is about all most other filesystems get these days... Disclaimer: I don't work here. If you want a definitive answer, wait for someone at Namesys to confirm or deny what I've just said.
Re: future r4 maintenance question
Just like many other applications that support plugins/extensions/addons ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised if Reiser4 doesn't spawn a large community of people completely independent of Namesys that create plugins from the useless, to the amazingly useful for years to come. if it ever gets finished ;) :)
Re: future r4 maintenance question
We are going to give changing the paradigm a try. The difference between 4.1-beta and 4.0 is that different plugins are the default, and the experimental code is in the plugins you see when mounting with the mount option 4.1-beta. Let's see if it works in practice. Hans Pysiak Satriani wrote: >Hello, > >one of the concerns of LKML folks about r4 is that namesys after >finishing their work with r4 will "rush off" to reiser5 leaving >r4 as "stable" in a "bugfix-mode only" > >I understand that way of things - you don't mess with a "stable" branch >software. > >But a question arises. Will namesys keep r4 in a bugfix-only mode >(as with reiserfs v3) or will it have resources planned not only >for bugfixes but for updates too ? > >I think that's a thing that needs to be very clear. > >Best regards, >Maciej > > > > > >
Re: future r4 maintenance question
On Sat, 2006-07-22 at 17:32 +0200, Pysiak Satriani wrote: > Hello, > > one of the concerns of LKML folks about r4 is that namesys after > finishing their work with r4 will "rush off" to reiser5 leaving > r4 as "stable" in a "bugfix-mode only" > > I understand that way of things - you don't mess with a "stable" branch > software. > > But a question arises. Will namesys keep r4 in a bugfix-only mode > (as with reiserfs v3) or will it have resources planned not only > for bugfixes but for updates too ? > > I think that's a thing that needs to be very clear. > I don't understand the concerns here? Kernel v2.4 is essentially in "bug-fix only" mode right now, yet I don't see many people complaining that the kernel developers have abandoned it. I personally think its crazy to expect major features be back ported to old versions. For as long as I've been alive programmers have released a major version of their software, and only maintained it with minor version bug fixes while they work on the next major version to introduce features to it. Hans himself has pretty much followed that same path with ReiserFS V3, and now V4. There just happened to be a several year gap in between. I don't know what Hans has in mind in regards to ReiserV5, but ReiserV4 was designed with plugins in mind so adding additional features (even advanced ones) to it should be much easier than most other file systems. Just like many other applications that support plugins/extensions/addons ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised if Reiser4 doesn't spawn a large community of people completely independent of Namesys that create plugins from the useless, to the amazingly useful for years to come. -- Mike Benoit <[EMAIL PROTECTED]> signature.asc Description: This is a digitally signed message part
portage tree (Was: Re: reiser4 status (correction))
Hi,
The portage tree is such a fine testing object since it should be sort
of a "best case scenario" for reiser filesystems, and needs no real
backup in case of a screwup during tests.
I've been on Gentoo for years now, used reiser3 since the days when you
had to patch it into a 2.2 kernel and tried out reiser4 on my portage
tree first of all. During my comparison, reiser3 + notail resulted in
huge amounts of wasted disk space, obviously it is not very smart to use
up a whole 4k block for a file of only few bytes. reiser3 with packed
tails was a lot better, but still it was a remarkable difference from
there to reiser4, and I am only talking storage efficiency here which
directly translates to the amount of I/O necessary to get the same data
all the way to RAM and CPU. The reiser4 dev team did a great job there!
With reiser3, I made another observation wrt. fragmentation, most of
which should be alleviated by reiser4 and its delayed allocation. Over
time, doing package database updates and querying for pending updates to
the local machine became slower. Packing up everything in a tarball,
mkfs, mount with -onoatime, unpack usually fixed the "problem". Since
the performance reduction in reiser4 is a lot less (I can hardly tell if
it makes a difference before vs. after the backup/restore "repacking") I
hardly do it any more, but once in a while I still use my script
initially written for the comparison test.
Attached is the shell script I used for comparing various mount options
for ReiserFS v3 and later v4. It is capable of converting between
ext2|3, reiser3|4 and xfs, provided the target fs stores the data
efficiently enough that everything fits inside. (I had "interesting"
results when moving from reiser3 to ext3 for testing, esp. wrt. portage
tree ; )
The script falls short at handling arbitrary mount options, but may be
trivially edited to use any desired options for a single use. Reading
and understanding is recommended before feeding your viable data to it.
I hope somebody may find it useful, or at least inspiring. Any comments
appreciated!
Kind regards,
Chris
#!/bin/sh
# ! WARNING !
# As always, BE CAREFUL AND KNOW WHAT YOU DO WHEN RUNNING THIS PROGRAM!
# At best, read the code, understand what it does, and take appropriate
# precautions. Backups of valuable data are a requirement anyway, so go
# for it BEFORE you try this out!
#
#
# SYNOPSIS:
# fsconvert.sh
#
# where fstype may be any of ext[2|3], reiser[fs|4] or xfs
# and compression method is a choice of compress, gzip or bzip2.
#
#
# USE:
# This script was initially intended to re-create old reiserfs 3
# filesystems for performance reasons, and can reasonably be considered
# useful when run once every year or two for any filesystem under heavy
# read/write load. Simply give the existing filesystem type as the fstype
# argument.
#
# Though it was written for that single purpose at first, it was recently
# adapted for filesystem type migration kind of operations, as it did
# never care about the fs type in the beginning and now takes the
# destination fs type as an argument.
#
# The script REQUIRES the concerned fs to be mounted, to have an entry in
# /etc/fstab, to have no further active mountpoints in a subdirectory,
# enough free disk space in $archdir to back up data and of course the
# required tools (mkfs., tar, compression tool of choice) to do
# all the magic.
#
# Please send any comment, modification, failure report etc. to
# ctrefzer AT gmx DOT de
#
# Have fun!
# Chris
# Changelog:
#
# 2005-10-03
# Enhanced commandline interface to support multiple target file system types
# and compression methods.
# Filesystems supported so far:
# - ext2 / 3
# - reiser 3 / 4
# - xfs
#
# Compression methods which seemed appropriate:
# - none (uncompressed archive for almost no CPU load)
# - compress (quite fast even on old machines)
# - gzip (realtime on new machines, yet impressive I/O reduction)
# - bzip2 (in case of tight disk space, but takes ages wrt. the others)
### Configure here:
# where to store the archives:
archdir="${PWD}/.fsconvert"
# Sane defaults for compression levels (and guidelines): use fast gzip on
# modern CPUs where GZIP -3 gives better compression than LZW, yet still at
# realtime, whereas BZIP2 should only be used when disk space is _really_
# critical - it will take forever. LZW (compress) is advised on not-so-recent
# machines to save some I/O without slowing things down. A nifty compromise
# may be GZIP -1 on P2/P3 class machines.
export GZIP="-1"
export BZIP2="-9"
### No modifications required below here!
### Some helper functions:
function e_report {
echo -en "${1}..."
}
function e_done {
# Paranoia is good! It's all about the data...
sync
echo " Done."
}
function e_fail {
echo " FAILED!"
exit ${1}
}
function e_usage {
echo "USAGE: ${0} "
exit 1
}
if [ -z ${1} ]
then
e_usage
else
case ${1} in
future r4 maintenance question
Hello, one of the concerns of LKML folks about r4 is that namesys after finishing their work with r4 will "rush off" to reiser5 leaving r4 as "stable" in a "bugfix-mode only" I understand that way of things - you don't mess with a "stable" branch software. But a question arises. Will namesys keep r4 in a bugfix-only mode (as with reiserfs v3) or will it have resources planned not only for bugfixes but for updates too ? I think that's a thing that needs to be very clear. Best regards, Maciej
Re: storing images thumbnails as pseudo files?
Hello Hans, Saturday, July 22, 2006, 2:46:55 AM, you wrote: > David Masover wrote: >> Disclaimer: I don't speak for Namesys, and I don't work here. While >> I'm pretty confident I understand their vision, the final word on >> anything Reiser is always from Hans Reiser. > David described my views pretty well, and saved me much typing.:) :-) Hehe. David, Hans, thanks for your answers. Now I'm really looking forward to r4. -- Maciek
Re: reiser4 status (correction)
Mike Benoit wrote: Could you not also write a small little app that gathers all kinds of stats about a file system and sends it to a Namesys server in hopes of finding better statistical data? I'm sure there are thousands of users Assuming the results are all made available, essentially public domain. If it becomes "improve Reiser" instead of "improve filesystems", then only fans of Reiser will do it. A much better approach in my opinion would be to have Reiser4 perform well in the majority of cases without the repacker, and sell the repacker to peopleisn' something who need that extra bit of performance. If I'm not mistaken this is actually Hans intent. Hans? Yes, that's the idea. Only sysadmins of large corps are likely to buy. We throw in service and support as well for those who purchase it. Personally, as much as I would like it all to be free, I think I would be much more willing to pay for compression/encryption (on both servers and desktops) then I would be for a repacker. Hard disks cost money, and if I can compress the vast majority of my data and save on purchasing a new hard disk, that is well worth it. I also have some important data that I would really like to encrypt, which is also worth spending money on. But gaining ~10% in performance probably isn't worthwhile spending money on as I most likely wouldn't notice a difference in my day to day life, unless my server was incredibly busy. Assuming the performance gain is only 10%, you may be right. Still, faster disks, controllers, buses, and CPUs also cost money. I would be willing to pay for both, even, if the price was reasonable (I think there was a scheme based on amount of space placed under the FS?), and if I could be guaranteed updates (bug fixes, new kernels, etc) for at least the functionality that I've paid for, with no additional cost. Reiser4 lazy writes make a huge difference on a laptop, and my current laptop is a Mac. That means that around when the next Mac OS rolls out, it will be perfectly reasonable to spend some $50 or $100 to make my Linux faster instead of my Mac OS, especially since I'm trying to migrate off of Mac OS as much as possible. (I miss package management.) One more suggestion: Maybe make it free for non-commercial use only? And by that I mean, based on how the FS is actually being used. I don't know if piracy is or has been a problem for Namesys, but at least the economics of it makes sense: A fifteen year old hacker won't want to pay for an FS, but might have a lot to contribute. But if your main market is large servers, have those people pay -- they are running their business off your FS. There is no doubt there is a market for a repacker, but I think people are much more likely to spend money on something that is immediately tangible, like disk space instantly being free'd up by compression, or data instantly being encrypted. As compared to something that is much The compression will probably mostly be about speed. Remember, if we're talking about people who want to see tangible, visceral results, we're probably also talking about end-users. And trust me, the vast majority of most of my data (as an end-user) is not very compressible. Ok, I lied: I love games, and you can make a very modern-looking game in 96K: http://produkkt.abraxas-medien.de/kkrieger But while that could be seen as a kind of compression, it's done by hand, and is fairly irrelevant to filesystems. No, mostly we're talking about things like office documents, the majority of which fit in less than a gigabyte, and multimedia (music, movies, games) which will gain very little from compression. If anything, the benefit would be mostly in compressing software. less tangible like fragmentation percentages and minor I/O throughput improvements. I used to work at a large, world wide web hosting company and I could see making a case to management for purchasing Reiser4 compression would be pretty easy for our shared servers. Instantly freeing up large amounts of disk space (where .html/.php files were the vast majority) would save huge amounts of money on disk drives, especially since most of the servers used RAID1 and adding new drives was a huge pain in the neck. Making a case to purchase a repacker would be much, much more difficult. Hmm, the problem is, if storage space is really the big deal, it's been done before, and some of these efforts are still usable and free: http://parallel.vub.ac.be/~johan/compFUSEd/ http://www.miio.net/fusecompress/ http://north.one.pl/~kazik/pub/LZOlayer/ http://apfs.humorgraficojr.com/apfs_ingles.html And while we're on the topic, here's an FS that does unpacking of archives, probably about the same way we imagined it in Reiser4 pseudofiles/magic: http://www.nongnu.org/unpackfs/ But regardless, as far as I can tell, the only real, tangible benefit of using Reiser4 compression instead of one of those four FUSE filesystems is s
Re: rephrase..
On Sat, 22 Jul 2006 12:48:48 +0200 Dirk <[EMAIL PROTECTED]> wrote: > will fsck.reiser4 --build-fs repair the partition by formatting it?? No, you should get your data back, unless you hit a bug in fsck (I am not aware of any such report recently). The only problem with fsck.reiser4 is that it's hard to figure out what files got actually damaged - most of its messages are probably useful only to people familiar with Reiser4 internals. If you are running tripwire or something similar, I recommend rechecking the files as soon as you repair the filesystem. -- Jindrich Makovicka
rephrase..
will fsck.reiser4 --build-fs repair the partition by formatting it?? Thanks, Dirk
Re: serious Reiser4 fsck problem
Łukasz Mierzwa wrote: > Dnia Sat, 22 Jul 2006 12:08:58 +0200, Dirk <[EMAIL PROTECTED]> napisał: > >> Hello, >> will using >> >> fsck.reiser4 --build-fs >> >> result in data loss? I mean to fix the partition - NOT to lose the data >> on it.. >> >> The manual page is also not very specific about if data will be lost or >> not when using this option... >> >> Using --fix resulted in a message that I should use --build-fs to remove >> corruption... > > > You will probably end up with some amount of files in lost&found dir, > those will be the files that fsck can't recognize from what dir are > coming and what are named, depending on what corruption You have there > can always be some files that You won't get back. As always, backups > are a must have on any fs. > > The hdd was full and a process was still trying to write and create new files... So I guess those files might end up in lost+found... Just want to make sure that --build-fs will not build a _new_ fs, meaning formatting the fs by formatting it?!? That wasn't very clear from what I'd read in the manual... Dirk
Re: serious Reiser4 fsck problem
Dnia Sat, 22 Jul 2006 12:08:58 +0200, Dirk <[EMAIL PROTECTED]> napisał: Hello, will using fsck.reiser4 --build-fs result in data loss? I mean to fix the partition - NOT to lose the data on it.. The manual page is also not very specific about if data will be lost or not when using this option... Using --fix resulted in a message that I should use --build-fs to remove corruption... You will probably end up with some amount of files in lost&found dir, those will be the files that fsck can't recognize from what dir are coming and what are named, depending on what corruption You have there can always be some files that You won't get back. As always, backups are a must have on any fs.
serious Reiser4 fsck problem
Hello, will using fsck.reiser4 --build-fs result in data loss? I mean to fix the partition - NOT to lose the data on it.. The manual page is also not very specific about if data will be lost or not when using this option... Using --fix resulted in a message that I should use --build-fs to remove corruption... Thanks, Dirk
Re: reiser4 status (correction)
On Fri, 2006-07-21 at 23:53 -0600, Hans Reiser wrote: > We have 3 levels of optimization: 1) at each modification, 2) at each > flush, and 3) at each repack. Each of these operates on a different > time scale, and all 3 are worthy of doing as right as we can. > > Now, the issue of where should airholes be? Why, that is the grand > experiment that will start to happen in a few months. Nobody knows yet > what defaults we should have, and whatever we choose, there will be some > users who gain from explicit control of it. > Wouldn't the most recently modified files give a good hint to the repacker? The files larger then a few kilobytes most recently modified would be packed with air holes, and the files least recently modified would be packed more tightly? Perhaps even files that are currently the most fragmented would be repacked with air holes, as compared to files least fragmented would be packed more tightly. Could you not also write a small little app that gathers all kinds of stats about a file system and sends it to a Namesys server in hopes of finding better statistical data? I'm sure there are thousands of users that would be willing to run this app for the "greater good" regardless or not if they used ReiserFS in the first place. Things like the number of files on the disk, what percentage of those files have been modified in the last week, which files are the most/least fragmented, and when they were last modified, etc...? > > > >> A much better approach in my opinion would be to have Reiser4 perform > >> well in the majority of cases without the repacker, and sell the > >> repacker to peopleisn' something who need that extra bit of performance. > >> If I'm not > >> mistaken this is actually Hans intent. > > > > > > Hans? > > Yes, that's the idea. Only sysadmins of large corps are likely to buy. > We throw in service and support as well for those who purchase it. > > If I was making money, I would not do this, but I am not. I am not > really willing to work a day job for the rest of my life supporting guys > in Russia, it is only ok to do as a temporary measure. I am getting > tired > Personally, as much as I would like it all to be free, I think I would be much more willing to pay for compression/encryption (on both servers and desktops) then I would be for a repacker. Hard disks cost money, and if I can compress the vast majority of my data and save on purchasing a new hard disk, that is well worth it. I also have some important data that I would really like to encrypt, which is also worth spending money on. But gaining ~10% in performance probably isn't worthwhile spending money on as I most likely wouldn't notice a difference in my day to day life, unless my server was incredibly busy. There is no doubt there is a market for a repacker, but I think people are much more likely to spend money on something that is immediately tangible, like disk space instantly being free'd up by compression, or data instantly being encrypted. As compared to something that is much less tangible like fragmentation percentages and minor I/O throughput improvements. I used to work at a large, world wide web hosting company and I could see making a case to management for purchasing Reiser4 compression would be pretty easy for our shared servers. Instantly freeing up large amounts of disk space (where .html/.php files were the vast majority) would save huge amounts of money on disk drives, especially since most of the servers used RAID1 and adding new drives was a huge pain in the neck. Making a case to purchase a repacker would be much, much more difficult. See, customers who used lots of CPU were easy to up-sell to a dedicated server because page load times were tangible and if they didn't move we would be forced to shut them off. However customers who used gobs of disk space were much more difficult to up-sell to dedicated servers because it didn't affect themselves or other customers in a tangible way. They wouldn't notice any difference by moving to a much more expensive dedicated server. I would like to see Namesys succeed and become incredibly profitable for Hans, if nothing else for the fact that he has given a huge amount to the open source community already. A profitable Namesys only means we'll have a greater chance of seeing even more interesting stuff from them in the future. -- Mike Benoit <[EMAIL PROTECTED]> signature.asc Description: This is a digitally signed message part
