Hello, Le vendredi 28 mai 2010 10:05:01, Henrik Johansen a écrit : > Hi, > > > You are right, we have a current project to add a new device format that > > will be able to be compatible with dedup layer. I don't know yet how it > > will work because I can imagine that each dedup system works > > differently, and finding a common denominator won't be easy. A first > > proof of concept will certainly use LessFS (It is already in my radar > > scope). But as you said, depending on block size, alignment, etc... it's > > not so easy. > > I can provide access (with KVM capabilities) to one or more > (Open)Solaris machine running dedup enabled ZFS with any version of > Bacula you may desire for as long as you need and with as much storage / > CPU / RAM as you need.
Ok thanks you, but we are still on the drawing board :-) > WRT the volume format - if you can provide tunables to adjust the volume > block size (powers of 2 within the 4-128 KB range) so that it matches a > given dedup enabled filesystem and keep the associated job information > out of the volume itself (perhaps in a seperate metadata file) that > would be a wonderful starting point for most dedup solutions. The biggest problem I see, is how to store efficiently small files and big one. For example, if storing 200 bytes requires a block of 4KB, it's not so nice. (Same if we need 2 blocks of 4KB to store 4.1KB) Now, maybe what we loose with alignment will be compensated by FS dedup and compression (or sparse things). We can also take ideas from filesystem like zfs, btrfs or reiserfs. They store small files directly with metadata, so their usage is very fast and it doesn't disturb big chunks. After a certain point, it can also be easier to implement dedup our self rather than relying on a FS. For example, we can imagine having a migration job that do deduplication during the process, so the result will store data and references to other volumes. In any cases, this is a very interesting subject. Bye -- Need professional help and support for Bacula ? Visit http://www.baculasystems.com ------------------------------------------------------------------------------ _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel
