On Saturday 24 July 2010 18:43:17 Joakim Tjernlund wrote: > Kern Sibbald <k...@sibbald.com> wrote on 2010/07/24 17:56:46: > > On Saturday 24 July 2010 17:32:58 Joakim Tjernlund wrote: > > > Kern Sibbald <k...@sibbald.com> wrote on 2010/07/24 16:46:58: > > > > On Saturday 24 July 2010 16:22:54 Joakim Tjernlund wrote: > > > > > Kern Sibbald <k...@sibbald.com> wrote on 2010/07/24 14:21:46: > > > > > > On Saturday 24 July 2010 14:01:59 Joakim Tjernlund wrote: > > > > > > > Joakim Tjernlund <joakim.tjernl...@transmode.se> wrote on > > > > > > > 2010/07/20 > > > > > > > > > > > > 20:02:53: > > > > > > > > From: Joakim Tjernlund <joakim.tjernl...@transmode.se> > > > > > > > > To: bacula-devel@lists.sourceforge.net > > > > > > > > Cc: Joakim Tjernlund <joakim.tjernl...@transmode.se> > > > > > > > > Date: 2010/07/20 20:24 > > > > > > > > Subject: [Bacula-devel] [PATCH 1/2] Initial crc32 > > > > > > > > optimization. > > > > > > > > > > > > > > > > This is a strait port of the Linux kernel crc32 optimization > > > > > > > > I did a while ago. This impl. is several times faster than > > > > > > > > the one it replaces. Adapted to bacula usage. > > > > > > > > --- > > > > > > > > > > > > > > > > I have not tested this yet as I went/is on vacation but I > > > > > > > > figured perhaps someone else is interrested enough to do that > > > > > > > > :) > > > > > > > > > > > > > > > > bacula/src/lib/crc32.c | 386 > > > > > > > > ++++++++++++++++++++++++++++++++++++++++-------- 1 files > > > > > > > > changed, 325 insertions(+), 61 deletions(-) > > > > > > > > > > > > > > So it has been a few days and I was hoping for one or two > > > > > > > comments by now. > > > > > > > > > > > > I have been terribly busy with *big* problems and past due > > > > > > deadlines. > > > > > > > > > > > > > One thing I have noticed is that Bacula seems to rely on > > > > > > > dynamic endian detection instead of detecting endian at > > > > > > > configure/compile time. > > > > > > > > > > > > Yes. > > > > > > > > > > OK. > > > > > > > > > > > > Is this so and does it need to stay this way? > > > > > > > > > > > > I am not sure. A Mac developer was trying to create a single > > > > > > binary that would run on multiple architectures, which is a good > > > > > > idea if it will work. I don't know whether he actually uses it > > > > > > or not. > > > > > > > > > > ehh, how can a single binary run on multiple archs? > > > > > If all platforms use autconf I think we can let autoconf > > > > > detect the hosts endian, see AC_C_BIGENDIAN > > > > > > > > > > We need to figure this one out before I can move on. If it needs > > > > > to be dynamic then I must add an init crc32 table routine that > > > > > runs "tole" on the crc32 table. Where should this init routine > > > > > be called from? > > > > > > > > For the moment, to keep your life simple, I suggest you assume that > > > > the dynamic endian detection is not critical. If it is we can find > > > > out when it is necessary, and deal with it by either calling your > > > > routine or another one or by generating your tables in the two endian > > > > manners in each execution or something. > > > > > > > > Hopefully, you can proceed without that blocking you and we deal with > > > > it later. > > > > > > OK, lets leave it for now. > > > > > > > > > > Currently the new crc32 code needs compile time detection of > > > > > > > endianess and I like to keep it that way. Furthermore I suspect > > > > > > > I won't be able to do much more Bacula work as vacation ends in > > > > > > > one week and work at my dayjob is piling up. > > > > > > > > > > > > The first thing to do is to fill out an FLA and send it in. See > > > > > > www.bacula.org -> FLA License > > > > > > > > > > I rather not, lets just call this "a few lines of code" which won't > > > > > require a FLA. As you can see I have not claimed any CopyRight in > > > > > the patch I sent. > > > > > > > > If you release it into the public domain or put a 3 clause BSD > > > > copyright on it or LGPL or something like that I will not need the > > > > FLA, but I need either public domain (explicit) or some simply > > > > copyright to not require the FLA. > > > > > > I see, how about GPLv2 then, same as Linux ? The I could just import > > > test code and table code from there. > > > > Unfortunately, Bacula's code is not compatible with other people's code > > (Linux) licensed under GPLv2. If you are the author of the code, then > > you should be able to use any license you want. There are a number of > > choices to get it into Bacula: > > > > 1. Some BSD type license > > 2. Put it into the public domain > > 3. Put it under the Bacula license and sign an FLA > > > > If it is not your code, I need to see the original copyright to know how > > to be able to include it, but if it is not your code, then it is unlikely > > we can use it. > > It is my code, as you can see there is nothing left of the original code > after applying my patch. > Earlier you mentioned LGPL, is that OK? > > > > > > > Then the next step is to ensure that your new code and the old > > > > > > code can be run in unit test and can read files for CRC from > > > > > > STDIN. Then we can compare the > > > > > > > > > > Which unit tests? I cannot find any(note that I have never touched > > > > > Bacula code until now so I might have missed it) > > > > > > > > Most my little routines like that have a #define TEST_PROGRAM at the > > > > end that create a main(), ... and when you compile it into a binary, > > > > it tests itself, in some very primitive way. It looks like this one > > > > doesn't have such a routine. Take a look at src/lib/edit.c for a > > > > really simple case, or alist.c, or scan.c ... > > > > > > > > Actually, the code in src/lib/md5.c may be very close to what you > > > > want to change to do what I ask. > > > > > > OK, will have a look. > > > > > > > > > results of the old code with the new code and the speed. Based > > > > > > on those results we can see if we need to dynamically detect > > > > > > which algorithm to use, and if yours is not identical, and is > > > > > > faster, then we would need to modify the Volume format to account > > > > > > for this and do the dynamic detection based on the volume format. > > > > > > All future versions of Bacula *must* be able to correctly read > > > > > > older Volume versions. > > > > > > > > > > Not sure I follow you here but it seems like you thing that the new > > > > > crc32 impl. is incompatible with the current one? This is not so, > > > > > the end result, the calculated crc32, is the same(barring any impl. > > > > > bugs) as the old one. > > > > > > > > OK, I didn't re-read your previous email, but it seems to me you said > > > > you were not sure. In any case, before I can replace the current > > > > algorithm, I must > > > > > > I am not sure I didn't made a mistake in porting the code from Linux > > > into Bacula. The general idea has been proven for some time. > > > > Unfortunately, the way the GPL works, you cannot port code written by > > someone else from Linux to Bacula. The licenses are both Open Source GPL > > but not compatible. > > As stated above, it is my code that I have submitted to Linux. > > > If you wrote the code in the Linux kernel that you are porting, then you > > have the choices to license it as I indicate above ... > > Yes, I know. > > > > > prove to myself (or much faster would be you prove to me) by seeing > > > > the output that it is the same. I suggest something like the md5.c > > > > or sha1.c programs where it reads a file and creates the hash -- or > > > > in your case the CRC. > > > > > > > > > There won't be any need to test which one to use, this impl. is > > > > > much faster than the current one for all archs(unless you want to > > > > > crc one byte at a time which the Bacula bcrc32 isn't designed for) > > > > > > > > I am all for much faster code. The current CRC takes a good deal of > > > > time, and I recently added the ability to disable it, but I would > > > > really like something faster so that no one disables it :-) > > > > > > Yes, I noticed this 10 years ago when we started to use the JFFS2 file > > > system in Linux. JFFS2 crc32 all its data and it took time. At the time > > > the Linux crc32 impl. was like yours. I then optimized it significantly > > > and sent it in and all was well. Recently I did another optimization > > > pass and it was twice as fast on x86 than the previous one. > > > > If the original Linux version was like mine, then it was derived from > > "public domain code" and as long as you made the vast majority of the > > modifications, and you so desire, the code would still be "public > > domain", so you just have to explicitly put in the code that your changes > > to the RFC 2083 algoritm are public domain, then there is no license > > issue. > > There are many impl. of crc32 like similar to yours with different > licences. Here is what Linux says > * Oct 15, 2000 Matt Domsch <matt_dom...@dell.com> > * Nicer crc32 functions/docs submitted by li...@horizon.com. Thanks! > * Code was from the public domain, copyright abandoned. Code was > * subsequently included in the kernel, thus was re-licensed under the > * GNU GPL v2.
Unless they totally re-wrote it, they cannot change the license only the author can do that unless he explicitly gives you the right to copyright it. However, that is not my affair. So, if you want to move forward, I need to have a submission from you with it licensed under the LGPL, and if you want it in right away, I need to "see" that it produces the same results, otherwise it will wait until I get around to writing a test program for my code and for your new code. > > Doesn't matter though, the current code has been completely rewritten since > then. OK > > > > OT, I am pushing our IT here to use Bacula but there was one thing > > > that Bacula could not do(or so they think) that we need. > > > We backup to disk for easy access when we need to restore some file. > > > then we transfer on disk backup to tape for safe and long time storage. > > > The on disk backups gets recycled after some time after which we only > > > have the tape copy left. > > > How can we do this in Bacula and make Bacula understand where to > > > look for files, either on disk if it is still there or on the > > > corresponding tape? > > > > You backup to disk, then when you want it on tape, you run a Copy job, > > and put the copy on tape. When the disk copy is purged, Bacula will > > promote the Copy to a Backup, and it will use that version. > > I see, I will relay this info to IT, thanks. > Good luck. Kern ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
