Kern Sibbald <k...@sibbald.com> wrote on 2010/07/24 20:46:24: > On Saturday 24 July 2010 18:43:17 Joakim Tjernlund wrote: > > Kern Sibbald <k...@sibbald.com> wrote on 2010/07/24 17:56:46: > > > On Saturday 24 July 2010 17:32:58 Joakim Tjernlund wrote: > > > > Kern Sibbald <k...@sibbald.com> wrote on 2010/07/24 16:46:58: > > > > > On Saturday 24 July 2010 16:22:54 Joakim Tjernlund wrote: > > > > > > Kern Sibbald <k...@sibbald.com> wrote on 2010/07/24 14:21:46: > > > > > > > On Saturday 24 July 2010 14:01:59 Joakim Tjernlund wrote: > > > > > > > > Joakim Tjernlund <joakim.tjernl...@transmode.se> wrote on > > > > > > > > 2010/07/20 > > > > > > > > > > > > > > 20:02:53: > > > > > > > > > From: Joakim Tjernlund <joakim.tjernl...@transmode.se> > > > > > > > > > To: bacula-devel@lists.sourceforge.net > > > > > > > > > Cc: Joakim Tjernlund <joakim.tjernl...@transmode.se> > > > > > > > > > Date: 2010/07/20 20:24 > > > > > > > > > Subject: [Bacula-devel] [PATCH 1/2] Initial crc32 > > > > > > > > > optimization. > > > > > > > > > > > > > > > > > > This is a strait port of the Linux kernel crc32 optimization > > > > > > > > > I did a while ago. This impl. is several times faster than > > > > > > > > > the one it replaces. Adapted to bacula usage. > > > > > > > > > --- > > > > > > > > > > > > > > > > > > I have not tested this yet as I went/is on vacation but I > > > > > > > > > figured perhaps someone else is interrested enough to do that > > > > > > > > > :) > > > > > > > > > > > > > > > > > > bacula/src/lib/crc32.c | 386 > > > > > > > > > ++++++++++++++++++++++++++++++++++++++++-------- 1 files > > > > > > > > > changed, 325 insertions(+), 61 deletions(-) > > > > > > > > > > > > > > > > So it has been a few days and I was hoping for one or two > > > > > > > > comments by now. > > > > > > > > > > > > > > I have been terribly busy with *big* problems and past due > > > > > > > deadlines. > > > > > > > > > > > > > > > One thing I have noticed is that Bacula seems to rely on > > > > > > > > dynamic endian detection instead of detecting endian at > > > > > > > > configure/compile time. > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > OK. > > > > > > > > > > > > > > Is this so and does it need to stay this way? > > > > > > > > > > > > > > I am not sure. A Mac developer was trying to create a single > > > > > > > binary that would run on multiple architectures, which is a good > > > > > > > idea if it will work. I don't know whether he actually uses it > > > > > > > or not. > > > > > > > > > > > > ehh, how can a single binary run on multiple archs? > > > > > > If all platforms use autconf I think we can let autoconf > > > > > > detect the hosts endian, see AC_C_BIGENDIAN > > > > > > > > > > > > We need to figure this one out before I can move on. If it needs > > > > > > to be dynamic then I must add an init crc32 table routine that > > > > > > runs "tole" on the crc32 table. Where should this init routine > > > > > > be called from? > > > > > > > > > > For the moment, to keep your life simple, I suggest you assume that > > > > > the dynamic endian detection is not critical. If it is we can find > > > > > out when it is necessary, and deal with it by either calling your > > > > > routine or another one or by generating your tables in the two endian > > > > > manners in each execution or something. > > > > > > > > > > Hopefully, you can proceed without that blocking you and we deal with > > > > > it later. > > > > > > > > OK, lets leave it for now. > > > > > > > > > > > > Currently the new crc32 code needs compile time detection of > > > > > > > > endianess and I like to keep it that way. Furthermore I suspect > > > > > > > > I won't be able to do much more Bacula work as vacation ends in > > > > > > > > one week and work at my dayjob is piling up. > > > > > > > > > > > > > > The first thing to do is to fill out an FLA and send it in. See > > > > > > > www.bacula.org -> FLA License > > > > > > > > > > > > I rather not, lets just call this "a few lines of code" which won't > > > > > > require a FLA. As you can see I have not claimed any CopyRight in > > > > > > the patch I sent. > > > > > > > > > > If you release it into the public domain or put a 3 clause BSD > > > > > copyright on it or LGPL or something like that I will not need the > > > > > FLA, but I need either public domain (explicit) or some simply > > > > > copyright to not require the FLA. > > > > > > > > I see, how about GPLv2 then, same as Linux ? The I could just import > > > > test code and table code from there. > > > > > > Unfortunately, Bacula's code is not compatible with other people's code > > > (Linux) licensed under GPLv2. If you are the author of the code, then > > > you should be able to use any license you want. There are a number of > > > choices to get it into Bacula: > > > > > > 1. Some BSD type license > > > 2. Put it into the public domain > > > 3. Put it under the Bacula license and sign an FLA > > > > > > If it is not your code, I need to see the original copyright to know how > > > to be able to include it, but if it is not your code, then it is unlikely > > > we can use it. > > > > It is my code, as you can see there is nothing left of the original code > > after applying my patch. > > Earlier you mentioned LGPL, is that OK? > > > > > > > > > Then the next step is to ensure that your new code and the old > > > > > > > code can be run in unit test and can read files for CRC from > > > > > > > STDIN. Then we can compare the > > > > > > > > > > > > Which unit tests? I cannot find any(note that I have never touched > > > > > > Bacula code until now so I might have missed it) > > > > > > > > > > Most my little routines like that have a #define TEST_PROGRAM at the > > > > > end that create a main(), ... and when you compile it into a binary, > > > > > it tests itself, in some very primitive way. It looks like this one > > > > > doesn't have such a routine. Take a look at src/lib/edit.c for a > > > > > really simple case, or alist.c, or scan.c ... > > > > > > > > > > Actually, the code in src/lib/md5.c may be very close to what you > > > > > want to change to do what I ask. > > > > > > > > OK, will have a look. > > > > > > > > > > > results of the old code with the new code and the speed. Based > > > > > > > on those results we can see if we need to dynamically detect > > > > > > > which algorithm to use, and if yours is not identical, and is > > > > > > > faster, then we would need to modify the Volume format to account > > > > > > > for this and do the dynamic detection based on the volume format. > > > > > > > All future versions of Bacula *must* be able to correctly read > > > > > > > older Volume versions. > > > > > > > > > > > > Not sure I follow you here but it seems like you thing that the new > > > > > > crc32 impl. is incompatible with the current one? This is not so, > > > > > > the end result, the calculated crc32, is the same(barring any impl. > > > > > > bugs) as the old one. > > > > > > > > > > OK, I didn't re-read your previous email, but it seems to me you said > > > > > you were not sure. In any case, before I can replace the current > > > > > algorithm, I must > > > > > > > > I am not sure I didn't made a mistake in porting the code from Linux > > > > into Bacula. The general idea has been proven for some time. > > > > > > Unfortunately, the way the GPL works, you cannot port code written by > > > someone else from Linux to Bacula. The licenses are both Open Source GPL > > > but not compatible. > > > > As stated above, it is my code that I have submitted to Linux. > > > > > If you wrote the code in the Linux kernel that you are porting, then you > > > have the choices to license it as I indicate above ... > > > > Yes, I know. > > > > > > > prove to myself (or much faster would be you prove to me) by seeing > > > > > the output that it is the same. I suggest something like the md5.c > > > > > or sha1.c programs where it reads a file and creates the hash -- or > > > > > in your case the CRC. > > > > > > > > > > > There won't be any need to test which one to use, this impl. is > > > > > > much faster than the current one for all archs(unless you want to > > > > > > crc one byte at a time which the Bacula bcrc32 isn't designed for) > > > > > > > > > > I am all for much faster code. The current CRC takes a good deal of > > > > > time, and I recently added the ability to disable it, but I would > > > > > really like something faster so that no one disables it :-) > > > > > > > > Yes, I noticed this 10 years ago when we started to use the JFFS2 file > > > > system in Linux. JFFS2 crc32 all its data and it took time. At the time > > > > the Linux crc32 impl. was like yours. I then optimized it significantly > > > > and sent it in and all was well. Recently I did another optimization > > > > pass and it was twice as fast on x86 than the previous one. > > > > > > If the original Linux version was like mine, then it was derived from > > > "public domain code" and as long as you made the vast majority of the > > > modifications, and you so desire, the code would still be "public > > > domain", so you just have to explicitly put in the code that your changes > > > to the RFC 2083 algoritm are public domain, then there is no license > > > issue. > > > > There are many impl. of crc32 like similar to yours with different > > licences. Here is what Linux says > > * Oct 15, 2000 Matt Domsch <matt_dom...@dell.com> > > * Nicer crc32 functions/docs submitted by li...@horizon.com. Thanks! > > * Code was from the public domain, copyright abandoned. Code was > > * subsequently included in the kernel, thus was re-licensed under the > > * GNU GPL v2. > > Unless they totally re-wrote it, they cannot change the license only the > author can do that unless he explicitly gives you the right to copyright it. > However, that is not my affair. > > So, if you want to move forward, I need to have a submission from you with it > licensed under the LGPL, and if you want it in right away, I need to "see" > that it produces the same results, otherwise it will wait until I get around > to writing a test program for my code and for your new code.
OK, I will give it a go, lets see where it gets us. I am travelling a few days as of tmw so it will be a few days before I come back. Jocke ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
