On Tue, Jan 20, 2009 at 05:24:40PM +0100, Morten Jagd Christensen (MJC) wrote: > Hi all > > I have been playing around with comparing a small number of C files > (<100)against each other. With the new fix, bsam-engine works fine > with filter_C. Thanks for a quick reply Neal!
You're welcome. Thanks for finding the bug. :-) > My idea was to build a list of high matching files/functions > os our projects against a large code base of GPL based software. > When a high match occurs I want to quickly see the ogiginal > source codes for files "A" and "B" for comparison. > > That made me wonder: Is there any way to get from the bsam-engine > output a linenumber of the functions in files A and B? From the > symbolic > alignment documentation I suspect that this is what the types 0x131 and > 0x132 > (Byte Offset Start/End) in the bSAM Cache File Format are reserved for? (For other people who want to know where these hex tag numbers come from, look at trunk/fossology/agents/reuseagent/CODES.txt or http://fossology.org/symbolic_alignment_matrix#bsam_cache_file_format) The 0x0131, 0x0132, and 0x0138 tags are for matching the cache file tokens to the pre-tokenized strings. Filter_C needs to be modified to fill out all three of these values. (Right now, it doesn't.) 0x0131: use ftell() to find the offset into the C file of the first token. 0x0132: use ftell() to find the end. The range specified by 0x0131 and 0x0132 are inclusive. 0x0138: this is the length between starts of strings. For example, let's say that there are 5 tokens: "this is a test." (period is a token) The 0x0131 will point to the first "t". The 0x0132 will point to the last period. The 0x0138 lengths will be: 5 3 2 4 1. 5 is "this " -- with space. (Distance between starts.) The final length is actually ignored since there is no next start. If you only do 0x0131 and 0x0132, then you will know where the functions are, but not which tokens matched. With 0x0138, you will know which tokens matched. (This field is the magic behind the highlighting in the "view license" web page.) There is one caveat: all lengths are limited to two bytes. If there is a large comment in source code (e.g., a C program that includes the entire GPL as a header comment), then there will be a problem here. Similarly, if there is a function with more than 32767 tokens (tokens are two bytes each), then there will be a problem. Eventually we should change all lengths to be four bytes rather than two. An acceptable workaround is to split the comment and make sure to add a null token in 0x0108. As an aside: if you're going to do these changes to Filter_C, could you also do them for Filter_Java? :-) (It's the exact same thing.) In contrast, Filter_class (for matching compiled Java classes) and Filter_objdump_exec (for matching functions from compiled binaries) need lots more work than just outputting move tags. Filter_class doesn't create bsam cache files, and Filter_objdump_exec probably doesn't work with 64-bit compiled binaries. > I'd love to hear from others using bsam-engine and filter_C. Me too! This was my pet project but it kept getting (understandably) usurped by other high priority tasks. :-) And since development is being moved to other folks at the end of the month, I'm hoping this doesn't get pushed too far down the wish-list. Wish list: http://fossology.org/task_list It is the item marked "Write Code Re-Use Agent". -Neal _______________________________________________ fossology mailing list [email protected] http://fossology.org/mailman/listinfo/fossology
