Thanks John, the patch seemed to have helped. May be I am asking too much, but do you mind help me understand what the issue was and how the fix helped? Also the patch mentioned that the setBit is going to be expensive and is to be avoided. What do you recommend we use instead to specifically set selective bits ?
Thanks much Srikanth Sundarrajan On Tue, May 7, 2013 at 1:13 PM, K. John Wu <[email protected]> wrote: > Hi, Srikanth, > > Just checked in some temporary fixes to the bitwise logical operations > as SVN Revision 628. You can download it with > > svn checkout https://codeforge.lbl.gov/anonscm/fastbit > > > I'd still like to have your test program if you are able to share it > with us. > > Thanks. > > John > > > > On 5/6/13 11:48 PM, Srikanth Sundarrajan wrote: > > Thanks much for looking into this so quickly. > > > > This is part of a unit test to verify the thread safety of the > > bitvector class in java. The unit test essentially creates a fixed > > number of bitvector 64 objects each of size 4096 bits through the > > function below. > > > > JNIEXPORT jlong JNICALL > > Java_com_inmobi_shadowfax_FastbitWrapper_IcreateBitset > > (JNIEnv *env, jobject obj, jlong numbits) { > > > > ibis::bitvector64* bitsetptr = new ibis::bitvector64(); > > bitsetptr->setBit((long)numbits, 0); > > > > return (long)bitsetptr; > > } > > > > and 3 random bits are set using the following set method > > > > JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_IsetBit > > (JNIEnv *env, jobject obj, jlong bitsetptr, jlong pos) { > > > > ((ibis::bitvector64*)bitsetptr)->setBit(pos, 1); > > > > } > > > > on this bit map a series of bit operations are performed from parallel > > threads in java > > > > Implementations of and/or/xor are given below > > > > JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_Iand > > (JNIEnv *env, jobject obj, jlong targetbitsetptr, jlong sourcebitsetptr) > { > > > ((ibis::bitvector64*)targetbitsetptr)->operator&=(*(ibis::bitvector64*)sourcebitsetptr); > > } > > > > JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_Ior > > (JNIEnv *env, jobject obj, jlong targetbitsetptr, jlong sourcebitsetptr) > { > > > ((ibis::bitvector64*)targetbitsetptr)->operator|=(*(ibis::bitvector64*)sourcebitsetptr); > > } > > > > > > JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_Ixor > > (JNIEnv *env, jobject obj, jlong targetbitsetptr, jlong sourcebitsetptr) > { > > > ((ibis::bitvector64*)targetbitsetptr)->operator^=(*(ibis::bitvector64*)sourcebitsetptr); > > } > > > > > > Regards > > Srikanth Sundarrajan > > > > > > On Tue, May 7, 2013 at 12:13 PM, K. John Wu <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi, Srikanth, > > > > I can verify that the problem was caused by those two unexpected > > words. This appears to be a flaw in the design of the compression > > scheme that left some special cases unhandled. Since it would > > introduce extra cost to handle this special case, it might be > > worthwhile to figure out how to prevent it being generated in the > > first place. Would you be willing to explain how you generate those > > bitvector64 objects? > > > > Thanks. > > > > John > > > > > > > > On 5/6/13 10:22 PM, Srikanth Sundarrajan wrote: > > > Hi, > > > I am fairly new to fasbit and have been using it for around a > > > month. Have been running into an issue with or_c2 & xor_c2 when > run > > > at scale from parallel threads (am using fastbit lib via jni > > from java). > > > > > > The error looks something like > > > > > > ERROR bitvector64::or_c2 -- serious problem here ... > > > terminate called throwing an exception > > > Error -- bitvector64::or_c2 expects to exhaust i0 but there are > -159 > > > word(s) left > > > > > > I tried adding print statement to print the copy (deep) of the > > > original lhs & rhs value of the bitmaps and looks like the bitmaps > > > weren't mutated while the function was running. > > > > > > lhs - copy of original > > > > > > This bitvector64 stores 4095 bits of a 4097-bit (25 set) > > sequence in a > > > 36-word array and 2 bits in the active word > > > 0 0 1 1 2 2 3 3 4 4 5 5 6 > > > 012345678901234567890123456789012345678901234567890123456789012 > > > --------------------------------------------------------------- > > > > > > 00000000000000000000000000000000000000000000000000000000000000000000000000000000 > > > > > > 14000000000000000100000000000000000000000000000000000000000000000000000000000000 > > > 28000000000000002126*0 > > > > > > 30000000000000040000000000000000000000000000000000000000000000000000000001000000 > > > 4800000000000000163*0 > > > > > > 50020000000000000000000000100000000000000000000000000000000000000000000000000000 > > > > > > 60004000000000000000000000000100000000000000000000000000000000000000000000000000 > > > > > > 70000001000000000000000000000000000000000001000000000000000000000000000000000000 > > > > > > 80000002000000044000000000000000000000000010000000000000000000000000000001000100 > > > 980000000000000000*0 > > > > > > 100000001000000000000000000000000000000000001000000000000000000000000000000000000 > > > 118000000000000004252*0 > > > > > > 120000400000000000000000000000000010000000000000000000000000000000000000000000000 > > > 138000000000000002126*0 > > > > > > 140000008000000000000000000000000000000001000000000000000000000000000000000000000 > > > > > > 150040000000000020000000001000000000000000000000000000000000000000000000000100000 > > > 1680000000000000000*0 > > > > > > 170000000000080000000000000000000000000000000000000000000000010000000000000000000 > > > 18800000000000000163*0 > > > > > > 190000000000102000000000000000000000000000000000000000000000100000010000000000000 > > > > > > 200000000000000000000000000000000000000000000000000000000000000000000000000000000 > > > > > > 210000000000800000000000000000000000000000000000000000000100000000000000000000000 > > > 228000000000000002126*0 > > > > > > 230000000002000000000000000000000000000000000000000000010000000000000000000000000 > > > 248000000000000004252*0 > > > > > > 250000000100000000000000000000000000000000000000100000000000000000000000000000000 > > > 268000000000000002126*0 > > > > > > 270000000000020000000000000000000000000000000000000000000000000100000000000000000 > > > 28800000000000000163*0 > > > > > > 290000000000001800000000000000000000000000000000000000000000000000001100000000000 > > > > > > 300000000004000000000000000000000000000000000000000000100000000000000000000000000 > > > 31800000000000000163*0 > > > > > > 320000000080000000000000000000000000000000000000010000000000000000000000000000000 > > > 338000000000000006378*0 > > > > > > 340000010000000000000000000000000000000010000000000000000000000000000000000000000 > > > 3580000000000000111071*0 > > > 000000000000000000 > > > > > > lhs - original > > > > > > This bitvector64 stores 4095 bits of a 4097-bit (25 set) > > sequence in a > > > 36-word array and 2 bits in the active word > > > 0 0 1 1 2 2 3 3 4 4 5 5 6 > > > 012345678901234567890123456789012345678901234567890123456789012 > > > --------------------------------------------------------------- > > > > > > 00000000000000000000000000000000000000000000000000000000000000000000000000000000 > > > > > > 14000000000000000100000000000000000000000000000000000000000000000000000000000000 > > > 28000000000000002126*0 > > > > > > 30000000000000040000000000000000000000000000000000000000000000000000000001000000 > > > 4800000000000000163*0 > > > > > > 50020000000000000000000000100000000000000000000000000000000000000000000000000000 > > > > > > 60004000000000000000000000000100000000000000000000000000000000000000000000000000 > > > > > > 70000001000000000000000000000000000000000001000000000000000000000000000000000000 > > > > > > 80000002000000044000000000000000000000000010000000000000000000000000000001000100 > > > 980000000000000000*0 > > > > > > 100000001000000000000000000000000000000000001000000000000000000000000000000000000 > > > 118000000000000004252*0 > > > > > > 120000400000000000000000000000000010000000000000000000000000000000000000000000000 > > > 138000000000000002126*0 > > > > > > 140000008000000000000000000000000000000001000000000000000000000000000000000000000 > > > > > > 150040000000000020000000001000000000000000000000000000000000000000000000000100000 > > > 1680000000000000000*0 > > > > > > 170000000000080000000000000000000000000000000000000000000000010000000000000000000 > > > 18800000000000000163*0 > > > > > > 190000000000102000000000000000000000000000000000000000000000100000010000000000000 > > > > > > 200000000000000000000000000000000000000000000000000000000000000000000000000000000 > > > > > > 210000000000800000000000000000000000000000000000000000000100000000000000000000000 > > > 228000000000000002126*0 > > > > > > 230000000002000000000000000000000000000000000000000000010000000000000000000000000 > > > 248000000000000004252*0 > > > > > > 250000000100000000000000000000000000000000000000100000000000000000000000000000000 > > > 268000000000000002126*0 > > > > > > 270000000000020000000000000000000000000000000000000000000000000100000000000000000 > > > 28800000000000000163*0 > > > > > > 290000000000001800000000000000000000000000000000000000000000000000001100000000000 > > > > > > 300000000004000000000000000000000000000000000000000000100000000000000000000000000 > > > 31800000000000000163*0 > > > > > > 320000000080000000000000000000000000000000000000010000000000000000000000000000000 > > > 338000000000000006378*0 > > > > > > 340000010000000000000000000000000000000010000000000000000000000000000000000000000 > > > 3580000000000000111071*0 > > > 000000000000000000 > > > > > > rhs - copy of original > > > > > > This bitvector64 stores 4095 bits of a 4097-bit (3 set) sequence > > in a > > > 7-word array and 2 bits in the active word > > > 0 0 1 1 2 2 3 3 4 4 5 5 6 > > > 012345678901234567890123456789012345678901234567890123456789012 > > > --------------------------------------------------------------- > > > 0800000000000000e882*0 > > > > > > 10000000000040000000000000000000000000000000000000000000000001000000000000000000 > > > 28000000000000005315*0 > > > > > > 30000010000000000000000000000000000000010000000000000000000000000000000000000000 > > > 4800000000000001a1638*0 > > > > > > 50000080000000000000000000000000000010000000000000000000000000000000000000000000 > > > 680000000000000111071*0 > > > 000000000000000000 > > > > > > rhs - original > > > > > > This bitvector64 stores 4095 bits of a 4097-bit (3 set) sequence > > in a > > > 7-word array and 2 bits in the active word > > > 0 0 1 1 2 2 3 3 4 4 5 5 6 > > > 012345678901234567890123456789012345678901234567890123456789012 > > > --------------------------------------------------------------- > > > 0800000000000000e882*0 > > > > > > 10000000000040000000000000000000000000000000000000000000000001000000000000000000 > > > 28000000000000005315*0 > > > > > > 30000010000000000000000000000000000000010000000000000000000000000000000000000000 > > > 4800000000000001a1638*0 > > > > > > 50000080000000000000000000000000000010000000000000000000000000000000000000000000 > > > 680000000000000111071*0 > > > 000000000000000000 > > > > > > Any guiding pointers are appreciated. > > > > > > Regards > > > Srikanth Sundarrajan > > > > > > > > > > > > _____________________________________________________________ > > > The information contained in this communication is intended > > solely for > > > the use of the individual or entity to whom it is addressed and > > others > > > authorized to receive it. It may contain confidential or legally > > > privileged information. If you are not the intended recipient > > you are > > > hereby notified that any disclosure, copying, distribution or > taking > > > any action in reliance on the contents of this information is > > strictly > > > prohibited and may be unlawful. If you have received this > > > communication in error, please notify us immediately by > > responding to > > > this email and then delete it from your system. The firm is neither > > > liable for the proper and complete transmission of the information > > > contained in this communication nor for any delay in its receipt. > > > > > > > > > _______________________________________________ > > > FastBit-users mailing list > > > [email protected] <mailto:[email protected]> > > > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > > > > > > > > > > _____________________________________________________________ > > The information contained in this communication is intended solely for > > the use of the individual or entity to whom it is addressed and others > > authorized to receive it. It may contain confidential or legally > > privileged information. If you are not the intended recipient you are > > hereby notified that any disclosure, copying, distribution or taking > > any action in reliance on the contents of this information is strictly > > prohibited and may be unlawful. If you have received this > > communication in error, please notify us immediately by responding to > > this email and then delete it from your system. The firm is neither > > liable for the proper and complete transmission of the information > > contained in this communication nor for any delay in its receipt. > -- _____________________________________________________________ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
