Thanks much for looking into this so quickly.
This is part of a unit test to verify the thread safety of the bitvector
class in java. The unit test essentially creates a fixed number of
bitvector 64 objects each of size 4096 bits through the function below.
JNIEXPORT jlong JNICALL
Java_com_inmobi_shadowfax_FastbitWrapper_IcreateBitset
(JNIEnv *env, jobject obj, jlong numbits) {
ibis::bitvector64* bitsetptr = new ibis::bitvector64();
bitsetptr->setBit((long)numbits, 0);
return (long)bitsetptr;
}
and 3 random bits are set using the following set method
JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_IsetBit
(JNIEnv *env, jobject obj, jlong bitsetptr, jlong pos) {
((ibis::bitvector64*)bitsetptr)->setBit(pos, 1);
}
on this bit map a series of bit operations are performed from parallel
threads in java
Implementations of and/or/xor are given below
JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_Iand
(JNIEnv *env, jobject obj, jlong targetbitsetptr, jlong sourcebitsetptr) {
((ibis::bitvector64*)targetbitsetptr)->operator&=(*(ibis::bitvector64*)sourcebitsetptr);
}
JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_Ior
(JNIEnv *env, jobject obj, jlong targetbitsetptr, jlong sourcebitsetptr) {
((ibis::bitvector64*)targetbitsetptr)->operator|=(*(ibis::bitvector64*)sourcebitsetptr);
}
JNIEXPORT void JNICALL Java_com_inmobi_shadowfax_FastbitWrapper_Ixor
(JNIEnv *env, jobject obj, jlong targetbitsetptr, jlong sourcebitsetptr) {
((ibis::bitvector64*)targetbitsetptr)->operator^=(*(ibis::bitvector64*)sourcebitsetptr);
}
Regards
Srikanth Sundarrajan
On Tue, May 7, 2013 at 12:13 PM, K. John Wu <[email protected]> wrote:
> Hi, Srikanth,
>
> I can verify that the problem was caused by those two unexpected
> words. This appears to be a flaw in the design of the compression
> scheme that left some special cases unhandled. Since it would
> introduce extra cost to handle this special case, it might be
> worthwhile to figure out how to prevent it being generated in the
> first place. Would you be willing to explain how you generate those
> bitvector64 objects?
>
> Thanks.
>
> John
>
>
>
> On 5/6/13 10:22 PM, Srikanth Sundarrajan wrote:
> > Hi,
> > I am fairly new to fasbit and have been using it for around a
> > month. Have been running into an issue with or_c2 & xor_c2 when run
> > at scale from parallel threads (am using fastbit lib via jni from java).
> >
> > The error looks something like
> >
> > ERROR bitvector64::or_c2 -- serious problem here ...
> > terminate called throwing an exception
> > Error -- bitvector64::or_c2 expects to exhaust i0 but there are -159
> > word(s) left
> >
> > I tried adding print statement to print the copy (deep) of the
> > original lhs & rhs value of the bitmaps and looks like the bitmaps
> > weren't mutated while the function was running.
> >
> > lhs - copy of original
> >
> > This bitvector64 stores 4095 bits of a 4097-bit (25 set) sequence in a
> > 36-word array and 2 bits in the active word
> > 0 0 1 1 2 2 3 3 4 4 5 5 6
> > 012345678901234567890123456789012345678901234567890123456789012
> > ---------------------------------------------------------------
> >
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> >
> 14000000000000000100000000000000000000000000000000000000000000000000000000000000
> > 28000000000000002126*0
> >
> 30000000000000040000000000000000000000000000000000000000000000000000000001000000
> > 4800000000000000163*0
> >
> 50020000000000000000000000100000000000000000000000000000000000000000000000000000
> >
> 60004000000000000000000000000100000000000000000000000000000000000000000000000000
> >
> 70000001000000000000000000000000000000000001000000000000000000000000000000000000
> >
> 80000002000000044000000000000000000000000010000000000000000000000000000001000100
> > 980000000000000000*0
> >
> 100000001000000000000000000000000000000000001000000000000000000000000000000000000
> > 118000000000000004252*0
> >
> 120000400000000000000000000000000010000000000000000000000000000000000000000000000
> > 138000000000000002126*0
> >
> 140000008000000000000000000000000000000001000000000000000000000000000000000000000
> >
> 150040000000000020000000001000000000000000000000000000000000000000000000000100000
> > 1680000000000000000*0
> >
> 170000000000080000000000000000000000000000000000000000000000010000000000000000000
> > 18800000000000000163*0
> >
> 190000000000102000000000000000000000000000000000000000000000100000010000000000000
> >
> 200000000000000000000000000000000000000000000000000000000000000000000000000000000
> >
> 210000000000800000000000000000000000000000000000000000000100000000000000000000000
> > 228000000000000002126*0
> >
> 230000000002000000000000000000000000000000000000000000010000000000000000000000000
> > 248000000000000004252*0
> >
> 250000000100000000000000000000000000000000000000100000000000000000000000000000000
> > 268000000000000002126*0
> >
> 270000000000020000000000000000000000000000000000000000000000000100000000000000000
> > 28800000000000000163*0
> >
> 290000000000001800000000000000000000000000000000000000000000000000001100000000000
> >
> 300000000004000000000000000000000000000000000000000000100000000000000000000000000
> > 31800000000000000163*0
> >
> 320000000080000000000000000000000000000000000000010000000000000000000000000000000
> > 338000000000000006378*0
> >
> 340000010000000000000000000000000000000010000000000000000000000000000000000000000
> > 3580000000000000111071*0
> > 000000000000000000
> >
> > lhs - original
> >
> > This bitvector64 stores 4095 bits of a 4097-bit (25 set) sequence in a
> > 36-word array and 2 bits in the active word
> > 0 0 1 1 2 2 3 3 4 4 5 5 6
> > 012345678901234567890123456789012345678901234567890123456789012
> > ---------------------------------------------------------------
> >
> 00000000000000000000000000000000000000000000000000000000000000000000000000000000
> >
> 14000000000000000100000000000000000000000000000000000000000000000000000000000000
> > 28000000000000002126*0
> >
> 30000000000000040000000000000000000000000000000000000000000000000000000001000000
> > 4800000000000000163*0
> >
> 50020000000000000000000000100000000000000000000000000000000000000000000000000000
> >
> 60004000000000000000000000000100000000000000000000000000000000000000000000000000
> >
> 70000001000000000000000000000000000000000001000000000000000000000000000000000000
> >
> 80000002000000044000000000000000000000000010000000000000000000000000000001000100
> > 980000000000000000*0
> >
> 100000001000000000000000000000000000000000001000000000000000000000000000000000000
> > 118000000000000004252*0
> >
> 120000400000000000000000000000000010000000000000000000000000000000000000000000000
> > 138000000000000002126*0
> >
> 140000008000000000000000000000000000000001000000000000000000000000000000000000000
> >
> 150040000000000020000000001000000000000000000000000000000000000000000000000100000
> > 1680000000000000000*0
> >
> 170000000000080000000000000000000000000000000000000000000000010000000000000000000
> > 18800000000000000163*0
> >
> 190000000000102000000000000000000000000000000000000000000000100000010000000000000
> >
> 200000000000000000000000000000000000000000000000000000000000000000000000000000000
> >
> 210000000000800000000000000000000000000000000000000000000100000000000000000000000
> > 228000000000000002126*0
> >
> 230000000002000000000000000000000000000000000000000000010000000000000000000000000
> > 248000000000000004252*0
> >
> 250000000100000000000000000000000000000000000000100000000000000000000000000000000
> > 268000000000000002126*0
> >
> 270000000000020000000000000000000000000000000000000000000000000100000000000000000
> > 28800000000000000163*0
> >
> 290000000000001800000000000000000000000000000000000000000000000000001100000000000
> >
> 300000000004000000000000000000000000000000000000000000100000000000000000000000000
> > 31800000000000000163*0
> >
> 320000000080000000000000000000000000000000000000010000000000000000000000000000000
> > 338000000000000006378*0
> >
> 340000010000000000000000000000000000000010000000000000000000000000000000000000000
> > 3580000000000000111071*0
> > 000000000000000000
> >
> > rhs - copy of original
> >
> > This bitvector64 stores 4095 bits of a 4097-bit (3 set) sequence in a
> > 7-word array and 2 bits in the active word
> > 0 0 1 1 2 2 3 3 4 4 5 5 6
> > 012345678901234567890123456789012345678901234567890123456789012
> > ---------------------------------------------------------------
> > 0800000000000000e882*0
> >
> 10000000000040000000000000000000000000000000000000000000000001000000000000000000
> > 28000000000000005315*0
> >
> 30000010000000000000000000000000000000010000000000000000000000000000000000000000
> > 4800000000000001a1638*0
> >
> 50000080000000000000000000000000000010000000000000000000000000000000000000000000
> > 680000000000000111071*0
> > 000000000000000000
> >
> > rhs - original
> >
> > This bitvector64 stores 4095 bits of a 4097-bit (3 set) sequence in a
> > 7-word array and 2 bits in the active word
> > 0 0 1 1 2 2 3 3 4 4 5 5 6
> > 012345678901234567890123456789012345678901234567890123456789012
> > ---------------------------------------------------------------
> > 0800000000000000e882*0
> >
> 10000000000040000000000000000000000000000000000000000000000001000000000000000000
> > 28000000000000005315*0
> >
> 30000010000000000000000000000000000000010000000000000000000000000000000000000000
> > 4800000000000001a1638*0
> >
> 50000080000000000000000000000000000010000000000000000000000000000000000000000000
> > 680000000000000111071*0
> > 000000000000000000
> >
> > Any guiding pointers are appreciated.
> >
> > Regards
> > Srikanth Sundarrajan
> >
> >
> >
> > _____________________________________________________________
> > The information contained in this communication is intended solely for
> > the use of the individual or entity to whom it is addressed and others
> > authorized to receive it. It may contain confidential or legally
> > privileged information. If you are not the intended recipient you are
> > hereby notified that any disclosure, copying, distribution or taking
> > any action in reliance on the contents of this information is strictly
> > prohibited and may be unlawful. If you have received this
> > communication in error, please notify us immediately by responding to
> > this email and then delete it from your system. The firm is neither
> > liable for the proper and complete transmission of the information
> > contained in this communication nor for any delay in its receipt.
> >
> >
> > _______________________________________________
> > FastBit-users mailing list
> > [email protected]
> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
> >
>
--
_____________________________________________________________
The information contained in this communication is intended solely for the
use of the individual or entity to whom it is addressed and others
authorized to receive it. It may contain confidential or legally privileged
information. If you are not the intended recipient you are hereby notified
that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be
unlawful. If you have received this communication in error, please notify
us immediately by responding to this email and then delete it from your
system. The firm is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt.
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
