Hi, Alexandre, You are right, I misunderstood your original message and only realized my mistake after I have sent out the first message. We have adopted your changes to ibis::colFloats::segment and have made changes to other versions of the function segment.
Hope you have got a chance to try the latest snapshot <https://codeforge.lbl.gov/snapshots.php?group_id=44>. Please let us know if it correctly resolves the problem for you. Again, thanks for the feedback and your interest in FastBit. John PS: Since we have accepted your modified version of ibis::colFloats::segment, your name is now listed in the list of contributors <https://codeforge.lbl.gov/plugins/scmsvn/viewcvs.php/trunk/AUTHORS?root=fastbit&view=markup> along with a more detailed information in ChangeLog <https://codeforge.lbl.gov/plugins/scmsvn/viewcvs.php/trunk/ChangeLog?root=fastbit&view=markup>. On 8/28/2009 2:13 AM, Alexandre Maurel wrote: > Hello John, > > You suspect that the last row is not in the table because of a missing > end-of-line, but it is not the case. > If you look at the last column of the result set (with counts), the last > row has counts equals to 2. > > Moreover the following query : > ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row = 4" > return the correct result. > > Barcode, Row, Col, Cycle, Sample (with counts) > 100220, 4, 4, 38, sample2, 1 > > which is the last row of the table. > > Note that the segment method implementation I sent is working properly > on this data set. > > Alexandre > > 2009/8/27 <[email protected] > <mailto:[email protected]>> > > Send FastBit-users mailing list submissions to > [email protected] > <mailto:[email protected]> > > To subscribe or unsubscribe via the World Wide Web, visit > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > or, via email, send a message with subject or body 'help' to > [email protected] > <mailto:[email protected]> > > You can reach the person managing the list at > [email protected] > <mailto:[email protected]> > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of FastBit-users digest..." > > > Today's Topics: > > 1. problem in segment method (Alexandre Maurel) > 2. problem in segment method (Alexandre Maurel) > 3. Re: problem in segment method (K. John Wu) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 27 Aug 2009 17:55:57 +0200 > From: Alexandre Maurel <[email protected] > <mailto:[email protected]>> > Subject: [FastBit-users] problem in segment method > To: [email protected] <mailto:[email protected]> > Message-ID: > <[email protected] > <mailto:[email protected]>> > Content-Type: text/plain; charset="iso-8859-1" > > Hello, > > I used ardea to generate a table using the following : > > Barcode Row Col Cycle Sample > 100220 1 1 30 sample1 > 100220 1 1 36 sample1 > 100220 1 2 30 sample1 > 100220 1 2 36 sample1 > 100220 2 1 30 sample1 > 100220 2 1 36 sample1 > 100220 2 2 30 sample1 > 100220 2 2 36 sample1 > 100220 3 3 38 sample2 > 100220 4 4 38 sample2 > > ardea -d tmp -m > "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" -t > Array.txt > > then using ibis, > > ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row > 0" > > result gives .... > > Barcode, Row, Col, Cycle, Sample (with counts) > 100220, 1, 1, 30, sample1, 1 > 100220, 1, 1, 36, sample1, 1 > 100220, 1, 2, 30, sample1, 1 > 100220, 1, 2, 36, sample1, 1 > 100220, 2, 1, 30, sample1, 1 > 100220, 2, 1, 36, sample1, 1 > 100220, 2, 2, 30, sample1, 1 > 100220, 2, 2, 36, sample1, 1 > 100220, 3, 3, 38, sample2, 2 > > which is wrong the last row is missing .... > > Here how I solved this bug > > array_t<uint32_t>* > ibis::colFloats::segment(const array_t<uint32_t>* old) const { > array_t<uint32_t>* res = new array_t<uint32_t>; > // res->push_back(0); // the first number is always 0 > uint32_t j = 0; > float target = *(array->begin()); > const uint32_t nelm = array->size(); > > if (old != 0 && old->size()>2) { > // find segments with in the previously defined segments > for (uint32_t i=0; i<old->size()-1; ++i) { > if (j < nelm) { > res->push_back(j); > target = (*array)[j]; > ++ j; > if (j < nelm) { > do { > while (j < (*old)[i+1] && (*array)[j] == > target) ++ > j; > if (j < (*old)[i+1]) { > res->push_back(j); > if (j < nelm) { > target = (*array)[j]; > ++ j; > } > } > } while (j < (*old)[i+1]); > } > }; > } > } > else { // start with all elements in one segment > res->push_back(0); // the first number is always 0 > uint32_t j = 1; > while (j < nelm) { > while (j < nelm && (*array)[j] == target) > ++ j; > res->push_back(j); > if (j < nelm) { > target = (*array)[j]; > ++ j; > } > } > } > if (res->back() < nelm) > res->push_back(nelm); > return res; > } // ibis::colFloats::segment > > > Hope this help. > > Alexandre Maurel > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > http://hpcrdm.lbl.gov/pipermail/fastbit-users/attachments/20090827/962c3caa/attachment-0001.htm > > ------------------------------ > > Message: 2 > Date: Thu, 27 Aug 2009 17:59:12 +0200 > From: Alexandre Maurel <[email protected] > <mailto:[email protected]>> > Subject: [FastBit-users] problem in segment method > To: [email protected] <mailto:[email protected]> > Message-ID: > <[email protected] > <mailto:[email protected]>> > Content-Type: text/plain; charset="iso-8859-1" > > Hello, > > I used ardea to generate a table using the following : > > Barcode Row Col Cycle Sample > 100220 1 1 30 sample1 > 100220 1 1 36 sample1 > 100220 1 2 30 sample1 > 100220 1 2 36 sample1 > 100220 2 1 30 sample1 > 100220 2 1 36 sample1 > 100220 2 2 30 sample1 > 100220 2 2 36 sample1 > 100220 3 3 38 sample2 > 100220 4 4 38 sample2 > > ardea -d tmp -m > "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" -t > Array.txt > > then using ibis, > > ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row > 0" > > result gives .... > > Barcode, Row, Col, Cycle, Sample (with counts) > 100220, 1, 1, 30, sample1, 1 > 100220, 1, 1, 36, sample1, 1 > 100220, 1, 2, 30, sample1, 1 > 100220, 1, 2, 36, sample1, 1 > 100220, 2, 1, 30, sample1, 1 > 100220, 2, 1, 36, sample1, 1 > 100220, 2, 2, 30, sample1, 1 > 100220, 2, 2, 36, sample1, 1 > 100220, 3, 3, 38, sample2, 2 > > which is wrong the last row is missing .... > > Here how I solved this bug > > array_t<uint32_t>* > ibis::colFloats::segment(const array_t<uint32_t>* old) const { > array_t<uint32_t>* res = new array_t<uint32_t>; > // res->push_back(0); // the first number is always 0 > uint32_t j = 0; > float target = *(array->begin()); > const uint32_t nelm = array->size(); > > if (old != 0 && old->size()>2) { > // find segments with in the previously defined segments > for (uint32_t i=0; i<old->size()-1; ++i) { > if (j < nelm) { > res->push_back(j); > target = (*array)[j]; > ++ j; > if (j < nelm) { > do { > while (j < (*old)[i+1] && (*array)[j] == > target) ++ > j; > if (j < (*old)[i+1]) { > res->push_back(j); > if (j < nelm) { > target = (*array)[j]; > ++ j; > } > } > } while (j < (*old)[i+1]); > } > }; > } > } > else { // start with all elements in one segment > res->push_back(0); // the first number is always 0 > uint32_t j = 1; > while (j < nelm) { > while (j < nelm && (*array)[j] == target) > ++ j; > res->push_back(j); > if (j < nelm) { > target = (*array)[j]; > ++ j; > } > } > } > if (res->back() < nelm) > res->push_back(nelm); > return res; > } // ibis::colFloats::segment > > > Hope this help. > > Alexandre Maurel > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > http://hpcrdm.lbl.gov/pipermail/fastbit-users/attachments/20090827/0928f095/attachment-0001.htm > > ------------------------------ > > Message: 3 > Date: Thu, 27 Aug 2009 11:57:05 -0700 > From: "K. John Wu" <[email protected] <mailto:[email protected]>> > Subject: Re: [FastBit-users] problem in segment method > To: FastBit Users <[email protected] > <mailto:[email protected]>> > Message-ID: <[email protected] <mailto:[email protected]>> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi, Alexandre, > > Thanks for bring this issue to our attention. My first impression was > that your input data might be missing a end-of-line or some such > thing, but guess you suspect that ibis::colFloats::segment is in > trouble. Would you mind check whether your input CSV file is indeed > missing the end-of-line for the last line? In the mean time, I will > look into ibis::colFloats::segment.. > > John > > > On 8/27/2009 8:59 AM, Alexandre Maurel wrote: > > Hello, > > > > I used ardea to generate a table using the following : > > > > Barcode Row Col Cycle Sample > > 100220 1 1 30 sample1 > > 100220 1 1 36 sample1 > > 100220 1 2 30 sample1 > > 100220 1 2 36 sample1 > > 100220 2 1 30 sample1 > > 100220 2 1 36 sample1 > > 100220 2 2 30 sample1 > > 100220 2 2 36 sample1 > > 100220 3 3 38 sample2 > > 100220 4 4 38 sample2 > > > > ardea -d tmp -m > > "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" > -t Array.txt > > > > then using ibis, > > > > ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row > 0" > > > > result gives .... > > > > Barcode, Row, Col, Cycle, Sample (with counts) > > 100220, 1, 1, 30, sample1, 1 > > 100220, 1, 1, 36, sample1, 1 > > 100220, 1, 2, 30, sample1, 1 > > 100220, 1, 2, 36, sample1, 1 > > 100220, 2, 1, 30, sample1, 1 > > 100220, 2, 1, 36, sample1, 1 > > 100220, 2, 2, 30, sample1, 1 > > 100220, 2, 2, 36, sample1, 1 > > 100220, 3, 3, 38, sample2, 2 > > > > which is wrong the last row is missing .... > > > > Here how I solved this bug > > > > array_t<uint32_t>* > > ibis::colFloats::segment(const array_t<uint32_t>* old) const { > > array_t<uint32_t>* res = new array_t<uint32_t>; > > // res->push_back(0); // the first number is always 0 > > uint32_t j = 0; > > float target = *(array->begin()); > > const uint32_t nelm = array->size(); > > > > if (old != 0 && old->size()>2) { > > // find segments with in the previously defined segments > > for (uint32_t i=0; i<old->size()-1; ++i) { > > if (j < nelm) { > > res->push_back(j); > > target = (*array)[j]; > > ++ j; > > if (j < nelm) { > > do { > > while (j < (*old)[i+1] && (*array)[j] == > target) > > ++ j; > > if (j < (*old)[i+1]) { > > res->push_back(j); > > if (j < nelm) { > > target = (*array)[j]; > > ++ j; > > } > > } > > } while (j < (*old)[i+1]); > > } > > }; > > } > > } > > else { // start with all elements in one segment > > res->push_back(0); // the first number is always 0 > > uint32_t j = 1; > > while (j < nelm) { > > while (j < nelm && (*array)[j] == target) > > ++ j; > > res->push_back(j); > > if (j < nelm) { > > target = (*array)[j]; > > ++ j; > > } > > } > > } > > if (res->back() < nelm) > > res->push_back(nelm); > > return res; > > } // ibis::colFloats::segment > > > > > > Hope this help. > > > > Alexandre Maurel > > > > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > FastBit-users mailing list > > [email protected] <mailto:[email protected]> > > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > > ------------------------------ > > _______________________________________________ > FastBit-users mailing list > [email protected] <mailto:[email protected]> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > > End of FastBit-users Digest, Vol 24, Issue 6 > ******************************************** > > > > ------------------------------------------------------------------------ > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
