Hi John, I've been to fast in my testing of version 1.1.3, in fact it is not working !
But in the mean time I moved to version 1.1.4 and the problem is solved. Thanks, Alexandre [email protected] a écrit : > Send FastBit-users mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of FastBit-users digest..." > > > Today's Topics: > > 1. Re: problem in segment method (K. John Wu) > 2. Re: problem in segment method (Alexandre Maurel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 10 Nov 2009 16:38:07 -0800 > From: "K. John Wu" <[email protected]> > Subject: Re: [FastBit-users] problem in segment method > To: FastBit Users <[email protected]> > Message-ID: <[email protected]> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi, Alexandre, > > My apologies. I have messed up the modification you've given me earlier. > > We are very appreciative of your test data and test case. The problem > should be fixed now -- there was an extra ++j statement in my version > of the code. > > The change has been rolled into the new release ibis1.1.4. You may > find it at > <https://codeforge.lbl.gov/frs/download.php/124/fastbit-ibis1.1.4.tar.gz>. > A description of the changes is available at > <https://codeforge.lbl.gov/frs/shownotes.php?release_id=123>. > > Please give this new code a try and let us know if you find any > issues. Thanks. > > John > > > > On 11/10/2009 7:41 AM, Alexandre Maurel wrote: > >> Hi John, >> >> I update my code with fastbit snapshot fastbit-scm-2009-09-01. >> I did some test and it is not working for me. >> >> To make it work, I had to go back to the implementation of the segment >> method I first sent. >> >> As attached document you will find the dataset I used for the test. >> >> The dataset contains 152 rows, when I run a query (select <all columns>) >> that should return the complete dataset, 2 rows are missing. >> >> Here are the segment methods implementation I used to make it work. >> >> array_t<uint32_t>* >> ibis::colFloats::segment(const array_t<uint32_t>* old) const { >> array_t<uint32_t>* res = new array_t<uint32_t>; >> // res->push_back(0); // the first number is always 0 >> uint32_t j = 0; >> float target = *(array->begin()); >> const uint32_t nelm = array->size(); >> >> if (old != 0 && old->size()>2) { >> // find segments with in the previously defined segments >> for (uint32_t i=0; i<old->size()-1; ++i) { >> if (j < nelm) { >> res->push_back(j); >> target = (*array)[j]; >> ++ j; >> if (j < nelm) { >> do { >> while (j < (*old)[i+1] && (*array)[j] == >> target) ++ >> j; >> if (j < (*old)[i+1]) { >> res->push_back(j); >> if (j < nelm) { >> target = (*array)[j]; >> ++ j; >> } >> } >> } while (j < (*old)[i+1]); >> } >> }; >> } >> } >> else { // start with all elements in one segment >> res->push_back(0); // the first number is always 0 >> uint32_t j = 1; >> while (j < nelm) { >> while (j < nelm && (*array)[j] == target) >> ++ j; >> res->push_back(j); >> if (j < nelm) { >> target = (*array)[j]; >> ++ j; >> } >> } >> } >> if (res->back() < nelm) >> res->push_back(nelm); >> return res; >> } // ibis::colFloats::segment >> >> and >> >> ibis::array_t<uint32_t>* >> ibis::colUInts::segment(const array_t<uint32_t>* old) const { >> array_t<uint32_t>* res = new array_t<uint32_t>; >> // res->push_back(0); // the first number is always 0 >> uint32_t j = 0; >> uint32_t target = *(array->begin()); >> const uint32_t nelm = array->size(); >> >> if (old != 0 && old->size()>2) { >> // find segments with in the previously defined segments >> for (uint32_t i=0; i<old->size()-1; ++i) { >> if (j < nelm) { >> res->push_back(j); >> target = (*array)[j]; >> ++ j; >> if (j < nelm) { >> do { >> while (j < (*old)[i+1] && (*array)[j] == target) ++ >> j; >> if (j < (*old)[i+1]) { >> res->push_back(j); >> if (j < nelm) { >> target = (*array)[j]; >> ++ j; >> } >> } >> } while (j < (*old)[i+1]); >> } >> }; >> } >> } >> else { // start with all elements in one segment >> res->push_back(0); // the first number is always 0 >> uint32_t j = 1; >> while (j < nelm) { >> while (j < nelm && (*array)[j] == target) >> ++ j; >> res->push_back(j); >> if (j < nelm) { >> target = (*array)[j]; >> ++ j; >> } >> } >> } >> if (res->back() < nelm) >> res->push_back(nelm); >> return res; >> } // ibis::colUInts::segment >> >> Alexandre >> >> [email protected] a ?crit : >> >>> Send FastBit-users mailing list submissions to >>> [email protected] >>> >>> To subscribe or unsubscribe via the World Wide Web, visit >>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>> or, via email, send a message with subject or body 'help' to >>> [email protected] >>> >>> You can reach the person managing the list at >>> [email protected] >>> >>> When replying, please edit your Subject line so it is more specific >>> than "Re: Contents of FastBit-users digest..." >>> >>> >>> Today's Topics: >>> >>> 1. Re: problem in segment method (K. John Wu) >>> >>> >>> ---------------------------------------------------------------------- >>> >>> Message: 1 >>> Date: Fri, 28 Aug 2009 06:56:19 -0700 >>> From: "K. John Wu" <[email protected]> >>> Subject: Re: [FastBit-users] problem in segment method >>> To: FastBit Users <[email protected]> >>> Message-ID: <[email protected]> >>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>> >>> Hi, Alexandre, >>> >>> You are right, I misunderstood your original message and only realized >>> my mistake after I have sent out the first message. We have adopted >>> your changes to ibis::colFloats::segment and have made changes to >>> other versions of the function segment. >>> >>> Hope you have got a chance to try the latest snapshot >>> <https://codeforge.lbl.gov/snapshots.php?group_id=44>. Please let us >>> know if it correctly resolves the problem for you. >>> >>> Again, thanks for the feedback and your interest in FastBit. >>> >>> John >>> >>> PS: Since we have accepted your modified version of >>> ibis::colFloats::segment, your name is now listed in the list of >>> contributors >>> <https://codeforge.lbl.gov/plugins/scmsvn/viewcvs.php/trunk/AUTHORS?root=fastbit&view=markup> >>> >>> along with a more detailed information in ChangeLog >>> <https://codeforge.lbl.gov/plugins/scmsvn/viewcvs.php/trunk/ChangeLog?root=fastbit&view=markup>. >>> >>> >>> >>> On 8/28/2009 2:13 AM, Alexandre Maurel wrote: >>> >>> >>>> Hello John, >>>> >>>> You suspect that the last row is not in the table because of a >>>> missing end-of-line, but it is not the case. >>>> If you look at the last column of the result set (with counts), the >>>> last row has counts equals to 2. >>>> >>>> Moreover the following query : >>>> ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row = 4" >>>> return the correct result. >>>> >>>> Barcode, Row, Col, Cycle, Sample (with counts) >>>> 100220, 4, 4, 38, sample2, 1 >>>> >>>> which is the last row of the table. >>>> >>>> Note that the segment method implementation I sent is working >>>> properly on this data set. >>>> >>>> Alexandre >>>> >>>> 2009/8/27 <[email protected] >>>> <mailto:[email protected]>> >>>> >>>> Send FastBit-users mailing list submissions to >>>> [email protected] >>>> <mailto:[email protected]> >>>> >>>> To subscribe or unsubscribe via the World Wide Web, visit >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> or, via email, send a message with subject or body 'help' to >>>> [email protected] >>>> <mailto:[email protected]> >>>> >>>> You can reach the person managing the list at >>>> [email protected] >>>> <mailto:[email protected]> >>>> >>>> When replying, please edit your Subject line so it is more specific >>>> than "Re: Contents of FastBit-users digest..." >>>> >>>> >>>> Today's Topics: >>>> >>>> 1. problem in segment method (Alexandre Maurel) >>>> 2. problem in segment method (Alexandre Maurel) >>>> 3. Re: problem in segment method (K. John Wu) >>>> >>>> >>>> >>>> ---------------------------------------------------------------------- >>>> >>>> Message: 1 >>>> Date: Thu, 27 Aug 2009 17:55:57 +0200 >>>> From: Alexandre Maurel <[email protected] >>>> <mailto:[email protected]>> >>>> Subject: [FastBit-users] problem in segment method >>>> To: [email protected] >>>> <mailto:[email protected]> >>>> Message-ID: >>>> <[email protected] >>>> >>>> <mailto:[email protected]>> >>>> Content-Type: text/plain; charset="iso-8859-1" >>>> >>>> Hello, >>>> >>>> I used ardea to generate a table using the following : >>>> >>>> Barcode Row Col Cycle Sample >>>> 100220 1 1 30 sample1 >>>> 100220 1 1 36 sample1 >>>> 100220 1 2 30 sample1 >>>> 100220 1 2 36 sample1 >>>> 100220 2 1 30 sample1 >>>> 100220 2 1 36 sample1 >>>> 100220 2 2 30 sample1 >>>> 100220 2 2 36 sample1 >>>> 100220 3 3 38 sample2 >>>> 100220 4 4 38 sample2 >>>> >>>> ardea -d tmp -m >>>> "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" -t >>>> Array.txt >>>> >>>> then using ibis, >>>> >>>> ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row > 0" >>>> >>>> result gives .... >>>> >>>> Barcode, Row, Col, Cycle, Sample (with counts) >>>> 100220, 1, 1, 30, sample1, 1 >>>> 100220, 1, 1, 36, sample1, 1 >>>> 100220, 1, 2, 30, sample1, 1 >>>> 100220, 1, 2, 36, sample1, 1 >>>> 100220, 2, 1, 30, sample1, 1 >>>> 100220, 2, 1, 36, sample1, 1 >>>> 100220, 2, 2, 30, sample1, 1 >>>> 100220, 2, 2, 36, sample1, 1 >>>> 100220, 3, 3, 38, sample2, 2 >>>> >>>> which is wrong the last row is missing .... >>>> >>>> Here how I solved this bug >>>> >>>> array_t<uint32_t>* >>>> ibis::colFloats::segment(const array_t<uint32_t>* old) const { >>>> array_t<uint32_t>* res = new array_t<uint32_t>; >>>> // res->push_back(0); // the first number is always 0 >>>> uint32_t j = 0; >>>> float target = *(array->begin()); >>>> const uint32_t nelm = array->size(); >>>> >>>> if (old != 0 && old->size()>2) { >>>> // find segments with in the previously defined segments >>>> for (uint32_t i=0; i<old->size()-1; ++i) { >>>> if (j < nelm) { >>>> res->push_back(j); >>>> target = (*array)[j]; >>>> ++ j; >>>> if (j < nelm) { >>>> do { >>>> while (j < (*old)[i+1] && (*array)[j] == >>>> target) ++ >>>> j; >>>> if (j < (*old)[i+1]) { >>>> res->push_back(j); >>>> if (j < nelm) { >>>> target = (*array)[j]; >>>> ++ j; >>>> } >>>> } >>>> } while (j < (*old)[i+1]); >>>> } >>>> }; >>>> } >>>> } >>>> else { // start with all elements in one segment >>>> res->push_back(0); // the first number is always 0 >>>> uint32_t j = 1; >>>> while (j < nelm) { >>>> while (j < nelm && (*array)[j] == target) >>>> ++ j; >>>> res->push_back(j); >>>> if (j < nelm) { >>>> target = (*array)[j]; >>>> ++ j; >>>> } >>>> } >>>> } >>>> if (res->back() < nelm) >>>> res->push_back(nelm); >>>> return res; >>>> } // ibis::colFloats::segment >>>> >>>> >>>> Hope this help. >>>> >>>> Alexandre Maurel >>>> -------------- next part -------------- >>>> An HTML attachment was scrubbed... >>>> URL: >>>> >>>> http://hpcrdm.lbl.gov/pipermail/fastbit-users/attachments/20090827/962c3caa/attachment-0001.htm >>>> >>>> >>>> >>>> ------------------------------ >>>> >>>> Message: 2 >>>> Date: Thu, 27 Aug 2009 17:59:12 +0200 >>>> From: Alexandre Maurel <[email protected] >>>> <mailto:[email protected]>> >>>> Subject: [FastBit-users] problem in segment method >>>> To: [email protected] >>>> <mailto:[email protected]> >>>> Message-ID: >>>> <[email protected] >>>> >>>> <mailto:[email protected]>> >>>> Content-Type: text/plain; charset="iso-8859-1" >>>> >>>> Hello, >>>> >>>> I used ardea to generate a table using the following : >>>> >>>> Barcode Row Col Cycle Sample >>>> 100220 1 1 30 sample1 >>>> 100220 1 1 36 sample1 >>>> 100220 1 2 30 sample1 >>>> 100220 1 2 36 sample1 >>>> 100220 2 1 30 sample1 >>>> 100220 2 1 36 sample1 >>>> 100220 2 2 30 sample1 >>>> 100220 2 2 36 sample1 >>>> 100220 3 3 38 sample2 >>>> 100220 4 4 38 sample2 >>>> >>>> ardea -d tmp -m >>>> "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" -t >>>> Array.txt >>>> >>>> then using ibis, >>>> >>>> ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row > 0" >>>> >>>> result gives .... >>>> >>>> Barcode, Row, Col, Cycle, Sample (with counts) >>>> 100220, 1, 1, 30, sample1, 1 >>>> 100220, 1, 1, 36, sample1, 1 >>>> 100220, 1, 2, 30, sample1, 1 >>>> 100220, 1, 2, 36, sample1, 1 >>>> 100220, 2, 1, 30, sample1, 1 >>>> 100220, 2, 1, 36, sample1, 1 >>>> 100220, 2, 2, 30, sample1, 1 >>>> 100220, 2, 2, 36, sample1, 1 >>>> 100220, 3, 3, 38, sample2, 2 >>>> >>>> which is wrong the last row is missing .... >>>> >>>> Here how I solved this bug >>>> >>>> array_t<uint32_t>* >>>> ibis::colFloats::segment(const array_t<uint32_t>* old) const { >>>> array_t<uint32_t>* res = new array_t<uint32_t>; >>>> // res->push_back(0); // the first number is always 0 >>>> uint32_t j = 0; >>>> float target = *(array->begin()); >>>> const uint32_t nelm = array->size(); >>>> >>>> if (old != 0 && old->size()>2) { >>>> // find segments with in the previously defined segments >>>> for (uint32_t i=0; i<old->size()-1; ++i) { >>>> if (j < nelm) { >>>> res->push_back(j); >>>> target = (*array)[j]; >>>> ++ j; >>>> if (j < nelm) { >>>> do { >>>> while (j < (*old)[i+1] && (*array)[j] == >>>> target) ++ >>>> j; >>>> if (j < (*old)[i+1]) { >>>> res->push_back(j); >>>> if (j < nelm) { >>>> target = (*array)[j]; >>>> ++ j; >>>> } >>>> } >>>> } while (j < (*old)[i+1]); >>>> } >>>> }; >>>> } >>>> } >>>> else { // start with all elements in one segment >>>> res->push_back(0); // the first number is always 0 >>>> uint32_t j = 1; >>>> while (j < nelm) { >>>> while (j < nelm && (*array)[j] == target) >>>> ++ j; >>>> res->push_back(j); >>>> if (j < nelm) { >>>> target = (*array)[j]; >>>> ++ j; >>>> } >>>> } >>>> } >>>> if (res->back() < nelm) >>>> res->push_back(nelm); >>>> return res; >>>> } // ibis::colFloats::segment >>>> >>>> >>>> Hope this help. >>>> >>>> Alexandre Maurel >>>> -------------- next part -------------- >>>> An HTML attachment was scrubbed... >>>> URL: >>>> >>>> http://hpcrdm.lbl.gov/pipermail/fastbit-users/attachments/20090827/0928f095/attachment-0001.htm >>>> >>>> >>>> >>>> ------------------------------ >>>> >>>> Message: 3 >>>> Date: Thu, 27 Aug 2009 11:57:05 -0700 >>>> From: "K. John Wu" <[email protected] <mailto:[email protected]>> >>>> Subject: Re: [FastBit-users] problem in segment method >>>> To: FastBit Users <[email protected] >>>> <mailto:[email protected]>> >>>> Message-ID: <[email protected] >>>> <mailto:[email protected]>> >>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>>> >>>> Hi, Alexandre, >>>> >>>> Thanks for bring this issue to our attention. My first >>>> impression was >>>> that your input data might be missing a end-of-line or some such >>>> thing, but guess you suspect that ibis::colFloats::segment is in >>>> trouble. Would you mind check whether your input CSV file is indeed >>>> missing the end-of-line for the last line? In the mean time, I will >>>> look into ibis::colFloats::segment.. >>>> >>>> John >>>> >>>> >>>> On 8/27/2009 8:59 AM, Alexandre Maurel wrote: >>>> > Hello, >>>> > >>>> > I used ardea to generate a table using the following : >>>> > >>>> > Barcode Row Col Cycle Sample >>>> > 100220 1 1 30 sample1 >>>> > 100220 1 1 36 sample1 >>>> > 100220 1 2 30 sample1 >>>> > 100220 1 2 36 sample1 >>>> > 100220 2 1 30 sample1 >>>> > 100220 2 1 36 sample1 >>>> > 100220 2 2 30 sample1 >>>> > 100220 2 2 36 sample1 >>>> > 100220 3 3 38 sample2 >>>> > 100220 4 4 38 sample2 >>>> > >>>> > ardea -d tmp -m >>>> > "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" >>>> -t Array.txt >>>> > >>>> > then using ibis, >>>> > >>>> > ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row >>>> >>>>> 0" >>>>> >>>> > >>>> > result gives .... >>>> > >>>> > Barcode, Row, Col, Cycle, Sample (with counts) >>>> > 100220, 1, 1, 30, sample1, 1 >>>> > 100220, 1, 1, 36, sample1, 1 >>>> > 100220, 1, 2, 30, sample1, 1 >>>> > 100220, 1, 2, 36, sample1, 1 >>>> > 100220, 2, 1, 30, sample1, 1 >>>> > 100220, 2, 1, 36, sample1, 1 >>>> > 100220, 2, 2, 30, sample1, 1 >>>> > 100220, 2, 2, 36, sample1, 1 >>>> > 100220, 3, 3, 38, sample2, 2 >>>> > >>>> > which is wrong the last row is missing .... >>>> > >>>> > Here how I solved this bug >>>> > >>>> > array_t<uint32_t>* >>>> > ibis::colFloats::segment(const array_t<uint32_t>* old) const { >>>> > array_t<uint32_t>* res = new array_t<uint32_t>; >>>> > // res->push_back(0); // the first number is always 0 >>>> > uint32_t j = 0; >>>> > float target = *(array->begin()); >>>> > const uint32_t nelm = array->size(); >>>> > >>>> > if (old != 0 && old->size()>2) { >>>> > // find segments with in the previously defined segments >>>> > for (uint32_t i=0; i<old->size()-1; ++i) { >>>> > if (j < nelm) { >>>> > res->push_back(j); >>>> > target = (*array)[j]; >>>> > ++ j; >>>> > if (j < nelm) { >>>> > do { >>>> > while (j < (*old)[i+1] && (*array)[j] == >>>> target) >>>> > ++ j; >>>> > if (j < (*old)[i+1]) { >>>> > res->push_back(j); >>>> > if (j < nelm) { >>>> > target = (*array)[j]; >>>> > ++ j; >>>> > } >>>> > } >>>> > } while (j < (*old)[i+1]); >>>> > } >>>> > }; >>>> > } >>>> > } >>>> > else { // start with all elements in one segment >>>> > res->push_back(0); // the first number is always 0 >>>> > uint32_t j = 1; >>>> > while (j < nelm) { >>>> > while (j < nelm && (*array)[j] == target) >>>> > ++ j; >>>> > res->push_back(j); >>>> > if (j < nelm) { >>>> > target = (*array)[j]; >>>> > ++ j; >>>> > } >>>> > } >>>> > } >>>> > if (res->back() < nelm) >>>> > res->push_back(nelm); >>>> > return res; >>>> > } // ibis::colFloats::segment >>>> > >>>> > >>>> > Hope this help. >>>> > >>>> > Alexandre Maurel >>>> > >>>> > >>>> > >>>> > >>>> >>>> ------------------------------------------------------------------------ >>>> > >>>> > _______________________________________________ >>>> > FastBit-users mailing list >>>> > [email protected] >>>> <mailto:[email protected]> >>>> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> >>>> >>>> ------------------------------ >>>> >>>> _______________________________________________ >>>> FastBit-users mailing list >>>> [email protected] <mailto:[email protected]> >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> >>>> >>>> End of FastBit-users Digest, Vol 24, Issue 6 >>>> ******************************************** >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> FastBit-users mailing list >>>> [email protected] >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> >>>> >>> ------------------------------ >>> >>> _______________________________________________ >>> FastBit-users mailing list >>> [email protected] >>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>> >>> >>> End of FastBit-users Digest, Vol 24, Issue 8 >>> ******************************************** >>> >>> >>> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> > > > ------------------------------ > > Message: 2 > Date: Wed, 11 Nov 2009 09:32:45 +0100 > From: Alexandre Maurel <[email protected]> > Subject: Re: [FastBit-users] problem in segment method > To: [email protected] > Message-ID: <[email protected]> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hello John, > > I tried version 1.1.3 it's working. > But not now I am confused, as far as I can see the segment methods are > the same in both version ...! > > Alexandre > > [email protected] a ?crit : > >> Send FastBit-users mailing list submissions to >> [email protected] >> >> To subscribe or unsubscribe via the World Wide Web, visit >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> or, via email, send a message with subject or body 'help' to >> [email protected] >> >> You can reach the person managing the list at >> [email protected] >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of FastBit-users digest..." >> >> >> Today's Topics: >> >> 1. Re: problem in segment method (K. John Wu) >> >> >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Tue, 10 Nov 2009 08:46:38 -0800 >> From: "K. John Wu" <[email protected]> >> Subject: Re: [FastBit-users] problem in segment method >> To: FastBit Users <[email protected]> >> Message-ID: <[email protected]> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> Hi, Alexandre, >> >> Would it be possible for you to try version 1.1.3, which was released >> on Sept 10 (and newer than the snapshot you used)? >> >> In the mean time, I will take a more careful look at differences >> between your code and what I have now... We are in the middle of >> trying to get the next release ready. The current SVN repository >> should stabilize in the next few days.. >> >> John >> >> >> On 11/10/2009 7:41 AM, Alexandre Maurel wrote: >> >> >>> Hi John, >>> >>> I update my code with fastbit snapshot fastbit-scm-2009-09-01. >>> I did some test and it is not working for me. >>> >>> To make it work, I had to go back to the implementation of the segment >>> method I first sent. >>> >>> As attached document you will find the dataset I used for the test. >>> >>> The dataset contains 152 rows, when I run a query (select <all columns>) >>> that should return the complete dataset, 2 rows are missing. >>> >>> Here are the segment methods implementation I used to make it work. >>> >>> array_t<uint32_t>* >>> ibis::colFloats::segment(const array_t<uint32_t>* old) const { >>> array_t<uint32_t>* res = new array_t<uint32_t>; >>> // res->push_back(0); // the first number is always 0 >>> uint32_t j = 0; >>> float target = *(array->begin()); >>> const uint32_t nelm = array->size(); >>> >>> if (old != 0 && old->size()>2) { >>> // find segments with in the previously defined segments >>> for (uint32_t i=0; i<old->size()-1; ++i) { >>> if (j < nelm) { >>> res->push_back(j); >>> target = (*array)[j]; >>> ++ j; >>> if (j < nelm) { >>> do { >>> while (j < (*old)[i+1] && (*array)[j] == >>> target) ++ >>> j; >>> if (j < (*old)[i+1]) { >>> res->push_back(j); >>> if (j < nelm) { >>> target = (*array)[j]; >>> ++ j; >>> } >>> } >>> } while (j < (*old)[i+1]); >>> } >>> }; >>> } >>> } >>> else { // start with all elements in one segment >>> res->push_back(0); // the first number is always 0 >>> uint32_t j = 1; >>> while (j < nelm) { >>> while (j < nelm && (*array)[j] == target) >>> ++ j; >>> res->push_back(j); >>> if (j < nelm) { >>> target = (*array)[j]; >>> ++ j; >>> } >>> } >>> } >>> if (res->back() < nelm) >>> res->push_back(nelm); >>> return res; >>> } // ibis::colFloats::segment >>> >>> and >>> >>> ibis::array_t<uint32_t>* >>> ibis::colUInts::segment(const array_t<uint32_t>* old) const { >>> array_t<uint32_t>* res = new array_t<uint32_t>; >>> // res->push_back(0); // the first number is always 0 >>> uint32_t j = 0; >>> uint32_t target = *(array->begin()); >>> const uint32_t nelm = array->size(); >>> >>> if (old != 0 && old->size()>2) { >>> // find segments with in the previously defined segments >>> for (uint32_t i=0; i<old->size()-1; ++i) { >>> if (j < nelm) { >>> res->push_back(j); >>> target = (*array)[j]; >>> ++ j; >>> if (j < nelm) { >>> do { >>> while (j < (*old)[i+1] && (*array)[j] == target) ++ >>> j; >>> if (j < (*old)[i+1]) { >>> res->push_back(j); >>> if (j < nelm) { >>> target = (*array)[j]; >>> ++ j; >>> } >>> } >>> } while (j < (*old)[i+1]); >>> } >>> }; >>> } >>> } >>> else { // start with all elements in one segment >>> res->push_back(0); // the first number is always 0 >>> uint32_t j = 1; >>> while (j < nelm) { >>> while (j < nelm && (*array)[j] == target) >>> ++ j; >>> res->push_back(j); >>> if (j < nelm) { >>> target = (*array)[j]; >>> ++ j; >>> } >>> } >>> } >>> if (res->back() < nelm) >>> res->push_back(nelm); >>> return res; >>> } // ibis::colUInts::segment >>> >>> Alexandre >>> >>> [email protected] a ?crit : >>> >>> >>>> Send FastBit-users mailing list submissions to >>>> [email protected] >>>> >>>> To subscribe or unsubscribe via the World Wide Web, visit >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> or, via email, send a message with subject or body 'help' to >>>> [email protected] >>>> >>>> You can reach the person managing the list at >>>> [email protected] >>>> >>>> When replying, please edit your Subject line so it is more specific >>>> than "Re: Contents of FastBit-users digest..." >>>> >>>> >>>> Today's Topics: >>>> >>>> 1. Re: problem in segment method (K. John Wu) >>>> >>>> >>>> ---------------------------------------------------------------------- >>>> >>>> Message: 1 >>>> Date: Fri, 28 Aug 2009 06:56:19 -0700 >>>> From: "K. John Wu" <[email protected]> >>>> Subject: Re: [FastBit-users] problem in segment method >>>> To: FastBit Users <[email protected]> >>>> Message-ID: <[email protected]> >>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>>> >>>> Hi, Alexandre, >>>> >>>> You are right, I misunderstood your original message and only realized >>>> my mistake after I have sent out the first message. We have adopted >>>> your changes to ibis::colFloats::segment and have made changes to >>>> other versions of the function segment. >>>> >>>> Hope you have got a chance to try the latest snapshot >>>> <https://codeforge.lbl.gov/snapshots.php?group_id=44>. Please let us >>>> know if it correctly resolves the problem for you. >>>> >>>> Again, thanks for the feedback and your interest in FastBit. >>>> >>>> John >>>> >>>> PS: Since we have accepted your modified version of >>>> ibis::colFloats::segment, your name is now listed in the list of >>>> contributors >>>> <https://codeforge.lbl.gov/plugins/scmsvn/viewcvs.php/trunk/AUTHORS?root=fastbit&view=markup> >>>> >>>> along with a more detailed information in ChangeLog >>>> <https://codeforge.lbl.gov/plugins/scmsvn/viewcvs.php/trunk/ChangeLog?root=fastbit&view=markup>. >>>> >>>> >>>> >>>> On 8/28/2009 2:13 AM, Alexandre Maurel wrote: >>>> >>>> >>>> >>>>> Hello John, >>>>> >>>>> You suspect that the last row is not in the table because of a >>>>> missing end-of-line, but it is not the case. >>>>> If you look at the last column of the result set (with counts), the >>>>> last row has counts equals to 2. >>>>> >>>>> Moreover the following query : >>>>> ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row = 4" >>>>> return the correct result. >>>>> >>>>> Barcode, Row, Col, Cycle, Sample (with counts) >>>>> 100220, 4, 4, 38, sample2, 1 >>>>> >>>>> which is the last row of the table. >>>>> >>>>> Note that the segment method implementation I sent is working >>>>> properly on this data set. >>>>> >>>>> Alexandre >>>>> >>>>> 2009/8/27 <[email protected] >>>>> <mailto:[email protected]>> >>>>> >>>>> Send FastBit-users mailing list submissions to >>>>> [email protected] >>>>> <mailto:[email protected]> >>>>> >>>>> To subscribe or unsubscribe via the World Wide Web, visit >>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>>> or, via email, send a message with subject or body 'help' to >>>>> [email protected] >>>>> <mailto:[email protected]> >>>>> >>>>> You can reach the person managing the list at >>>>> [email protected] >>>>> <mailto:[email protected]> >>>>> >>>>> When replying, please edit your Subject line so it is more specific >>>>> than "Re: Contents of FastBit-users digest..." >>>>> >>>>> >>>>> Today's Topics: >>>>> >>>>> 1. problem in segment method (Alexandre Maurel) >>>>> 2. problem in segment method (Alexandre Maurel) >>>>> 3. Re: problem in segment method (K. John Wu) >>>>> >>>>> >>>>> >>>>> ---------------------------------------------------------------------- >>>>> >>>>> Message: 1 >>>>> Date: Thu, 27 Aug 2009 17:55:57 +0200 >>>>> From: Alexandre Maurel <[email protected] >>>>> <mailto:[email protected]>> >>>>> Subject: [FastBit-users] problem in segment method >>>>> To: [email protected] >>>>> <mailto:[email protected]> >>>>> Message-ID: >>>>> <[email protected] >>>>> >>>>> <mailto:[email protected]>> >>>>> Content-Type: text/plain; charset="iso-8859-1" >>>>> >>>>> Hello, >>>>> >>>>> I used ardea to generate a table using the following : >>>>> >>>>> Barcode Row Col Cycle Sample >>>>> 100220 1 1 30 sample1 >>>>> 100220 1 1 36 sample1 >>>>> 100220 1 2 30 sample1 >>>>> 100220 1 2 36 sample1 >>>>> 100220 2 1 30 sample1 >>>>> 100220 2 1 36 sample1 >>>>> 100220 2 2 30 sample1 >>>>> 100220 2 2 36 sample1 >>>>> 100220 3 3 38 sample2 >>>>> 100220 4 4 38 sample2 >>>>> >>>>> ardea -d tmp -m >>>>> "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" -t >>>>> Array.txt >>>>> >>>>> then using ibis, >>>>> >>>>> ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row > 0" >>>>> >>>>> result gives .... >>>>> >>>>> Barcode, Row, Col, Cycle, Sample (with counts) >>>>> 100220, 1, 1, 30, sample1, 1 >>>>> 100220, 1, 1, 36, sample1, 1 >>>>> 100220, 1, 2, 30, sample1, 1 >>>>> 100220, 1, 2, 36, sample1, 1 >>>>> 100220, 2, 1, 30, sample1, 1 >>>>> 100220, 2, 1, 36, sample1, 1 >>>>> 100220, 2, 2, 30, sample1, 1 >>>>> 100220, 2, 2, 36, sample1, 1 >>>>> 100220, 3, 3, 38, sample2, 2 >>>>> >>>>> which is wrong the last row is missing .... >>>>> >>>>> Here how I solved this bug >>>>> >>>>> array_t<uint32_t>* >>>>> ibis::colFloats::segment(const array_t<uint32_t>* old) const { >>>>> array_t<uint32_t>* res = new array_t<uint32_t>; >>>>> // res->push_back(0); // the first number is always 0 >>>>> uint32_t j = 0; >>>>> float target = *(array->begin()); >>>>> const uint32_t nelm = array->size(); >>>>> >>>>> if (old != 0 && old->size()>2) { >>>>> // find segments with in the previously defined segments >>>>> for (uint32_t i=0; i<old->size()-1; ++i) { >>>>> if (j < nelm) { >>>>> res->push_back(j); >>>>> target = (*array)[j]; >>>>> ++ j; >>>>> if (j < nelm) { >>>>> do { >>>>> while (j < (*old)[i+1] && (*array)[j] == >>>>> target) ++ >>>>> j; >>>>> if (j < (*old)[i+1]) { >>>>> res->push_back(j); >>>>> if (j < nelm) { >>>>> target = (*array)[j]; >>>>> ++ j; >>>>> } >>>>> } >>>>> } while (j < (*old)[i+1]); >>>>> } >>>>> }; >>>>> } >>>>> } >>>>> else { // start with all elements in one segment >>>>> res->push_back(0); // the first number is always 0 >>>>> uint32_t j = 1; >>>>> while (j < nelm) { >>>>> while (j < nelm && (*array)[j] == target) >>>>> ++ j; >>>>> res->push_back(j); >>>>> if (j < nelm) { >>>>> target = (*array)[j]; >>>>> ++ j; >>>>> } >>>>> } >>>>> } >>>>> if (res->back() < nelm) >>>>> res->push_back(nelm); >>>>> return res; >>>>> } // ibis::colFloats::segment >>>>> >>>>> >>>>> Hope this help. >>>>> >>>>> Alexandre Maurel >>>>> -------------- next part -------------- >>>>> An HTML attachment was scrubbed... >>>>> URL: >>>>> >>>>> http://hpcrdm.lbl.gov/pipermail/fastbit-users/attachments/20090827/962c3caa/attachment-0001.htm >>>>> >>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> Message: 2 >>>>> Date: Thu, 27 Aug 2009 17:59:12 +0200 >>>>> From: Alexandre Maurel <[email protected] >>>>> <mailto:[email protected]>> >>>>> Subject: [FastBit-users] problem in segment method >>>>> To: [email protected] >>>>> <mailto:[email protected]> >>>>> Message-ID: >>>>> <[email protected] >>>>> >>>>> <mailto:[email protected]>> >>>>> Content-Type: text/plain; charset="iso-8859-1" >>>>> >>>>> Hello, >>>>> >>>>> I used ardea to generate a table using the following : >>>>> >>>>> Barcode Row Col Cycle Sample >>>>> 100220 1 1 30 sample1 >>>>> 100220 1 1 36 sample1 >>>>> 100220 1 2 30 sample1 >>>>> 100220 1 2 36 sample1 >>>>> 100220 2 1 30 sample1 >>>>> 100220 2 1 36 sample1 >>>>> 100220 2 2 30 sample1 >>>>> 100220 2 2 36 sample1 >>>>> 100220 3 3 38 sample2 >>>>> 100220 4 4 38 sample2 >>>>> >>>>> ardea -d tmp -m >>>>> "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" -t >>>>> Array.txt >>>>> >>>>> then using ibis, >>>>> >>>>> ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row > 0" >>>>> >>>>> result gives .... >>>>> >>>>> Barcode, Row, Col, Cycle, Sample (with counts) >>>>> 100220, 1, 1, 30, sample1, 1 >>>>> 100220, 1, 1, 36, sample1, 1 >>>>> 100220, 1, 2, 30, sample1, 1 >>>>> 100220, 1, 2, 36, sample1, 1 >>>>> 100220, 2, 1, 30, sample1, 1 >>>>> 100220, 2, 1, 36, sample1, 1 >>>>> 100220, 2, 2, 30, sample1, 1 >>>>> 100220, 2, 2, 36, sample1, 1 >>>>> 100220, 3, 3, 38, sample2, 2 >>>>> >>>>> which is wrong the last row is missing .... >>>>> >>>>> Here how I solved this bug >>>>> >>>>> array_t<uint32_t>* >>>>> ibis::colFloats::segment(const array_t<uint32_t>* old) const { >>>>> array_t<uint32_t>* res = new array_t<uint32_t>; >>>>> // res->push_back(0); // the first number is always 0 >>>>> uint32_t j = 0; >>>>> float target = *(array->begin()); >>>>> const uint32_t nelm = array->size(); >>>>> >>>>> if (old != 0 && old->size()>2) { >>>>> // find segments with in the previously defined segments >>>>> for (uint32_t i=0; i<old->size()-1; ++i) { >>>>> if (j < nelm) { >>>>> res->push_back(j); >>>>> target = (*array)[j]; >>>>> ++ j; >>>>> if (j < nelm) { >>>>> do { >>>>> while (j < (*old)[i+1] && (*array)[j] == >>>>> target) ++ >>>>> j; >>>>> if (j < (*old)[i+1]) { >>>>> res->push_back(j); >>>>> if (j < nelm) { >>>>> target = (*array)[j]; >>>>> ++ j; >>>>> } >>>>> } >>>>> } while (j < (*old)[i+1]); >>>>> } >>>>> }; >>>>> } >>>>> } >>>>> else { // start with all elements in one segment >>>>> res->push_back(0); // the first number is always 0 >>>>> uint32_t j = 1; >>>>> while (j < nelm) { >>>>> while (j < nelm && (*array)[j] == target) >>>>> ++ j; >>>>> res->push_back(j); >>>>> if (j < nelm) { >>>>> target = (*array)[j]; >>>>> ++ j; >>>>> } >>>>> } >>>>> } >>>>> if (res->back() < nelm) >>>>> res->push_back(nelm); >>>>> return res; >>>>> } // ibis::colFloats::segment >>>>> >>>>> >>>>> Hope this help. >>>>> >>>>> Alexandre Maurel >>>>> -------------- next part -------------- >>>>> An HTML attachment was scrubbed... >>>>> URL: >>>>> >>>>> http://hpcrdm.lbl.gov/pipermail/fastbit-users/attachments/20090827/0928f095/attachment-0001.htm >>>>> >>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> Message: 3 >>>>> Date: Thu, 27 Aug 2009 11:57:05 -0700 >>>>> From: "K. John Wu" <[email protected] <mailto:[email protected]>> >>>>> Subject: Re: [FastBit-users] problem in segment method >>>>> To: FastBit Users <[email protected] >>>>> <mailto:[email protected]>> >>>>> Message-ID: <[email protected] >>>>> <mailto:[email protected]>> >>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>>>> >>>>> Hi, Alexandre, >>>>> >>>>> Thanks for bring this issue to our attention. My first >>>>> impression was >>>>> that your input data might be missing a end-of-line or some such >>>>> thing, but guess you suspect that ibis::colFloats::segment is in >>>>> trouble. Would you mind check whether your input CSV file is indeed >>>>> missing the end-of-line for the last line? In the mean time, I will >>>>> look into ibis::colFloats::segment.. >>>>> >>>>> John >>>>> >>>>> >>>>> On 8/27/2009 8:59 AM, Alexandre Maurel wrote: >>>>> > Hello, >>>>> > >>>>> > I used ardea to generate a table using the following : >>>>> > >>>>> > Barcode Row Col Cycle Sample >>>>> > 100220 1 1 30 sample1 >>>>> > 100220 1 1 36 sample1 >>>>> > 100220 1 2 30 sample1 >>>>> > 100220 1 2 36 sample1 >>>>> > 100220 2 1 30 sample1 >>>>> > 100220 2 1 36 sample1 >>>>> > 100220 2 2 30 sample1 >>>>> > 100220 2 2 36 sample1 >>>>> > 100220 3 3 38 sample2 >>>>> > 100220 4 4 38 sample2 >>>>> > >>>>> > ardea -d tmp -m >>>>> > "Barcode:float,Row:float,Col:float,Cycle:float,Sample:category" >>>>> -t Array.txt >>>>> > >>>>> > then using ibis, >>>>> > >>>>> > ibis -d tmp -q "select Barcode,Row,Col,Cycle,Sample where Row >>>>> >>>>> >>>>>> 0" >>>>>> >>>>>> >>>>> > >>>>> > result gives .... >>>>> > >>>>> > Barcode, Row, Col, Cycle, Sample (with counts) >>>>> > 100220, 1, 1, 30, sample1, 1 >>>>> > 100220, 1, 1, 36, sample1, 1 >>>>> > 100220, 1, 2, 30, sample1, 1 >>>>> > 100220, 1, 2, 36, sample1, 1 >>>>> > 100220, 2, 1, 30, sample1, 1 >>>>> > 100220, 2, 1, 36, sample1, 1 >>>>> > 100220, 2, 2, 30, sample1, 1 >>>>> > 100220, 2, 2, 36, sample1, 1 >>>>> > 100220, 3, 3, 38, sample2, 2 >>>>> > >>>>> > which is wrong the last row is missing .... >>>>> > >>>>> > Here how I solved this bug >>>>> > >>>>> > array_t<uint32_t>* >>>>> > ibis::colFloats::segment(const array_t<uint32_t>* old) const { >>>>> > array_t<uint32_t>* res = new array_t<uint32_t>; >>>>> > // res->push_back(0); // the first number is always 0 >>>>> > uint32_t j = 0; >>>>> > float target = *(array->begin()); >>>>> > const uint32_t nelm = array->size(); >>>>> > >>>>> > if (old != 0 && old->size()>2) { >>>>> > // find segments with in the previously defined segments >>>>> > for (uint32_t i=0; i<old->size()-1; ++i) { >>>>> > if (j < nelm) { >>>>> > res->push_back(j); >>>>> > target = (*array)[j]; >>>>> > ++ j; >>>>> > if (j < nelm) { >>>>> > do { >>>>> > while (j < (*old)[i+1] && (*array)[j] == >>>>> target) >>>>> > ++ j; >>>>> > if (j < (*old)[i+1]) { >>>>> > res->push_back(j); >>>>> > if (j < nelm) { >>>>> > target = (*array)[j]; >>>>> > ++ j; >>>>> > } >>>>> > } >>>>> > } while (j < (*old)[i+1]); >>>>> > } >>>>> > }; >>>>> > } >>>>> > } >>>>> > else { // start with all elements in one segment >>>>> > res->push_back(0); // the first number is always 0 >>>>> > uint32_t j = 1; >>>>> > while (j < nelm) { >>>>> > while (j < nelm && (*array)[j] == target) >>>>> > ++ j; >>>>> > res->push_back(j); >>>>> > if (j < nelm) { >>>>> > target = (*array)[j]; >>>>> > ++ j; >>>>> > } >>>>> > } >>>>> > } >>>>> > if (res->back() < nelm) >>>>> > res->push_back(nelm); >>>>> > return res; >>>>> > } // ibis::colFloats::segment >>>>> > >>>>> > >>>>> > Hope this help. >>>>> > >>>>> > Alexandre Maurel >>>>> > >>>>> > >>>>> > >>>>> > >>>>> >>>>> ------------------------------------------------------------------------ >>>>> > >>>>> > _______________________________________________ >>>>> > FastBit-users mailing list >>>>> > [email protected] >>>>> <mailto:[email protected]> >>>>> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> _______________________________________________ >>>>> FastBit-users mailing list >>>>> [email protected] <mailto:[email protected]> >>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>>> >>>>> >>>>> End of FastBit-users Digest, Vol 24, Issue 6 >>>>> ******************************************** >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> _______________________________________________ >>>>> FastBit-users mailing list >>>>> [email protected] >>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>>> >>>>> >>>>> >>>> ------------------------------ >>>> >>>> _______________________________________________ >>>> FastBit-users mailing list >>>> [email protected] >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> >>>> >>>> End of FastBit-users Digest, Vol 24, Issue 8 >>>> ******************************************** >>>> >>>> >>>> >>>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> FastBit-users mailing list >>> [email protected] >>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>> >>> >> ------------------------------ >> >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> >> >> End of FastBit-users Digest, Vol 27, Issue 9 >> ******************************************** >> >> >> > > > ------------------------------ > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > > End of FastBit-users Digest, Vol 27, Issue 10 > ********************************************* > > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
