Hi, Alexandre,
An update has been checked into to address the problem you've
observed. The problem was caused by an incorrect return value in the
code that does the evaluation of the operator '!'. Please give the
new code from the nightly snapshot a try when you get a chance. (To
verify that the modification is included, you can take a look at the
file ChangeLog and make sure the entry for 2009-12-11 is present)
Please let us know if you find any additional issues.
Thanks.
John
On 12/11/2009 1:38 AM, Alexandre Maurel wrote:
> Hi John,
>
> I have adapted the query to fit with test case dataset Array.txt.
>
> %IBIS% -no-estimation -l log2.txt -d .\testCase\dimensions\ColPart -q
> "select Col,Array where (((((( ! Array == "D09") AND ( ! Array ==
> "H07")) AND ( ! Array == "B04")) AND ( ! (Array == "B04" AND Visual_QC
> == "OK"))) AND Visual_QC == "OK") AND 2 < Col) "
>
> %IBIS% -l log2.txt -d .\testCase\dimensions\ColPart -q "select Col,Array
> where (((((( ! Array == "D09") AND ( ! Array == "H07")) AND ( ! Array ==
> "B04")) AND ( ! (Array == "B04" AND Visual_QC == "OK"))) AND Visual_QC
> == "OK") AND 2 < Col) "
>
> I first run the query with the parameter -no-estimation, I get the
> following bundle
>
> Bundle PKY4o3gX2ds----0 contains 93 distinct 2-tuples
> Col, Array (with counts)
> 1, A01, 12
> 1, B01, 12
> 1, C01, 12
> 1, D01, 12
> 1, E01, 12
> 1, F01, 12
> 1, G01, 12
> 1, H01, 12
> 2, A02, 12
> 2, B02, 12
> 2, C02, 12
> 2, D02, 12
> 2, E02, 12
> 2, F02, 12
> 2, G02, 12
> 2, H02, 12
> 3, A03, 12
> 3, B03, 12
> 3, C03, 12
> 3, D03, 12
> 3, E03, 12
> 3, F03, 12
> 3, G03, 12
> 3, H03, 12
> 4, A04, 12
> 4, C04, 12
> 4, D04, 12
> 4, E04, 12
> 4, F04, 12
> 4, G04, 12
> 4, H04, 12
> 5, A05, 12
> 5, B05, 12
> 5, C05, 12
> 5, D05, 12
> 5, E05, 12
> 5, F05, 12
> 5, G05, 12
> 5, H05, 12
> 6, A06, 12
> 6, B06, 12
> 6, C06, 12
> 6, D06, 12
> 6, E06, 12
> 6, F06, 12
> 6, G06, 12
> 6, H06, 12
> 7, A07, 12
> 7, B07, 12
> 7, C07, 12
> 7, D07, 12
> 7, E07, 12
> 7, F07, 12
> 7, G07, 12
> 8, A08, 12
> 8, B08, 12
> 8, C08, 12
> 8, D08, 12
> 8, E08, 12
> 8, F08, 12
> 8, G08, 12
> 8, H08, 12
> 9, A09, 12
> 9, B09, 12
> 9, C09, 12
> 9, E09, 12
> 9, F09, 12
> 9, G09, 12
> 9, H09, 12
> 10, A10, 12
> 10, B10, 12
> 10, C10, 12
> 10, D10, 12
> 10, E10, 12
> 10, F10, 12
> 10, G10, 12
> 10, H10, 12
> 11, A11, 12
> 11, B11, 12
> 11, C11, 12
> 11, D11, 12
> 11, E11, 12
> 11, F11, 12
> 11, G11, 12
> 11, H11, 12
> 12, A12, 12
> 12, B12, 12
> 12, C12, 12
> 12, D12, 12
> 12, E12, 12
> 12, F12, 12
> 12, G12, 12
> 12, H12, 12
>
> then same query but with estimation
>
> Bundle PKY4o3gX2ds----0 contains 77 distinct 2-tuples
> Col, Array (with counts)
> 3, A03, 12
> 3, B03, 12
> 3, C03, 12
> 3, D03, 12
> 3, E03, 12
> 3, F03, 12
> 3, G03, 12
> 3, H03, 12
> 4, A04, 12
> 4, C04, 12
> 4, D04, 12
> 4, E04, 12
> 4, F04, 12
> 4, G04, 12
> 4, H04, 12
> 5, A05, 12
> 5, B05, 12
> 5, C05, 12
> 5, D05, 12
> 5, E05, 12
> 5, F05, 12
> 5, G05, 12
> 5, H05, 12
> 6, A06, 12
> 6, B06, 12
> 6, C06, 12
> 6, D06, 12
> 6, E06, 12
> 6, F06, 12
> 6, G06, 12
> 6, H06, 12
> 7, A07, 12
> 7, B07, 12
> 7, C07, 12
> 7, D07, 12
> 7, E07, 12
> 7, F07, 12
> 7, G07, 12
> 8, A08, 12
> 8, B08, 12
> 8, C08, 12
> 8, D08, 12
> 8, E08, 12
> 8, F08, 12
> 8, G08, 12
> 8, H08, 12
> 9, A09, 12
> 9, B09, 12
> 9, C09, 12
> 9, E09, 12
> 9, F09, 12
> 9, G09, 12
> 9, H09, 12
> 10, A10, 12
> 10, B10, 12
> 10, C10, 12
> 10, D10, 12
> 10, E10, 12
> 10, F10, 12
> 10, G10, 12
> 10, H10, 12
> 11, A11, 12
> 11, B11, 12
> 11, C11, 12
> 11, D11, 12
> 11, E11, 12
> 11, F11, 12
> 11, G11, 12
> 11, H11, 12
> 12, A12, 12
> 12, B12, 12
> 12, C12, 12
> 12, D12, 12
> 12, E12, 12
> 12, F12, 12
> 12, G12, 12
> 12, H12, 12
>
> As you can see the first run do not take the condition "2 < Col" into
> account.
>
> Is this a bug or calling estimate method is required before calling
> evaluate method ?
>
>
> Alexandre
>
>
> K. John Wu a écrit :
>> Hi, Alexandre,
>>
>> I suspect that a couple of column names are not what they appear in
>> the data file. I would guess that VisualsQC probably should be
>> Visual_QC, not sure what Comments1 should be.
>>
>> If the names are correct, it might be helpful to show me the log file
>> produced by your FastBit program. If you do want to send me a log
>> file, please be sure to set verboseness level to some positive number.
>> If you use ibis, you can achieve that by specifying '-v', '-v 2' or
>> '-v 3'. In your own program, you can either call ibis::init with
>> positive number or directly assign a value to ibis::gVerbose.
>>
>> John
>>
>>
>> On 12/10/2009 9:06 AM, Alexandre Maurel wrote:
>>> Hi John,
>>>
>>> You are right, the nightly snapshot I downloaded have latest
>>> ChangeLog in date of 2009-12-07 !!
>>> I will check the ChangeLog in the future.
>>>
>>> By the way, I have delivered the first version of my application with
>>> Fastbit , and one of the user have found a bug.
>>> So I'm now fighting with the following query
>>>
>>> (((((( ! Array == "D09") AND ( ! Array == "H07")) AND ( ! Array ==
>>> "B04")) AND ( ! (Array == "B04" AND VisualsQC == "OK"))) AND
>>> VisualsQC == "OK") AND 0 < Comments1)
>>>
>>> You would ask, why one need to evaluate such query, where ( ! Array
>>> == "B04") and ! (Array == "B04" AND VisualsQC == "OK") are clearly
>>> redundant, but in the application users are allowed to create query
>>> "visualy", by dragging and dropping columns and the query is auto
>>> generated ....
>>>
>>> As far as I can see Fastbit fails to evaluate such query. I suspect
>>> doEvaluate method not being able to properly handle such type of query.
>>> I give up for today, but I will continue tomorrow, I would appreciate
>>> an advice.
>>> What do you think ?
>>>
>>> Thanks,
>>>
>>> Alexandre
>>>
>>> K. John Wu a écrit :
>>>> Hi, Alexandre,
>>>>
>>>> Attached is the current source tar ball. I suspect that the nightly
>>>> snapshot might have something that is just a little too old -- I
>>>> remembering encountering this issue you encountered and thought I
>>>> have addressed it already. On way to verify this might to be look
>>>> at file ChangeLog and see how many entries are there for
>>>> 2009-12-08. The latest file should have 5 entries and the second
>>>> one from the top (about src/whereParser.yy) is the one related to
>>>> this particular problem. Last time the parser was modified was
>>>> 2009-12-03; that one apparently has the behavior you've observed.
>>>>
>>>> Please give this tar ball a try or go down the nightly snapshot
>>>> again. Hope you don't mind all this back-and-forth. You are being
>>>> very helpful to us and we really appreciate it.
>>>>
>>>> Thanks.
>>>>
>>>> John
>>>>
>>>>
>>>> On 12/10/2009 4:33 AM, Alexandre Maurel wrote:
>>>>> Hello John,
>>>>>
>>>>> The problem is solved for the following query :
>>>>>
>>>>> %IBIS% -l log.txt -d .\dimensions\ColPart -q "select Inhibitor
>>>>> where Inhibitor = 'Imatinib'"
>>>>>
>>>>> but this one failed :
>>>>>
>>>>> %IBIS% -l log.txt -d .\dimensions\ColPart -q "select Inhibitor
>>>>> where Inhibitor IN ('Imatinib')"
>>>>>
>>>>> with the following message :
>>>>>
>>>>> ibis::whereParser encountered syntax error at location Inhibitor IN
>>>>> ('Imatinib'):1.14-23
>>>>> Warning -- whereClause(Inhibitor IN ('Imatinib')) failed to parse
>>>>> the string into an expression tree
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Alexandre
>>>>>
>>>>>
>>>>>
>>>>> [email protected] a écrit :
>>>>>> Send FastBit-users mailing list submissions to
>>>>>> [email protected]
>>>>>>
>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>> [email protected]
>>>>>>
>>>>>> You can reach the person managing the list at
>>>>>> [email protected]
>>>>>>
>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>> than "Re: Contents of FastBit-users digest..."
>>>>>>
>>>>>>
>>>>>> Today's Topics:
>>>>>>
>>>>>> 1. Re: query error (Alexandre Maurel)
>>>>>> 2. Re: query error (K. John Wu)
>>>>>>
>>>>>>
>>>>>> ----------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>> Message: 1
>>>>>> Date: Tue, 08 Dec 2009 10:30:47 +0100
>>>>>> From: Alexandre Maurel <[email protected]>
>>>>>> Subject: Re: [FastBit-users] query error
>>>>>> To: [email protected]
>>>>>> Message-ID: <[email protected]>
>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> Sorry for this late reply, some how I miss your answer.
>>>>>>
>>>>>> Here is the command line I used :
>>>>>>
>>>>>> .\Debug\ardea.exe -d .\testCase\dimensions\ColPart -m
>>>>>> Barcode:k,Row:f,Col:f,Filter:f,Exposure_time:f,Cycle:f,Image:k,Temperature:f,Sample_name:k,Array:k,Sample_volume:f,Inhibitor:k,Inhibitor_conc_:f,Antibody:k,ATP_conc_:f,Visual_QC:k,Strip:f,Imatinib_Resistant:f,Imatinib:k
>>>>>>
>>>>>> -t .\testCase\Array.txt
>>>>>>
>>>>>> Note that this command line is stored as the description parameter
>>>>>> of the partition generated by ardea.exe (see -part.txt file I sent
>>>>>> ...) ;-)
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Alexandre
>>>>>>
>>>>>> [email protected] a ?crit :
>>>>>>
>>>>>>> Send FastBit-users mailing list submissions to
>>>>>>> [email protected]
>>>>>>>
>>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>>> [email protected]
>>>>>>>
>>>>>>> You can reach the person managing the list at
>>>>>>> [email protected]
>>>>>>>
>>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>>> than "Re: Contents of FastBit-users digest..."
>>>>>>>
>>>>>>>
>>>>>>> Today's Topics:
>>>>>>>
>>>>>>> 1. Re: query error (Alexandre Maurel)
>>>>>>> 2. Re: query error (K. John Wu)
>>>>>>>
>>>>>>>
>>>>>>> ----------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>> Message: 1
>>>>>>> Date: Thu, 03 Dec 2009 10:41:35 +0100
>>>>>>> From: Alexandre Maurel <[email protected]>
>>>>>>> Subject: Re: [FastBit-users] query error
>>>>>>> To: [email protected]
>>>>>>> Message-ID: <[email protected]>
>>>>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>>>>
>>>>>>> Hi John,
>>>>>>>
>>>>>>> The problem is still there.
>>>>>>> As attached document you will find ibis log file , generated by
>>>>>>> the following command,
>>>>>>>
>>>>>>> %IBIS% -l log.txt -d .\dimensions\ColPart -q "select Inhibitor
>>>>>>> where Inhibitor = 'Imatinib'"
>>>>>>> %IBIS% -l log.txt -d .\dimensions\ColPart -q "select Inhibitor
>>>>>>> where Inhibitor IN ('Imatinib')"
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Alexandre
>>>>>>>
>>>>>>> [email protected] a ?crit :
>>>>>>>
>>>>>>>> Send FastBit-users mailing list submissions to
>>>>>>>> [email protected]
>>>>>>>>
>>>>>>>> Message: 2
>>>>>>>> Date: Mon, 23 Nov 2009 09:55:20 -0800
>>>>>>>> From: "K. John Wu" <[email protected]>
>>>>>>>> Subject: Re: [FastBit-users] query error
>>>>>>>> To: FastBit Users <[email protected]>
>>>>>>>> Message-ID: <[email protected]>
>>>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>>>
>>>>>>>> Hi, Alexandre,
>>>>>>>>
>>>>>>>> Thanks for reporting the problem and exploring the solutions.
>>>>>>>> It turned out that we can tweak the lexer a little to recognize
>>>>>>>> all quoted strings as string literals -- which eliminate the
>>>>>>>> ambiguity in whether a string a column name or a string literal
>>>>>>>> and correctly represents the original intend of your query
>>>>>>>> expression. The change has been checked into the SVN
>>>>>>>> repository. Please give the nightly snapshot a try when you get
>>>>>>>> the chance.
>>>>>>>>
>>>>>>>> John
>>>>>>>>
>>>>>>>> PS: The nightly snapshot is available at
>>>>>>>> <https://codeforge.lbl.gov/snapshots.php?group_id=44> and it is
>>>>>>>> built in the early morning hours Pacific Time -- which means you
>>>>>>>> will need to wait till tomorrow morning in order to get your
>>>>>>>> hand on it. If you want to try it earlier, please let me know,
>>>>>>>> I will send you the source code separately.
>>>>>>>>
>>>>>>>>
>>>>>>> -------------- next part --------------
>>>>>>> An embedded and charset-unspecified text was scrubbed...
>>>>>>> Name: log.txt
>>>>>>> Url:
>>>>>>> http://hpcrdm.lbl.gov/pipermail/fastbit-users/attachments/20091203/08d6c17e/attachment-0001.txt
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> Message: 2
>>>>>>> Date: Thu, 03 Dec 2009 10:35:11 -0800
>>>>>>> From: "K. John Wu" <[email protected]>
>>>>>>> Subject: Re: [FastBit-users] query error
>>>>>>> To: FastBit Users <[email protected]>
>>>>>>> Message-ID: <[email protected]>
>>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>>
>>>>>>> Hi, Alexandre,
>>>>>>>
>>>>>>> Thanks for getting back to us. To make it a little easier for us
>>>>>>> to verify the problem has been fixed correctly, would you mind
>>>>>>> send us the column names and types you used to process the CSV
>>>>>>> data in Array.txt (you sent us earlier)? I think I am fixing
>>>>>>> correctly this time around, but need to verify it to be sure ;-)
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>> PS: You can either send us the command line of ardea you used
>>>>>>> process Array.txt or a file containing the names and types in the
>>>>>>> order as they appear in the file Array.txt. The content of
>>>>>>> -part.txt has all the right names and types, but they are not in
>>>>>>> the right order -- I have trouble matching the orders.
>>>>>>>
>>>>>>>
>>>>>>> On 12/3/2009 1:41 AM, Alexandre Maurel wrote:
>>>>>>>
>>>>>>>> Hi John,
>>>>>>>>
>>>>>>>> The problem is still there.
>>>>>>>> As attached document you will find ibis log file , generated by
>>>>>>>> the following command,
>>>>>>>>
>>>>>>>> %IBIS% -l log.txt -d .\dimensions\ColPart -q "select Inhibitor
>>>>>>>> where Inhibitor = 'Imatinib'"
>>>>>>>> %IBIS% -l log.txt -d .\dimensions\ColPart -q "select Inhibitor
>>>>>>>> where Inhibitor IN ('Imatinib')"
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Alexandre
>>>>>>>>
>>>>>>>> [email protected] a ?crit :
>>>>>>>>
>>>>>>>>> Send FastBit-users mailing list submissions to
>>>>>>>>> [email protected]
>>>>>>>>>
>>>>>>>>> Message: 2
>>>>>>>>> Date: Mon, 23 Nov 2009 09:55:20 -0800
>>>>>>>>> From: "K. John Wu" <[email protected]>
>>>>>>>>> Subject: Re: [FastBit-users] query error
>>>>>>>>> To: FastBit Users <[email protected]>
>>>>>>>>> Message-ID: <[email protected]>
>>>>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>>>>
>>>>>>>>> Hi, Alexandre,
>>>>>>>>>
>>>>>>>>> Thanks for reporting the problem and exploring the solutions.
>>>>>>>>> It turned out that we can tweak the lexer a little to recognize
>>>>>>>>> all quoted strings as string literals -- which eliminate the
>>>>>>>>> ambiguity in whether a string a column name or a string literal
>>>>>>>>> and correctly represents the original intend of your query
>>>>>>>>> expression. The change has been checked into the SVN
>>>>>>>>> repository. Please give the nightly snapshot a try when you
>>>>>>>>> get the chance.
>>>>>>>>>
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>> PS: The nightly snapshot is available at
>>>>>>>>> <https://codeforge.lbl.gov/snapshots.php?group_id=44> and it is
>>>>>>>>> built in the early morning hours Pacific Time -- which means
>>>>>>>>> you will need to wait till tomorrow morning in order to get
>>>>>>>>> your hand on it. If you want to try it earlier, please let me
>>>>>>>>> know, I will send you the source code separately.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> FastBit-users mailing list
>>>>>>>> [email protected]
>>>>>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> FastBit-users mailing list
>>>>>>> [email protected]
>>>>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>>>>>>
>>>>>>>
>>>>>>> End of FastBit-users Digest, Vol 28, Issue 1
>>>>>>> ********************************************
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 2
>>>>>> Date: Tue, 08 Dec 2009 10:21:47 -0800
>>>>>> From: "K. John Wu" <[email protected]>
>>>>>> Subject: Re: [FastBit-users] query error
>>>>>> To: FastBit Users <[email protected]>
>>>>>> Message-ID: <[email protected]>
>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>
>>>>>> Hi, Alexandre,
>>>>>>
>>>>>> Thanks for the information. I have just successfully run the
>>>>>> query that was giving you trouble. The code has been checked into
>>>>>> the SVN repository. The nightly snapshot is available at
>>>>>> <https://codeforge.lbl.gov/snapshots.php?group_id=44> and is
>>>>>> generated daily around 2AM PST. When you get a chance, please
>>>>>> give it a try and let us know if it works for you.
>>>>>>
>>>>>> We would like to turn your test data (Array.txt) and sample query
>>>>>> into part of the built-in tests. Would you mind grand us the
>>>>>> permission to do so? In addition, would you like to add a few
>>>>>> more queries for this purpose?
>>>>>>
>>>>>> John
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> _______________________________________________
>>>>>> FastBit-users mailing list
>>>>>> [email protected]
>>>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>>>>>
>>>>>>
>>>>>> End of FastBit-users Digest, Vol 28, Issue 2
>>>>>> ********************************************
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> FastBit-users mailing list
>>>>> [email protected]
>>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
