[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-04-10 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12487826 ] A B commented on DERBY-47: -- The issue filed by Andrew (DERBY-2500) has been resolved with no apparent fallout. I have also

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-29 Thread Mike Matrigali
If there are no query plans printed then maybe the right thing to do is to consider running the test multiple times, with different properties set. Considering you found a bug by effectively doing so it seems worthwhile - but I don't know how worthwhile. One the benefits of the whole junit

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-29 Thread Army
Mike Matrigali wrote: If there are no query plans printed then maybe the right thing to do is to consider running the test multiple times, with different properties set. [snip] maybe we could configure all the non-queryplan printing tests into a suite and then enable or disable the

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-29 Thread Mike Matrigali
Army wrote: Mike Matrigali wrote: If there are no query plans printed then maybe the right thing to do is to consider running the test multiple times, with different properties set. [snip] maybe we could configure all the non-queryplan printing tests into a suite and then enable or

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-29 Thread Army
Mike Matrigali wrote: I agree, I don't think we should run every test, under every configuration, in suites.ALL. [...] So for instance it would be interesting to run all the tests under the non-cost based optimizer once per release/per week/per ???, but maybe not so interesting to do it as

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-28 Thread Army
Laura Stewart (JIRA) wrote: In the Tuning Guide, the IN operator is mentioned in the optimization and query sections, as shown below: snip sections Thank you, Laura. I filed DERBY-2499 for documentation of the new IN list behavior. I'll try to look into this sometime in the near future...

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-28 Thread Andrew McIntyre (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12484985 ] Andrew McIntyre commented on DERBY-47: -- Hi Army, I just checked in a converted JUnit test for the old

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-28 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12484994 ] A B commented on DERBY-47: -- Hi Andrew, Thank you for reporting this! My guess is that the difference between the JUnit

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-28 Thread Andrew McIntyre (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485002 ] Andrew McIntyre commented on DERBY-47: -- Hah! I had already deleted all the properties files for the old tests in

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-28 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485008 ] A B commented on DERBY-47: -- 1. shall I open a new issue for the test failure? Yes, I think that'd be good. Note though

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-28 Thread Andrew McIntyre (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485028 ] Andrew McIntyre commented on DERBY-47: -- Army Note though that this is an actual engine bug, not just a test

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-28 Thread Army
Andrew McIntyre (JIRA) wrote: and since all the tests pass with or without ruleBasedOptimization set to true, I'll leave that off. Not really worth investigating, since it seems to have no effect on the new test. Unless of course setting that property to true revealed an error once upon a

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-27 Thread Laura Stewart (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12484620 ] Laura Stewart commented on DERBY-47: Army - Just FYI In the Tuning Guide, the IN operator is mentioned in the

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-21 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483010 ] Bryan Pendleton commented on DERBY-47: -- Thanks for posting the measurement analysis, Army; it's good to see the

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-17 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12481865 ] Bryan Pendleton commented on DERBY-47: -- InListMultiProbeTest applied and built and passed in my environment.

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-16 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12481760 ] Bryan Pendleton commented on DERBY-47: -- Thanks for all the attention to detail, Army! The mp_cleanup_v1 patch

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-14 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12480827 ] A B commented on DERBY-47: -- Thanks again for the _excellent_ review, Mike. Unless I hear otherwise I plan to commit the

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-13 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12480534 ] Bryan Pendleton commented on DERBY-47: -- Hi Army, I had a look at the preprocess and master patches and they look

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-13 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12480546 ] A B commented on DERBY-47: -- 1) In one of the comments, you said: We intentionally use a parameter node instead of a

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-12 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12480146 ] A B commented on DERBY-47: -- Bryan Pendleton (JIRA) wrote: I finally got around to reading through the patches. I haven't

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-12 Thread Manish Khettry
4) The code which traverses predicate lists seems to always traverse them in backwards order, e.g. this code from HashJoinStrategy.java: for (int i = storeRestrictionList.size() - 1; i = 0; i--) Why do we always traverse these backwards? Is this just an optimization

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-11 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12479933 ] Bryan Pendleton commented on DERBY-47: -- Hi Army, I finally got around to reading through the patches. Sorry it

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-08 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12479350 ] A B commented on DERBY-47: -- maybe it's possible to explore why we ended up with 26 arguments here? Good question, Bryan.

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-08 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12479416 ] Bryan Pendleton commented on DERBY-47: -- Thanks Army for the good explanation. I didn't realize that the calls to

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-07 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12479232 ] Bryan Pendleton commented on DERBY-47: -- I was reading through d47_mp_codeGen_v1.patch, and the new

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-03-06 Thread Army
Bryan Pendleton (JIRA) wrote: Hi Army, just wanted to let you know that I've been reading your notes and reading the patches. Thank you very much, Bryan! I'm *really* glad to hear this--it's good to know someone else is at least sanity-checking the notes and changes. More eyes the better.

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-27 Thread Bryan Pendleton (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476489 ] Bryan Pendleton commented on DERBY-47: -- Something about this SQL statement just tickled my fancy and made me

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-26 Thread Army
A B (JIRA) wrote: A B commented on DERBY-47: -- Thank you very much for your excellent questions, Mike. My attempted answers are below... Hi Mike, Just wondering if you had any additional questions/inquiries regarding my previous responses to your comments? If

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-26 Thread Mike Matrigali
please go ahead with your patch - I think the changes are good incremental steps. The approach with using the flag and code reuse seems reasonable. I am interested in the costing but I think that is separate from the work you are doing. What you have implemented seems in line with the current

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-23 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12475414 ] A B commented on DERBY-47: -- Thank you very much for your excellent questions, Mike. My attempted answers are below... Can

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-22 Thread James Synge (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12475147 ] James Synge commented on DERBY-47: -- Army, this sounds like great progress, thanks. Will this impact UPDATES? I want

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-22 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12475176 ] A B commented on DERBY-47: -- Thank you very much for feedback, James. Will this impact UPDATES? I want to ensure that each

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-22 Thread James Synge (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12475227 ] James Synge commented on DERBY-47: -- Army, yes that's the behavior I was hoping to see. Thanks for double checking.

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2007-02-21 Thread A B (JIRA)
[ https://issues.apache.org/jira/browse/DERBY-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12474895 ] A B commented on DERBY-47: -- In one of my previous comments I mentioned that d47_engine_doNOTCommit_v1.patch has a problem

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-22 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12460530 ] James Synge commented on DERBY-47: -- As stated in my previous comment, I've been debugging the compilation of this query: SELECT * FROM tableA WHERE columnB IN (

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-22 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12460578 ] James Synge commented on DERBY-47: -- Some success at last... I forced SubqueryNode.preprocess to flatten the sub-query mentioned previously by modifying

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-21 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12460327 ] James Synge commented on DERBY-47: -- I've been thinking more about the suggestion of using a table constructor. I was concerned about several issues: 1) When I

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-20 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459973 ] James Synge commented on DERBY-47: -- I think I've found the code that caused a problem for me when I tried using VALUES to construct a table using parameters.

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-20 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459979 ] James Synge commented on DERBY-47: -- I've changed InListOperatorNode.preprocess to replace the node with a new SubqueryNode in the case where the list is all

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-19 Thread James Synge
Army wrote: In the bindNonVTITables() method of FromVTI there is the following call: ... I took a quick look at DMLStatementNode and it looks like binding of a top level ResultSetNode (which includes a SELECT node) involves calling the following three methods: bindNonVTITables(...)

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-19 Thread Daniel John Debrunner (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459697 ] Daniel John Debrunner commented on DERBY-47: Are there possible existing temporary table mechanisms that could be used instead of relying on VTIs? I

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-19 Thread Daniel John Debrunner (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459716 ] Daniel John Debrunner commented on DERBY-47: In DERBY-2152 James wrote: I'm experimenting with transforming a query such as:

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-19 Thread Daniel John Debrunner (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459722 ] Daniel John Debrunner commented on DERBY-47: The next question is how do I pass information to the VTI implementation regarding which parameters of

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-19 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459767 ] James Synge commented on DERBY-47: -- Dan wrote: Is this an step to solving DERBY-47, or does re-writing the query this way solve the performance problem? The

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-19 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459768 ] James Synge commented on DERBY-47: -- Dan wrote: ... store it as a saved object. Thanks for the pointer to CompilerContext.addSavedObject, I'd not come across it

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-19 Thread Daniel John Debrunner (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459777 ] Daniel John Debrunner commented on DERBY-47: Do you think that I could just pass InListOperatorNode.rightOperandList (a ValueNodeList) to

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-18 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459466 ] James Synge commented on DERBY-47: -- I'm currently looking to add the following if/else branch to InListOperatorNode: else if ((leftOperand

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-18 Thread Army
James Synge (JIRA) wrote: The basic question is when should I replace the IN predicate? The two basic choices would seem to be during preprocessing or during getNextDecoratedPermutation. Very good question. I guess I don't really know what the answer here is. As you say, the first way

Re: [jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-18 Thread Army
James Synge (JIRA) wrote: // Issue: how do I assign the table number for the FromVTI? In the bindNonVTITables() method of FromVTI there is the following call: /* Assign the tableNumber. (All other work done in bindVTITables() */ if (tableNumber == -1) // allow re-bind, in which

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-12-16 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12459063 ] James Synge commented on DERBY-47: -- I'm (finally) preparing to work on this issue. I have a general approach, which is to replace the IN LIST with an IN

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-11-21 Thread Bryan Pendleton (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12451812 ] Bryan Pendleton commented on DERBY-47: -- Note that there is a Wiki page related to this issue: http://wiki.apache.org/db-derby/DerbyBug47 Some possible

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2006-09-05 Thread James Synge (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12432636 ] James Synge commented on DERBY-47: -- I spent much of last week tracking down a performance problem in an application that I'm working on, and it turned out to be

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2005-11-24 Thread Daniel James Neades (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12358463 ] Daniel James Neades commented on DERBY-47: -- I'll try to schedule some time for one of our engineers to contribute a test case with data. We're in the middle of a

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2005-11-18 Thread Satheesh Bandaram (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12358064 ] Satheesh Bandaram commented on DERBY-47: Is it possible to contribute a test case with data that I can quickly setup to try on my machine? Make sure your data is

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2005-11-15 Thread Kevin Hore (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12357700 ] Kevin Hore commented on DERBY-47: - I have evidence (presented below) that the query optimizer is making some very poor choices when deciding how to make use of the available

[jira] Commented: (DERBY-47) Some possible improvements to IN optimization

2005-11-14 Thread Kevin Hore (JIRA)
[ http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12357601 ] Kevin Hore commented on DERBY-47: - I have also found this issue to be a problem and would like to know whether there are any plans to fix it? What follows is a copy of