[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726659#comment-14726659 ] Susheel Kumar commented on SOLR-7560: - Hi Joel, I have created a JIRA SOLR-8002 for field aliases. Also noticed the JIRA SOLR-7986 you have created and design thoughts on creating Connection, Statement & ResultSet classes. Thanks, Susheel > Parallel SQL Support > > > Key: SOLR-7560 > URL: https://issues.apache.org/jira/browse/SOLR-7560 > Project: Solr > Issue Type: New Feature > Components: clients - java, search >Reporter: Joel Bernstein > Fix For: Trunk > > Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, > SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch > > > This ticket provides support for executing *Parallel SQL* queries across > SolrCloud collections. The SQL engine will be built on top of the Streaming > API (SOLR-7082), which provides support for *parallel relational algebra* and > *real-time map-reduce*. > Basic design: > 1) A new SQLHandler will be added to process SQL requests. The SQL statements > will be compiled to live Streaming API objects for parallel execution across > SolrCloud worker nodes. > 2) SolrCloud collections will be abstracted as *Relational Tables*. > 3) The Presto SQL parser will be used to parse the SQL statements. > 4) A JDBC thin client will be added as a Solrj client. > This ticket will focus on putting the framework in place and providing basic > SELECT support and GROUP BY aggregate support. > Future releases will build on this framework to provide additional SQL > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712381#comment-14712381 ] Susheel Kumar commented on SOLR-7560: - Thanks, Joel. I'll try to write tests around the supported features. Do you have any list of items/tickets for future releases or If I can help to maintain. Also i will try to understand how presto is being plugged with Solr but if you have any pointers please let me know. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712441#comment-14712441 ] Joel Bernstein commented on SOLR-7560: -- SOLR-7903 and SOLR-7905 are currently being worked on. Field aliases are the next thing I'd like to tackle, but there isn't a ticket for it yet. Presto is currently only used to parse the SQL queries. The code for this is in the SQLHandler. One area that might be interesting for you to think about is the JDBC driver. If you wanted to create a ticket for that I'd be happy to help with the design. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709862#comment-14709862 ] Joel Bernstein commented on SOLR-7560: -- Hi Susheel, The two queries you tried are not yet supported. They are both high priorities but might not be in the initial release. The documentation link (https://cwiki.apache.org/confluence/display/solr/Parallel+SQL+Interface) describes the types of queries currently supported in trunk. Joel Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708682#comment-14708682 ] Susheel Kumar commented on SOLR-7560: - Hi Joel, I started with two basic tests on my local box a) add field alias e.g. select id,name as product_name from techproducts limit 20 which currently fails as data returned contains still name as the field/column key than product_name b) I wanted to get additional field returned from SQL e.g select id,name,manu,mul(price,weight) from techproducts limit 20 which currently fails with error Aggregate functions only supported with group by queries. while actually I just want to have additional calculated field based on some function/formula for every document. I checked SQLHandler.java which currently throws this out due to presence of parenthesis without any group by/aggregate function. Please let me know your suggestion on this. Thanks, Susheel Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705552#comment-14705552 ] Susheel Kumar commented on SOLR-7560: - Thanks, Eric for pointing server dist target. Now I am able to run basic SQL. Will start looking into deeper. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703667#comment-14703667 ] Susheel Kumar commented on SOLR-7560: - Hi, I can help to test the Parallel SQL Support feature which is very useful for analytical purpose. Can I get some info on setting up SQLHandler / some instructions to get started. Thanks, Susheel Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703784#comment-14703784 ] Joel Bernstein commented on SOLR-7560: -- I will spend some time working on the documentation. So far the docs mostly covers how to form queries. The link to docs is here: https://cwiki.apache.org/confluence/display/solr/Parallel+SQL+Interface I'll post back to this ticket when I've added the sections on sending the query, configuration and the architecture. If you want to get a jump on things the sample configs in trunk have the request handlers already setup. You can take a look at the test cases to see how to send a SQL query: https://svn.apache.org/repos/asf/lucene/dev/trunk/solr/core/src/test/org/apache/solr/handler/TestSQLHandler.java Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704223#comment-14704223 ] Susheel Kumar commented on SOLR-7560: - Thanks, Joel. Found the SQLHandler config and I did checkout the trunk and compiled using 'ant compile' but getting error when starting solr. Will look into classpath which may be causing the issue. ./bin/solr status Found 1 Solr nodes: Solr process 11789 running on port 8983 Error: Could not find or load main class org.apache.solr.util.SolrCLI. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-7560) Parallel SQL Support
Try these two targets instead: ant server dist to build both the sever and distribution targets, that way you'll have everything you need, including the jars that you can path to from, say, SolrJ programs which will be in the dist directory and the solrj-lib child of dist. On Wed, Aug 19, 2015 at 7:59 PM, Susheel Kumar (JIRA) j...@apache.org wrote: [ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704223#comment-14704223 ] Susheel Kumar commented on SOLR-7560: - Thanks, Joel. Found the SQLHandler config and I did checkout the trunk and compiled using 'ant compile' but getting error when starting solr. Will look into classpath which may be causing the issue. ./bin/solr status Found 1 Solr nodes: Solr process 11789 running on port 8983 Error: Could not find or load main class org.apache.solr.util.SolrCLI. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: Trunk Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679326#comment-14679326 ] Joel Bernstein commented on SOLR-7560: -- Going to stick with the Presto Parser and release this ticket with Solr 6. There is a patch on this ticket that uses the Calcite SQL parser. In the future we can refer to that patch if we decide to switch to Calcite. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679322#comment-14679322 ] ASF subversion and git services commented on SOLR-7560: --- Commit 1694905 from [~joel.bernstein] in branch 'dev/trunk' [ https://svn.apache.org/r1694905 ] SOLR-7560: Updated CHANGES.txt Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.4 Attachments: SOLR-7560.calcite.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637434#comment-14637434 ] ASF subversion and git services commented on SOLR-7560: --- Commit 1692319 from [~joel.bernstein] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1692319 ] SOLR-7560: Parallel SQL Support Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637741#comment-14637741 ] ASF subversion and git services commented on SOLR-7560: --- Commit 1692331 from [~joel.bernstein] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1692331 ] SOLR-7560: Reverting 5x commit Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637743#comment-14637743 ] Joel Bernstein commented on SOLR-7560: -- Ok, I've reverting everything I believe. Bummer. I need to think about how get this released now. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637675#comment-14637675 ] Shawn Heisey commented on SOLR-7560: The requirements on the page for facebook/presto (which is the parent for presto-parser) show Java 8 as a requirement: https://github.com/facebook/presto Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637613#comment-14637613 ] Steve Rowe commented on SOLR-7560: -- Same error here: http://jenkins.sarowe.net/job/Lucene-Solr-tests-5.x-Java7/903/ Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637682#comment-14637682 ] Joel Bernstein commented on SOLR-7560: -- Ok, looks like I'll need to revert. I'll start the process. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637607#comment-14637607 ] Steve Rowe commented on SOLR-7560: -- Joel, your branch_5x commit won't compile under java7 for me: {noformat} common.compile-core: [mkdir] Created dir: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/build/solr-core/classes/java [javac] Compiling 835 source files to /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/build/solr-core/classes/java [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/ExpressionFormatter.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/handler/SQLHandler.java:54: error: cannot find symbol [javac] import java.util.Optional; [javac] ^ [javac] symbol: class Optional [javac] location: package java.util [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/parser/SqlParser.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/AstVisitor.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/SortItem.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/Expression.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/Node.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/Unnest.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/Query.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/QuerySpecification.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/ComparisonExpression.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/Select.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/SingleColumn.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning: /Users/sarowe/svn/lucene/dev/branches/branch_5x/solr/core/lib/presto-parser-0.108.jar(com/facebook/presto/sql/tree/AllColumns.class): major version 52 is newer than 51, the highest major version supported by this compiler. [javac] It is recommended that the compiler be upgraded. [javac] warning:
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637989#comment-14637989 ] Joel Bernstein commented on SOLR-7560: -- Investigating using the Apache Calcite SQL parser rather the Presto SQL parser. Calcite is Java 7 compatible and used by Hive and Drill, so we'll be in good company. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635770#comment-14635770 ] Joel Bernstein commented on SOLR-7560: -- Just wrapped up the work on SOLR-7441 which really makes the SQL interface much more robust and user friendly when errors occur. I think the SQL work is ready to backport to the 5x branch. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601826#comment-14601826 ] ASF subversion and git services commented on SOLR-7560: --- Commit 1687623 from [~joel.bernstein] in branch 'dev/trunk' [ https://svn.apache.org/r1687623 ] SOLR-7560: Removing Presto 0.107 SHA1 Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601798#comment-14601798 ] Joel Bernstein commented on SOLR-7560: -- I think we only need the latest version. I'll remove presto-parser-0.107.jar.sha1 shortly. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601435#comment-14601435 ] Joel Bernstein commented on SOLR-7560: -- ok, great. I'll switch over to the 0.108 parser. thanks! Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601430#comment-14601430 ] Christine Poerschke commented on SOLR-7560: --- Hi [~joel.bernstein] - just wanted to mention that the {{0.108}} presto-parser release that came out late yesterday didn't have the difficulties that {{0.107}} (and {{0.106}}) had. I haven't yet looked what the changes between 0.107 and 0.108 are, but 'ant resolve' and 'ant dist' certainly work again now (still with the above customised {{lucene/ivy-settings.xml}}). Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601696#comment-14601696 ] ASF subversion and git services commented on SOLR-7560: --- Commit 1687603 from [~joel.bernstein] in branch 'dev/trunk' [ https://svn.apache.org/r1687603 ] SOLR-7560: Bump Presto Parser version to 0.108 Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601776#comment-14601776 ] Christine Poerschke commented on SOLR-7560: --- great, thanks for bumping the version up to {{0.108}} - when refreshing my local branch of trunk i noticed that it acquired a {{solr/licenses/presto-parser-0.108.jar.sha1}} file as part of the version increase, hadn't noticed that file before - question (please excuse my lack of knowledge on this): can the {{solr/licenses/presto-parser-0.107.jar.sha1}} be removed because the 0.108 equivalent replaces it or is there a need or obligation to keep it around with perhaps only solr releases containing the latest version's .sha1 files? Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591948#comment-14591948 ] Joel Bernstein commented on SOLR-7560: -- [~cpoerschke], thanks for the comments and the link was very informative. Just tried another fresh pull and the warnings are still there so the SHA1 files haven't been updated yet. A couple things I'd like to understand better based on your comments: 1) How come your builds were failing, while mine were successful with warnings? Are you building with Maven? 2) Before a release it will be important to nail down the correct ivy conf. What should the scope be and whether to include all the Presto Parser decencies or just the ones being used in this ticket. This ticket uses about 10% of the full capabilities of the Presto Parser so I didn't want to have un-needed jars in the release, but possibly that was not the right decision. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592049#comment-14592049 ] Christine Poerschke commented on SOLR-7560: --- It was the 'ant resolve' (or 'ant compile' or 'ant dist') that was failing. {{ant -verbose resolve}} gave this output (for presto-parser 0.107) {code} [ivy:retrieve] == resolving dependencies org.apache.solr#core;working@hostname-com.facebook.presto#presto-parser;0.107 [compile-master] [ivy:retrieve] bb-default: Checking cache for: dependency: com.facebook.presto#presto-parser;0.107 {compile=[master], compile.hadoop=[master], test=[master], test.DfsMiniCluster=[master], test.MiniKdc=[master]} [ivy:retrieve] don't use cache for com.facebook.presto#presto-parser;0.107: checkModified=true [ivy:retrieve] tried /mydirectory/.ivy2/local/com.facebook.presto/presto-parser/0.107/ivys/ivy.xml [ivy:retrieve] tried /mydirectory/.ivy2/local/com.facebook.presto/presto-parser/0.107/jars/presto-parser.jar [ivy:retrieve] local: no ivy file nor artifact found for com.facebook.presto#presto-parser;0.107 [ivy:retrieve] tried http://.../artifactory/repo1/com/facebook/presto/presto-parser/0.107/presto-parser-0.107.pom [ivy:retrieve] CLIENT ERROR: Conflict url=http://.../artifactory/repo1/com/facebook/presto/presto-parser/0.107/presto-parser-0.107.pom [ivy:retrieve] tried http://.../artifactory/repo1/com/facebook/presto/presto-parser/0.107/presto-parser-0.107.jar [ivy:retrieve] bb-art-main: no ivy file found for com.facebook.presto#presto-parser;0.107: using default data [ivy:retrieve] found com.facebook.presto#presto-parser;0.107 in bb-art-main {code} using customised {{lucene/ivy-settings.xml}} so that the machine running the build need not directly access external repositories. This is what the customisation patch looks like: {code} - settings defaultResolver=default/ + settings defaultResolver=bb-default/ ... resolvers -ibiblio name=sonatype-releases root=https://oss.sonatype.org/content/repositories/releases; m2compatible=true / + +ibiblio name=bb-art-main root=http://.../artifactory/repo1; m2compatible=true / +ibiblio name=bb-art-jcenter root=http://.../artifactory/jcenter; m2compatible=true / +ibiblio name=bb-art-cloudera root=http://.../artifactory/cloudera.repo; m2compatible=true / + +ibiblio name=bb-art-sonatype-releases root=http://.../artifactory/sonatype-public; m2compatible=true / +ibiblio name=bb-art-maven-restlet root=http://.../artifactory/maven-restlet; m2compatible=true / + +ibiblio name=bb-nexus-main root=http://.../content/repositories/repo1; m2compatible=true / +ibiblio name=bb-nexus-central root=http://.../content/repositories/central; m2compatible=true / +ibiblio name=bb-nexus-cloudera root=http://.../content/repositories/cloudera-public; m2compatible=true / +ibiblio name=bb-nexus-sonatype-releases root=http://.../content/repositories/sonatype-public; m2compatible=true / + +chain name=bb-default returnFirst=true checkmodified=true changingPattern=.*SNAPSHOT + resolver ref=local/ + resolver ref=bb-art-main/ + resolver ref=bb-art-jcenter/ + resolver ref=bb-art-cloudera/ +/chain + +ibiblio name=sonatype-releases root=http://oss.sonatype.org/content/repositories/releases; m2compatible=true / ... {code} Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591603#comment-14591603 ] Christine Poerschke commented on SOLR-7560: --- Thanks [~joel.bernstein], i will follow the presto-parser project ticket then from here on. (Hadn't mentioned before but changing {{lucene/ivy-versions.properties}}'s {{/com.facebook.presto/presto-parser}} from {{0.107}} to {{0.105}} worked i.e. only 0.107 and 0.106 caused the error and 0.105 worked without 'conf' attribute addition i.e. retrospectively that would then also point towards a difference/issue w.r.t. presto-parser project.) Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590651#comment-14590651 ] Joel Bernstein commented on SOLR-7560: -- Just blew away the ivy cache, did a fresh truck checkout and 'ant compile'. The compile was successful but I did get these warnings: ivy:retrieve] WARNINGS [ivy:retrieve] problem while downloading module descriptor: http://repo1.maven.org/maven2/com/facebook/presto/presto-parser/0.107/presto-parser-0.107.pom: invalid sha1: expected=7f2f854a24db4cc9b5c26c3e9c6e46da37abbed1 computed=661fee4927f5b356f91880c308e5bf586568ff3c (180ms) [ivy:retrieve] problem while downloading module descriptor: http://repo1.maven.org/maven2/com/facebook/presto/presto-root/0.107/presto-root-0.107.pom: invalid sha1: expected=3e2a2729fd8b728fdc29c4062f5dc9a138c433cd computed=2cdf6c93caab02a26e6280862ed25452b862e5bf (218ms) [ivy:retrie I'll take a look and see what the issue is. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590673#comment-14590673 ] Joel Bernstein commented on SOLR-7560: -- It appears that sha1 files are incorrect for: http://repo1.maven.org/maven2/com/facebook/presto/presto-parser/0.107/presto-parser-0.107.pom http://repo1.maven.org/maven2/com/facebook/presto/presto-root/0.107/presto-root-0.107.pom Wondering if this causing your errors? Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590619#comment-14590619 ] Joel Bernstein commented on SOLR-7560: -- Can you paste the steps you're using to reproduce the error into the ticket? Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590683#comment-14590683 ] Joel Bernstein commented on SOLR-7560: -- It looks like the issue needs to be fixed on the Presto project. I will notify the project. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591095#comment-14591095 ] Joel Bernstein commented on SOLR-7560: -- Here is the ticket on the presto-parser project: https://github.com/facebook/presto/issues/3126 Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590518#comment-14590518 ] Christine Poerschke commented on SOLR-7560: --- Looked into and learnt some more on this. The original {{'ant resolve'}} error was {code} [ivy:retrieve] :: [ivy:retrieve] :: UNRESOLVED DEPENDENCIES :: [ivy:retrieve] :: [ivy:retrieve] :: com.facebook.presto#presto-parser;0.107: configuration not found in com.facebook.presto#presto-parser;0.107: 'master'. It was required from org.apache.solr#core;working@hostname compile.hadoop [ivy:retrieve] :: {code} and i found http://wrongnotes.blogspot.co.uk/2014/02/simplest-explanation-of-ivy.html useful to understand the meaning of the error and what the {{conf=compile-*}} attribute addition means (still in combination with customised {{lucene/ivy-settings.xml}} though). Following on from that, i cleared my ivy cache and tried this alternative, shorter dependency declaration {code} -dependency org=com.facebook.presto name=presto-parser rev=${/com.facebook.presto/presto-parser}/ -dependency org=org.antlr name=antlr4-runtime rev=${/org.antlr/antlr4-runtime}/ -dependency org=io.airlift name=slice rev=${/io.airlift/slice}/ +dependency org=com.facebook.presto name=presto-parser rev=${/com.facebook.presto/presto-parser} conf=compile.hadoop-*/ {code} and that also worked but since the https://repo1.maven.org/maven2/com/facebook/presto/presto-parser/0.107/presto-parser-0.107.pom file has additional dependencies apart from {{org.antlr antlr4-runtime}} and {{io.airlift slice}} the shorter alternative is perhaps/probably not equivalent to the original dependency trio. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588279#comment-14588279 ] Joel Bernstein commented on SOLR-7560: -- It doesn't seem to be breaking jenkins. I'll clear my ivy cache and see if I can reproduce the errors. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588177#comment-14588177 ] Christine Poerschke commented on SOLR-7560: --- Hello, was experiencing {{'ant resolve'}} difficulties related to the {{com.facebook.presto}} dependency that was added above. We have tried a few things and just adding {{conf=compile-*}} attributes for the new dependencies in {{solr/core/ivy.xml}} makes it work (this is in combination with customised {{lucene/ivy-settings.xml}} though). {code} -dependency org=com.facebook.presto name=presto-parser rev=${/com.facebook.presto/presto-parser}/ -dependency org=org.antlr name=antlr4-runtime rev=${/org.antlr/antlr4-runtime}/ -dependency org=io.airlift name=slice rev=${/io.airlift/slice}/ +dependency org=com.facebook.presto name=presto-parser rev=${/com.facebook.presto/presto-parser} conf=compile-*/ +dependency org=org.antlr name=antlr4-runtime rev=${/org.antlr/antlr4-runtime} conf=compile-*/ +dependency org=io.airlift name=slice rev=${/io.airlift/slice} conf=compile-*/ {code} Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588294#comment-14588294 ] Joel Bernstein commented on SOLR-7560: -- Wasn't able to reproduce. I blew away the ivy caches, did a fresh trunk pull, and the presto dependancies were pulled right down during an ant compile. I'll do a little more research ... Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585271#comment-14585271 ] Joel Bernstein commented on SOLR-7560: -- There is still some more work to do before it's ready for a release but I believe this ticket is far enough along to commit to *trunk*. The main goals have been accomplished: Two SQL constructs are supported: 1) simple SELECT statements with complex nested WHERE clauses, ORDER BY and LIMIT. 2) GROUP BY statements with aggregate functions, complex nested WHERE and HAVING clauses, ORDER BY and LIMIT. The functions supported are SUM, COUNT, AVG, MIN, MAX. The GROUP BY aggregations can be run in parallel across a SolrCloud worker collection, providing true *Parallel SQL framework* for SolrCloud. A new /sql request (SQLHandler.java) has been created to handle SQL requests and stream back the results. Initial test cases demonstrate the functionality described above. The excellent *Presto SQL parser* has been integrated into the SQLHandler and all the licenses for new dependencies have been added to the project. The JDBC driver has not been built as part of this ticket so the SQLHandler's HTTP interface will need to be accessed directly. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585386#comment-14585386 ] ASF subversion and git services commented on SOLR-7560: --- Commit 1685497 from [~joel.bernstein] in branch 'dev/trunk' [ https://svn.apache.org/r1685497 ] SOLR-7560: Parallel SQL Support Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch, SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578750#comment-14578750 ] Dennis Gove commented on SOLR-7560: --- Possible expression syntax for the RollupStream {code} rollup( someStream(), over=fieldA, fieldB, fieldC, min(fieldA), max(fieldA), min(fieldB), mean(fieldD), sum(fieldC) ) {code} This would require making the *Metric types Expressible but I think that ends up as a good thing. Would make it real easy to support other options on metrics like excluding outliers, for example find the sum of values within 3 standard deviations from the mean could be {code} sum(fieldC, limit=standardDev(3)) {code} (note, how that particular calculation could be implemented is left as an exercise for the reader, I'm just using it as an example of adding additional options on a relatively simple metric). Another option example is what to do with null values. For example, in some cases a null should not impact a mean but in others it should. You could express those as {code} mean(fieldA, replace(null, 0)) // replace null values with 0 thus leading to an impact on the mean mean(fieldA, includeNull=true) // nulls are counted in the denominator but nothing added to numerator mean(fieldA, includeNull=false) // nulls neither counted in denominator nor added to numerator mean(fieldA, replace(null, fieldB), includeNull=true) // if fieldA is null replace it with fieldB, include null fieldB in mean {code} so on and so forth. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578827#comment-14578827 ] Joel Bernstein commented on SOLR-7560: -- Looks great! A little background on this discussion. [~dpgove] is describing the Streaming Expression (SOLR-7377) syntax for the RollupStream which is being added in this ticket to support the SQL aggregate functions. The RollupStream does Map/Reduce style aggregations where group by fields are sorted first and the aggregates are rolled up one group at a time. This technique will be very strong for time series rollups and aggregating high cardinality fields. There will be other aggregation streams added in the future that tap into Solr faceting directly. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein Fix For: 5.3 Attachments: SOLR-7560.patch This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555966#comment-14555966 ] Joel Bernstein commented on SOLR-7560: -- Still working on the details of the queries. I will post more information with patch. Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7560) Parallel SQL Support
[ https://issues.apache.org/jira/browse/SOLR-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552839#comment-14552839 ] Noble Paul commented on SOLR-7560: -- Looking forward to this Can you post some example queries Parallel SQL Support Key: SOLR-7560 URL: https://issues.apache.org/jira/browse/SOLR-7560 Project: Solr Issue Type: New Feature Components: clients - java, search Reporter: Joel Bernstein This ticket provides support for executing *Parallel SQL* queries across SolrCloud collections. The SQL engine will be built on top of the Streaming API (SOLR-7082), which provides support for *parallel relational algebra* and *real-time map-reduce*. Basic design: 1) A new SQLHandler will be added to process SQL requests. The SQL statements will be compiled to live Streaming API objects for parallel execution across SolrCloud worker nodes. 2) SolrCloud collections will be abstracted as *Relational Tables*. 3) The Presto SQL parser will be used to parse the SQL statements. 4) A JDBC thin client will be added as a Solrj client. This ticket will focus on putting the framework in place and providing basic SELECT support and GROUP BY aggregate support. Future releases will build on this framework to provide additional SQL features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org