[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13739267#comment-13739267 ] Navis commented on HIVE-4579: - Sorry for late comment, but would it be better to remove MINA dependency only for IdentityHashSet? Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.12.0 Attachments: h-4579.patch, HIVE-4579.4.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13737283#comment-13737283 ] Hudson commented on HIVE-4579: -- SUCCESS: Integrated in Hive-trunk-hadoop1-ptest #124 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/124/]) HIVE-4579: Create a SARG interface for RecordReaders (Owen O'Malley via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1513029) * /hive/trunk/ivy/libraries.properties * /hive/trunk/ql/ivy.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgument.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg/TestSearchArgumentImpl.java Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.12.0 Attachments: h-4579.patch, HIVE-4579.4.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736456#comment-13736456 ] Phabricator commented on HIVE-4579: --- hagleitn has accepted the revision HIVE-4579 [jira] Create a SARG interface for RecordReaders. LGTM +1 REVISION DETAIL https://reviews.facebook.net/D11409 BRANCH h-4579 ARCANIST PROJECT hive To: JIRA, hagleitn, omalley Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736458#comment-13736458 ] Gunther Hagleitner commented on HIVE-4579: -- +1 I really like this new interface. Planning commit in a couple of hours (going off of Ed's first +1 24 hours ago). Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736487#comment-13736487 ] Gunther Hagleitner commented on HIVE-4579: -- 20/20S look good, but compile fails for 23 because of missing mina dependency (IdentityHashSet in test code). Will fix, upload new patch and commit. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736499#comment-13736499 ] Gunther Hagleitner commented on HIVE-4579: -- Committed to trunk. Thanks Owen! Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.4.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736517#comment-13736517 ] Hudson commented on HIVE-4579: -- FAILURE: Integrated in Hive-trunk-hadoop2 #353 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/353/]) HIVE-4579: Create a SARG interface for RecordReaders (Owen O'Malley via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1513029) * /hive/trunk/ivy/libraries.properties * /hive/trunk/ql/ivy.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgument.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg/TestSearchArgumentImpl.java Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.12.0 Attachments: h-4579.patch, HIVE-4579.4.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736565#comment-13736565 ] Hudson commented on HIVE-4579: -- FAILURE: Integrated in Hive-trunk-h0.21 #2262 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2262/]) HIVE-4579: Create a SARG interface for RecordReaders (Owen O'Malley via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1513029) * /hive/trunk/ivy/libraries.properties * /hive/trunk/ql/ivy.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgument.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg/TestSearchArgumentImpl.java Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.12.0 Attachments: h-4579.patch, HIVE-4579.4.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736566#comment-13736566 ] Hudson commented on HIVE-4579: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/54/]) HIVE-4579: Create a SARG interface for RecordReaders (Owen O'Malley via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1513029) * /hive/trunk/ivy/libraries.properties * /hive/trunk/ql/ivy.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgument.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/sarg/TestSearchArgumentImpl.java Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.12.0 Attachments: h-4579.patch, HIVE-4579.4.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736103#comment-13736103 ] Owen O'Malley commented on HIVE-4579: - All of the unit tests pass on my machine and those tests are unrelated to the code in this patch. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736108#comment-13736108 ] Edward Capriolo commented on HIVE-4579: --- I think I roughly understand the interface, we are going to pass a Sarg into the conf of the RecordReader, then the record reader can apply these directly on the input row? Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[
https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736110#comment-13736110
]
Edward Capriolo commented on HIVE-4579:
---
{code}
+private final StackExpressionTree currentTree =
+new StackExpressionTree();
{code}
Can you use Deque and ArrayDeque here instead.
Create a SARG interface for RecordReaders
-
Key: HIVE-4579
URL: https://issues.apache.org/jira/browse/HIVE-4579
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: h-4579.patch, HIVE-4579.D11409.1.patch,
HIVE-4579.D11409.2.patch, pushdown.pdf
I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable)
interface for RecordReaders. For a first pass, I'll create an API that uses
the value stored in hive.io.filter.expr.serialized.
The desire is to define an simpler interface that the direct AST expression
that is provided by hive.io.filter.expr.serialized so that the code to
evaluate expressions can be generalized instead of put inside a particular
RecordReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736112#comment-13736112 ] Edward Capriolo commented on HIVE-4579: --- Other then the Deque and ArrayDeque I am +1. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[
https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736113#comment-13736113
]
Edward Capriolo commented on HIVE-4579:
---
One other q.
{quote}
+ public static enum Type {
+INTEGER, // all of the integer types
+FLOAT, // float and double
+STRING
+ }
{quote}
Should we call these Integral, Real instead of INTEGER, FLOAT? Or should we
call them LONG and DOUBLE? Because naming them the widest type might make more
sense?
Create a SARG interface for RecordReaders
-
Key: HIVE-4579
URL: https://issues.apache.org/jira/browse/HIVE-4579
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: h-4579.patch, HIVE-4579.D11409.1.patch,
HIVE-4579.D11409.2.patch, pushdown.pdf
I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable)
interface for RecordReaders. For a first pass, I'll create an API that uses
the value stored in hive.io.filter.expr.serialized.
The desire is to define an simpler interface that the direct AST expression
that is provided by hive.io.filter.expr.serialized so that the code to
evaluate expressions can be generalized instead of put inside a particular
RecordReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736162#comment-13736162 ] Owen O'Malley commented on HIVE-4579: - Ed, I changed the Stack to a Deque with the corresponding change of method names. I think integer and float are easier for people to remember than integral and real. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736172#comment-13736172 ] Owen O'Malley commented on HIVE-4579: - Ed, I forgot to answer your previous question. The serialized predicate is stored into the jobconf and the sarg api provides a simplified and shared mechanism for record readers to implement predicate push down based on the serialized predicate. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[
https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736193#comment-13736193
]
Hive QA commented on HIVE-4579:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12597321/HIVE-4579.D11409.3.patch
{color:green}SUCCESS:{color} +1 2787 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/389/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/389/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
Create a SARG interface for RecordReaders
-
Key: HIVE-4579
URL: https://issues.apache.org/jira/browse/HIVE-4579
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: h-4579.patch, HIVE-4579.D11409.1.patch,
HIVE-4579.D11409.2.patch, HIVE-4579.D11409.3.patch, pushdown.pdf
I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable)
interface for RecordReaders. For a first pass, I'll create an API that uses
the value stored in hive.io.filter.expr.serialized.
The desire is to define an simpler interface that the direct AST expression
that is provided by hive.io.filter.expr.serialized so that the code to
evaluate expressions can be generalized instead of put inside a particular
RecordReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735462#comment-13735462 ] Owen O'Malley commented on HIVE-4579: - Eric, In the current subset of SARGS, it isn't strictly required to track nulls, but if we want to support a little richer expressions we will. For example (c1 = c2) = (c1 != c2) is true if either c1 or c2 is null. Gunther, * +1 for changing the hive storage handlers to use the SARG interface. * I'll add a comment about regenerating the test cases. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[
https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735773#comment-13735773
]
Hive QA commented on HIVE-4579:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12597224/HIVE-4579.D11409.2.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 2787 tests executed
*Failed tests:*
{noformat}
org.apache.hcatalog.fileformats.TestOrcDynamicPartitioned.testHCatDynamicPartitionedTable
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/376/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/376/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
Create a SARG interface for RecordReaders
-
Key: HIVE-4579
URL: https://issues.apache.org/jira/browse/HIVE-4579
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: h-4579.patch, HIVE-4579.D11409.1.patch,
HIVE-4579.D11409.2.patch, pushdown.pdf
I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable)
interface for RecordReaders. For a first pass, I'll create an API that uses
the value stored in hive.io.filter.expr.serialized.
The desire is to define an simpler interface that the direct AST expression
that is provided by hive.io.filter.expr.serialized so that the code to
evaluate expressions can be generalized instead of put inside a particular
RecordReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731448#comment-13731448 ] Gunther Hagleitner commented on HIVE-4579: -- This is cool. Looks good. A few questions/comments: - What about [~ehans]'s recommendation? Is it possible to simplify what truth values you care about (filters v value-based expr)? - I think we should open a jira and use the same/similar interface for storage handlers. I've gotten requests in the past where ppl are asking for simpler structures to be pushed down. - Can you add comments on how to regenerate the serialized expression trees in the test cases? I'm thinking about the case where the serialized data goes stale because of changes to the expression tree code. If that happens we might have to regen the tests. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13711611#comment-13711611 ] Owen O'Malley commented on HIVE-4579: - I need to update HIVE-4242, but all that HIVE-4242 does is always enable predicate pushdown for file formats. Otherwise, the user needs to set hive.optimize.index.filter to enable the use of SARGs. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13701760#comment-13701760 ] Ashutosh Chauhan commented on HIVE-4579: Does this interface make HIVE-4242 redundant? Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, HIVE-4579.D11409.1.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669027#comment-13669027 ] Owen O'Malley commented on HIVE-4579: - I need to add the null-safe equals as an primitive clause. Having NULL as a boolean value makes things strange and I need to keep in mind that: * P and not P is not always false (it may be null) * P or not P is not always true (it may be null) I see why you wanted HIVE-4478 since if a column has no nulls, the rules for predicate evaluation are much tighter. :) Although we are going to want it in the index as well as in the stripe metadata. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669472#comment-13669472 ] Owen O'Malley commented on HIVE-4579: - Just to be clear, the sql semantics are: || Arg1 || Arg2 || Arg1 OR Arg2 || | false | false | false | | false | true | true | | false | null | null | | true | false | true | | true | true | true | | true | null | true | | null | false | null | | null | true | true | | null | null | null | || Arg1 || Arg2 || Arg1 AND Arg2 || | false | false | false | | false | true | false | | false | null | false | | true | false | false | | true | true | true | | true | null | null | | null | false | false | | null | true | null | | null | null | null | || Arg1 || NOT Arg1 || | false | true | | true | false | | null | null | Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669545#comment-13669545 ] Eric Hanson commented on HIVE-4579: --- Since SARGs are always for filters, not for value-based expressions, the null-preserving logic be ignored when testing SARGs, the way I understand it. e.g. Column 10 is always TRUE or FALSE when used as a filter, but may be TRUE, FALSE, or NULL when retrieved as a value-based expression. Also, it appears that maybe values in a SARG test against stripe metadata can just be consider TRUE for the purpose of that test since you have to look at the stripe in that case. Of course, that means the predicate in the SARG will have to be applied either at the row-by-row level again, or else applied as a residual by the query executor. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669548#comment-13669548 ] Mark Wagner commented on HIVE-4579: --- This looks good. My one comment is that it would be good to extend this to support nested columns, especially since all the recent columnar file formats that have come out support storing nested columns individually. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668881#comment-13668881 ] Eric Hanson commented on HIVE-4579: --- Overall this looks great. One issue that I think needs to be rephrased involves NOT. The expression ^(a b) is the same as a = b in SQL semantics. In general, for all the basic relational operators, , , =, =, , =, you can invert them when you apply NOT to them, thus eliminating the NOT, because any comparison with NULL is false. Of course, not all negated Boolean clauses can be converted to equivalent clauses without negation, and negation will have to be handled as a first class thing for those cases. I'm not sure what you mean by maybe answers. Can you be more explicit about All functions and udfs will be considered as indeterminate and eliminate the disjunction sub-expression that they occur in? I am not sure what that means exactly. I think you mean that such clauses will be removed from the SARG and treated as a residual. For The primitive clauses will be extended to include presence in a bloom filter do you mean that bloom filters will be added as a primitive clause type later? Please clarify. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4579.patch, pushdown.pdf I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[
https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669008#comment-13669008
]
Owen O'Malley commented on HIVE-4579:
-
Eric, thanks for the comments!
You're right that I was mistaken when I thought that 10 NULL was FALSE. :( It
is NULL and NOT(NULL) is still NULL. Therefore, you are right that you can
translate NOT(COL 10) to COL = 10.
The maybe answers are the parts of the SARG that can't be answered
definitively by the information in the index. For example, if the SARG is age
99 and the min age is 1 and the max age is 101, the value of the predicate
is maybe.
{quote}
I think you mean that such clauses will be removed from the SARG and treated as
a residual.{quote}
Yes
{quote}
For The primitive clauses will be
extended to include presence in a bloom filter do you mean that bloom filters
will be added as a primitive clause type later?
{quote}
Yes
Create a SARG interface for RecordReaders
-
Key: HIVE-4579
URL: https://issues.apache.org/jira/browse/HIVE-4579
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: h-4579.patch, pushdown.pdf
I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable)
interface for RecordReaders. For a first pass, I'll create an API that uses
the value stored in hive.io.filter.expr.serialized.
The desire is to define an simpler interface that the direct AST expression
that is provided by hive.io.filter.expr.serialized so that the code to
evaluate expressions can be generalized instead of put inside a particular
RecordReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664293#comment-13664293 ] Ashit Gosalia commented on HIVE-4579: - This is a broad enough interface. You may also want to consider supporting IN and BETWEEN clauses because particular RecordReaders may efficiently implement these special forms. Looking at the TPC-H and TPC-DS queries, top level ANDs also seem to be a common case. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[ https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664321#comment-13664321 ] Eric Hanson commented on HIVE-4579: --- Consider adding Column IN (list-of-constants) as a SIMPLE_COND. This is really commonly used. Create a SARG interface for RecordReaders - Key: HIVE-4579 URL: https://issues.apache.org/jira/browse/HIVE-4579 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable) interface for RecordReaders. For a first pass, I'll create an API that uses the value stored in hive.io.filter.expr.serialized. The desire is to define an simpler interface that the direct AST expression that is provided by hive.io.filter.expr.serialized so that the code to evaluate expressions can be generalized instead of put inside a particular RecordReader. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[
https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663437#comment-13663437
]
Owen O'Malley commented on HIVE-4579:
-
Another alternative is to put the expression into disjunctive normal form:
{code}
SARG: ALTERNATIVE | ALTERNATIVE or SARG
ALTERNATIVE: COND | COND and ALTERNATIVE
{code}
that forces the or conditions to be top level operators. Of course the most
effective filters will have a very small number of alternatives since the
effectiveness decreases rapidly as the number of alternatives increases.
Create a SARG interface for RecordReaders
-
Key: HIVE-4579
URL: https://issues.apache.org/jira/browse/HIVE-4579
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable)
interface for RecordReaders. For a first pass, I'll create an API that uses
the value stored in hive.io.filter.expr.serialized.
The desire is to define an simpler interface that the direct AST expression
that is provided by hive.io.filter.expr.serialized so that the code to
evaluate expressions can be generalized instead of put inside a particular
RecordReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4579) Create a SARG interface for RecordReaders
[
https://issues.apache.org/jira/browse/HIVE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13662486#comment-13662486
]
Owen O'Malley commented on HIVE-4579:
-
I plan to support SARGs such as:
{code}
SARG: COND | SARG or SARG | SARG and SARG
COND: SIMPLE_COND | not SIMPLE_COND
SIMPLE_COND: COLUMN OPERATOR VALUE
OPERATOR: | = | | = | == |
{code}
Create a SARG interface for RecordReaders
-
Key: HIVE-4579
URL: https://issues.apache.org/jira/browse/HIVE-4579
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
I think we should create a SARG (http://en.wikipedia.org/wiki/Sargable)
interface for RecordReaders. For a first pass, I'll create an API that uses
the value stored in hive.io.filter.expr.serialized.
The desire is to define an simpler interface that the direct AST expression
that is provided by hive.io.filter.expr.serialized so that the code to
evaluate expressions can be generalized instead of put inside a particular
RecordReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
