[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787164#comment-13787164 ] Hudson commented on HIVE-4888: -- FAILURE: Integrated in Hive-trunk-hadoop2 #480 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/480/]) HIVE-4888 : listPartitionsByFilter doesn't support lt/gt/lte/gte (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1529423) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java * /hive/trunk/ql/src/test/queries/clientpositive/filter_numeric.q * /hive/trunk/ql/src/test/results/clientpositive/filter_numeric.q.out * /hive/trunk/serde/if/serde.thrift * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h * /hive/trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java * /hive/trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php * /hive/trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py * /hive/trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, D13101.5.patch, D13101.6.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.05.patch, HIVE-4888.06.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787331#comment-13787331 ] Hudson commented on HIVE-4888: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #125 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/125/]) HIVE-4888 : listPartitionsByFilter doesn't support lt/gt/lte/gte (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1529423) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java * /hive/trunk/ql/src/test/queries/clientpositive/filter_numeric.q * /hive/trunk/ql/src/test/results/clientpositive/filter_numeric.q.out * /hive/trunk/serde/if/serde.thrift * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h * /hive/trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java * /hive/trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php * /hive/trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py * /hive/trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, D13101.5.patch, D13101.6.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.05.patch, HIVE-4888.06.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787355#comment-13787355 ] Hudson commented on HIVE-4888: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #191 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/191/]) HIVE-4888 : listPartitionsByFilter doesn't support lt/gt/lte/gte (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1529423) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java * /hive/trunk/ql/src/test/queries/clientpositive/filter_numeric.q * /hive/trunk/ql/src/test/results/clientpositive/filter_numeric.q.out * /hive/trunk/serde/if/serde.thrift * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h * /hive/trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java * /hive/trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php * /hive/trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py * /hive/trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, D13101.5.patch, D13101.6.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.05.patch, HIVE-4888.06.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787472#comment-13787472 ] Hudson commented on HIVE-4888: -- FAILURE: Integrated in Hive-trunk-h0.21 #2382 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2382/]) HIVE-4888 : listPartitionsByFilter doesn't support lt/gt/lte/gte (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1529423) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java * /hive/trunk/ql/src/test/queries/clientpositive/filter_numeric.q * /hive/trunk/ql/src/test/results/clientpositive/filter_numeric.q.out * /hive/trunk/serde/if/serde.thrift * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp * /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h * /hive/trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java * /hive/trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php * /hive/trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py * /hive/trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, D13101.5.patch, D13101.6.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.05.patch, HIVE-4888.06.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[
https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786945#comment-13786945
]
Hive QA commented on HIVE-4888:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12606861/D13101.6.patch
{color:green}SUCCESS:{color} +1 4054 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1038/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1038/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
listPartitionsByFilter doesn't support lt/gt/lte/gte
Key: HIVE-4888
URL: https://issues.apache.org/jira/browse/HIVE-4888
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch,
D13101.4.patch, D13101.5.patch, D13101.6.patch, HIVE-4888.00.patch,
HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.05.patch,
HIVE-4888.06.patch, HIVE-4888.on-top-of-4914.patch
Filter pushdown could be improved. Based on my experiments there's no
reasonable way to do it with DN 2.0, due to DN bug in substring and
Collection.get(int) not being implemented.
With version as low as 2.1 we can use values.get on partition to extract
values to compare to. Type compatibility is an issue, but is easy for strings
and integral values.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13785297#comment-13785297 ] Ashutosh Chauhan commented on HIVE-4888: Patch looks good. Just for documentation purposes, we can now push: * Filters on integral types to underlying db via directSQL for eq/lt/gt/gte/lte * Filters on string to underlying db via directSQL for eq * Filters on string to underlying db via DN for lt/gt/gte/lte [~sershe] .q tests test correctness, but don't test optimization done here. Will writing a junit test be better here? listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13785782#comment-13785782 ] Sergey Shelukhin commented on HIVE-4888: added more tests listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.05.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13785805#comment-13785805 ] Ashutosh Chauhan commented on HIVE-4888: +1 Thanks Sergey for adding tests! listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, D13101.5.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch, HIVE-4888.05.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[
https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13785919#comment-13785919
]
Hive QA commented on HIVE-4888:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12606706/D13101.5.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4053 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testPartitionFilter
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testPartitionFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testPartitionFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testPartitionFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartitionFilter
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1026/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1026/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
listPartitionsByFilter doesn't support lt/gt/lte/gte
Key: HIVE-4888
URL: https://issues.apache.org/jira/browse/HIVE-4888
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch,
D13101.4.patch, D13101.5.patch, HIVE-4888.00.patch, HIVE-4888.01.patch,
HIVE-4888.04.patch, HIVE-4888.05.patch, HIVE-4888.on-top-of-4914.patch
Filter pushdown could be improved. Based on my experiments there's no
reasonable way to do it with DN 2.0, due to DN bug in substring and
Collection.get(int) not being implemented.
With version as low as 2.1 we can use values.get on partition to extract
values to compare to. Type compatibility is an issue, but is easy for strings
and integral values.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[
https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13783788#comment-13783788
]
Hive QA commented on HIVE-4888:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12606218/HIVE-4888.04.patch
{color:green}SUCCESS:{color} +1 4078 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/991/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/991/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
listPartitionsByFilter doesn't support lt/gt/lte/gte
Key: HIVE-4888
URL: https://issues.apache.org/jira/browse/HIVE-4888
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch,
D13101.4.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.04.patch,
HIVE-4888.on-top-of-4914.patch
Filter pushdown could be improved. Based on my experiments there's no
reasonable way to do it with DN 2.0, due to DN bug in substring and
Collection.get(int) not being implemented.
With version as low as 2.1 we can use values.get on partition to extract
values to compare to. Type compatibility is an issue, but is easy for strings
and integral values.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13783450#comment-13783450 ] Sergey Shelukhin commented on HIVE-4888: Many of these queries pass for me. It looks like the patch that was tested was the first one out of the two, which had an issue. Let me update to trunk and re-submit the patch listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, D13101.2.patch, D13101.3.patch, D13101.4.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13780592#comment-13780592 ] Sergey Shelukhin commented on HIVE-4888: it looks like derby is evaluating a cast before it filters the values by join, or something like that. All the values it should test via cast are numbers and can be cast if the queries are issued for this specific set of values via separate queries, but in filter form it doesn't work. Let me verify if it works in real databases, if it does I will disable sql for this test, or do some workaround for derby. listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, D13101.2.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[
https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13778855#comment-13778855
]
Hive QA commented on HIVE-4888:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12605164/HIVE-4888.01.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 3163 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_partitioned
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynamic_partition_skip_default
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partcols1
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/913/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/913/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
listPartitionsByFilter doesn't support lt/gt/lte/gte
Key: HIVE-4888
URL: https://issues.apache.org/jira/browse/HIVE-4888
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: D13101.1.patch, D13101.2.patch, HIVE-4888.00.patch,
HIVE-4888.01.patch, HIVE-4888.on-top-of-4914.patch
Filter pushdown could be improved. Based on my experiments there's no
reasonable way to do it with DN 2.0, due to DN bug in substring and
Collection.get(int) not being implemented.
With version as low as 2.1 we can use values.get on partition to extract
values to compare to. Type compatibility is an issue, but is easy for strings
and integral values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13779461#comment-13779461 ] Sergey Shelukhin commented on HIVE-4888: All these tests have non-int values in int columns. For most of them the problem is solved by making it JDO-compatible by doing equal compare as a string, the same way JDO does. For other one I am looking. listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, D13101.2.patch, HIVE-4888.00.patch, HIVE-4888.01.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13778191#comment-13778191 ] Sergey Shelukhin commented on HIVE-4888: [~ashutoshc] this JIRA is now active :) listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, HIVE-4888.00.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13778241#comment-13778241 ] Ashutosh Chauhan commented on HIVE-4888: There are no test cases. Also, it seems you have hand modified a generated file(serdeConstants.java). listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: D13101.1.patch, HIVE-4888.00.patch, HIVE-4888.on-top-of-4914.patch Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4888) listPartitionsByFilter doesn't support lt/gt/lte/gte
[ https://issues.apache.org/jira/browse/HIVE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724158#comment-13724158 ] Sergey Shelukhin commented on HIVE-4888: There doesn't appear to be any equivalent of Long.parse in DN/JDO. Casts are not it. I am playing with writing a DN SQLMethod plugin, but it would only work if DN is backed by SQL store as far as I see. I will file DN jira. HIVE-4051 makes pushdown work for SQL, so I might end up doing HIVE-4914 instead and having server decide to do pushdown to SQL or no pushdown for JDOQL for those. Depends on how seamless the plugin would be. listPartitionsByFilter doesn't support lt/gt/lte/gte Key: HIVE-4888 URL: https://issues.apache.org/jira/browse/HIVE-4888 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Filter pushdown could be improved. Based on my experiments there's no reasonable way to do it with DN 2.0, due to DN bug in substring and Collection.get(int) not being implemented. With version as low as 2.1 we can use values.get on partition to extract values to compare to. Type compatibility is an issue, but is easy for strings and integral values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
