[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions for equals
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13722009#comment-13722009
]
Ashutosh Chauhan commented on HIVE-2702:
I ran the tests on current patch on latest trunk. All tests passed except
alter_partition_coltype.q for which you just need to update .q.out file. Can
you also include that update in your next refresh of the patch?
listPartitionsByFilter only supports string partitions for equals
-
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Fix For: 0.12.0
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702.D11715.2.patch,
HIVE-2702.D11715.3.patch, HIVE-2702.D11847.1.patch, HIVE-2702.patch,
HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions for equals
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13721198#comment-13721198
]
Ashutosh Chauhan commented on HIVE-2702:
Do you want to improve the description of ticket ... something like
Enhance listPartitionsByFilter to add support for integral types both for
equality and non-equality
listPartitionsByFilter only supports string partitions for equals
-
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Fix For: 0.12.0
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702.D11715.2.patch,
HIVE-2702.D11715.3.patch, HIVE-2702.D11847.1.patch, HIVE-2702.patch,
HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions for equals
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13721209#comment-13721209
]
Phabricator commented on HIVE-2702:
---
ashutoshc has accepted the revision HIVE-2702 [jira] listPartitionsByFilter
only supports string partitions for equals.
+1 Looks good.
Few minor nits.
INLINE COMMENTS
metastore/src/java/org/apache/hadoop/hive/metastore/parser/Filter.g:160 Do u
want to name this as IntegralLiteral now ?
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java:312
Do you instead want to say in this TODO that this will be dealt in HIVE-4888
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:2188 Do you want to
say for 2nd TODO that this will be dealt in HIVE-4888
REVISION DETAIL
https://reviews.facebook.net/D11847
BRANCH
HIVE-2702-2
ARCANIST PROJECT
hive
To: JIRA, ashutoshc, sershe
listPartitionsByFilter only supports string partitions for equals
-
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Fix For: 0.12.0
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702.D11715.2.patch,
HIVE-2702.D11715.3.patch, HIVE-2702.D11847.1.patch, HIVE-2702.patch,
HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions for equals
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13716638#comment-13716638
]
Phabricator commented on HIVE-2702:
---
ashutoshc has accepted the revision HIVE-2702 [jira] listPartitionsByFilter
only supports string partitions.
+1.
Sergey, How did the tests go?
REVISION DETAIL
https://reviews.facebook.net/D11715
BRANCH
HIVE-2702
ARCANIST PROJECT
hive
To: JIRA, ashutoshc, sershe
listPartitionsByFilter only supports string partitions for equals
-
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Fix For: 0.12.0
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702.D11715.2.patch,
HIVE-2702.D11715.3.patch, HIVE-2702.patch, HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions for equals
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715607#comment-13715607
]
Phabricator commented on HIVE-2702:
---
ashutoshc has requested changes to the revision HIVE-2702 [jira]
listPartitionsByFilter only supports string partitions.
Some comments.
INLINE COMMENTS
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java:161
Its better to say numeric type here, since we can also support byte and short
(both of which are valid hive types) as well.
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java:262
As stated earlier, lets rename this method and variable as
doesOpSupportNumeric()
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java:272
Why are you checking value instanceof Long here ? This parsing is not done via
grammar described in Filter.g
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java:299
I presume this depends on DN upgrade, lets postpone it till than.
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:7699
HIVE-3059, I assume you want to say.
REVISION DETAIL
https://reviews.facebook.net/D11715
BRANCH
HIVE-2702
ARCANIST PROJECT
hive
To: JIRA, ashutoshc, sershe
listPartitionsByFilter only supports string partitions for equals
-
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Fix For: 0.12.0
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702.D11715.2.patch,
HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions for equals
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714187#comment-13714187
]
Sergey Shelukhin commented on HIVE-2702:
ran some perf tests, filter pushdown can cut about 300ms for getting partitions
on 16k-partition table on my machine, with standalone metastore on the same
machine (for equivalent queries e.g. n=0 vs n=0 and n=0)
listPartitionsByFilter only supports string partitions for equals
-
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Fix For: 0.12.0
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702.D11715.2.patch,
HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713123#comment-13713123
]
Sergey Shelukhin commented on HIVE-2702:
Actually, as a quick workaround I can change the part of HIVE-2249 that changes
the legitimately derived integer type to double if the other column is string.
I don't think it makes sense... I feel like an archeologist by now
listPartitionsByFilter only supports string partitions
--
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Fix For: 0.12.0
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13711847#comment-13711847
]
Sergey Shelukhin commented on HIVE-2702:
gt/lt/etc. hit HIVE-2609 unfortunately; I simplified valString to avoid nested
substrings but that didn't help either, and Collection get method is not
supported on DN 2.0.
Making blocked on DN upgrade
listPartitionsByFilter only supports string partitions
--
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Sergey Shelukhin
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch, HIVE-2702.D11715.1.patch, HIVE-2702-v0.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709315#comment-13709315
]
Sergey Shelukhin commented on HIVE-2702:
There hasn't been activity for a while, I will take over if you don't mind
listPartitionsByFilter only supports string partitions
--
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Aniket Mokashi
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629724#comment-13629724
]
cyril liao commented on HIVE-2702:
--
Drop partition get org.apache.hadoop.hive.ql.parse.SemanticException because of
this design.
Drop partition is a base function,why this design exist if a base function can
not work!
listPartitionsByFilter only supports string partitions
--
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Aniket Mokashi
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2702.D2043.1.patch,
HIVE-2702.1.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2702) listPartitionsByFilter only supports string partitions
[
https://issues.apache.org/jira/browse/HIVE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13438357#comment-13438357
]
Jakob Homan commented on HIVE-2702:
---
This is very limiting for HCat as it means HCat clients cannot access
non-string partitions from Pig or MR, whereas they are available via Hive
directly. The JDOQL that's used in generateJDOFilter() to generate the query
is not powerful enough to support stripping out the value, casting it to a
number, and then sorting it thusly. The best solution is probably to re-write
ObjectStore.listPartitionNamesByFilter() to not use the Partitions table (which
stores the values as PARTITION=FOO but rather take advantage of
PARTITION_KEY_VALS, which stores the partition values by themselves. Any
objections to this approach?
listPartitionsByFilter only supports string partitions
--
Key: HIVE-2702
URL: https://issues.apache.org/jira/browse/HIVE-2702
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.1
Reporter: Aniket Mokashi
Assignee: Aniket Mokashi
Attachments: HIVE-2702.1.patch, HIVE-2702.D2043.1.patch
listPartitionsByFilter supports only non-string partitions. This is because
its explicitly specified in generateJDOFilterOverPartitions in
ExpressionTree.java.
//Can only support partitions whose types are string
if( ! table.getPartitionKeys().get(partitionColumnIndex).
getType().equals(org.apache.hadoop.hive.serde.Constants.STRING_TYPE_NAME) ) {
throw new MetaException
(Filtering is supported only on partition keys of type string);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
