[jira] [Created] (HIVE-11593) Add aes_encrypt and aes_decrypt UDFs
Alexander Pivovarov created HIVE-11593: -- Summary: Add aes_encrypt and aes_decrypt UDFs Key: HIVE-11593 URL: https://issues.apache.org/jira/browse/HIVE-11593 Project: Hive Issue Type: New Feature Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov AES (Advanced Encryption Standard) algorithm. Oracle JRE supports AES-128 out of the box AES-192 and AES-256 are supported if Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files installed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] hive pull request: HIVE-11593 Add aes_encrypt and aes_decrypt UDFs
GitHub user apivovarov opened a pull request: https://github.com/apache/hive/pull/48 HIVE-11593 Add aes_encrypt and aes_decrypt UDFs You can merge this pull request into a Git repository by running: $ git pull https://github.com/apivovarov/hive AES Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/48.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #48 commit e33b81ad8a12ed2ffcbe79dc2d12bb4e07e44595 Author: Alexander Pivovarov apivova...@gmail.com Date: 2015-08-17T01:36:32Z HIVE-11593 Add aes_encrypt and aes_decrypt UDFs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HIVE-11594) Analyze Table For Columns cannot handle columns with embedded spaces
Gopal V created HIVE-11594:
--
Summary: Analyze Table For Columns cannot handle columns with
embedded spaces
Key: HIVE-11594
URL: https://issues.apache.org/jira/browse/HIVE-11594
Project: Hive
Issue Type: Bug
Components: Statistics
Reporter: Gopal V
{code}
create temporary table events(`user id` bigint, `user name` string);
explain analyze table events compute statistics for columns `user id`;
FAILED: SemanticException [Error 30009]: Encountered parse error while parsing
rewritten query
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Hive-0.14 - Build # 1047 - Still Failing
Changes for Build #1026 Changes for Build #1027 Changes for Build #1028 Changes for Build #1029 Changes for Build #1030 Changes for Build #1031 Changes for Build #1032 Changes for Build #1033 Changes for Build #1034 Changes for Build #1035 Changes for Build #1036 Changes for Build #1037 Changes for Build #1038 Changes for Build #1039 Changes for Build #1040 Changes for Build #1041 Changes for Build #1042 Changes for Build #1043 Changes for Build #1044 Changes for Build #1045 Changes for Build #1046 Changes for Build #1047 No tests ran. The Apache Jenkins build system has built Hive-0.14 (build #1047) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-0.14/1047/ to view the results.
[jira] [Created] (HIVE-11595) refactor ORC footer reading to make it usable from outside
Sergey Shelukhin created HIVE-11595: --- Summary: refactor ORC footer reading to make it usable from outside Key: HIVE-11595 URL: https://issues.apache.org/jira/browse/HIVE-11595 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin If ORC footer is read from cache, we want to parse it without having the reader, opening a file, etc. I thought it would be as simple as protobuf parseFrom bytes, but apparently there's bunch of stuff going on there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 37563: HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty conversion
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37563/#review95749
---
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
(line 443)
https://reviews.apache.org/r/37563/#comment150862
Is the conversion of extra fields to NULL done in convert(). i.e. how do we
handle a scenario where the input is a UNION with {int, string, boolean, char}
and output is a UNION with {string, int, boolean}. Is the above scenario valid
?
1. If this conversion is not possible, we should make sure that
inputTagsOIs and outputTagsOIs have the same size. i.e. we need to make sure
that there is an one to one mapping between inputTagsOIs and outputTagsOIs.
2. In the above example, if the conversion is valid; on the other hand, we
need to make sure that we infact return NULL when the user tries to call
convert on inputFieldTag 3.
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
(line 467)
https://reviews.apache.org/r/37563/#comment150863
Can you make sure that (Byte)inputFieldTag).intValue() is = 0
fieldConverters.size(). If not, you can either throw an exception or NULL based
on how you handle extra tags.
- Hari Sankar Sivarama Subramaniyan
On Aug. 18, 2015, 3:35 a.m., Swarnim Kulkarni wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37563/
---
(Updated Aug. 18, 2015, 3:35 a.m.)
Review request for hive and Hari Sankar Sivarama Subramaniyan.
Repository: hive-git
Description
---
HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty
conversion
Diffs
-
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
8ef8ce1736d50f0f9163cd5e3fd00ddd4bd810da
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableUnionObjectInspector.java
a64aee074d05a14c3c72079ff960039811936419
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardUnionObjectInspector.java
d1b11e82730e57f6894145478aae7c0c0c26e518
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java
11852833577149152cefaef50ee49328733c9dde
Diff: https://reviews.apache.org/r/37563/diff/
Testing
---
Unit tests added.
Thanks,
Swarnim Kulkarni
Re: Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- (Updated Aug. 18, 2015, 9:51 p.m.) Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
[jira] [Created] (HIVE-11596) nvl(x, y) throws NPE if type x and type y doesn't match, rather than throwing the meaningful error
Aihua Xu created HIVE-11596:
---
Summary: nvl(x, y) throws NPE if type x and type y doesn't match,
rather than throwing the meaningful error
Key: HIVE-11596
URL: https://issues.apache.org/jira/browse/HIVE-11596
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 2.0.0
Reporter: Aihua Xu
Assignee: Aihua Xu
{noformat}
create table test(key string);
select nvl(key, true) from test;
{noformat}
The query above will throw NPE rather than the meaningful error The first and
seconds arguments of function NLV should have the same type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 37563: HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty conversion
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37563/ --- (Updated Aug. 18, 2015, 8:27 p.m.) Review request for hive and Hari Sankar Sivarama Subramaniyan. Repository: hive-git Description --- HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty conversion Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 8ef8ce1736d50f0f9163cd5e3fd00ddd4bd810da serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableUnionObjectInspector.java a64aee074d05a14c3c72079ff960039811936419 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardUnionObjectInspector.java d1b11e82730e57f6894145478aae7c0c0c26e518 serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 11852833577149152cefaef50ee49328733c9dde Diff: https://reviews.apache.org/r/37563/diff/ Testing --- Unit tests added. Thanks, Swarnim Kulkarni
Re: Review Request 37563: HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty conversion
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37563/#review95776 --- Ship it! Ship It! - Hari Sankar Sivarama Subramaniyan On Aug. 18, 2015, 8:27 p.m., Swarnim Kulkarni wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37563/ --- (Updated Aug. 18, 2015, 8:27 p.m.) Review request for hive and Hari Sankar Sivarama Subramaniyan. Repository: hive-git Description --- HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty conversion Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 8ef8ce1736d50f0f9163cd5e3fd00ddd4bd810da serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableUnionObjectInspector.java a64aee074d05a14c3c72079ff960039811936419 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardUnionObjectInspector.java d1b11e82730e57f6894145478aae7c0c0c26e518 serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 11852833577149152cefaef50ee49328733c9dde Diff: https://reviews.apache.org/r/37563/diff/ Testing --- Unit tests added. Thanks, Swarnim Kulkarni
Request for Review Commit
Hi, would anyone mind taking a look at https://issues.apache.org/jira/browse/HIVE-8583 https://issues.apache.org/jira/browse/HIVE-8007 for me? I believe both have been ready for commit for a few months now. Thank you very much. Cheers, Lars
[jira] [Created] (HIVE-11598) Document Configuration for Parquet Files
Shannon Ladymon created HIVE-11598: -- Summary: Document Configuration for Parquet Files Key: HIVE-11598 URL: https://issues.apache.org/jira/browse/HIVE-11598 Project: Hive Issue Type: Task Components: Documentation Reporter: Shannon Ladymon In addition to Parquet's own documentation, there seems to be a need for documenting configuration properties for Parquet that can be set through Hive (see the [Hive user mailing list discussion on this | http://mail-archives.apache.org/mod_mbox/hive-user/201508.mbox/%3CCAKOFcwo7X0QZ-K1XJjzK0nNLN9fTkw-rAqnnwFkytA-kxNh6wA%40mail.gmail.com%3E]). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
On Aug. 18, 2015, 10:51 p.m., Szehon Ho wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java, line 90 https://reviews.apache.org/r/37591/diff/2/?file=1043884#file1043884line90 Seems a little risky to me, what if there are some bad GC pause? If it's fully-inited it should get notified , so how about a longer timeout? Szehon Ho wrote: Sorry, my question should be, why can't we sleep the thread, and wait for it to get notified? Jimmy Xiang wrote: We have a while loop, so it is fine to take longer than this much time to init. Here, wait() achieves the same effect as sleep and get notified. It's kind of sleep for up to that much time unless it is got notified earlier. Szehon Ho wrote: I mean, should we wait without a timeout then? Not sure if we will ever miss a notify? Jimmy Xiang wrote: I see. That should work too. With a timeout, so that we can periodically run a check and throw RuntimeException if things go wrong. I'm not so sure, I think it just does the same check in a while loop without any other exit condition, right? Interrupt exception will happen anyway regardless of the timeout. - Szehon --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/#review95789 --- On Aug. 18, 2015, 9:51 p.m., Jimmy Xiang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- (Updated Aug. 18, 2015, 9:51 p.m.) Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
Re: Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/#review95789 --- It makes sense, but I just had some basic questions. serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java (line 90) https://reviews.apache.org/r/37591/#comment150918 Seems a little risky to me, what if there are some bad GC pause? If it's fully-inited it should get notified , so how about a longer timeout? serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java (line 165) https://reviews.apache.org/r/37591/#comment150917 Very basic question I'm sure you thought of, why can't we synchronize the entire init() method? I assume this method will init all the children. Then we won't need the isFullyInit methods? - Szehon Ho On Aug. 18, 2015, 9:51 p.m., Jimmy Xiang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- (Updated Aug. 18, 2015, 9:51 p.m.) Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
Re: Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
On Aug. 18, 2015, 10:51 p.m., Szehon Ho wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java, line 165 https://reviews.apache.org/r/37591/diff/2/?file=1043885#file1043885line165 Very basic question I'm sure you thought of, why can't we synchronize the entire init() method? I assume this method will init all the children. Then we won't need the isFullyInit methods? Jimmy Xiang wrote: Since some type could refer to itself recursively, for example the MyStruct class in the test, if we synchronize the entire init() method without the isFullyInited method, we will run into deadlocks. Ah I see, that's unfortunate. On Aug. 18, 2015, 10:51 p.m., Szehon Ho wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java, line 90 https://reviews.apache.org/r/37591/diff/2/?file=1043884#file1043884line90 Seems a little risky to me, what if there are some bad GC pause? If it's fully-inited it should get notified , so how about a longer timeout? Szehon Ho wrote: Sorry, my question should be, why can't we sleep the thread, and wait for it to get notified? Jimmy Xiang wrote: We have a while loop, so it is fine to take longer than this much time to init. Here, wait() achieves the same effect as sleep and get notified. It's kind of sleep for up to that much time unless it is got notified earlier. I mean, should we wait without a timeout then? Not sure if we will ever miss a notify? - Szehon --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/#review95789 --- On Aug. 18, 2015, 9:51 p.m., Jimmy Xiang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- (Updated Aug. 18, 2015, 9:51 p.m.) Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
Re: Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
On Aug. 18, 2015, 10:51 p.m., Szehon Ho wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java, line 90 https://reviews.apache.org/r/37591/diff/2/?file=1043884#file1043884line90 Seems a little risky to me, what if there are some bad GC pause? If it's fully-inited it should get notified , so how about a longer timeout? Szehon Ho wrote: Sorry, my question should be, why can't we sleep the thread, and wait for it to get notified? Jimmy Xiang wrote: We have a while loop, so it is fine to take longer than this much time to init. Here, wait() achieves the same effect as sleep and get notified. It's kind of sleep for up to that much time unless it is got notified earlier. Szehon Ho wrote: I mean, should we wait without a timeout then? Not sure if we will ever miss a notify? I see. That should work too. With a timeout, so that we can periodically run a check and throw RuntimeException if things go wrong. - Jimmy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/#review95789 --- On Aug. 18, 2015, 9:51 p.m., Jimmy Xiang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- (Updated Aug. 18, 2015, 9:51 p.m.) Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
[jira] [Created] (HIVE-11597) [CBO new return path] Handling of strings of zero-length
Ashutosh Chauhan created HIVE-11597: --- Summary: [CBO new return path] Handling of strings of zero-length Key: HIVE-11597 URL: https://issues.apache.org/jira/browse/HIVE-11597 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Exposed by load_dyn_part14.q -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
On Aug. 18, 2015, 10:51 p.m., Szehon Ho wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java, line 90 https://reviews.apache.org/r/37591/diff/2/?file=1043884#file1043884line90 Seems a little risky to me, what if there are some bad GC pause? If it's fully-inited it should get notified , so how about a longer timeout? Sorry, my question should be, why can't we sleep the thread, and wait for it to get notified? - Szehon --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/#review95789 --- On Aug. 18, 2015, 9:51 p.m., Jimmy Xiang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- (Updated Aug. 18, 2015, 9:51 p.m.) Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
Re: Review Request 37591: HIVE-11586 ObjectInspectorFactory.getReflectionObjectInspector is not thread-safe
On Aug. 18, 2015, 10:51 p.m., Szehon Ho wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java, line 90 https://reviews.apache.org/r/37591/diff/2/?file=1043884#file1043884line90 Seems a little risky to me, what if there are some bad GC pause? If it's fully-inited it should get notified , so how about a longer timeout? Szehon Ho wrote: Sorry, my question should be, why can't we sleep the thread, and wait for it to get notified? We have a while loop, so it is fine to take longer than this much time to init. Here, wait() achieves the same effect as sleep and get notified. It's kind of sleep for up to that much time unless it is got notified earlier. On Aug. 18, 2015, 10:51 p.m., Szehon Ho wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java, line 165 https://reviews.apache.org/r/37591/diff/2/?file=1043885#file1043885line165 Very basic question I'm sure you thought of, why can't we synchronize the entire init() method? I assume this method will init all the children. Then we won't need the isFullyInit methods? Since some type could refer to itself recursively, for example the MyStruct class in the test, if we synchronize the entire init() method without the isFullyInited method, we will run into deadlocks. - Jimmy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/#review95789 --- On Aug. 18, 2015, 9:51 p.m., Jimmy Xiang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37591/ --- (Updated Aug. 18, 2015, 9:51 p.m.) Review request for hive. Bugs: HIVE-11586 https://issues.apache.org/jira/browse/HIVE-11586 Repository: hive-git Description --- Made ObjectInspectorFactory.getReflectionObjectInspector thread-safe. When it returns, the object inspector is fully initialized. Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 97bb715 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java 78e6066 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ThriftUnionObjectInspector.java 600abbb serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java e2408c6 Diff: https://reviews.apache.org/r/37591/diff/ Testing --- Unit test Thanks, Jimmy Xiang
[jira] [Created] (HIVE-11599) Add metastore command to dump it's configs
Eugene Koifman created HIVE-11599: - Summary: Add metastore command to dump it's configs Key: HIVE-11599 URL: https://issues.apache.org/jira/browse/HIVE-11599 Project: Hive Issue Type: Bug Components: HiveServer2, Metastore Affects Versions: 1.0.0 Reporter: Eugene Koifman We should have equivalent of Hive CLI set command on Metastore (and likely HS2) which can dump out all properties this particular process is running with. cc [~thejas] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Question about PredicateTransitivePropagate
For cases like: select * from t1 join t2 on t1.col = t2.col where t1.col = 1; Is rule PredicateTransitivePropagate supposed to propagate predicate t1.col = 1 to t2 via join condition t1.col = t2.col? Assuming so, I found that the predicate t1.col = 1 has not been pushed down to table scan of t1, thus PredicateTransitivePropagate wouldn't see the predicate. Then I tried to put PredicateTransitivePropagate after PredicatePushDown, I saw predicate t1.col = 1 was propagated to t2. Could anyone enlighten me? Thanks. Ruoxi SUN
Re: Review Request 37150: HIVE-11375 Broken processing of queries containing NOT (x IS NOT NULL and x 0)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37150/ --- (Updated Aug. 18, 2015, 3:03 p.m.) Review request for hive. Repository: hive-git Description --- HIVE-11375 Broken processing of queries containing NOT (x IS NOT NULL and x 0) Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 55ad0ce ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java e8df5d3 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPAnd.java db7fbac ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPEqual.java 428ced7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPEqualOrGreaterThan.java d9556cc ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPEqualOrLessThan.java 1d9eaf6 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPGreaterThan.java 8e1f2b1 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPLessThan.java 101b348 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNotEqual.java b5da57a ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNotNull.java d22b35d ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNull.java fe20e9a ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPOr.java 4160610 ql/src/test/queries/clientpositive/folder_predicate.q PRE-CREATION ql/src/test/results/clientpositive/annotate_stats_filter.q.out af1e1c3 ql/src/test/results/clientpositive/decimal_udf.q.out f36670c ql/src/test/results/clientpositive/filter_cond_pushdown.q.out e09057a ql/src/test/results/clientpositive/filter_join_breaktask.q.out 948ad0d ql/src/test/results/clientpositive/fold_when.q.out 21748da ql/src/test/results/clientpositive/folder_predicate.q.out PRE-CREATION ql/src/test/results/clientpositive/input_testxpath2.q.out d3a6f29 ql/src/test/results/clientpositive/list_bucket_query_oneskew_3.q.out fabd0d6 ql/src/test/results/clientpositive/rand_partitionpruner3.q.out ef8ee2f ql/src/test/results/clientpositive/select_unquote_not.q.out 64de433 ql/src/test/results/clientpositive/spark/filter_join_breaktask.q.out 28eeb0d ql/src/test/results/clientpositive/tez/filter_join_breaktask.q.out 06f1ca4 ql/src/test/results/clientpositive/tez/vector_decimal_udf.q.out b24ebcc ql/src/test/results/clientpositive/udf_isnull_isnotnull.q.out d459ce3 ql/src/test/results/clientpositive/udf_size.q.out efb06a1 Diff: https://reviews.apache.org/r/37150/diff/ Testing --- Thanks, Aihua Xu
Re: Review Request 37563: HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty conversion
On Aug. 18, 2015, 6:06 p.m., Hari Sankar Sivarama Subramaniyan wrote: serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java, line 467 https://reviews.apache.org/r/37563/diff/1/?file=1042755#file1042755line467 Can you make sure that (Byte)inputFieldTag).intValue() is = 0 fieldConverters.size(). If not, you can either throw an exception or NULL based on how you handle extra tags. Ah..good call. I think we only need to care about extra fields in the input as that is what we are parsing the tags from. I'll get this updated. - Swarnim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37563/#review95749 --- On Aug. 18, 2015, 3:35 a.m., Swarnim Kulkarni wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37563/ --- (Updated Aug. 18, 2015, 3:35 a.m.) Review request for hive and Hari Sankar Sivarama Subramaniyan. Repository: hive-git Description --- HIVE-10697 Fix for ObjectInspectorConvertors#UnionConvertor doing a faulty conversion Diffs - serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 8ef8ce1736d50f0f9163cd5e3fd00ddd4bd810da serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableUnionObjectInspector.java a64aee074d05a14c3c72079ff960039811936419 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardUnionObjectInspector.java d1b11e82730e57f6894145478aae7c0c0c26e518 serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 11852833577149152cefaef50ee49328733c9dde Diff: https://reviews.apache.org/r/37563/diff/ Testing --- Unit tests added. Thanks, Swarnim Kulkarni
