[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14158954#comment-14158954 ] Lefty Leverenz commented on HIVE-6148: -- By the time Hive 0.15 is released, this should be documented in the wiki: * [Hive HBase Integration | https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration] Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Labels: TODOC15 Fix For: 0.15.0 Attachments: HIVE-6148.1.patch.txt, HIVE-6148.2.patch.txt, HIVE-6148.3.patch.txt We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151846#comment-14151846 ] Swarnim Kulkarni commented on HIVE-6148: The failed test seems flaky and unrelated to my changes here. Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6148.1.patch.txt, HIVE-6148.2.patch.txt, HIVE-6148.3.patch.txt We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152287#comment-14152287 ] Brock Noland commented on HIVE-6148: +1 Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6148.1.patch.txt, HIVE-6148.2.patch.txt, HIVE-6148.3.patch.txt We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[
https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150139#comment-14150139
]
Hive QA commented on HIVE-6148:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12671376/HIVE-6148.3.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6357 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority2
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1001/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1001/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1001/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12671376
Support arbitrary structs stored in HBase
-
Key: HIVE-6148
URL: https://issues.apache.org/jira/browse/HIVE-6148
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.12.0
Reporter: Swarnim Kulkarni
Assignee: Swarnim Kulkarni
Attachments: HIVE-6148.1.patch.txt, HIVE-6148.2.patch.txt,
HIVE-6148.3.patch.txt
We should add support to be able to query arbitrary structs stored in HBase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147754#comment-14147754 ] Swarnim Kulkarni commented on HIVE-6148: The above test failure is unrelated to the change. Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6148.1.patch.txt, HIVE-6148.2.patch.txt We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[
https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148427#comment-14148427
]
Brock Noland commented on HIVE-6148:
Patch looks good and I am +1, but I still see:
{noformat}
@@ -1403,6 +1484,8 @@ private void deserializeAndSerializeHiveAvro(HBaseSerDe
serDe, Result r, Put p,
assertNotNull(fieldData);
assertEquals(expectedFieldsData[j], fieldData.toString().trim());
}
+
+SerDeUtils.getJSONString(row, soi);
// Now serialize
Put put = ((PutWritable) serDe.serialize(row, soi)).getPut();
{noformat}
which doesn't seem to make sense?
Support arbitrary structs stored in HBase
-
Key: HIVE-6148
URL: https://issues.apache.org/jira/browse/HIVE-6148
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.12.0
Reporter: Swarnim Kulkarni
Assignee: Swarnim Kulkarni
Attachments: HIVE-6148.1.patch.txt, HIVE-6148.2.patch.txt
We should add support to be able to query arbitrary structs stored in HBase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[
https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14146978#comment-14146978
]
Hive QA commented on HIVE-6148:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12670972/HIVE-6148.2.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6347 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/967/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/967/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-967/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12670972
Support arbitrary structs stored in HBase
-
Key: HIVE-6148
URL: https://issues.apache.org/jira/browse/HIVE-6148
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.12.0
Reporter: Swarnim Kulkarni
Assignee: Swarnim Kulkarni
Attachments: HIVE-6148.1.patch.txt, HIVE-6148.2.patch.txt
We should add support to be able to query arbitrary structs stored in HBase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141281#comment-14141281 ] Swarnim Kulkarni commented on HIVE-6148: [~brocknoland][~xuefuz] If you guys get a chance to review this, I would highly appreciate that. The above one test failure is unrelated to my patch. Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Attachments: HIVE-6148.1.patch.txt We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141309#comment-14141309 ] Brock Noland commented on HIVE-6148: Thank you! Comments on RB. Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6148.1.patch.txt We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[
https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135336#comment-14135336
]
Hive QA commented on HIVE-6148:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668872/HIVE-6148.1.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6277 tests executed
*Failed tests:*
{noformat}
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/818/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/818/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-818/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12668872
Support arbitrary structs stored in HBase
-
Key: HIVE-6148
URL: https://issues.apache.org/jira/browse/HIVE-6148
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.12.0
Reporter: Swarnim Kulkarni
Attachments: HIVE-6148.1.patch.txt
We should add support to be able to query arbitrary structs stored in HBase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134621#comment-14134621 ] Swarnim Kulkarni commented on HIVE-6148: RB: https://reviews.apache.org/r/25669 Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Attachments: HIVE-6148.1.patch.txt We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863520#comment-13863520 ] Sergey Shelukhin commented on HIVE-6148: Can you elaborate? HBase provides an option for structured keys w/type support and correct multi-field ordering (as of 0.96 IIRC, [~ndimiduk] might know more). Using these, if they fit the use case, might be nice for both sides (they are already there so no new code, and this will provide the real use case for them). We (ok, at least myself) can volunteer to fix things quickly to make it work :) Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[ https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863648#comment-13863648 ] Swarnim Kulkarni commented on HIVE-6148: Hi Sergey, This bug is similar to other two bugs[1][2] that I logged to get an out-of-the-box support for querying structs stored in HBase. The other two issues mainly deal with supporting protobufs, thrift and avros which can be deeply nested while this one was for simple structs that don't fall in either category (fix being pretty similar to one suggested in HIVE-2599). I kind of already have something which is working but I can completely throw it away if we can come up with a better strategy. :) [1] https://issues.apache.org/jira/browse/HIVE-3555 [2] https://issues.apache.org/jira/browse/HIVE-6147 Support arbitrary structs stored in HBase - Key: HIVE-6148 URL: https://issues.apache.org/jira/browse/HIVE-6148 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni We should add support to be able to query arbitrary structs stored in HBase. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6148) Support arbitrary structs stored in HBase
[
https://issues.apache.org/jira/browse/HIVE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863738#comment-13863738
]
Nick Dimiduk commented on HIVE-6148:
How is this not a duplicate of HIVE-3211?
I think it would be trivial to create Avro and Protobuf implementations of
[{{org.apache.hadoop.hbase.types.DataType}}|https://github.com/apache/hbase/blob/hbase-0.96.0/hbase-common/src/main/java/org/apache/hadoop/hbase/types/DataType.java].
The details will be similar to what's already provided in
[{{Struct}}|https://github.com/apache/hbase/blob/hbase-0.96.0/hbase-common/src/main/java/org/apache/hadoop/hbase/types/Struct.java]
(which I suggested you examine over in HIVE-2599). If you build on top of that
interface, all HBase users will benefit from from you code, not just Hive +
HBase users.
Support arbitrary structs stored in HBase
-
Key: HIVE-6148
URL: https://issues.apache.org/jira/browse/HIVE-6148
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.12.0
Reporter: Swarnim Kulkarni
We should add support to be able to query arbitrary structs stored in HBase.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
