[jira] [Commented] (HIVE-6259) Support truncate for non-native tables
[
https://issues.apache.org/jira/browse/HIVE-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882642#comment-13882642
]
Hive QA commented on HIVE-6259:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625316/HIVE-6259.2.patch.txt
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1040/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1040/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625316
Support truncate for non-native tables
--
Key: HIVE-6259
URL: https://issues.apache.org/jira/browse/HIVE-6259
Project: Hive
Issue Type: Bug
Components: StorageHandler
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-6259.1.patch.txt, HIVE-6259.2.patch.txt
Tables on HBase might be truncated by similar method in HBaseShell.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: Review Request 16951: HIVE-6109: Support customized location for EXTERNAL tables created by Dynamic Partitioning
On Jan. 22, 2014, 10:25 p.m., Sushanth Sowmyan wrote: Hi, Sorry for the delay, I thought I'd published this review over the weekend, but reviewboard was unresponsive, and it looks like it didn't save. Thanks Sushanth for the review. On Jan. 22, 2014, 10:25 p.m., Sushanth Sowmyan wrote: hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java, line 73 https://reviews.apache.org/r/16951/diff/1/?file=424591#file424591line73 _DYN is already defined in FosterStorageHandler, needs to have one place where it's defined. I'm okay with it being defined here if the FosterStorageHandler constant is removed and references to that are changed to reference this. Will refactor it now. I didn't want to touch unrelated code in the first cut. On Jan. 22, 2014, 10:25 p.m., Sushanth Sowmyan wrote: hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java, line 272 https://reviews.apache.org/r/16951/diff/1/?file=424591#file424591line272 whitespace errors - git refers to a bunch of these through the patch when we try to apply, please correct for final patch upload. Will do. On Jan. 22, 2014, 10:25 p.m., Sushanth Sowmyan wrote: hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java, line 721 https://reviews.apache.org/r/16951/diff/1/?file=424591#file424591line721 A bit about code readability - if we add a special case, then it makes sense to add the special case as an else, rather than as an if - that way, the default behaviour is visible first, and then the special case - please swap this around so that this is a if (!customDynamicLocationUsed) structure. Fine, will do that. On Jan. 22, 2014, 10:25 p.m., Sushanth Sowmyan wrote: hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java, line 765 https://reviews.apache.org/r/16951/diff/1/?file=424591#file424591line765 This is now significant amount of code repetition from line 720-741 above, please see if we can refactor this into a separate method. I will see if it can be easily refactored into a private method. On Jan. 22, 2014, 10:25 p.m., Sushanth Sowmyan wrote: hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/OutputJobInfo.java, line 178 https://reviews.apache.org/r/16951/diff/1/?file=424594#file424594line178 This becomes the primary API point with this change, wherein, a user that is using HCatOutputFormat will generate an OutputJobInfo, and then call setCustomDynamicLocation on it. This is fine for M/R users of HCat, but is something that will wind up having to be implemented for each M/R user. It might have been better to define a constant in HCatConstants, say hcat.dynamic.partitioning.custom.pattern, and to use that as a JobInfo parameter. That makes it easier for other tools to integrate with this feature. For example, with your patch, we still do not support the ability for the HCatStorer from pig to be able to write to custom dynamic partitions, while we do want to keep feature parity where possible between HCatOutputFormat and HCatStorer. In fact, as a design goal for HCat, we're trying to move away from letting(requiring) users explicitly muck around with OutputJobInfo and InputJobInfo, and stick to static calls to HCatInputFormat/HCatOutputFormat. I would like to see this call be something the HCatOutputFormat automatically calls if a jobConf parameter(as above) is set. That way, we can solve pig compatibility as well easily. Thanks for this feedback Sushanth. I had realized that with the current patch, HCatStorer is still not covered (only M/R jobs are covered), and was thinking of exposing equivalent APIs in HCatStorer to achieve it. However your suggestion sounds cleaner to me. Will do that! - Satish --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16951/#review32561 --- On Jan. 16, 2014, 12:09 p.m., Satish Mittal wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16951/ --- (Updated Jan. 16, 2014, 12:09 p.m.) Review request for hive and Sushanth Sowmyan. Repository: hive-git Description --- - Attaching the patch that implements the functionality to support custom location for external tables in dynamic partitioning. Diffs - hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java a5ae1be hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java
[jira] [Commented] (HIVE-6313) Minimr tests in hadoop-1 hangs on shutdown
[
https://issues.apache.org/jira/browse/HIVE-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882732#comment-13882732
]
Hive QA commented on HIVE-6313:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625332/HIVE-6313.1.patch.txt
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1044/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1044/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625332
Minimr tests in hadoop-1 hangs on shutdown
--
Key: HIVE-6313
URL: https://issues.apache.org/jira/browse/HIVE-6313
Project: Hive
Issue Type: Bug
Components: Tests
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-6313.1.patch.txt
It takes minutes after all tests run waiting for all task trackers shutdown.
Just shutting down JobTracker after killing pending jobs seemed enough.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: [jira] [Commented] (HIVE-6311) Design a new logo?
The elephant-bee is cute and its color matches the Hadoop logo (almost). What's not to like? -- Lefty On Sun, Jan 26, 2014 at 1:21 PM, Edward Capriolo (JIRA) j...@apache.orgwrote: [ https://issues.apache.org/jira/browse/HIVE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882432#comment-13882432] Edward Capriolo commented on HIVE-6311: --- I really like the hive logo. I surely can be re-drawn high res etc, but fundamentally I like the elephant/bee hybrid. Design a new logo? -- Key: HIVE-6311 URL: https://issues.apache.org/jira/browse/HIVE-6311 Project: Hive Issue Type: Task Reporter: Brock Noland I have heard some folks saying we should create a new logo so I am creating a jira for their comment, -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6285) DESCRIBE EXTENDED shows numRows as zero
[ https://issues.apache.org/jira/browse/HIVE-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882776#comment-13882776 ] Lefty Leverenz commented on HIVE-6285: -- I documented this in the wiki, along with ANALYZE TABLE for gathering statistics in an existing table. Please review and correct any errors: [Describe Table/View/Column|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DescribeTable/View/Column]. DESCRIBE EXTENDED shows numRows as zero --- Key: HIVE-6285 URL: https://issues.apache.org/jira/browse/HIVE-6285 Project: Hive Issue Type: Bug Components: CLI, HiveServer2 Affects Versions: 0.12.0 Reporter: Haroon Running DESCRIBE EXTENDED always shows numRows as zero. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6290) Add support for hbase filters for composite keys
[ https://issues.apache.org/jira/browse/HIVE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882862#comment-13882862 ] Brock Noland commented on HIVE-6290: Hi Swarnim, Yes those tests are failing on trunk at present. There is work going on to fix them. Thank you!! Brock Add support for hbase filters for composite keys Key: HIVE-6290 URL: https://issues.apache.org/jira/browse/HIVE-6290 Project: Hive Issue Type: Sub-task Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6290.1.patch.txt Add support for filters to be provided via the composite key class -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congratulations, Sergey and Jason! --Xuefu On Mon, Jan 27, 2014 at 8:36 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl
[ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl
[jira] [Commented] (HIVE-6302) annotate_stats_*.q are failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882970#comment-13882970 ] Owen O'Malley commented on HIVE-6302: - [~navis] Sorry for breaking trunk. I'll be more careful. annotate_stats_*.q are failing on trunk --- Key: HIVE-6302 URL: https://issues.apache.org/jira/browse/HIVE-6302 Project: Hive Issue Type: Task Components: Tests Reporter: Navis Assignee: Navis Attachments: HIVE-6302.1.patch.txt I'm checking it out -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6310) Fix a few minimr test failures
[
https://issues.apache.org/jira/browse/HIVE-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882978#comment-13882978
]
Xuefu Zhang commented on HIVE-6310:
---
Test failures above seems strange. All of them passed when being manually
started. file_with_header_footer_negative.q seems suffering the same problem as
the previous patch was to address. Thus, the new patch, #1, includes fix for
that too.
Fix a few minimr test failures
--
Key: HIVE-6310
URL: https://issues.apache.org/jira/browse/HIVE-6310
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-6310.patch
These test cases are:
{code}
ql/src/test/queries/clientpositive/import_exported_table.q
ql/src/test/queries/clientpositive/load_hdfs_file_with_space_in_the_name.q
ql/src/test/queries/clientpositive/root_dir_external_table.q
{code}
They are failing because of existing hdfs:///tmp/test, possible left over by
other tests.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6310) Fix a few minimr test failures
[
https://issues.apache.org/jira/browse/HIVE-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6310:
--
Attachment: HIVE-6310.1.patch
Fix a few minimr test failures
--
Key: HIVE-6310
URL: https://issues.apache.org/jira/browse/HIVE-6310
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-6310.1.patch, HIVE-6310.patch
These test cases are:
{code}
ql/src/test/queries/clientpositive/import_exported_table.q
ql/src/test/queries/clientpositive/load_hdfs_file_with_space_in_the_name.q
ql/src/test/queries/clientpositive/root_dir_external_table.q
{code}
They are failing because of existing hdfs:///tmp/test, possible left over by
other tests.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6243) error in high-precision division for Decimal128
[ https://issues.apache.org/jira/browse/HIVE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882974#comment-13882974 ] Eric Hanson commented on HIVE-6243: --- All the failed tests named annotate_stats* are unrelated to this patch and were failing in other builds going on around the same time (e.g. the one for HIVE-6263). This patch is also unrelated to testTableCreateDrop. error in high-precision division for Decimal128 --- Key: HIVE-6243 URL: https://issues.apache.org/jira/browse/HIVE-6243 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-6243.01.patch, HIVE-6243.02.patch, divide-error.01.patch a = 213474114411690 b = 5062120663 a * b = 1080631725579042037750470 (a * b) / b == actual: 251599050984618 expected: 213474114411690 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congratulations Sergey and Jason, good job! Jarcec On Mon, Jan 27, 2014 at 08:36:37AM -0800, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl signature.asc Description: Digital signature
[jira] [Work started] (HIVE-6139) Implement vectorized decimal division and modulo
[ https://issues.apache.org/jira/browse/HIVE-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-6139 started by Eric Hanson. Implement vectorized decimal division and modulo Key: HIVE-6139 URL: https://issues.apache.org/jira/browse/HIVE-6139 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-6139.01.patch, HIVE-6139.02.patch, HIVE-6139.07.patch, HIVE-6139.07.patch, HIVE-6139.08.patch Support column-scalar, scalar-column, and column-column versions for division and modulo. Include unit tests. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6139) Implement vectorized decimal division and modulo
[ https://issues.apache.org/jira/browse/HIVE-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-6139: -- Status: Patch Available (was: In Progress) Implement vectorized decimal division and modulo Key: HIVE-6139 URL: https://issues.apache.org/jira/browse/HIVE-6139 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-6139.01.patch, HIVE-6139.02.patch, HIVE-6139.07.patch, HIVE-6139.07.patch, HIVE-6139.08.patch Support column-scalar, scalar-column, and column-column versions for division and modulo. Include unit tests. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6243) error in high-precision division for Decimal128
[ https://issues.apache.org/jira/browse/HIVE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-6243: -- Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Committed to trunk error in high-precision division for Decimal128 --- Key: HIVE-6243 URL: https://issues.apache.org/jira/browse/HIVE-6243 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Assignee: Eric Hanson Fix For: 0.13.0 Attachments: HIVE-6243.01.patch, HIVE-6243.02.patch, divide-error.01.patch a = 213474114411690 b = 5062120663 a * b = 1080631725579042037750470 (a * b) / b == actual: 251599050984618 expected: 213474114411690 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6139) Implement vectorized decimal division and modulo
[ https://issues.apache.org/jira/browse/HIVE-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883004#comment-13883004 ] Eric Hanson commented on HIVE-6139: --- Re-submitted patch to start unit tests Implement vectorized decimal division and modulo Key: HIVE-6139 URL: https://issues.apache.org/jira/browse/HIVE-6139 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-6139.01.patch, HIVE-6139.02.patch, HIVE-6139.07.patch, HIVE-6139.07.patch, HIVE-6139.08.patch Support column-scalar, scalar-column, and column-column versions for division and modulo. Include unit tests. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6297) [Refactor] Move new Auth Interface to common/
[ https://issues.apache.org/jira/browse/HIVE-6297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6297: --- Assignee: Ashutosh Chauhan Status: Patch Available (was: Open) [Refactor] Move new Auth Interface to common/ -- Key: HIVE-6297 URL: https://issues.apache.org/jira/browse/HIVE-6297 Project: Hive Issue Type: Task Components: Security Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6297.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5959) SQL std auth - bootstrap SUPERUSER, PUBLIC roles
[ https://issues.apache.org/jira/browse/HIVE-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-5959: --- Attachment: HIVE-5959.patch This patch adds root role at metastore startup time. SQL std auth - bootstrap SUPERUSER, PUBLIC roles Key: HIVE-5959 URL: https://issues.apache.org/jira/browse/HIVE-5959 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5959.patch Original Estimate: 72h Remaining Estimate: 72h SUPERUSER and PUBLIC are two roles that are always present, these need to be added automatically on metastore startup. This would be similar to creation of the default database from HMSHandler.init(). A config param in hive-site.xml will be used to specify the list of users who belong to the SUPERUSER role. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5181) RetryingRawStore should not retry on logical failures (e.g. from commit)
[ https://issues.apache.org/jira/browse/HIVE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883039#comment-13883039 ] Sergey Shelukhin commented on HIVE-5181: They are probably two different issues. RetryingRawStore should not retry on logical failures (e.g. from commit) Key: HIVE-5181 URL: https://issues.apache.org/jira/browse/HIVE-5181 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Prasad Mujumdar Priority: Minor Fix For: 0.13.0 Attachments: HIVE-5181.1.patch, HIVE-5181.3.patch RetryingRawStore retries calls. Some method (e.g. drop_table_core in HiveMetaStore) explicitly call openTransaction and commitTransaction on RawStore. When the commit call fails due to some real issue, it is retried, and instead of a real cause for failure one gets some bogus exception about transaction open count. I doesn't make sense to retry logical errors, especially not from commitTransaction. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5959) SQL std auth - bootstrap SUPERUSER, PUBLIC roles
[ https://issues.apache.org/jira/browse/HIVE-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-5959: --- Assignee: Ashutosh Chauhan Status: Patch Available (was: Open) SQL std auth - bootstrap SUPERUSER, PUBLIC roles Key: HIVE-5959 URL: https://issues.apache.org/jira/browse/HIVE-5959 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Attachments: HIVE-5959.patch Original Estimate: 72h Remaining Estimate: 72h SUPERUSER and PUBLIC are two roles that are always present, these need to be added automatically on metastore startup. This would be similar to creation of the default database from HMSHandler.init(). A config param in hive-site.xml will be used to specify the list of users who belong to the SUPERUSER role. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Thanks guys! On Mon, Jan 27, 2014 at 9:24 AM, Jarek Jarcec Cecho jar...@apache.orgwrote: Congratulations Sergey and Jason, good job! Jarcec On Mon, Jan 27, 2014 at 08:36:37AM -0800, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congrats!! Sergey and Jason.. Thanks Prasanth Jayachandran On Jan 27, 2014, at 10:19 AM, Sergey Shelukhin ser...@hortonworks.com wrote: Thanks guys! On Mon, Jan 27, 2014 at 9:24 AM, Jarek Jarcec Cecho jar...@apache.org wrote: Congratulations Sergey and Jason, good job! Jarcec On Mon, Jan 27, 2014 at 08:36:37AM -0800, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-5814) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat
[ https://issues.apache.org/jira/browse/HIVE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-5814: - Status: Open (was: Patch Available) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat - Key: HIVE-5814 URL: https://issues.apache.org/jira/browse/HIVE-5814 Project: Hive Issue Type: New Feature Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-5814 HCat-Pig type mapping.pdf, HIVE-5814.patch Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well.Also note that CHAR was added recently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6251) Add ability to specify delimiter in HCatalog Java API to create tables - HCatCreateTableDesc
[
https://issues.apache.org/jira/browse/HIVE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-6251:
-
Status: Open (was: Patch Available)
Add ability to specify delimiter in HCatalog Java API to create tables -
HCatCreateTableDesc
Key: HIVE-6251
URL: https://issues.apache.org/jira/browse/HIVE-6251
Project: Hive
Issue Type: Bug
Components: HCatalog, WebHCat
Affects Versions: 0.13.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-6251.patch
Per
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTablerow_format,
the following is supported when creating a table.
{code}
: DELIMITED [FIELDS TERMINATED BY char [ESCAPED BY char]] [COLLECTION ITEMS
TERMINATED BY char]
[MAP KEYS TERMINATED BY char] [LINES TERMINATED BY char]
[NULL DEFINED AS char] (Note: Only available starting with Hive 0.13)
| SERDE serde_name [WITH SERDEPROPERTIES (property_name=property_value,
property_name=property_value, ...)]
{code}
Need to add support for specifying 4 delimiters plus escape and NULL in
HCatCreateTableDesc. create(String dbName, String tableName,
ListHCatFieldSchema columns) API.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-5814) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat
[ https://issues.apache.org/jira/browse/HIVE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-5814: - Status: Patch Available (was: Open) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat - Key: HIVE-5814 URL: https://issues.apache.org/jira/browse/HIVE-5814 Project: Hive Issue Type: New Feature Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-5814 HCat-Pig type mapping.pdf, HIVE-5814.patch Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well.Also note that CHAR was added recently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6251) Add ability to specify delimiter in HCatalog Java API to create tables - HCatCreateTableDesc
[
https://issues.apache.org/jira/browse/HIVE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-6251:
-
Status: Patch Available (was: Open)
Add ability to specify delimiter in HCatalog Java API to create tables -
HCatCreateTableDesc
Key: HIVE-6251
URL: https://issues.apache.org/jira/browse/HIVE-6251
Project: Hive
Issue Type: Bug
Components: HCatalog, WebHCat
Affects Versions: 0.13.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-6251.patch
Per
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTablerow_format,
the following is supported when creating a table.
{code}
: DELIMITED [FIELDS TERMINATED BY char [ESCAPED BY char]] [COLLECTION ITEMS
TERMINATED BY char]
[MAP KEYS TERMINATED BY char] [LINES TERMINATED BY char]
[NULL DEFINED AS char] (Note: Only available starting with Hive 0.13)
| SERDE serde_name [WITH SERDEPROPERTIES (property_name=property_value,
property_name=property_value, ...)]
{code}
Need to add support for specifying 4 delimiters plus escape and NULL in
HCatCreateTableDesc. create(String dbName, String tableName,
ListHCatFieldSchema columns) API.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congratulations Sergey and Jason! Thanks, Gunther. On Mon, Jan 27, 2014 at 10:20 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congrats!! Sergey and Jason.. Thanks Prasanth Jayachandran On Jan 27, 2014, at 10:19 AM, Sergey Shelukhin ser...@hortonworks.com wrote: Thanks guys! On Mon, Jan 27, 2014 at 9:24 AM, Jarek Jarcec Cecho jar...@apache.org wrote: Congratulations Sergey and Jason, good job! Jarcec On Mon, Jan 27, 2014 at 08:36:37AM -0800, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-6315) MetaStoreDirectSql ctor should not throw
Sergey Shelukhin created HIVE-6315: -- Summary: MetaStoreDirectSql ctor should not throw Key: HIVE-6315 URL: https://issues.apache.org/jira/browse/HIVE-6315 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor There are unprotected places in the ctor that may throw in some rare circumstances, it should never throw but rather self-disable. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congrats Sergey and Jason! Thanks Vikram. On Jan 27, 2014, at 8:36 AM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-6316) Document support for new types in HCat
Eugene Koifman created HIVE-6316: Summary: Document support for new types in HCat Key: HIVE-6316 URL: https://issues.apache.org/jira/browse/HIVE-6316 Project: Hive Issue Type: Sub-task Components: Documentation, HCatalog Affects Versions: 0.13.0 Reporter: Eugene Koifman HIVE-5814 added support for new types in HCat. The PDF file in that bug explains exactly how these map to Pig types. This should be added to the Wiki somewhere (probably here https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore). In particular it should be highlighted that copying data from Hive TIMESTAMP to Pig DATETIME, any 'nanos' in the timestamp will be lost. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 17005: Vectorized reader for DECIMAL datatype for ORC format.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17005/#review32855 --- Please see my comment about the scale. Otherwise, it looks good to me. ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java https://reviews.apache.org/r/17005/#comment61854 For safety, you could put a check here and throw an exception if scratchScaleVector.vector[I] is not equal to result.vector[I].scale. Alternatively, you could make the value NULL if that is the case, and maybe log an error. - Eric Hanson On Jan. 24, 2014, 10:28 p.m., Jitendra Pandey wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17005/ --- (Updated Jan. 24, 2014, 10:28 p.m.) Review request for hive and Eric Hanson. Bugs: HIVE-6178 https://issues.apache.org/jira/browse/HIVE-6178 Repository: hive-git Description --- vectorized reader for DECIMAL datatype for ORC format. Diffs - common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 3939511 common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java d71ebb3 common/src/test/org/apache/hadoop/hive/common/type/TestUnsignedInt128.java fbb2aa0 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java 23564bb ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 0df82b9 ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java 0d5b7ff Diff: https://reviews.apache.org/r/17005/diff/ Testing --- Thanks, Jitendra Pandey
[jira] [Updated] (HIVE-6315) MetaStoreDirectSql ctor should not throw
[ https://issues.apache.org/jira/browse/HIVE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6315: --- Attachment: HIVE-6315.patch Simple patch. Also removed outdated todo that was already addressed MetaStoreDirectSql ctor should not throw Key: HIVE-6315 URL: https://issues.apache.org/jira/browse/HIVE-6315 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HIVE-6315.patch There are unprotected places in the ctor that may throw in some rare circumstances, it should never throw but rather self-disable. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6315) MetaStoreDirectSql ctor should not throw
[ https://issues.apache.org/jira/browse/HIVE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6315: --- Status: Patch Available (was: Open) MetaStoreDirectSql ctor should not throw Key: HIVE-6315 URL: https://issues.apache.org/jira/browse/HIVE-6315 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HIVE-6315.patch There are unprotected places in the ctor that may throw in some rare circumstances, it should never throw but rather self-disable. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congrats Jason and Sergey! Well deserved! Looking forward to your help in getting the patch available counts down (its at 225 now)! On Mon, Jan 27, 2014 at 10:55 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Sergey and Jason! --Vaibhav On Mon, Jan 27, 2014 at 10:47 AM, Vikram Dixit vik...@hortonworks.com wrote: Congrats Sergey and Jason! Thanks Vikram. On Jan 27, 2014, at 8:36 AM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congrats Sergey and Jason! --Vaibhav On Mon, Jan 27, 2014 at 10:47 AM, Vikram Dixit vik...@hortonworks.comwrote: Congrats Sergey and Jason! Thanks Vikram. On Jan 27, 2014, at 8:36 AM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congratulations Sergey and Jason! On Mon, Jan 27, 2014 at 10:58 AM, Thejas Nair the...@hortonworks.comwrote: Congrats Jason and Sergey! Well deserved! Looking forward to your help in getting the patch available counts down (its at 225 now)! On Mon, Jan 27, 2014 at 10:55 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Sergey and Jason! --Vaibhav On Mon, Jan 27, 2014 at 10:47 AM, Vikram Dixit vik...@hortonworks.com wrote: Congrats Sergey and Jason! Thanks Vikram. On Jan 27, 2014, at 8:36 AM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6315) MetaStoreDirectSql ctor should not throw
[ https://issues.apache.org/jira/browse/HIVE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883136#comment-13883136 ] Roshan Naik commented on HIVE-6315: --- I see the following exception with this patch applied: org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The class org.apache.hadoop.hive.metastore.model.MVersionTable is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:380) at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732) at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752) at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6025) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:5935) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:5913) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:122) at com.sun.proxy.$Proxy7.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:389) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:427) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:314) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:274) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.init(RetryingHMSHandler.java:54) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59) at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4175) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.init(HiveMetaStoreClient.java:115) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.init(HiveMetaStoreClient.java:98) at org.apache.hive.streaming.TestStreaming.setup(TestStreaming.java:45) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:24) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runner.JUnitCore.run(JUnitCore.java:157) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77) at
[jira] [Commented] (HIVE-5814) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat
[ https://issues.apache.org/jira/browse/HIVE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883139#comment-13883139 ] Sushanth Sowmyan commented on HIVE-5814: The patch looks good from HCat's perspective. It does not apply cleanly on trunk right now, but from the intent, I'm good on this. If you can update the patch and set it to patch available again (and if the precommit tests are working again), we can get the precommit tests running on this. Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat - Key: HIVE-5814 URL: https://issues.apache.org/jira/browse/HIVE-5814 Project: Hive Issue Type: New Feature Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-5814 HCat-Pig type mapping.pdf, HIVE-5814.patch Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well.Also note that CHAR was added recently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6310) Fix a few minimr test failures
[
https://issues.apache.org/jira/browse/HIVE-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883140#comment-13883140
]
Hive QA commented on HIVE-6310:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625381/HIVE-6310.1.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 4961 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1045/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1045/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625381
Fix a few minimr test failures
--
Key: HIVE-6310
URL: https://issues.apache.org/jira/browse/HIVE-6310
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-6310.1.patch, HIVE-6310.patch
These test cases are:
{code}
ql/src/test/queries/clientpositive/import_exported_table.q
ql/src/test/queries/clientpositive/load_hdfs_file_with_space_in_the_name.q
ql/src/test/queries/clientpositive/root_dir_external_table.q
{code}
They are failing because of existing hdfs:///tmp/test, possible left over by
other tests.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6315) MetaStoreDirectSql ctor should not throw
[ https://issues.apache.org/jira/browse/HIVE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883167#comment-13883167 ] Sergey Shelukhin commented on HIVE-6315: This is not related to this patch. This patch makes some code not throw, that's it. Is it on the same setup where the ctor originally failed? Maybe the setup is broken and JDO/DataNucleus don't work at all? Is this on Hive trunk, or does it have changes on top? MetaStoreDirectSql ctor should not throw Key: HIVE-6315 URL: https://issues.apache.org/jira/browse/HIVE-6315 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HIVE-6315.patch There are unprotected places in the ctor that may throw in some rare circumstances, it should never throw but rather self-disable. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Thanks everyone! Jason On Jan 27, 2014, at 11:01 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congratulations Sergey and Jason! On Mon, Jan 27, 2014 at 10:58 AM, Thejas Nair the...@hortonworks.com wrote: Congrats Jason and Sergey! Well deserved! Looking forward to your help in getting the patch available counts down (its at 225 now)! On Mon, Jan 27, 2014 at 10:55 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Sergey and Jason! --Vaibhav On Mon, Jan 27, 2014 at 10:47 AM, Vikram Dixit vik...@hortonworks.com wrote: Congrats Sergey and Jason! Thanks Vikram. On Jan 27, 2014, at 8:36 AM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Review Request 17262: HIVE-6246: Sign(a) UDF is not supported for decimal type
On Jan. 23, 2014, 10:57 p.m., Mohammad Islam wrote: Overall looks good. Is it possible to add a .q test or append to an existing .q test? Xuefu Zhang wrote: Actually unit test such as that provided is preferrable. Additional .q test only prelongs the build process while not providing much value for this case. ok with me. I agree the build-timing is important. but, in some instances,i found people prefer .q test. On Jan. 23, 2014, 10:57 p.m., Mohammad Islam wrote: ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFSign.java, line 31 https://reviews.apache.org/r/17262/diff/1/?file=436440#file436440line31 Minor issue: does testByte() is a good name? Xuefu Zhang wrote: A result of CP. Will change it. sounds good. - Mohammad --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17262/#review32677 --- On Jan. 23, 2014, 8:57 p.m., Xuefu Zhang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17262/ --- (Updated Jan. 23, 2014, 8:57 p.m.) Review request for hive. Bugs: HIVE-6246 https://issues.apache.org/jira/browse/HIVE-6246 Repository: hive-git Description --- Please see the JIRA description. It's believed that this has nevered worked. Added a method in UDFSign class to handle Decimal data type to make it work. This method returns INT instead of doulbe to be inline with other data bases. Diffs - common/src/java/org/apache/hadoop/hive/common/type/HiveDecimal.java 729908a ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSign.java 0fef283 ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFSign.java PRE-CREATION Diff: https://reviews.apache.org/r/17262/diff/ Testing --- Unit test is added. Thanks, Xuefu Zhang
[jira] [Created] (HIVE-6317) Add eclipse code formatter to hive projects
Swarnim Kulkarni created HIVE-6317: -- Summary: Add eclipse code formatter to hive projects Key: HIVE-6317 URL: https://issues.apache.org/jira/browse/HIVE-6317 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Currently on hive trunk, it seems like the eclipse formatter doesn't get automatically imported(it used to happen sometime ago). We should probably fix that so all changes going forward are formatted consistently according to this formatter. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 17162: HIVE-6157 Fetching column stats slow
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17162/#review32861
---
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
https://reviews.apache.org/r/17162/#comment61867
are there 12 columns in column stats table? can 12 be made as a static
const?
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
https://reviews.apache.org/r/17162/#comment61870
Is there any reason why this additional map conversion is needed? If
neededColumns is not empty then it can be returned else iterate the schema and
ignore the virtual columns.
if(neededColumns.isEmpty()) {
return list of non-hidden/vitual columns from schema;
} else {
return neededColumns;
}
- Prasanth_J
On Jan. 23, 2014, 7:53 p.m., Sergey Shelukhin wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17162/
---
(Updated Jan. 23, 2014, 7:53 p.m.)
Review request for hive and Gunther Hagleitner.
Repository: hive-git
Description
---
see jira
Diffs
-
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
1b01238
metastore/if/hive_metastore.thrift e4e816d
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
58f9957
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
ed05790
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
4288781
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
16f43e9
metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
794fadd
metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java f5ea2ef
metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java
PRE-CREATION
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
003dc9c
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
6dd0852
metastore/src/test/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java
c683fc9
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java d32deea
ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java 608bef2
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java b815ea2
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 384b49e
ql/src/test/results/clientpositive/metadataonly1.q.out 3500fd2
Diff: https://reviews.apache.org/r/17162/diff/
Testing
---
Thanks,
Sergey Shelukhin
[jira] [Commented] (HIVE-6157) Fetching column stats slower than the 101 during rush hour
[ https://issues.apache.org/jira/browse/HIVE-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883216#comment-13883216 ] Prasanth J commented on HIVE-6157: -- Mostly looks good. Left minor comments on RB. Fetching column stats slower than the 101 during rush hour -- Key: HIVE-6157 URL: https://issues.apache.org/jira/browse/HIVE-6157 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Gunther Hagleitner Assignee: Sergey Shelukhin Attachments: HIVE-6157.01.patch, HIVE-6157.01.patch, HIVE-6157.03.patch, HIVE-6157.03.patch, HIVE-6157.nogen.patch, HIVE-6157.nogen.patch, HIVE-6157.prelim.patch hive.stats.fetch.column.stats controls whether the column stats for a table are fetched during explain (in Tez: during query planning). On my setup (1 table 4000 partitions, 24 columns) the time spent in semantic analyze goes from ~1 second to ~66 seconds when turning the flag on. 65 seconds spent fetching column stats... The reason is probably that the APIs force you to make separate metastore calls for each column in each partition. That's probably the first thing that has to change. The question is if in addition to that we need to cache this in the client or store the stats as a single blob in the database to further cut down on the time. However, the way it stands right now column stats seem unusable. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6122) Implement show grant on resource
[
https://issues.apache.org/jira/browse/HIVE-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883219#comment-13883219
]
Ashutosh Chauhan commented on HIVE-6122:
Thinking more on this, I think its fine to have this syntax, since this part of
syntax is not part of standard anyways. Also, I mistakenly thought patch is
introducing new keyword 'ALL' but 'ALL' is already a keyword both in Hive and
in standard, so we should be good there.
+1
Implement show grant on resource
--
Key: HIVE-6122
URL: https://issues.apache.org/jira/browse/HIVE-6122
Project: Hive
Issue Type: Improvement
Components: Authorization
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-6122.1.patch.txt, HIVE-6122.2.patch.txt,
HIVE-6122.3.patch.txt
Currently, hive shows privileges owned by a principal. Reverse API is also
needed, which shows all principals for a resource.
{noformat}
show grant user hive_test_user on database default;
show grant user hive_test_user on table dummy;
show grant user hive_test_user on all;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6122) Implement show grant on resource
[
https://issues.apache.org/jira/browse/HIVE-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883224#comment-13883224
]
Ashutosh Chauhan commented on HIVE-6122:
Unfortunately, patch has gone stale. Navis, can you rebase it, lets get this in.
Implement show grant on resource
--
Key: HIVE-6122
URL: https://issues.apache.org/jira/browse/HIVE-6122
Project: Hive
Issue Type: Improvement
Components: Authorization
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-6122.1.patch.txt, HIVE-6122.2.patch.txt,
HIVE-6122.3.patch.txt
Currently, hive shows privileges owned by a principal. Reverse API is also
needed, which shows all principals for a resource.
{noformat}
show grant user hive_test_user on database default;
show grant user hive_test_user on table dummy;
show grant user hive_test_user on all;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congrats Jason and Sergey! On Mon, Jan 27, 2014 at 11:41 AM, Jason Dere jd...@hortonworks.com wrote: Thanks everyone! Jason On Jan 27, 2014, at 11:01 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congratulations Sergey and Jason! On Mon, Jan 27, 2014 at 10:58 AM, Thejas Nair the...@hortonworks.comwrote: Congrats Jason and Sergey! Well deserved! Looking forward to your help in getting the patch available counts down (its at 225 now)! On Mon, Jan 27, 2014 at 10:55 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Sergey and Jason! --Vaibhav On Mon, Jan 27, 2014 at 10:47 AM, Vikram Dixit vik...@hortonworks.com wrote: Congrats Sergey and Jason! Thanks Vikram. On Jan 27, 2014, at 8:36 AM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Review Request 17262: HIVE-6246: Sign(a) UDF is not supported for decimal type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17262/#review32876 --- ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSign.java https://reviews.apache.org/r/17262/#comment61871 Nit: Add Javadoc. ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSign.java https://reviews.apache.org/r/17262/#comment61872 Nit: Add javadoc to this new method. - Swarnim Kulkarni On Jan. 23, 2014, 8:57 p.m., Xuefu Zhang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17262/ --- (Updated Jan. 23, 2014, 8:57 p.m.) Review request for hive. Bugs: HIVE-6246 https://issues.apache.org/jira/browse/HIVE-6246 Repository: hive-git Description --- Please see the JIRA description. It's believed that this has nevered worked. Added a method in UDFSign class to handle Decimal data type to make it work. This method returns INT instead of doulbe to be inline with other data bases. Diffs - common/src/java/org/apache/hadoop/hive/common/type/HiveDecimal.java 729908a ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSign.java 0fef283 ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFSign.java PRE-CREATION Diff: https://reviews.apache.org/r/17262/diff/ Testing --- Unit test is added. Thanks, Xuefu Zhang
[jira] [Commented] (HIVE-6246) Sign(a) UDF is not supported for decimal type
[ https://issues.apache.org/jira/browse/HIVE-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883248#comment-13883248 ] Swarnim Kulkarni commented on HIVE-6246: Non committer +1 Sign(a) UDF is not supported for decimal type - Key: HIVE-6246 URL: https://issues.apache.org/jira/browse/HIVE-6246 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.12.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-6246.patch java.sql.SQLException: Error while compiling statement: FAILED: SemanticException [Error 10014]: Line 1:86 Wrong arguments 'a': No matching method for class org.apache.hadoop.hive.ql.udf.UDFSign with (decimal(38,10)). Possible choices: _FUNC_(double) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5728) Make ORC InputFormat/OutputFormat usable outside Hive
[ https://issues.apache.org/jira/browse/HIVE-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5728: - Status: Open (was: Patch Available) Make ORC InputFormat/OutputFormat usable outside Hive - Key: HIVE-5728 URL: https://issues.apache.org/jira/browse/HIVE-5728 Project: Hive Issue Type: Improvement Components: File Formats Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5728-1.patch, HIVE-5728-10.patch, HIVE-5728-2.patch, HIVE-5728-3.patch, HIVE-5728-4.patch, HIVE-5728-5.patch, HIVE-5728-6.patch, HIVE-5728-7.patch, HIVE-5728-8.patch, HIVE-5728-9.patch, HIVE-5728.10.patch ORC InputFormat/OutputFormat is currently not usable outside Hive. There are several issues need to solve: 1. Several class is not public, eg: OrcStruct 2. There is no InputFormat/OutputFormat for new api (Some tools such as Pig need new api) 3. Has no way to push WriteOption to OutputFormat outside Hive -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5728) Make ORC InputFormat/OutputFormat usable outside Hive
[ https://issues.apache.org/jira/browse/HIVE-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5728: - Attachment: HIVE-5728.10.patch Rename the patch for jenkins pickup. Make ORC InputFormat/OutputFormat usable outside Hive - Key: HIVE-5728 URL: https://issues.apache.org/jira/browse/HIVE-5728 Project: Hive Issue Type: Improvement Components: File Formats Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5728-1.patch, HIVE-5728-10.patch, HIVE-5728-2.patch, HIVE-5728-3.patch, HIVE-5728-4.patch, HIVE-5728-5.patch, HIVE-5728-6.patch, HIVE-5728-7.patch, HIVE-5728-8.patch, HIVE-5728-9.patch, HIVE-5728.10.patch ORC InputFormat/OutputFormat is currently not usable outside Hive. There are several issues need to solve: 1. Several class is not public, eg: OrcStruct 2. There is no InputFormat/OutputFormat for new api (Some tools such as Pig need new api) 3. Has no way to push WriteOption to OutputFormat outside Hive -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5728) Make ORC InputFormat/OutputFormat usable outside Hive
[ https://issues.apache.org/jira/browse/HIVE-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5728: - Status: Patch Available (was: Open) Make ORC InputFormat/OutputFormat usable outside Hive - Key: HIVE-5728 URL: https://issues.apache.org/jira/browse/HIVE-5728 Project: Hive Issue Type: Improvement Components: File Formats Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5728-1.patch, HIVE-5728-10.patch, HIVE-5728-2.patch, HIVE-5728-3.patch, HIVE-5728-4.patch, HIVE-5728-5.patch, HIVE-5728-6.patch, HIVE-5728-7.patch, HIVE-5728-8.patch, HIVE-5728-9.patch, HIVE-5728.10.patch ORC InputFormat/OutputFormat is currently not usable outside Hive. There are several issues need to solve: 1. Several class is not public, eg: OrcStruct 2. There is no InputFormat/OutputFormat for new api (Some tools such as Pig need new api) 3. Has no way to push WriteOption to OutputFormat outside Hive -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6300) Add documentation for stats configs to hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-6300: - Attachment: HIVE-6300.2.patch [~leftylev] Thanks for pointing out the nitty-gritty mistakes. That was really usefull. I removed the hive config hive.stats.avg.row.size as it is no longer required. Average row size will be computed from data size and schema in very stages of operator tree (TableScanOperator), so it should be available to all operators (including LIMIT). Can you please skim through it again to see if it looks good? Add documentation for stats configs to hive-default.xml.template Key: HIVE-6300 URL: https://issues.apache.org/jira/browse/HIVE-6300 Project: Hive Issue Type: Sub-task Components: Query Processor, Statistics Affects Versions: 0.13.0 Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Fix For: 0.13.0 Attachments: HIVE-6300.1.patch, HIVE-6300.2.patch Add documentation for the following configs hive.stats.max.variable.length hive.stats.list.num.entries hive.stats.map.num.entries hive.stats.map.parallelism hive.stats.fetch.column.stats hive.stats.avg.row.size hive.stats.join.factor hive.stats.deserialization.factor hive.stats.fetch.partition.stats -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6300) Add documentation for stats configs to hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-6300: - Status: Patch Available (was: Open) Making it as patch available for HIVE QA to pickup. The new patch removed a config. So needs precommit tests. Add documentation for stats configs to hive-default.xml.template Key: HIVE-6300 URL: https://issues.apache.org/jira/browse/HIVE-6300 Project: Hive Issue Type: Sub-task Components: Query Processor, Statistics Affects Versions: 0.13.0 Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Fix For: 0.13.0 Attachments: HIVE-6300.1.patch, HIVE-6300.2.patch Add documentation for the following configs hive.stats.max.variable.length hive.stats.list.num.entries hive.stats.map.num.entries hive.stats.map.parallelism hive.stats.fetch.column.stats hive.stats.avg.row.size hive.stats.join.factor hive.stats.deserialization.factor hive.stats.fetch.partition.stats -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Review Request 17420: HiveServer2 JDBC SSL binary client transport should not call a TTransport#open
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17420/ --- Review request for hive and Thejas Nair. Bugs: HIVE-6249 https://issues.apache.org/jira/browse/HIVE-6249 Repository: hive-git Description --- Check https://issues.apache.org/jira/browse/HIVE-6249 Diffs - jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java aabb5cb Diff: https://reviews.apache.org/r/17420/diff/ Testing --- Thanks, Vaibhav Gumashta
[jira] [Commented] (HIVE-6249) HiveServer2 JDBC SSL binary client transport should not call a TTransport#open
[ https://issues.apache.org/jira/browse/HIVE-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883316#comment-13883316 ] Vaibhav Gumashta commented on HIVE-6249: Review board: https://reviews.apache.org/r/17420/ HiveServer2 JDBC SSL binary client transport should not call a TTransport#open --- Key: HIVE-6249 URL: https://issues.apache.org/jira/browse/HIVE-6249 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6249.1.patch Thrift's TSSLTransportFactory#getClientSocket already returns an open socket. HiveConnection#openTransport should call open only on sockets which are not bound. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5826) Add https support to HiveServer2 http mode
[ https://issues.apache.org/jira/browse/HIVE-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-5826: --- Attachment: HIVE-5826.2.patch Add https support to HiveServer2 http mode --- Key: HIVE-5826 URL: https://issues.apache.org/jira/browse/HIVE-5826 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-5826.1.patch, HIVE-5826.2.patch Current http implementation does not have support for https -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6139) Implement vectorized decimal division and modulo
[
https://issues.apache.org/jira/browse/HIVE-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883322#comment-13883322
]
Hive QA commented on HIVE-6139:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624866/HIVE-6139.08.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 4966 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_parallel_orderby
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
org.apache.hcatalog.hbase.TestHBaseHCatStorageHandler.testTableCreateDrop
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1047/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1047/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624866
Implement vectorized decimal division and modulo
Key: HIVE-6139
URL: https://issues.apache.org/jira/browse/HIVE-6139
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Eric Hanson
Attachments: HIVE-6139.01.patch, HIVE-6139.02.patch,
HIVE-6139.07.patch, HIVE-6139.07.patch, HIVE-6139.08.patch
Support column-scalar, scalar-column, and column-column versions for division
and modulo. Include unit tests.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-5826) Add https support to HiveServer2 http mode
[ https://issues.apache.org/jira/browse/HIVE-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-5826: --- Attachment: HIVE-5826.2.patch Add https support to HiveServer2 http mode --- Key: HIVE-5826 URL: https://issues.apache.org/jira/browse/HIVE-5826 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-5826.1.patch, HIVE-5826.2.patch Current http implementation does not have support for https -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5826) Add https support to HiveServer2 http mode
[ https://issues.apache.org/jira/browse/HIVE-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-5826: --- Attachment: (was: HIVE-5826.2.patch) Add https support to HiveServer2 http mode --- Key: HIVE-5826 URL: https://issues.apache.org/jira/browse/HIVE-5826 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-5826.1.patch, HIVE-5826.2.patch Current http implementation does not have support for https -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Review Request 17422: Add https support to HiveServer2 http mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17422/ --- Review request for hive and Thejas Nair. Bugs: HIVE-5826 https://issues.apache.org/jira/browse/HIVE-5826 Repository: hive-git Description --- Here: https://issues.apache.org/jira/browse/HIVE-5826 Diffs - itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java b271d65 itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestSSL.java d0c4fc2 itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java PRE-CREATION itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java a65e678 itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHiveServer2.java 910de9b itests/hive-unit/src/test/java/org/apache/hive/service/cli/thrift/TestThriftHttpCLIService.java 65177dd jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java aabb5cb pom.xml 41f5337 service/pom.xml dff3174 service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java b5a6138 service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java e487a7f Diff: https://reviews.apache.org/r/17422/diff/ Testing --- New tests added to TestSSL, some refactoring of MiniHS2 as well. Thanks, Vaibhav Gumashta
[jira] [Commented] (HIVE-5826) Add https support to HiveServer2 http mode
[ https://issues.apache.org/jira/browse/HIVE-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883325#comment-13883325 ] Vaibhav Gumashta commented on HIVE-5826: Review board: https://reviews.apache.org/r/17422/ Add https support to HiveServer2 http mode --- Key: HIVE-5826 URL: https://issues.apache.org/jira/browse/HIVE-5826 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-5826.1.patch, HIVE-5826.2.patch Current http implementation does not have support for https -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6297) [Refactor] Move new Auth Interface to common/
[
https://issues.apache.org/jira/browse/HIVE-6297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883327#comment-13883327
]
Hive QA commented on HIVE-6297:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624945/HIVE-6297.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1048/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1048/console
Messages:
{noformat}
This message was trimmed, see log for full details
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:108:5:
Decision can match input such as KW_ORDER KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:121:5:
Decision can match input such as KW_CLUSTER KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:133:5:
Decision can match input such as KW_PARTITION KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:144:5:
Decision can match input such as KW_DISTRIBUTE KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:155:5:
Decision can match input such as KW_SORT KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:172:7:
Decision can match input such as STAR using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:185:5:
Decision can match input such as KW_STRUCT using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:185:5:
Decision can match input such as KW_ARRAY using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:185:5:
Decision can match input such as KW_UNIONTYPE using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:267:5:
Decision can match input such as KW_NULL using multiple alternatives: 1, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:267:5:
Decision can match input such as KW_FALSE using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:267:5:
Decision can match input such as KW_TRUE using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:267:5:
Decision can match input such as KW_DATE StringLiteral using multiple
alternatives: 2, 3
As a result, alternative(s) 3 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as KW_BETWEEN KW_MAP LPAREN using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_INTO using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_LATERAL
KW_VIEW using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:399:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN
using
[jira] [Created] (HIVE-6318) Document SSL support added to HiveServer2
Vaibhav Gumashta created HIVE-6318: -- Summary: Document SSL support added to HiveServer2 Key: HIVE-6318 URL: https://issues.apache.org/jira/browse/HIVE-6318 Project: Hive Issue Type: Task Components: HiveServer2, JDBC Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 SSL support is/will be added to HiveServer2 running in both binary and http mode, in unsecured auth modes. Need to document the usage and setup. Linking relevant jiras. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5826) Add https support to HiveServer2 http mode
[ https://issues.apache.org/jira/browse/HIVE-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-5826: --- Status: Patch Available (was: Open) Add https support to HiveServer2 http mode --- Key: HIVE-5826 URL: https://issues.apache.org/jira/browse/HIVE-5826 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-5826.1.patch, HIVE-5826.2.patch Current http implementation does not have support for https -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6312) doAs with plain sasl auth should be session aware
[ https://issues.apache.org/jira/browse/HIVE-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883362#comment-13883362 ] Vaibhav Gumashta commented on HIVE-6312: Hi [~navis]. Thanks a lot for the patch! Can you also upload it to rb since it's easier to read there? doAs with plain sasl auth should be session aware - Key: HIVE-6312 URL: https://issues.apache.org/jira/browse/HIVE-6312 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Navis Assignee: Navis Attachments: HIVE-6312.1.patch.txt TUGIContainingProcessor creates new Subject for each invocation which induces FileSystem leakage when cache is enable(true by default). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6246) Sign(a) UDF is not supported for decimal type
[ https://issues.apache.org/jira/browse/HIVE-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-6246: -- Attachment: HIVE-6246.1.patch Patch #1 has no real code changes: comments and method name. Sign(a) UDF is not supported for decimal type - Key: HIVE-6246 URL: https://issues.apache.org/jira/browse/HIVE-6246 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.12.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-6246.1.patch, HIVE-6246.patch java.sql.SQLException: Error while compiling statement: FAILED: SemanticException [Error 10014]: Line 1:86 Wrong arguments 'a': No matching method for class org.apache.hadoop.hive.ql.udf.UDFSign with (decimal(38,10)). Possible choices: _FUNC_(double) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6246) Sign(a) UDF is not supported for decimal type
[ https://issues.apache.org/jira/browse/HIVE-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-6246: -- Attachment: HIVE-6246.1.patch Sign(a) UDF is not supported for decimal type - Key: HIVE-6246 URL: https://issues.apache.org/jira/browse/HIVE-6246 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.12.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-6246.1.patch, HIVE-6246.1.patch, HIVE-6246.patch java.sql.SQLException: Error while compiling statement: FAILED: SemanticException [Error 10014]: Line 1:86 Wrong arguments 'a': No matching method for class org.apache.hadoop.hive.ql.udf.UDFSign with (decimal(38,10)). Possible choices: _FUNC_(double) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5826) Add https support to HiveServer2 http mode
[ https://issues.apache.org/jira/browse/HIVE-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883386#comment-13883386 ] Vaibhav Gumashta commented on HIVE-5826: Some formatting issues remain in the patch - will upload a new one after the reviews. Add https support to HiveServer2 http mode --- Key: HIVE-5826 URL: https://issues.apache.org/jira/browse/HIVE-5826 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-5826.1.patch, HIVE-5826.2.patch Current http implementation does not have support for https -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 16728: Implement non-staged MapJoin
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16728/#review32885
---
itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java
https://reviews.apache.org/r/16728/#comment61876
This would break tez tests.
itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java
https://reviews.apache.org/r/16728/#comment61875
This would eliminate tez unit tests. Was this intentional?
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/LocalMapJoinProcFactory.java
https://reviews.apache.org/r/16728/#comment61880
Could you raise a jira for this.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/LocalMapJoinProcFactory.java
https://reviews.apache.org/r/16728/#comment61882
can it be only these 2 operators? Maybe common join operator can be used?
- Vikram Dixit Kumaraswamy
On Jan. 20, 2014, 5 a.m., Navis Ryu wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16728/
---
(Updated Jan. 20, 2014, 5 a.m.)
Review request for hive.
Bugs: HIVE-6144
https://issues.apache.org/jira/browse/HIVE-6144
Repository: hive-git
Description
---
For map join, all data in small aliases are hashed and stored into temporary
file in MapRedLocalTask. But for some aliases without filter or projection,
it seemed not necessary to do that. For example.
{noformat}
select a.* from src a join src b on a.key=b.key;
{noformat}
makes plan like this.
{noformat}
STAGE PLANS:
Stage: Stage-4
Map Reduce Local Work
Alias - Map Local Tables:
a
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
a
TableScan
alias: a
HashTable Sink Operator
condition expressions:
0 {key} {value}
1
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
Position of Big Table: 1
Stage: Stage-3
Map Reduce
Alias - Map Operator Tree:
b
TableScan
alias: b
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {key} {value}
1
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
outputColumnNames: _col0, _col1
Position of Big Table: 1
Select Operator
File Output Operator
Local Work:
Map Reduce Local Work
Stage: Stage-0
Fetch Operator
{noformat}
table src(a) is fetched and stored as-is in MRLocalTask. With this patch,
plan can be like below.
{noformat}
Stage: Stage-3
Map Reduce
Alias - Map Operator Tree:
b
TableScan
alias: b
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {key} {value}
1
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
outputColumnNames: _col0, _col1
Position of Big Table: 1
Select Operator
File Output Operator
Local Work:
Map Reduce Local Work
Alias - Map Local Tables:
a
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
a
TableScan
alias: a
Has Any Stage Alias: false
Stage: Stage-0
Fetch Operator
{noformat}
Diffs
-
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a78b72f
conf/hive-default.xml.template 7cd8a1f
itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 9ad5986
itests/util/src/main/java/org/apache/hadoop/hive/ql/hooks/MapJoinCounterHook.java
1b0d57e
ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java
d8f4eb4
ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableLoader.java a080fcc
ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java
fc08b28
ql/src/java/org/apache/hadoop/hive/ql/exec/JoinUtil.java 1e0314d
ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java bdc85b9
ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 56676df
ql/src/java/org/apache/hadoop/hive/ql/exec/TemporaryHashSinkOperator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java
[jira] [Commented] (HIVE-6144) Implement non-staged MapJoin
[
https://issues.apache.org/jira/browse/HIVE-6144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883387#comment-13883387
]
Vikram Dixit K commented on HIVE-6144:
--
I have left some comments on the review-board.
The 2 test failures can be ignored. These pass on my machine.
Thanks
Vikram.
Implement non-staged MapJoin
Key: HIVE-6144
URL: https://issues.apache.org/jira/browse/HIVE-6144
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-6144.1.patch.txt, HIVE-6144.2.patch.txt,
HIVE-6144.3.patch.txt, HIVE-6144.4.patch.txt, HIVE-6144.5.patch.txt,
HIVE-6144.6.patch.txt, HIVE-6144.7.patch.txt
For map join, all data in small aliases are hashed and stored into temporary
file in MapRedLocalTask. But for some aliases without filter or projection,
it seemed not necessary to do that. For example.
{noformat}
select a.* from src a join src b on a.key=b.key;
{noformat}
makes plan like this.
{noformat}
STAGE PLANS:
Stage: Stage-4
Map Reduce Local Work
Alias - Map Local Tables:
a
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
a
TableScan
alias: a
HashTable Sink Operator
condition expressions:
0 {key} {value}
1
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
Position of Big Table: 1
Stage: Stage-3
Map Reduce
Alias - Map Operator Tree:
b
TableScan
alias: b
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {key} {value}
1
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
outputColumnNames: _col0, _col1
Position of Big Table: 1
Select Operator
File Output Operator
Local Work:
Map Reduce Local Work
Stage: Stage-0
Fetch Operator
{noformat}
table src(a) is fetched and stored as-is in MRLocalTask. With this patch,
plan can be like below.
{noformat}
Stage: Stage-3
Map Reduce
Alias - Map Operator Tree:
b
TableScan
alias: b
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {key} {value}
1
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
outputColumnNames: _col0, _col1
Position of Big Table: 1
Select Operator
File Output Operator
Local Work:
Map Reduce Local Work
Alias - Map Local Tables:
a
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
a
TableScan
alias: a
Has Any Stage Alias: false
Stage: Stage-0
Fetch Operator
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congratulations, Sergey and Jason. From: Carl Steinbach c...@apache.org Subject: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere Date: January 27, 2014 8:36:37 AM PST To: dev@hive.apache.org dev@hive.apache.org, u...@hive.apache.org u...@hive.apache.org, jd...@hortonworks.com, ser...@hortonworks.com Reply-To: dev@hive.apache.org The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl
[jira] [Commented] (HIVE-6262) Remove unnecessary copies of schema + table desc from serialized plan
[ https://issues.apache.org/jira/browse/HIVE-6262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883389#comment-13883389 ] Vikram Dixit K commented on HIVE-6262: -- This is really good in terms of memory efficiency. LGTM +1. Remove unnecessary copies of schema + table desc from serialized plan - Key: HIVE-6262 URL: https://issues.apache.org/jira/browse/HIVE-6262 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-6262.1.patch Currently for a partitioned table the following are true: - for each partitiondesc we send a copy of the corresponding tabledesc - for each partitiondesc we send two copies of the schema (in different formats). Obviously we need to send different schemas if they are required by schema evolution, but in our case we'll always end up with multiple copies. The effect can be dramatic. The reductions by removing those on partitioned tables easily be can be 8-10x in size. Plans themselves can be 10s to 100s of mb (even with kryo). The size difference also plays out in every task on the cluster we run. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Congrats!! On Mon, Jan 27, 2014 at 3:54 PM, Ted Yu yuzhih...@gmail.com wrote: Congratulations, Sergey and Jason. From: Carl Steinbach c...@apache.org Subject: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere Date: January 27, 2014 8:36:37 AM PST To: dev@hive.apache.org dev@hive.apache.org, u...@hive.apache.org u...@hive.apache.org, jd...@hortonworks.com, ser...@hortonworks.com Reply-To: dev@hive.apache.org The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
[jira] [Updated] (HIVE-5814) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat
[ https://issues.apache.org/jira/browse/HIVE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-5814: - Attachment: HIVE-5814.2.patch same patch, diff name; trying to get pre commit test to run Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat - Key: HIVE-5814 URL: https://issues.apache.org/jira/browse/HIVE-5814 Project: Hive Issue Type: New Feature Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-5814 HCat-Pig type mapping.pdf, HIVE-5814.2.patch Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well.Also note that CHAR was added recently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5929) SQL std auth - Access control statement updates
[ https://issues.apache.org/jira/browse/HIVE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5929: Status: Open (was: Patch Available) SQL std auth - Access control statement updates --- Key: HIVE-5929 URL: https://issues.apache.org/jira/browse/HIVE-5929 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5929.1.patch, HIVE-5929.2.patch, HIVE-5929.3.patch, HIVE-5929.4.patch, HIVE-5929.5.patch Original Estimate: 48h Remaining Estimate: 48h Subtask for sql standard based auth, for performing the updates to metastore from newly supported access control statements . -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5929) SQL std auth - Access control statement updates
[ https://issues.apache.org/jira/browse/HIVE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5929: Attachment: HIVE-5929.5.patch SQL std auth - Access control statement updates --- Key: HIVE-5929 URL: https://issues.apache.org/jira/browse/HIVE-5929 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5929.1.patch, HIVE-5929.2.patch, HIVE-5929.3.patch, HIVE-5929.4.patch, HIVE-5929.5.patch Original Estimate: 48h Remaining Estimate: 48h Subtask for sql standard based auth, for performing the updates to metastore from newly supported access control statements . -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5814) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat
[ https://issues.apache.org/jira/browse/HIVE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-5814: - Status: Open (was: Patch Available) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat - Key: HIVE-5814 URL: https://issues.apache.org/jira/browse/HIVE-5814 Project: Hive Issue Type: New Feature Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-5814 HCat-Pig type mapping.pdf Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well.Also note that CHAR was added recently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5814) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat
[ https://issues.apache.org/jira/browse/HIVE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-5814: - Attachment: (was: HIVE-5814.patch) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat - Key: HIVE-5814 URL: https://issues.apache.org/jira/browse/HIVE-5814 Project: Hive Issue Type: New Feature Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-5814 HCat-Pig type mapping.pdf Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well.Also note that CHAR was added recently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5814) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat
[ https://issues.apache.org/jira/browse/HIVE-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-5814: - Status: Patch Available (was: Open) Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat - Key: HIVE-5814 URL: https://issues.apache.org/jira/browse/HIVE-5814 Project: Hive Issue Type: New Feature Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-5814 HCat-Pig type mapping.pdf, HIVE-5814.2.patch Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well.Also note that CHAR was added recently. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5929) SQL std auth - Access control statement updates
[ https://issues.apache.org/jira/browse/HIVE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5929: Status: Patch Available (was: Open) SQL std auth - Access control statement updates --- Key: HIVE-5929 URL: https://issues.apache.org/jira/browse/HIVE-5929 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5929.1.patch, HIVE-5929.2.patch, HIVE-5929.3.patch, HIVE-5929.4.patch, HIVE-5929.5.patch Original Estimate: 48h Remaining Estimate: 48h Subtask for sql standard based auth, for performing the updates to metastore from newly supported access control statements . -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6251) Add ability to specify delimiter in HCatalog Java API to create tables - HCatCreateTableDesc
[
https://issues.apache.org/jira/browse/HIVE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-6251:
-
Attachment: HIVE-6251.2.patch
Add ability to specify delimiter in HCatalog Java API to create tables -
HCatCreateTableDesc
Key: HIVE-6251
URL: https://issues.apache.org/jira/browse/HIVE-6251
Project: Hive
Issue Type: Bug
Components: HCatalog, WebHCat
Affects Versions: 0.13.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-6251.2.patch, HIVE-6251.patch
Per
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTablerow_format,
the following is supported when creating a table.
{code}
: DELIMITED [FIELDS TERMINATED BY char [ESCAPED BY char]] [COLLECTION ITEMS
TERMINATED BY char]
[MAP KEYS TERMINATED BY char] [LINES TERMINATED BY char]
[NULL DEFINED AS char] (Note: Only available starting with Hive 0.13)
| SERDE serde_name [WITH SERDEPROPERTIES (property_name=property_value,
property_name=property_value, ...)]
{code}
Need to add support for specifying 4 delimiters plus escape and NULL in
HCatCreateTableDesc. create(String dbName, String tableName,
ListHCatFieldSchema columns) API.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6251) Add ability to specify delimiter in HCatalog Java API to create tables - HCatCreateTableDesc
[
https://issues.apache.org/jira/browse/HIVE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-6251:
-
Status: Patch Available (was: Open)
Add ability to specify delimiter in HCatalog Java API to create tables -
HCatCreateTableDesc
Key: HIVE-6251
URL: https://issues.apache.org/jira/browse/HIVE-6251
Project: Hive
Issue Type: Bug
Components: HCatalog, WebHCat
Affects Versions: 0.13.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-6251.2.patch, HIVE-6251.patch
Per
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTablerow_format,
the following is supported when creating a table.
{code}
: DELIMITED [FIELDS TERMINATED BY char [ESCAPED BY char]] [COLLECTION ITEMS
TERMINATED BY char]
[MAP KEYS TERMINATED BY char] [LINES TERMINATED BY char]
[NULL DEFINED AS char] (Note: Only available starting with Hive 0.13)
| SERDE serde_name [WITH SERDEPROPERTIES (property_name=property_value,
property_name=property_value, ...)]
{code}
Need to add support for specifying 4 delimiters plus escape and NULL in
HCatCreateTableDesc. create(String dbName, String tableName,
ListHCatFieldSchema columns) API.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6251) Add ability to specify delimiter in HCatalog Java API to create tables - HCatCreateTableDesc
[
https://issues.apache.org/jira/browse/HIVE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-6251:
-
Status: Open (was: Patch Available)
Add ability to specify delimiter in HCatalog Java API to create tables -
HCatCreateTableDesc
Key: HIVE-6251
URL: https://issues.apache.org/jira/browse/HIVE-6251
Project: Hive
Issue Type: Bug
Components: HCatalog, WebHCat
Affects Versions: 0.13.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-6251.2.patch, HIVE-6251.patch
Per
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTablerow_format,
the following is supported when creating a table.
{code}
: DELIMITED [FIELDS TERMINATED BY char [ESCAPED BY char]] [COLLECTION ITEMS
TERMINATED BY char]
[MAP KEYS TERMINATED BY char] [LINES TERMINATED BY char]
[NULL DEFINED AS char] (Note: Only available starting with Hive 0.13)
| SERDE serde_name [WITH SERDEPROPERTIES (property_name=property_value,
property_name=property_value, ...)]
{code}
Need to add support for specifying 4 delimiters plus escape and NULL in
HCatCreateTableDesc. create(String dbName, String tableName,
ListHCatFieldSchema columns) API.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Work logged] (HIVE-6258) sql std auth - disallow cycles between roles
[ https://issues.apache.org/jira/browse/HIVE-6258?focusedWorklogId=15735page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-15735 ] Thejas M Nair logged work on HIVE-6258: --- Author: Thejas M Nair Created on: 27/Jan/14 22:52 Start Date: 27/Jan/14 22:52 Worklog Time Spent: 16h Issue Time Tracking --- Worklog Id: (was: 15735) Time Spent: 16h Remaining Estimate: 8h (was: 24h) sql std auth - disallow cycles between roles Key: HIVE-6258 URL: https://issues.apache.org/jira/browse/HIVE-6258 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6258.1.patch Original Estimate: 24h Time Spent: 16h Remaining Estimate: 8h It should not be possible to have cycles in role relationships. If a grant role statement would end up adding such a cycle, it should result in an error. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6060) Define API for RecordUpdater and UpdateReader
[ https://issues.apache.org/jira/browse/HIVE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-6060: Attachment: h-5317.patch This patch updates the input/output apis: * Add AcidInputFormat and AcidOutputFormat * Add RecordIdentifier * Makes OrcInputFormat and OrcOutputFormat implement AcidInputFormat and AcidOutputFormat. * Adds some simple stubs to OrcOutputFormat so that I can see the calls to the RecordUpdater. Define API for RecordUpdater and UpdateReader - Key: HIVE-6060 URL: https://issues.apache.org/jira/browse/HIVE-6060 Project: Hive Issue Type: Sub-task Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-5317.patch, h-5317.patch, h-5317.patch We need to define some new APIs for how Hive interacts with the file formats since it needs to be much richer than the current RecordReader and RecordWriter. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5959) SQL std auth - bootstrap SUPERUSER, PUBLIC roles
[
https://issues.apache.org/jira/browse/HIVE-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883484#comment-13883484
]
Hive QA commented on HIVE-5959:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625390/HIVE-5959.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4962 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
org.apache.hadoop.hive.metastore.TestMetastoreVersion.testVersionMisMatch
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1049/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1049/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625390
SQL std auth - bootstrap SUPERUSER, PUBLIC roles
Key: HIVE-5959
URL: https://issues.apache.org/jira/browse/HIVE-5959
Project: Hive
Issue Type: Sub-task
Components: Authorization
Reporter: Thejas M Nair
Assignee: Ashutosh Chauhan
Attachments: HIVE-5959.patch
Original Estimate: 72h
Remaining Estimate: 72h
SUPERUSER and PUBLIC are two roles that are always present, these need to be
added automatically on metastore startup. This would be similar to creation
of the default database from HMSHandler.init().
A config param in hive-site.xml will be used to specify the list of users who
belong to the SUPERUSER role.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6310) Fix a few minimr test failures
[
https://issues.apache.org/jira/browse/HIVE-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6310:
--
Attachment: HIVE-6310.2.patch
Patch #2 updated with fix for import_exported_table.q. Still having no clue why
the other two tests are failing. Will have to exclude them if they keep failing.
Fix a few minimr test failures
--
Key: HIVE-6310
URL: https://issues.apache.org/jira/browse/HIVE-6310
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-6310.1.patch, HIVE-6310.2.patch, HIVE-6310.patch
These test cases are:
{code}
ql/src/test/queries/clientpositive/import_exported_table.q
ql/src/test/queries/clientpositive/load_hdfs_file_with_space_in_the_name.q
ql/src/test/queries/clientpositive/root_dir_external_table.q
{code}
They are failing because of existing hdfs:///tmp/test, possible left over by
other tests.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere
Kudos, Jason! Kudos, Sergey! -- Lefty On Mon, Jan 27, 2014 at 1:58 PM, Brock Noland br...@cloudera.com wrote: Congrats!! On Mon, Jan 27, 2014 at 3:54 PM, Ted Yu yuzhih...@gmail.com wrote: Congratulations, Sergey and Jason. From: Carl Steinbach c...@apache.org Subject: [ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere Date: January 27, 2014 8:36:37 AM PST To: dev@hive.apache.org dev@hive.apache.org, u...@hive.apache.org u...@hive.apache.org, jd...@hortonworks.com, ser...@hortonworks.com Reply-To: dev@hive.apache.org The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere committers on the Apache Hive Project. Please join me in congratulating Sergey and Jason! Thanks. Carl -- Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: Review Request 17005: Vectorized reader for DECIMAL datatype for ORC format.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17005/ --- (Updated Jan. 27, 2014, 11:20 p.m.) Review request for hive and Eric Hanson. Bugs: HIVE-6178 https://issues.apache.org/jira/browse/HIVE-6178 Repository: hive-git Description --- vectorized reader for DECIMAL datatype for ORC format. Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 15025cf common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java d71ebb3 common/src/test/org/apache/hadoop/hive/common/type/TestUnsignedInt128.java fbb2aa0 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java 23564bb ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 0df82b9 ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java 0d5b7ff Diff: https://reviews.apache.org/r/17005/diff/ Testing --- Thanks, Jitendra Pandey
Re: Review Request 17163: HIVE-5929 - SQL std auth - Access control statement updates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17163/ --- (Updated Jan. 27, 2014, 11:22 p.m.) Review request for hive and Ashutosh Chauhan. Changes --- HIVE-5929.5.patch - fixes the unit test failures (change in SessionState.start, to create a copy of HiveConf before calling Hive.get(conf) ). Bugs: HIVE-5929 https://issues.apache.org/jira/browse/HIVE-5929 Repository: hive-git Description --- Subtask for sql standard based auth, for performing the updates to metastore from newly supported access control statements . Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/Driver.java cb0c1a5 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 92ed55b ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c65bf28 ql/src/java/org/apache/hadoop/hive/ql/parse/authorization/HiveAuthorizationTaskFactoryImpl.java c41cd0f ql/src/java/org/apache/hadoop/hive/ql/security/authorization/AuthorizationUtils.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/DefaultHiveAuthorizerFactory.java 7470e9d ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAccessController.java 8e4114f ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizationPluginException.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizationValidator.java 63046f5 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizer.java c10a2ac ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizerImpl.java ca95bfc ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeInfo.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAuthorizationValidator.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAuthorizerFactory.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 08f7fec ql/src/test/queries/clientpositive/authorization_1_sql_std.q PRE-CREATION ql/src/test/queries/clientpositive/authorization_role_grant1.q 1a375a5 ql/src/test/results/clientpositive/authorization_1_sql_std.q.out PRE-CREATION ql/src/test/results/clientpositive/authorization_role_grant1.q.out 496687c Diff: https://reviews.apache.org/r/17163/diff/ Testing --- .q tests included. Thanks, Thejas Nair
[jira] [Commented] (HIVE-5929) SQL std auth - Access control statement updates
[ https://issues.apache.org/jira/browse/HIVE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883513#comment-13883513 ] Thejas M Nair commented on HIVE-5929: - HIVE-5929.5.patch - fixes the unit test failures (change in SessionState.java, to create a copy of HiveConf before calling Hive.get(conf) ). This problem was not seen earlier because one of the calls from SessionState.setupAuth() was leading to a new HiveObject being created with HiveConf copy. As a side effect of not calling SessionState.setupAuth() from SessionState.start() this bug showed up. SQL std auth - Access control statement updates --- Key: HIVE-5929 URL: https://issues.apache.org/jira/browse/HIVE-5929 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5929.1.patch, HIVE-5929.2.patch, HIVE-5929.3.patch, HIVE-5929.4.patch, HIVE-5929.5.patch Original Estimate: 48h Remaining Estimate: 48h Subtask for sql standard based auth, for performing the updates to metastore from newly supported access control statements . -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 17005: Vectorized reader for DECIMAL datatype for ORC format.
On Jan. 27, 2014, 6:54 p.m., Eric Hanson wrote: ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java, line 1126 https://reviews.apache.org/r/17005/diff/2/?file=450612#file450612line1126 For safety, you could put a check here and throw an exception if scratchScaleVector.vector[I] is not equal to result.vector[I].scale. Alternatively, you could make the value NULL if that is the case, and maybe log an error. The code in the latest patch re-scales the decimal value to the expected scale if there is a mismatch. - Jitendra --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17005/#review32855 --- On Jan. 27, 2014, 11:20 p.m., Jitendra Pandey wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17005/ --- (Updated Jan. 27, 2014, 11:20 p.m.) Review request for hive and Eric Hanson. Bugs: HIVE-6178 https://issues.apache.org/jira/browse/HIVE-6178 Repository: hive-git Description --- vectorized reader for DECIMAL datatype for ORC format. Diffs - common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 15025cf common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java d71ebb3 common/src/test/org/apache/hadoop/hive/common/type/TestUnsignedInt128.java fbb2aa0 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java 23564bb ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 0df82b9 ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java 0d5b7ff Diff: https://reviews.apache.org/r/17005/diff/ Testing --- Thanks, Jitendra Pandey
[jira] [Work logged] (HIVE-5929) SQL std auth - Access control statement updates
[ https://issues.apache.org/jira/browse/HIVE-5929?focusedWorklogId=15736page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-15736 ] Thejas M Nair logged work on HIVE-5929: --- Author: Thejas M Nair Created on: 27/Jan/14 23:27 Start Date: 27/Jan/14 23:27 Worklog Time Spent: 1.6h Issue Time Tracking --- Worklog Id: (was: 15736) Time Spent: 1.6h Remaining Estimate: 8h (was: 48h) SQL std auth - Access control statement updates --- Key: HIVE-5929 URL: https://issues.apache.org/jira/browse/HIVE-5929 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5929.1.patch, HIVE-5929.2.patch, HIVE-5929.3.patch, HIVE-5929.4.patch, HIVE-5929.5.patch Original Estimate: 48h Time Spent: 1.6h Remaining Estimate: 8h Subtask for sql standard based auth, for performing the updates to metastore from newly supported access control statements . -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6178) Implement vectorized reader for DECIMAL datatype for ORC format.
[ https://issues.apache.org/jira/browse/HIVE-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6178: --- Status: Patch Available (was: Open) Updated patch to fix the scale if the data has a different scale than the one in schema. Implement vectorized reader for DECIMAL datatype for ORC format. Key: HIVE-6178 URL: https://issues.apache.org/jira/browse/HIVE-6178 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6178.1.patch, HIVE-6178.2.patch, HIVE-6178.3.patch Implement vectorized reader for DECIMAL datatype for ORC format. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6178) Implement vectorized reader for DECIMAL datatype for ORC format.
[ https://issues.apache.org/jira/browse/HIVE-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6178: --- Attachment: HIVE-6178.3.patch Implement vectorized reader for DECIMAL datatype for ORC format. Key: HIVE-6178 URL: https://issues.apache.org/jira/browse/HIVE-6178 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6178.1.patch, HIVE-6178.2.patch, HIVE-6178.3.patch Implement vectorized reader for DECIMAL datatype for ORC format. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Work logged] (HIVE-5928) Add a hive authorization plugin api that does not assume privileges needed
[ https://issues.apache.org/jira/browse/HIVE-5928?focusedWorklogId=15737page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-15737 ] Thejas M Nair logged work on HIVE-5928: --- Author: Thejas M Nair Created on: 27/Jan/14 23:29 Start Date: 27/Jan/14 23:29 Worklog Time Spent: 4h Issue Time Tracking --- Worklog Id: (was: 15737) Time Spent: 6h (was: 2h) Remaining Estimate: 0h (was: 12h) Add a hive authorization plugin api that does not assume privileges needed -- Key: HIVE-5928 URL: https://issues.apache.org/jira/browse/HIVE-5928 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.13.0 Attachments: HIVE-5928.1.patch, HIVE-5928.2.patch, hive_auth_class_preview.txt Original Estimate: 120h Time Spent: 6h Remaining Estimate: 0h The existing HiveAuthorizationProvider interface implementations can be used to support custom authorization models. But this interface limits the customization for these reasons - 1. It has assumptions about the privileges required for an action. 2. It does have not functions that you can implement for having custom ways of doing the actions of access control statements. This jira proposes a new interface HiveAuthorizer that does not make assumptions of the privileges required for the actions. The authorize() functions will be equivalent of authorize(operation type, input objects, output objects). It will also have functions that will be called from the access control statements. The current HiveAuthorizationProvider will continue to be supported for backward compatibility. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Work logged] (HIVE-5932) SQL std auth - add list_all_roles to metastore api
[ https://issues.apache.org/jira/browse/HIVE-5932?focusedWorklogId=15738page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-15738 ] Thejas M Nair logged work on HIVE-5932: --- Author: Thejas M Nair Created on: 27/Jan/14 23:31 Start Date: 27/Jan/14 23:31 Worklog Time Spent: 1h Issue Time Tracking --- Worklog Id: (was: 15738) Time Spent: 1h Remaining Estimate: 0h (was: 24h) SQL std auth - add list_all_roles to metastore api -- Key: HIVE-5932 URL: https://issues.apache.org/jira/browse/HIVE-5932 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Original Estimate: 24h Time Spent: 1h Remaining Estimate: 0h This is needed for 'show all roles' statement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
