Re: Review Request 14523: HIVE-5486 HiveServer2 should create base scratch directories at startup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14523/ --- (Updated Oct. 9, 2013, 6:34 a.m.) Review request for hive. Changes --- Changes per review feedback Bugs: HIVE-5486 https://issues.apache.org/jira/browse/HIVE-5486 Repository: hive-git Description --- With impersonation enabled, the same base directory is used by all sessions/queries. For a new deployment, this directory gets created on first invocation by the user running that session. This would cause directory permission conflict for other users. The patch is creating base scratch dirs at startup if it doesn't exist. Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d0895e1 service/src/java/org/apache/hive/service/cli/CLIService.java 1a7f338 service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java ae7bb6b service/src/test/org/apache/hive/service/cli/TestEmbeddedThriftBinaryCLIService.java da325da service/src/test/org/apache/hive/service/cli/TestScratchDir.java PRE-CREATION Diff: https://reviews.apache.org/r/14523/diff/ Testing --- Added new test Thanks, Prasad Mujumdar
[jira] [Updated] (HIVE-5486) HiveServer2 should create base scratch directories at startup
[ https://issues.apache.org/jira/browse/HIVE-5486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-5486: -- Attachment: HIVE-5486.3.patch HiveServer2 should create base scratch directories at startup - Key: HIVE-5486 URL: https://issues.apache.org/jira/browse/HIVE-5486 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0, 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5486.2.patch, HIVE-5486.3.patch With impersonation enabled, the same base directory is used by all sessions/queries. For a new deployment, this directory gets created on first invocation by the user running that session. This would cause directory permission conflict for other users. HiveServer2 should create the base scratch dirs if it doesn't exist. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5493) duplicate jars with different versions for guava, commons-logging
[
https://issues.apache.org/jira/browse/HIVE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790104#comment-13790104
]
Hive QA commented on HIVE-5493:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12607449/HIVE-5493.1.patch
{color:green}SUCCESS:{color} +1 4390 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1080/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1080/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
duplicate jars with different versions for guava, commons-logging
-
Key: HIVE-5493
URL: https://issues.apache.org/jira/browse/HIVE-5493
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-5493.1.patch
Duplicate jars with different versions for guava and commons-logging are
present in build/dist/lib .
hive should ship with just one version of each jar.
guava-r08 and guava-0.11.0.2 are present, and commons-logging has 1.0.4 and
1.1.1 .
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (HIVE-5499) can not show chinese comments
alex.lv created HIVE-5499: - Summary: can not show chinese comments Key: HIVE-5499 URL: https://issues.apache.org/jira/browse/HIVE-5499 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.11.0 Environment: hadoop-cdh3u6 Reporter: alex.lv desc formatted tablename1 can not show chinese comments and the result is messy code -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5499) can not show chinese comments
[ https://issues.apache.org/jira/browse/HIVE-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] alex.lv updated HIVE-5499: -- Attachment: HIVE-5499.patch can not show chinese comments - Key: HIVE-5499 URL: https://issues.apache.org/jira/browse/HIVE-5499 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.11.0 Environment: hadoop-cdh3u6 Reporter: alex.lv Attachments: HIVE-5499.patch desc formatted tablename1 can not show chinese comments and the result is messy code -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5499) can not show chinese comments
[ https://issues.apache.org/jira/browse/HIVE-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790106#comment-13790106 ] alex.lv commented on HIVE-5499: --- the patch is for hive-0.11.0,and i'm not sure it can be used on other hive versions can not show chinese comments - Key: HIVE-5499 URL: https://issues.apache.org/jira/browse/HIVE-5499 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.11.0 Environment: hadoop-cdh3u6 Reporter: alex.lv Attachments: HIVE-5499.patch desc formatted tablename1 can not show chinese comments and the result is messy code -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (HIVE-5499) can not show chinese comments
[ https://issues.apache.org/jira/browse/HIVE-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] alex.lv resolved HIVE-5499. --- Resolution: Fixed can not show chinese comments - Key: HIVE-5499 URL: https://issues.apache.org/jira/browse/HIVE-5499 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.11.0 Environment: hadoop-cdh3u6 Reporter: alex.lv Attachments: HIVE-5499.patch desc formatted tablename1 can not show chinese comments and the result is messy code -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5484) TestSchemaTool failures when Hive version has more than 3 revision numbers
[ https://issues.apache.org/jira/browse/HIVE-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-5484: --- Status: Patch Available (was: Open) Marking Patch Available to get QA run. TestSchemaTool failures when Hive version has more than 3 revision numbers -- Key: HIVE-5484 URL: https://issues.apache.org/jira/browse/HIVE-5484 Project: Hive Issue Type: Bug Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-5484.1.patch, HIVE-5484.2.patch If Hive is created with a version string with more than 3 numbers, we end up with a couple of test failures in TestSchemaTool, because the metastore is expecting a version with the format of majorVersion.minorVersion.changeVersion. error message=Unknown version specified for initialization: 0.12.0.2.0.6.0-61 type=org.apache.hadoop.hive.metastore.HiveMetaExceptionorg.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for initialization: 0.12.0.2.0.6.0-61 at org.apache.hadoop.hive.metastore.MetaStoreSchemaInfo.generateInitFileName(MetaStoreSchemaInfo.java:113) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:269) at org.apache.hive.beeline.src.test.TestSchemaTool.testSchemaInit(TestSchemaTool.java:104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) /error /testcase testcase classname=org.apache.hive.beeline.src.test.TestSchemaTool name=testSchemaUpgrade time=2.164 error message=Found unexpected schema version 0.12.0 type=org.apache.hadoop.hive.metastore.HiveMetaExceptionorg.apache.hadoop.hive.metastore.HiveMetaException: Found unexpected schema version 0.12.0 at org.apache.hive.beeline.HiveSchemaTool.verifySchemaVersion(HiveSchemaTool.java:192) at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:242) at org.apache.hive.beeline.src.test.TestSchemaTool.testSchemaUpgrade(TestSchemaTool.java:128) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) /error -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4898) make vectorized math functions work end-to-end (update VectorizationContext.java)
[ https://issues.apache.org/jira/browse/HIVE-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790141#comment-13790141 ] Ashutosh Chauhan commented on HIVE-4898: Patch has gone stale. [~ehans] Can you rebase your patch? make vectorized math functions work end-to-end (update VectorizationContext.java) - Key: HIVE-4898 URL: https://issues.apache.org/jira/browse/HIVE-4898 Project: Hive Issue Type: Sub-task Affects Versions: vectorization-branch Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-4898.3.patch, HIVE-4898.3.patch The vectorized math function VectorExpression classes were added in HIVE-4822. This JIRA is to allow those to actually be used in a SQL query end-to-end. This requires updating VectorizationContext to use the new classes in vectorized expression creation. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5476) Authorization-provider tests fail in sequential run
[ https://issues.apache.org/jira/browse/HIVE-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-5476: --- Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Sushanth! Authorization-provider tests fail in sequential run --- Key: HIVE-5476 URL: https://issues.apache.org/jira/browse/HIVE-5476 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Sushanth Sowmyan Fix For: 0.13.0 Attachments: HIVE-5476.2.patch, HIVE-5476.patch As seen in 0.12 build with hadoop1 - https://builds.apache.org/job/Hive-branch-0.12-hadoop1/lastCompletedBuild/testReport/ Following tests fail - org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedClientSideAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5493) duplicate jars with different versions for guava, commons-logging
[ https://issues.apache.org/jira/browse/HIVE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-5493: --- Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Thejas! duplicate jars with different versions for guava, commons-logging - Key: HIVE-5493 URL: https://issues.apache.org/jira/browse/HIVE-5493 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Priority: Blocker Fix For: 0.13.0 Attachments: HIVE-5493.1.patch Duplicate jars with different versions for guava and commons-logging are present in build/dist/lib . hive should ship with just one version of each jar. guava-r08 and guava-0.11.0.2 are present, and commons-logging has 1.0.4 and 1.1.1 . -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi reassigned HIVE-5490: Assignee: Teddy Choi SUBSTR(col, 1, 0) returns wrong result in vectorized mode - Key: HIVE-5490 URL: https://issues.apache.org/jira/browse/HIVE-5490 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Teddy Choi The query select substr(cstring1, 1, 0) from alltypesorc; returns all empty strings when set hive.vectorized.execution.enabled = false; which is the correct result, and returns non-empty strings when set hive.vectorized.execution.enabled = true; which is not correct. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4846) Implement Vectorized Limit Operator
[
https://issues.apache.org/jira/browse/HIVE-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790192#comment-13790192
]
Hive QA commented on HIVE-4846:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12607479/HIVE-4846.2.patch
{color:green}SUCCESS:{color} +1 4394 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1081/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1081/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
Implement Vectorized Limit Operator
---
Key: HIVE-4846
URL: https://issues.apache.org/jira/browse/HIVE-4846
Project: Hive
Issue Type: Sub-task
Reporter: Sarvesh Sakalanaga
Assignee: Sarvesh Sakalanaga
Attachments: Hive-4846.0.patch, Hive-4846.1.patch, HIVE-4846.2.patch
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5424) Document schema tool usage
[ https://issues.apache.org/jira/browse/HIVE-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790193#comment-13790193 ] Lefty Leverenz commented on HIVE-5424: -- Added hive.metastore.schema.verification to the Configuration doc at the end of the [MetaStore section|https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-MetaStore] and revised the description of datanucleus.autoCreateSchema, but didn't find fixedDataStore for datanucleus. Also, I'm not sure I got the default right -- is hive.metastore.schema.verification false or true by default? The new wikidoc implies it's true by default, and javadoc in the patch says (Default) in the explanation of true but gives false as the value: property namehive.metastore.schema.verification/name valuefalse/value description Enforce metastore schema version consistency. True: Verify that version information stored in metastore matches with one from Hive jars. Also disable automatic schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures proper metastore schema migration. (Default) False: Warn if the version information stored in metastore doesn't match with one from in Hive jars. /description /property So please correct the Configuration doc if it ought to say true by default, because I left it as false. And if there's another config property called datanucleus.fixedDataStore, please add it to the doc. Document schema tool usage -- Key: HIVE-5424 URL: https://issues.apache.org/jira/browse/HIVE-5424 Project: Hive Issue Type: Test Components: Documentation, Metastore Affects Versions: 0.12.0, 0.13.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Document the tool usage details -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5252) Add ql syntax for inline java code creation
[
https://issues.apache.org/jira/browse/HIVE-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790259#comment-13790259
]
Hive QA commented on HIVE-5252:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12607505/HIVE-5252.1.patch.txt
{color:green}SUCCESS:{color} +1 4391 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1082/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1082/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
Add ql syntax for inline java code creation
---
Key: HIVE-5252
URL: https://issues.apache.org/jira/browse/HIVE-5252
Project: Hive
Issue Type: Sub-task
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Attachments: HIVE-5252.1.patch.txt
Something to the effect of compile 'my code here' using 'groovycompiler'.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5486) HiveServer2 should create base scratch directories at startup
[
https://issues.apache.org/jira/browse/HIVE-5486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790312#comment-13790312
]
Hive QA commented on HIVE-5486:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12607521/HIVE-5486.3.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4391 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1083/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1083/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
HiveServer2 should create base scratch directories at startup
-
Key: HIVE-5486
URL: https://issues.apache.org/jira/browse/HIVE-5486
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.11.0, 0.12.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Attachments: HIVE-5486.2.patch, HIVE-5486.3.patch
With impersonation enabled, the same base directory is used by all
sessions/queries. For a new deployment, this directory gets created on first
invocation by the user running that session. This would cause directory
permission conflict for other users.
HiveServer2 should create the base scratch dirs if it doesn't exist.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-5490: - Status: Patch Available (was: Open) SUBSTR(col, 1, 0) returns wrong result in vectorized mode - Key: HIVE-5490 URL: https://issues.apache.org/jira/browse/HIVE-5490 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Teddy Choi Attachments: HIVE-5490.1.patch.txt The query select substr(cstring1, 1, 0) from alltypesorc; returns all empty strings when set hive.vectorized.execution.enabled = false; which is the correct result, and returns non-empty strings when set hive.vectorized.execution.enabled = true; which is not correct. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-5490: - Attachment: HIVE-5490.1.patch.txt StringSubstrColStartLen#populateSubstrOffsets had an off-by-one error. I solved in a quick way, which returns an empty range when its given length is zero. SUBSTR(col, 1, 0) returns wrong result in vectorized mode - Key: HIVE-5490 URL: https://issues.apache.org/jira/browse/HIVE-5490 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Teddy Choi Attachments: HIVE-5490.1.patch.txt The query select substr(cstring1, 1, 0) from alltypesorc; returns all empty strings when set hive.vectorized.execution.enabled = false; which is the correct result, and returns non-empty strings when set hive.vectorized.execution.enabled = true; which is not correct. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790343#comment-13790343 ] Teddy Choi commented on HIVE-5490: -- Review request on https://reviews.apache.org/r/14558/ SUBSTR(col, 1, 0) returns wrong result in vectorized mode - Key: HIVE-5490 URL: https://issues.apache.org/jira/browse/HIVE-5490 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Teddy Choi Attachments: HIVE-5490.1.patch.txt The query select substr(cstring1, 1, 0) from alltypesorc; returns all empty strings when set hive.vectorized.execution.enabled = false; which is the correct result, and returns non-empty strings when set hive.vectorized.execution.enabled = true; which is not correct. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4850) Implement vectorized JOIN operators
[ https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-4850: --- Attachment: HIVE-4850.03.patch Added test query but my local environment does not pass trunk clean. Uploading to get a pre-commit build infra run on the patch. Implement vectorized JOIN operators --- Key: HIVE-4850 URL: https://issues.apache.org/jira/browse/HIVE-4850 Project: Hive Issue Type: Sub-task Reporter: Remus Rusanu Assignee: Remus Rusanu Attachments: HIVE-4850.03.patch, HIVE-4850.1.patch, HIVE-4850.2.patch Easysauce -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: Review Request 13059: HIVE-4850 Implement vector mode map join
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13059/ --- (Updated Oct. 9, 2013, 1:50 p.m.) Review request for hive, Eric Hanson and Jitendra Pandey. Bugs: HIVE-4850 https://issues.apache.org/jira/browse/HIVE-4850 Repository: hive-git Description --- This is a working implementation based on current trunk. It is simpler than the .1 patch in as it delegates the JOIN entirely to the row-mode MapJoinOperator. The vectorized operator is literally calling the row-mode implementaiton for each row in the input batch and collects the row-mode forward into the output batch. This is not as bad as it seems because the JOIN operators has to resort to row-mode operations anyway, due to the small tables (hashtables) being row-mode (objects and object-inspectors). By delegating the entire join logic to the row mode we piggyback on the correctness of exiting implementation. I do plan to come up with a full-vectorized mode implementation but that would require changes to the hash table creation-serialization. Note that the filtering and key evaluation of the big table does use vectorized operators. the row mode applies only to the key HT lookup and to the JOIN logic Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java d320b47 ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java 86db044 ql/src/java/org/apache/hadoop/hive/ql/exec/JoinUtil.java fa9ee35 ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 153b8ea ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 8ab5395 ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java cde1a59 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 8b4c615 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssign.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssignFactory.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorHashKeyWrapperBatch.java 9955d09 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorReduceSinkOperator.java 6df3551 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java 0fb763a ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java bd0955e ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java ff13f89 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorExpressionWriterFactory.java 9e189c9 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java df1c5a6 ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java a72ec8b ql/src/test/queries/clientpositive/vectorized_mapjoin.q PRE-CREATION ql/src/test/results/clientpositive/vectorized_mapjoin.q.out PRE-CREATION Diff: https://reviews.apache.org/r/13059/diff/ Testing --- Manually run some join queries on alltypes_orc table. Thanks, Remus Rusanu
[jira] [Commented] (HIVE-5484) TestSchemaTool failures when Hive version has more than 3 revision numbers
[
https://issues.apache.org/jira/browse/HIVE-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790378#comment-13790378
]
Hive QA commented on HIVE-5484:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12607452/HIVE-5484.2.patch
{color:green}SUCCESS:{color} +1 4390 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1084/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1084/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
TestSchemaTool failures when Hive version has more than 3 revision numbers
--
Key: HIVE-5484
URL: https://issues.apache.org/jira/browse/HIVE-5484
Project: Hive
Issue Type: Bug
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-5484.1.patch, HIVE-5484.2.patch
If Hive is created with a version string with more than 3 numbers, we end up
with a couple of test failures in TestSchemaTool, because the metastore is
expecting a version with the format of
majorVersion.minorVersion.changeVersion.
error message=Unknown version specified for initialization:
0.12.0.2.0.6.0-61
type=org.apache.hadoop.hive.metastore.HiveMetaExceptionorg.apache.hadoop.hive.metastore.HiveMetaException:
Unknown version specified for initialization: 0.12.0.2.0.6.0-61
at
org.apache.hadoop.hive.metastore.MetaStoreSchemaInfo.generateInitFileName(MetaStoreSchemaInfo.java:113)
at
org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:269)
at
org.apache.hive.beeline.src.test.TestSchemaTool.testSchemaInit(TestSchemaTool.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
/error
/testcase
testcase classname=org.apache.hive.beeline.src.test.TestSchemaTool
name=testSchemaUpgrade time=2.164
error message=Found unexpected schema version 0.12.0
type=org.apache.hadoop.hive.metastore.HiveMetaExceptionorg.apache.hadoop.hive.metastore.HiveMetaException:
Found unexpected schema version 0.12.0
at
org.apache.hive.beeline.HiveSchemaTool.verifySchemaVersion(HiveSchemaTool.java:192)
at
org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:242)
at
org.apache.hive.beeline.src.test.TestSchemaTool.testSchemaUpgrade(TestSchemaTool.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at
[jira] [Commented] (HIVE-5489) NOTICE copyright dates are out of date, README needs update
[ https://issues.apache.org/jira/browse/HIVE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790440#comment-13790440 ] Hudson commented on HIVE-5489: -- FAILURE: Integrated in Hive-trunk-h0.21 #2389 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2389/]) HIVE-5489 : NOTICE copyright dates are out of date, README needs update (Thejas M Nair via Carl Steinbach) (thejas: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530474) * /hive/trunk/NOTICE * /hive/trunk/README.txt NOTICE copyright dates are out of date, README needs update --- Key: HIVE-5489 URL: https://issues.apache.org/jira/browse/HIVE-5489 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Priority: Blocker Fix For: 0.12.0 Attachments: HIVE-5489.1.patch This needs to be updated for 0.12 release NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5488) some files are missing apache license headers
[ https://issues.apache.org/jira/browse/HIVE-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790439#comment-13790439 ] Hudson commented on HIVE-5488: -- FAILURE: Integrated in Hive-trunk-h0.21 #2389 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2389/]) HIVE-5488 : some files are missing apache license headers (Thejas M Nair via Ashutosh Chauhan) (thejas: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530473) * /hive/trunk/hbase-handler/src/test/templates/TestHBaseNegativeCliDriver.vm * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/BoundaryDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/CurrentRowDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/OrderDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/OrderExpressionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFExpressionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFInputDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFQueryInputDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PartitionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PartitionedTableFunctionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/RangeBoundaryDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/ShapeDetails.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/ValueBoundaryDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowExpressionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowFrameDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowFunctionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowTableFunctionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/GenericUDFDecode.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/GenericUDFEncode.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFBase64.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnbase64.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUnixTimeStamp.java * /hive/trunk/ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13OutputFormat.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFAbs.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDecode.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFEncode.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestToInteger.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFBase64.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFHex.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFUnbase64.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFUnhex.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableUnionObjectInspector1.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe3.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe4.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe5.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableUnionObjectInspector.java * /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/primitive/TestPrimitiveObjectInspectorUtils.java * /hive/trunk/service/src/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java * /hive/trunk/service/src/java/org/apache/hive/service/auth/TUGIContainingProcessor.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/AddResourceOperation.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/DeleteResourceOperation.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/DfsOperation.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SetOperation.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java * /hive/trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java *
Re: Review Request 14523: HIVE-5486 HiveServer2 should create base scratch directories at startup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14523/#review26820 --- Ship it! Ship It! - Brock Noland On Oct. 9, 2013, 6:34 a.m., Prasad Mujumdar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14523/ --- (Updated Oct. 9, 2013, 6:34 a.m.) Review request for hive. Bugs: HIVE-5486 https://issues.apache.org/jira/browse/HIVE-5486 Repository: hive-git Description --- With impersonation enabled, the same base directory is used by all sessions/queries. For a new deployment, this directory gets created on first invocation by the user running that session. This would cause directory permission conflict for other users. The patch is creating base scratch dirs at startup if it doesn't exist. Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d0895e1 service/src/java/org/apache/hive/service/cli/CLIService.java 1a7f338 service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java ae7bb6b service/src/test/org/apache/hive/service/cli/TestEmbeddedThriftBinaryCLIService.java da325da service/src/test/org/apache/hive/service/cli/TestScratchDir.java PRE-CREATION Diff: https://reviews.apache.org/r/14523/diff/ Testing --- Added new test Thanks, Prasad Mujumdar
[jira] [Commented] (HIVE-5486) HiveServer2 should create base scratch directories at startup
[ https://issues.apache.org/jira/browse/HIVE-5486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790523#comment-13790523 ] Brock Noland commented on HIVE-5486: New test passed: https://builds.apache.org/job/PreCommit-HIVE-Build/1083/testReport/org.apache.hive.service.cli/TestScratchDir/ +1 HiveServer2 should create base scratch directories at startup - Key: HIVE-5486 URL: https://issues.apache.org/jira/browse/HIVE-5486 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0, 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5486.2.patch, HIVE-5486.3.patch With impersonation enabled, the same base directory is used by all sessions/queries. For a new deployment, this directory gets created on first invocation by the user running that session. This would cause directory permission conflict for other users. HiveServer2 should create the base scratch dirs if it doesn't exist. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5252) Add ql syntax for inline java code creation
[ https://issues.apache.org/jira/browse/HIVE-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790528#comment-13790528 ] Brock Noland commented on HIVE-5252: LGTM but I don't understand the new ant dep in ql? What are thoughts on a negative test when the code has a syntax error? Add ql syntax for inline java code creation --- Key: HIVE-5252 URL: https://issues.apache.org/jira/browse/HIVE-5252 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: HIVE-5252.1.patch.txt Something to the effect of compile 'my code here' using 'groovycompiler'. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5488) some files are missing apache license headers
[ https://issues.apache.org/jira/browse/HIVE-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790532#comment-13790532 ] Hudson commented on HIVE-5488: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #197 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/197/]) HIVE-5488 : some files are missing apache license headers (Thejas M Nair via Ashutosh Chauhan) (thejas: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530473) * /hive/trunk/hbase-handler/src/test/templates/TestHBaseNegativeCliDriver.vm * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/BoundaryDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/CurrentRowDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/OrderDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/OrderExpressionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFExpressionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFInputDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFQueryInputDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PartitionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PartitionedTableFunctionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/RangeBoundaryDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/ShapeDetails.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/ValueBoundaryDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowExpressionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowFrameDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowFunctionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/WindowTableFunctionDef.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/GenericUDFDecode.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/GenericUDFEncode.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFBase64.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnbase64.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUnixTimeStamp.java * /hive/trunk/ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13OutputFormat.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFAbs.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDecode.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFEncode.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestToInteger.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFBase64.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFHex.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFUnbase64.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFUnhex.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableUnionObjectInspector1.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe3.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe4.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe5.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableUnionObjectInspector.java * /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/primitive/TestPrimitiveObjectInspectorUtils.java * /hive/trunk/service/src/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java * /hive/trunk/service/src/java/org/apache/hive/service/auth/TUGIContainingProcessor.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/AddResourceOperation.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/DeleteResourceOperation.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/DfsOperation.java * /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SetOperation.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java * /hive/trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java *
[jira] [Commented] (HIVE-5253) Create component to compile and jar dynamic code
[ https://issues.apache.org/jira/browse/HIVE-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790535#comment-13790535 ] Hudson commented on HIVE-5253: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #197 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/197/]) HIVE-5253: Create component to compile and jar dynamic code (Edward Capriolo via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530394) * /hive/trunk/eclipse-templates/.classpath * /hive/trunk/ivy/libraries.properties * /hive/trunk/ql/ivy.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/CompileProcessor.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/processors/TestCompileProcessor.java Create component to compile and jar dynamic code Key: HIVE-5253 URL: https://issues.apache.org/jira/browse/HIVE-5253 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Fix For: 0.13.0 Attachments: HIVE-5253.10.patch.txt, HIVE-5253.11.patch.txt, HIVE-5253.1.patch.txt, HIVE-5253.3.patch.txt, HIVE-5253.3.patch.txt, HIVE-5253.3.patch.txt, HIVE-5253.8.patch.txt, HIVE-5253.9.patch.txt, HIVE-5253.patch.txt -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5489) NOTICE copyright dates are out of date, README needs update
[ https://issues.apache.org/jira/browse/HIVE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790534#comment-13790534 ] Hudson commented on HIVE-5489: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #197 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/197/]) HIVE-5489 : NOTICE copyright dates are out of date, README needs update (Thejas M Nair via Carl Steinbach) (thejas: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530474) * /hive/trunk/NOTICE * /hive/trunk/README.txt NOTICE copyright dates are out of date, README needs update --- Key: HIVE-5489 URL: https://issues.apache.org/jira/browse/HIVE-5489 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Priority: Blocker Fix For: 0.12.0 Attachments: HIVE-5489.1.patch This needs to be updated for 0.12 release NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5478) WebHCat e2e testsuite for hcat authorization tests needs some fixes
[ https://issues.apache.org/jira/browse/HIVE-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790536#comment-13790536 ] Hudson commented on HIVE-5478: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #197 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/197/]) HIVE-5478 : WebHCat e2e testsuite for hcat authorization tests needs some fixes (Deepesh Khandelwal via Thejas Nair) (thejas: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530384) * /hive/trunk/hcatalog/src/test/e2e/templeton/build.xml * /hive/trunk/hcatalog/src/test/e2e/templeton/tests/hcatperms.conf WebHCat e2e testsuite for hcat authorization tests needs some fixes --- Key: HIVE-5478 URL: https://issues.apache.org/jira/browse/HIVE-5478 Project: Hive Issue Type: Bug Components: Tests, WebHCat Affects Versions: 0.12.0 Reporter: Deepesh Khandelwal Assignee: Deepesh Khandelwal Fix For: 0.13.0 Attachments: HIVE-5478.1.patch, HIVE-5478.patch Here are the issues: 1. The HARNESS_ROOT in the test-hcat-authorization testsuite needs to be testdist root otherwise the ant command fails to look for resource/default.res. 2. A few tests DB_OPS_5 and TABLE_OPS_2 were relying on default permissions on the hive warehouse directory which can vary based on the environment, improved the test to check what is set. 3. DB_OPS_18 error message is old, now we get a more specific message, updated to verify the new one. NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5460) invalid offsets in lag lead should return an exception (per ISO-SQL)
[ https://issues.apache.org/jira/browse/HIVE-5460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790533#comment-13790533 ] Hudson commented on HIVE-5460: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #197 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/197/]) HIVE-5460: invalid offsets in lag lead should return an exception (per ISO-SQL) (Edward Capriolo via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530393) * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingExprNodeEvaluatorFactory.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFLeadLag.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLag.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLead.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLeadLag.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/LeadLagBuffer.java * /hive/trunk/ql/src/test/queries/clientnegative/windowing_ll_no_neg.q * /hive/trunk/ql/src/test/results/clientnegative/windowing_ll_no_neg.q.out invalid offsets in lag lead should return an exception (per ISO-SQL) - Key: HIVE-5460 URL: https://issues.apache.org/jira/browse/HIVE-5460 Project: Hive Issue Type: Bug Affects Versions: 0.11.0 Reporter: N Campbell Assignee: Edward Capriolo Priority: Minor Fix For: 0.13.0 Attachments: HIVE-5460.1.patch.txt ISO-SQL 2011 defines how lag and lead should behave when invalid offsets are provided to the functions. i.e. select tint.rnum,tint.cint, lag( tint.cint, -100 ) over ( order by tint.rnum) from tint tint Instead of a meaningful error (as other vendors will emit) you get Error: Query returned non-zero code: 2, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask SQLState: 08S01 ErrorCode: 2 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5252) Add ql syntax for inline java code creation
[ https://issues.apache.org/jira/browse/HIVE-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790531#comment-13790531 ] Edward Capriolo commented on HIVE-5252: --- Groovyc (groovy compiler) requires ant. Ant is on our classpath for development but we need to add it as a ql. dep because otherwise it does not get added to hive/lib in the package. Add ql syntax for inline java code creation --- Key: HIVE-5252 URL: https://issues.apache.org/jira/browse/HIVE-5252 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: HIVE-5252.1.patch.txt Something to the effect of compile 'my code here' using 'groovycompiler'. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5493) duplicate jars with different versions for guava, commons-logging
[ https://issues.apache.org/jira/browse/HIVE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790537#comment-13790537 ] Hudson commented on HIVE-5493: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #197 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/197/]) HIVE-5493 : duplicate jars with different versions for guava, commons-logging (Thejas Nair via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530518) * /hive/trunk/ivy/libraries.properties * /hive/trunk/metastore/ivy.xml duplicate jars with different versions for guava, commons-logging - Key: HIVE-5493 URL: https://issues.apache.org/jira/browse/HIVE-5493 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Priority: Blocker Fix For: 0.13.0 Attachments: HIVE-5493.1.patch Duplicate jars with different versions for guava and commons-logging are present in build/dist/lib . hive should ship with just one version of each jar. guava-r08 and guava-0.11.0.2 are present, and commons-logging has 1.0.4 and 1.1.1 . -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5476) Authorization-provider tests fail in sequential run
[ https://issues.apache.org/jira/browse/HIVE-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790538#comment-13790538 ] Hudson commented on HIVE-5476: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #197 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/197/]) HIVE-5476 : Authorization-provider tests fail in sequential run (Sushanth Sowmyan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1530516) * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/security/TestClientSideAuthorizationProvider.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/security/TestMetastoreAuthorizationProvider.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/security/TestStorageBasedClientSideAuthorizationProvider.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/security/TestStorageBasedMetastoreAuthorizationProvider.java Authorization-provider tests fail in sequential run --- Key: HIVE-5476 URL: https://issues.apache.org/jira/browse/HIVE-5476 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Sushanth Sowmyan Fix For: 0.13.0 Attachments: HIVE-5476.2.patch, HIVE-5476.patch As seen in 0.12 build with hadoop1 - https://builds.apache.org/job/Hive-branch-0.12-hadoop1/lastCompletedBuild/testReport/ Following tests fail - org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedClientSideAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-2436) Update project naming and description in Hive website
[ https://issues.apache.org/jira/browse/HIVE-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-2436: --- Resolution: Fixed Status: Resolved (was: Patch Available) Carl thank you for the review! I have committed this. Update project naming and description in Hive website - Key: HIVE-2436 URL: https://issues.apache.org/jira/browse/HIVE-2436 Project: Hive Issue Type: Sub-task Reporter: John Sichi Assignee: Brock Noland Attachments: HIVE-2436.patch http://www.apache.org/foundation/marks/pmcs.html#naming -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5500) Update my username on credits page
[ https://issues.apache.org/jira/browse/HIVE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5500: --- Description: My apache username is brock not brocknoland NO PRECOMMIT TESTS was:My apache username is brock not brocknoland Update my username on credits page -- Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Priority: Minor My apache username is brock not brocknoland NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HIVE-5500) Update my username on credits page
Brock Noland created HIVE-5500: -- Summary: Update my username on credits page Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Priority: Minor My apache username is brock not brocknoland -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5484) TestSchemaTool failures when Hive version has more than 3 revision numbers
[ https://issues.apache.org/jira/browse/HIVE-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-5484: --- Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Jason! TestSchemaTool failures when Hive version has more than 3 revision numbers -- Key: HIVE-5484 URL: https://issues.apache.org/jira/browse/HIVE-5484 Project: Hive Issue Type: Bug Reporter: Jason Dere Assignee: Jason Dere Fix For: 0.13.0 Attachments: HIVE-5484.1.patch, HIVE-5484.2.patch If Hive is created with a version string with more than 3 numbers, we end up with a couple of test failures in TestSchemaTool, because the metastore is expecting a version with the format of majorVersion.minorVersion.changeVersion. error message=Unknown version specified for initialization: 0.12.0.2.0.6.0-61 type=org.apache.hadoop.hive.metastore.HiveMetaExceptionorg.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for initialization: 0.12.0.2.0.6.0-61 at org.apache.hadoop.hive.metastore.MetaStoreSchemaInfo.generateInitFileName(MetaStoreSchemaInfo.java:113) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:269) at org.apache.hive.beeline.src.test.TestSchemaTool.testSchemaInit(TestSchemaTool.java:104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) /error /testcase testcase classname=org.apache.hive.beeline.src.test.TestSchemaTool name=testSchemaUpgrade time=2.164 error message=Found unexpected schema version 0.12.0 type=org.apache.hadoop.hive.metastore.HiveMetaExceptionorg.apache.hadoop.hive.metastore.HiveMetaException: Found unexpected schema version 0.12.0 at org.apache.hive.beeline.HiveSchemaTool.verifySchemaVersion(HiveSchemaTool.java:192) at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:242) at org.apache.hive.beeline.src.test.TestSchemaTool.testSchemaUpgrade(TestSchemaTool.java:128) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) /error -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5500) Update my username on credits page
[ https://issues.apache.org/jira/browse/HIVE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790543#comment-13790543 ] Brock Noland commented on HIVE-5500: cc: [~cwsteinbach] Update my username on credits page -- Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Priority: Minor Attachments: HIVE-5500.patch My apache username is brock not brocknoland NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5500) Update my username on credits page
[ https://issues.apache.org/jira/browse/HIVE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5500: --- Attachment: HIVE-5500.patch Update my username on credits page -- Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Priority: Minor Attachments: HIVE-5500.patch My apache username is brock not brocknoland NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5492) Explain query fails with NPE if a client doesn't call getResultSetSchema()
[
https://issues.apache.org/jira/browse/HIVE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790546#comment-13790546
]
Brock Noland commented on HIVE-5492:
+1
Explain query fails with NPE if a client doesn't call getResultSetSchema()
--
Key: HIVE-5492
URL: https://issues.apache.org/jira/browse/HIVE-5492
Project: Hive
Issue Type: Bug
Components: Thrift API
Affects Versions: 0.11.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Fix For: 0.13.0
Attachments: HIVE-5492.patch
If a thrift client makes an explain query and fetches results without calling
getResultSetSchema() first, NullPointerException will occur in Hive.
{code}
org.apache.hive.service.cli.HiveSQLException: java.lang.NullPointerException
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:262)
at
org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
at
org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:422)
at
org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:333)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:413)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TUGIContainingProcessor$1.run(TUGIContainingProcessor.java:40)
at
org.apache.hive.service.auth.TUGIContainingProcessor$1.run(TUGIContainingProcessor.java:37)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:527)
at
org.apache.hive.service.auth.TUGIContainingProcessor.process(TUGIContainingProcessor.java:37)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.NullPointerException
at org.apache.hive.service.cli.Row.init(Row.java:45)
at org.apache.hive.service.cli.RowSet.addRow(RowSet.java:61)
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:254)
... 19 more
{code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-2436) Update project naming and description in Hive website
[ https://issues.apache.org/jira/browse/HIVE-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2436: --- Fix Version/s: 0.13.0 Update project naming and description in Hive website - Key: HIVE-2436 URL: https://issues.apache.org/jira/browse/HIVE-2436 Project: Hive Issue Type: Sub-task Reporter: John Sichi Assignee: Brock Noland Fix For: 0.13.0 Attachments: HIVE-2436.patch http://www.apache.org/foundation/marks/pmcs.html#naming -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790552#comment-13790552 ] Eric Hanson commented on HIVE-5490: --- +1 Thanks Teddy! SUBSTR(col, 1, 0) returns wrong result in vectorized mode - Key: HIVE-5490 URL: https://issues.apache.org/jira/browse/HIVE-5490 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Teddy Choi Attachments: HIVE-5490.1.patch.txt The query select substr(cstring1, 1, 0) from alltypesorc; returns all empty strings when set hive.vectorized.execution.enabled = false; which is the correct result, and returns non-empty strings when set hive.vectorized.execution.enabled = true; which is not correct. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-2436) Update project naming and description in Hive website
[ https://issues.apache.org/jira/browse/HIVE-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790557#comment-13790557 ] Brock Noland commented on HIVE-2436: Thanks [~ashutoshc] for updating the fix version. I wasn't sure if website updates should have a fix version. Update project naming and description in Hive website - Key: HIVE-2436 URL: https://issues.apache.org/jira/browse/HIVE-2436 Project: Hive Issue Type: Sub-task Reporter: John Sichi Assignee: Brock Noland Fix For: 0.13.0 Attachments: HIVE-2436.patch http://www.apache.org/foundation/marks/pmcs.html#naming -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.util.HostUtil for Hive 0.10.0
thats why mapreduce.util.HostUtil is from mr2, and you use (i think) mr1,
hive has dummy code :
common/java/org/apache/hadoop/hive/shims/ShimLoader.java -
String getMajorVersion() {
String vers = VersionInfo.getVersion();
thus you gets something like : 2.0.0-cdh4.1.0, instead you use old hadoop
for example : 2.0.0-mr1-cdh4.1.0
i will write to cloudera to fix it, but simple fix is to fake:
HADOOP_SHIM_CLASSES.put(0.23,
org.apache.hadoop.hive.shims.Hadoop20SShims);
not
HADOOP_SHIM_CLASSES.put(0.23,
org.apache.hadoop.hive.shims.Hadoop23Shims);
среда, 27 февраля 2013 г., 6:39:36 UTC+4 пользователь Eric Chu написал:
(+hue-user since this issue prevents me from successfully installing Hue
from source)
Hi,
I recently did the following with both the Hive-0.10 and Hive-0.9, and had
a problem with 0.10 that I didn't see with 0.9
- Checked out the respective branch from github
- Did an ant package
- Copied the dist folder to /usr/lib/hive on the right machine
- Copied mysql-connector-java-5.1.22-bin.jar to /usr/lib/hive/lib
- Configured /etc/hive/conf (so same for both versions)
The problem is that when I use Hive-0.10, doing a select count(1) from
table (or anything that requires MR) would return a NoClassDefFound error
(see *Error Msg *below), whereas when I use Hive-0.9, the job would run
fine. Has anyone run into this problem? I can't use Hive-0.9 b/c it has
libthrift-0.7.0.jar, while Hue-2.2 (which I'm also using) requires
libthrift-0.9.0.jar (available in Hive-10).
Any insights would be much appreciated. Googling on this error doesn't get
very far. Thanks!
*Error Msg:*
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=number
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=number
In order to set a constant number of reducers:
set mapred.reduce.tasks=number
Starting Job = job_201302201756_0009, Tracking URL =
http://master-hadoop.pww-arp-dev.rfiserve.net:50030/jobdetails.jsp?jobid=job_201302201756_0009
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_201302201756_0009
Hadoop job information for Stage-1: number of mappers: 1; number of
reducers: 1
2013-02-26 21:05:01,060 Stage-1 map = 0%, reduce = 0%
2013-02-26 21:05:32,286 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201302201756_0009 with errors
Error during job, obtaining debugging information...
Job Tracking URL:
http://master-hadoop.pww-arp-dev.rfiserve.net:50030/jobdetails.jsp?jobid=job_201302201756_0009
Examining task ID: task_201302201756_0009_m_02 (and more) from job
job_201302201756_0009
Exception in thread Thread-29 java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/util/HostUtil
at
org.apache.hadoop.hive.shims.Hadoop23Shims.getTaskAttemptLogUrl(Hadoop23Shims.java:53)
at
org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.getTaskInfos(JobDebugger.java:186)
at
org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.run(JobDebugger.java:142)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.mapreduce.util.HostUtil
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 4 more
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
[jira] [Updated] (HIVE-4898) make vectorized math functions work end-to-end (update VectorizationContext.java)
[ https://issues.apache.org/jira/browse/HIVE-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-4898: -- Attachment: HIVE-4898.4.patch Rebased the patch on current trunk. make vectorized math functions work end-to-end (update VectorizationContext.java) - Key: HIVE-4898 URL: https://issues.apache.org/jira/browse/HIVE-4898 Project: Hive Issue Type: Sub-task Affects Versions: vectorization-branch Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-4898.3.patch, HIVE-4898.3.patch, HIVE-4898.4.patch The vectorized math function VectorExpression classes were added in HIVE-4822. This JIRA is to allow those to actually be used in a SQL query end-to-end. This requires updating VectorizationContext to use the new classes in vectorized expression creation. -- This message was sent by Atlassian JIRA (v6.1#6144)
Single Mapper - HIVE 0.11
Hi, I am trying to run a join using two tables stored in ORC file format. The first table has 34 million records and the second has around 300,000 records. Setting set hive.auto.convert.join=true makes the entire query run via a single mapper. In case I am setting set hive.auto.convert.join=false then there are two mappers first one reads the second table and then the entire large table goes through the second mapper. Is there something that I am doing wrong because there are three nodes in the HADOOP cluster currently and I was expecting that at least 6 mappers should have been used. Thanks and Regards, Gourav
[jira] [Commented] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790613#comment-13790613 ] Timothy Chen commented on HIVE-5490: Nice! was just planning to look at it today and it's already done! Thanks for fixing this Teddy. SUBSTR(col, 1, 0) returns wrong result in vectorized mode - Key: HIVE-5490 URL: https://issues.apache.org/jira/browse/HIVE-5490 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Teddy Choi Attachments: HIVE-5490.1.patch.txt The query select substr(cstring1, 1, 0) from alltypesorc; returns all empty strings when set hive.vectorized.execution.enabled = false; which is the correct result, and returns non-empty strings when set hive.vectorized.execution.enabled = true; which is not correct. -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: Single Mapper - HIVE 0.11
whats the size of the table? (in GBs? ) Whats the max and min split sizes have you provied? On Wed, Oct 9, 2013 at 10:28 PM, Gourav Sengupta gourav.had...@gmail.comwrote: Hi, I am trying to run a join using two tables stored in ORC file format. The first table has 34 million records and the second has around 300,000 records. Setting set hive.auto.convert.join=true makes the entire query run via a single mapper. In case I am setting set hive.auto.convert.join=false then there are two mappers first one reads the second table and then the entire large table goes through the second mapper. Is there something that I am doing wrong because there are three nodes in the HADOOP cluster currently and I was expecting that at least 6 mappers should have been used. Thanks and Regards, Gourav -- Nitin Pawar
[jira] [Created] (HIVE-5501) Filter on partitioning column shouldn't be present at execution time
Ashutosh Chauhan created HIVE-5501: -- Summary: Filter on partitioning column shouldn't be present at execution time Key: HIVE-5501 URL: https://issues.apache.org/jira/browse/HIVE-5501 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Ashutosh Chauhan Since such filters are already processed via partitioning pruning, having it present in operator pipeline is unnecessary overhead. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5501) Filter on partitioning column shouldn't be present at execution time
[
https://issues.apache.org/jira/browse/HIVE-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790624#comment-13790624
]
Ashutosh Chauhan commented on HIVE-5501:
{code}
hive create table t3 (a string, b int) partitioned by (p1 string);
hive explain select count(*) from t3 where p1='3';
OK
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME t3))) (TOK_INSERT
(TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR
(TOK_FUNCTIONSTAR count))) (TOK_WHERE (= (TOK_TABLE_OR_COL p1) '3'
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias - Map Operator Tree:
t3
TableScan
alias: t3
Filter Operator
predicate:
expr: (p1 = '3')
type: boolean
Select Operator
Group By Operator
aggregations:
expr: count()
bucketGroup: false
mode: hash
outputColumnNames: _col0
Reduce Output Operator
sort order:
tag: -1
value expressions:
expr: _col0
type: bigint
Reduce Operator Tree:
Group By Operator
aggregations:
expr: count(VALUE._col0)
bucketGroup: false
mode: mergepartial
outputColumnNames: _col0
Select Operator
expressions:
expr: _col0
type: bigint
outputColumnNames: _col0
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
{code}
Filter marked by is not useful at execution time and should have been
eliminated by optimizer since partitioning pruning will already take care of it.
Filter on partitioning column shouldn't be present at execution time
Key: HIVE-5501
URL: https://issues.apache.org/jira/browse/HIVE-5501
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Ashutosh Chauhan
Since such filters are already processed via partitioning pruning, having it
present in operator pipeline is unnecessary overhead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
Re: Single Mapper - HIVE 0.11
What is your ORC file stripe size? How many ORC files are there in each of the tables? It could be possible that ORC compressed the file so much that the file size is less than the HDFS block size. Can you please report the file size of the two ORC files? Another possibility is that there are many small files. In that case by default hive uses CombineHiveInputFormat which combines many small files into a single large file. Hence you will see less number of mappers. If you are expecting one mapper per hdfs file, then try disabling CombineHiveInputFormat by set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;. Another way to control the number of mappers is by adjusting the min and max split size. Thanks Prasanth Jayachandran On Oct 9, 2013, at 10:03 AM, Nitin Pawar nitinpawar...@gmail.com wrote: whats the size of the table? (in GBs? ) Whats the max and min split sizes have you provied? On Wed, Oct 9, 2013 at 10:28 PM, Gourav Sengupta gourav.had...@gmail.comwrote: Hi, I am trying to run a join using two tables stored in ORC file format. The first table has 34 million records and the second has around 300,000 records. Setting set hive.auto.convert.join=true makes the entire query run via a single mapper. In case I am setting set hive.auto.convert.join=false then there are two mappers first one reads the second table and then the entire large table goes through the second mapper. Is there something that I am doing wrong because there are three nodes in the HADOOP cluster currently and I was expecting that at least 6 mappers should have been used. Thanks and Regards, Gourav -- Nitin Pawar -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-5500) Update my username on credits page
[ https://issues.apache.org/jira/browse/HIVE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790641#comment-13790641 ] Carl Steinbach commented on HIVE-5500: -- +1 Update my username on credits page -- Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Priority: Minor Attachments: HIVE-5500.patch My apache username is brock not brocknoland NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4850) Implement vectorized JOIN operators
[
https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790654#comment-13790654
]
Hive QA commented on HIVE-4850:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12607559/HIVE-4850.03.patch
{color:red}ERROR:{color} -1 due to 70 failed/errored test(s), 4363 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_filters
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_smb_mapjoin_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_joins
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_column_access_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_join1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_sets4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join0
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join18_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_1to1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_casesensitive
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters_overlap
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_nulls
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_nullsafe
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_leftsemijoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_louter_join_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_no_hooks
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nonblock_op_deduplicate
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_outer_join_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_udf_case
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_vc
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_print_header
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_router_join_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt17
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt18
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_6
[jira] [Created] (HIVE-5502) ORC TestFileDump is flaky
Brock Noland created HIVE-5502:
--
Summary: ORC TestFileDump is flaky
Key: HIVE-5502
URL: https://issues.apache.org/jira/browse/HIVE-5502
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Priority: Minor
I found in my maven work that TestFileDump is non-deterministic. For example
sometimes the output ORC file is much larger
{noformat}
pass:
-rwxrwxrwx 1 brock brock 290055 Oct 9 12:02 TestFileDump.testDump.orc
fail:
-rwxrwxrwx 1 brock brock 1938634 Oct 9 12:08 TestFileDump.testDump.orc
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5502) ORC TestFileDump is flaky
[
https://issues.apache.org/jira/browse/HIVE-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brock Noland updated HIVE-5502:
---
Attachment: TestFileDump.tar.gz
Attached is the output of a failed and passing test.
ORC TestFileDump is flaky
-
Key: HIVE-5502
URL: https://issues.apache.org/jira/browse/HIVE-5502
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Priority: Minor
Attachments: TestFileDump.tar.gz
I found in my maven work that TestFileDump is non-deterministic. For example
sometimes the output ORC file is much larger
{noformat}
pass:
-rwxrwxrwx 1 brock brock 290055 Oct 9 12:02 TestFileDump.testDump.orc
fail:
-rwxrwxrwx 1 brock brock 1938634 Oct 9 12:08 TestFileDump.testDump.orc
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Assigned] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair reassigned HIVE-5497:
---
Assignee: Thejas M Nair
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5493) duplicate jars with different versions for guava, commons-logging
[ https://issues.apache.org/jira/browse/HIVE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5493: Fix Version/s: (was: 0.13.0) 0.12.0 duplicate jars with different versions for guava, commons-logging - Key: HIVE-5493 URL: https://issues.apache.org/jira/browse/HIVE-5493 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Priority: Blocker Fix For: 0.12.0 Attachments: HIVE-5493.1.patch Duplicate jars with different versions for guava and commons-logging are present in build/dist/lib . hive should ship with just one version of each jar. guava-r08 and guava-0.11.0.2 are present, and commons-logging has 1.0.4 and 1.1.1 . -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5493) duplicate jars with different versions for guava, commons-logging
[ https://issues.apache.org/jira/browse/HIVE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790693#comment-13790693 ] Thejas M Nair commented on HIVE-5493: - Patch committed to 0.12 branch. duplicate jars with different versions for guava, commons-logging - Key: HIVE-5493 URL: https://issues.apache.org/jira/browse/HIVE-5493 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Priority: Blocker Fix For: 0.12.0 Attachments: HIVE-5493.1.patch Duplicate jars with different versions for guava and commons-logging are present in build/dist/lib . hive should ship with just one version of each jar. guava-r08 and guava-0.11.0.2 are present, and commons-logging has 1.0.4 and 1.1.1 . -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5502) ORC TestFileDump is flaky
[
https://issues.apache.org/jira/browse/HIVE-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790696#comment-13790696
]
Prasanth J commented on HIVE-5502:
--
[~brocknoland] I think this is because of the difference in JVM heap size. More
JVM heap size will generate lesser number of stripes and hence the file dump
information will be less. Can you please try using the same heap size (probably
with same ANT_OPTS and MAVEN_OPTS) and see if this issue happens again?
ORC TestFileDump is flaky
-
Key: HIVE-5502
URL: https://issues.apache.org/jira/browse/HIVE-5502
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Priority: Minor
Attachments: TestFileDump.tar.gz
I found in my maven work that TestFileDump is non-deterministic. For example
sometimes the output ORC file is much larger
{noformat}
pass:
-rwxrwxrwx 1 brock brock 290055 Oct 9 12:02 TestFileDump.testDump.orc
fail:
-rwxrwxrwx 1 brock brock 1938634 Oct 9 12:08 TestFileDump.testDump.orc
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (HIVE-5503) TopN optimization in VectorReduceSink
Jitendra Nath Pandey created HIVE-5503: -- Summary: TopN optimization in VectorReduceSink Key: HIVE-5503 URL: https://issues.apache.org/jira/browse/HIVE-5503 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Sergey Shelukhin We need to add TopN optimization to VectorReduceSink as well, it would be great if ReduceSink and VectorReduceSink share this code. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4850) Implement vectorized JOIN operators
[ https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-4850: --- Status: Open (was: Patch Available) Implement vectorized JOIN operators --- Key: HIVE-4850 URL: https://issues.apache.org/jira/browse/HIVE-4850 Project: Hive Issue Type: Sub-task Reporter: Remus Rusanu Assignee: Remus Rusanu Attachments: HIVE-4850.03.patch, HIVE-4850.1.patch, HIVE-4850.2.patch Easysauce -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5502) ORC TestFileDump is flaky
[
https://issues.apache.org/jira/browse/HIVE-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790703#comment-13790703
]
Brock Noland commented on HIVE-5502:
Hey [~prasanth_j] this is generated by running the test twice in a row in the
exact same environment. Meaning I executed these exact commands in succession:
{noformat}
$ rm -rf /tmp/fail; mkdir /tmp/fail; while mvn -o clean test
-Dtest=TestFileDump; do ls -l target/tmp/TestFileDump.testDump.orc;done; cp -R
target/* /tmp/fail
$ rm -rf /tmp/pass; mkdir /tmp/pass; while ! mvn -o clean test
-Dtest=TestFileDump; do ls -l target/tmp/TestFileDump.testDump.orc;done; cp -R
target/* /tmp/pass
{noformat}
to generate my data and and the failures. Note however the above commands won't
work on the maven branch since I am working on a patch to fix the branch post
vectorization.
ORC TestFileDump is flaky
-
Key: HIVE-5502
URL: https://issues.apache.org/jira/browse/HIVE-5502
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Priority: Minor
Attachments: TestFileDump.tar.gz
I found in my maven work that TestFileDump is non-deterministic. For example
sometimes the output ORC file is much larger
{noformat}
pass:
-rwxrwxrwx 1 brock brock 290055 Oct 9 12:02 TestFileDump.testDump.orc
fail:
-rwxrwxrwx 1 brock brock 1938634 Oct 9 12:08 TestFileDump.testDump.orc
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (HIVE-5504) HCatOutputFormat does not honor orc.compress tblproperty
Venkat Ranganathan created HIVE-5504: Summary: HCatOutputFormat does not honor orc.compress tblproperty Key: HIVE-5504 URL: https://issues.apache.org/jira/browse/HIVE-5504 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.11.0, 0.12.0 Reporter: Venkat Ranganathan When we import data into a HCatalog table created with the following storage description .. stored as orc tblproperties (orc.compress=SNAPPY) the resultant orc file still uses the default zlib compression It looks like HCatOutputFormat is ignoring the tblproperties specified. show tblproperties shows that the table indeed has the properties properly saved. An insert/select into the table has the resulting orc file honor the tbl property. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790715#comment-13790715
]
Thejas M Nair commented on HIVE-5497:
-
Yes, its related to that change, but in that patch the call was being made only
after checking if(ShimLoader.getHadoopShims().isSecurityEnabled() - In this
change, the call was made without the check -
https://github.com/apache/hive/commit/adfc51b807953e69164c49a6bfd470b423d67cb0
I will add the check in the new place as well .
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790729#comment-13790729
]
Brock Noland commented on HIVE-5497:
Good find! Sorry about that.
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5500) Update my username on credits page
[ https://issues.apache.org/jira/browse/HIVE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5500: --- Fix Version/s: 0.13.0 Update my username on credits page -- Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Assignee: Brock Noland Priority: Minor Fix For: 0.13.0 Attachments: HIVE-5500.patch My apache username is brock not brocknoland NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (HIVE-5500) Update my username on credits page
[ https://issues.apache.org/jira/browse/HIVE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland resolved HIVE-5500. Resolution: Fixed Assignee: Brock Noland Thank you very much! Update my username on credits page -- Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Assignee: Brock Noland Priority: Minor Attachments: HIVE-5500.patch My apache username is brock not brocknoland NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5500) Update Brock Noland's username on credits page
[ https://issues.apache.org/jira/browse/HIVE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5500: --- Summary: Update Brock Noland's username on credits page (was: Update my username on credits page) Update Brock Noland's username on credits page -- Key: HIVE-5500 URL: https://issues.apache.org/jira/browse/HIVE-5500 Project: Hive Issue Type: Task Reporter: Brock Noland Assignee: Brock Noland Priority: Minor Fix For: 0.13.0 Attachments: HIVE-5500.patch My apache username is brock not brocknoland NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[
https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790740#comment-13790740
]
Hive QA commented on HIVE-5490:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12607555/HIVE-5490.1.patch.txt
{color:green}SUCCESS:{color} +1 4390 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1086/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1086/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
SUBSTR(col, 1, 0) returns wrong result in vectorized mode
-
Key: HIVE-5490
URL: https://issues.apache.org/jira/browse/HIVE-5490
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Eric Hanson
Assignee: Teddy Choi
Attachments: HIVE-5490.1.patch.txt
The query
select substr(cstring1, 1, 0) from alltypesorc;
returns all empty strings when
set hive.vectorized.execution.enabled = false;
which is the correct result, and returns non-empty strings when
set hive.vectorized.execution.enabled = true;
which is not correct.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5476) Authorization-provider tests fail in sequential run
[ https://issues.apache.org/jira/browse/HIVE-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790744#comment-13790744 ] Thejas M Nair commented on HIVE-5476: - Patch committed to 0.12 branch Authorization-provider tests fail in sequential run --- Key: HIVE-5476 URL: https://issues.apache.org/jira/browse/HIVE-5476 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Sushanth Sowmyan Fix For: 0.12.0 Attachments: HIVE-5476.2.patch, HIVE-5476.patch As seen in 0.12 build with hadoop1 - https://builds.apache.org/job/Hive-branch-0.12-hadoop1/lastCompletedBuild/testReport/ Following tests fail - org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedClientSideAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5476) Authorization-provider tests fail in sequential run
[ https://issues.apache.org/jira/browse/HIVE-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5476: Fix Version/s: (was: 0.13.0) 0.12.0 Authorization-provider tests fail in sequential run --- Key: HIVE-5476 URL: https://issues.apache.org/jira/browse/HIVE-5476 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Sushanth Sowmyan Fix For: 0.12.0 Attachments: HIVE-5476.2.patch, HIVE-5476.patch As seen in 0.12 build with hadoop1 - https://builds.apache.org/job/Hive-branch-0.12-hadoop1/lastCompletedBuild/testReport/ Following tests fail - org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedClientSideAuthorizationProvider.testSimplePrivileges 12 sec 1 org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges 12 sec 1 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-5497:
Status: Patch Available (was: Open)
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-5497.1.patch
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-5497:
Attachment: HIVE-5497.1.patch
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-5497.1.patch
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790759#comment-13790759
]
Brock Noland commented on HIVE-5497:
+1
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-5497.1.patch
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790768#comment-13790768
]
Thejas M Nair commented on HIVE-5497:
-
Thanks for the prompt review Brock!
I hope we can make an exception to the 24 hr embargo for this one, as this is
the last piece needed for new RC. It is also a minor change.
I am planning to wait for a few hours for the unit tests to finish.
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-5497.1.patch
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5335) Milestone 4: Most tests pass under maven
[ https://issues.apache.org/jira/browse/HIVE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5335: --- Description: NO PRECOMMIT TESTS Milestone 4: Most tests pass under maven Key: HIVE-5335 URL: https://issues.apache.org/jira/browse/HIVE-5335 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Attachments: HIVE-5335-no-qfiles.patch, HIVE-5335.patch NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5335) Milestone 4: Most tests pass under maven
[ https://issues.apache.org/jira/browse/HIVE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5335: --- Attachment: HIVE-5335-no-qfiles.patch HIVE-5335.patch Attached are two patches. The larger one contains the qfile updates required while the smaller one has no qfile updates and thus should be easier to review. The big change here is that this patch breaks the itests package into three separate module util, unit, and qfile. The unit module contains items such as QTestUtil and HBaseSetup while unit contains any unit tests which had to be moved to the itests module due to circular dependencies. The qfile module contains all the qfile tests. Another change in this patch is that the itests module is not part of the hive project. It inherits the pom but but is build separately. This is because it depends on the jars having been built. There are other ways to this and we can revisit this in the future if people would like. The smallest of the large changes is that a bunch more metastore unit tests have been moved to itests/unit because they depend on the ql module. The following modules have all their tests passing: common serde service ql hwi cli beeline contrib metastore hbase-handler and most of the qfile tests pass. The nest big work item is fix the hcatalog tests and the remaining qfile tests. Milestone 4: Most tests pass under maven Key: HIVE-5335 URL: https://issues.apache.org/jira/browse/HIVE-5335 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Attachments: HIVE-5335-no-qfiles.patch, HIVE-5335.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5335) Milestone 4: Most tests pass under maven
[ https://issues.apache.org/jira/browse/HIVE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5335: --- Assignee: Brock Noland Status: Patch Available (was: Open) Milestone 4: Most tests pass under maven Key: HIVE-5335 URL: https://issues.apache.org/jira/browse/HIVE-5335 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-5335-no-qfiles.patch, HIVE-5335.patch NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5335) Milestone 4: Most tests pass under maven
[ https://issues.apache.org/jira/browse/HIVE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790775#comment-13790775 ] Brock Noland commented on HIVE-5335: Ahh shoot. I meant to say this is ready for commit. HCatalog and future qfile tests will be addressed in future jiras. Milestone 4: Most tests pass under maven Key: HIVE-5335 URL: https://issues.apache.org/jira/browse/HIVE-5335 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-5335-no-qfiles.patch, HIVE-5335.patch NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5335) Milestone 4: Most tests pass under maven
[ https://issues.apache.org/jira/browse/HIVE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790773#comment-13790773 ] Brock Noland commented on HIVE-5335: Additionally the hcatalog tests write files all over hell. It'd be great o get those into target/. Milestone 4: Most tests pass under maven Key: HIVE-5335 URL: https://issues.apache.org/jira/browse/HIVE-5335 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-5335-no-qfiles.patch, HIVE-5335.patch NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5497) Hive trunk broken against hadoop 0.20.2
[
https://issues.apache.org/jira/browse/HIVE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790776#comment-13790776
]
Brock Noland commented on HIVE-5497:
Yes I think we can. I won't be around when the unit tests finish so feel free
to commit in my absence.
Hive trunk broken against hadoop 0.20.2
---
Key: HIVE-5497
URL: https://issues.apache.org/jira/browse/HIVE-5497
Project: Hive
Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-5497.1.patch
ommon-0.13.0-SNAPSHOT.jar!/hive-log4j.properties
hive compile `import org.apache.hadoop.hive.ql.exec.UDF \;
public class Pyth extends UDF {
public double evaluate(double a, double b){
return Math.sqrt((a*a) + (b*b)) \;
}
} ` AS GROOVY NAMED Pyth.groovy;
Added /tmp/0_1381290655403.jar to class path
Added resource: /tmp/0_1381290655403.jar
hive create temporary function Pyth as 'Pyth';
OK
Time taken: 0.445 seconds
hive select Pyth(a,b) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Exception in thread main java.lang.UnsupportedOperationException: Kerberos
not supported in current hadoop version
at
org.apache.hadoop.hive.shims.Hadoop20Shims.getTokenFileLocEnvName(Hadoop20Shims.java:775)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:653)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/edward/hive.log
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
hive
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5459) Add --version option to hive script
[ https://issues.apache.org/jira/browse/HIVE-5459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5459: Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Patch committed to trunk. Thanks for the contribution Prasad! Add --version option to hive script --- Key: HIVE-5459 URL: https://issues.apache.org/jira/browse/HIVE-5459 Project: Hive Issue Type: Bug Components: Diagnosability Affects Versions: 0.11.0, 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.13.0 Attachments: HIVE-5459.1.patch, HIVE-5459.1.patch Hive jars already contain all the build information, similar to hadoop. This was added as part of HiveServer2 feature. We are still missing the command line wrapper to extract that information -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5457) Concurrent calls to getTable() result in: MetaException: org.datanucleus.exceptions.NucleusException: Invalid index 1 for DataStoreMapping. NucleusException: Invalid in
[
https://issues.apache.org/jira/browse/HIVE-5457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790803#comment-13790803
]
Brock Noland commented on HIVE-5457:
Lenni, I am curious if you had the configuration decribed here
https://issues.apache.org/jira/browse/HIVE-4762?focusedCommentId=13688852page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13688852
in your hive-site?
I don't think that is a catch all fix for our concurrency issues I am just
curious if that would help resolve this specific issue.
Concurrent calls to getTable() result in: MetaException:
org.datanucleus.exceptions.NucleusException: Invalid index 1 for
DataStoreMapping. NucleusException: Invalid index 1 for DataStoreMapping
---
Key: HIVE-5457
URL: https://issues.apache.org/jira/browse/HIVE-5457
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.10.0
Reporter: Lenni Kuff
Priority: Critical
Concurrent calls to getTable() result in: MetaException:
org.datanucleus.exceptions.NucleusException: Invalid index 1 for
DataStoreMapping. NucleusException: Invalid index 1 for DataStoreMapping
This happens when using a Hive Metastore Service directly connecting to the
backend metastore db. I have been able to hit this with as few as 2
concurrent calls. When I update my app to serialize all calls to getTable()
this problem is resolved.
Stack Trace:
{code}
Caused by: org.datanucleus.exceptions.NucleusException: Invalid index 1 for
DataStoreMapping.
at
org.datanucleus.store.mapped.mapping.PersistableMapping.getDatastoreMapping(PersistableMapping.java:307)
at
org.datanucleus.store.rdbms.scostore.RDBMSElementContainerStoreSpecialization.getSizeStmt(RDBMSElementContainerStoreSpecialization.java:407)
at
org.datanucleus.store.rdbms.scostore.RDBMSElementContainerStoreSpecialization.getSize(RDBMSElementContainerStoreSpecialization.java:257)
at
org.datanucleus.store.rdbms.scostore.RDBMSJoinListStoreSpecialization.getSize(RDBMSJoinListStoreSpecialization.java:46)
at
org.datanucleus.store.mapped.scostore.ElementContainerStore.size(ElementContainerStore.java:440)
at org.datanucleus.sco.backed.List.size(List.java:557)
at
org.apache.hadoop.hive.metastore.ObjectStore.convertToSkewedValues(ObjectStore.java:1029)
at
org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1007)
at
org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1017)
at
org.apache.hadoop.hive.metastore.ObjectStore.convertToTable(ObjectStore.java:872)
at
org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:743)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
at $Proxy6.getTable(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1349)
{code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (HIVE-5505) PerfLogger statements for Tez
Gunther Hagleitner created HIVE-5505: Summary: PerfLogger statements for Tez Key: HIVE-5505 URL: https://issues.apache.org/jira/browse/HIVE-5505 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Fix For: tez-branch Need to add some perf logger statements for the Tez path as well. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5502) ORC TestFileDump is flaky
[
https://issues.apache.org/jira/browse/HIVE-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790809#comment-13790809
]
Prasanth J commented on HIVE-5502:
--
Hi [~brocknoland].. seems like the test case failure is not related to the file
size of TestFileDump.testDump.orc file.. TestFileDump unit test file contains
two test cases ( testDump() and testDictionaryThreshold() ). These two test
cases creates an ORC file with the same name (look for testFilePath variable
initialization in openFileSystem()). This should be fixed to write to two
different files which is based on the test case function name.
I think the reason for seeing 2 different file size in your case is the passing
test case contains the output of testDictionaryThreshold() whereas failing test
case contains the output of testDump(). But the file size of
TestFileDump.testDump.orc is not really important for these test cases. Its the
contents of orc-file-dump.out file that is more important. Doing a diff of
generated orc-file-dump.out vs golden file shows that 1st strip expects 5000
rows but it got only 4000 rows. This is the reason for test case failure. I
faced similar non-determinism when I run the test case from eclipse vs from
console. From console I always get the correct result but when I try to run the
test case from eclipse it fails all the time with the same issue (4000 rows vs
5000 rows). The golden file in this case might have been generated by running
ant test -Dtestcase=TestFileDump. Since now you are testing using maven there
might be some difference in ANT_OPTS vs MAVEN_OPTS. Thats my guess.
Moving forward there are two ways this can be fixed
1) Implement a deterministic memory manager that doesn't depend on the
available memory for ORC test cases
2) Overwrite golden file when we move to maven
ORC TestFileDump is flaky
-
Key: HIVE-5502
URL: https://issues.apache.org/jira/browse/HIVE-5502
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Priority: Minor
Attachments: TestFileDump.tar.gz
I found in my maven work that TestFileDump is non-deterministic. For example
sometimes the output ORC file is much larger
{noformat}
pass:
-rwxrwxrwx 1 brock brock 290055 Oct 9 12:02 TestFileDump.testDump.orc
fail:
-rwxrwxrwx 1 brock brock 1938634 Oct 9 12:08 TestFileDump.testDump.orc
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5505) PerfLogger statements for Tez
[ https://issues.apache.org/jira/browse/HIVE-5505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-5505: - Attachment: HIVE-5505.1.patch PerfLogger statements for Tez - Key: HIVE-5505 URL: https://issues.apache.org/jira/browse/HIVE-5505 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Fix For: tez-branch Attachments: HIVE-5505.1.patch Need to add some perf logger statements for the Tez path as well. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5502) ORC TestFileDump is flaky
[
https://issues.apache.org/jira/browse/HIVE-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790828#comment-13790828
]
Brock Noland commented on HIVE-5502:
Gotcha. Thanks for taking a look.
bq. Since now you are testing using maven there might be some difference in
ANT_OPTS vs MAVEN_OPTS. Thats my guess.
bq. Implement a deterministic memory manager
OK these two together make sense as I was seeing the failures under maven with
the exact same M2_OPTS. I suppose it's possible that that some no M2_OPTS
cause non-determinism and my current ANT_OPTS do not.
bq. Overwrite golden file when we move to maven
I am not sure this will work since we cannot control. M2_OPTS on developers
machines.
ORC TestFileDump is flaky
-
Key: HIVE-5502
URL: https://issues.apache.org/jira/browse/HIVE-5502
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Priority: Minor
Attachments: TestFileDump.tar.gz
I found in my maven work that TestFileDump is non-deterministic. For example
sometimes the output ORC file is much larger
{noformat}
pass:
-rwxrwxrwx 1 brock brock 290055 Oct 9 12:02 TestFileDump.testDump.orc
fail:
-rwxrwxrwx 1 brock brock 1938634 Oct 9 12:08 TestFileDump.testDump.orc
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5504) OrcOutputFormat honors compression properties only from within hive
[ https://issues.apache.org/jira/browse/HIVE-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated HIVE-5504: - Summary: OrcOutputFormat honors compression properties only from within hive (was: HCatOutputFormat does not honor orc.compress tblproperty) OrcOutputFormat honors compression properties only from within hive - Key: HIVE-5504 URL: https://issues.apache.org/jira/browse/HIVE-5504 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.11.0, 0.12.0 Reporter: Venkat Ranganathan When we import data into a HCatalog table created with the following storage description .. stored as orc tblproperties (orc.compress=SNAPPY) the resultant orc file still uses the default zlib compression It looks like HCatOutputFormat is ignoring the tblproperties specified. show tblproperties shows that the table indeed has the properties properly saved. An insert/select into the table has the resulting orc file honor the tbl property. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4898) make vectorized math functions work end-to-end (update VectorizationContext.java)
[ https://issues.apache.org/jira/browse/HIVE-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-4898: --- Status: Open (was: Patch Available) make vectorized math functions work end-to-end (update VectorizationContext.java) - Key: HIVE-4898 URL: https://issues.apache.org/jira/browse/HIVE-4898 Project: Hive Issue Type: Sub-task Affects Versions: vectorization-branch Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-4898.3.patch, HIVE-4898.3.patch, HIVE-4898.4.patch The vectorized math function VectorExpression classes were added in HIVE-4822. This JIRA is to allow those to actually be used in a SQL query end-to-end. This requires updating VectorizationContext to use the new classes in vectorized expression creation. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4898) make vectorized math functions work end-to-end (update VectorizationContext.java)
[ https://issues.apache.org/jira/browse/HIVE-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-4898: --- Status: Patch Available (was: Open) make vectorized math functions work end-to-end (update VectorizationContext.java) - Key: HIVE-4898 URL: https://issues.apache.org/jira/browse/HIVE-4898 Project: Hive Issue Type: Sub-task Affects Versions: vectorization-branch Reporter: Eric Hanson Assignee: Eric Hanson Attachments: HIVE-4898.3.patch, HIVE-4898.3.patch, HIVE-4898.4.patch The vectorized math function VectorExpression classes were added in HIVE-4822. This JIRA is to allow those to actually be used in a SQL query end-to-end. This requires updating VectorizationContext to use the new classes in vectorized expression creation. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5490) SUBSTR(col, 1, 0) returns wrong result in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13790848#comment-13790848 ] Ashutosh Chauhan commented on HIVE-5490: +1 SUBSTR(col, 1, 0) returns wrong result in vectorized mode - Key: HIVE-5490 URL: https://issues.apache.org/jira/browse/HIVE-5490 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.13.0 Reporter: Eric Hanson Assignee: Teddy Choi Attachments: HIVE-5490.1.patch.txt The query select substr(cstring1, 1, 0) from alltypesorc; returns all empty strings when set hive.vectorized.execution.enabled = false; which is the correct result, and returns non-empty strings when set hive.vectorized.execution.enabled = true; which is not correct. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HIVE-5506) Hive SPLIT function does not return array correctly
John Omernik created HIVE-5506: -- Summary: Hive SPLIT function does not return array correctly Key: HIVE-5506 URL: https://issues.apache.org/jira/browse/HIVE-5506 Project: Hive Issue Type: Bug Components: SQL, UDF Affects Versions: 0.11.0, 0.10.0, 0.9.0 Environment: Hive Reporter: John Omernik Hello all, I think I have outlined a bug in the hive split function: Summary: When calling split on a string of data, it will only return all array items if the the last array item has a value. For example, if I have a string of text delimited by tab with 7 columns, and the first four are filled, but the last three are blank, split will only return a 4 position array. If any number of middle columns are empty, but the last item still has a value, then it will return the proper number of columns. This was tested in Hive 0.9 and hive 0.11. Data: (Note \t represents a tab char, \x09 the line endings should be \n (UNIX style) not sure what email will do to them). Basically my data is 7 lines of data with the first 7 letters separated by tab. On some lines I've left out certain letters, but kept the number of tabs exactly the same. input.txt a\tb\tc\td\te\tf\tg a\tb\tc\td\te\t\tg a\tb\t\td\t\tf\tg \t\t\td\te\tf\tg a\tb\tc\td\t\t\t a\t\t\t\te\tf\tg a\t\t\td\t\t\tg I then created a table with one column from that data: DROP TABLE tmp_jo_tab_test; CREATE table tmp_jo_tab_test (message_line STRING) STORED AS TEXTFILE; LOAD DATA LOCAL INPATH '/tmp/input.txt' OVERWRITE INTO TABLE tmp_jo_tab_test; Ok just to validate I created a python counting script: #!/usr/bin/python import sys for line in sys.stdin: line = line[0:-1] out = line.split(\t) print len(out) The output there is : $ cat input.txt |./cnt_tabs.py 7 7 7 7 7 7 7 Based on that information, split on tab should return me 7 for each line as well: hive -e select size(split(message_line, '\\t')) from tmp_jo_tab_test; 7 7 7 7 4 7 7 However it does not. It would appear that the line where only the first four letters are filled in(and blank is passed in on the last three) only returns 4 splits, where there should technically be 7, 4 for letters included, and three blanks. a\tb\tc\td\t\t\t -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5507) [WebHCat] test.other.user.name parameter is missing from build.xml in e2e harness
[ https://issues.apache.org/jira/browse/HIVE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5507: - Attachment: HIVE-5507-1.patch [WebHCat] test.other.user.name parameter is missing from build.xml in e2e harness - Key: HIVE-5507 URL: https://issues.apache.org/jira/browse/HIVE-5507 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5507-1.patch When we run templeton e2e tests, we need to specify test.other.user.name parameter for a second templeton user. This is missing in build.xml. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HIVE-5507) [WebHCat] test.other.user.name parameter is missing from build.xml in e2e harness
Daniel Dai created HIVE-5507: Summary: [WebHCat] test.other.user.name parameter is missing from build.xml in e2e harness Key: HIVE-5507 URL: https://issues.apache.org/jira/browse/HIVE-5507 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5507-1.patch When we run templeton e2e tests, we need to specify test.other.user.name parameter for a second templeton user. This is missing in build.xml. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5507) [WebHCat] test.other.user.name parameter is missing from build.xml in e2e harness
[ https://issues.apache.org/jira/browse/HIVE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5507: - Status: Patch Available (was: Open) [WebHCat] test.other.user.name parameter is missing from build.xml in e2e harness - Key: HIVE-5507 URL: https://issues.apache.org/jira/browse/HIVE-5507 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5507-1.patch When we run templeton e2e tests, we need to specify test.other.user.name parameter for a second templeton user. This is missing in build.xml. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HIVE-5508) [WebHCat] ignore log collector e2e tests for Hadoop 2
Daniel Dai created HIVE-5508: Summary: [WebHCat] ignore log collector e2e tests for Hadoop 2 Key: HIVE-5508 URL: https://issues.apache.org/jira/browse/HIVE-5508 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Log collector currently only works with Hadoop 1. If run under Hadoop 2, no log will be collected. Templeton e2e tests check the existence of those logs, so they will fail under Hadoop 2. Need to disable them when run under Hadoop 2. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5508) [WebHCat] ignore log collector e2e tests for Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5508: - Attachment: HIVE-5508-1.patch [WebHCat] ignore log collector e2e tests for Hadoop 2 - Key: HIVE-5508 URL: https://issues.apache.org/jira/browse/HIVE-5508 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5508-1.patch Log collector currently only works with Hadoop 1. If run under Hadoop 2, no log will be collected. Templeton e2e tests check the existence of those logs, so they will fail under Hadoop 2. Need to disable them when run under Hadoop 2. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-5509) [WebHCat] TestDriverCurl to use string comparison for jobid
[
https://issues.apache.org/jira/browse/HIVE-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-5509:
-
Attachment: HIVE-5509-1.patch
[WebHCat] TestDriverCurl to use string comparison for jobid
---
Key: HIVE-5509
URL: https://issues.apache.org/jira/browse/HIVE-5509
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.12.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.13.0
Attachments: HIVE-5509-1.patch
In TestDriverCurl.pm, we sort job status array returned by templeton using:
{code}
sort { $a-{id} = $b-{id} }
{code}
However, = is used to compare numbers, jobid is string, so comparison is
wrong. This results test JOBS_4 fail in some cases.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
