[jira] [Commented] (HIVE-9228) Problem with subquery using windowing functions
[ https://issues.apache.org/jira/browse/HIVE-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279016#comment-14279016 ] Aihua Xu commented on HIVE-9228: [~rhbutani] Do you have comments on this issue? Problem with subquery using windowing functions --- Key: HIVE-9228 URL: https://issues.apache.org/jira/browse/HIVE-9228 Project: Hive Issue Type: Bug Components: PTF-Windowing Affects Versions: 0.13.1 Reporter: Aihua Xu Assignee: Aihua Xu Attachments: create_table_tab1.sql, tab1.csv Original Estimate: 96h Remaining Estimate: 96h The following query with window functions failed. The internal query works fine. select col1, col2, col3 from (select col1,col2, col3, count(case when col4=1 then 1 end ) over (partition by col1, col2) as col5, row_number() over (partition by col1, col2 order by col4) as col6 from tab1) t; HIVE generates an execution plan with 2 jobs. 1. The first job is to basically calculate window function for col5. 2. The second job is to calculate window function for col6 and output. The plan says the first job outputs the columns (col1, col2, col3, col4) to a tmp file since only these columns are used in later stage. While, the PTF operator for the first job outputs (_wcol0, col1, col2, col3, col4) with _wcol0 as the result of the window function even it's not used. In the second job, the map operator still reads the 4 columns (col1, col2, col3, col4) from the temp file using the plan. That causes the exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9357) Create ADD_MONTHS UDF
[
https://issues.apache.org/jira/browse/HIVE-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279039#comment-14279039
]
Alexander Pivovarov commented on HIVE-9357:
---
build 2366 has the same 3 errors as prev build 2365
Create ADD_MONTHS UDF
-
Key: HIVE-9357
URL: https://issues.apache.org/jira/browse/HIVE-9357
Project: Hive
Issue Type: Improvement
Components: UDF
Reporter: Alexander Pivovarov
Assignee: Alexander Pivovarov
Attachments: HIVE-9357.1.patch, HIVE-9357.2.patch
ADD_MONTHS adds a number of months to startdate:
add_months('2015-01-14', 1) = '2015-02-14'
add_months('2015-01-31', 1) = '2015-02-28'
add_months('2015-02-28', 2) = '2015-04-30'
add_months('2015-02-28', 12) = '2016-02-29'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9369) fix arguments length checking in Upper and Lower UDF
[ https://issues.apache.org/jira/browse/HIVE-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279079#comment-14279079 ] Alexander Pivovarov commented on HIVE-9369: --- looks like these 4 errors were not caused by the patch 1 fix arguments length checking in Upper and Lower UDF Key: HIVE-9369 URL: https://issues.apache.org/jira/browse/HIVE-9369 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0, 0.14.1 Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Priority: Trivial Attachments: HIVE-9369.1.patch currently initialize method checks that arguments.length 0 it should check if arguments.length != 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9389) INSERT OVERWRITE DIRECTORY fails to delete old data files
Andy Skelton created HIVE-9389:
--
Summary: INSERT OVERWRITE DIRECTORY fails to delete old data files
Key: HIVE-9389
URL: https://issues.apache.org/jira/browse/HIVE-9389
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Environment: CDH 5.3.0, non-secure hdfs, perm checking off
Reporter: Andy Skelton
{code:sql}
FROM myview INSERT OVERWRITE DIRECTORY 'hdfs://nameservice/path/' SELECT
COUNT(DISTINCT mycol);
{code}
This always produces one row. Sometimes the output is two files, {{00_0}}
and {{00_1}}, one of which is empty. Sometimes we have seen new results in
{{00_0}} while old results remain in {{00_1}}.
We were alerted to this because Sqoop was exporting the output files in order
by filename, writing first the new value and then overwriting with the old
value, triggering an alert because the value stopped increasing in our database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9038) Join tests fail on Tez
[
https://issues.apache.org/jira/browse/HIVE-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279076#comment-14279076
]
Hive QA commented on HIVE-9038:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692388/HIVE-9038.3.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7315 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2375/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2375/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2375/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692388 - PreCommit-HIVE-TRUNK-Build
Join tests fail on Tez
--
Key: HIVE-9038
URL: https://issues.apache.org/jira/browse/HIVE-9038
Project: Hive
Issue Type: Bug
Components: Tests, Tez
Reporter: Ashutosh Chauhan
Assignee: Vikram Dixit K
Attachments: HIVE-9038.1.patch, HIVE-9038.2.patch, HIVE-9038.3.patch
Tez doesn't run all tests. But, if you run them, following tests fail with
runt time exception pointing to bugs.
{{auto_join21.q,auto_join29.q,auto_join30.q
,auto_join_filters.q,auto_join_nulls.q}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9234) HiveServer2 leaks FileSystem objects in FileSystem.CACHE
[
https://issues.apache.org/jira/browse/HIVE-9234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-9234:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Test failure unrelated. Patch committed to trunk and 14.1. Thanks for reviewing
[~thejas] and [~brocknoland].
HiveServer2 leaks FileSystem objects in FileSystem.CACHE
Key: HIVE-9234
URL: https://issues.apache.org/jira/browse/HIVE-9234
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.12.0, 0.13.0, 0.12.1, 0.14.0, 0.13.1
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 0.14.1
Attachments: HIVE-9234.1.patch, HIVE-9234.2.patch, HIVE-9234.2.patch,
HIVE-9234.branch-14.patch
Running over extended period (48+ hrs), we've noticed HiveServer2 leaking
FileSystem objects in FileSystem.CACHE. Linked jiras were previous attempts
to fix it, but the issue still seems to be there. A workaround is to disable
the caching (by setting {{fs.hdfs.impl.disable.cache}} and
{{fs.file.impl.disable.cache}} to {{true}}), but creating new FileSystem
objects is expensive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9264) Merge encryption branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9264: --- Affects Version/s: (was: encryption-branch) 0.15.0 Status: Patch Available (was: Open) Merge encryption branch to trunk Key: HIVE-9264 URL: https://issues.apache.org/jira/browse/HIVE-9264 Project: Hive Issue Type: Sub-task Affects Versions: 0.15.0 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9264.1.patch The team working on the encryption branch would like to merge their work to trunk. This jira will track that effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9255) Fastpath for limited fetches from unpartitioned tables
[
https://issues.apache.org/jira/browse/HIVE-9255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278999#comment-14278999
]
Ashutosh Chauhan commented on HIVE-9255:
+1
you can edit comments while commiting
Fastpath for limited fetches from unpartitioned tables
--
Key: HIVE-9255
URL: https://issues.apache.org/jira/browse/HIVE-9255
Project: Hive
Issue Type: Bug
Components: Physical Optimizer
Affects Versions: 0.14.0, 0.15.0
Reporter: Gopal V
Assignee: Gopal V
Priority: Minor
Attachments: HIVE-9255.1.patch, HIVE-9255.2.patch, HIVE-9255.3.patch
Currently, for flat tables, the threshold check is applicable for a query
like {{select * from lineitem limit 1;}}.
This is not necessary as without a filter clause, this can be executed
entirely via FetchTask. Running a cluster task is redundant for this case.
This fastpath is applicable for partitioned tables already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9390) Enhance retry logic wrt DB access in TxnHandler
Eugene Koifman created HIVE-9390: Summary: Enhance retry logic wrt DB access in TxnHandler Key: HIVE-9390 URL: https://issues.apache.org/jira/browse/HIVE-9390 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Eugene Koifman example of error 2015-01-13 16:09:21,148 ERROR metastore.RetryingHMSHandler (RetryingHMSHandler.java:invoke(141)) - org.apache.thrift.TException: MetaException(message:Unable to get jdbc connection from pool, Read timed out) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_open_txns(HiveMetaStore.java:5324) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102) at com.sun.proxy.$Proxy11.get_open_txns(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1696) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90) at com.sun.proxy.$Proxy12.getValidTxns(Unknown Source) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:289) at org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:882) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:399) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:303) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1067) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1061) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:100) at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:171) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:256) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363) at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60) at com.sun.proxy.$Proxy21.executeStatementAsync(Unknown Source) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:247) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TServlet.doPost(TServlet.java:83) at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:101) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479) at
[jira] [Commented] (HIVE-9369) fix arguments length checking in Upper and Lower UDF
[
https://issues.apache.org/jira/browse/HIVE-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278952#comment-14278952
]
Hive QA commented on HIVE-9369:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692114/HIVE-9369.1.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 7311 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_alter_merge_orc
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2374/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2374/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2374/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692114 - PreCommit-HIVE-TRUNK-Build
fix arguments length checking in Upper and Lower UDF
Key: HIVE-9369
URL: https://issues.apache.org/jira/browse/HIVE-9369
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0, 0.14.1
Reporter: Alexander Pivovarov
Assignee: Alexander Pivovarov
Priority: Trivial
Attachments: HIVE-9369.1.patch
currently initialize method checks that arguments.length 0
it should check if arguments.length != 1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9390) Enhance retry logic wrt DB access in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-9390:
-
Description:
example of error
{noformat}
2015-01-13 16:09:21,148 ERROR metastore.RetryingHMSHandler
(RetryingHMSHandler.java:invoke(141)) - org.apache.thrift.TException:
MetaException(message:Unable to get jdbc connection from pool, Read timed out)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_open_txns(HiveMetaStore.java:5324)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102)
at com.sun.proxy.$Proxy11.get_open_txns(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy12.getValidTxns(Unknown Source)
at
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:289)
at org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:882)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:399)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:303)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1067)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1061)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:100)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:171)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:256)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy21.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:247)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
at
org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:101)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
at
[jira] [Commented] (HIVE-9248) Vectorization : Tez Reduce vertex not getting vectorized when GROUP BY is Hash mode
[ https://issues.apache.org/jira/browse/HIVE-9248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279027#comment-14279027 ] Ashutosh Chauhan commented on HIVE-9248: Looks good, but some of above test failures looks legit and needs to be looked at. Vectorization : Tez Reduce vertex not getting vectorized when GROUP BY is Hash mode --- Key: HIVE-9248 URL: https://issues.apache.org/jira/browse/HIVE-9248 Project: Hive Issue Type: Bug Components: Tez, Vectorization Affects Versions: 0.14.0 Reporter: Matt McCline Assignee: Matt McCline Priority: Critical Attachments: HIVE-9248.01.patch, HIVE-9248.02.patch, HIVE-9248.03.patch, HIVE-9248.04.patch Under Tez and Vectorization, ReduceWork not getting vectorized unless it GROUP BY operator is MergePartial. Add valid cases where GROUP BY is Hash (and presumably there are downstream reducers that will do MergePartial). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9264) Merge encryption branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9264: --- Attachment: HIVE-9264.1.patch Merge encryption branch to trunk Key: HIVE-9264 URL: https://issues.apache.org/jira/browse/HIVE-9264 Project: Hive Issue Type: Sub-task Affects Versions: encryption-branch Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9264.1.patch The team working on the encryption branch would like to merge their work to trunk. This jira will track that effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9383) Improve schema verification error message
[ https://issues.apache.org/jira/browse/HIVE-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9383: --- Status: Patch Available (was: Open) Improve schema verification error message - Key: HIVE-9383 URL: https://issues.apache.org/jira/browse/HIVE-9383 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Brock Noland Priority: Minor Attachments: HIVE-9383.patch Currently the error message just says the schema found. It should say the schema expected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9366) wrong date in description annotation in date_add() and date_sub() udf
[ https://issues.apache.org/jira/browse/HIVE-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279081#comment-14279081 ] Alexander Pivovarov commented on HIVE-9366: --- I think these 2 errors were not caused by the patch 3 wrong date in description annotation in date_add() and date_sub() udf - Key: HIVE-9366 URL: https://issues.apache.org/jira/browse/HIVE-9366 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.1 Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Priority: Trivial Attachments: HIVE-9366.1.patch, HIVE-9366.2.patch, HIVE-9366.3.patch files: ql/src/test/results/clientpositive/udf_date_add.q.out ql/src/test/results/beelinepositive/udf_date_add.q.out last line shows '2009-31-07' but it should be '2009-07-31' instead the @Description annotation needs to be fixed for both date_add() and date_sub() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HIVE-9378) Spark qfile tests should reuse RSC [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang reopened HIVE-9378: --- Assignee: Jimmy Xiang Spark qfile tests should reuse RSC [Spark Branch] - Key: HIVE-9378 URL: https://issues.apache.org/jira/browse/HIVE-9378 Project: Hive Issue Type: Improvement Components: Spark Affects Versions: spark-branch Reporter: Jimmy Xiang Assignee: Jimmy Xiang Run several qfile tests, use jps to monitor the java processes. You will find several SparkSubmitDriverBootstrapper processes are created (not the same time of course). It seems to me that we create a RSC for each qfile, then terminate it when this qfile test is done. The RSC seems not shared among qfiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9272) Tests for utf-8 support
[ https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepesh Khandelwal updated HIVE-9272: - Description: Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character NO PRECOMMIT TESTS was:Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character We should ignore the above failures. Add the flag to skip pre-commit tests as these are changes in the E2E test suite that doesn't get run as part of pre-commit tests. Tests for utf-8 support --- Key: HIVE-9272 URL: https://issues.apache.org/jira/browse/HIVE-9272 Project: Hive Issue Type: Test Components: Tests, WebHCat Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.patch Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9385) Sum a Double using an ORC table
[
https://issues.apache.org/jira/browse/HIVE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damien Carol updated HIVE-9385:
---
Description:
I’m storing a sales amount column as a double in an ORC table and when I do:
{code:sql}
select sum(x) from sometable
{code}
I get a value like {{4.79165141174808E9}}
A visual inspection of the column values reveals no glaring anomalies…all looks
pretty normal.
If I do the same thing in a textfile table I get a perfectly fine aggregation
of the double field.
was:
I’m storing a sales amount column as a double in an ORC table and when I do:
select sum(x) from sometable
I get a value like 4.79165141174808E9
A visual inspection of the column values reveals no glaring anomalies…all looks
pretty normal.
If I do the same thing in a textfile table I get a perfectly fine aggregation
of the double field.
Sum a Double using an ORC table
---
Key: HIVE-9385
URL: https://issues.apache.org/jira/browse/HIVE-9385
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Environment: HDP 2.x, Hive
Reporter: Nick Martin
Priority: Minor
I’m storing a sales amount column as a double in an ORC table and when I do:
{code:sql}
select sum(x) from sometable
{code}
I get a value like {{4.79165141174808E9}}
A visual inspection of the column values reveals no glaring anomalies…all
looks pretty normal.
If I do the same thing in a textfile table I get a perfectly fine aggregation
of the double field.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9385) Sum a Double using an ORC table
[
https://issues.apache.org/jira/browse/HIVE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278823#comment-14278823
]
Damien Carol commented on HIVE-9385:
[~nimartin] I don't understand your problem.
Do you complain about double format inaccuracy? Or did you expected another
result?
Sum a Double using an ORC table
---
Key: HIVE-9385
URL: https://issues.apache.org/jira/browse/HIVE-9385
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Environment: HDP 2.x, Hive
Reporter: Nick Martin
Priority: Minor
I’m storing a sales amount column as a double in an ORC table and when I do:
{code:sql}
select sum(x) from sometable
{code}
I get a value like {{4.79165141174808E9}}
A visual inspection of the column values reveals no glaring anomalies…all
looks pretty normal.
If I do the same thing in a textfile table I get a perfectly fine aggregation
of the double field.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-9384) Sum a Double using an ORC table
[ https://issues.apache.org/jira/browse/HIVE-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Carol resolved HIVE-9384. Resolution: Duplicate Sum a Double using an ORC table --- Key: HIVE-9384 URL: https://issues.apache.org/jira/browse/HIVE-9384 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Environment: HDP 2.x, Hive Reporter: Nick Martin Priority: Minor I’m storing a sales amount column as a double in an ORC table and when I do: select sum(x) from sometable I get a value like 4.79165141174808E9 A visual inspection of the column values reveals no glaring anomalies…all looks pretty normal. If I do the same thing in a textfile table I get a perfectly fine aggregation of the double field. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9367) CombineFileInputFormatShim#getDirIndices is expensive
[
https://issues.apache.org/jira/browse/HIVE-9367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278836#comment-14278836
]
Hive QA commented on HIVE-9367:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692326/HIVE-9367.2.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 7311 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_stats_counter
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union13
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2373/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2373/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2373/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692326 - PreCommit-HIVE-TRUNK-Build
CombineFileInputFormatShim#getDirIndices is expensive
-
Key: HIVE-9367
URL: https://issues.apache.org/jira/browse/HIVE-9367
Project: Hive
Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: HIVE-9367.1.patch, HIVE-9367.2.patch
[~lirui] found out that we spent quite some time on
CombineFileInputFormatShim#getDirIndices. Looked into it and it seems to me
we should be able to get rid of this method completely if we can enhance
CombineFileInputFormatShim a little.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9392) JoinStatsRule miscalculates join cardinality as incorrect NDV is used due to column names having duplicated fqColumnName
[ https://issues.apache.org/jira/browse/HIVE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar updated HIVE-9392: -- Summary: JoinStatsRule miscalculates join cardinality as incorrect NDV is used due to column names having duplicated fqColumnName (was: Hive : JoinStatsRule overwrites the column statistics in HashMap because column names have duplicate fqColumnName) JoinStatsRule miscalculates join cardinality as incorrect NDV is used due to column names having duplicated fqColumnName Key: HIVE-9392 URL: https://issues.apache.org/jira/browse/HIVE-9392 Project: Hive Issue Type: Bug Components: Physical Optimizer Affects Versions: 0.14.0 Reporter: Mostafa Mokhtar Assignee: Prasanth Jayachandran Priority: Critical Fix For: 0.15.0 In JoinStatsRule.process the join column statistics are stored in HashMap joinedColStats, the key used which is the ColStatistics.fqColName is duplicated between join column in the same vertex, as a result distinctVals ends up having duplicated values which negatively affects the join cardinality estimation. The duplicate keys are usually named KEY.reducesinkkey0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9391) Spark doesn't obey setting hive.multigroupby.singlereducer
[
https://issues.apache.org/jira/browse/HIVE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HIVE-9391:
--
Assignee: (was: Jimmy Xiang)
Spark doesn't obey setting hive.multigroupby.singlereducer
--
Key: HIVE-9391
URL: https://issues.apache.org/jira/browse/HIVE-9391
Project: Hive
Issue Type: Bug
Reporter: Jimmy Xiang
Priority: Minor
Fix For: spark-branch
With hive.multigroupby.singlereducer set to true, Spark still generates plan
with multiple groupby reducers. For example, in working on HIVE-9378, I
found, for test groupby_complex_types_multi_single_reducer.q, we have plan
like
{noformat}
+Reducer 2 - Map 6 (GROUP, 2)
+Reducer 4 - Map 7 (GROUP, 2)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9357) Create ADD_MONTHS UDF
[
https://issues.apache.org/jira/browse/HIVE-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279163#comment-14279163
]
Jason Dere commented on HIVE-9357:
--
Agreed, org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1,
org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection have
been failing on other builds, and
TestSparkCliDriver.testCliDriver_vectorization_part passes locally for me.
+1
Create ADD_MONTHS UDF
-
Key: HIVE-9357
URL: https://issues.apache.org/jira/browse/HIVE-9357
Project: Hive
Issue Type: Improvement
Components: UDF
Reporter: Alexander Pivovarov
Assignee: Alexander Pivovarov
Attachments: HIVE-9357.1.patch, HIVE-9357.2.patch
ADD_MONTHS adds a number of months to startdate:
add_months('2015-01-14', 1) = '2015-02-14'
add_months('2015-01-31', 1) = '2015-02-28'
add_months('2015-02-28', 2) = '2015-04-30'
add_months('2015-02-28', 12) = '2016-02-29'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9272) Tests for utf-8 support
[ https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-9272: - Affects Version/s: 0.14.0 Tests for utf-8 support --- Key: HIVE-9272 URL: https://issues.apache.org/jira/browse/HIVE-9272 Project: Hive Issue Type: Test Components: Tests, WebHCat Affects Versions: 0.14.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.patch Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9272) Tests for utf-8 support
[ https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279305#comment-14279305 ] Eugene Koifman commented on HIVE-9272: -- +1 Tests for utf-8 support --- Key: HIVE-9272 URL: https://issues.apache.org/jira/browse/HIVE-9272 Project: Hive Issue Type: Test Components: Tests, WebHCat Affects Versions: 0.14.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.patch Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9272) Tests for utf-8 support
[ https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279326#comment-14279326 ] Eugene Koifman commented on HIVE-9272: -- sorry, I think the +1 was premature after applying this patch on trunk and running mvn clean package install -Phadoop-2,dist -DskipTests I end up with an error [INFO] Hive Packaging . FAILURE [ 23.908 s] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 03:46 min [INFO] Finished at: 2015-01-15T13:35:44-08:00 [INFO] Final Memory: 187M/1055M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.3:single (assemble) on project hive-packaging: Failed to create assembly: Error creating assembly archive src: hcatalog/src/test/e2e/templeton/inpdir/artof\344\266\264\343\204\251\351\274\276\344\270\204\347\213\234\343\200\207war.txt not found. - [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn goals -rf :hive-packaging ekoifman:svnhive ekoifman$ Tests for utf-8 support --- Key: HIVE-9272 URL: https://issues.apache.org/jira/browse/HIVE-9272 Project: Hive Issue Type: Test Components: Tests, WebHCat Affects Versions: 0.14.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.patch Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9391) Spark doesn't obey setting hive.multigroupby.singlereducer
Jimmy Xiang created HIVE-9391:
-
Summary: Spark doesn't obey setting hive.multigroupby.singlereducer
Key: HIVE-9391
URL: https://issues.apache.org/jira/browse/HIVE-9391
Project: Hive
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: spark-branch
With hive.multigroupby.singlereducer set to true, Spark still generates plan
with multiple groupby reducers. For example, in working on HIVE-9378, I found,
for test groupby_complex_types_multi_single_reducer.q, we have plan like
{noformat}
+Reducer 2 - Map 6 (GROUP, 2)
+Reducer 4 - Map 7 (GROUP, 2)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 28964: HIVE-8121 Create micro-benchmarks for ParquetSerde and evaluate performance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28964/ --- (Updated Ene. 15, 2015, 8:08 p.m.) Review request for hive, Brock Noland, cheng xu, and Dong Chen. Bugs: HIVE-8121 https://issues.apache.org/jira/browse/HIVE-8121 Repository: hive-git Description --- This is a new tool used to test ORC PARQUET file format performance. Diffs - itests/hive-jmh/pom.xml PRE-CREATION itests/hive-jmh/src/main/java/org/apache/hive/benchmark/storage/ColumnarStorageBench.java PRE-CREATION itests/pom.xml e1e88f65ee3c8ad0e9067ec2b94004477c86bece Diff: https://reviews.apache.org/r/28964/diff/ Testing --- Thanks, Sergio Pena
Re: Review Request 28964: HIVE-8121 Create micro-benchmarks for ParquetSerde and evaluate performance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28964/ --- (Updated Ene. 15, 2015, 8:08 p.m.) Review request for hive, Brock Noland and cheng xu. Changes --- Sorry guys for more changes. I removed repeated code, and apply changes that Dong recommended me. Bugs: HIVE-8121 https://issues.apache.org/jira/browse/HIVE-8121 Repository: hive-git Description --- This is a new tool used to test ORC PARQUET file format performance. Diffs (updated) - itests/hive-jmh/pom.xml PRE-CREATION itests/hive-jmh/src/main/java/org/apache/hive/benchmark/storage/ColumnarStorageBench.java PRE-CREATION itests/pom.xml e1e88f65ee3c8ad0e9067ec2b94004477c86bece Diff: https://reviews.apache.org/r/28964/diff/ Testing --- Thanks, Sergio Pena
[jira] [Commented] (HIVE-9248) Vectorization : Tez Reduce vertex not getting vectorized when GROUP BY is Hash mode
[ https://issues.apache.org/jira/browse/HIVE-9248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279325#comment-14279325 ] Matt McCline commented on HIVE-9248: Ok, thanks [~ashutoshc], I will work on the related test failure and come back with a new patch. Vectorization : Tez Reduce vertex not getting vectorized when GROUP BY is Hash mode --- Key: HIVE-9248 URL: https://issues.apache.org/jira/browse/HIVE-9248 Project: Hive Issue Type: Bug Components: Tez, Vectorization Affects Versions: 0.14.0 Reporter: Matt McCline Assignee: Matt McCline Priority: Critical Attachments: HIVE-9248.01.patch, HIVE-9248.02.patch, HIVE-9248.03.patch, HIVE-9248.04.patch Under Tez and Vectorization, ReduceWork not getting vectorized unless it GROUP BY operator is MergePartial. Add valid cases where GROUP BY is Hash (and presumably there are downstream reducers that will do MergePartial). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9224) CBO (Calcite Return Path): Inline Table, Properties
[ https://issues.apache.org/jira/browse/HIVE-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279134#comment-14279134 ] Ashutosh Chauhan commented on HIVE-9224: +1 CBO (Calcite Return Path): Inline Table, Properties --- Key: HIVE-9224 URL: https://issues.apache.org/jira/browse/HIVE-9224 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Fix For: 0.15.0 Attachments: HIVE-9224.1.patch, HIVE-9224.2.patch, HIVE-9224.3.patch, HIVE-9224.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9038) Join tests fail on Tez
[
https://issues.apache.org/jira/browse/HIVE-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damien Carol updated HIVE-9038:
---
Description:
Tez doesn't run all tests. But, if you run them, following tests fail with runt
time exception pointing to bugs.
* {{auto_join21.q}}
* {{auto_join29.q}}
* {{auto_join30.q}}
* {{auto_join_filters.q}}
* {{auto_join_nulls.q}}
was:Tez doesn't run all tests. But, if you run them, following tests fail
with runt time exception pointing to bugs.
{{auto_join21.q,auto_join29.q,auto_join30.q
,auto_join_filters.q,auto_join_nulls.q}}
Join tests fail on Tez
--
Key: HIVE-9038
URL: https://issues.apache.org/jira/browse/HIVE-9038
Project: Hive
Issue Type: Bug
Components: Tests, Tez
Reporter: Ashutosh Chauhan
Assignee: Vikram Dixit K
Attachments: HIVE-9038.1.patch, HIVE-9038.2.patch, HIVE-9038.3.patch
Tez doesn't run all tests. But, if you run them, following tests fail with
runt time exception pointing to bugs.
* {{auto_join21.q}}
* {{auto_join29.q}}
* {{auto_join30.q}}
* {{auto_join_filters.q}}
* {{auto_join_nulls.q}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9358) Create LAST_DAY UDF
[
https://issues.apache.org/jira/browse/HIVE-9358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279219#comment-14279219
]
Hive QA commented on HIVE-9358:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692157/HIVE-9358.1.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 7312 tests executed
*Failed tests:*
{noformat}
TestCustomAuthentication - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats13
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2376/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2376/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2376/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692157 - PreCommit-HIVE-TRUNK-Build
Create LAST_DAY UDF
---
Key: HIVE-9358
URL: https://issues.apache.org/jira/browse/HIVE-9358
Project: Hive
Issue Type: Improvement
Components: UDF
Reporter: Alexander Pivovarov
Assignee: Alexander Pivovarov
Attachments: HIVE-9358.1.patch
LAST_DAY returns the date of the last day of the month that contains date:
last_day('2015-01-14') = '2015-01-31'
last_day('2016-02-01') = '2016-02-29'
last_day function went from oracle
http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions072.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 28964: HIVE-8121 Create micro-benchmarks for ParquetSerde and evaluate performance
On Ene. 15, 2015, 7:47 a.m., Dong Chen wrote: itests/hive-jmh/src/main/java/org/apache/hive/benchmark/storage/ColumnarStorageBench.java, line 355 https://reviews.apache.org/r/28964/diff/4/?file=822007#file822007line355 OrcRecordWriter load its internal writer lazily, which means writing the first record is different with writing following records. Should we consider this when defining and evaluating benchmark? Thanks for the tip. The write test now calls the write() method with many rows intead. - Sergio --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28964/#review68215 --- On Ene. 14, 2015, 11:48 p.m., Sergio Pena wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28964/ --- (Updated Ene. 14, 2015, 11:48 p.m.) Review request for hive, Brock Noland and cheng xu. Bugs: HIVE-8121 https://issues.apache.org/jira/browse/HIVE-8121 Repository: hive-git Description --- This is a new tool used to test ORC PARQUET file format performance. Diffs - itests/hive-jmh/pom.xml PRE-CREATION itests/hive-jmh/src/main/java/org/apache/hive/benchmark/storage/ColumnarStorageBench.java PRE-CREATION itests/pom.xml 40e7b9b8114d693f352f28ad94e6abcf96d1372a Diff: https://reviews.apache.org/r/28964/diff/ Testing --- Thanks, Sergio Pena
[jira] [Created] (HIVE-9392) Hive : JoinStatsRule overwrites the column statistics in HashMap because column names have duplicate fqColumnName
Mostafa Mokhtar created HIVE-9392: - Summary: Hive : JoinStatsRule overwrites the column statistics in HashMap because column names have duplicate fqColumnName Key: HIVE-9392 URL: https://issues.apache.org/jira/browse/HIVE-9392 Project: Hive Issue Type: Bug Components: Physical Optimizer Affects Versions: 0.14.0 Reporter: Mostafa Mokhtar Assignee: Prasanth Jayachandran Priority: Critical Fix For: 0.15.0 In JoinStatsRule.process the join column statistics are stored in HashMap joinedColStats, the key used which is the ColStatistics.fqColName is duplicated between join column in the same vertex, as a result distinctVals ends up having duplicated values which negatively affects the join cardinality estimation. The duplicate keys are usually named KEY.reducesinkkey0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9331) get rid of pre-optimized-hashtable memory optimizations
[ https://issues.apache.org/jira/browse/HIVE-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279191#comment-14279191 ] Sergey Shelukhin commented on HIVE-9331: Test failure is unrelated, will commit later today get rid of pre-optimized-hashtable memory optimizations --- Key: HIVE-9331 URL: https://issues.apache.org/jira/browse/HIVE-9331 Project: Hive Issue Type: Bug Affects Versions: 0.15.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.15.0 Attachments: HIVE-9331.01.patch, HIVE-9331.01.patch, HIVE-9331.02.patch, HIVE-9331.patch, HIVE-9331.patch These were added in 13 because optimized hashtable couldn't make it in; they reduced memory usage by some amount (10-25%), and informed the design of the optimized hashtable, but now extra settings and code branches are just confusing and may have their own bugs. Might as well remove them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9390) Enhance retry logic wrt DB access in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-9390:
-
Attachment: HIVE-9390.patch
Enhance retry logic wrt DB access in TxnHandler
---
Key: HIVE-9390
URL: https://issues.apache.org/jira/browse/HIVE-9390
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 0.14.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-9390.patch
example of error
{noformat}
2015-01-13 16:09:21,148 ERROR metastore.RetryingHMSHandler
(RetryingHMSHandler.java:invoke(141)) - org.apache.thrift.TException:
MetaException(message:Unable to get jdbc connection from pool, Read timed out)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_open_txns(HiveMetaStore.java:5324)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102)
at com.sun.proxy.$Proxy11.get_open_txns(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy12.getValidTxns(Unknown Source)
at
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:289)
at org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:882)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:399)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:303)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1067)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1061)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:100)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:171)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:256)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy21.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:247)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
at
org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:101)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
[jira] [Updated] (HIVE-9390) Enhance retry logic wrt DB access in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-9390:
-
Status: Patch Available (was: Open)
Enhance retry logic wrt DB access in TxnHandler
---
Key: HIVE-9390
URL: https://issues.apache.org/jira/browse/HIVE-9390
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 0.14.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-9390.patch
example of error
{noformat}
2015-01-13 16:09:21,148 ERROR metastore.RetryingHMSHandler
(RetryingHMSHandler.java:invoke(141)) - org.apache.thrift.TException:
MetaException(message:Unable to get jdbc connection from pool, Read timed out)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_open_txns(HiveMetaStore.java:5324)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102)
at com.sun.proxy.$Proxy11.get_open_txns(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy12.getValidTxns(Unknown Source)
at
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:289)
at org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:882)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:399)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:303)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1067)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1061)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:100)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:171)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:256)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy21.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:247)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
at
org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:101)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
[jira] [Commented] (HIVE-9358) Create LAST_DAY UDF
[
https://issues.apache.org/jira/browse/HIVE-9358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279252#comment-14279252
]
Alexander Pivovarov commented on HIVE-9358:
---
The errors are not related to patch 1
Create LAST_DAY UDF
---
Key: HIVE-9358
URL: https://issues.apache.org/jira/browse/HIVE-9358
Project: Hive
Issue Type: Improvement
Components: UDF
Reporter: Alexander Pivovarov
Assignee: Alexander Pivovarov
Attachments: HIVE-9358.1.patch
LAST_DAY returns the date of the last day of the month that contains date:
last_day('2015-01-14') = '2015-01-31'
last_day('2016-02-01') = '2016-02-29'
last_day function went from oracle
http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions072.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9248) Vectorization : Tez Reduce vertex not getting vectorized when GROUP BY is Hash mode
[
https://issues.apache.org/jira/browse/HIVE-9248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279300#comment-14279300
]
Hive QA commented on HIVE-9248:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692170/HIVE-9248.04.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 7316 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2377/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2377/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2377/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692170 - PreCommit-HIVE-TRUNK-Build
Vectorization : Tez Reduce vertex not getting vectorized when GROUP BY is
Hash mode
---
Key: HIVE-9248
URL: https://issues.apache.org/jira/browse/HIVE-9248
Project: Hive
Issue Type: Bug
Components: Tez, Vectorization
Affects Versions: 0.14.0
Reporter: Matt McCline
Assignee: Matt McCline
Priority: Critical
Attachments: HIVE-9248.01.patch, HIVE-9248.02.patch,
HIVE-9248.03.patch, HIVE-9248.04.patch
Under Tez and Vectorization, ReduceWork not getting vectorized unless it
GROUP BY operator is MergePartial. Add valid cases where GROUP BY is Hash
(and presumably there are downstream reducers that will do MergePartial).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9315) CBO (Calcite Return Path): Inline FileSinkOperator, Properties
[ https://issues.apache.org/jira/browse/HIVE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279364#comment-14279364 ] Laljo John Pullokkaran commented on HIVE-9315: -- Committed to trunk CBO (Calcite Return Path): Inline FileSinkOperator, Properties -- Key: HIVE-9315 URL: https://issues.apache.org/jira/browse/HIVE-9315 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9315.01.patch, HIVE-9315.02.patch, HIVE-9315.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9331) get rid of pre-optimized-hashtable memory optimizations
[ https://issues.apache.org/jira/browse/HIVE-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-9331: --- Resolution: Fixed Status: Resolved (was: Patch Available) committed to trunk get rid of pre-optimized-hashtable memory optimizations --- Key: HIVE-9331 URL: https://issues.apache.org/jira/browse/HIVE-9331 Project: Hive Issue Type: Bug Affects Versions: 0.15.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.15.0 Attachments: HIVE-9331.01.patch, HIVE-9331.01.patch, HIVE-9331.02.patch, HIVE-9331.patch, HIVE-9331.patch These were added in 13 because optimized hashtable couldn't make it in; they reduced memory usage by some amount (10-25%), and informed the design of the optimized hashtable, but now extra settings and code branches are just confusing and may have their own bugs. Might as well remove them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9200) CBO (Calcite Return Path): Inline Join, Properties
[ https://issues.apache.org/jira/browse/HIVE-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279366#comment-14279366 ] Laljo John Pullokkaran commented on HIVE-9200: -- [~jcamachorodriguez] Could you rebase the patch? doesn't apply cleanly after HIVE-9315. CBO (Calcite Return Path): Inline Join, Properties -- Key: HIVE-9200 URL: https://issues.apache.org/jira/browse/HIVE-9200 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9200.01.patch, HIVE-9200.02.patch, HIVE-9200.03.patch, HIVE-9200.04.patch, HIVE-9200.05.patch, HIVE-9200.06.patch, HIVE-9200.07.patch, HIVE-9200.08.patch, HIVE-9200.09.patch, HIVE-9200.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9393) reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG
[ https://issues.apache.org/jira/browse/HIVE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279450#comment-14279450 ] Yongzhi Chen commented on HIVE-9393: [~brocknoland], could you review and commit the patch? Thanks. reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG --- Key: HIVE-9393 URL: https://issues.apache.org/jira/browse/HIVE-9393 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.1 Reporter: Yongzhi Chen Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-9393.1.patch From Hive 0.13 the log level of ColumnarSerDe.java:116 was upgraded from DEBUG to INFO, this has introduced an very large amount of noise into the logs causing the underlying filesystem to fill up. This request is to drop is back to DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9378) Spark qfile tests should reuse RSC [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9378: -- Status: Patch Available (was: Reopened) Spark qfile tests should reuse RSC [Spark Branch] - Key: HIVE-9378 URL: https://issues.apache.org/jira/browse/HIVE-9378 Project: Hive Issue Type: Improvement Components: Spark Affects Versions: spark-branch Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: HIVE-9378.1-spark.patch Run several qfile tests, use jps to monitor the java processes. You will find several SparkSubmitDriverBootstrapper processes are created (not the same time of course). It seems to me that we create a RSC for each qfile, then terminate it when this qfile test is done. The RSC seems not shared among qfiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9378) Spark qfile tests should reuse RSC [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9378: -- Fix Version/s: spark-branch Spark qfile tests should reuse RSC [Spark Branch] - Key: HIVE-9378 URL: https://issues.apache.org/jira/browse/HIVE-9378 Project: Hive Issue Type: Improvement Components: Spark Affects Versions: spark-branch Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: spark-branch Attachments: HIVE-9378.1-spark.patch Run several qfile tests, use jps to monitor the java processes. You will find several SparkSubmitDriverBootstrapper processes are created (not the same time of course). It seems to me that we create a RSC for each qfile, then terminate it when this qfile test is done. The RSC seems not shared among qfiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9378) Spark qfile tests should reuse RSC [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9378: -- Attachment: HIVE-9378.1-spark.patch Spark qfile tests should reuse RSC [Spark Branch] - Key: HIVE-9378 URL: https://issues.apache.org/jira/browse/HIVE-9378 Project: Hive Issue Type: Improvement Components: Spark Affects Versions: spark-branch Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: spark-branch Attachments: HIVE-9378.1-spark.patch Run several qfile tests, use jps to monitor the java processes. You will find several SparkSubmitDriverBootstrapper processes are created (not the same time of course). It seems to me that we create a RSC for each qfile, then terminate it when this qfile test is done. The RSC seems not shared among qfiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-9393) reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG
[ https://issues.apache.org/jira/browse/HIVE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzhi Chen reassigned HIVE-9393: -- Assignee: Yongzhi Chen reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG --- Key: HIVE-9393 URL: https://issues.apache.org/jira/browse/HIVE-9393 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.1 Reporter: Yongzhi Chen Assignee: Yongzhi Chen Priority: Minor From Hive 0.13 the log level of ColumnarSerDe.java:116 was upgraded from DEBUG to INFO, this has introduced an very large amount of noise into the logs causing the underlying filesystem to fill up. This request is to drop is back to DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9393) reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG
Yongzhi Chen created HIVE-9393: -- Summary: reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG Key: HIVE-9393 URL: https://issues.apache.org/jira/browse/HIVE-9393 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.1 Reporter: Yongzhi Chen Priority: Minor From Hive 0.13 the log level of ColumnarSerDe.java:116 was upgraded from DEBUG to INFO, this has introduced an very large amount of noise into the logs causing the underlying filesystem to fill up. This request is to drop is back to DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7313) Allow in-memory/ssd session-level temp-tables
[ https://issues.apache.org/jira/browse/HIVE-7313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-7313: -- Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk, thanks [~hagleitn]. Allow in-memory/ssd session-level temp-tables - Key: HIVE-7313 URL: https://issues.apache.org/jira/browse/HIVE-7313 Project: Hive Issue Type: Improvement Components: Tez Affects Versions: 0.14.0 Reporter: Gopal V Assignee: Gopal V Labels: InMemory, Performance, TODOC15 Fix For: 0.15.0 Attachments: HIVE-7313.1.patch, HIVE-7313.2.patch With HDFS storage policies implementation, temporary tables can be written with different storage/reliability policies. In-session temporary tables can be targetted at both SSD and memory storage policies, with fallbacks onto the disk and the associated reliability trade-offs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9393) reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG
[ https://issues.apache.org/jira/browse/HIVE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzhi Chen updated HIVE-9393: --- Status: Patch Available (was: Open) Need code review. reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG --- Key: HIVE-9393 URL: https://issues.apache.org/jira/browse/HIVE-9393 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.1 Reporter: Yongzhi Chen Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-9393.1.patch From Hive 0.13 the log level of ColumnarSerDe.java:116 was upgraded from DEBUG to INFO, this has introduced an very large amount of noise into the logs causing the underlying filesystem to fill up. This request is to drop is back to DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9393) reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG
[ https://issues.apache.org/jira/browse/HIVE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzhi Chen updated HIVE-9393: --- Attachment: HIVE-9393.1.patch reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG --- Key: HIVE-9393 URL: https://issues.apache.org/jira/browse/HIVE-9393 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.1 Reporter: Yongzhi Chen Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-9393.1.patch From Hive 0.13 the log level of ColumnarSerDe.java:116 was upgraded from DEBUG to INFO, this has introduced an very large amount of noise into the logs causing the underlying filesystem to fill up. This request is to drop is back to DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9393) reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG
[ https://issues.apache.org/jira/browse/HIVE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279459#comment-14279459 ] Brock Noland commented on HIVE-9393: Seems fine, anyone can enable debug mode for that class if needed. +1 pending tests reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG --- Key: HIVE-9393 URL: https://issues.apache.org/jira/browse/HIVE-9393 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.1 Reporter: Yongzhi Chen Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-9393.1.patch From Hive 0.13 the log level of ColumnarSerDe.java:116 was upgraded from DEBUG to INFO, this has introduced an very large amount of noise into the logs causing the underlying filesystem to fill up. This request is to drop is back to DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9200) CBO (Calcite Return Path): Inline Join, Properties
[ https://issues.apache.org/jira/browse/HIVE-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9200: -- Attachment: HIVE-9200.10.patch Uploading rebased patch. CBO (Calcite Return Path): Inline Join, Properties -- Key: HIVE-9200 URL: https://issues.apache.org/jira/browse/HIVE-9200 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9200.01.patch, HIVE-9200.02.patch, HIVE-9200.03.patch, HIVE-9200.04.patch, HIVE-9200.05.patch, HIVE-9200.06.patch, HIVE-9200.07.patch, HIVE-9200.08.patch, HIVE-9200.09.patch, HIVE-9200.10.patch, HIVE-9200.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9292) CBO (Calcite Return Path): Inline GroupBy, Properties
[ https://issues.apache.org/jira/browse/HIVE-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279527#comment-14279527 ] Jesus Camacho Rodriguez commented on HIVE-9292: --- Yes, I could get rid of the groupOpToInputTables object in ParseContext. However, ParseContext is still used by RewriteQueryUsingAggregateIndexCtx, as it needs to retrieve the RowResolver for the group operator that is retrieved from the tree. Once HIVE-9327 goes in, RewriteQueryUsingAggregateIndexCtx won't use any of the data structures in ParseContext anymore. I will wait till HIVE-9200 goes in to rebase the patch and upload it again. CBO (Calcite Return Path): Inline GroupBy, Properties - Key: HIVE-9292 URL: https://issues.apache.org/jira/browse/HIVE-9292 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9292.01.patch, HIVE-9292.02.patch, HIVE-9292.03.patch, HIVE-9292.04.patch, HIVE-9292.patch, HIVE-9292.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9292) CBO (Calcite Return Path): Inline GroupBy, Properties
[ https://issues.apache.org/jira/browse/HIVE-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279368#comment-14279368 ] Laljo John Pullokkaran commented on HIVE-9292: -- [~jcamachorodriguez] Does the latest patch address the review comments? CBO (Calcite Return Path): Inline GroupBy, Properties - Key: HIVE-9292 URL: https://issues.apache.org/jira/browse/HIVE-9292 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9292.01.patch, HIVE-9292.02.patch, HIVE-9292.03.patch, HIVE-9292.04.patch, HIVE-9292.patch, HIVE-9292.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9194) Support select distinct *
[ https://issues.apache.org/jira/browse/HIVE-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279384#comment-14279384 ] Laljo John Pullokkaran commented on HIVE-9194: -- committed to trunk. Thanks [~pxiong] Support select distinct * - Key: HIVE-9194 URL: https://issues.apache.org/jira/browse/HIVE-9194 Project: Hive Issue Type: Sub-task Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong Attachments: HIVE-9194.00.patch, HIVE-9194.01.patch, HIVE-9194.02.patch, HIVE-9194.03.patch, HIVE-9194.04.patch As per [~jpullokkaran]'s review comments, implement select distinct * -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9194) Support select distinct *
[ https://issues.apache.org/jira/browse/HIVE-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-9194: - Resolution: Fixed Status: Resolved (was: Patch Available) Support select distinct * - Key: HIVE-9194 URL: https://issues.apache.org/jira/browse/HIVE-9194 Project: Hive Issue Type: Sub-task Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong Attachments: HIVE-9194.00.patch, HIVE-9194.01.patch, HIVE-9194.02.patch, HIVE-9194.03.patch, HIVE-9194.04.patch As per [~jpullokkaran]'s review comments, implement select distinct * -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9315) CBO (Calcite Return Path): Inline FileSinkOperator, Properties
[ https://issues.apache.org/jira/browse/HIVE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-9315: - Resolution: Fixed Status: Resolved (was: Patch Available) CBO (Calcite Return Path): Inline FileSinkOperator, Properties -- Key: HIVE-9315 URL: https://issues.apache.org/jira/browse/HIVE-9315 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9315.01.patch, HIVE-9315.02.patch, HIVE-9315.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9347) Bug with max() together with rank() and grouping sets
[
https://issues.apache.org/jira/browse/HIVE-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279431#comment-14279431
]
Hive QA commented on HIVE-9347:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692183/HIVE-9347.1.patch.txt
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 7312 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_window
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_limit_pushdown
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_limit_pushdown
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_cube1
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_rollup1
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_limit_pushdown
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_semijoin
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2378/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2378/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2378/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692183 - PreCommit-HIVE-TRUNK-Build
Bug with max() together with rank() and grouping sets
-
Key: HIVE-9347
URL: https://issues.apache.org/jira/browse/HIVE-9347
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Environment: Amazon Elastic Map Reduce, AMI 3.3.1, Hadoop Amazon
2.4.0, Hive 0.13.1
Reporter: Michal Krawczyk
Assignee: Navis
Attachments: HIVE-9347.1.patch.txt
It looks like the query below returns incorrect results on Hive 0.13.1, but
it was working fine on Hive 0.11.
I have the following table:
CREATE TABLE `t`(
`category` int,
`live` int,
`comments` int)
with the following data:
hive select * from t;
OK
3 0 2
2 0 2
8 0 2
The query:
hive select category, max(live) live, max(comments) comments, rank() OVER
(PARTITION BY category ORDER BY comments) rank1
FROM t
GROUP BY category
GROUPING SETS ((), (category))
HAVING max(comments) 0;
return the following results:
NULL1 48 1
2 1 49 1
3 1 49 1
8 1 49 1
When using grouping sets with the rank() function the max() function return
incorrect results. Everything works fine if I remove grouping sets clause and
split the query into two independent queries or remove the rank() function.
This looks like a bug to me but please review. That said, I'm not sure if
it's just Amazon issue or general Hive issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9272) Tests for utf-8 support
[ https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aswathy Chellammal Sreekumar updated HIVE-9272: --- Attachment: HIVE-9272.3.patch Attaching the patch with encoded characters replaced with original characters in file name. Please review the same. Tests for utf-8 support --- Key: HIVE-9272 URL: https://issues.apache.org/jira/browse/HIVE-9272 Project: Hive Issue Type: Test Components: Tests, WebHCat Affects Versions: 0.14.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.3.patch, HIVE-9272.patch Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9292) CBO (Calcite Return Path): Inline GroupBy, Properties
[
https://issues.apache.org/jira/browse/HIVE-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279437#comment-14279437
]
Hive QA commented on HIVE-9292:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692505/HIVE-9292.04.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2379/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2379/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2379/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2379/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets4.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_cube1.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets6.q.out'
Reverted 'ql/src/test/results/clientpositive/annotate_stats_groupby2.q.out'
Reverted 'ql/src/test/results/clientpositive/annotate_stats_groupby.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets3.q.out'
Reverted
'ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets5.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_rollup1.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets2.q.out'
Reverted
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java'
Reverted
'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20S/target
shims/0.23/target shims/aggregator/target shims/common/target
shims/scheduler/target packaging/target hbase-handler/target testutils/target
jdbc/target metastore/target itests/target itests/thirdparty
itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target
itests/hive-unit-hadoop2/target itests/hive-minikdc/target
itests/hive-unit/target itests/custom-serde/target itests/util/target
itests/qtest-spark/target hcatalog/target hcatalog/core/target
hcatalog/streaming/target hcatalog/server-extensions/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target accumulo-handler/target hwi/target
common/target common/src/gen spark-client/target contrib/target service/target
serde/target beeline/target odbc/target cli/target
ql/dependency-reduced-pom.xml ql/target
ql/src/test/queries/clientpositive/groupby_grouping_window.q
+ svn update
Ushims/0.20S/src/main/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java
Ushims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
Ushims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java
Uitests/src/test/resources/testconfiguration.properties
Ucommon/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Aql/src/test/results/clientnegative/selectDistinctStarNeg_1.q.out
Aql/src/test/results/clientnegative/selectDistinctStarNeg_2.q.out
Aql/src/test/results/clientpositive/selectDistinctStar.q.out
A
[jira] [Commented] (HIVE-9376) Add back some APIs which changed from 0.13 for Spark SQL
[
https://issues.apache.org/jira/browse/HIVE-9376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279570#comment-14279570
]
Hive QA commented on HIVE-9376:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692298/HIVE-9376.1.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 7315 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2381/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2381/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2381/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692298 - PreCommit-HIVE-TRUNK-Build
Add back some APIs which changed from 0.13 for Spark SQL
Key: HIVE-9376
URL: https://issues.apache.org/jira/browse/HIVE-9376
Project: Hive
Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Brock Noland
Fix For: 0.15.0
Attachments: HIVE-9376.1.patch
As [discussed on
HIVE-9364|https://issues.apache.org/jira/browse/HIVE-9364?focusedCommentId=14276100page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14276100]
some APIs were changed which broke SparkSQL. This change adds them back,
marks them as Deprecated, Unstable, and in-use by SparkSQL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Review Request 29954: HIVE-9179. Add listener API to JobHandle.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29954/ --- Review request for hive, Brock Noland, chengxiang li, and Xuefu Zhang. Bugs: HIVE-9179 https://issues.apache.org/jira/browse/HIVE-9179 Repository: hive-git Description --- HIVE-9179. Add listener API to JobHandle. Diffs - spark-client/pom.xml 77016df61a0bcbd94058bcbd2825c6c210a70e14 spark-client/src/main/java/org/apache/hive/spark/client/BaseProtocol.java f9c10b196ab47b5b4f4c0126ad455869ab68f0ca spark-client/src/main/java/org/apache/hive/spark/client/JobHandle.java e760ce35d92bedf4d301b08ec57d1c2dc37a39f0 spark-client/src/main/java/org/apache/hive/spark/client/JobHandleImpl.java 1b8feedb0b23aa7897dc6ac37ea5c0209e71d573 spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java 0d49ed3d9e33ca08d6a7526c1c434a0dd0a06a67 spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java a30d8cbbaae9d25b1cffdc286b546f549e439545 spark-client/src/test/java/org/apache/hive/spark/client/TestJobHandle.java PRE-CREATION spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab Diff: https://reviews.apache.org/r/29954/diff/ Testing --- Thanks, Marcelo Vanzin
[jira] [Updated] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9179: - Status: Patch Available (was: Open) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch] Key: HIVE-9179 URL: https://issues.apache.org/jira/browse/HIVE-9179 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Xuefu Zhang Assignee: Marcelo Vanzin Attachments: HIVE-9179.1-spark.patch Based on discussion in HIVE-8972, it seems nice to add listeners on a job handle such that state changes of a submitted a job can be notified instead of the current approach of client polling for such changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9179: - Attachment: HIVE-9179.1-spark.patch Add listeners on JobHandle so job status change can be notified to the client [Spark Branch] Key: HIVE-9179 URL: https://issues.apache.org/jira/browse/HIVE-9179 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Xuefu Zhang Assignee: Marcelo Vanzin Attachments: HIVE-9179.1-spark.patch Based on discussion in HIVE-8972, it seems nice to add listeners on a job handle such that state changes of a submitted a job can be notified instead of the current approach of client polling for such changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9200) CBO (Calcite Return Path): Inline Join, Properties
[ https://issues.apache.org/jira/browse/HIVE-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-9200: - Status: Patch Available (was: Open) CBO (Calcite Return Path): Inline Join, Properties -- Key: HIVE-9200 URL: https://issues.apache.org/jira/browse/HIVE-9200 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9200.01.patch, HIVE-9200.02.patch, HIVE-9200.03.patch, HIVE-9200.04.patch, HIVE-9200.05.patch, HIVE-9200.06.patch, HIVE-9200.07.patch, HIVE-9200.08.patch, HIVE-9200.09.patch, HIVE-9200.10.patch, HIVE-9200.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9200) CBO (Calcite Return Path): Inline Join, Properties
[ https://issues.apache.org/jira/browse/HIVE-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-9200: - Status: Open (was: Patch Available) CBO (Calcite Return Path): Inline Join, Properties -- Key: HIVE-9200 URL: https://issues.apache.org/jira/browse/HIVE-9200 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9200.01.patch, HIVE-9200.02.patch, HIVE-9200.03.patch, HIVE-9200.04.patch, HIVE-9200.05.patch, HIVE-9200.06.patch, HIVE-9200.07.patch, HIVE-9200.08.patch, HIVE-9200.09.patch, HIVE-9200.10.patch, HIVE-9200.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9264) Merge encryption branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-9264: --- Attachment: HIVE-9264.2.patch reattach to trigger the jenkins test Merge encryption branch to trunk Key: HIVE-9264 URL: https://issues.apache.org/jira/browse/HIVE-9264 Project: Hive Issue Type: Sub-task Affects Versions: 0.15.0 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9264.1.patch, HIVE-9264.2.patch The team working on the encryption branch would like to merge their work to trunk. This jira will track that effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9394) SparkCliDriver tests have sporadic timeout error
Szehon Ho created HIVE-9394:
---
Summary: SparkCliDriver tests have sporadic timeout error
Key: HIVE-9394
URL: https://issues.apache.org/jira/browse/HIVE-9394
Project: Hive
Issue Type: Test
Components: Tests
Affects Versions: 0.15.0
Reporter: Szehon Ho
Assignee: Szehon Ho
There have been some sporadic exceptions in pre-commit tests like:
{noformat}
2015-01-15 08:31:40,805 WARN [main]: client.SparkClientImpl
(SparkClientImpl.java:init(90)) - Error while waiting for client to connect.
java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException:
Timed out waiting for client connection.
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
at
org.apache.hive.spark.client.SparkClientImpl.init(SparkClientImpl.java:88)
at
org.apache.hive.spark.client.SparkClientFactory.createClient(SparkClientFactory.java:75)
at
org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.init(RemoteHiveSparkClient.java:82)
at
org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactory.createHiveSparkClient(HiveSparkClientFactory.java:53)
at
org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:56)
at
org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionManagerImpl.getSession(SparkSessionManagerImpl.java:128)
at
org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.getSparkSession(SparkUtilities.java:84)
at
org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:96)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1634)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1393)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1179)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:305)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:882)
at
org.apache.hadoop.hive.cli.TestSparkCliDriver.runTest(TestSparkCliDriver.java:234)
at
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_alter_merge_orc(TestSparkCliDriver.java:162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:176)
at junit.framework.TestCase.runBare(TestCase.java:141)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:129)
at junit.framework.TestSuite.runTest(TestSuite.java:255)
at junit.framework.TestSuite.run(TestSuite.java:250)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for client
connection.
at org.apache.hive.spark.client.rpc.RpcServer$2.run(RpcServer.java:125)
at
io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at
io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:123)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:380)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)
[jira] [Commented] (HIVE-9264) Merge encryption branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279596#comment-14279596 ] Ferdinand Xu commented on HIVE-9264: Thanks [~brocknoland] for your patch. +1 Merge encryption branch to trunk Key: HIVE-9264 URL: https://issues.apache.org/jira/browse/HIVE-9264 Project: Hive Issue Type: Sub-task Affects Versions: 0.15.0 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9264.1.patch The team working on the encryption branch would like to merge their work to trunk. This jira will track that effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9377) UDF in_file() in WHERE predicate causes NPE.
[
https://issues.apache.org/jira/browse/HIVE-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279661#comment-14279661
]
Hive QA commented on HIVE-9377:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692320/HIVE-9377.1.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 7315 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2382/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2382/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2382/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692320 - PreCommit-HIVE-TRUNK-Build
UDF in_file() in WHERE predicate causes NPE.
Key: HIVE-9377
URL: https://issues.apache.org/jira/browse/HIVE-9377
Project: Hive
Issue Type: Bug
Components: UDF
Reporter: Mithun Radhakrishnan
Assignee: Mithun Radhakrishnan
Attachments: HIVE-9377.1.patch
Consider the following query:
{code:sql}
SELECT foo, bar from mythdb.foobar where in_file( bar, '/tmp/bar_list.txt' );
{code}
Using {{in_file()}} in a WHERE predicate causes the following NPE:
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getWritableConstantValue(ObjectInspectorUtils.java:1041)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFInFile.getRequiredFiles(GenericUDFInFile.java:93)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.isDeterministicUdf(ConstantPropagateProcFactory.java:303)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:226)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.access$000(ConstantPropagateProcFactory.java:92)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory$ConstantPropagateFilterProc.process(ConstantPropagateProcFactory.java:623)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagate$ConstantPropagateWalker.walk(ConstantPropagate.java:147)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
at
org.apache.hadoop.hive.ql.optimizer.ConstantPropagate.transform(ConstantPropagate.java:117)
at
org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:177)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10032)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:189)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:420)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1108)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1156)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:701)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:674)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29954/#review68392 --- Ship it! Ship It! - chengxiang li On 一月 16, 2015, 1:05 a.m., Marcelo Vanzin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29954/ --- (Updated 一月 16, 2015, 1:05 a.m.) Review request for hive, Brock Noland, chengxiang li, and Xuefu Zhang. Bugs: HIVE-9179 https://issues.apache.org/jira/browse/HIVE-9179 Repository: hive-git Description --- HIVE-9179. Add listener API to JobHandle. Diffs - spark-client/pom.xml 77016df61a0bcbd94058bcbd2825c6c210a70e14 spark-client/src/main/java/org/apache/hive/spark/client/BaseProtocol.java f9c10b196ab47b5b4f4c0126ad455869ab68f0ca spark-client/src/main/java/org/apache/hive/spark/client/JobHandle.java e760ce35d92bedf4d301b08ec57d1c2dc37a39f0 spark-client/src/main/java/org/apache/hive/spark/client/JobHandleImpl.java 1b8feedb0b23aa7897dc6ac37ea5c0209e71d573 spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java 0d49ed3d9e33ca08d6a7526c1c434a0dd0a06a67 spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java a30d8cbbaae9d25b1cffdc286b546f549e439545 spark-client/src/test/java/org/apache/hive/spark/client/TestJobHandle.java PRE-CREATION spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab Diff: https://reviews.apache.org/r/29954/diff/ Testing --- Thanks, Marcelo Vanzin
[jira] [Commented] (HIVE-9264) Merge encryption branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279734#comment-14279734 ] Szehon Ho commented on HIVE-9264: - +1 pending tests. Applied the patch cleanly and went through. New methods are well-documented and there's a good number of tests. Also I'm familiar with the work, having done some permission inheritance testing for this branch. Merge encryption branch to trunk Key: HIVE-9264 URL: https://issues.apache.org/jira/browse/HIVE-9264 Project: Hive Issue Type: Sub-task Affects Versions: 0.15.0 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9264.1.patch, HIVE-9264.2.patch The team working on the encryption branch would like to merge their work to trunk. This jira will track that effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279735#comment-14279735
]
Hive QA commented on HIVE-9179:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692657/HIVE-9179.1-spark.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7312 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/647/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/647/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-647/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692657 - PreCommit-HIVE-SPARK-Build
Add listeners on JobHandle so job status change can be notified to the client
[Spark Branch]
Key: HIVE-9179
URL: https://issues.apache.org/jira/browse/HIVE-9179
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Xuefu Zhang
Assignee: Marcelo Vanzin
Attachments: HIVE-9179.1-spark.patch
Based on discussion in HIVE-8972, it seems nice to add listeners on a job
handle such that state changes of a submitted a job can be notified instead
of the current approach of client polling for such changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9298) Support reading alternate timestamp formats
[
https://issues.apache.org/jira/browse/HIVE-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279749#comment-14279749
]
Hive QA commented on HIVE-9298:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692344/HIVE-9298.1.patch
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 7321 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_overwrite_local_directory_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nested_complex
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_create
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_array_null_element
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_create
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_nested_complex
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_map_queries_prefix
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.testOutputFormat
org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2383/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2383/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2383/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692344 - PreCommit-HIVE-TRUNK-Build
Support reading alternate timestamp formats
---
Key: HIVE-9298
URL: https://issues.apache.org/jira/browse/HIVE-9298
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-9298.1.patch
There are some users who want to be able to parse ISO-8601 timestamps, as
well to set their own custom timestamp formats. We may be able to support
this in LazySimpleSerDe through the use of a SerDe parameter to specify one
or more alternative timestamp patterns to use to parse timestamp values from
string.
If we are doing this it might also be nice to work in support for HIVE-3844,
to parse numeric strings as timestamp by treating the numeric value as millis
since Unix epoch. This can be enabled through the SerDe params as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9352) Merge from spark to trunk (follow-up of HIVE-9257)
[
https://issues.apache.org/jira/browse/HIVE-9352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278422#comment-14278422
]
Szehon Ho commented on HIVE-9352:
-
Failures do not look related. TestMTQueries and TestStreaming fail in the next
two builds as well, see
([2366|http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2366]/
[2367|http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2367]).
The spark failure is non-reproducible and from [test
logs|http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2365/failed/TestSparkCliDriver-auto_join_reordering_values.q-auto_sortmerge_join_7.q-insert1.q-and-12-more/hive.log]
is due to the following error:
{noformat}
2015-01-14 18:47:49,859 WARN [main]: client.SparkClientImpl
(SparkClientImpl.java:init(90)) - Error while waiting for client to connect.
java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException:
Timed out waiting for client connection.
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
at
org.apache.hive.spark.client.SparkClientImpl.init(SparkClientImpl.java:88)
at
org.apache.hive.spark.client.SparkClientFactory.createClient(SparkClientFactory.java:75)
at
org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.init(RemoteHiveSparkClient.java:82)
at
org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactory.createHiveSparkClient(HiveSparkClientFactory.java:53)
at
org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:56)
at
org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionManagerImpl.getSession(SparkSessionManagerImpl.java:128)
at
org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.getSparkSession(SparkUtilities.java:84)
{noformat}
I believe its the same one I've discussed with [~chengxiang li] in HIVE-9094 ,
and I think it can be fixed by increasing a client-connect timeout. I'll take
a look after this.
Merge from spark to trunk (follow-up of HIVE-9257)
--
Key: HIVE-9352
URL: https://issues.apache.org/jira/browse/HIVE-9352
Project: Hive
Issue Type: Sub-task
Components: Spark
Affects Versions: 0.15.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-9352.patch, HIVE-9352.patch
Will include following JIRA's (not-inclusive list)
HIVE-7674 (remove spark-snapshot dependency)
HIVE-9335 (cleanup)
HIVE-9340 (cleanup 2, including removing spark snapshot repo)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9372) Parallel checking non-combinable paths in CombineHiveInputFormat
[ https://issues.apache.org/jira/browse/HIVE-9372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-9372: - Attachment: HIVE-9372.3-spark.patch Upload a spark version Parallel checking non-combinable paths in CombineHiveInputFormat Key: HIVE-9372 URL: https://issues.apache.org/jira/browse/HIVE-9372 Project: Hive Issue Type: Improvement Reporter: Rui Li Assignee: Rui Li Attachments: HIVE-9372.1.patch, HIVE-9372.2.patch, HIVE-9372.3-spark.patch Checking if an input path is combinable is expensive. So we should make it parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9315) CBO (Calcite Return Path): Inline FileSinkOperator, Properties
[ https://issues.apache.org/jira/browse/HIVE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9315: -- Status: Open (was: Patch Available) CBO (Calcite Return Path): Inline FileSinkOperator, Properties -- Key: HIVE-9315 URL: https://issues.apache.org/jira/browse/HIVE-9315 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9315.01.patch, HIVE-9315.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9315) CBO (Calcite Return Path): Inline FileSinkOperator, Properties
[ https://issues.apache.org/jira/browse/HIVE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9315: -- Status: Patch Available (was: Open) CBO (Calcite Return Path): Inline FileSinkOperator, Properties -- Key: HIVE-9315 URL: https://issues.apache.org/jira/browse/HIVE-9315 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9315.01.patch, HIVE-9315.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9292) CBO (Calcite Return Path): Inline GroupBy, Properties
[ https://issues.apache.org/jira/browse/HIVE-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9292: -- Status: Patch Available (was: Open) CBO (Calcite Return Path): Inline GroupBy, Properties - Key: HIVE-9292 URL: https://issues.apache.org/jira/browse/HIVE-9292 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9292.01.patch, HIVE-9292.02.patch, HIVE-9292.03.patch, HIVE-9292.04.patch, HIVE-9292.patch, HIVE-9292.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9315) CBO (Calcite Return Path): Inline FileSinkOperator, Properties
[ https://issues.apache.org/jira/browse/HIVE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9315: -- Attachment: (was: HIVE-9315.02.patch) CBO (Calcite Return Path): Inline FileSinkOperator, Properties -- Key: HIVE-9315 URL: https://issues.apache.org/jira/browse/HIVE-9315 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9315.01.patch, HIVE-9315.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9292) CBO (Calcite Return Path): Inline GroupBy, Properties
[ https://issues.apache.org/jira/browse/HIVE-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9292: -- Attachment: HIVE-9292.04.patch CBO (Calcite Return Path): Inline GroupBy, Properties - Key: HIVE-9292 URL: https://issues.apache.org/jira/browse/HIVE-9292 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9292.01.patch, HIVE-9292.02.patch, HIVE-9292.03.patch, HIVE-9292.04.patch, HIVE-9292.patch, HIVE-9292.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9272) Tests for utf-8 support
[
https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278589#comment-14278589
]
Hive QA commented on HIVE-9272:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692284/HIVE-9272.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7311 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2370/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2370/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2370/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692284 - PreCommit-HIVE-TRUNK-Build
Tests for utf-8 support
---
Key: HIVE-9272
URL: https://issues.apache.org/jira/browse/HIVE-9272
Project: Hive
Issue Type: Test
Components: Tests, WebHCat
Reporter: Aswathy Chellammal Sreekumar
Assignee: Aswathy Chellammal Sreekumar
Priority: Minor
Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.patch
Including some test cases for utf8 support in webhcat. The first four tests
invoke hive, pig, mapred and streaming apis for testing the utf8 support for
data processed, file names and job name. The last test case tests the
filtering of job name with utf8 character
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9370) Enable Hive on Spark for BigBench and run Query 8, the test failed [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278557#comment-14278557 ] Rui Li commented on HIVE-9370: -- At client side we wait 30s for a job to be submitted before we decide the job submission has failed, i.e. by trying to get a job ID from the driver. [~chengxiang li] mentioned that in some cases (especially spark on yarn mode) it may take a longer time for a job to be submitted so we may need to increase that timeout. Enable Hive on Spark for BigBench and run Query 8, the test failed [Spark Branch] - Key: HIVE-9370 URL: https://issues.apache.org/jira/browse/HIVE-9370 Project: Hive Issue Type: Sub-task Components: Spark Reporter: yuyun.chen enable hive on spark and run BigBench Query 8 then got the following exception: 2015-01-14 11:43:46,057 INFO [main]: impl.RemoteSparkJobStatus (RemoteSparkJobStatus.java:getSparkJobInfo(143)) - Job hasn't been submitted after 30s. Aborting it. 2015-01-14 11:43:46,061 INFO [main]: impl.RemoteSparkJobStatus (RemoteSparkJobStatus.java:getSparkJobInfo(143)) - Job hasn't been submitted after 30s. Aborting it. 2015-01-14 11:43:46,061 ERROR [main]: status.SparkJobMonitor (SessionState.java:printError(839)) - Status: Failed 2015-01-14 11:43:46,062 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - /PERFLOG method=SparkRunJob start=1421206996052 end=1421207026062 duration=30010 from=org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - 15/01/14 11:43:46 INFO RemoteDriver: Failed to run job 0a9a7782-0e0b-4561-8468-959a6d8df0a3 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - java.lang.InterruptedException 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at java.lang.Object.wait(Native Method) 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at java.lang.Object.wait(Object.java:503) 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.scheduler.JobWaiter.awaitResult(JobWaiter.scala:73) 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:514) 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.SparkContext.runJob(SparkContext.scala:1282) 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.SparkContext.runJob(SparkContext.scala:1300) 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.SparkContext.runJob(SparkContext.scala:1314) 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.SparkContext.runJob(SparkContext.scala:1328) 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.rdd.RDD.collect(RDD.scala:780) 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.RangePartitioner$.sketch(Partitioner.scala:262) 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.RangePartitioner.init(Partitioner.scala:124) 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.rdd.OrderedRDDFunctions.sortByKey(OrderedRDDFunctions.scala:63) 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.api.java.JavaPairRDD.sortByKey(JavaPairRDD.scala:894) 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.spark.api.java.JavaPairRDD.sortByKey(JavaPairRDD.scala:864) 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at org.apache.hadoop.hive.ql.exec.spark.SortByShuffler.shuffle(SortByShuffler.java:48) 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) -at
[jira] [Updated] (HIVE-9315) CBO (Calcite Return Path): Inline FileSinkOperator, Properties
[ https://issues.apache.org/jira/browse/HIVE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9315: -- Attachment: HIVE-9315.02.patch CBO (Calcite Return Path): Inline FileSinkOperator, Properties -- Key: HIVE-9315 URL: https://issues.apache.org/jira/browse/HIVE-9315 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9315.01.patch, HIVE-9315.02.patch, HIVE-9315.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6332) HCatConstants Documentation needed
[ https://issues.apache.org/jira/browse/HIVE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278534#comment-14278534 ] Lefty Leverenz commented on HIVE-6332: -- +1 [~sushanth] finished the documentation in the Setup section of HCatalog Config Properties in the wiki, so this issue can be resolved as fixed. * [HCatalog Config Properties -- Setup | https://cwiki.apache.org/confluence/display/Hive/HCatalog+Config+Properties#HCatalogConfigProperties-Setup:] HCatConstants Documentation needed -- Key: HIVE-6332 URL: https://issues.apache.org/jira/browse/HIVE-6332 Project: Hive Issue Type: Task Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan HCatConstants documentation is near non-existent, being defined only as comments in code for the various parameters. Given that a lot of api winds up being implemented as knobs that can be tweaked here, we should have a public facing doc for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9387) Shutting down cli takes quite some time [Spark Branch]
Rui Li created HIVE-9387:
Summary: Shutting down cli takes quite some time [Spark Branch]
Key: HIVE-9387
URL: https://issues.apache.org/jira/browse/HIVE-9387
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Rui Li
It's quite slow to shut down cli driver when running HOS. And the following
warning can be found in log:
{noformat}
2015-01-15 11:22:13,651 WARN [main]: client.SparkClientImpl
(SparkClientImpl.java:stop(125)) - Timed out waiting for driver to respond to
stop request.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9292) CBO (Calcite Return Path): Inline GroupBy, Properties
[ https://issues.apache.org/jira/browse/HIVE-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9292: -- Status: Open (was: Patch Available) CBO (Calcite Return Path): Inline GroupBy, Properties - Key: HIVE-9292 URL: https://issues.apache.org/jira/browse/HIVE-9292 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9292.01.patch, HIVE-9292.02.patch, HIVE-9292.03.patch, HIVE-9292.patch, HIVE-9292.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9292) CBO (Calcite Return Path): Inline GroupBy, Properties
[ https://issues.apache.org/jira/browse/HIVE-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9292: -- Attachment: (was: HIVE-9292.04.patch) CBO (Calcite Return Path): Inline GroupBy, Properties - Key: HIVE-9292 URL: https://issues.apache.org/jira/browse/HIVE-9292 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 0.15.0 Attachments: HIVE-9292.01.patch, HIVE-9292.02.patch, HIVE-9292.03.patch, HIVE-9292.patch, HIVE-9292.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9309) schematool fails on Postgres 8.1
[
https://issues.apache.org/jira/browse/HIVE-9309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lefty Leverenz updated HIVE-9309:
-
Labels: TODOC15 (was: )
schematool fails on Postgres 8.1
Key: HIVE-9309
URL: https://issues.apache.org/jira/browse/HIVE-9309
Project: Hive
Issue Type: Bug
Components: Database/Schema
Reporter: Mohit Sabharwal
Assignee: Mohit Sabharwal
Labels: TODOC15
Fix For: 0.15.0
Attachments: HIVE-9309.patch
Postgres upgrade scripts set {{standard_conforming_strings}} which is not
allowed in 8.1:
{code}
ERROR: parameter standard_conforming_strings cannot be changed
(state=55P02,code=0)
{code}
Postgres [8.1 Release
notes|http://www.postgresql.org/docs/8.2/static/release-8-1.html] say that
standard_conforming_strings value is read-only
Postgres [8.2
notes|http://www.postgresql.org/docs/8.2/static/release-8-2.html] say that it
can be set at runtime.
It'd be nice to address this for those still using Postgres 8.1
This patch provides a schemaTool db option postgres.filter.81 which, if
set, filters out the standard_conforming_strings statement from upgrade
scripts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9352) Merge from spark to trunk (follow-up of HIVE-9257)
[ https://issues.apache.org/jira/browse/HIVE-9352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-9352: Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk, thanks for the review Merge from spark to trunk (follow-up of HIVE-9257) -- Key: HIVE-9352 URL: https://issues.apache.org/jira/browse/HIVE-9352 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: 0.15.0 Reporter: Szehon Ho Assignee: Szehon Ho Fix For: 0.15.0 Attachments: HIVE-9352.patch, HIVE-9352.patch Will include following JIRA's (not-inclusive list) HIVE-7674 (remove spark-snapshot dependency) HIVE-9335 (cleanup) HIVE-9340 (cleanup 2, including removing spark snapshot repo) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9331) get rid of pre-optimized-hashtable memory optimizations
[
https://issues.apache.org/jira/browse/HIVE-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278431#comment-14278431
]
Hive QA commented on HIVE-9331:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692027/HIVE-9331.02.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7313 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2368/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2368/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2368/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692027 - PreCommit-HIVE-TRUNK-Build
get rid of pre-optimized-hashtable memory optimizations
---
Key: HIVE-9331
URL: https://issues.apache.org/jira/browse/HIVE-9331
Project: Hive
Issue Type: Bug
Affects Versions: 0.15.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.15.0
Attachments: HIVE-9331.01.patch, HIVE-9331.01.patch,
HIVE-9331.02.patch, HIVE-9331.patch, HIVE-9331.patch
These were added in 13 because optimized hashtable couldn't make it in; they
reduced memory usage by some amount (10-25%), and informed the design of the
optimized hashtable, but now extra settings and code branches are just
confusing and may have their own bugs. Might as well remove them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8966) Delta files created by hive hcatalog streaming cannot be compacted
[
https://issues.apache.org/jira/browse/HIVE-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278515#comment-14278515
]
Hive QA commented on HIVE-8966:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692048/HIVE-8966.5.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7330 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2369/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2369/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2369/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692048 - PreCommit-HIVE-TRUNK-Build
Delta files created by hive hcatalog streaming cannot be compacted
--
Key: HIVE-8966
URL: https://issues.apache.org/jira/browse/HIVE-8966
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.14.0
Environment: hive
Reporter: Jihong Liu
Assignee: Alan Gates
Priority: Critical
Fix For: 0.14.1
Attachments: HIVE-8966.2.patch, HIVE-8966.3.patch, HIVE-8966.4.patch,
HIVE-8966.5.patch, HIVE-8966.patch
hive hcatalog streaming will also create a file like bucket_n_flush_length in
each delta directory. Where n is the bucket number. But the
compactor.CompactorMR think this file also needs to compact. However this
file of course cannot be compacted, so compactor.CompactorMR will not
continue to do the compaction.
Did a test, after removed the bucket_n_flush_length file, then the alter
table partition compact finished successfully. If don't delete that file,
nothing will be compacted.
This is probably a very severity bug. Both 0.13 and 0.14 have this issue
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9387) Shutting down cli takes quite some time [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9387:
-
Attachment: HIVE-9387.1-spark.patch
Seems we shouldn't wait for a reply from an EndSession as the channel is
closed. And we already have a daemon thread to watch the driver process.
The patch can solve the issue on my machine.
Shutting down cli takes quite some time [Spark Branch]
--
Key: HIVE-9387
URL: https://issues.apache.org/jira/browse/HIVE-9387
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Rui Li
Assignee: Rui Li
Attachments: HIVE-9387.1-spark.patch
It's quite slow to shut down cli driver when running HOS. And the following
warning can be found in log:
{noformat}
2015-01-15 11:22:13,651 WARN [main]: client.SparkClientImpl
(SparkClientImpl.java:stop(125)) - Timed out waiting for driver to respond to
stop request.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9366) wrong date in description annotation in date_add() and date_sub() udf
[
https://issues.apache.org/jira/browse/HIVE-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278648#comment-14278648
]
Hive QA commented on HIVE-9366:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12692118/HIVE-9366.3.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 7311 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2371/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2371/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2371/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12692118 - PreCommit-HIVE-TRUNK-Build
wrong date in description annotation in date_add() and date_sub() udf
-
Key: HIVE-9366
URL: https://issues.apache.org/jira/browse/HIVE-9366
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.1
Reporter: Alexander Pivovarov
Assignee: Alexander Pivovarov
Priority: Trivial
Attachments: HIVE-9366.1.patch, HIVE-9366.2.patch, HIVE-9366.3.patch
files:
ql/src/test/results/clientpositive/udf_date_add.q.out
ql/src/test/results/beelinepositive/udf_date_add.q.out
last line shows '2009-31-07' but it should be '2009-07-31' instead
the @Description annotation needs to be fixed for both date_add() and
date_sub()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-9387) Shutting down cli takes quite some time [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-9387:
Assignee: Rui Li
Shutting down cli takes quite some time [Spark Branch]
--
Key: HIVE-9387
URL: https://issues.apache.org/jira/browse/HIVE-9387
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Rui Li
Assignee: Rui Li
It's quite slow to shut down cli driver when running HOS. And the following
warning can be found in log:
{noformat}
2015-01-15 11:22:13,651 WARN [main]: client.SparkClientImpl
(SparkClientImpl.java:stop(125)) - Timed out waiting for driver to respond to
stop request.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9388) HiveServer2 fails to reconnect to MetaStore after MetaStore restart
Piotr Ackermann created HIVE-9388:
-
Summary: HiveServer2 fails to reconnect to MetaStore after
MetaStore restart
Key: HIVE-9388
URL: https://issues.apache.org/jira/browse/HIVE-9388
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.1, 0.14.0
Reporter: Piotr Ackermann
How to reproduce:
# Use Hue to connect to HiveServer2
# Restart Metastore
# Try to execute any query in que
HiveServer2 report error:
{{
ERROR hive.log: Got exception: org.apache.thrift.transport.TTransportException
null
org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:355)
at org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:432)
at org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:414)
at
org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.hadoop.hive.thrift.TFilterTransport.readAll(TFilterTransport.java:62)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:600)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:587)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:837)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy10.getDatabases(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:1681)
at com.sun.proxy.$Proxy10.getDatabases(Unknown Source)
at
org.apache.hive.service.cli.operation.GetSchemasOperation.run(GetSchemasOperation.java:62)
at
org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:715)
at
org.apache.hive.service.cli.session.HiveSessionImpl.getSchemas(HiveSessionImpl.java:438)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:502)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy19.getSchemas(Unknown Source)
at org.apache.hive.service.cli.CLIService.getSchemas(CLIService.java:277)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.GetSchemas(ThriftCLIService.java:436)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1433)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1418)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:677)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at
[jira] [Updated] (HIVE-9388) HiveServer2 fails to reconnect to MetaStore after MetaStore restart
[
https://issues.apache.org/jira/browse/HIVE-9388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Ackermann updated HIVE-9388:
--
Description:
How to reproduce:
# Use Hue to connect to HiveServer2
# Restart Metastore
# Try to execute any query in que
HiveServer2 report error:
{quote}
ERROR hive.log: Got exception: org.apache.thrift.transport.TTransportException
null
org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:355)
at org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:432)
at org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:414)
at
org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.hadoop.hive.thrift.TFilterTransport.readAll(TFilterTransport.java:62)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:600)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:587)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:837)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy10.getDatabases(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:1681)
at com.sun.proxy.$Proxy10.getDatabases(Unknown Source)
at
org.apache.hive.service.cli.operation.GetSchemasOperation.run(GetSchemasOperation.java:62)
at
org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:715)
at
org.apache.hive.service.cli.session.HiveSessionImpl.getSchemas(HiveSessionImpl.java:438)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:502)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy19.getSchemas(Unknown Source)
at org.apache.hive.service.cli.CLIService.getSchemas(CLIService.java:277)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.GetSchemas(ThriftCLIService.java:436)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1433)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1418)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:677)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
15/01/12 14:53:08 [HiveServer2-Handler-Pool: Thread-20]: ERROR hive.log:
