[jira] [Resolved] (HIVE-10357) LLAP: Name IO elevator threads
[ https://issues.apache.org/jira/browse/HIVE-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-10357. -- Resolution: Fixed Committed to llap branch. LLAP: Name IO elevator threads -- Key: HIVE-10357 URL: https://issues.apache.org/jira/browse/HIVE-10357 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10357.patch LLAP IO elevator thread pool thread names should be given a non-default name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10357) LLAP: Name IO elevator threads
[ https://issues.apache.org/jira/browse/HIVE-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-10357: - Attachment: HIVE-10357.patch LLAP: Name IO elevator threads -- Key: HIVE-10357 URL: https://issues.apache.org/jira/browse/HIVE-10357 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10357.patch LLAP IO elevator thread pool thread names should be given a non-default name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5672) Insert with custom separator not supported for non-local directory
[
https://issues.apache.org/jira/browse/HIVE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nemon Lou updated HIVE-5672:
Attachment: HIVE-5672.3.patch
Insert with custom separator not supported for non-local directory
--
Key: HIVE-5672
URL: https://issues.apache.org/jira/browse/HIVE-5672
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0, 1.0.0
Reporter: Romain Rigaux
Assignee: Nemon Lou
Attachments: HIVE-5672.1.patch, HIVE-5672.2.patch, HIVE-5672.3.patch
https://issues.apache.org/jira/browse/HIVE-3682 is great but non local
directory don't seem to be supported:
{code}
insert overwrite directory '/tmp/test-02'
row format delimited
FIELDS TERMINATED BY ':'
select description FROM sample_07
{code}
{code}
Error while compiling statement: FAILED: ParseException line 2:0 cannot
recognize input near 'row' 'format' 'delimited' in select clause
{code}
This works (with 'local'):
{code}
insert overwrite local directory '/tmp/test-02'
row format delimited
FIELDS TERMINATED BY ':'
select code, description FROM sample_07
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10358) LLAP: Tez in-place UI gets misaligned when state is TERMINATING
[ https://issues.apache.org/jira/browse/HIVE-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-10358: - Attachment: HIVE-10358.patch LLAP: Tez in-place UI gets misaligned when state is TERMINATING --- Key: HIVE-10358 URL: https://issues.apache.org/jira/browse/HIVE-10358 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10358.patch In-place UI goes misaligned for TERMINATING state as length of TERMINATING string is greater than header. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7797) upgrade hive schema from 0.9.0 to 0.13.1 failed when back end db is postgres
[ https://issues.apache.org/jira/browse/HIVE-7797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated HIVE-7797: Description: Using the following command to upgrade hive schema : schematool -dbType postgres -upgradeSchemaFrom 0.9.0 then got this: ERROR: null value in column SCHEMA_VERSION violates not-null constraint Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed. The sql in it is : INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '', 'Initial value'); And the result is: ERROR: null value in column SCHEMA_VERSION violates not-null constraint DETAIL: Failing row contains (1, null, Initial value). was: Using the following command to upgrade hive schema : schematool -dbType postgres -upgradeSchemaFrom 0.9.0 ERROR: null value in column SCHEMA_VERSION violates not-null constraint Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed. The sql in it is : INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '', 'Initial value'); And the result is: ERROR: null value in column SCHEMA_VERSION violates not-null constraint DETAIL: Failing row contains (1, null, Initial value). upgrade hive schema from 0.9.0 to 0.13.1 failed when back end db is postgres -- Key: HIVE-7797 URL: https://issues.apache.org/jira/browse/HIVE-7797 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0, 0.13.1 Reporter: Nemon Lou Assignee: Nemon Lou Attachments: HIVE-7797.1.patch Using the following command to upgrade hive schema : schematool -dbType postgres -upgradeSchemaFrom 0.9.0 then got this: ERROR: null value in column SCHEMA_VERSION violates not-null constraint Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed. The sql in it is : INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '', 'Initial value'); And the result is: ERROR: null value in column SCHEMA_VERSION violates not-null constraint DETAIL: Failing row contains (1, null, Initial value). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7797) upgrade hive schema from 0.9.0 to 0.13.1 failed when back end db is postgres
[ https://issues.apache.org/jira/browse/HIVE-7797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated HIVE-7797: Description: Using the following command to upgrade hive schema : schematool -dbType postgres -upgradeSchemaFrom 0.9.0 ERROR: null value in column SCHEMA_VERSION violates not-null constraint Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed. The sql in it is : INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '', 'Initial value'); And the result is: ERROR: null value in column SCHEMA_VERSION violates not-null constraint DETAIL: Failing row contains (1, null, Initial value). was: Using hive schema tool with the following command to upgrade hive schema failed: schematool -dbType postgres -upgradeSchemaFrom 0.9.0 ERROR: null value in column SCHEMA_VERSION violates not-null constraint Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed. The sql in it is : INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '', 'Initial value'); And the result is: ERROR: null value in column SCHEMA_VERSION violates not-null constraint DETAIL: Failing row contains (1, null, Initial value). upgrade hive schema from 0.9.0 to 0.13.1 failed when back end db is postgres -- Key: HIVE-7797 URL: https://issues.apache.org/jira/browse/HIVE-7797 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0, 0.13.1 Reporter: Nemon Lou Assignee: Nemon Lou Attachments: HIVE-7797.1.patch Using the following command to upgrade hive schema : schematool -dbType postgres -upgradeSchemaFrom 0.9.0 ERROR: null value in column SCHEMA_VERSION violates not-null constraint Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed. The sql in it is : INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '', 'Initial value'); And the result is: ERROR: null value in column SCHEMA_VERSION violates not-null constraint DETAIL: Failing row contains (1, null, Initial value). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-10358) LLAP: Tez in-place UI gets misaligned when state is TERMINATING
[ https://issues.apache.org/jira/browse/HIVE-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-10358. -- Resolution: Fixed Committed to llap branch. LLAP: Tez in-place UI gets misaligned when state is TERMINATING --- Key: HIVE-10358 URL: https://issues.apache.org/jira/browse/HIVE-10358 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10358.patch In-place UI goes misaligned for TERMINATING state as length of TERMINATING string is greater than header. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10361) CBO (Calcite Return Path): Overriding toString method in join algorithms
[ https://issues.apache.org/jira/browse/HIVE-10361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10361: --- Attachment: HIVE-10361.cbo.patch CBO (Calcite Return Path): Overriding toString method in join algorithms Key: HIVE-10361 URL: https://issues.apache.org/jira/browse/HIVE-10361 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-10361.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-7797) using hive schema tool to upgrade hive schema in postgres from 0.9.0 to 0.13.1 failed
[ https://issues.apache.org/jira/browse/HIVE-7797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou reassigned HIVE-7797: --- Assignee: Nemon Lou using hive schema tool to upgrade hive schema in postgres from 0.9.0 to 0.13.1 failed --- Key: HIVE-7797 URL: https://issues.apache.org/jira/browse/HIVE-7797 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0, 0.13.1 Reporter: Nemon Lou Assignee: Nemon Lou Attachments: HIVE-7797.1.patch Using hive schema tool with the following command to upgrade hive schema failed: schematool -dbType postgres -upgradeSchemaFrom 0.9.0 ERROR: null value in column SCHEMA_VERSION violates not-null constraint Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed. The sql in it is : INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '', 'Initial value'); And the result is: ERROR: null value in column SCHEMA_VERSION violates not-null constraint DETAIL: Failing row contains (1, null, Initial value). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10331) ORC : Is null SARG filters out all row groups written in old ORC format
[
https://issues.apache.org/jira/browse/HIVE-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mostafa Mokhtar updated HIVE-10331:
---
Attachment: HIVE-10331.03.patch
HIVE-10331.03.patch
ORC : Is null SARG filters out all row groups written in old ORC format
---
Key: HIVE-10331
URL: https://issues.apache.org/jira/browse/HIVE-10331
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 1.1.0
Reporter: Mostafa Mokhtar
Assignee: Mostafa Mokhtar
Fix For: 1.2.0
Attachments: HIVE-10331.01.patch, HIVE-10331.02.patch,
HIVE-10331.03.patch, HIVE-10331.03.patch
Queries are returning wrong results as all row groups gets filtered out and
no rows get scanned.
{code}
SELECT
count(*)
FROM
store_sales
WHERE
ss_addr_sk IS NULL
{code}
With hive.optimize.index.filter disabled we get the correct results
In pickRowGroups stats show that hasNull_ is fales, while the rowgroup
actually has null.
Same query runs fine for newly loaded ORC tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10331) ORC : Is null SARG filters out all row groups written in old ORC format
[
https://issues.apache.org/jira/browse/HIVE-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mostafa Mokhtar updated HIVE-10331:
---
Attachment: HIVE-10331.04.patch
[~prasanth_j]
Makes sense.
ORC : Is null SARG filters out all row groups written in old ORC format
---
Key: HIVE-10331
URL: https://issues.apache.org/jira/browse/HIVE-10331
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 1.1.0
Reporter: Mostafa Mokhtar
Assignee: Mostafa Mokhtar
Fix For: 1.2.0
Attachments: HIVE-10331.01.patch, HIVE-10331.02.patch,
HIVE-10331.03.patch, HIVE-10331.03.patch, HIVE-10331.04.patch
Queries are returning wrong results as all row groups gets filtered out and
no rows get scanned.
{code}
SELECT
count(*)
FROM
store_sales
WHERE
ss_addr_sk IS NULL
{code}
With hive.optimize.index.filter disabled we get the correct results
In pickRowGroups stats show that hasNull_ is fales, while the rowgroup
actually has null.
Same query runs fine for newly loaded ORC tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10361) CBO (Calcite Return Path): Overriding toString method in join algorithms
[ https://issues.apache.org/jira/browse/HIVE-10361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10361: --- Attachment: HIVE-10361.cbo.patch CBO (Calcite Return Path): Overriding toString method in join algorithms Key: HIVE-10361 URL: https://issues.apache.org/jira/browse/HIVE-10361 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-10361.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10222) Upgrade Calcite dependency to newest version
[ https://issues.apache.org/jira/browse/HIVE-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10222: --- Attachment: HIVE-10222.03.patch Running QA on new release. Upgrade Calcite dependency to newest version Key: HIVE-10222 URL: https://issues.apache.org/jira/browse/HIVE-10222 Project: Hive Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Attachments: HIVE-10222.01.patch, HIVE-10222.02.patch, HIVE-10222.03.patch, HIVE-10222.patch Upgrade Calcite version to 1.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10288) Cannot call permanent UDFs
[
https://issues.apache.org/jira/browse/HIVE-10288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497858#comment-14497858
]
Chinna Rao Lalam commented on HIVE-10288:
-
Failed tests seems not related to this patch.
Cannot call permanent UDFs
--
Key: HIVE-10288
URL: https://issues.apache.org/jira/browse/HIVE-10288
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Nezih Yigitbasi
Assignee: Chinna Rao Lalam
Attachments: HIVE-10288.1.patch, HIVE-10288.patch
Just pulled the trunk and built the hive binary. If I create a permanent udf
and exit the cli, and then open the cli and try calling the udf it fails with
the exception below. However, the call succeeds if I call the udf right after
registering the permanent udf (without exiting the cli). The call also
succeeds with the apache-hive-1.0.0 release.
{code}
15-04-13 17:04:54,004 INFO org.apache.hadoop.hive.ql.log.PerfLogger
(PerfLogger.java:PerfLogEnd(148)) - /PERFLOG method=parse
start=1428969893115 end=1428969894004 duration=889
from=org.apache.hadoop.hive.ql.Driver
2015-04-13 17:04:54,007 DEBUG org.apache.hadoop.hive.ql.Driver
(Driver.java:recordValidTxns(939)) - Encoding valid txns info
9223372036854775807:
2015-04-13 17:04:54,007 INFO org.apache.hadoop.hive.ql.log.PerfLogger
(PerfLogger.java:PerfLogBegin(121)) - PERFLOG method=semanticAnalyze
from=org.apache.hadoop.hive.ql.Driver
2015-04-13 17:04:54,052 INFO org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:analyzeInternal(9997)) - Starting Semantic Analysis
2015-04-13 17:04:54,053 DEBUG org.apache.hadoop.hive.ql.exec.FunctionRegistry
(FunctionRegistry.java:getGenericUDAFResolver(942)) - Looking up GenericUDAF:
hour_now
2015-04-13 17:04:54,053 INFO org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:genResolvedParseTree(9980)) - Completed phase 1 of
Semantic Analysis
2015-04-13 17:04:54,053 INFO org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:getMetaData(1530)) - Get metadata for source tables
2015-04-13 17:04:54,054 INFO org.apache.hadoop.hive.metastore.HiveMetaStore
(HiveMetaStore.java:logInfo(744)) - 0: get_table : db=default tbl=test_table
2015-04-13 17:04:54,054 INFO
org.apache.hadoop.hive.metastore.HiveMetaStore.audit
(HiveMetaStore.java:logAuditEvent(369)) - ugi=nyigitbasi ip=unknown-ip-addr
cmd=get_table : db=default tbl=test_table
2015-04-13 17:04:54,054 DEBUG org.apache.hadoop.hive.metastore.ObjectStore
(ObjectStore.java:debugLog(6776)) - Open transaction: count = 1, isActive =
true at:
org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:927)
2015-04-13 17:04:54,054 DEBUG org.apache.hadoop.hive.metastore.ObjectStore
(ObjectStore.java:debugLog(6776)) - Open transaction: count = 2, isActive =
true at:
org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:990)
2015-04-13 17:04:54,104 DEBUG org.apache.hadoop.hive.metastore.ObjectStore
(ObjectStore.java:debugLog(6776)) - Commit transaction: count = 1, isactive
true at:
org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:998)
2015-04-13 17:04:54,232 DEBUG org.apache.hadoop.hive.metastore.ObjectStore
(ObjectStore.java:debugLog(6776)) - Commit transaction: count = 0, isactive
true at:
org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:929)
2015-04-13 17:04:54,242 INFO org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:getMetaData(1682)) - Get metadata for subqueries
2015-04-13 17:04:54,247 INFO org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:getMetaData(1706)) - Get metadata for destination
tables
2015-04-13 17:04:54,256 INFO org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:genResolvedParseTree(9984)) - Completed getting
MetaData in Semantic Analysis
2015-04-13 17:04:54,259 INFO
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer
(CalcitePlanner.java:canHandleAstForCbo(369)) - Not invoking CBO because the
statement has too few joins
2015-04-13 17:04:54,344 DEBUG
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
(LazySimpleSerDe.java:initialize(135)) -
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with:
columnNames=[_c0, _c1] columnTypes=[int, int] separator=[[B@6e6d4780]
nullstring=\N lastColumnTakesRest=false timestampFormats=null
2015-04-13 17:04:54,406 DEBUG org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:genTablePlan(9458)) - Created Table Plan for
test_table TS[0]
2015-04-13 17:04:54,410 DEBUG org.apache.hadoop.hive.ql.parse.CalcitePlanner
(SemanticAnalyzer.java:genBodyPlan(8815)) - RR before GB test_table{(_c0,_c0:
[jira] [Commented] (HIVE-7594) Hive JDBC client: out of sequence response on large long running query
[
https://issues.apache.org/jira/browse/HIVE-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497954#comment-14497954
]
Shrey Mehrotra commented on HIVE-7594:
--
I am also facing same issue with Hive 1.0.0. Any patch or workaround available?
Hive JDBC client: out of sequence response on large long running query
Key: HIVE-7594
URL: https://issues.apache.org/jira/browse/HIVE-7594
Project: Hive
Issue Type: Bug
Components: Clients, HiveServer2
Affects Versions: 0.13.0
Environment: HDP2.1
Reporter: Hari Sekhon
When executing a long running query in a JDBC client (Squirrel) to
HiveServer2 after several minutes I get this error in the client:
{code}
Error: org.apache.thrift.TApplicationException: ExecuteStatement failed: out
of sequence response
SQLState: 08S01
ErrorCode: 0
{code}
I've seen this before in, iirc when running 2 queries in 1 session but I've
closed the client and run only this single query in a new session each time.
I did a search and saw HIVE-6893 referring to a Metastore exception which I
have in some older logs but not corresponding / recent in these recent
instances, the error seems different in this case but may be related.
The query to reproduce is select count(*) from myTable where myTable is a
1TB table of 620 million rows. This happens in both MR and Tez execution
engines running on Yarn.
Here are all the jars I've added to the classpath (taken from Hortonworks doc
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1-latest/bk_dataintegration/content/ch_using-hive-2.html,
plus added hadoop-common, hive-exec and slf4j-api to solve class not found
issues on top of that):
commons-codec-1.4.jar
commons-logging-1.1.3.jar
hadoop-common-2.4.0.2.1.3.0-563.jar
hive-exec-0.13.0.2.1.3.0-563.jar
hive-jdbc-0.13.0.2.1.3.0-563.jar
hive-service-0.13.0.2.1.3.0-563.jar
httpclient-4.2.5.jar
httpcore-4.2.5.jar
libthrift-0.9.0.jar
slf4j-api-1.7.5.jar
I am seeing errors like this in the hiveserver2.log:
{code}
2014-08-01 15:04:31,358 ERROR [pool-5-thread-3]: server.TThreadPoolServer
(TThreadPoolServer.java:run(215)) - Error occurred during processing of
message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
at
org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 4 more
...
2014-08-01 15:06:31,520 ERROR [pool-5-thread-3]: server.TThreadPoolServer
(TThreadPoolServer.java:run(215)) - Error occurred during processing of
message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
at
org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
at
[jira] [Commented] (HIVE-10228) Changes to Hive Export/Import/DropTable/DropPartition to support replication semantics
[
https://issues.apache.org/jira/browse/HIVE-10228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497895#comment-14497895
]
Alan Gates commented on HIVE-10228:
---
Wow, when I saw it was a 150K patch I was hoping it was mostly generated code.
No such luck.
Code level comments on review board, higher level below:
This stuff needs some major doc work as you're introducing a new concept of a
table being replicated or generated from replication. Is there a doc JIRA for
the replication work yet? If so we should link it to this JIRA.
Parser changes:
I don't understand why DROP TABLE needs the replication clause. As far as I
can tell from the changes in DDLSemanticAnalyzer this is semantically
equivalent to IF EXISTS. Why not use that?
Adding METADATA and REPLICATION as keywords is not backwards compatible. We
either need to explicitly note that in this JIRA or add them to the list of
reserved keywords allowed as identifiers in IdentifiersParser.g. I suspect the
latter is a better choice.
Changes to Hive Export/Import/DropTable/DropPartition to support replication
semantics
--
Key: HIVE-10228
URL: https://issues.apache.org/jira/browse/HIVE-10228
Project: Hive
Issue Type: Sub-task
Components: Import/Export
Affects Versions: 1.2.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-10228.2.patch, HIVE-10228.3.patch, HIVE-10228.patch
We need to update a couple of hive commands to support replication semantics.
To wit, we need the following:
EXPORT ... [FOR [METADATA] REPLICATION(“comment”)]
Export will now support an extra optional clause to tell it that this export
is being prepared for the purpose of replication. There is also an additional
optional clause here, that allows for the export to be a metadata-only
export, to handle cases of capturing the diff for alter statements, for
example.
Also, if done for replication, the non-presence of a table, or a table being
a view/offline table/non-native table is not considered an error, and
instead, will result in a successful no-op.
IMPORT ... (as normal) – but handles new semantics
No syntax changes for import, but import will have to change to be able to
handle all the permutations of export dumps possible. Also, import will have
to ensure that it should update the object only if the update being imported
is not older than the state of the object. Also, import currently does not
work with dbname.tablename kind of specification, this should be fixed to
work.
DROP TABLE ... FOR REPLICATION('eventid')
Drop Table now has an additional clause, to specify that this drop table is
being done for replication purposes, and that the dop should not actually
drop the table if the table is newer than that event id specified.
ALTER TABLE ... DROP PARTITION (...) FOR REPLICATION('eventid')
Similarly, Drop Partition also has an equivalent change to Drop Table.
=
In addition, we introduce a new property repl.last.id, which when tagged on
to table properties or partition properties on a replication-destination,
holds the effective state identifier of the object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10360) CBO (Calcite Return Path): Remove HiveCostUtil
[ https://issues.apache.org/jira/browse/HIVE-10360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10360: --- Summary: CBO (Calcite Return Path): Remove HiveCostUtil (was: CBO (Calcite Return Path): Prevent cost overflow) CBO (Calcite Return Path): Remove HiveCostUtil -- Key: HIVE-10360 URL: https://issues.apache.org/jira/browse/HIVE-10360 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10360) CBO (Calcite Return Path): Remove HiveCostUtil
[
https://issues.apache.org/jira/browse/HIVE-10360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-10360:
---
Description: The methods were already moved to {{HiveAlgorithmsUtil}}.
CBO (Calcite Return Path): Remove HiveCostUtil
--
Key: HIVE-10360
URL: https://issues.apache.org/jira/browse/HIVE-10360
Project: Hive
Issue Type: Sub-task
Components: CBO
Affects Versions: cbo-branch
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: cbo-branch
The methods were already moved to {{HiveAlgorithmsUtil}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10121) Implement a hive --service udflint command to check UDF jars for common shading mistakes
[ https://issues.apache.org/jira/browse/HIVE-10121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdelrahman Shettia updated HIVE-10121: --- Attachment: HIVE-10121.1.patch Implement a hive --service udflint command to check UDF jars for common shading mistakes Key: HIVE-10121 URL: https://issues.apache.org/jira/browse/HIVE-10121 Project: Hive Issue Type: New Feature Components: UDF Reporter: Gopal V Assignee: Abdelrahman Shettia Fix For: 1.2.0 Attachments: HIVE-10121.1.patch Several SerDe and UDF jars tend to shade in various parts of the dependencies including hadoop-common or guava without relocation. Implement a simple udflint tool which automates some part of the class path and shaded resources audit process required when upgrading a hive install from an old version to a new one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10121) Implement a hive --service udflint command to check UDF jars for common shading mistakes
[ https://issues.apache.org/jira/browse/HIVE-10121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498076#comment-14498076 ] Abdelrahman Shettia commented on HIVE-10121: Hi Gopal, I am attaching the following use cases output files: bad_udfs.out bad_udfs_verbose.out good_udfs.out good_udfs_verbose.out Usage: Normal mode: $ hive --service UDFLint -file /tmp/hive_udf-1.0.0.jar Verbose mode: hive --service UDFLint -jar yi/hive-json-serde-0.3.jar -v Without any options: [root@sandbox test]# hive --service UDFLint usage: udflint -h,--helpprint help message --hiveconf property=value Use value for given property --jar arg Comma separated list of jars to validate -v,--verbose Verbose mode (Run the tool in debug mode) Please let me know if you have questions. Thanks -Rahman Implement a hive --service udflint command to check UDF jars for common shading mistakes Key: HIVE-10121 URL: https://issues.apache.org/jira/browse/HIVE-10121 Project: Hive Issue Type: New Feature Components: UDF Reporter: Gopal V Assignee: Abdelrahman Shettia Fix For: 1.2.0 Attachments: HIVE-10121.1.patch, bad_udfs.out, bad_udfs_verbose.out, good_udfs.out, good_udfs_verbose.out Several SerDe and UDF jars tend to shade in various parts of the dependencies including hadoop-common or guava without relocation. Implement a simple udflint tool which automates some part of the class path and shaded resources audit process required when upgrading a hive install from an old version to a new one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10306) We need to print tez summary when hive.server2.logging.level = PERFORMANCE.
[
https://issues.apache.org/jira/browse/HIVE-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498118#comment-14498118
]
Hive QA commented on HIVE-10306:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725753/HIVE-10306.4.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8707 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3457/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3457/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3457/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725753 - PreCommit-HIVE-TRUNK-Build
We need to print tez summary when hive.server2.logging.level = PERFORMANCE.
-
Key: HIVE-10306
URL: https://issues.apache.org/jira/browse/HIVE-10306
Project: Hive
Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-10306.1.patch, HIVE-10306.2.patch,
HIVE-10306.3.patch, HIVE-10306.4.patch
We need to print tez summary when hive.server2.logging.level = PERFORMANCE.
We introduced this parameter via HIVE-10119.
The logging param for levels is only relevant to HS2, so for hive-cli users
the hive.tez.exec.print.summary still makes sense. We can check for log-level
param as well, in places we are checking value of
hive.tez.exec.print.summary. Ie, consider hive.tez.exec.print.summary=true if
log.level = PERFORMANCE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10364) The HMS upgrade script test does not publish results when prepare.sh fails.
[ https://issues.apache.org/jira/browse/HIVE-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-10364: --- Attachment: HIVE-10364.1.patch The HMS upgrade script test does not publish results when prepare.sh fails. --- Key: HIVE-10364 URL: https://issues.apache.org/jira/browse/HIVE-10364 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Sergio Peña Assignee: Sergio Peña Attachments: HIVE-10364.1.patch The HMS upgrade script must publish succeed or failure results to JIRA. This bug is not publishing any results on JIRA is the prepare.sh script fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10190) CBO: AST mode checks for TABLESAMPLE with AST.toString().contains(TOK_TABLESPLITSAMPLE)
[
https://issues.apache.org/jira/browse/HIVE-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-10190:
---
Attachment: HIVE-10190.05.patch
Re-upload patch to allow jenkins run the tests again.
CBO: AST mode checks for TABLESAMPLE with
AST.toString().contains(TOK_TABLESPLITSAMPLE)
-
Key: HIVE-10190
URL: https://issues.apache.org/jira/browse/HIVE-10190
Project: Hive
Issue Type: Bug
Components: CBO
Affects Versions: 1.2.0
Reporter: Gopal V
Assignee: Reuben Kuhnert
Priority: Trivial
Labels: perfomance
Attachments: HIVE-10190-querygen.py, HIVE-10190.01.patch,
HIVE-10190.02.patch, HIVE-10190.03.patch, HIVE-10190.04.patch,
HIVE-10190.05.patch, HIVE-10190.05.patch
{code}
public static boolean validateASTForUnsupportedTokens(ASTNode ast) {
String astTree = ast.toStringTree();
// if any of following tokens are present in AST, bail out
String[] tokens = { TOK_CHARSETLITERAL, TOK_TABLESPLITSAMPLE };
for (String token : tokens) {
if (astTree.contains(token)) {
return false;
}
}
return true;
}
{code}
This is an issue for a SQL query which is bigger in AST form than in text
(~700kb).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9917) After HIVE-3454 is done, make int to timestamp conversion configurable
[
https://issues.apache.org/jira/browse/HIVE-9917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498235#comment-14498235
]
Hive QA commented on HIVE-9917:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725748/HIVE-9917.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 8704 tests
executed
*Failed tests:*
{noformat}
TestCustomAuthentication - did not produce a TEST-*.xml file
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3458/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3458/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3458/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725748 - PreCommit-HIVE-TRUNK-Build
After HIVE-3454 is done, make int to timestamp conversion configurable
--
Key: HIVE-9917
URL: https://issues.apache.org/jira/browse/HIVE-9917
Project: Hive
Issue Type: Improvement
Reporter: Aihua Xu
Assignee: Aihua Xu
Attachments: HIVE-9917.patch
After HIVE-3454 is fixed, we will have correct behavior of converting int to
timestamp. While the customers are using such incorrect behavior for so long,
better to make it configurable so that in one release, it will default to
old/inconsistent way and the next release will default to new/consistent way.
And then we will deprecate it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10365) First job fails with StackOverflowError [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HIVE-10365:
---
Summary: First job fails with StackOverflowError [Spark Branch] (was:
First job fails with StackOverflowError)
First job fails with StackOverflowError [Spark Branch]
--
Key: HIVE-10365
URL: https://issues.apache.org/jira/browse/HIVE-10365
Project: Hive
Issue Type: Bug
Affects Versions: spark-branch
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
When running some queries on Yarn with standalone Hadoop, the first query
fails with StackOverflowError:
{noformat}
java.lang.StackOverflowError
at
java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
at
java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1145)
at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:464)
at java.lang.ClassLoader.loadClass(ClassLoader.java:405)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10313) Literal Decimal ExprNodeConstantDesc should contain value of HiveDecimal instead of String
[
https://issues.apache.org/jira/browse/HIVE-10313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498280#comment-14498280
]
Jimmy Xiang commented on HIVE-10313:
+1
Literal Decimal ExprNodeConstantDesc should contain value of HiveDecimal
instead of String
--
Key: HIVE-10313
URL: https://issues.apache.org/jira/browse/HIVE-10313
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 1.0.0
Reporter: Chaoyu Tang
Assignee: Chaoyu Tang
Attachments: HIVE-10313.1.patch, HIVE-10313.patch
In TyepCheckProcFactory.NumExprProcessor, the ExprNodeConstantDesc is created
from strVal:
{code}
else if (expr.getText().endsWith(BD)) {
// Literal decimal
String strVal = expr.getText().substring(0, expr.getText().length()
- 2);
HiveDecimal hd = HiveDecimal.create(strVal);
int prec = 1;
int scale = 0;
if (hd != null) {
prec = hd.precision();
scale = hd.scale();
}
DecimalTypeInfo typeInfo = TypeInfoFactory.getDecimalTypeInfo(prec,
scale);
return new ExprNodeConstantDesc(typeInfo, strVal);
}
{code}
It should use HiveDecmal:
return new ExprNodeConstantDesc(typeInfo, hd);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8950) Add support in ParquetHiveSerde to create table schema from a parquet file
[ https://issues.apache.org/jira/browse/HIVE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498249#comment-14498249 ] Ashish K Singh commented on HIVE-8950: -- [~kamrul] for now you will have to create hive table with proper schema yourself. The patch is almost ready, it got side tracked for a bit. There are a few things I have to address. Will soon submit an updated patch. Add support in ParquetHiveSerde to create table schema from a parquet file -- Key: HIVE-8950 URL: https://issues.apache.org/jira/browse/HIVE-8950 Project: Hive Issue Type: Improvement Reporter: Ashish K Singh Assignee: Ashish K Singh Attachments: HIVE-8950.1.patch, HIVE-8950.2.patch, HIVE-8950.3.patch, HIVE-8950.4.patch, HIVE-8950.5.patch, HIVE-8950.6.patch, HIVE-8950.patch PARQUET-76 and PARQUET-47 ask for creating parquet backed tables without having to specify the column names and types. As, parquet files store schema in their footer, it is possible to generate hive schema from parquet file's metadata. This will improve usability of parquet backed tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9923) No clear message when from is missing
[
https://issues.apache.org/jira/browse/HIVE-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498288#comment-14498288
]
Yongzhi Chen commented on HIVE-9923:
Hive support simple select without from statement (when there is a UDF). When
select in an insert statement, hive need the from clause. The new patch only
check from clause null or not in the insert statement.
No clear message when from is missing
---
Key: HIVE-9923
URL: https://issues.apache.org/jira/browse/HIVE-9923
Project: Hive
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Jeff Zhang
Assignee: Yongzhi Chen
Attachments: HIVE-9923.1.patch
For the following sql, from is missing but it throw NPE which is not clear
for user.
{code}
hive insert overwrite directory '/tmp/hive-3' select sb1.name, sb2.age
student_bucketed sb1 join student_bucketed sb2 on sb1.name=sb2.name;
FAILED: NullPointerException null
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9917) After HIVE-3454 is done, make int to timestamp conversion configurable
[ https://issues.apache.org/jira/browse/HIVE-9917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498355#comment-14498355 ] Aihua Xu commented on HIVE-9917: The test failures are unrelated to the patch. After HIVE-3454 is done, make int to timestamp conversion configurable -- Key: HIVE-9917 URL: https://issues.apache.org/jira/browse/HIVE-9917 Project: Hive Issue Type: Improvement Reporter: Aihua Xu Assignee: Aihua Xu Attachments: HIVE-9917.patch After HIVE-3454 is fixed, we will have correct behavior of converting int to timestamp. While the customers are using such incorrect behavior for so long, better to make it configurable so that in one release, it will default to old/inconsistent way and the next release will default to new/consistent way. And then we will deprecate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10366) Multiple Date Functions inside Named Struct return the wrong values
[
https://issues.apache.org/jira/browse/HIVE-10366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Traynham updated HIVE-10366:
-
Description:
Using multiple date functions inside a named struct results in incorrect
values. The first date function is correct. All subsequent date functions
receive the value of the first date function, unless the function has a
different valued argument.
As shown below, the 3rd query returns an incorrect value for `month(datetime)`.
A not so pleasant workaround is to substr datetime resulting in a different
value (4th query).
DESCRIBE my_datamart;
OK
datetimestring
SELECT datetime FROM my_datamart LIMIT 1;
OK
2015-04-14T00:00:00
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(datetime)), day(datetime), month(datetime) FROM my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:14} 14 4
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(substr(datetime, 0, 10))), day(datetime), month(datetime) FROM
my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:4} 144
I havn't tried all the other date functions, but this is happens with
weekofyear as well. Looks like a caching issue...
was:
Using multiple date functions inside a named struct results in incorrect
values. The first date function is correct. All subsequent date functions
receive the value of the first date function, unless the function has a
different valued argument.
As shown below, the 3rd query returns an incorrect value for `month(datetime)`.
A not so pleasant workaround is to substr datetime resulting in a different
value (4th query).
DESCRIBE my_datamart;
OK
datetimestring
SELECT datetime FROM my_datamart LIMIT 1;
OK
2015-04-14T00:00:00
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(datetime)), day(datetime), month(datetime) FROM my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:14} 14 4
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(substr(datetime, 0, 10))), day(datetime), month(datetime) FROM
my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:4} 14 4
I havn't tried all the other date functions, but this is happens with
weekofyear as well. Looks like a caching issue...
Multiple Date Functions inside Named Struct return the wrong values
---
Key: HIVE-10366
URL: https://issues.apache.org/jira/browse/HIVE-10366
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 0.14.0
Reporter: Matt Traynham
Using multiple date functions inside a named struct results in incorrect
values. The first date function is correct. All subsequent date functions
receive the value of the first date function, unless the function has a
different valued argument.
As shown below, the 3rd query returns an incorrect value for
`month(datetime)`. A not so pleasant workaround is to substr datetime
resulting in a different value (4th query).
DESCRIBE my_datamart;
OK
datetime string
SELECT datetime FROM my_datamart LIMIT 1;
OK
2015-04-14T00:00:00
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(datetime)), day(datetime), month(datetime) FROM my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:14} 14 4
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(substr(datetime, 0, 10))), day(datetime), month(datetime) FROM
my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:4} 14 4
I havn't tried all the other date functions, but this is happens with
weekofyear as well. Looks like a caching issue...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10059) Make udaf_percentile_approx_23.q test more stable
[ https://issues.apache.org/jira/browse/HIVE-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498359#comment-14498359 ] Aihua Xu commented on HIVE-10059: - There is a failure to have a different value. 342.0 --- 341.5 Make udaf_percentile_approx_23.q test more stable - Key: HIVE-10059 URL: https://issues.apache.org/jira/browse/HIVE-10059 Project: Hive Issue Type: Bug Components: Tests Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Fix For: 1.2.0 Attachments: HIVE-10059.1.patch I think the result of the test is not deterministic Sometimes line 626 in the output shows 256.0 sometimes 255.5 Both results are correct because this UDAF returns approximate result. Probably results in the range of 255-257 should be considered as correct -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10346) Tez on HBase has problems with settings again
[ https://issues.apache.org/jira/browse/HIVE-10346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498375#comment-14498375 ] Gunther Hagleitner commented on HIVE-10346: --- LGTM +1 Things you might want to consider: Do you want to do the logging in the second code patch (checkOutputSpec) on failure too? Since you're creating the format at init time now, do we still need the checkOutputSpecs at all? Tez on HBase has problems with settings again - Key: HIVE-10346 URL: https://issues.apache.org/jira/browse/HIVE-10346 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-10346.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10367) org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
[
https://issues.apache.org/jira/browse/HIVE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sushanth Sowmyan updated HIVE-10367:
Affects Version/s: 1.2.0
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
Key: HIVE-10367
URL: https://issues.apache.org/jira/browse/HIVE-10367
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-10367.1.patch
{code}
java.io.IOException: Failed to execute create table encrypted_table(a int, b
string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'.
Driver returned 1 Error: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.lang.IllegalArgumentException: Pathname
/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
from
hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
is not a valid DFS filename.)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197)
{code}
It looks like the error is happening because of the below query:
create table encrypted_table(a int, b string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable'
This gets called from createTableInSpecifiedPath() in
TestHCatLoaderEncryption. I think an invalid HDFS path is passed here when
we are running the tests on Windows OS, so in line 197, changing
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
encryptedTablePath, driver);
{code}
to the below line might resolve the issue.
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
WindowsPathUtil.getHdfsUriString(encryptedTablePath), driver);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10368) VectorExpressionWriter doesn't match vectorColumn during row spilling in HybridGraceHashJoin
[
https://issues.apache.org/jira/browse/HIVE-10368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Zheng updated HIVE-10368:
-
Description:
This problem was exposed by HIVE-10284, when testing vectorized_context.q
Below is the query and backtrace:
{code}
select store.s_city, ss_net_profit
from store_sales
JOIN store ON store_sales.ss_store_sk = store.s_store_sk
JOIN household_demographics ON store_sales.ss_hdemo_sk =
household_demographics.hd_demo_sk
limit 100
{code}
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector cannot be cast to
org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterLong.writeValue(VectorExpressionWriterFactory.java:175)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.getRowObject(VectorMapJoinOperator.java:347)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.spillBigTableRow(VectorMapJoinOperator.java:306)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:390)
... 24 more
was:
This problem was exposed by HIVE-10284, when testing vectorized_context
Below is the query and backtrace:
{code}
select store.s_city, ss_net_profit
from store_sales
JOIN store ON store_sales.ss_store_sk = store.s_store_sk
JOIN household_demographics ON store_sales.ss_hdemo_sk =
household_demographics.hd_demo_sk
limit 100
{code}
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector cannot be cast to
org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterLong.writeValue(VectorExpressionWriterFactory.java:175)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.getRowObject(VectorMapJoinOperator.java:347)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.spillBigTableRow(VectorMapJoinOperator.java:306)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:390)
... 24 more
VectorExpressionWriter doesn't match vectorColumn during row spilling in
HybridGraceHashJoin
Key: HIVE-10368
URL: https://issues.apache.org/jira/browse/HIVE-10368
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Wei Zheng
Assignee: Wei Zheng
This problem was exposed by HIVE-10284, when testing vectorized_context.q
Below is the query and backtrace:
{code}
select store.s_city, ss_net_profit
from store_sales
JOIN store ON store_sales.ss_store_sk = store.s_store_sk
JOIN household_demographics ON store_sales.ss_hdemo_sk =
household_demographics.hd_demo_sk
limit 100
{code}
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector cannot be cast to
org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterLong.writeValue(VectorExpressionWriterFactory.java:175)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.getRowObject(VectorMapJoinOperator.java:347)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.spillBigTableRow(VectorMapJoinOperator.java:306)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:390)
... 24 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10284) enable container reuse for grace hash join
[ https://issues.apache.org/jira/browse/HIVE-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498665#comment-14498665 ] Wei Zheng commented on HIVE-10284: -- HIVE-10368 has been created for this specific issue. enable container reuse for grace hash join --- Key: HIVE-10284 URL: https://issues.apache.org/jira/browse/HIVE-10284 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Wei Zheng Attachments: HIVE-10284.1.patch, HIVE-10284.2.patch, HIVE-10284.3.patch, HIVE-10284.4.patch, HIVE-10284.5.patch, HIVE-10284.6.patch, HIVE-10284.7.patch, HIVE-10284.8.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10367) org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
[ https://issues.apache.org/jira/browse/HIVE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-10367: - Description: java.io.IOException: Failed to execute create table encrypted_table(a int, b string) location 'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'. Driver returned 1 Error: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.lang.IllegalArgumentException: Pathname /D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable from hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable is not a valid DFS filename.) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197) It looks like the error is happening because of the below query: create table encrypted_table(a int, b string) location 'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable' This gets called from createTableInSpecifiedPath() in TestHCatLoaderEncryption. I think an invalid HDFS path is passed here when we are running the tests on Windows OS, so in line 197, changing createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string, encryptedTablePath, driver); to the below line might resolve the issue. createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string, WindowsPathUtil.getHdfsUriString(encryptedTablePath), driver); Thanks Hari was: java.io.IOException: Failed to execute create table encrypted_table(a int, b string) location 'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'. Driver returned 1 Error: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.lang.IllegalArgumentException: Pathname /D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable from hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable is not a valid DFS filename.) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197) org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows Key: HIVE-10367 URL: https://issues.apache.org/jira/browse/HIVE-10367 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan java.io.IOException: Failed to execute create table encrypted_table(a int, b string) location 'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'. Driver returned 1 Error: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.lang.IllegalArgumentException: Pathname /D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable from hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable is not a valid DFS filename.) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423) at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197) It looks like the error is happening because of the below query: create table encrypted_table(a int, b string) location 'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable' This gets called from
[jira] [Updated] (HIVE-10367) org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
[
https://issues.apache.org/jira/browse/HIVE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sankar Sivarama Subramaniyan updated HIVE-10367:
-
Description:
{code}
java.io.IOException: Failed to execute create table encrypted_table(a int, b
string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'.
Driver returned 1 Error: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.lang.IllegalArgumentException: Pathname
/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
from
hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
is not a valid DFS filename.)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197)
{code}
It looks like the error is happening because of the below query:
create table encrypted_table(a int, b string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable'
This gets called from createTableInSpecifiedPath() in TestHCatLoaderEncryption.
I think an invalid HDFS path is passed here when we are running the tests on
Windows OS, so in line 197, changing
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
encryptedTablePath, driver);
{code}
to the below line might resolve the issue.
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
WindowsPathUtil.getHdfsUriString(encryptedTablePath), driver);
{code}
was:
java.io.IOException: Failed to execute create table encrypted_table(a int, b
string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'.
Driver returned 1 Error: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.lang.IllegalArgumentException: Pathname
/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
from
hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
is not a valid DFS filename.)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197)
It looks like the error is happening because of the below query:
create table encrypted_table(a int, b string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable'
This gets called from createTableInSpecifiedPath() in TestHCatLoaderEncryption.
I think an invalid HDFS path is passed here when we are running the tests on
Windows OS, so in line 197, changing
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
encryptedTablePath, driver);
to the below line might resolve the issue.
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
WindowsPathUtil.getHdfsUriString(encryptedTablePath), driver);
Thanks
Hari
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
Key: HIVE-10367
URL: https://issues.apache.org/jira/browse/HIVE-10367
Project: Hive
Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
{code}
java.io.IOException: Failed to execute create table encrypted_table(a int, b
string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'.
Driver returned 1 Error: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.lang.IllegalArgumentException: Pathname
/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
from
[jira] [Commented] (HIVE-10029) LLAP: Scheduling of work from different queries within the daemon
[ https://issues.apache.org/jira/browse/HIVE-10029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498716#comment-14498716 ] Siddharth Seth commented on HIVE-10029: --- Yes, to the most part. We'll likely a need a follow up to provide data to the pre-emption queue. LLAP: Scheduling of work from different queries within the daemon - Key: HIVE-10029 URL: https://issues.apache.org/jira/browse/HIVE-10029 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Fix For: llap The current implementation is a simple queue - whichever query wins the race to submit work to a daemon will execute first. A policy around this may be useful - potentially a fair share, or a first query in gets all slots approach. Also, prioritiy associated with work within a query should be considered. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10303) HIVE-9471 broke forward compatibility of ORC files
[ https://issues.apache.org/jira/browse/HIVE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498816#comment-14498816 ] Prasanth Jayachandran commented on HIVE-10303: -- [~owen.omalley]/[~gopalv].. ping! HIVE-9471 broke forward compatibility of ORC files -- Key: HIVE-10303 URL: https://issues.apache.org/jira/browse/HIVE-10303 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 1.2.0 Reporter: Owen O'Malley Assignee: Prasanth Jayachandran Fix For: 1.2.0 Attachments: HIVE-10303.1.patch The change suppresses the streams in ORC files for ORC dictionaries with 0 entries. This causes NPE on ORC readers for all versions of Hive 0.11 to 1.1 and needs to be reverted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10222) Upgrade Calcite dependency to newest version
[ https://issues.apache.org/jira/browse/HIVE-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-10222: Attachment: HIVE-10222.04.patch Upgrade Calcite dependency to newest version Key: HIVE-10222 URL: https://issues.apache.org/jira/browse/HIVE-10222 Project: Hive Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Attachments: HIVE-10222.01.patch, HIVE-10222.02.patch, HIVE-10222.03.patch, HIVE-10222.04.patch, HIVE-10222.patch Upgrade Calcite version to 1.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10367) org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
[
https://issues.apache.org/jira/browse/HIVE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498588#comment-14498588
]
Sushanth Sowmyan commented on HIVE-10367:
-
+1
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
Key: HIVE-10367
URL: https://issues.apache.org/jira/browse/HIVE-10367
Project: Hive
Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-10367.1.patch
{code}
java.io.IOException: Failed to execute create table encrypted_table(a int, b
string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'.
Driver returned 1 Error: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.lang.IllegalArgumentException: Pathname
/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
from
hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
is not a valid DFS filename.)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197)
{code}
It looks like the error is happening because of the below query:
create table encrypted_table(a int, b string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable'
This gets called from createTableInSpecifiedPath() in
TestHCatLoaderEncryption. I think an invalid HDFS path is passed here when
we are running the tests on Windows OS, so in line 197, changing
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
encryptedTablePath, driver);
{code}
to the below line might resolve the issue.
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
WindowsPathUtil.getHdfsUriString(encryptedTablePath), driver);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10366) Multiple Date Functions inside Named Struct return the wrong values
[
https://issues.apache.org/jira/browse/HIVE-10366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498696#comment-14498696
]
Matt Traynham commented on HIVE-10366:
--
Possible dupe of https://issues.apache.org/jira/browse/HIVE-9278. Took a while
skimming to find it.
Multiple Date Functions inside Named Struct return the wrong values
---
Key: HIVE-10366
URL: https://issues.apache.org/jira/browse/HIVE-10366
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 0.14.0
Reporter: Matt Traynham
Using multiple date functions inside a named struct results in incorrect
values. The first date function is correct. All subsequent date functions
receive the value of the first date function, unless the function has a
different valued argument.
As shown below, the 3rd query returns an incorrect value for
`month(datetime)`. A not so pleasant workaround is to substr datetime
resulting in a different value (4th query).
DESCRIBE my_datamart;
OK
datetime string
SELECT datetime FROM my_datamart LIMIT 1;
OK
2015-04-14T00:00:00
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(datetime)), day(datetime), month(datetime) FROM my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:14} 14 4
SELECT named_struct('dayofmonth', day(datetime), 'monthofyear',
month(substr(datetime, 0, 10))), day(datetime), month(datetime) FROM
my_datamart LIMIT 1;
OK
{dayofmonth:14,monthofyear:4} 14 4
I havn't tried all the other date functions, but this is happens with
weekofyear as well. Looks like a caching issue...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10239) Create scripts to do metastore upgrade tests on jenkins for Derby, Oracle and PostgreSQL
[ https://issues.apache.org/jira/browse/HIVE-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-10239: - Attachment: HIVE-10239.00.patch This appears to also be an issue for postgreSQL scripts. Appears that this is an issue when wget gets redirected from http to https URL. I have added a check to ensure that the apt-transport-https package gets installed before installing the corresponding DB packages. Thanks Create scripts to do metastore upgrade tests on jenkins for Derby, Oracle and PostgreSQL Key: HIVE-10239 URL: https://issues.apache.org/jira/browse/HIVE-10239 Project: Hive Issue Type: Improvement Affects Versions: 1.1.0 Reporter: Naveen Gangam Assignee: Naveen Gangam Attachments: HIVE-10239-donotcommit.patch, HIVE-10239.0.patch, HIVE-10239.0.patch, HIVE-10239.00.patch, HIVE-10239.patch Need to create DB-implementation specific scripts to use the framework introduced in HIVE-9800 to have any metastore schema changes tested across all supported databases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10331) ORC : Is null SARG filters out all row groups written in old ORC format
[
https://issues.apache.org/jira/browse/HIVE-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498648#comment-14498648
]
Hive QA commented on HIVE-10331:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725797/HIVE-10331.04.patch
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 8711 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testReadFormat_0_11[0]
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testReadFormat_0_11[1]
org.apache.hadoop.hive.ql.io.orc.TestOrcSerDeStats.testSerdeStatsOldFormat
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testBetween
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testDateWritableEqualsBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testDateWritableInBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testDecimalEqualsBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testDecimalInBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testDoubleEqualsBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testDoubleInBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testEquals
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testIn
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testIntEqualsBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testIntInBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testIsNull
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testLessThan
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testLessThanEquals
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testNullsInBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testStringEqualsBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testStringInBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testTimestampEqualsBloomFilter
org.apache.hadoop.hive.ql.io.orc.TestRecordReaderImpl.testTimestampInBloomFilter
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3461/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3461/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3461/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 36 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725797 - PreCommit-HIVE-TRUNK-Build
ORC : Is null SARG filters out all row groups written in old ORC format
---
Key: HIVE-10331
URL: https://issues.apache.org/jira/browse/HIVE-10331
Project: Hive
Issue Type: Bug
[jira] [Commented] (HIVE-10222) Upgrade Calcite dependency to newest version
[ https://issues.apache.org/jira/browse/HIVE-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498503#comment-14498503 ] Ashutosh Chauhan commented on HIVE-10222: - +1 Upgrade Calcite dependency to newest version Key: HIVE-10222 URL: https://issues.apache.org/jira/browse/HIVE-10222 Project: Hive Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Attachments: HIVE-10222.01.patch, HIVE-10222.02.patch, HIVE-10222.03.patch, HIVE-10222.patch Upgrade Calcite version to 1.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10324) Hive metatool should take table_param_key to allow for changes to avro serde's schema url key
[
https://issues.apache.org/jira/browse/HIVE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498516#comment-14498516
]
Hive QA commented on HIVE-10324:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725773/HIVE-10324.1.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8702 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hive.jdbc.TestSSL.testSSLFetchHttp
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3460/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3460/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3460/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725773 - PreCommit-HIVE-TRUNK-Build
Hive metatool should take table_param_key to allow for changes to avro
serde's schema url key
-
Key: HIVE-10324
URL: https://issues.apache.org/jira/browse/HIVE-10324
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 1.1.0
Reporter: Szehon Ho
Assignee: Ferdinand Xu
Attachments: HIVE-10324.1.patch, HIVE-10324.patch,
HIVE-10324.patch.WIP
HIVE-3443 added support to change the serdeParams from 'metatool
updateLocation' command.
However, in avro it is possible to specify the schema via the tableParams:
{noformat}
CREATE TABLE `testavro`(
`test` string COMMENT 'from deserializer')
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES (
'avro.schema.url'='hdfs://namenode:8020/tmp/test.avsc',
'kite.compression.type'='snappy',
'transient_lastDdlTime'='1427996456')
{noformat}
Hence for those tables the 'metatool updateLocation' will not help.
This is necessary in case like upgrade the namenode to HA where the absolute
paths have changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10367) org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
[
https://issues.apache.org/jira/browse/HIVE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sankar Sivarama Subramaniyan updated HIVE-10367:
-
Attachment: HIVE-10367.1.patch
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
Key: HIVE-10367
URL: https://issues.apache.org/jira/browse/HIVE-10367
Project: Hive
Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-10367.1.patch
{code}
java.io.IOException: Failed to execute create table encrypted_table(a int, b
string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'.
Driver returned 1 Error: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.lang.IllegalArgumentException: Pathname
/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
from
hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
is not a valid DFS filename.)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197)
{code}
It looks like the error is happening because of the below query:
create table encrypted_table(a int, b string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable'
This gets called from createTableInSpecifiedPath() in
TestHCatLoaderEncryption. I think an invalid HDFS path is passed here when
we are running the tests on Windows OS, so in line 197, changing
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
encryptedTablePath, driver);
{code}
to the below line might resolve the issue.
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
WindowsPathUtil.getHdfsUriString(encryptedTablePath), driver);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7797) upgrade hive schema from 0.9.0 to 0.13.1 failed when back end db is postgres
[
https://issues.apache.org/jira/browse/HIVE-7797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498779#comment-14498779
]
Hive QA commented on HIVE-7797:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12687447/HIVE-7797.1.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 8711 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testMetastoreProxyUser
org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testSaslWithHiveMetaStore
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3462/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3462/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3462/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12687447 - PreCommit-HIVE-TRUNK-Build
upgrade hive schema from 0.9.0 to 0.13.1 failed when back end db is postgres
--
Key: HIVE-7797
URL: https://issues.apache.org/jira/browse/HIVE-7797
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0, 0.13.1
Reporter: Nemon Lou
Assignee: Nemon Lou
Attachments: HIVE-7797.1.patch
Using the following command to upgrade hive schema :
schematool -dbType postgres -upgradeSchemaFrom 0.9.0
then got this:
ERROR: null value in column SCHEMA_VERSION violates not-null constraint
Log shows that the upgrade sql file 014-HIVE-3764.postgres.sql failed.
The sql in it is :
INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES
(1, '', 'Initial value');
And the result is:
ERROR: null value in column SCHEMA_VERSION violates not-null constraint
DETAIL: Failing row contains (1, null, Initial value).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10367) org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
[
https://issues.apache.org/jira/browse/HIVE-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498545#comment-14498545
]
Hari Sankar Sivarama Subramaniyan commented on HIVE-10367:
--
Either [~sushanth] or [~thejas] can review this minor change.
Thanks
Hari
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows
Key: HIVE-10367
URL: https://issues.apache.org/jira/browse/HIVE-10367
Project: Hive
Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-10367.1.patch
{code}
java.io.IOException: Failed to execute create table encrypted_table(a int, b
string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable'.
Driver returned 1 Error: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.lang.IllegalArgumentException: Pathname
/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
from
hdfs://127.0.0.1:64735/D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429172673960/warehouse/encryptedTable
is not a valid DFS filename.)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.executeStatementOnDriver(TestHCatLoaderEncryption.java:163)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.createTableInSpecifiedPath(TestHCatLoaderEncryption.java:423)
at
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:197)
{code}
It looks like the error is happening because of the below query:
create table encrypted_table(a int, b string) location
'D:/w/hv/hcatalog/hcatalog-pig-adapter/target/tmp/org.apache.hive.hcatalog.pig.TestHCatLoader-1429056939184/warehouse/encryptedTable'
This gets called from createTableInSpecifiedPath() in
TestHCatLoaderEncryption. I think an invalid HDFS path is passed here when
we are running the tests on Windows OS, so in line 197, changing
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
encryptedTablePath, driver);
{code}
to the below line might resolve the issue.
{code}
createTableInSpecifiedPath(ENCRYPTED_TABLE, a int, b string,
WindowsPathUtil.getHdfsUriString(encryptedTablePath), driver);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10239) Create scripts to do metastore upgrade tests on jenkins for Derby, Oracle and PostgreSQL
[
https://issues.apache.org/jira/browse/HIVE-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498596#comment-14498596
]
Naveen Gangam commented on HIVE-10239:
--
I have been able to replicate the issue on a brand new VM by removing the
apt-transport-https package before I did anything else.
{code}
100%[]
1,743 --.-K/s in 0s
2015-04-16 12:56:00 (15.8 MB/s) - written to stdout [1743/1743]
OK
+ apt-get update
E: The method driver /usr/lib/apt/methods/https could not be found.
N: Is the package apt-transport-https installed?
+ /bin/true
+ apt-get install -y --force-yes oracle-xe
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package oracle-xe
{code}
Will upload a fix shortly.
Create scripts to do metastore upgrade tests on jenkins for Derby, Oracle and
PostgreSQL
Key: HIVE-10239
URL: https://issues.apache.org/jira/browse/HIVE-10239
Project: Hive
Issue Type: Improvement
Affects Versions: 1.1.0
Reporter: Naveen Gangam
Assignee: Naveen Gangam
Attachments: HIVE-10239-donotcommit.patch, HIVE-10239.0.patch,
HIVE-10239.0.patch, HIVE-10239.patch
Need to create DB-implementation specific scripts to use the framework
introduced in HIVE-9800 to have any metastore schema changes tested across
all supported databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10284) enable container reuse for grace hash join
[ https://issues.apache.org/jira/browse/HIVE-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zheng updated HIVE-10284: - Attachment: HIVE-10284.9.patch Updated test outputs for several q tests. Upload path 9 for testing. enable container reuse for grace hash join --- Key: HIVE-10284 URL: https://issues.apache.org/jira/browse/HIVE-10284 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Wei Zheng Attachments: HIVE-10284.1.patch, HIVE-10284.2.patch, HIVE-10284.3.patch, HIVE-10284.4.patch, HIVE-10284.5.patch, HIVE-10284.6.patch, HIVE-10284.7.patch, HIVE-10284.8.patch, HIVE-10284.9.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10222) Upgrade Calcite dependency to newest version
[
https://issues.apache.org/jira/browse/HIVE-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498921#comment-14498921
]
Hive QA commented on HIVE-10222:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725949/HIVE-10222.04.patch
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 8711 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3463/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3463/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3463/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725949 - PreCommit-HIVE-TRUNK-Build
Upgrade Calcite dependency to newest version
Key: HIVE-10222
URL: https://issues.apache.org/jira/browse/HIVE-10222
Project: Hive
Issue Type: Bug
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Attachments: HIVE-10222.01.patch, HIVE-10222.02.patch,
HIVE-10222.03.patch, HIVE-10222.04.patch, HIVE-10222.patch
Upgrade Calcite version to 1.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10284) enable container reuse for grace hash join
[ https://issues.apache.org/jira/browse/HIVE-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498961#comment-14498961 ] Gunther Hagleitner commented on HIVE-10284: --- +1 assuming tests will pass. enable container reuse for grace hash join --- Key: HIVE-10284 URL: https://issues.apache.org/jira/browse/HIVE-10284 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Wei Zheng Attachments: HIVE-10284.1.patch, HIVE-10284.2.patch, HIVE-10284.3.patch, HIVE-10284.4.patch, HIVE-10284.5.patch, HIVE-10284.6.patch, HIVE-10284.7.patch, HIVE-10284.8.patch, HIVE-10284.9.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10374) Revert deprecation message to HiveCLI
[ https://issues.apache.org/jira/browse/HIVE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-10374: - Attachment: HIVE-10374.patch Revert deprecation message to HiveCLI - Key: HIVE-10374 URL: https://issues.apache.org/jira/browse/HIVE-10374 Project: Hive Issue Type: Bug Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-10374.patch Reverting HIVE-10304 while there is a discussion about this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10277) Unable to process Comment line '--' in HIVE-1.1.0
[
https://issues.apache.org/jira/browse/HIVE-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499016#comment-14499016
]
Hive QA commented on HIVE-10277:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725857/HIVE-10277.2.patch
{color:red}ERROR:{color} -1 due to 632 failed/errored test(s), 8711 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_multiple
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alias_casted_column
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_char1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table2_h23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table_h23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_protect_mode
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition_authorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_serde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_serde2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_varchar1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_reordering_values
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_add_column
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_add_column2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_add_column3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_change_schema
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_comments
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_compression_enabled
[jira] [Resolved] (HIVE-10371) CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter
[ https://issues.apache.org/jira/browse/HIVE-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-10371. - Resolution: Fixed Committed to branch. Thanks, Jesus! CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter Key: HIVE-10371 URL: https://issues.apache.org/jira/browse/HIVE-10371 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-10371.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10268) Merge cbo branch into trunk
[ https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-10268: Affects Version/s: (was: cbo-branch) Merge cbo branch into trunk --- Key: HIVE-10268 URL: https://issues.apache.org/jira/browse/HIVE-10268 Project: Hive Issue Type: Task Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch, HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.patch Merge patch generated on basis of diffs of trunk with cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10222) Upgrade Calcite dependency to newest version
[ https://issues.apache.org/jira/browse/HIVE-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-10222: Issue Type: Task (was: Bug) Upgrade Calcite dependency to newest version Key: HIVE-10222 URL: https://issues.apache.org/jira/browse/HIVE-10222 Project: Hive Issue Type: Task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.2.0 Attachments: HIVE-10222.01.patch, HIVE-10222.02.patch, HIVE-10222.03.patch, HIVE-10222.04.patch, HIVE-10222.patch Upgrade Calcite version to 1.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10264) Document Replication support on wiki
[ https://issues.apache.org/jira/browse/HIVE-10264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-10264: Attachment: EXIMReplicationReplayProtocol.pdf EXIM Replay Protocol Document Replication support on wiki Key: HIVE-10264 URL: https://issues.apache.org/jira/browse/HIVE-10264 Project: Hive Issue Type: Sub-task Components: Import/Export Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: EXIMReplicationReplayProtocol.pdf, apache_hivedr.0.pdf -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10228) Changes to Hive Export/Import/DropTable/DropPartition to support replication semantics
[ https://issues.apache.org/jira/browse/HIVE-10228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498877#comment-14498877 ] Sushanth Sowmyan commented on HIVE-10228: - Sorry, yeah, this is a big patch. :) It's really a cumulative patch of a bunch of work, but a lot of that was overwriting itself so much that splitting them out into a bunch of patches would have been difficult. Forking hive to do dev of this on a separate branch and merging in one go might have been easier. I'd created https://issues.apache.org/jira/browse/HIVE-10264 as a doc jira, and I've attached a presentation-like document there outlining various points of why we're doing a bunch of what we're doing, but that still needs some wiki-fication that I am working on. I've also attached the replay-protocol document on that jira after updating it slightly with your question on DROP TABLE here. I'll reply to code-level comments on review board, and reply to your higher-level comments here. DROP TABLE : This is not quite a DROP TABLE IF EXISTS, it's a DROP TABLE IF OLDER THAN(x). There are a couple of cases this can happen in: a) To make it more resilient in cases of parallelization of events (in the cases of a worker that times out and does not respond back, for eg., but might still be running, albeit slowly in the background), one of the goals of all Commands generated by Replication is that they should be idempotent, and reprocessing of events older than the state of an object should not cause any error. So, if one drone that's processing events (41,42,43) might perform 41 and then not respond back for a significant amount of time, causing Falcon to queue another HiveDR job that starts performing (41,42,43), and 43 might return successfully before the other job performs 42, and then failing. So, one of the early design goals was that all commands should be resilient to repeats. This is a way of achieving that goal. b) In the case of a CREATE1-DROP1-CREATE2-REPL(CREATE1)-REPL(DROP1)-REPL(CREATE2), since the REPL(CREATE1) occurs after CREATE2, it picks up a newer state of the table, and the destination is at a newer state than the table which was dropped. Thus, by making the DROP ignore the destination table if it's already newer than the event that spawned the DROP, we can optimize away a bit of re-importing that REPL(CREATE2) would have needed to do. In the future, we'll add in event-nullification, and can do it at a higher level if we batch events, but this helps out even when processing at an individual level. c) In addition to a DROP-IF-OLDER, it also acts like a recursive DROP-TABLE-IF-OLDER for cases where it doesn't result in the dropping of the table, it will still result in dropping older partitions in a newer table. For eg., if a T(state=50) has partitions P1(state=45) and P2(state=53), then DROP_TABLE_IF_OLDER_THAN(47) will drop P1 but not P2. This is because a Drop-table event does not result in a series of DropPtn events that are associated with the appropriate table. So, given that our replication works on an per-object basis, if DropTable should not drop the destination table because the destination table is newer than the origin table at the time of the drop, it might still contain older partitions which should be nuked. (This mode is tested in one of the tests in TestCommands in HIVE-10227 if you want to have a look at an example of what's expected) -- Regarding the kewword addition, thanks for the feedback, it was not my intent to make them reserved keywords. I talked to [~pxiong] and [~ashutoshc] about it, and the latter is the way that makes sense. As long as I add them to the nonReserved entry in IdentifiersParser.g, it should be good. So, I'll add that in and have another update here. Changes to Hive Export/Import/DropTable/DropPartition to support replication semantics -- Key: HIVE-10228 URL: https://issues.apache.org/jira/browse/HIVE-10228 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 1.2.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-10228.2.patch, HIVE-10228.3.patch, HIVE-10228.patch We need to update a couple of hive commands to support replication semantics. To wit, we need the following: EXPORT ... [FOR [METADATA] REPLICATION(“comment”)] Export will now support an extra optional clause to tell it that this export is being prepared for the purpose of replication. There is also an additional optional clause here, that allows for the export to be a metadata-only export, to handle cases of capturing the diff for alter statements, for example. Also, if done for replication, the non-presence of a table, or a table
[jira] [Updated] (HIVE-10371) CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter
[ https://issues.apache.org/jira/browse/HIVE-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10371: --- Attachment: HIVE-10371.cbo.patch [~ashutoshc], could you take a look? Thanks CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter Key: HIVE-10371 URL: https://issues.apache.org/jira/browse/HIVE-10371 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-10371.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10373) CBO (Calcite Return Path): Complete translation for all kinds of RexNode in Windowing
[ https://issues.apache.org/jira/browse/HIVE-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10373: --- Attachment: HIVE-10373.cbo.patch [~jpullokkaran], could you review it? Thanks CBO (Calcite Return Path): Complete translation for all kinds of RexNode in Windowing - Key: HIVE-10373 URL: https://issues.apache.org/jira/browse/HIVE-10373 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-10373.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10350) CBO: Use total size instead of bucket count to determine number of splits parallelism
[
https://issues.apache.org/jira/browse/HIVE-10350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499159#comment-14499159
]
Laljo John Pullokkaran commented on HIVE-10350:
---
Fixed by HIVE-10369
CBO: Use total size instead of bucket count to determine number of splits
parallelism
Key: HIVE-10350
URL: https://issues.apache.org/jira/browse/HIVE-10350
Project: Hive
Issue Type: Sub-task
Components: CBO
Affects Versions: 1.2.0
Reporter: Mostafa Mokhtar
Assignee: Mostafa Mokhtar
Fix For: 1.2.0
Attachments: HIVE-10331.01.patch, HIVE-10350.2.patch
Not an overflow but parallelism ends up being -1 as it uses number of buckets
{code}
final int parallelism = RelMetadataQuery.splitCount(join) == null
? 1 : RelMetadataQuery.splitCount(join);
{code}
{code}
2015-04-13 18:19:09,154 DEBUG [main]: cost.HiveCostModel
(HiveCostModel.java:getJoinCost(62)) - COMMON_JOIN cost: {1600892.857142857
rows, 2.4463782008994658E7 cpu, 8.54445445875E10 io}
2015-04-13 18:19:09,155 DEBUG [main]: cost.HiveCostModel
(HiveCostModel.java:getJoinCost(62)) - MAP_JOIN cost: {1600892.857142857
rows, 1601785.714285714 cpu, -1698787.48 io}
2015-04-13 18:19:09,155 DEBUG [main]: cost.HiveCostModel
(HiveCostModel.java:getJoinCost(72)) - MAP_JOIN selected
2015-04-13 18:19:09,157 DEBUG [main]: parse.CalcitePlanner
(CalcitePlanner.java:apply(862)) - Plan After Join Reordering:
HiveSort(fetch=[100]): rowcount = 6006.726049749041, cumulative cost =
{1.1468867492063493E8 rows, 1.166177684126984E8 cpu, -1.1757664816220238E9
io}, id = 3000
HiveSort(sort0=[$0], dir0=[ASC]): rowcount = 6006.726049749041, cumulative
cost = {1.1468867492063493E8 rows, 1.166177684126984E8 cpu,
-1.1757664816220238E9 io}, id = 2998
HiveProject(customer_id=[$4], customername=[concat($9, ', ', $8)]):
rowcount = 6006.726049749041, cumulative cost = {1.1468867492063493E8 rows,
1.166177684126984E8 cpu, -1.1757664816220238E9 io}, id = 3136
HiveJoin(condition=[=($1, $5)], joinType=[inner],
joinAlgorithm=[map_join], cost=[{5.557820341269841E7 rows,
5.557840182539682E7 cpu, -4299694.122023809 io}]): rowcount =
6006.726049749041, cumulative cost = {1.1468867492063493E8 rows,
1.166177684126984E8 cpu, -1.1757664816220238E9 io}, id = 3132
HiveJoin(condition=[=($0, $1)], joinType=[inner],
joinAlgorithm=[map_join], cost=[{5.7498805E7 rows, 5.9419605E7 cpu,
-1.15248E9 io}]): rowcount = 5.5578005E7, cumulative cost = {5.7498805E7
rows, 5.9419605E7 cpu, -1.15248E9 io}, id = 3100
HiveProject(sr_cdemo_sk=[$4]): rowcount = 5.5578005E7, cumulative
cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2992
HiveTableScan(table=[[tpcds_bin_orc_200.store_returns]]):
rowcount = 5.5578005E7, cumulative cost = {0}, id = 2878
HiveProject(cd_demo_sk=[$0]): rowcount = 1920800.0, cumulative cost
= {0.0 rows, 0.0 cpu, 0.0 io}, id = 2978
HiveTableScan(table=[[tpcds_bin_orc_200.customer_demographics]]):
rowcount = 1920800.0, cumulative cost = {0}, id = 2868
HiveJoin(condition=[=($10, $1)], joinType=[inner],
joinAlgorithm=[map_join], cost=[{1787.9365079365077 rows, 1790.15873015873
cpu, -8000.0 io}]): rowcount = 198.4126984126984, cumulative cost =
{1611666.507936508 rows, 1619761.5873015872 cpu, -1.89867875E7 io}, id = 3130
HiveJoin(condition=[=($0, $4)], joinType=[inner],
joinAlgorithm=[map_join], cost=[{8985.714285714286 rows, 16185.714285714286
cpu, -1.728E7 io}]): rowcount = 1785.7142857142856, cumulative cost =
{1609878.5714285714 rows, 1617971.4285714284 cpu, -1.89787875E7 io}, id = 3128
HiveProject(hd_demo_sk=[$0], hd_income_band_sk=[$1]): rowcount =
7200.0, cumulative cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2982
HiveTableScan(table=[[tpcds_bin_orc_200.household_demographics]]): rowcount =
7200.0, cumulative cost = {0}, id = 2871
HiveJoin(condition=[=($3, $6)], joinType=[inner],
joinAlgorithm=[map_join], cost=[{1600892.857142857 rows, 1601785.714285714
cpu, -1698787.48 io}]): rowcount = 1785.7142857142856, cumulative
cost = {1600892.857142857 rows, 1601785.714285714 cpu, -1698787.48
io}, id = 3105
HiveProject(c_customer_id=[$1], c_current_cdemo_sk=[$2],
c_current_hdemo_sk=[$3], c_current_addr_sk=[$4], c_first_name=[$8],
c_last_name=[$9]): rowcount = 160.0, cumulative cost = {0.0 rows, 0.0
cpu, 0.0 io}, id = 2970
HiveTableScan(table=[[tpcds_bin_orc_200.customer]]): rowcount
= 160.0, cumulative cost = {0}, id = 2862
HiveProject(ca_address_sk=[$0], ca_city=[$6]): rowcount =
892.8571428571428, cumulative
[jira] [Updated] (HIVE-10306) We need to print tez summary when hive.server2.logging.level = PERFORMANCE.
[ https://issues.apache.org/jira/browse/HIVE-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-10306: - Attachment: HIVE-10306.5.patch [~thejas] Made the change to move dependencies to hadoop-2 profile. We need to print tez summary when hive.server2.logging.level = PERFORMANCE. - Key: HIVE-10306 URL: https://issues.apache.org/jira/browse/HIVE-10306 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-10306.1.patch, HIVE-10306.2.patch, HIVE-10306.3.patch, HIVE-10306.4.patch, HIVE-10306.5.patch We need to print tez summary when hive.server2.logging.level = PERFORMANCE. We introduced this parameter via HIVE-10119. The logging param for levels is only relevant to HS2, so for hive-cli users the hive.tez.exec.print.summary still makes sense. We can check for log-level param as well, in places we are checking value of hive.tez.exec.print.summary. Ie, consider hive.tez.exec.print.summary=true if log.level = PERFORMANCE. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8950) Add support in ParquetHiveSerde to create table schema from a parquet file
[ https://issues.apache.org/jira/browse/HIVE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated HIVE-8950: - Attachment: HIVE-8950.7.patch Add support in ParquetHiveSerde to create table schema from a parquet file -- Key: HIVE-8950 URL: https://issues.apache.org/jira/browse/HIVE-8950 Project: Hive Issue Type: Improvement Reporter: Ashish K Singh Assignee: Ashish K Singh Attachments: HIVE-8950.1.patch, HIVE-8950.2.patch, HIVE-8950.3.patch, HIVE-8950.4.patch, HIVE-8950.5.patch, HIVE-8950.6.patch, HIVE-8950.7.patch, HIVE-8950.patch PARQUET-76 and PARQUET-47 ask for creating parquet backed tables without having to specify the column names and types. As, parquet files store schema in their footer, it is possible to generate hive schema from parquet file's metadata. This will improve usability of parquet backed tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-10369) CBO: Don't use HiveDefaultCostModel when With Tez and hive.cbo.costmodel.extended enabled
[
https://issues.apache.org/jira/browse/HIVE-10369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran resolved HIVE-10369.
---
Resolution: Fixed
CBO: Don't use HiveDefaultCostModel when With Tez and
hive.cbo.costmodel.extended enabled
--
Key: HIVE-10369
URL: https://issues.apache.org/jira/browse/HIVE-10369
Project: Hive
Issue Type: Sub-task
Components: CBO
Affects Versions: 1.2.0
Reporter: Mostafa Mokhtar
Assignee: Laljo John Pullokkaran
Fix For: 1.2.0
Attachments: HIVE-10369.patch
When calculating parallelism, we end up using HiveDefaultCostModel.
getSplitCount which returns null instead of HiveOnTezCostModel.getSplitCount
which results in wrong parallelism.
This happens for this join
{code}
org.apache.calcite.plan.RelOptUtil.toString(join)
(java.lang.String) HiveJoin(condition=[=($1, $3)], joinType=[inner],
algorithm=[none], cost=[not available])
HiveProject(cs_sold_date_sk=[$0], cs_bill_customer_sk=[$3],
cs_sales_price=[$21])
HiveTableScan(table=[[tpcds_bin_orc_200.catalog_sales]])
HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[MapJoin],
cost=[{240.0 rows, 6.48E11 cpu, 1294.6098 io}])
HiveProject(c_customer_sk=[$0], c_current_addr_sk=[$4])
HiveTableScan(table=[[tpcds_bin_orc_200.customer]])
HiveProject(ca_address_sk=[$0], ca_state=[$8], ca_zip=[$9])
HiveTableScan(table=[[tpcds_bin_orc_200.customer_address]])
{code}
The issue appears to be happening very early when calling
{code}
if (pushDownTree != null) {
costPushDown =
RelMetadataQuery.getCumulativeCost(pushDownTree.getJoinTree());
}
{code}
As pushDownTree.getJoinTree().joinAlgorithm =
HiveOnTezCostModel$TezMapJoinAlgorithm
Call stack.
{code}
HiveDefaultCostModel$DefaultJoinAlgorithm.getSplitCount(HiveJoin) line: 114
HiveJoin.getSplitCount() line: 136
HiveRelMdParallelism.splitCount(HiveJoin) line: 63
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ReflectiveRelMetadataProvider$1$1.invoke(Object, Method, Object[]) line: 182
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
CachingRelMetadataProvider$CachingInvocationHandler.invoke(Object, Method,
Object[]) line: 132
$Proxy46.splitCount() line: not available
RelMetadataQuery.splitCount(RelNode) line: 401
HiveOnTezCostModel$TezMapJoinAlgorithm.getCost(HiveJoin) line: 255
HiveOnTezCostModel(HiveCostModel).getJoinCost(HiveJoin) line: 64
HiveRelMdCost.getNonCumulativeCost(HiveJoin) line: 56
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ReflectiveRelMetadataProvider$1$1.invoke(Object, Method, Object[]) line: 182
$Proxy41.getNonCumulativeCost() line: not available
GeneratedMethodAccessor22.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy41.getNonCumulativeCost() line: not available
GeneratedMethodAccessor22.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
[jira] [Commented] (HIVE-10304) Add deprecation message to HiveCLI
[ https://issues.apache.org/jira/browse/HIVE-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498900#comment-14498900 ] Mithun Radhakrishnan commented on HIVE-10304: - Hello, [~szehon]. Pardon the delay; I wish I'd responded sooner. There was a discussion on the dev user-list that concluded that we shouldn't be deprecating the Hive command-line until we have interface/error-code parity between beeline and the CLI. [Here|http://mail-archives.apache.org/mod_mbox/hive-dev/201412.mbox/%3ccabgngzfnjhnfv0p15+glmznf-gogw6dm9xotgoqh+dnyg3z...@mail.gmail.com%3E] is one thread. To quote: bq. +1 to the idea of embedding beeline within hive cli, and retaining core behavior such as exit codes in hive-cli while doing that... users don't have to specify parameters like jdbc url, username etc. The issue I see here is that there are still Hive installations that depend on the CLI, and don't depend entirely on HS2 deploys. (Where I work, for instance.) I'd be very keen to see the embedded-beeline option in working order. Could we please discuss this check-in? I don't know if it's a good idea to push this into the impending release. I fear that the deprecation will be too disruptive, without proper recourse. Add deprecation message to HiveCLI -- Key: HIVE-10304 URL: https://issues.apache.org/jira/browse/HIVE-10304 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 1.1.0 Reporter: Szehon Ho Assignee: Szehon Ho Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-10304.2.patch, HIVE-10304.3.patch, HIVE-10304.patch As Beeline is now the recommended command line tool to Hive, we should add a message to HiveCLI to indicate that it is deprecated and redirect them to Beeline. This is not suggesting to remove HiveCLI for now, but just a helpful direction for user to know the direction to focus attention in Beeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5672) Insert with custom separator not supported for non-local directory
[
https://issues.apache.org/jira/browse/HIVE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499023#comment-14499023
]
Lefty Leverenz commented on HIVE-5672:
--
How's this?
{quote}
As of Hive 0.11.0 the separator used can be specified, in earlier versions it
was always the ^A character (\001). However, custom separators are currently
only supported for LOCAL writes; this is a bug that will be fixed by HIVE-5672.
{quote}
* [DML -- Writing data into the filesystem from queries -- Notes |
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Notes.2]
Insert with custom separator not supported for non-local directory
--
Key: HIVE-5672
URL: https://issues.apache.org/jira/browse/HIVE-5672
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0, 1.0.0
Reporter: Romain Rigaux
Assignee: Nemon Lou
Attachments: HIVE-5672.1.patch, HIVE-5672.2.patch, HIVE-5672.3.patch
https://issues.apache.org/jira/browse/HIVE-3682 is great but non local
directory don't seem to be supported:
{code}
insert overwrite directory '/tmp/test-02'
row format delimited
FIELDS TERMINATED BY ':'
select description FROM sample_07
{code}
{code}
Error while compiling statement: FAILED: ParseException line 2:0 cannot
recognize input near 'row' 'format' 'delimited' in select clause
{code}
This works (with 'local'):
{code}
insert overwrite local directory '/tmp/test-02'
row format delimited
FIELDS TERMINATED BY ':'
select code, description FROM sample_07
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10364) The HMS upgrade script test does not publish results when prepare.sh fails.
[
https://issues.apache.org/jira/browse/HIVE-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499102#comment-14499102
]
Hive QA commented on HIVE-10364:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725894/HIVE-10364.1.patch
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 8711 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testMetastoreProxyUser
org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testSaslWithHiveMetaStore
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3466/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3466/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3466/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725894 - PreCommit-HIVE-TRUNK-Build
The HMS upgrade script test does not publish results when prepare.sh fails.
---
Key: HIVE-10364
URL: https://issues.apache.org/jira/browse/HIVE-10364
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-10364.1.patch
The HMS upgrade script must publish succeed or failure results to JIRA. This
bug is not publishing any results on JIRA is the prepare.sh script fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9710) HiveServer2 should support cookie based authentication, when using HTTP transport.
[ https://issues.apache.org/jira/browse/HIVE-9710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498989#comment-14498989 ] Lefty Leverenz commented on HIVE-9710: -- Doc note: In addition to general documentation (release notes), these new configuration parameters need to be documented: * hive.server2.thrift.http.cookie.auth.enabled * hive.server2.thrift.http.cookie.max.age * hive.server2.thrift.http.cookie.domain * hive.server2.thrift.http.cookie.path * hive.server2.thrift.http.cookie.is.secure * hive.server2.thrift.http.cookie.is.httponly Here are the wikidocs for HS2 and its configs: * [Setting Up HiveServer2 | https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] * [HiveServer2 Clients | https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients] * [Configuration Properties -- HiveServer2 | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveServer2] HiveServer2 should support cookie based authentication, when using HTTP transport. -- Key: HIVE-9710 URL: https://issues.apache.org/jira/browse/HIVE-9710 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 1.2.0 Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-9710.1.patch, HIVE-9710.2.patch, HIVE-9710.3.patch, HIVE-9710.4.patch, HIVE-9710.5.patch, HIVE-9710.6.patch, HIVE-9710.7.patch, HIVE-9710.8.patch HiveServer2 should generate cookies and validate the client cookie send to it so that it need not perform User/Password or a Kerberos based authentication on each HTTP request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10369) CBO: Don't use HiveDefaultCostModel when With Tez and hive.cbo.costmodel.extended enabled
[
https://issues.apache.org/jira/browse/HIVE-10369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498904#comment-14498904
]
Mostafa Mokhtar commented on HIVE-10369:
[~jcamachorodriguez]
CBO: Don't use HiveDefaultCostModel when With Tez and
hive.cbo.costmodel.extended enabled
--
Key: HIVE-10369
URL: https://issues.apache.org/jira/browse/HIVE-10369
Project: Hive
Issue Type: Sub-task
Components: CBO
Affects Versions: 1.2.0
Reporter: Mostafa Mokhtar
Assignee: Laljo John Pullokkaran
Fix For: 1.2.0
When calculating parallelism, we end up using HiveDefaultCostModel.
getSplitCount which returns null instead of HiveOnTezCostModel.getSplitCount
which results in wrong parallelism.
This happens for this join
{code}
org.apache.calcite.plan.RelOptUtil.toString(join)
(java.lang.String) HiveJoin(condition=[=($1, $3)], joinType=[inner],
algorithm=[none], cost=[not available])
HiveProject(cs_sold_date_sk=[$0], cs_bill_customer_sk=[$3],
cs_sales_price=[$21])
HiveTableScan(table=[[tpcds_bin_orc_200.catalog_sales]])
HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[MapJoin],
cost=[{240.0 rows, 6.48E11 cpu, 1294.6098 io}])
HiveProject(c_customer_sk=[$0], c_current_addr_sk=[$4])
HiveTableScan(table=[[tpcds_bin_orc_200.customer]])
HiveProject(ca_address_sk=[$0], ca_state=[$8], ca_zip=[$9])
HiveTableScan(table=[[tpcds_bin_orc_200.customer_address]])
{code}
The issue appears to be happening very early when calling
{code}
if (pushDownTree != null) {
costPushDown =
RelMetadataQuery.getCumulativeCost(pushDownTree.getJoinTree());
}
{code}
As pushDownTree.getJoinTree().joinAlgorithm =
HiveOnTezCostModel$TezMapJoinAlgorithm
Call stack.
{code}
HiveDefaultCostModel$DefaultJoinAlgorithm.getSplitCount(HiveJoin) line: 114
HiveJoin.getSplitCount() line: 136
HiveRelMdParallelism.splitCount(HiveJoin) line: 63
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ReflectiveRelMetadataProvider$1$1.invoke(Object, Method, Object[]) line: 182
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
CachingRelMetadataProvider$CachingInvocationHandler.invoke(Object, Method,
Object[]) line: 132
$Proxy46.splitCount() line: not available
RelMetadataQuery.splitCount(RelNode) line: 401
HiveOnTezCostModel$TezMapJoinAlgorithm.getCost(HiveJoin) line: 255
HiveOnTezCostModel(HiveCostModel).getJoinCost(HiveJoin) line: 64
HiveRelMdCost.getNonCumulativeCost(HiveJoin) line: 56
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ReflectiveRelMetadataProvider$1$1.invoke(Object, Method, Object[]) line: 182
$Proxy41.getNonCumulativeCost() line: not available
GeneratedMethodAccessor22.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy41.getNonCumulativeCost() line: not available
GeneratedMethodAccessor22.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
[jira] [Updated] (HIVE-10268) Merge cbo branch into trunk
[ https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-10268: Attachment: HIVE-10268.4.patch Merge cbo branch into trunk --- Key: HIVE-10268 URL: https://issues.apache.org/jira/browse/HIVE-10268 Project: Hive Issue Type: Task Components: CBO Affects Versions: cbo-branch Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch, HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.patch Merge patch generated on basis of diffs of trunk with cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10371) CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter
[ https://issues.apache.org/jira/browse/HIVE-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10371: --- Attachment: HIVE-10371.cbo.patch CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter Key: HIVE-10371 URL: https://issues.apache.org/jira/browse/HIVE-10371 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-10371.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10222) Upgrade Calcite dependency to newest version
[ https://issues.apache.org/jira/browse/HIVE-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-10222: Component/s: CBO Upgrade Calcite dependency to newest version Key: HIVE-10222 URL: https://issues.apache.org/jira/browse/HIVE-10222 Project: Hive Issue Type: Bug Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.2.0 Attachments: HIVE-10222.01.patch, HIVE-10222.02.patch, HIVE-10222.03.patch, HIVE-10222.04.patch, HIVE-10222.patch Upgrade Calcite version to 1.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10304) Add deprecation message to HiveCLI
[ https://issues.apache.org/jira/browse/HIVE-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499176#comment-14499176 ] Szehon Ho commented on HIVE-10304: -- OK, filed HIVE-10374 for reversion while discussion is ongoing. Add deprecation message to HiveCLI -- Key: HIVE-10304 URL: https://issues.apache.org/jira/browse/HIVE-10304 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 1.1.0 Reporter: Szehon Ho Assignee: Szehon Ho Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-10304.2.patch, HIVE-10304.3.patch, HIVE-10304.patch As Beeline is now the recommended command line tool to Hive, we should add a message to HiveCLI to indicate that it is deprecated and redirect them to Beeline. This is not suggesting to remove HiveCLI for now, but just a helpful direction for user to know the direction to focus attention in Beeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10190) CBO: AST mode checks for TABLESAMPLE with AST.toString().contains(TOK_TABLESPLITSAMPLE)
[
https://issues.apache.org/jira/browse/HIVE-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499153#comment-14499153
]
Hive QA commented on HIVE-10190:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725898/HIVE-10190.05.patch
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 8711 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_decode_name
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_special_char
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_printf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_reverse
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_substr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udtf_json_tuple
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udtf_parse_url_tuple
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union35
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union_decimal
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3467/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3467/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3467/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725898 - PreCommit-HIVE-TRUNK-Build
CBO: AST mode checks for TABLESAMPLE with
AST.toString().contains(TOK_TABLESPLITSAMPLE)
-
Key: HIVE-10190
URL: https://issues.apache.org/jira/browse/HIVE-10190
Project: Hive
Issue Type: Bug
Components: CBO
Affects Versions: 1.2.0
Reporter: Gopal V
Assignee: Reuben Kuhnert
Priority: Trivial
Labels: perfomance
Attachments: HIVE-10190-querygen.py, HIVE-10190.01.patch,
HIVE-10190.02.patch, HIVE-10190.03.patch, HIVE-10190.04.patch,
HIVE-10190.05.patch, HIVE-10190.05.patch
{code}
public static boolean validateASTForUnsupportedTokens(ASTNode ast) {
String astTree = ast.toStringTree();
// if any of following tokens are present in AST, bail out
String[] tokens = { TOK_CHARSETLITERAL, TOK_TABLESPLITSAMPLE };
for (String token : tokens) {
if (astTree.contains(token)) {
return false;
}
}
return true;
}
{code}
This is an issue for a SQL query which is bigger in AST form than in text
(~700kb).
[jira] [Updated] (HIVE-10372) Bump parquet version to 1.6.0
[ https://issues.apache.org/jira/browse/HIVE-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-10372: Attachment: HIVE-10372.patch Hi [~spena], the maven central repo is not updated yet. Will be pending on that. Bump parquet version to 1.6.0 - Key: HIVE-10372 URL: https://issues.apache.org/jira/browse/HIVE-10372 Project: Hive Issue Type: Bug Reporter: Ferdinand Xu Assignee: Ferdinand Xu Attachments: HIVE-10372.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10324) Hive metatool should take table_param_key to allow for changes to avro serde's schema url key
[
https://issues.apache.org/jira/browse/HIVE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499055#comment-14499055
]
Ferdinand Xu commented on HIVE-10324:
-
Thanks [~szehon] for your review. And the failed cases are irrelevant.
Hive metatool should take table_param_key to allow for changes to avro
serde's schema url key
-
Key: HIVE-10324
URL: https://issues.apache.org/jira/browse/HIVE-10324
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 1.1.0
Reporter: Szehon Ho
Assignee: Ferdinand Xu
Attachments: HIVE-10324.1.patch, HIVE-10324.patch,
HIVE-10324.patch.WIP
HIVE-3443 added support to change the serdeParams from 'metatool
updateLocation' command.
However, in avro it is possible to specify the schema via the tableParams:
{noformat}
CREATE TABLE `testavro`(
`test` string COMMENT 'from deserializer')
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES (
'avro.schema.url'='hdfs://namenode:8020/tmp/test.avsc',
'kite.compression.type'='snappy',
'transient_lastDdlTime'='1427996456')
{noformat}
Hence for those tables the 'metatool updateLocation' will not help.
This is necessary in case like upgrade the namenode to HA where the absolute
paths have changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10369) CBO: Don't use HiveDefaultCostModel when With Tez and hive.cbo.costmodel.extended enabled
[
https://issues.apache.org/jira/browse/HIVE-10369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-10369:
--
Attachment: HIVE-10369.patch
CBO: Don't use HiveDefaultCostModel when With Tez and
hive.cbo.costmodel.extended enabled
--
Key: HIVE-10369
URL: https://issues.apache.org/jira/browse/HIVE-10369
Project: Hive
Issue Type: Sub-task
Components: CBO
Affects Versions: 1.2.0
Reporter: Mostafa Mokhtar
Assignee: Laljo John Pullokkaran
Fix For: 1.2.0
Attachments: HIVE-10369.patch
When calculating parallelism, we end up using HiveDefaultCostModel.
getSplitCount which returns null instead of HiveOnTezCostModel.getSplitCount
which results in wrong parallelism.
This happens for this join
{code}
org.apache.calcite.plan.RelOptUtil.toString(join)
(java.lang.String) HiveJoin(condition=[=($1, $3)], joinType=[inner],
algorithm=[none], cost=[not available])
HiveProject(cs_sold_date_sk=[$0], cs_bill_customer_sk=[$3],
cs_sales_price=[$21])
HiveTableScan(table=[[tpcds_bin_orc_200.catalog_sales]])
HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[MapJoin],
cost=[{240.0 rows, 6.48E11 cpu, 1294.6098 io}])
HiveProject(c_customer_sk=[$0], c_current_addr_sk=[$4])
HiveTableScan(table=[[tpcds_bin_orc_200.customer]])
HiveProject(ca_address_sk=[$0], ca_state=[$8], ca_zip=[$9])
HiveTableScan(table=[[tpcds_bin_orc_200.customer_address]])
{code}
The issue appears to be happening very early when calling
{code}
if (pushDownTree != null) {
costPushDown =
RelMetadataQuery.getCumulativeCost(pushDownTree.getJoinTree());
}
{code}
As pushDownTree.getJoinTree().joinAlgorithm =
HiveOnTezCostModel$TezMapJoinAlgorithm
Call stack.
{code}
HiveDefaultCostModel$DefaultJoinAlgorithm.getSplitCount(HiveJoin) line: 114
HiveJoin.getSplitCount() line: 136
HiveRelMdParallelism.splitCount(HiveJoin) line: 63
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ReflectiveRelMetadataProvider$1$1.invoke(Object, Method, Object[]) line: 182
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy46.splitCount() line: not available
GeneratedMethodAccessor26.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
CachingRelMetadataProvider$CachingInvocationHandler.invoke(Object, Method,
Object[]) line: 132
$Proxy46.splitCount() line: not available
RelMetadataQuery.splitCount(RelNode) line: 401
HiveOnTezCostModel$TezMapJoinAlgorithm.getCost(HiveJoin) line: 255
HiveOnTezCostModel(HiveCostModel).getJoinCost(HiveJoin) line: 64
HiveRelMdCost.getNonCumulativeCost(HiveJoin) line: 56
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ReflectiveRelMetadataProvider$1$1.invoke(Object, Method, Object[]) line: 182
$Proxy41.getNonCumulativeCost() line: not available
GeneratedMethodAccessor22.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(Object, Method,
Object[]) line: 109
$Proxy41.getNonCumulativeCost() line: not available
GeneratedMethodAccessor22.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
[jira] [Updated] (HIVE-10307) Support to use number literals in partition column
[ https://issues.apache.org/jira/browse/HIVE-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-10307: --- Attachment: HIVE-10307.2.patch Uploaded revised patch based on Jimmy's review. Support to use number literals in partition column -- Key: HIVE-10307 URL: https://issues.apache.org/jira/browse/HIVE-10307 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 1.0.0 Reporter: Chaoyu Tang Assignee: Chaoyu Tang Attachments: HIVE-10307.1.patch, HIVE-10307.2.patch, HIVE-10307.patch Data types like TinyInt, SmallInt, BigInt or Decimal can be expressed as literals with postfix like Y, S, L, or BD appended to the number. These literals work in most Hive queries, but do not when they are used as partition column value. For a partitioned table like: create table partcoltypenum (key int, value string) partitioned by (tint tinyint, sint smallint, bint bigint); insert into partcoltypenum partition (tint=100Y, sint=1S, bint=1000L) select key, value from src limit 30; Queries like select, describe and drop partition do not work. For an example select * from partcoltypenum where tint=100Y and sint=1S and bint=1000L; does not return any rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[ https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498411#comment-14498411 ] Eugene Koifman commented on HIVE-10242: --- The fix also logging improvements and a fix to the following situation where a lock (by client3) could be acquired when it should not be. client1: update table T client2: select * from T client3: update table T with all 3 clients executing concurrently and starting in above order, client 3 should be blocked until client1 finishes. ACID: insert overwrite prevents create table command Key: HIVE-10242 URL: https://issues.apache.org/jira/browse/HIVE-10242 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 1.2.0 Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, HIVE-10242.4.patch, HIVE-10242.patch 1. insert overwirte table DB.T1 select ... from T2: this takes X lock on DB.T1 and S lock on T2. X lock makes sense because we don't want anyone reading T1 while it's overwritten. S lock on T2 prevents if from being dropped while the query is in progress. 2. create table DB.T3: takes S lock on DB. This S lock gets blocked by X lock on T1. S lock prevents the DB from being dropped while create table is executed. If the insert statement is long running, this blocks DDL ops on the same database. This is a usability issue. There is no good reason why X lock on a table within a DB and S lock on DB should be in conflict. (this is different from a situation where X lock is on a partition and S lock is on the table to which this partition belongs. Here it makes sense. Basically there is no SQL way to address all tables in a DB but you can easily refer to all partitions of a table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10270) Cannot use Decimal constants less than 0.1BD
[
https://issues.apache.org/jira/browse/HIVE-10270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498417#comment-14498417
]
Gunther Hagleitner commented on HIVE-10270:
---
LGTM +1
Cannot use Decimal constants less than 0.1BD
Key: HIVE-10270
URL: https://issues.apache.org/jira/browse/HIVE-10270
Project: Hive
Issue Type: Bug
Components: Types
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-10270.1.patch, HIVE-10270.2.patch,
HIVE-10270.3.patch, HIVE-10270.4.patch, HIVE-10270.5.patch
{noformat}
hive select 0.09765625BD;
FAILED: IllegalArgumentException Decimal scale must be less than or equal to
precision
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9710) HiveServer2 should support cookie based authentication, when using HTTP transport.
[ https://issues.apache.org/jira/browse/HIVE-9710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498430#comment-14498430 ] Vaibhav Gumashta commented on HIVE-9710: [~hsubramaniyan] It will be good to add documentation in the release notes section on how to enable/disable and use this feature. Once [~leftylev] reviews it, we can add it to the wiki. Thanks. HiveServer2 should support cookie based authentication, when using HTTP transport. -- Key: HIVE-9710 URL: https://issues.apache.org/jira/browse/HIVE-9710 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 1.2.0 Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-9710.1.patch, HIVE-9710.2.patch, HIVE-9710.3.patch, HIVE-9710.4.patch, HIVE-9710.5.patch, HIVE-9710.6.patch, HIVE-9710.7.patch, HIVE-9710.8.patch HiveServer2 should generate cookies and validate the client cookie send to it so that it need not perform User/Password or a Kerberos based authentication on each HTTP request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10346) Tez on HBase has problems with settings again
[ https://issues.apache.org/jira/browse/HIVE-10346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498448#comment-14498448 ] Sergey Shelukhin commented on HIVE-10346: - checkOutputSpec is actually an API on outputformat; the one of FileSink is just a translation call between HiveOutputFormatImpl and real output formats inside the filesink... so needs to be called. Tez on HBase has problems with settings again - Key: HIVE-10346 URL: https://issues.apache.org/jira/browse/HIVE-10346 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-10346.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10356) LLAP: query80 fails with vectorization cast issue
[
https://issues.apache.org/jira/browse/HIVE-10356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498473#comment-14498473
]
Sergey Shelukhin commented on HIVE-10356:
-
Is that one going to be fixed? :)
LLAP: query80 fails with vectorization cast issue
--
Key: HIVE-10356
URL: https://issues.apache.org/jira/browse/HIVE-10356
Project: Hive
Issue Type: Sub-task
Reporter: Sergey Shelukhin
Assignee: Matt McCline
Reducer 6 fails:
{noformat}
Error: Failure while running task:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing vector batch (tag=0)
\N\N09.285817653506076E84.639990363237801E7-1.1814318134524737E8
\N\N01.2847032699693155E96.41569738480791E7-5.956161019898126E8
\N\N04.682909323885761E82.288924051203157E7-5.995957665973593E7
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:332)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:180)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing vector batch (tag=0)
\N\N09.285817653506076E84.639990363237801E7-1.1814318134524737E8
\N\N01.2847032699693155E96.41569738480791E7-5.956161019898126E8
\N\N04.682909323885761E82.288924051203157E7-5.995957665973593E7
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:267)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:254)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing vector batch (tag=0)
\N\N09.285817653506076E84.639990363237801E7-1.1814318134524737E8
\N\N01.2847032699693155E96.41569738480791E7-5.956161019898126E8
\N\N04.682909323885761E82.288924051203157E7-5.995957665973593E7
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectors(ReduceRecordSource.java:394)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:252)
... 16 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.exec.vector.DoubleColumnVector cannot be cast to
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupKeyHelper.copyGroupKey(VectorGroupKeyHelper.java:94)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeGroupBatches.processBatch(VectorGroupByOperator.java:729)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.process(VectorGroupByOperator.java:878)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectors(ReduceRecordSource.java:378)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex
vertex_1428572510173_0231_1_24 [Reducer 5] killed/failed due to:null]Vertex
killed, vertexName=Reducer 6, vertexId=vertex_1428572510173_0231_1_25,
diagnostics=[Vertex received Kill while in RUNNING state., Vertex killed as
other vertex failed. failedTasks:0, Vertex vertex_1428572510173_0231_1_25
[Reducer 6] killed/failed due to:null]DAG failed due to vertex failure.
failedVertices:1 killedVertices:1
{noformat}
How to repro: run query80 on scale factor 200. I might look tomorrow to see
if this is specific to LLAP or not
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-4625) HS2 should not attempt to get delegation token from metastore if using embedded metastore
[ https://issues.apache.org/jira/browse/HIVE-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-4625: Attachment: HIVE-4625.2.patch HS2 should not attempt to get delegation token from metastore if using embedded metastore - Key: HIVE-4625 URL: https://issues.apache.org/jira/browse/HIVE-4625 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-4625.1.patch, HIVE-4625.2.patch In kerberos secure mode, with doas enabled, Hive server2 tries to get delegation token from metastore even if the metastore is being used in embedded mode. To avoid failure in that case, it uses catch block for UnsupportedOperationException thrown that does nothing. But this leads to an error being logged by lower levels and can mislead users into thinking that there is a problem. It should check if delegation token mode is supported with current configuration before calling the function. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8136) Reduce table locking
[ https://issues.apache.org/jira/browse/HIVE-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498498#comment-14498498 ] Chaoyu Tang commented on HIVE-8136: --- [~Ferd] I have a question, when Hive processes a query, it needs fetch Table/Partition/Serde metadata from HMS, is this metadata cached at HMS client side (as Table/Partition) or to be retrieved from HMS each time it is needed, no matter the metadata is cached at HMS (as JDO entity object)? If the first one is the case, it is easer and I think the change should be fine. Otherwise, I think we should be more conservative and might even need add DDL_EXCLUSIVE on anything which may affect other query behaviors (e.g. ADDPROP etc). For example, querying data for an Avro or HBase backed tables, the Avro schema url/literal or HBase column mapping is specified as a property. Can we guarantee that Hive only need access these data once or all the metadata has been cached at its HMS client? Reduce table locking Key: HIVE-8136 URL: https://issues.apache.org/jira/browse/HIVE-8136 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Assignee: Ferdinand Xu Attachments: HIVE-8136.1.patch, HIVE-8136.patch When using ZK for concurrency control, some statements require an exclusive table lock when they are atomic. Such as setting a tables location. This JIRA is to analyze the scope of statements like ALTER TABLE and see if we can reduce the locking required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
