[jira] [Updated] (HIVE-10407) separate out the timestamp ranges for testing purposes
[ https://issues.apache.org/jira/browse/HIVE-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-10407: - Attachment: HIVE-10407.patch Fixed the cases of year = that Prasanth found. separate out the timestamp ranges for testing purposes -- Key: HIVE-10407 URL: https://issues.apache.org/jira/browse/HIVE-10407 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: HIVE-10407.patch, HIVE-10407.patch, HIVE-10407.patch Some platforms have limits for date ranges, so separate out the test cases that are outside of the range 1970 to 2038. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10324) Hive metatool should take table_param_key to allow for changes to avro serde's schema url key
[
https://issues.apache.org/jira/browse/HIVE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504121#comment-14504121
]
Lefty Leverenz commented on HIVE-10324:
---
Super, thanks [~Ferd]!
Hive metatool should take table_param_key to allow for changes to avro
serde's schema url key
-
Key: HIVE-10324
URL: https://issues.apache.org/jira/browse/HIVE-10324
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 1.1.0
Reporter: Szehon Ho
Assignee: Ferdinand Xu
Fix For: 1.2.0
Attachments: HIVE-10324.1.patch, HIVE-10324.patch,
HIVE-10324.patch.WIP
HIVE-3443 added support to change the serdeParams from 'metatool
updateLocation' command.
However, in avro it is possible to specify the schema via the tableParams:
{noformat}
CREATE TABLE `testavro`(
`test` string COMMENT 'from deserializer')
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES (
'avro.schema.url'='hdfs://namenode:8020/tmp/test.avsc',
'kite.compression.type'='snappy',
'transient_lastDdlTime'='1427996456')
{noformat}
Hence for those tables the 'metatool updateLocation' will not help.
This is necessary in case like upgrade the namenode to HA where the absolute
paths have changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-10400:
Affects Version/s: 1.2.0
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Affects Versions: 1.2.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9917) After HIVE-3454 is done, make int to timestamp conversion configurable
[ https://issues.apache.org/jira/browse/HIVE-9917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503780#comment-14503780 ] Jason Dere commented on HIVE-9917: -- You're right, thanks for looking into that. Sorry to make you change the patch again .. can you put hbase_timestamp.q back to what it looked like before HIVE-3454? After HIVE-3454 is done, make int to timestamp conversion configurable -- Key: HIVE-9917 URL: https://issues.apache.org/jira/browse/HIVE-9917 Project: Hive Issue Type: Improvement Reporter: Aihua Xu Assignee: Aihua Xu Attachments: HIVE-9917.patch After HIVE-3454 is fixed, we will have correct behavior of converting int to timestamp. While the customers are using such incorrect behavior for so long, better to make it configurable so that in one release, it will default to old/inconsistent way and the next release will default to new/consistent way. And then we will deprecate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10407) separate out the timestamp ranges for testing purposes
[ https://issues.apache.org/jira/browse/HIVE-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503819#comment-14503819 ] Prasanth Jayachandran commented on HIVE-10407: -- TestOrcTimezone1 and TestOrcTimezone2 classes also have year test cases. separate out the timestamp ranges for testing purposes -- Key: HIVE-10407 URL: https://issues.apache.org/jira/browse/HIVE-10407 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: HIVE-10407.patch, HIVE-10407.patch Some platforms have limits for date ranges, so separate out the test cases that are outside of the range 1970 to 2038. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10410) Apparent race condition in HiveServer2 causing intermittent query failures
[
https://issues.apache.org/jira/browse/HIVE-10410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504025#comment-14504025
]
Mostafa Mokhtar commented on HIVE-10410:
[~ekoifman] FYI.
Apparent race condition in HiveServer2 causing intermittent query failures
--
Key: HIVE-10410
URL: https://issues.apache.org/jira/browse/HIVE-10410
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.1
Environment: CDH 5.3.3
CentOS 6.4
Reporter: Richard Williams
On our secure Hadoop cluster, queries submitted to HiveServer2 through JDBC
occasionally trigger odd Thrift exceptions with messages such as Read a
negative frame size (-2147418110)! or out of sequence response in
HiveServer2's connections to the metastore. For certain metastore calls (for
example, showDatabases), these Thrift exceptions are converted to
MetaExceptions in HiveMetaStoreClient, which prevents RetryingMetaStoreClient
from retrying these calls and thus causes the failure to bubble out to the
JDBC client.
Note that as far as we can tell, this issue appears to only affect queries
that are submitted with the runAsync flag on TExecuteStatementReq set to true
(which, in practice, seems to mean all JDBC queries), and it appears to only
manifest when HiveServer2 is using the new HTTP transport mechanism. When
both these conditions hold, we are able to fairly reliably reproduce the
issue by spawning about 100 simple, concurrent hive queries (we have been
using show databases), two or three of which typically fail. However, when
either of these conditions do not hold, we are no longer able to reproduce
the issue.
Some example stack traces from the HiveServer2 logs:
{noformat}
2015-04-16 13:54:55,486 ERROR hive.log: Got exception:
org.apache.thrift.transport.TTransportException Read a negative frame size
(-2147418110)!
org.apache.thrift.transport.TTransportException: Read a negative frame size
(-2147418110)!
at
org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:435)
at
org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:414)
at
org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.hadoop.hive.thrift.TFilterTransport.readAll(TFilterTransport.java:62)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:600)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:587)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:837)
at
org.apache.sentry.binding.metastore.SentryHiveMetaStoreClient.getDatabases(SentryHiveMetaStoreClient.java:60)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy6.getDatabases(Unknown Source)
at
org.apache.hadoop.hive.ql.metadata.Hive.getDatabasesByPattern(Hive.java:1139)
at
org.apache.hadoop.hive.ql.exec.DDLTask.showDatabases(DDLTask.java:2445)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:364)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1554)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1321)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1139)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:957)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:145)
at
org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:69)
at
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:200)
at
[jira] [Commented] (HIVE-10347) Merge spark to trunk 4/15/2015
[ https://issues.apache.org/jira/browse/HIVE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503559#comment-14503559 ] Szehon Ho commented on HIVE-10347: -- Clarification, I mean to say the com.jackson stuff is not actually in the uber-jar. It seems it is just pulled in for compile/test purpose. Merge spark to trunk 4/15/2015 -- Key: HIVE-10347 URL: https://issues.apache.org/jira/browse/HIVE-10347 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-10347.2.patch, HIVE-10347.2.patch, HIVE-10347.3.patch, HIVE-10347.4.patch, HIVE-10347.5.patch, HIVE-10347.patch CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10347) Merge spark to trunk 4/15/2015
[ https://issues.apache.org/jira/browse/HIVE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-10347: - Attachment: HIVE-10347.5.patch Rebase to be safe. Merge spark to trunk 4/15/2015 -- Key: HIVE-10347 URL: https://issues.apache.org/jira/browse/HIVE-10347 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-10347.2.patch, HIVE-10347.2.patch, HIVE-10347.3.patch, HIVE-10347.4.patch, HIVE-10347.5.patch, HIVE-10347.5.patch, HIVE-10347.patch CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10409) Webhcat tests need to be updated, to accomodate HADOOP-10193
[ https://issues.apache.org/jira/browse/HIVE-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aswathy Chellammal Sreekumar updated HIVE-10409: Attachment: HIVE-10409.patch Webhcat tests need to be updated, to accomodate HADOOP-10193 Key: HIVE-10409 URL: https://issues.apache.org/jira/browse/HIVE-10409 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 1.2.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Fix For: 1.2.0 Attachments: HIVE-10409.patch Webhcat tests need to be updated to accommodate the url change brought in by HADOOP-10193. Add ?user.name=user-name for the templeton calls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-10411) LLAP: NPE caused by HIVE-10397
[ https://issues.apache.org/jira/browse/HIVE-10411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-10411. -- Resolution: Fixed Committed to llap branch LLAP: NPE caused by HIVE-10397 -- Key: HIVE-10411 URL: https://issues.apache.org/jira/browse/HIVE-10411 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10411.patch Fix NPE caused by HIVE-10397 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10385) Optionally disable partition creation to speedup ETL jobs
[ https://issues.apache.org/jira/browse/HIVE-10385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504116#comment-14504116 ] Xuefu Zhang commented on HIVE-10385: Not sure if I understand the request correctly. If we load a table with dynamic partitioning w/o creating these partitions at the end, why do we even bother using dynamic partitioning at all. A use case would help. Optionally disable partition creation to speedup ETL jobs - Key: HIVE-10385 URL: https://issues.apache.org/jira/browse/HIVE-10385 Project: Hive Issue Type: Improvement Components: Hive Reporter: Slava Markeyev Priority: Minor Attachments: HIVE-10385.patch ETL jobs that create dynamic partitions with high cardinality perform the expensive step of metastore partition creation after query completion. Until bulk partition creation can be optimized there should be a way of optionally skipping this step. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7948) Add an E2E test to verify fix for HIVE-7155
[ https://issues.apache.org/jira/browse/HIVE-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503585#comment-14503585 ] Eugene Koifman commented on HIVE-7948: -- +1 Add an E2E test to verify fix for HIVE-7155 Key: HIVE-7948 URL: https://issues.apache.org/jira/browse/HIVE-7948 Project: Hive Issue Type: Test Components: Tests, WebHCat Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Attachments: HIVE-7948.1.patch, HIVE-7948.2.patch, HIVE-7948.patch E2E Test to verify webhcat property templeton.mapper.memory.mb correctly overrides mapreduce.map.memory.mb. The feature was added as part of HIVE-7155. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10120) Disallow create table with dot/colon in column name
[
https://issues.apache.org/jira/browse/HIVE-10120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503683#comment-14503683
]
Pengcheng Xiong commented on HIVE-10120:
[~jpullokkaran] and [~ashutoshc], I checked Hive 0.13,0.14,1.1, in all of these
versions, it throws exception if we would like to select the column with
dot/colon. And, in the previous patch, I already included negative tests. Thus,
I think it is ready to go. Thanks.
Disallow create table with dot/colon in column name
---
Key: HIVE-10120
URL: https://issues.apache.org/jira/browse/HIVE-10120
Project: Hive
Issue Type: Improvement
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong
Attachments: HIVE-10120.01.patch, HIVE-10120.02.patch
Since we don't allow users to query column names with dot in the middle such
as emp.no, don't allow users to create tables with such columns that cannot
be queried. Fix the documentation to reflect this fix.
Here is an example. Consider this table:
{code}
CREATE TABLE a (`emp.no` string);
select `emp.no` from a; fails with this message:
FAILED: RuntimeException java.lang.RuntimeException: cannot find field emp
from [0:emp.no]
{code}
The hive documentation needs to be fixed:
{code}
(https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL) seems
to indicate that any Unicode character can go between the backticks in the
select statement, but it doesn’t like the dot/colon or even select * when
there is a column that has a dot/colon.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10250) Optimize AuthorizationPreEventListener to reuse TableWrapper objects
[
https://issues.apache.org/jira/browse/HIVE-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503743#comment-14503743
]
Sushanth Sowmyan commented on HIVE-10250:
-
I looked into the test failure, and here's what I see:
Using
PartitionWrapper(org.apache.hadoop.hive.ql.metadata.Table,org.apache.hadoop.hive.metastore.api.Partition)
expects the Table object to be initialized, and the Partition.getSd() to not
be null.
As of the time the preEventListener is fired, the
org.apache.hadoop.hive.metastore.api.Partition has not yet had its sd
initialized. So, we might want to add a bit to the
PartitionWrapper(org.apache.hadoop.hive.ql.metadata.Table,org.apache.hadoop.hive.metastore.api.Partition)
ctor so that it's similar to the
PartitionWrapper(org.apache.hadoop.hive.metastore.api.Partition,
PreEventContext) ctor, and then we can use a TableWrapper instead. Note that if
the TableWrapper itself has a null sd, we have a potential failure condition -
fetching using context.getHandler().get_table_core is the right call instead
of using a TableWrapper then. It might even be worth fetching a Table once and
caching it, instead of creating a TableWrapper and caching it.
Optimize AuthorizationPreEventListener to reuse TableWrapper objects
Key: HIVE-10250
URL: https://issues.apache.org/jira/browse/HIVE-10250
Project: Hive
Issue Type: Bug
Components: Authorization
Reporter: Mithun Radhakrishnan
Assignee: Mithun Radhakrishnan
Attachments: HIVE-10250.1.patch
Here's the {{PartitionWrapper}} class in {{AuthorizationPreEventListener}}:
{code:java|title=AuthorizationPreEventListener.java}
public static class PartitionWrapper extends
org.apache.hadoop.hive.ql.metadata.Partition {
...
public PartitionWrapper(org.apache.hadoop.hive.metastore.api.Partition
mapiPart, PreEventContext context) throws ... {
Partition wrapperApiPart = mapiPart.deepCopy();
Table t = context.getHandler().get_table_core(
mapiPart.getDbName(),
mapiPart.getTableName());
...
}
{code}
{{PreAddPartitionEvent}} (and soon, {{PreDropPartitionEvent}}) correspond not
just to a single partition, but an entire set of partitions added atomically.
When the event is authorized, {{HMSHandler.get_table_core()}} will be called
once for every partition in the Event instance.
Since we already make the assumption that the partition-sets correspond to a
single table, we might as well make a single call.
I'll have a patch for this, shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10409) Webhcat tests need to be updated, to accomodate HADOOP-10193
[ https://issues.apache.org/jira/browse/HIVE-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503843#comment-14503843 ] Aswathy Chellammal Sreekumar commented on HIVE-10409: - Please review the test case changes to update the templeton url, with user.name appended. Attached the patch. Webhcat tests need to be updated, to accomodate HADOOP-10193 Key: HIVE-10409 URL: https://issues.apache.org/jira/browse/HIVE-10409 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 1.2.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Fix For: 1.2.0 Attachments: HIVE-10409.patch Webhcat tests need to be updated to accommodate the url change brought in by HADOOP-10193. Add ?user.name=user-name for the templeton calls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10382) Aggregate stats cache for RDBMS based metastore codepath
[
https://issues.apache.org/jira/browse/HIVE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504096#comment-14504096
]
Hive QA commented on HIVE-10382:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726644/HIVE-10382.1.patch
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 8731 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3505/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3505/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3505/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726644 - PreCommit-HIVE-TRUNK-Build
Aggregate stats cache for RDBMS based metastore codepath
Key: HIVE-10382
URL: https://issues.apache.org/jira/browse/HIVE-10382
Project: Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 1.2.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 1.2.0
Attachments: HIVE-10382.1.patch, HIVE-10382.1.patch
Similar to the work done on the HBase branch (HIVE-9693), the stats cache can
potentially have performance gains.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[
https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503644#comment-14503644
]
Hive QA commented on HIVE-10227:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726608/HIVE-10227.4.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 8746 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_partitioned
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3502/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3502/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3502/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726608 - PreCommit-HIVE-TRUNK-Build
Concrete implementation of Export/Import based ReplicationTaskFactory
-
Key: HIVE-10227
URL: https://issues.apache.org/jira/browse/HIVE-10227
Project: Hive
Issue Type: Sub-task
Components: Import/Export
Affects Versions: 1.2.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-10227.2.patch, HIVE-10227.3.patch,
HIVE-10227.4.patch, HIVE-10227.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-7948) Add an E2E test to verify fix for HIVE-7155
[ https://issues.apache.org/jira/browse/HIVE-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved HIVE-7948. -- Resolution: Fixed Fix Version/s: 1.2.0 Committed to trunk. Thanks [~asreekumar] for the contribution. Add an E2E test to verify fix for HIVE-7155 Key: HIVE-7948 URL: https://issues.apache.org/jira/browse/HIVE-7948 Project: Hive Issue Type: Test Components: Tests, WebHCat Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Fix For: 1.2.0 Attachments: HIVE-7948.1.patch, HIVE-7948.2.patch, HIVE-7948.patch E2E Test to verify webhcat property templeton.mapper.memory.mb correctly overrides mapreduce.map.memory.mb. The feature was added as part of HIVE-7155. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10407) separate out the timestamp ranges for testing purposes
[ https://issues.apache.org/jira/browse/HIVE-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503911#comment-14503911 ] Prasanth Jayachandran commented on HIVE-10407: -- LGTM, +1. Pending tests. separate out the timestamp ranges for testing purposes -- Key: HIVE-10407 URL: https://issues.apache.org/jira/browse/HIVE-10407 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: HIVE-10407.patch, HIVE-10407.patch, HIVE-10407.patch Some platforms have limits for date ranges, so separate out the test cases that are outside of the range 1970 to 2038. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10408) LLAP: query fails - NPE (old exception I posted was bogus)
[
https://issues.apache.org/jira/browse/HIVE-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-10408:
Summary: LLAP: query fails - NPE (old exception I posted was bogus) (was:
LLAP: query fails - execution is rejected when it shouldn't be)
LLAP: query fails - NPE (old exception I posted was bogus)
--
Key: HIVE-10408
URL: https://issues.apache.org/jira/browse/HIVE-10408
Project: Hive
Issue Type: Sub-task
Reporter: Sergey Shelukhin
Assignee: Siddharth Seth
{noformat}
java.lang.NullPointerException
at
org.apache.tez.dag.app.rm.LlapTaskSchedulerService.deallocateTask(LlapTaskSchedulerService.java:388)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.handleTASucceeded(TaskSchedulerEventHandler.java:339)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.handleEvent(TaskSchedulerEventHandler.java:224)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler$1.run(TaskSchedulerEventHandler.java:493)
{noformat}
The query, running alone on 10-node cluster, dumped 1000 mappers into
running; with 3 completed it failed with that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10408) LLAP: query fails - execution is rejected when it shouldn't be
[
https://issues.apache.org/jira/browse/HIVE-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-10408:
Description:
{noformat}
java.lang.NullPointerException
at
org.apache.tez.dag.app.rm.LlapTaskSchedulerService.deallocateTask(LlapTaskSchedulerService.java:388)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.handleTASucceeded(TaskSchedulerEventHandler.java:339)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.handleEvent(TaskSchedulerEventHandler.java:224)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler$1.run(TaskSchedulerEventHandler.java:493)
{noformat}
The query, running alone on 10-node cluster, dumped 1000 mappers into running;
with 3 completed it failed with that.
was:
{noformat}
Caused by:
org.apache.hadoop.ipc.RemoteException(java.util.concurrent.RejectedExecutionException):
Queues are full. Rejecting request.
at
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService.schedule(TaskExecutorService.java:182)
at
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService.schedule(TaskExecutorService.java:63)
at
org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:202)
at
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:258)
at
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemonProtocolServerImpl.submitWork(LlapDaemonProtocolServerImpl.java:71)
at
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:8698)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:972)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2056)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2052)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2050)
at org.apache.hadoop.ipc.Client.call(Client.java:1492)
at org.apache.hadoop.ipc.Client.call(Client.java:1423)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
... 8 more
{noformat}
The query, running alone on 10-node cluster, dumped 1000 mappers into running;
with 3 completed it failed with that.
LLAP: query fails - execution is rejected when it shouldn't be
--
Key: HIVE-10408
URL: https://issues.apache.org/jira/browse/HIVE-10408
Project: Hive
Issue Type: Sub-task
Reporter: Sergey Shelukhin
Assignee: Siddharth Seth
{noformat}
java.lang.NullPointerException
at
org.apache.tez.dag.app.rm.LlapTaskSchedulerService.deallocateTask(LlapTaskSchedulerService.java:388)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.handleTASucceeded(TaskSchedulerEventHandler.java:339)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.handleEvent(TaskSchedulerEventHandler.java:224)
at
org.apache.tez.dag.app.rm.TaskSchedulerEventHandler$1.run(TaskSchedulerEventHandler.java:493)
{noformat}
The query, running alone on 10-node cluster, dumped 1000 mappers into
running; with 3 completed it failed with that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-10368) VectorExpressionWriter doesn't match vectorColumn during row spilling in HybridGraceHashJoin
[
https://issues.apache.org/jira/browse/HIVE-10368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Zheng reassigned HIVE-10368:
Assignee: Matt McCline (was: Wei Zheng)
[~mmccline] Can you take a look at this issue? Thanks.
VectorExpressionWriter doesn't match vectorColumn during row spilling in
HybridGraceHashJoin
Key: HIVE-10368
URL: https://issues.apache.org/jira/browse/HIVE-10368
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Wei Zheng
Assignee: Matt McCline
This problem was exposed by HIVE-10284, when testing vectorized_context.q
Below is the query and backtrace:
{code}
select store.s_city, ss_net_profit
from store_sales
JOIN store ON store_sales.ss_store_sk = store.s_store_sk
JOIN household_demographics ON store_sales.ss_hdemo_sk =
household_demographics.hd_demo_sk
limit 100
{code}
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector cannot be cast to
org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterLong.writeValue(VectorExpressionWriterFactory.java:175)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.getRowObject(VectorMapJoinOperator.java:347)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.spillBigTableRow(VectorMapJoinOperator.java:306)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:390)
... 24 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10411) LLAP: NPE caused by HIVE-10397
[ https://issues.apache.org/jira/browse/HIVE-10411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-10411: - Attachment: HIVE-10411.patch LLAP: NPE caused by HIVE-10397 -- Key: HIVE-10411 URL: https://issues.apache.org/jira/browse/HIVE-10411 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10411.patch Fix NPE caused by HIVE-10397 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8890) HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe
[ https://issues.apache.org/jira/browse/HIVE-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-8890: --- Affects Version/s: 1.1.0 HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe Key: HIVE-8890 URL: https://issues.apache.org/jira/browse/HIVE-8890 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.14.0, 1.0.0, 1.1.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 1.2.0 Attachments: HIVE-8890.1.patch, HIVE-8890.2.patch, HIVE-8890.3.patch Using this recipe gives better reliability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8890) HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe
[ https://issues.apache.org/jira/browse/HIVE-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-8890: --- Priority: Critical (was: Major) HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe Key: HIVE-8890 URL: https://issues.apache.org/jira/browse/HIVE-8890 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.14.0, 1.0.0, 1.1.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Priority: Critical Fix For: 1.2.0 Attachments: HIVE-8890.1.patch, HIVE-8890.2.patch, HIVE-8890.3.patch Using this recipe gives better reliability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8890) HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe
[ https://issues.apache.org/jira/browse/HIVE-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-8890: --- Fix Version/s: (was: 1.1.0) 1.2.0 HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe Key: HIVE-8890 URL: https://issues.apache.org/jira/browse/HIVE-8890 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.14.0, 1.0.0, 1.1.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 1.2.0 Attachments: HIVE-8890.1.patch, HIVE-8890.2.patch, HIVE-8890.3.patch Using this recipe gives better reliability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10410) Apparent race condition in HiveServer2 causing intermittent query failures
[
https://issues.apache.org/jira/browse/HIVE-10410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504058#comment-14504058
]
Eugene Koifman commented on HIVE-10410:
---
In HIVE-10404 the out of sequence response is caused by threads sharing
instance of Hive which effectively shares MetaStoreClient which itself is not
thread safe. Maybe something similar is happening here.
Apparent race condition in HiveServer2 causing intermittent query failures
--
Key: HIVE-10410
URL: https://issues.apache.org/jira/browse/HIVE-10410
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.1
Environment: CDH 5.3.3
CentOS 6.4
Reporter: Richard Williams
On our secure Hadoop cluster, queries submitted to HiveServer2 through JDBC
occasionally trigger odd Thrift exceptions with messages such as Read a
negative frame size (-2147418110)! or out of sequence response in
HiveServer2's connections to the metastore. For certain metastore calls (for
example, showDatabases), these Thrift exceptions are converted to
MetaExceptions in HiveMetaStoreClient, which prevents RetryingMetaStoreClient
from retrying these calls and thus causes the failure to bubble out to the
JDBC client.
Note that as far as we can tell, this issue appears to only affect queries
that are submitted with the runAsync flag on TExecuteStatementReq set to true
(which, in practice, seems to mean all JDBC queries), and it appears to only
manifest when HiveServer2 is using the new HTTP transport mechanism. When
both these conditions hold, we are able to fairly reliably reproduce the
issue by spawning about 100 simple, concurrent hive queries (we have been
using show databases), two or three of which typically fail. However, when
either of these conditions do not hold, we are no longer able to reproduce
the issue.
Some example stack traces from the HiveServer2 logs:
{noformat}
2015-04-16 13:54:55,486 ERROR hive.log: Got exception:
org.apache.thrift.transport.TTransportException Read a negative frame size
(-2147418110)!
org.apache.thrift.transport.TTransportException: Read a negative frame size
(-2147418110)!
at
org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:435)
at
org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:414)
at
org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.hadoop.hive.thrift.TFilterTransport.readAll(TFilterTransport.java:62)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:600)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:587)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:837)
at
org.apache.sentry.binding.metastore.SentryHiveMetaStoreClient.getDatabases(SentryHiveMetaStoreClient.java:60)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy6.getDatabases(Unknown Source)
at
org.apache.hadoop.hive.ql.metadata.Hive.getDatabasesByPattern(Hive.java:1139)
at
org.apache.hadoop.hive.ql.exec.DDLTask.showDatabases(DDLTask.java:2445)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:364)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1554)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1321)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1139)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:957)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:145)
at
[jira] [Updated] (HIVE-10268) Merge cbo branch into trunk
[ https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-10268: -- Labels: TODOC1.2 (was: ) Merge cbo branch into trunk --- Key: HIVE-10268 URL: https://issues.apache.org/jira/browse/HIVE-10268 Project: Hive Issue Type: Task Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch, HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.5.patch, HIVE-10268.6.patch, HIVE-10268.patch Merge patch generated on basis of diffs of trunk with cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10347) Merge spark to trunk 4/15/2015
[
https://issues.apache.org/jira/browse/HIVE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504183#comment-14504183
]
Hive QA commented on HIVE-10347:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726675/HIVE-10347.5.patch
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 8761 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union20
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_cbo_simple_select
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union_remove_6_subq
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3506/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3506/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3506/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726675 - PreCommit-HIVE-TRUNK-Build
Merge spark to trunk 4/15/2015
--
Key: HIVE-10347
URL: https://issues.apache.org/jira/browse/HIVE-10347
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-10347.2.patch, HIVE-10347.2.patch,
HIVE-10347.3.patch, HIVE-10347.4.patch, HIVE-10347.5.patch,
HIVE-10347.5.patch, HIVE-10347.patch
CLEAR LIBRARY CACHE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10396) decimal_precision2.q test is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504366#comment-14504366 ] Ashutosh Chauhan commented on HIVE-10396: - +1 decimal_precision2.q test is failing on trunk - Key: HIVE-10396 URL: https://issues.apache.org/jira/browse/HIVE-10396 Project: Hive Issue Type: Test Components: Types Affects Versions: 1.2.0 Reporter: Ashutosh Chauhan Assignee: Jason Dere Attachments: HIVE-10396.1.patch seems like missing golden file update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10268) Merge cbo branch into trunk
[ https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504192#comment-14504192 ] Lefty Leverenz commented on HIVE-10268: --- Doc note: Two issues in the CBO branch added 8 configuration parameters to this merge (HIVE-9581 and HIVE-10343) so I've added TODOC1.2 labels to them as well as here. What documentation does this need besides those 8 new configs? Merge cbo branch into trunk --- Key: HIVE-10268 URL: https://issues.apache.org/jira/browse/HIVE-10268 Project: Hive Issue Type: Task Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch, HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.5.patch, HIVE-10268.6.patch, HIVE-10268.patch Merge patch generated on basis of diffs of trunk with cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10383) Add tests for partition level statistics + refactor stats tests of TestHBaseStore [hbase-metastore branch]
[ https://issues.apache.org/jira/browse/HIVE-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504146#comment-14504146 ] Alan Gates commented on HIVE-10383: --- +1, assuming all of the HBase metastore unit and iunit tests pass. Add tests for partition level statistics + refactor stats tests of TestHBaseStore [hbase-metastore branch] -- Key: HIVE-10383 URL: https://issues.apache.org/jira/browse/HIVE-10383 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Attachments: HIVE-10383.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9272) Tests for utf-8 support
[ https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aswathy Chellammal Sreekumar updated HIVE-9272: --- Attachment: HIVE-9272.7.patch Tests for utf-8 support --- Key: HIVE-9272 URL: https://issues.apache.org/jira/browse/HIVE-9272 Project: Hive Issue Type: Test Components: Tests, WebHCat Affects Versions: 0.14.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Fix For: 1.2.0 Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.3.patch, HIVE-9272.4.patch, HIVE-9272.5.patch, HIVE-9272.6.patch, HIVE-9272.7.patch, HIVE-9272.patch Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9272) Tests for utf-8 support
[ https://issues.apache.org/jira/browse/HIVE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504161#comment-14504161 ] Aswathy Chellammal Sreekumar commented on HIVE-9272: [~ekoifman] Please find attached the patch with the above suggestions included. Tests for utf-8 support --- Key: HIVE-9272 URL: https://issues.apache.org/jira/browse/HIVE-9272 Project: Hive Issue Type: Test Components: Tests, WebHCat Affects Versions: 0.14.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Aswathy Chellammal Sreekumar Priority: Minor Fix For: 1.2.0 Attachments: HIVE-9272.1.patch, HIVE-9272.2.patch, HIVE-9272.3.patch, HIVE-9272.4.patch, HIVE-9272.5.patch, HIVE-9272.6.patch, HIVE-9272.patch Including some test cases for utf8 support in webhcat. The first four tests invoke hive, pig, mapred and streaming apis for testing the utf8 support for data processed, file names and job name. The last test case tests the filtering of job name with utf8 character -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-9913) LLAP: Avoid fetching data multiple times in case of broadcast
[ https://issues.apache.org/jira/browse/HIVE-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth resolved HIVE-9913. -- Resolution: Fixed LLAP: Avoid fetching data multiple times in case of broadcast - Key: HIVE-9913 URL: https://issues.apache.org/jira/browse/HIVE-9913 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Siddharth Seth Fix For: llap Attachments: HIVE-9913.1.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10396) decimal_precision2.q test is failing on trunk
[
https://issues.apache.org/jira/browse/HIVE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504352#comment-14504352
]
Hive QA commented on HIVE-10396:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726670/HIVE-10396.1.patch
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 8708 tests
executed
*Failed tests:*
{noformat}
TestCompareCliDriver - did not produce a TEST-*.xml file
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-parallel_join1.q-ptf_general_queries.q-avro_joins.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3507/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3507/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3507/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726670 - PreCommit-HIVE-TRUNK-Build
decimal_precision2.q test is failing on trunk
-
Key: HIVE-10396
URL: https://issues.apache.org/jira/browse/HIVE-10396
Project: Hive
Issue Type: Test
Components: Types
Affects Versions: 1.2.0
Reporter: Ashutosh Chauhan
Assignee: Jason Dere
Attachments: HIVE-10396.1.patch
seems like missing golden file update.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-9581) CBO (Calcite Return Path): Translate Join to Hive Op [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504182#comment-14504182 ] Lefty Leverenz edited comment on HIVE-9581 at 4/21/15 2:38 AM: --- HIVE-10268 merged the CBO branch to trunk, so I added a TODOC1.2 label to this issue. was (Author: le...@hortonworks.com): HIVE-10268 merged the CBO branch to trunk, so I added a TODOC-1.2 label to this issue. CBO (Calcite Return Path): Translate Join to Hive Op [CBO branch] - Key: HIVE-9581 URL: https://issues.apache.org/jira/browse/HIVE-9581 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: TODOC-CBO, TODOC1.2 Fix For: cbo-branch Attachments: HIVE-9581.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10268) Merge cbo branch into trunk
[ https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504204#comment-14504204 ] Laljo John Pullokkaran commented on HIVE-10268: --- [~leftylev] These are advanced configuration and only used when extended cost model is turned on. I will put in a write up about these parameters and how to set them. Merge cbo branch into trunk --- Key: HIVE-10268 URL: https://issues.apache.org/jira/browse/HIVE-10268 Project: Hive Issue Type: Task Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch, HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.5.patch, HIVE-10268.6.patch, HIVE-10268.patch Merge patch generated on basis of diffs of trunk with cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9913) LLAP: Avoid fetching data multiple times in case of broadcast
[ https://issues.apache.org/jira/browse/HIVE-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-9913: - Attachment: HIVE-9913.1.txt Patch delays the start to when the Input is actually used for Unordered cases (broadcast and non-broadcast for now), which is soon after the Processor starts running. LLAP: Avoid fetching data multiple times in case of broadcast - Key: HIVE-9913 URL: https://issues.apache.org/jira/browse/HIVE-9913 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Siddharth Seth Fix For: llap Attachments: HIVE-9913.1.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[
https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504313#comment-14504313
]
Sushanth Sowmyan commented on HIVE-10227:
-
Hm, that's definitely food for thought and it does change my mind about how I
think it ought to work.
a) I do agree now that it should not reset to null, but for a slightly
different reason - if there's no hope for success, there's no point in
attempting to re-instantiate it every time - it's unlikely that the HiveConf
would have changed. So, nulling it out, only to set it again and fail again is
pointless.
b) I do still think it should absolutely error every time a user attempts to
call ReplicationTask.create from then on, since we should not have any case of
silent successes here via NoopFactory being instantiated, since this could
cause a tool like Falcon to assume that it has successfully processed that
eventid, and move its pointer forward, thereby missing the event altogether in
the future.
However, this might mean that the optimal route here is going to go further in
the direction you warn against, but that will only happen in cases where the
tool calling ReplicationTask is misbehaving badly. Thoughts?
To wit, this is the change I now think appropriate:
{code}
diff --git
a/hcatalog/webhcat/java-client/src/main/java/org/apache/hive/hcatalog/api/repl/ReplicationTask.java
b/hcatalog/webhcat/java-client/src/main/java/org/apache/hive/hcatalog/api/repl/ReplicationTask.ja
index e73cc0c..811eeb8 100644
---
a/hcatalog/webhcat/java-client/src/main/java/org/apache/hive/hcatalog/api/repl/ReplicationTask.java
+++
b/hcatalog/webhcat/java-client/src/main/java/org/apache/hive/hcatalog/api/repl/ReplicationTask.java
@@ -24,6 +24,7 @@
import org.apache.hive.hcatalog.api.HCatNotificationEvent;
import org.apache.hive.hcatalog.common.HCatConstants;
import org.apache.hive.hcatalog.messaging.MessageFactory;
+import sun.plugin.dom.exception.InvalidStateException;
/**
@@ -81,6 +82,20 @@ public ReplicationTask create(HCatClient client,
HCatNotificationEvent event) {
}
}
+ /**
+ * Dummy factory to indicate that ReplicationTask is in an invalid state,
+ * possibly due to an incorrect Factory having been configured. To reset
+ * use, please call resetFactory on a valid ReplicationTask.Factory impl.
+ */
+ public static class InvalidStateFactory implements Factory {
+@Override
+public ReplicationTask create(HCatClient client, HCatNotificationEvent
event) {
+ throw new InvalidStateException(Error instantiating
ReplicationTask.Factory +
+
HiveConf.ConfVars.HIVE_REPL_TASK_FACTORY.varname+=+factoryClassName +
+ . Call resetFactory() if you need to reset to a valid one.);
+}
+ }
+
private static Factory getFactoryInstance(HCatClient client) {
if (factoryInstance == null){
createFactoryInstance(client);
@@ -112,7 +127,7 @@ private synchronized static void
createFactoryInstance(HCatClient client) {
Class? extends Factory factoryClass = (Class? extends Factory)
Class.forName(factoryClassName);
factoryInstance = factoryClass.newInstance();
} catch (Exception e) {
- factoryClassName = null; // reset the classname for future
evaluations.
+ factoryInstance = new InvalidStateFactory();
throw new RuntimeException(Error instantiating
ReplicationTask.Factory +
HiveConf.ConfVars.HIVE_REPL_TASK_FACTORY.varname+=+factoryClassName);
}
{code}
Concrete implementation of Export/Import based ReplicationTaskFactory
-
Key: HIVE-10227
URL: https://issues.apache.org/jira/browse/HIVE-10227
Project: Hive
Issue Type: Sub-task
Components: Import/Export
Affects Versions: 1.2.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-10227.2.patch, HIVE-10227.3.patch,
HIVE-10227.4.patch, HIVE-10227.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10239) Create scripts to do metastore upgrade tests on jenkins for Derby, Oracle and PostgreSQL
[
https://issues.apache.org/jira/browse/HIVE-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503539#comment-14503539
]
Sergio Peña commented on HIVE-10239:
[~ngangam] The HMS failed.
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/36/console
This time it has to do with the GPG repo key:
{noformat}
W: GPG error: http://oss.oracle.com unstable Release: The following signatures
were invalid: KEYEXPIRED 1378511808 KEYEXPIRED 1378511808 KEYEXPIRED 1378511808
W: Failed to fetch
http://oss.oracle.com/debian/dists/unstable/main/binary-amd64/Packages
HttpError404
Reading package lists...W: Failed to fetch
http://oss.oracle.com/debian/dists/unstable/non-free/binary-amd64/Packages
HttpError404
E: Some index files failed to download. They have been ignored, or old ones
used instead.
+ /bin/true
+ apt-get install -y --force-yes oracle-xe
Building dependency tree...
Reading state information...
{noformat}
Regarding the results not posted here yet, it has to do with a trap command
from some of the jenkins scripts that is exiting when an error occurs without
publishing the results. I don't know where that is yet, but I will find it.
Create scripts to do metastore upgrade tests on jenkins for Derby, Oracle and
PostgreSQL
Key: HIVE-10239
URL: https://issues.apache.org/jira/browse/HIVE-10239
Project: Hive
Issue Type: Improvement
Affects Versions: 1.1.0
Reporter: Naveen Gangam
Assignee: Naveen Gangam
Attachments: HIVE-10239-donotcommit.patch, HIVE-10239.0.patch,
HIVE-10239.0.patch, HIVE-10239.00.patch, HIVE-10239.patch
Need to create DB-implementation specific scripts to use the framework
introduced in HIVE-9800 to have any metastore schema changes tested across
all supported databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10405) LLAP: Provide runtime information to daemons to decide on preemption order
[ https://issues.apache.org/jira/browse/HIVE-10405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-10405: -- Attachment: HIVE-10405.1.txt The following information is sent into daemons at fragment submission time - start time of the dag - start time of the first attempt of a specific fragment - The priority of a fragment within an executing dag - determined by the topological order in the DAG (this is irrelevant across DAGs) - number of tasks in the current vertex + upstream to the current vertex - number of completed tasks in the current vertex + upstream to the current vertex. LLAP: Provide runtime information to daemons to decide on preemption order -- Key: HIVE-10405 URL: https://issues.apache.org/jira/browse/HIVE-10405 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Siddharth Seth Fix For: llap Attachments: HIVE-10405.1.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-10397) LLAP: Implement Tez SplitSizeEstimator for Orc
[ https://issues.apache.org/jira/browse/HIVE-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-10397. -- Resolution: Fixed Committed to llap branch. LLAP: Implement Tez SplitSizeEstimator for Orc -- Key: HIVE-10397 URL: https://issues.apache.org/jira/browse/HIVE-10397 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10397.patch This is patch for HIVE-7428. For now this will be in llap branch as hive has not bumped up the tez version yet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10396) decimal_precision2.q test is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502474#comment-14502474 ] Jason Dere commented on HIVE-10396: --- Looks like HIVE-10313, which was committed on the same day as HIVE-10207, changed the behavior of decimal constants and caused the golden file from HIVE-10207 to fail. Yeah we just need to regenerate the golden file. decimal_precision2.q test is failing on trunk - Key: HIVE-10396 URL: https://issues.apache.org/jira/browse/HIVE-10396 Project: Hive Issue Type: Test Components: Types Affects Versions: 1.2.0 Reporter: Ashutosh Chauhan seems like missing golden file update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10235) Loop optimization for SIMD in ColumnDivideColumn.txt
[
https://issues.apache.org/jira/browse/HIVE-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502543#comment-14502543
]
Hive QA commented on HIVE-10235:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726525/HIVE-10235.1.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8727 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3498/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3498/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3498/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726525 - PreCommit-HIVE-TRUNK-Build
Loop optimization for SIMD in ColumnDivideColumn.txt
Key: HIVE-10235
URL: https://issues.apache.org/jira/browse/HIVE-10235
Project: Hive
Issue Type: Sub-task
Components: Vectorization
Affects Versions: 1.1.0
Reporter: Chengxiang Li
Assignee: Chengxiang Li
Priority: Minor
Attachments: HIVE-10235.1.patch, HIVE-10235.1.patch
Found two loop which could be optimized for packed instruction set during
execution.
1. hasDivBy0 depends on the result of last loop, which prevent the loop be
executed vectorized.
{code:java}
for(int i = 0; i != n; i++) {
OperandType2 denom = vector2[i];
outputVector[i] = vector1[0] OperatorSymbol denom;
hasDivBy0 = hasDivBy0 || (denom == 0);
}
{code}
2. same as HIVE-10180, vector2\[0\] reference provent JVM optimizing loop
into packed instruction set.
{code:java}
for(int i = 0; i != n; i++) {
outputVector[i] = vector1[i] OperatorSymbol vector2[0];
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10397) LLAP: Implement Tez SplitSizeEstimator for Orc
[ https://issues.apache.org/jira/browse/HIVE-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502456#comment-14502456 ] Prasanth Jayachandran commented on HIVE-10397: -- We should port this back to trunk later when hive moves to new version of tez which has SplitSizeEstimator interface. LLAP: Implement Tez SplitSizeEstimator for Orc -- Key: HIVE-10397 URL: https://issues.apache.org/jira/browse/HIVE-10397 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10397.patch This is patch for HIVE-7428. For now this will be in llap branch as hive has not bumped up the tez version yet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-10396) decimal_precision2.q test is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-10396: - Assignee: Jason Dere decimal_precision2.q test is failing on trunk - Key: HIVE-10396 URL: https://issues.apache.org/jira/browse/HIVE-10396 Project: Hive Issue Type: Test Components: Types Affects Versions: 1.2.0 Reporter: Ashutosh Chauhan Assignee: Jason Dere seems like missing golden file update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9917) After HIVE-3454 is done, make int to timestamp conversion configurable
[ https://issues.apache.org/jira/browse/HIVE-9917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502858#comment-14502858 ] Aihua Xu commented on HIVE-9917: [~jdere] You are talking about vector_between_in.q.out, right? Yeah, I notice that too. I think it's caused by your previous fix in HIVE-10178. After HIVE-3454 is done, make int to timestamp conversion configurable -- Key: HIVE-9917 URL: https://issues.apache.org/jira/browse/HIVE-9917 Project: Hive Issue Type: Improvement Reporter: Aihua Xu Assignee: Aihua Xu Attachments: HIVE-9917.patch After HIVE-3454 is fixed, we will have correct behavior of converting int to timestamp. While the customers are using such incorrect behavior for so long, better to make it configurable so that in one release, it will default to old/inconsistent way and the next release will default to new/consistent way. And then we will deprecate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10312) SASL.QOP in JDBC URL is ignored for Delegation token Authentication
[ https://issues.apache.org/jira/browse/HIVE-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502944#comment-14502944 ] Mubashir Kazia commented on HIVE-10312: --- [~aihuaxu] If you are going to use the Hive JDBC driver from any framework that relies on Delegation tokens to authenticate to HS2 that has SASL.QOP to set to anything other than auth you'll have this issue. In my case it just happened to be oozie. Yes the driver would work the same with or without oozie. I have not seen any hive docs that says that QOP should not be set for DIGEST authentication mechanism. AFAIK Java supports QOP on DIGEST authentication mechanism, see http://docs.oracle.com/javase/jndi/tutorial/ldap/security/sasl.html. SASL.QOP in JDBC URL is ignored for Delegation token Authentication --- Key: HIVE-10312 URL: https://issues.apache.org/jira/browse/HIVE-10312 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 1.2.0 Reporter: Mubashir Kazia Fix For: 1.2.0 Attachments: HIVE-10312.1.patch When HS2 is configured for QOP other than auth (auth-int or auth-conf), Kerberos client connection works fine when the JDBC URL specifies the matching QOP, however when this HS2 is accessed through Oozie (Delegation token / Digest authentication), connections fails because the JDBC driver ignores the SASL.QOP parameters in the JDBC URL. SASL.QOP setting should be valid for DIGEST Auth mech. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9917) After HIVE-3454 is done, make int to timestamp conversion configurable
[ https://issues.apache.org/jira/browse/HIVE-9917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502860#comment-14502860 ] Aihua Xu commented on HIVE-9917: Sorry. Alex Pivovarov's fix. :) After HIVE-3454 is done, make int to timestamp conversion configurable -- Key: HIVE-9917 URL: https://issues.apache.org/jira/browse/HIVE-9917 Project: Hive Issue Type: Improvement Reporter: Aihua Xu Assignee: Aihua Xu Attachments: HIVE-9917.patch After HIVE-3454 is fixed, we will have correct behavior of converting int to timestamp. While the customers are using such incorrect behavior for so long, better to make it configurable so that in one release, it will default to old/inconsistent way and the next release will default to new/consistent way. And then we will deprecate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10312) SASL.QOP in JDBC URL is ignored for Delegation token Authentication
[ https://issues.apache.org/jira/browse/HIVE-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502923#comment-14502923 ] Aihua Xu commented on HIVE-10312: - [~mkazia] Regarding JDBC driver ignores the SASL.QOP, is this a known issue or a limitation of the driver? Would the driver work the same with/without oozie? Can you show any doc/reference on that? SASL.QOP in JDBC URL is ignored for Delegation token Authentication --- Key: HIVE-10312 URL: https://issues.apache.org/jira/browse/HIVE-10312 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 1.2.0 Reporter: Mubashir Kazia Fix For: 1.2.0 Attachments: HIVE-10312.1.patch When HS2 is configured for QOP other than auth (auth-int or auth-conf), Kerberos client connection works fine when the JDBC URL specifies the matching QOP, however when this HS2 is accessed through Oozie (Delegation token / Digest authentication), connections fails because the JDBC driver ignores the SASL.QOP parameters in the JDBC URL. SASL.QOP setting should be valid for DIGEST Auth mech. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-10401) splitCondition does not behave correctly when one side of the condition references columns from different inputs
[ https://issues.apache.org/jira/browse/HIVE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved HIVE-10401. Resolution: Invalid splitCondition does not behave correctly when one side of the condition references columns from different inputs Key: HIVE-10401 URL: https://issues.apache.org/jira/browse/HIVE-10401 Project: Hive Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502963#comment-14502963
]
Jesus Camacho Rodriguez commented on HIVE-10400:
[~jpullokkaran], can you take a look? Thanks
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[ https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503259#comment-14503259 ] Sushanth Sowmyan commented on HIVE-10227: - (And sorry, unable to create rb link for this - rb has been very slow last week, and availability has been spotty. Today, it's up, but does not like my patches until I generate with --full-index, and after I generate with --full-index, says it can't find the right hash to apply) Mithun was able to create a rb link for the .2.patch if you want to have a look at that : https://reviews.apache.org/r/7/ Concrete implementation of Export/Import based ReplicationTaskFactory - Key: HIVE-10227 URL: https://issues.apache.org/jira/browse/HIVE-10227 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 1.2.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-10227.2.patch, HIVE-10227.3.patch, HIVE-10227.4.patch, HIVE-10227.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10084) Improve common join performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503301#comment-14503301
]
Hive QA commented on HIVE-10084:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726596/HIVE-10084.1-spark.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8718 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucket6.q-scriptfile1_win.q-quotedid_smb.q-and-1-more - did
not produce a TEST-*.xml file
TestMinimrCliDriver-bucketizedhiveinputformat.q-empty_dir_in_table.q - did not
produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-infer_bucket_sort_map_operators.q-load_hdfs_file_with_space_in_the_name.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-import_exported_table.q-truncate_column_buckets.q-bucket_num_reducers2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-infer_bucket_sort_num_buckets.q-parallel_orderby.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-join1.q-infer_bucket_sort_bucketed_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-bucket5.q-infer_bucket_sort_merge.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-input16_cc.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-bucket_num_reducers.q-scriptfile1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx_cbo_2.q-bucketmapjoin6.q-bucket4.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-reduce_deduplicate.q-infer_bucket_sort_dyn_part.q-udf_using.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-uber_reduce.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-stats_counter_partitioned.q-external_table_with_space_in_location_path.q-disable_merge_for_bucketing.q-and-1-more
- did not produce a TEST-*.xml file
TestPigHBaseStorageHandler - did not produce a TEST-*.xml file
org.apache.hive.jdbc.TestSSL.testSSLConnectionWithProperty
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/831/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/831/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-831/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726596 - PreCommit-HIVE-SPARK-Build
Improve common join performance [Spark Branch]
--
Key: HIVE-10084
URL: https://issues.apache.org/jira/browse/HIVE-10084
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Xuefu Zhang
Attachments: HIVE-10084.1-spark.patch, HIVE-10084.1-spark.patch
Benchmark shows that Hive on Spark shows some numbers which indicate that
common join performance can be improved. This task is to investigate and fix
the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10312) SASL.QOP in JDBC URL is ignored for Delegation token Authentication
[ https://issues.apache.org/jira/browse/HIVE-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503241#comment-14503241 ] Lefty Leverenz commented on HIVE-10312: --- The Hive wiki only mentions QOP in two places: * [Setting Up HiveServer2 -- Integrity/Confidentiality Protection | https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-Integrity/ConfidentialityProtection] * [Configuration Properties -- hive.server2.thrift.sasl.qop | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.server2.thrift.sasl.qop] SASL.QOP in JDBC URL is ignored for Delegation token Authentication --- Key: HIVE-10312 URL: https://issues.apache.org/jira/browse/HIVE-10312 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 1.2.0 Reporter: Mubashir Kazia Fix For: 1.2.0 Attachments: HIVE-10312.1.patch When HS2 is configured for QOP other than auth (auth-int or auth-conf), Kerberos client connection works fine when the JDBC URL specifies the matching QOP, however when this HS2 is accessed through Oozie (Delegation token / Digest authentication), connections fails because the JDBC driver ignores the SASL.QOP parameters in the JDBC URL. SASL.QOP setting should be valid for DIGEST Auth mech. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503189#comment-14503189
]
Pengcheng Xiong commented on HIVE-10400:
Tested this with cbo_limit.q. Test passed.
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503216#comment-14503216
]
Jesus Camacho Rodriguez commented on HIVE-10400:
Awesome!
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[
https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503228#comment-14503228
]
Sushanth Sowmyan commented on HIVE-10227:
-
Hi Alan,
ReplicationTask.java:
{quote}
line 100: Instantiating new HiveConf objects is expensive. It also runs the
risk of ignoring conf values set elsewhere in the code. Is there not an
existing version of the conf that can be used here?
{quote}
This has been fixed already per feedback from Mithun, and is updated in patches
2 3.
{quote}
line 107: Why do you want to set factory class name to null? If someone
configures a bad class name this will result in an error message every time
this method is invoked, which appears to be everytime a caller attempt to read
the next replication task.
{quote}
Going by version 1 of this patch for line 107 (although the fundamental logic
hasn't changed for this bit across the versions, there are 2 ways for people to
configure/fix a factory configured on this:
* Through the HiveConf set on HCatClient (in the later versions - it was a
statically instantiated HiveConf in the first version you looked at)
* Through resetFactory(Class? extends Factory) (which is intended for
testing purposes)
If we cannot instantiate a factory, that is a fatal error for ReplicationTask,
and we must not allow it to continue until it's fixed. Thus, it is appropriate
for this to error out on each call. That said, the behaviour of erroring out on
each call will hold only in the case of it being set through HiveConf, since
the next time it runs, it again picks up the factoryClassName from HiveConf
(and thus, the nulling basically allows us to refetch the value to check in
case there has been any update to that). In the case of resetFactory-based
instantiation, nulling it out so that it goes to NoopFactory is an appropriate
fallback - that said, resetFactory is also more resilient, in that it will not
have a case of a bad classname, since it takes in a Class? extends Factory
itself(it can still error out in default construction, but that's a runtime
fault, in which case the failovers are appropriate).
DropDatabaseCommand.java and other ? extends HiveCommand methods:
{quote} line 43: These comments would be better in javadoc {quote}
Agreed, doing a trivial rebase to generate a .4.patch.
HiveCommand:
This class has been removed as a result of feedback as of .3.patch. HiveCommand
was an experiment, to push usage of .run() on Commands, rather than trying to
do a get() to get SQL commands. In that scenario, if we had other
ReplicationTaskFactory implementations that were returning their own Commands
which allowed Hive-MySQL replication, for instance, their get() might return
SQL commands intended for MySQL, for eg. But trying that was still premature,
and since HCatClient did not support import and export, that problem was not
being solved well either. This class and all references to it have been removed.
Concrete implementation of Export/Import based ReplicationTaskFactory
-
Key: HIVE-10227
URL: https://issues.apache.org/jira/browse/HIVE-10227
Project: Hive
Issue Type: Sub-task
Components: Import/Export
Affects Versions: 1.2.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-10227.2.patch, HIVE-10227.3.patch, HIVE-10227.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503227#comment-14503227
]
Hive QA commented on HIVE-10400:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726578/HIVE-10400.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 8727 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.testPigPopulation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3499/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3499/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3499/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726578 - PreCommit-HIVE-TRUNK-Build
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503185#comment-14503185
]
Jesus Camacho Rodriguez commented on HIVE-10400:
[~ashutoshc], this is the plan that we get from Calcite:
{noformat}
HiveSort(sort0=[$3], sort1=[$1], dir0=[DESC], dir1=[ASC])
HiveProject(c_int=[$0], c=[$1], _o__c2=[$2], _o__col3=[+(CAST($0):BIGINT,
$1)])
HiveAggregate(group=[{0, 1}], agg#0=[count()])
HiveProject($f0=[$6], $f1=[$2])
HiveFilter(condition=[OR(($1, 0), =($6, 0))])
HiveProject(a=[$2], b=[$3], c=[$4], p=[$5], q=[$6], key=[$0],
c_int=[$1])
HiveJoin(condition=[=($2, $0)], joinType=[inner], algorithm=[none],
cost=[not available])
HiveSortExchange(distribution=[org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelDistribution@4b21d04b],
collation=[[0]])
HiveProject(key=[$0], c_int=[$2])
HiveFilter(condition=[isnotnull($0)])
HiveTableScan(table=[[default.cbo_t3]], qbid:alias=[cbo_t3])
HiveSortExchange(distribution=[org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelDistribution@18f21276],
collation=[[0]])
HiveFilter(condition=[=(+($1, $4), 0)])
HiveProject(a=[$2], b=[$3], c=[$4], p=[$0], q=[$1])
HiveJoin(condition=[=($2, $0)], joinType=[inner],
algorithm=[none], cost=[not available])
HiveSortExchange(distribution=[org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelDistribution@4aa77ce6],
collation=[[0]])
HiveProject(p=[$0], q=[$1])
HiveFilter(condition=[isnotnull($0)])
HiveSort(sort0=[$3], sort1=[$2], dir0=[DESC],
dir1=[ASC])
HiveProject(p=[$2], q=[+($1, 1)], r=[$3], (/
(tok_table_or_col q) 10)=[/(CAST(+($1, 1)):DOUBLE, CAST(10):DOUBLE)])
HiveAggregate(group=[{0, 1, 2}],
agg#0=[sum($1)])
HiveProject($f0=[$3], $f1=[$2], $f2=[$0])
HiveFilter(condition=[AND(=(+($2, 1), 0),
OR(($2, 0), =($3, 0E0)))])
HiveTableScan(table=[[default.cbo_t2]],
qbid:alias=[cbo_t2:cbo_t2])
HiveSortExchange(distribution=[org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelDistribution@39f90654],
collation=[[0]])
HiveFilter(condition=[isnotnull($0)])
HiveSort(sort0=[$0], dir0=[ASC])
HiveProject(a=[$2], b=[+($1, 1)], c=[$3])
HiveAggregate(group=[{0, 1, 2}], agg#0=[sum($1)])
HiveProject($f0=[$3], $f1=[$2], $f2=[$0])
HiveFilter(condition=[AND(=(+($2, 1), 0),
OR(($2, 0), =($3, 0E0)))])
HiveTableScan(table=[[default.cbo_t1]],
qbid:alias=[cbo_t1:cbo_t1])
{noformat}
It seems we miss a projection on top. What do you think?
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10397) LLAP: Implement Tez SplitSizeEstimator for Orc
[ https://issues.apache.org/jira/browse/HIVE-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503033#comment-14503033 ] Gopal V commented on HIVE-10397: [~prasanth_j]: this is fine for LLAP branch, but this patch might break hadoop-1 builds. Because of the inheritance dependencies for Tez, you cannot make the InputFormat implement SplitSizeEstimator in trunk. The original idea to do this via an extra object (because java has no mixins) was to implement OrcSplit - ColumnarSplit + instantiate a generic ColumnarSplitEstimator only inside Tez codepath would do the trick. LLAP: Implement Tez SplitSizeEstimator for Orc -- Key: HIVE-10397 URL: https://issues.apache.org/jira/browse/HIVE-10397 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10397.patch This is patch for HIVE-7428. For now this will be in llap branch as hive has not bumped up the tez version yet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[ https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-10227: Attachment: HIVE-10227.4.patch Trivial update - moved some comments to javadoc per Alan's feedback. Concrete implementation of Export/Import based ReplicationTaskFactory - Key: HIVE-10227 URL: https://issues.apache.org/jira/browse/HIVE-10227 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 1.2.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-10227.2.patch, HIVE-10227.3.patch, HIVE-10227.4.patch, HIVE-10227.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503094#comment-14503094
]
Jesus Camacho Rodriguez commented on HIVE-10400:
Let me take a look. Thanks
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10382) Aggregate stats cache for RDBMS based metastore codepath
[ https://issues.apache.org/jira/browse/HIVE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-10382: Attachment: HIVE-10382.1.patch Aggregate stats cache for RDBMS based metastore codepath Key: HIVE-10382 URL: https://issues.apache.org/jira/browse/HIVE-10382 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 1.2.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 1.2.0 Attachments: HIVE-10382.1.patch Similar to the work done on the HBase branch (HIVE-9693), the stats cache can potentially have performance gains. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9711) ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN
[ https://issues.apache.org/jira/browse/HIVE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-9711: -- Attachment: HIVE-9711.3.patch ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN - Key: HIVE-9711 URL: https://issues.apache.org/jira/browse/HIVE-9711 Project: Hive Issue Type: Bug Components: File Formats, Vectorization Affects Versions: 1.2.0 Reporter: Gopal V Assignee: Gopal V Fix For: 1.2.0 Attachments: HIVE-9711.1.patch, HIVE-9711.2.patch, HIVE-9711.3.patch The isRepeating=true check uses Java equality, which results in NaN != NaN comparison operations. The noNulls case needs the current check folded into the previous loop, while the hasNulls case needs a logical AND of the isNull[] field instead of == comparisons. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[ https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503096#comment-14503096 ] Alan Gates commented on HIVE-10227: --- * ReplicationTask.java ** line 100: Instantiating new HiveConf objects is expensive. It also runs the risk of ignoring conf values set elsewhere in the code. Is there not an existing version of the conf that can be used here? ** line 107: Why do you want to set factory class name to null? If someone configures a bad class name this will result in an error message every time this method is invoked, which appears to be everytime a caller attempt to read the next replication task. * DropDatabaseCommand.java and other ? extends HiveCommand methods ** line 43: These comments would be better in javadoc * HiveCommand ** comments about methods should be in javadoc ** Is there any reason this should be separate from the Command interface. It says in the comments the point of this interface is to indicate that the get command returns SQL to be executed. What else could get return? Concrete implementation of Export/Import based ReplicationTaskFactory - Key: HIVE-10227 URL: https://issues.apache.org/jira/browse/HIVE-10227 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 1.2.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-10227.2.patch, HIVE-10227.3.patch, HIVE-10227.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9711) ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN
[ https://issues.apache.org/jira/browse/HIVE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-9711: -- Attachment: HIVE-9711.3.patch ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN - Key: HIVE-9711 URL: https://issues.apache.org/jira/browse/HIVE-9711 Project: Hive Issue Type: Bug Components: File Formats, Vectorization Affects Versions: 1.2.0 Reporter: Gopal V Assignee: Gopal V Fix For: 1.2.0 Attachments: HIVE-9711.1.patch, HIVE-9711.2.patch, HIVE-9711.3.patch The isRepeating=true check uses Java equality, which results in NaN != NaN comparison operations. The noNulls case needs the current check folded into the previous loop, while the hasNulls case needs a logical AND of the isNull[] field instead of == comparisons. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9711) ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN
[ https://issues.apache.org/jira/browse/HIVE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-9711: -- Attachment: (was: HIVE-9711.3.patch) ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN - Key: HIVE-9711 URL: https://issues.apache.org/jira/browse/HIVE-9711 Project: Hive Issue Type: Bug Components: File Formats, Vectorization Affects Versions: 1.2.0 Reporter: Gopal V Assignee: Gopal V Fix For: 1.2.0 Attachments: HIVE-9711.1.patch, HIVE-9711.2.patch, HIVE-9711.3.patch The isRepeating=true check uses Java equality, which results in NaN != NaN comparison operations. The noNulls case needs the current check folded into the previous loop, while the hasNulls case needs a logical AND of the isNull[] field instead of == comparisons. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503085#comment-14503085
]
Ashutosh Chauhan commented on HIVE-10400:
-
Testing this on cbo_gby.q, I was able to get past the problem of incorrect name
in OI, but result set has an extra column which looks wrong :
{noformat}
-1 12 6
-1 2 6
+1 12 6 13
+1 2 6 3
{noformat}
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10235) Loop optimization for SIMD in ColumnDivideColumn.txt
[
https://issues.apache.org/jira/browse/HIVE-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503106#comment-14503106
]
Gopal V commented on HIVE-10235:
[~chengxiang li]: Adding to tomorrow's build, will let you know.
Loop optimization for SIMD in ColumnDivideColumn.txt
Key: HIVE-10235
URL: https://issues.apache.org/jira/browse/HIVE-10235
Project: Hive
Issue Type: Sub-task
Components: Vectorization
Affects Versions: 1.1.0
Reporter: Chengxiang Li
Assignee: Chengxiang Li
Priority: Minor
Attachments: HIVE-10235.1.patch, HIVE-10235.1.patch
Found two loop which could be optimized for packed instruction set during
execution.
1. hasDivBy0 depends on the result of last loop, which prevent the loop be
executed vectorized.
{code:java}
for(int i = 0; i != n; i++) {
OperandType2 denom = vector2[i];
outputVector[i] = vector1[0] OperatorSymbol denom;
hasDivBy0 = hasDivBy0 || (denom == 0);
}
{code}
2. same as HIVE-10180, vector2\[0\] reference provent JVM optimizing loop
into packed instruction set.
{code:java}
for(int i = 0; i != n; i++) {
outputVector[i] = vector1[i] OperatorSymbol vector2[0];
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10084) Improve common join performance [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-10084: --- Attachment: HIVE-10084.1-spark.patch Improve common join performance [Spark Branch] -- Key: HIVE-10084 URL: https://issues.apache.org/jira/browse/HIVE-10084 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Xuefu Zhang Attachments: HIVE-10084.1-spark.patch, HIVE-10084.1-spark.patch Benchmark shows that Hive on Spark shows some numbers which indicate that common join performance can be improved. This task is to investigate and fix the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10084) Improve common join performance [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503124#comment-14503124 ] Rui Li commented on HIVE-10084: --- OOO and travelling abroad from 4/14 to 4/22. Please expect slow email response. Sorry for the inconvenience. Improve common join performance [Spark Branch] -- Key: HIVE-10084 URL: https://issues.apache.org/jira/browse/HIVE-10084 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Xuefu Zhang Attachments: HIVE-10084.1-spark.patch, HIVE-10084.1-spark.patch Benchmark shows that Hive on Spark shows some numbers which indicate that common join performance can be improved. This task is to investigate and fix the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[ https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503385#comment-14503385 ] Sushanth Sowmyan commented on HIVE-10227: - Also, as a note, none of the test failures noted in the run above are related to this patch, and the differences between .3.patch which was tested, and .4.patch are trivial and contain only comment updates. Concrete implementation of Export/Import based ReplicationTaskFactory - Key: HIVE-10227 URL: https://issues.apache.org/jira/browse/HIVE-10227 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 1.2.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-10227.2.patch, HIVE-10227.3.patch, HIVE-10227.4.patch, HIVE-10227.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10391) CBO (Calcite Return Path): HiveOpConverter always assumes that HiveFilter does not include a partition column
[ https://issues.apache.org/jira/browse/HIVE-10391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-10391: --- Assignee: Laljo John Pullokkaran (was: Pengcheng Xiong) CBO (Calcite Return Path): HiveOpConverter always assumes that HiveFilter does not include a partition column - Key: HIVE-10391 URL: https://issues.apache.org/jira/browse/HIVE-10391 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Pengcheng Xiong Assignee: Laljo John Pullokkaran Fix For: 1.2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10352) Investigate the failure of union_view.q
[ https://issues.apache.org/jira/browse/HIVE-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-10352: -- Attachment: HIVE-10352.2.patch Looks like an SVN issue messed up the test run. Investigate the failure of union_view.q Key: HIVE-10352 URL: https://issues.apache.org/jira/browse/HIVE-10352 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 1.2.0 Reporter: Aihua Xu Assignee: Vikram Dixit K Attachments: HIVE-10352.1.patch, HIVE-10352.2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10303) HIVE-9471 broke forward compatibility of ORC files
[ https://issues.apache.org/jira/browse/HIVE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503376#comment-14503376 ] Owen O'Malley commented on HIVE-10303: -- +1 LGTM HIVE-9471 broke forward compatibility of ORC files -- Key: HIVE-10303 URL: https://issues.apache.org/jira/browse/HIVE-10303 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 1.2.0 Reporter: Owen O'Malley Assignee: Prasanth Jayachandran Fix For: 1.2.0 Attachments: HIVE-10303.1.patch The change suppresses the streams in ORC files for ORC dictionaries with 0 entries. This causes NPE on ORC readers for all versions of Hive 0.11 to 1.1 and needs to be reverted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10394) LLAP: Notify AM of pre-emption
[ https://issues.apache.org/jira/browse/HIVE-10394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503373#comment-14503373 ] Siddharth Seth commented on HIVE-10394: --- The information isn't actually being sent across to the AM. What's handled right now is a response to the submitWork request. However, once a request moves onto the scheduler queue for execution at a later point - an RPC invocaiton will be required to inform the AM about the status of the task. This would be an addition to LlapTaskUmbilicalProtocol. LLAP: Notify AM of pre-emption -- Key: HIVE-10394 URL: https://issues.apache.org/jira/browse/HIVE-10394 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-10394.1.patch Pre-empted tasks should be notified to AM as killed/interrupted by system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10382) Aggregate stats cache for RDBMS based metastore codepath
[
https://issues.apache.org/jira/browse/HIVE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503374#comment-14503374
]
Hive QA commented on HIVE-10382:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726586/HIVE-10382.1.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8731 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3500/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3500/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3500/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726586 - PreCommit-HIVE-TRUNK-Build
Aggregate stats cache for RDBMS based metastore codepath
Key: HIVE-10382
URL: https://issues.apache.org/jira/browse/HIVE-10382
Project: Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 1.2.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 1.2.0
Attachments: HIVE-10382.1.patch
Similar to the work done on the HBase branch (HIVE-9693), the stats cache can
potentially have performance gains.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10400) CBO (Calcite Return Path): Exception when column name contains dot or colon characters
[
https://issues.apache.org/jira/browse/HIVE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503398#comment-14503398
]
Laljo John Pullokkaran commented on HIVE-10400:
---
+1
CBO (Calcite Return Path): Exception when column name contains dot or colon
characters
--
Key: HIVE-10400
URL: https://issues.apache.org/jira/browse/HIVE-10400
Project: Hive
Issue Type: Sub-task
Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.2.0
Attachments: HIVE-10400.patch
If return path is on, this query produces the problem:
{noformat}
select cbo_t3.c_int, c, count(*)
from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1
where (cbo_t1.c_int + 1 = 0) and (cbo_t1.c_int 0 or cbo_t1.c_float = 0)
group by c_float, cbo_t1.c_int, key order by a) cbo_t1
join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2
where (cbo_t2.c_int + 1 = 0) and (cbo_t2.c_int 0 or cbo_t2.c_float = 0)
group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on
cbo_t1.a=p
join cbo_t3 on cbo_t1.a=key
where (b + cbo_t2.q = 0) and (b 0 or c_int = 0)
group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10396) decimal_precision2.q test is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502394#comment-14502394 ] Ashutosh Chauhan commented on HIVE-10396: - FYI : [~jdere] decimal_precision2.q test is failing on trunk - Key: HIVE-10396 URL: https://issues.apache.org/jira/browse/HIVE-10396 Project: Hive Issue Type: Test Components: Types Affects Versions: 1.2.0 Reporter: Ashutosh Chauhan seems like missing golden file update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9711) ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN
[
https://issues.apache.org/jira/browse/HIVE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503505#comment-14503505
]
Hive QA commented on HIVE-9711:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726587/HIVE-9711.3.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8727 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3501/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3501/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3501/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726587 - PreCommit-HIVE-TRUNK-Build
ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN
-
Key: HIVE-9711
URL: https://issues.apache.org/jira/browse/HIVE-9711
Project: Hive
Issue Type: Bug
Components: File Formats, Vectorization
Affects Versions: 1.2.0
Reporter: Gopal V
Assignee: Gopal V
Fix For: 1.2.0
Attachments: HIVE-9711.1.patch, HIVE-9711.2.patch, HIVE-9711.3.patch
The isRepeating=true check uses Java equality, which results in NaN != NaN
comparison operations.
The noNulls case needs the current check folded into the previous loop, while
the hasNulls case needs a logical AND of the isNull[] field instead of ==
comparisons.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
