[jira] [Commented] (HIVE-10347) Merge spark to trunk 4/15/2015
[
https://issues.apache.org/jira/browse/HIVE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501521#comment-14501521
]
Hive QA commented on HIVE-10347:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726381/HIVE-10347.4.patch
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 8700 tests
executed
*Failed tests:*
{noformat}
TestCliDriver-udf_bitmap_empty.q-index_auto_update.q-quotedid_basic.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse
org.apache.hive.hcatalog.streaming.TestStreaming.testAddPartition
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3494/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3494/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3494/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726381 - PreCommit-HIVE-TRUNK-Build
Merge spark to trunk 4/15/2015
--
Key: HIVE-10347
URL: https://issues.apache.org/jira/browse/HIVE-10347
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-10347.2.patch, HIVE-10347.2.patch,
HIVE-10347.3.patch, HIVE-10347.4.patch, HIVE-10347.patch
CLEAR LIBRARY CACHE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10347) Merge spark to trunk 4/15/2015
[ https://issues.apache.org/jira/browse/HIVE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-10347: - Attachment: HIVE-10347.4.patch Merge spark to trunk 4/15/2015 -- Key: HIVE-10347 URL: https://issues.apache.org/jira/browse/HIVE-10347 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-10347.2.patch, HIVE-10347.2.patch, HIVE-10347.3.patch, HIVE-10347.4.patch, HIVE-10347.patch CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10268) Merge cbo branch into trunk
[ https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-10268: Attachment: HIVE-10268.6.patch Merge cbo branch into trunk --- Key: HIVE-10268 URL: https://issues.apache.org/jira/browse/HIVE-10268 Project: Hive Issue Type: Task Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch, HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.5.patch, HIVE-10268.6.patch, HIVE-10268.patch Merge patch generated on basis of diffs of trunk with cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10228) Changes to Hive Export/Import/DropTable/DropPartition to support replication semantics
[
https://issues.apache.org/jira/browse/HIVE-10228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501516#comment-14501516
]
Alan Gates commented on HIVE-10228:
---
+1
Changes to Hive Export/Import/DropTable/DropPartition to support replication
semantics
--
Key: HIVE-10228
URL: https://issues.apache.org/jira/browse/HIVE-10228
Project: Hive
Issue Type: Sub-task
Components: Import/Export
Affects Versions: 1.2.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-10228.2.patch, HIVE-10228.3.patch,
HIVE-10228.4.patch, HIVE-10228.5.patch, HIVE-10228.patch
We need to update a couple of hive commands to support replication semantics.
To wit, we need the following:
EXPORT ... [FOR [METADATA] REPLICATION(“comment”)]
Export will now support an extra optional clause to tell it that this export
is being prepared for the purpose of replication. There is also an additional
optional clause here, that allows for the export to be a metadata-only
export, to handle cases of capturing the diff for alter statements, for
example.
Also, if done for replication, the non-presence of a table, or a table being
a view/offline table/non-native table is not considered an error, and
instead, will result in a successful no-op.
IMPORT ... (as normal) – but handles new semantics
No syntax changes for import, but import will have to change to be able to
handle all the permutations of export dumps possible. Also, import will have
to ensure that it should update the object only if the update being imported
is not older than the state of the object. Also, import currently does not
work with dbname.tablename kind of specification, this should be fixed to
work.
DROP TABLE ... FOR REPLICATION('eventid')
Drop Table now has an additional clause, to specify that this drop table is
being done for replication purposes, and that the dop should not actually
drop the table if the table is newer than that event id specified.
ALTER TABLE ... DROP PARTITION (...) FOR REPLICATION('eventid')
Similarly, Drop Partition also has an equivalent change to Drop Table.
=
In addition, we introduce a new property repl.last.id, which when tagged on
to table properties or partition properties on a replication-destination,
holds the effective state identifier of the object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[ https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-10227: Affects Version/s: 1.2.0 Concrete implementation of Export/Import based ReplicationTaskFactory - Key: HIVE-10227 URL: https://issues.apache.org/jira/browse/HIVE-10227 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 1.2.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-10227.2.patch, HIVE-10227.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10028) LLAP: Create a fixed size execution queue for daemons
[ https://issues.apache.org/jira/browse/HIVE-10028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501578#comment-14501578 ] Lefty Leverenz commented on HIVE-10028: --- Does this need documentation? In particular, *task.scheduler.wait.queue.size* and *task.scheduler.enable.preemption* (and perhaps other configs in LlapConfiguration.java). LLAP: Create a fixed size execution queue for daemons - Key: HIVE-10028 URL: https://issues.apache.org/jira/browse/HIVE-10028 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Prasanth Jayachandran Fix For: llap Attachments: HIVE-10028.1.patch, HIVE-10028.2.patch, HIVE-10028.3.patch, HIVE-10028.4.patch Currently, this is unbounded. This should be a configurable size. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10028) LLAP: Create a fixed size execution queue for daemons
[ https://issues.apache.org/jira/browse/HIVE-10028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501594#comment-14501594 ] Prasanth Jayachandran commented on HIVE-10028: -- Yes. It does need documentation. I think there is already a doc jira for LLAP. LLAP: Create a fixed size execution queue for daemons - Key: HIVE-10028 URL: https://issues.apache.org/jira/browse/HIVE-10028 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Prasanth Jayachandran Fix For: llap Attachments: HIVE-10028.1.patch, HIVE-10028.2.patch, HIVE-10028.3.patch, HIVE-10028.4.patch Currently, this is unbounded. This should be a configurable size. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10268) Merge cbo branch into trunk
[
https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501570#comment-14501570
]
Hive QA commented on HIVE-10268:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726386/HIVE-10268.6.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8723 tests
executed
*Failed tests:*
{noformat}
TestCustomAuthentication - did not produce a TEST-*.xml file
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3495/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3495/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3495/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726386 - PreCommit-HIVE-TRUNK-Build
Merge cbo branch into trunk
---
Key: HIVE-10268
URL: https://issues.apache.org/jira/browse/HIVE-10268
Project: Hive
Issue Type: Task
Components: CBO
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch,
HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.5.patch,
HIVE-10268.6.patch, HIVE-10268.patch
Merge patch generated on basis of diffs of trunk with cbo-branch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10228) Changes to Hive Export/Import/DropTable/DropPartition to support replication semantics
[
https://issues.apache.org/jira/browse/HIVE-10228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501585#comment-14501585
]
Lefty Leverenz commented on HIVE-10228:
---
Doc note: HIVE-10264 will document everything related to replication,
including the configuration parameter added here
(*hive.exim.strict.repl.tables*).
Changes to Hive Export/Import/DropTable/DropPartition to support replication
semantics
--
Key: HIVE-10228
URL: https://issues.apache.org/jira/browse/HIVE-10228
Project: Hive
Issue Type: Sub-task
Components: Import/Export
Affects Versions: 1.2.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Fix For: 1.2.0
Attachments: HIVE-10228.2.patch, HIVE-10228.3.patch,
HIVE-10228.4.patch, HIVE-10228.5.patch, HIVE-10228.patch
We need to update a couple of hive commands to support replication semantics.
To wit, we need the following:
EXPORT ... [FOR [METADATA] REPLICATION(“comment”)]
Export will now support an extra optional clause to tell it that this export
is being prepared for the purpose of replication. There is also an additional
optional clause here, that allows for the export to be a metadata-only
export, to handle cases of capturing the diff for alter statements, for
example.
Also, if done for replication, the non-presence of a table, or a table being
a view/offline table/non-native table is not considered an error, and
instead, will result in a successful no-op.
IMPORT ... (as normal) – but handles new semantics
No syntax changes for import, but import will have to change to be able to
handle all the permutations of export dumps possible. Also, import will have
to ensure that it should update the object only if the update being imported
is not older than the state of the object. Also, import currently does not
work with dbname.tablename kind of specification, this should be fixed to
work.
DROP TABLE ... FOR REPLICATION('eventid')
Drop Table now has an additional clause, to specify that this drop table is
being done for replication purposes, and that the dop should not actually
drop the table if the table is newer than that event id specified.
ALTER TABLE ... DROP PARTITION (...) FOR REPLICATION('eventid')
Similarly, Drop Partition also has an equivalent change to Drop Table.
=
In addition, we introduce a new property repl.last.id, which when tagged on
to table properties or partition properties on a replication-destination,
holds the effective state identifier of the object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[
https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501595#comment-14501595
]
Hive QA commented on HIVE-10227:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726266/HIVE-10227.2.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 8746 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3496/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3496/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3496/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726266 - PreCommit-HIVE-TRUNK-Build
Concrete implementation of Export/Import based ReplicationTaskFactory
-
Key: HIVE-10227
URL: https://issues.apache.org/jira/browse/HIVE-10227
Project: Hive
Issue Type: Sub-task
Components: Import/Export
Affects Versions: 1.2.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-10227.2.patch, HIVE-10227.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-10375) CBO (Calcite Return Path): disable the identity project remover for some union operators
[ https://issues.apache.org/jira/browse/HIVE-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong resolved HIVE-10375. Resolution: Fixed CBO (Calcite Return Path): disable the identity project remover for some union operators Key: HIVE-10375 URL: https://issues.apache.org/jira/browse/HIVE-10375 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong Fix For: 1.2.0 Attachments: HIVE-10375.01.patch The identity project remover will remove the Project operator that is required by some Union Operators. A temporary solution is to disable it for them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-10391) CBO (Calcite Return Path): HiveOpConverter always assumes that HiveFilter does not include a partition column
[ https://issues.apache.org/jira/browse/HIVE-10391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong reassigned HIVE-10391: -- Assignee: Pengcheng Xiong CBO (Calcite Return Path): HiveOpConverter always assumes that HiveFilter does not include a partition column - Key: HIVE-10391 URL: https://issues.apache.org/jira/browse/HIVE-10391 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong Fix For: 1.2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10375) CBO (Calcite Return Path): disable the identity project remover for some union operators
[ https://issues.apache.org/jira/browse/HIVE-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501649#comment-14501649 ] Pengcheng Xiong commented on HIVE-10375: resolved in HIVE-10386 CBO (Calcite Return Path): disable the identity project remover for some union operators Key: HIVE-10375 URL: https://issues.apache.org/jira/browse/HIVE-10375 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong Fix For: 1.2.0 Attachments: HIVE-10375.01.patch The identity project remover will remove the Project operator that is required by some Union Operators. A temporary solution is to disable it for them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10227) Concrete implementation of Export/Import based ReplicationTaskFactory
[ https://issues.apache.org/jira/browse/HIVE-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501662#comment-14501662 ] Mithun Radhakrishnan commented on HIVE-10227: - +0.9. I have some nitpicks. I've made mention on the [Review Board|https://reviews.apache.org/r/7/]. Concrete implementation of Export/Import based ReplicationTaskFactory - Key: HIVE-10227 URL: https://issues.apache.org/jira/browse/HIVE-10227 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 1.2.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-10227.2.patch, HIVE-10227.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10390) CBO (Calcite Return Path): Wrong type for windowing UDFs that produce different values for each row
[ https://issues.apache.org/jira/browse/HIVE-10390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-10390: --- Attachment: HIVE-10390.cbo.patch [~ashutoshc], this solves the issue with the type difference in the windowing test. Could you take a look? Thanks CBO (Calcite Return Path): Wrong type for windowing UDFs that produce different values for each row --- Key: HIVE-10390 URL: https://issues.apache.org/jira/browse/HIVE-10390 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-10390.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-10304) Add deprecation message to HiveCLI
[ https://issues.apache.org/jira/browse/HIVE-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493527#comment-14493527 ] Lefty Leverenz edited comment on HIVE-10304 at 4/19/15 3:02 AM: Doc note: At the least, the CLI wikidoc should include this warning for 1.2.0 with a link to the Beeline doc. Since the CLI doc already has a stub of a Beeline section, perhaps that should be moved before the CLI section with some rewriting. Also, other places that mention the CLI should be updated -- Getting Started and the Tutorial for starters, maybe a few more. * [Hive CLI | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli] * Getting Started ** [Running Hive | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive] ** [Configuration Management Overview | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ConfigurationManagementOverview] ** [Runtime Configuration | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RuntimeConfiguration] ** [Error Logs | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs] * Tutorial ** [Built In Operators and Functions | https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-BuiltInOperatorsandFunctions] ** [Simple Query | https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-SimpleQuery] Edit: Another place is the introduction to the UDFs doc (SHOW FUNCTIONS and DESCRIBE FUNCTION). For now, I just added Beeline: * [Hive Operators and UDFs | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF] was (Author: le...@hortonworks.com): Doc note: At the least, the CLI wikidoc should include this warning for 1.2.0 with a link to the Beeline doc. Since the CLI doc already has a stub of a Beeline section, perhaps that should be moved before the CLI section with some rewriting. Also, other places that mention the CLI should be updated -- Getting Started and the Tutorial for starters, maybe a few more. * [Hive CLI | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli] * Getting Started ** [Running Hive | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive] ** [Configuration Management Overview | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ConfigurationManagementOverview] ** [Runtime Configuration | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RuntimeConfiguration] ** [Error Logs | https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs] * Tutorial ** [Built In Operators and Functions | https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-BuiltInOperatorsandFunctions] ** [Simple Query | https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-SimpleQuery] Add deprecation message to HiveCLI -- Key: HIVE-10304 URL: https://issues.apache.org/jira/browse/HIVE-10304 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 1.1.0 Reporter: Szehon Ho Assignee: Szehon Ho Labels: TODOC1.2 Fix For: 1.2.0 Attachments: HIVE-10304.2.patch, HIVE-10304.3.patch, HIVE-10304.patch As Beeline is now the recommended command line tool to Hive, we should add a message to HiveCLI to indicate that it is deprecated and redirect them to Beeline. This is not suggesting to remove HiveCLI for now, but just a helpful direction for user to know the direction to focus attention in Beeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10383) Add tests for partition level statistics + refactor stats tests of TestHBaseStore [hbase-metastore branch]
[ https://issues.apache.org/jira/browse/HIVE-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-10383: Attachment: HIVE-10383.1.patch Add tests for partition level statistics + refactor stats tests of TestHBaseStore [hbase-metastore branch] -- Key: HIVE-10383 URL: https://issues.apache.org/jira/browse/HIVE-10383 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Attachments: HIVE-10383.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10268) Merge cbo branch into trunk
[ https://issues.apache.org/jira/browse/HIVE-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501671#comment-14501671 ] Laljo John Pullokkaran commented on HIVE-10268: --- +1 Merge cbo branch into trunk --- Key: HIVE-10268 URL: https://issues.apache.org/jira/browse/HIVE-10268 Project: Hive Issue Type: Task Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-10268.1.patch, HIVE-10268.2.patch, HIVE-10268.3.patch, HIVE-10268.4.patch, HIVE-10268.5.patch, HIVE-10268.6.patch, HIVE-10268.patch Merge patch generated on basis of diffs of trunk with cbo-branch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-10028) LLAP: Create a fixed size execution queue for daemons
[ https://issues.apache.org/jira/browse/HIVE-10028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-10028. -- Resolution: Fixed Committed to llap branch. Thanks for the reviews [~seth.siddha...@gmail.com]! LLAP: Create a fixed size execution queue for daemons - Key: HIVE-10028 URL: https://issues.apache.org/jira/browse/HIVE-10028 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Prasanth Jayachandran Fix For: llap Attachments: HIVE-10028.1.patch, HIVE-10028.2.patch, HIVE-10028.3.patch, HIVE-10028.4.patch Currently, this is unbounded. This should be a configurable size. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10347) Merge spark to trunk 4/15/2015
[ https://issues.apache.org/jira/browse/HIVE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-10347: - Attachment: HIVE-10347.3.patch Merge spark to trunk 4/15/2015 -- Key: HIVE-10347 URL: https://issues.apache.org/jira/browse/HIVE-10347 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-10347.2.patch, HIVE-10347.2.patch, HIVE-10347.3.patch, HIVE-10347.patch CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9824) LLAP: Native Vectorization of Map Join so previously CPU bound queries shift their bottleneck to I/O and make it possible for the rest of LLAP to shine ;)
[
https://issues.apache.org/jira/browse/HIVE-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501138#comment-14501138
]
Hive QA commented on HIVE-9824:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726343/HIVE-9824.04.patch
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8743 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vector_decimal_mapjoin
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3492/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3492/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3492/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726343 - PreCommit-HIVE-TRUNK-Build
LLAP: Native Vectorization of Map Join so previously CPU bound queries shift
their bottleneck to I/O and make it possible for the rest of LLAP to shine ;)
--
Key: HIVE-9824
URL: https://issues.apache.org/jira/browse/HIVE-9824
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Priority: Critical
Attachments: HIVE-9824.01.patch, HIVE-9824.02.patch,
HIVE-9824.04.patch
Today's VectorMapJoinOperator is a pass-through that converts each row from a
vectorized row batch in a Java Object[] row and passes it to the
MapJoinOperator superclass.
This enhancement creates specialized vectorized map join operator classes
that are optimized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10347) Merge spark to trunk 4/15/2015
[
https://issues.apache.org/jira/browse/HIVE-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501227#comment-14501227
]
Hive QA commented on HIVE-10347:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12726348/HIVE-10347.3.patch
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 8758 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby3_map_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse
org.apache.hive.jdbc.TestSSL.testSSLConnectionWithProperty
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3493/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3493/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3493/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12726348 - PreCommit-HIVE-TRUNK-Build
Merge spark to trunk 4/15/2015
--
Key: HIVE-10347
URL: https://issues.apache.org/jira/browse/HIVE-10347
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-10347.2.patch, HIVE-10347.2.patch,
HIVE-10347.3.patch, HIVE-10347.patch
CLEAR LIBRARY CACHE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
