[jira] [Commented] (HIVE-17021) Support replication of concatenate operation.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086223#comment-16086223
 ] 

ASF GitHub Bot commented on HIVE-17021:
---

Github user sankarh closed the pull request at:

https://github.com/apache/hive/pull/202


> Support replication of concatenate operation.
> -
>
> Key: HIVE-17021
> URL: https://issues.apache.org/jira/browse/HIVE-17021
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive, repl
>Affects Versions: 2.1.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>  Labels: DR, replication
> Fix For: 3.0.0
>
> Attachments: HIVE-17021.01.patch
>
>
> We need to handle cases like ALTER TABLE ... CONCATENATE that also change the 
> files on disk, and potentially treat them similar to INSERT OVERWRITE, as it 
> does something equivalent to a compaction.
> Note that a ConditionalTask might also be fired at the end of inserts at the 
> end of a tez task (or other exec engine) if appropriate HiveConf settings are 
> set, to automatically do this operation - these also need to be taken care of 
> for replication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17021) Support replication of concatenate operation.

[Sankar Hariappan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086220#comment-16086220
 ] 

Sankar Hariappan commented on HIVE-17021:
-

Thanks for the commit [~daijy]!

> Support replication of concatenate operation.
> -
>
> Key: HIVE-17021
> URL: https://issues.apache.org/jira/browse/HIVE-17021
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive, repl
>Affects Versions: 2.1.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>  Labels: DR, replication
> Fix For: 3.0.0
>
> Attachments: HIVE-17021.01.patch
>
>
> We need to handle cases like ALTER TABLE ... CONCATENATE that also change the 
> files on disk, and potentially treat them similar to INSERT OVERWRITE, as it 
> does something equivalent to a compaction.
> Note that a ConditionalTask might also be fired at the end of inserts at the 
> end of a tez task (or other exec engine) if appropriate HiveConf settings are 
> set, to automatically do this operation - these also need to be taken care of 
> for replication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17021) Support replication of concatenate operation.

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084950#comment-16084950
 ] 

Daniel Dai commented on HIVE-17021:
---

+1. Will commit shortly.

> Support replication of concatenate operation.
> -
>
> Key: HIVE-17021
> URL: https://issues.apache.org/jira/browse/HIVE-17021
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive, repl
>Affects Versions: 2.1.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>  Labels: DR, replication
> Fix For: 3.0.0
>
> Attachments: HIVE-17021.01.patch
>
>
> We need to handle cases like ALTER TABLE ... CONCATENATE that also change the 
> files on disk, and potentially treat them similar to INSERT OVERWRITE, as it 
> does something equivalent to a compaction.
> Note that a ConditionalTask might also be fired at the end of inserts at the 
> end of a tez task (or other exec engine) if appropriate HiveConf settings are 
> set, to automatically do this operation - these also need to be taken care of 
> for replication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17021) Support replication of concatenate operation.

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080614#comment-16080614
 ] 

Hive QA commented on HIVE-17021:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12876435/HIVE-17021.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10836 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] 
(batchId=143)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=232)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
 (batchId=177)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
 (batchId=177)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation 
(batchId=177)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5932/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5932/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5932/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12876435 - PreCommit-HIVE-Build

> Support replication of concatenate operation.
> -
>
> Key: HIVE-17021
> URL: https://issues.apache.org/jira/browse/HIVE-17021
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive, repl
>Affects Versions: 2.1.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>  Labels: DR, replication
> Fix For: 3.0.0
>
> Attachments: HIVE-17021.01.patch
>
>
> We need to handle cases like ALTER TABLE ... CONCATENATE that also change the 
> files on disk, and potentially treat them similar to INSERT OVERWRITE, as it 
> does something equivalent to a compaction.
> Note that a ConditionalTask might also be fired at the end of inserts at the 
> end of a tez task (or other exec engine) if appropriate HiveConf settings are 
> set, to automatically do this operation - these also need to be taken care of 
> for replication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17021) Support replication of concatenate operation.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080518#comment-16080518
 ] 

ASF GitHub Bot commented on HIVE-17021:
---

GitHub user sankarh opened a pull request:

https://github.com/apache/hive/pull/202

HIVE-17021: Support replication of concatenate operation.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sankarh/hive HIVE-17021

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #202


commit e68853deef64037260fb951da329011ef6e5a3d5
Author: Sankar Hariappan 
Date:   2017-07-10T15:24:45Z

HIVE-17021: Support replication of concatenate operation.




> Support replication of concatenate operation.
> -
>
> Key: HIVE-17021
> URL: https://issues.apache.org/jira/browse/HIVE-17021
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive, repl
>Affects Versions: 2.1.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>  Labels: DR, replication
> Fix For: 3.0.0
>
> Attachments: HIVE-17021.01.patch
>
>
> We need to handle cases like ALTER TABLE ... CONCATENATE that also change the 
> files on disk, and potentially treat them similar to INSERT OVERWRITE, as it 
> does something equivalent to a compaction.
> Note that a ConditionalTask might also be fired at the end of inserts at the 
> end of a tez task (or other exec engine) if appropriate HiveConf settings are 
> set, to automatically do this operation - these also need to be taken care of 
> for replication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)