[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-20 Thread Vineet Garg (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446515#comment-16446515
 ] 

Vineet Garg commented on HIVE-19195:


Pushed the patch to branch-3

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-17 Thread Deepak Jaiswal (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441599#comment-16441599
 ] 

Deepak Jaiswal commented on HIVE-19195:
---

[~ashutoshc] thanks for the analysis. The last two should be easy to fix. I 
will take a look why tez_smb_1 is giving different plan again.

 

I have opened HIVE-19234 to track this.

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Deepak Jaiswal (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440137#comment-16440137
 ] 

Deepak Jaiswal commented on HIVE-19195:
---

I did not update result file for llap_smb for TestMiniLlapCliDriver which 
resulted in failure. I removed it from the list after noticing it as it already 
runs in LlapLocal.

Will look into tez_smb_1 why it keeps changing plan.

Thanks [~ashutoshc] for pointing this out.

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Ashutosh Chauhan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439978#comment-16439978
 ] 

Ashutosh Chauhan commented on HIVE-19195:
-

TestMiniLlapCliDriver.testCliDriver[llap_smb] failed in run of this patch. 
Doesn't seem like falkiness is resolved yet.

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Deepak Jaiswal (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439800#comment-16439800
 ] 

Deepak Jaiswal commented on HIVE-19195:
---

Looks like the way I created the patch these deleted files did not show in it.

Thankfully, I did clean them up locally.

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Ashutosh Chauhan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439746#comment-16439746
 ] 

Ashutosh Chauhan commented on HIVE-19195:
-

Following q.out needs to be deleted :
{code}

org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut

Failing for the past 1 build (Since Failed#10253 )
Took 17 ms.
Error Message
dangling qouts: 
[/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_14.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_15.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/llap_acid_fast.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_13.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/llap_acid.q.out,
 
/home/hiveptest/35.225.136.142-hiveptest-0/apache-github-source-source/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out]
{code}

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439728#comment-16439728
 ] 

Hive QA commented on HIVE-19195:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12919126/HIVE-19195.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 89 failed/errored test(s), 13823 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439610#comment-16439610
 ] 

Hive QA commented on HIVE-19195:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
53s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
33s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
17s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}  2m 43s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10253/dev-support/hive-personality.sh
 |
| git revision | master / c46c4c9 |
| asflicense | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10253/yetus/patch-asflicense-problems.txt
 |
| modules | C: itests ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10253/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Ashutosh Chauhan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439500#comment-16439500
 ] 

Ashutosh Chauhan commented on HIVE-19195:
-

+1 
You may also delete .q.out for these tests corresponding to TestCliDriver.

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19195) Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.

2018-04-16 Thread Deepak Jaiswal (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-19195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439009#comment-16439009
 ] 

Deepak Jaiswal commented on HIVE-19195:
---

[~ashutoshc] can you please review?

I clubbed changes for HIVE-17055 in the same patch. Moved out unnecessary tests 
listed in shared list which were also listed in llaplocal only list as well 
along with BMJ, SMB, tez and llap specific tests.

> Fix flaky tests and cleanup testconfiguration to run llap specific tests in 
> llap only.
> --
>
> Key: HIVE-19195
> URL: https://issues.apache.org/jira/browse/HIVE-19195
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-19195.1.patch
>
>
> This test is certainly flaky. Seems like it makes some assumption about 
> available memory. Consider dropping this altogether.
>  Also remove tests from llaplocal.shared list to llaplocal so that they dont 
> run in MR.
> Makes HIVE-17055 redundant.
> {code:java}
> Client Execution succeeded but contained differences (error code = 1) after 
> executing auto_sortmerge_join_2.q 
> 1101,1103d1100
> < Hive Runtime Error: Map local work exhausted memory
> < FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)