[jira] [Updated] (TEZ-3235) Modify Example TestOrderedWordCount job to test the IPC limit for large dag plans
[ https://issues.apache.org/jira/browse/TEZ-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushmitha Sreenivasan updated TEZ-3235: --- Attachment: Tez-3235.2.patch [~aplusplus] Thank you for the review comments. Made all the suggested changes. Please take a look when you get a chance. > Modify Example TestOrderedWordCount job to test the IPC limit for large dag > plans > - > > Key: TEZ-3235 > URL: https://issues.apache.org/jira/browse/TEZ-3235 > Project: Apache Tez > Issue Type: Task >Affects Versions: 0.8.3 >Reporter: Sushmitha Sreenivasan >Assignee: Sushmitha Sreenivasan > Attachments: TEZ-3235.1.patch, Tez-3235.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Success: TEZ-3235 PreCommit Build #1775
Jira: https://issues.apache.org/jira/browse/TEZ-3235
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1775/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3884 lines...]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-ui
[INFO] Build failures were ignored.
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12808743/Tez-3235.2.patch
against master revision a2d120b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1775//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1775//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
fc898c8d3528e02f1bc1bb48e48caed4a27b7d7f logged out
==
==
Finished build.
==
==
Archiving artifacts
[description-setter] Description set: TEZ-3235
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-3235) Modify Example TestOrderedWordCount job to test the IPC limit for large dag plans
[
https://issues.apache.org/jira/browse/TEZ-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319426#comment-15319426
]
TezQA commented on TEZ-3235:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12808743/Tez-3235.2.patch
against master revision a2d120b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1775//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1775//console
This message is automatically generated.
> Modify Example TestOrderedWordCount job to test the IPC limit for large dag
> plans
> -
>
> Key: TEZ-3235
> URL: https://issues.apache.org/jira/browse/TEZ-3235
> Project: Apache Tez
> Issue Type: Task
>Affects Versions: 0.8.3
>Reporter: Sushmitha Sreenivasan
>Assignee: Sushmitha Sreenivasan
> Attachments: TEZ-3235.1.patch, Tez-3235.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2846) Flaky test: TestCommit.testVertexCommit_OnDAGSuccess
[ https://issues.apache.org/jira/browse/TEZ-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319813#comment-15319813 ] Jeff Zhang commented on TEZ-2846: - +1 > Flaky test: TestCommit.testVertexCommit_OnDAGSuccess > > > Key: TEZ-2846 > URL: https://issues.apache.org/jira/browse/TEZ-2846 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-2846.1.patch > > > java.lang.AssertionError: expected:<0> but was:<1> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:743) > at org.junit.Assert.assertEquals(Assert.java:118) > at org.junit.Assert.assertEquals(Assert.java:555) > at org.junit.Assert.assertEquals(Assert.java:542) > at > org.apache.tez.dag.app.dag.impl.TestCommit.testVertexCommit_OnDAGSuccess(TestCommit.java:582) > From: > https://builds.apache.org/job/Tez-Build/1250/testReport/junit/org.apache.tez.dag.app.dag.impl/TestCommit/testVertexCommit_OnDAGSuccess/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3235) Modify Example TestOrderedWordCount job to test the IPC limit for large dag plans
[ https://issues.apache.org/jira/browse/TEZ-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushmitha Sreenivasan updated TEZ-3235: --- Attachment: Tez-3235.2.patch > Modify Example TestOrderedWordCount job to test the IPC limit for large dag > plans > - > > Key: TEZ-3235 > URL: https://issues.apache.org/jira/browse/TEZ-3235 > Project: Apache Tez > Issue Type: Task >Affects Versions: 0.8.3 >Reporter: Sushmitha Sreenivasan >Assignee: Sushmitha Sreenivasan > Attachments: TEZ-3235.1.patch, Tez-3235.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3235) Modify Example TestOrderedWordCount job to test the IPC limit for large dag plans
[
https://issues.apache.org/jira/browse/TEZ-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319462#comment-15319462
]
TezQA commented on TEZ-3235:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12808744/Tez-3235.2.patch
against master revision a2d120b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1776//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1776//console
This message is automatically generated.
> Modify Example TestOrderedWordCount job to test the IPC limit for large dag
> plans
> -
>
> Key: TEZ-3235
> URL: https://issues.apache.org/jira/browse/TEZ-3235
> Project: Apache Tez
> Issue Type: Task
>Affects Versions: 0.8.3
>Reporter: Sushmitha Sreenivasan
>Assignee: Sushmitha Sreenivasan
> Attachments: TEZ-3235.1.patch, Tez-3235.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Success: TEZ-3235 PreCommit Build #1776
Jira: https://issues.apache.org/jira/browse/TEZ-3235
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1776/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3884 lines...]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-ui
[INFO] Build failures were ignored.
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12808744/Tez-3235.2.patch
against master revision a2d120b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1776//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1776//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
5d1bd3ee59506731d59d1e2675d767c7cdc580a3 logged out
==
==
Finished build.
==
==
Archiving artifacts
[description-setter] Description set: TEZ-3235
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-2846) Flaky test: TestCommit.testVertexCommit_OnDAGSuccess
[ https://issues.apache.org/jira/browse/TEZ-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319434#comment-15319434 ] Hitesh Shah commented on TEZ-2846: -- [~zjffdu] Mind doing a review? > Flaky test: TestCommit.testVertexCommit_OnDAGSuccess > > > Key: TEZ-2846 > URL: https://issues.apache.org/jira/browse/TEZ-2846 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-2846.1.patch > > > java.lang.AssertionError: expected:<0> but was:<1> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:743) > at org.junit.Assert.assertEquals(Assert.java:118) > at org.junit.Assert.assertEquals(Assert.java:555) > at org.junit.Assert.assertEquals(Assert.java:542) > at > org.apache.tez.dag.app.dag.impl.TestCommit.testVertexCommit_OnDAGSuccess(TestCommit.java:582) > From: > https://builds.apache.org/job/Tez-Build/1250/testReport/junit/org.apache.tez.dag.app.dag.impl/TestCommit/testVertexCommit_OnDAGSuccess/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available
[
https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319172#comment-15319172
]
Gopal V commented on TEZ-3291:
--
bq. it will group as if all splits are on the same machine.
I think that's the exact issue here - they are not.
> Optimize splits grouping when locality information is not available
> ---
>
> Key: TEZ-3291
> URL: https://issues.apache.org/jira/browse/TEZ-3291
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: TEZ-3291.WIP.patch
>
>
> There are scenarios where splits might not contain the location details. S3
> is an example, where all splits would have "localhost" for the location
> details. In such cases, curent split computation does not go through the
> rack local and allow-small groups optimizations and ends up creating small
> number of splits. Depending on clusters this can end creating long running
> map jobs.
> Example with hive:
> ==
> 1. Inventory table in tpc-ds dataset is partitioned and is relatively a small
> table.
> 2. With query-22, hive requests with the original splits count as 52 and
> overall length of splits themselves is around 12061817 bytes.
> {{tez.grouping.min-size}} was set to 16 MB.
> 3. In tez splits grouping, this ends up creating a single split with 52+
> files be processed in the split. In clusters with split locations, this
> would have landed up with multiple splits since {{allowSmallGroups}} would
> have kicked in.
> But in S3, since everything would have "localhost" all splits get added to
> single group. This makes things a lot worse.
> 4. Depending on the dataset and the format, this can be problematic. For
> instance, file open calls and random seeks can be expensive in S3.
> 5. In this case, 52 files have to be opened and processed by single task in
> sequential fashion. Had it been processed by multiple tasks, response time
> would have drastically reduced.
> E.g log details
> {noformat}
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Grouping splits in Tez
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired splits: 110 too large. Desired
> splitLength: 109652 Min splitLength: 16777216 New desired splits: 1 Total
> length: 12061817 Original splits: 52
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired numSplits: 1 lengthPerGroup: 12061817
> numLocations: 1 numSplitsPerLocation: 52 numSplitsInGroup: 52 totalLength:
> 12061817 numOriginalSplits: 52 . Grouping by length: true count: false
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Number of splits desired: 1 created: 1
> splitsProcessed: 52
> {noformat}
> Alternate options:
> ==
> 1. Force Hadoop to provide bogus locations for S3. But not sure, if that
> would be accepted anytime soon. Ref: HADOOP-12878
> 2. Set {{tez.grouping.min-size}} to very very low value. But should the end
> user always be doing this on query to query basis?
> 3. When {{(lengthPerGroup < "tez.grouping.min-size")}}, recompute
> desiredNumSplits only when number of distinct locations in the splits is > 1.
> This would force more number of splits to be generated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (TEZ-3291) Optimize splits grouping when locality information is not available
[
https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319219#comment-15319219
]
Hitesh Shah edited comment on TEZ-3291 at 6/7/16 7:46 PM:
--
bq. it will group as if all splits are on the same machine.
Seems like a potential solution would be to generate a random location string
for each split ( "InvalidHost_N" where N auto-increments ) and then let the
rest of the grouping code do what it does today? Would need some better
handling to convert the split location into a host-ask against the RM I guess.
was (Author: hitesh):
bq. it will group as if all splits are on the same machine.
Seems like a better solution would be to generate a random location string for
each split ( "InvalidHost_N" where N auto-increments ) and then let the rest of
the grouping code do what it does today? Would need some better handling to
convert the split location into a host-ask against the RM I guess.
> Optimize splits grouping when locality information is not available
> ---
>
> Key: TEZ-3291
> URL: https://issues.apache.org/jira/browse/TEZ-3291
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: TEZ-3291.WIP.patch
>
>
> There are scenarios where splits might not contain the location details. S3
> is an example, where all splits would have "localhost" for the location
> details. In such cases, curent split computation does not go through the
> rack local and allow-small groups optimizations and ends up creating small
> number of splits. Depending on clusters this can end creating long running
> map jobs.
> Example with hive:
> ==
> 1. Inventory table in tpc-ds dataset is partitioned and is relatively a small
> table.
> 2. With query-22, hive requests with the original splits count as 52 and
> overall length of splits themselves is around 12061817 bytes.
> {{tez.grouping.min-size}} was set to 16 MB.
> 3. In tez splits grouping, this ends up creating a single split with 52+
> files be processed in the split. In clusters with split locations, this
> would have landed up with multiple splits since {{allowSmallGroups}} would
> have kicked in.
> But in S3, since everything would have "localhost" all splits get added to
> single group. This makes things a lot worse.
> 4. Depending on the dataset and the format, this can be problematic. For
> instance, file open calls and random seeks can be expensive in S3.
> 5. In this case, 52 files have to be opened and processed by single task in
> sequential fashion. Had it been processed by multiple tasks, response time
> would have drastically reduced.
> E.g log details
> {noformat}
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Grouping splits in Tez
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired splits: 110 too large. Desired
> splitLength: 109652 Min splitLength: 16777216 New desired splits: 1 Total
> length: 12061817 Original splits: 52
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired numSplits: 1 lengthPerGroup: 12061817
> numLocations: 1 numSplitsPerLocation: 52 numSplitsInGroup: 52 totalLength:
> 12061817 numOriginalSplits: 52 . Grouping by length: true count: false
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Number of splits desired: 1 created: 1
> splitsProcessed: 52
> {noformat}
> Alternate options:
> ==
> 1. Force Hadoop to provide bogus locations for S3. But not sure, if that
> would be accepted anytime soon. Ref: HADOOP-12878
> 2. Set {{tez.grouping.min-size}} to very very low value. But should the end
> user always be doing this on query to query basis?
> 3. When {{(lengthPerGroup < "tez.grouping.min-size")}}, recompute
> desiredNumSplits only when number of distinct locations in the splits is > 1.
> This would force more number of splits to be generated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available
[
https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319219#comment-15319219
]
Hitesh Shah commented on TEZ-3291:
--
bq. it will group as if all splits are on the same machine.
Seems like a better solution would be to generate a random location string for
each split ( "InvalidHost_N" where N auto-increments ) and then let the rest of
the grouping code do what it does today? Would need some better handling to
convert the split location into a host-ask against the RM I guess.
> Optimize splits grouping when locality information is not available
> ---
>
> Key: TEZ-3291
> URL: https://issues.apache.org/jira/browse/TEZ-3291
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: TEZ-3291.WIP.patch
>
>
> There are scenarios where splits might not contain the location details. S3
> is an example, where all splits would have "localhost" for the location
> details. In such cases, curent split computation does not go through the
> rack local and allow-small groups optimizations and ends up creating small
> number of splits. Depending on clusters this can end creating long running
> map jobs.
> Example with hive:
> ==
> 1. Inventory table in tpc-ds dataset is partitioned and is relatively a small
> table.
> 2. With query-22, hive requests with the original splits count as 52 and
> overall length of splits themselves is around 12061817 bytes.
> {{tez.grouping.min-size}} was set to 16 MB.
> 3. In tez splits grouping, this ends up creating a single split with 52+
> files be processed in the split. In clusters with split locations, this
> would have landed up with multiple splits since {{allowSmallGroups}} would
> have kicked in.
> But in S3, since everything would have "localhost" all splits get added to
> single group. This makes things a lot worse.
> 4. Depending on the dataset and the format, this can be problematic. For
> instance, file open calls and random seeks can be expensive in S3.
> 5. In this case, 52 files have to be opened and processed by single task in
> sequential fashion. Had it been processed by multiple tasks, response time
> would have drastically reduced.
> E.g log details
> {noformat}
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Grouping splits in Tez
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired splits: 110 too large. Desired
> splitLength: 109652 Min splitLength: 16777216 New desired splits: 1 Total
> length: 12061817 Original splits: 52
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired numSplits: 1 lengthPerGroup: 12061817
> numLocations: 1 numSplitsPerLocation: 52 numSplitsInGroup: 52 totalLength:
> 12061817 numOriginalSplits: 52 . Grouping by length: true count: false
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Number of splits desired: 1 created: 1
> splitsProcessed: 52
> {noformat}
> Alternate options:
> ==
> 1. Force Hadoop to provide bogus locations for S3. But not sure, if that
> would be accepted anytime soon. Ref: HADOOP-12878
> 2. Set {{tez.grouping.min-size}} to very very low value. But should the end
> user always be doing this on query to query basis?
> 3. When {{(lengthPerGroup < "tez.grouping.min-size")}}, recompute
> desiredNumSplits only when number of distinct locations in the splits is > 1.
> This would force more number of splits to be generated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available
[
https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319263#comment-15319263
]
Bikas Saha commented on TEZ-3291:
-
Then that would be a bug to fix. Hopefully thats what the patch is doing.
> Optimize splits grouping when locality information is not available
> ---
>
> Key: TEZ-3291
> URL: https://issues.apache.org/jira/browse/TEZ-3291
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: TEZ-3291.WIP.patch
>
>
> There are scenarios where splits might not contain the location details. S3
> is an example, where all splits would have "localhost" for the location
> details. In such cases, curent split computation does not go through the
> rack local and allow-small groups optimizations and ends up creating small
> number of splits. Depending on clusters this can end creating long running
> map jobs.
> Example with hive:
> ==
> 1. Inventory table in tpc-ds dataset is partitioned and is relatively a small
> table.
> 2. With query-22, hive requests with the original splits count as 52 and
> overall length of splits themselves is around 12061817 bytes.
> {{tez.grouping.min-size}} was set to 16 MB.
> 3. In tez splits grouping, this ends up creating a single split with 52+
> files be processed in the split. In clusters with split locations, this
> would have landed up with multiple splits since {{allowSmallGroups}} would
> have kicked in.
> But in S3, since everything would have "localhost" all splits get added to
> single group. This makes things a lot worse.
> 4. Depending on the dataset and the format, this can be problematic. For
> instance, file open calls and random seeks can be expensive in S3.
> 5. In this case, 52 files have to be opened and processed by single task in
> sequential fashion. Had it been processed by multiple tasks, response time
> would have drastically reduced.
> E.g log details
> {noformat}
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Grouping splits in Tez
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired splits: 110 too large. Desired
> splitLength: 109652 Min splitLength: 16777216 New desired splits: 1 Total
> length: 12061817 Original splits: 52
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired numSplits: 1 lengthPerGroup: 12061817
> numLocations: 1 numSplitsPerLocation: 52 numSplitsInGroup: 52 totalLength:
> 12061817 numOriginalSplits: 52 . Grouping by length: true count: false
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Number of splits desired: 1 created: 1
> splitsProcessed: 52
> {noformat}
> Alternate options:
> ==
> 1. Force Hadoop to provide bogus locations for S3. But not sure, if that
> would be accepted anytime soon. Ref: HADOOP-12878
> 2. Set {{tez.grouping.min-size}} to very very low value. But should the end
> user always be doing this on query to query basis?
> 3. When {{(lengthPerGroup < "tez.grouping.min-size")}}, recompute
> desiredNumSplits only when number of distinct locations in the splits is > 1.
> This would force more number of splits to be generated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TEZ-3235) Modify Example TestOrderedWordCount job to test the IPC limit for large dag plans
[ https://issues.apache.org/jira/browse/TEZ-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushmitha Sreenivasan updated TEZ-3235: --- Attachment: (was: Tez-3235.2.patch) > Modify Example TestOrderedWordCount job to test the IPC limit for large dag > plans > - > > Key: TEZ-3235 > URL: https://issues.apache.org/jira/browse/TEZ-3235 > Project: Apache Tez > Issue Type: Task >Affects Versions: 0.8.3 >Reporter: Sushmitha Sreenivasan >Assignee: Sushmitha Sreenivasan > Attachments: TEZ-3235.1.patch, Tez-3235.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-2846) Flaky test: TestCommit.testVertexCommit_OnDAGSuccess
[ https://issues.apache.org/jira/browse/TEZ-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reassigned TEZ-2846: Assignee: Hitesh Shah > Flaky test: TestCommit.testVertexCommit_OnDAGSuccess > > > Key: TEZ-2846 > URL: https://issues.apache.org/jira/browse/TEZ-2846 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > > java.lang.AssertionError: expected:<0> but was:<1> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:743) > at org.junit.Assert.assertEquals(Assert.java:118) > at org.junit.Assert.assertEquals(Assert.java:555) > at org.junit.Assert.assertEquals(Assert.java:542) > at > org.apache.tez.dag.app.dag.impl.TestCommit.testVertexCommit_OnDAGSuccess(TestCommit.java:582) > From: > https://builds.apache.org/job/Tez-Build/1250/testReport/junit/org.apache.tez.dag.app.dag.impl/TestCommit/testVertexCommit_OnDAGSuccess/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2846) Flaky test: TestCommit.testVertexCommit_OnDAGSuccess
[ https://issues.apache.org/jira/browse/TEZ-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319432#comment-15319432 ] Hitesh Shah edited comment on TEZ-2846 at 6/7/16 9:06 PM: -- The commitCounter gets incremented even though the commit is meant to block. This results in flaky runs as sometimes the commit gets kicked off before the commit count is checked. was (Author: hitesh): The commitCounter gets increment even though the commit is meant to block. This results in flaky runs as sometimes the commit gets kicked off before the commit count is checked. > Flaky test: TestCommit.testVertexCommit_OnDAGSuccess > > > Key: TEZ-2846 > URL: https://issues.apache.org/jira/browse/TEZ-2846 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-2846.1.patch > > > java.lang.AssertionError: expected:<0> but was:<1> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:743) > at org.junit.Assert.assertEquals(Assert.java:118) > at org.junit.Assert.assertEquals(Assert.java:555) > at org.junit.Assert.assertEquals(Assert.java:542) > at > org.apache.tez.dag.app.dag.impl.TestCommit.testVertexCommit_OnDAGSuccess(TestCommit.java:582) > From: > https://builds.apache.org/job/Tez-Build/1250/testReport/junit/org.apache.tez.dag.app.dag.impl/TestCommit/testVertexCommit_OnDAGSuccess/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2846) Flaky test: TestCommit.testVertexCommit_OnDAGSuccess
[ https://issues.apache.org/jira/browse/TEZ-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2846: - Attachment: TEZ-2846.1.patch The commitCounter gets increment even though the commit is meant to block. This results in flaky runs as sometimes the commit gets kicked off before the commit count is checked. > Flaky test: TestCommit.testVertexCommit_OnDAGSuccess > > > Key: TEZ-2846 > URL: https://issues.apache.org/jira/browse/TEZ-2846 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-2846.1.patch > > > java.lang.AssertionError: expected:<0> but was:<1> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:743) > at org.junit.Assert.assertEquals(Assert.java:118) > at org.junit.Assert.assertEquals(Assert.java:555) > at org.junit.Assert.assertEquals(Assert.java:542) > at > org.apache.tez.dag.app.dag.impl.TestCommit.testVertexCommit_OnDAGSuccess(TestCommit.java:582) > From: > https://builds.apache.org/job/Tez-Build/1250/testReport/junit/org.apache.tez.dag.app.dag.impl/TestCommit/testVertexCommit_OnDAGSuccess/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Success: TEZ-2846 PreCommit Build #1777
Jira: https://issues.apache.org/jira/browse/TEZ-2846
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1777/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3884 lines...]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-ui
[INFO] Build failures were ignored.
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12808760/TEZ-2846.1.patch
against master revision a2d120b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1777//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1777//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
4940e0c37267c38eec0302f8a434ad7b0378f43b logged out
==
==
Finished build.
==
==
Archiving artifacts
[description-setter] Description set: TEZ-2846
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-2846) Flaky test: TestCommit.testVertexCommit_OnDAGSuccess
[
https://issues.apache.org/jira/browse/TEZ-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319584#comment-15319584
]
TezQA commented on TEZ-2846:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12808760/TEZ-2846.1.patch
against master revision a2d120b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1777//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1777//console
This message is automatically generated.
> Flaky test: TestCommit.testVertexCommit_OnDAGSuccess
>
>
> Key: TEZ-2846
> URL: https://issues.apache.org/jira/browse/TEZ-2846
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Hitesh Shah
> Attachments: TEZ-2846.1.patch
>
>
> java.lang.AssertionError: expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.tez.dag.app.dag.impl.TestCommit.testVertexCommit_OnDAGSuccess(TestCommit.java:582)
> From:
> https://builds.apache.org/job/Tez-Build/1250/testReport/junit/org.apache.tez.dag.app.dag.impl/TestCommit/testVertexCommit_OnDAGSuccess/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (TEZ-2998) TestPreemption.testPreemptionWithSession fails on hadoop-2.4
[
https://issues.apache.org/jira/browse/TEZ-2998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah resolved TEZ-2998.
--
Resolution: Won't Fix
Havent seen this one show up after defaulting to min of 2.6
> TestPreemption.testPreemptionWithSession fails on hadoop-2.4
>
>
> Key: TEZ-2998
> URL: https://issues.apache.org/jira/browse/TEZ-2998
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Jeff Zhang
>
> {code}
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.714 sec <<<
> FAILURE!
> testPreemptionWithSession(org.apache.tez.dag.app.TestPreemption) Time
> elapsed: 5.573 sec <<< ERROR!
> org.apache.tez.dag.api.TezException: App master already running a DAG
> at
> org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:1307)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.submitDAG(DAGClientHandler.java:120)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:161)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7471)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1363)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy14.submitDAG(Unknown Source)
> at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:536)
> at org.apache.tez.client.TezClient.submitDAG(TezClient.java:452)
> at
> org.apache.tez.dag.app.TestPreemption.testPreemptionJob(TestPreemption.java:186)
> at
> org.apache.tez.dag.app.TestPreemption.testPreemptionMultiple(TestPreemption.java:176)
> at
> org.apache.tez.dag.app.TestPreemption.testPreemptionWithSession(TestPreemption.java:141)
> {code}
> https://builds.apache.org/job/Tez-Build-Hadoop-2.4/225/console
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available
[
https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318941#comment-15318941
]
Bikas Saha commented on TEZ-3291:
-
IIRC they should because localhost will be treated as a valid machine name and
it will group as if all splits are on the same machine. The code itself does
the same thing by adding a same bogus machine location name for all splits that
have no location. Thereafter the code works identically for splits that have
real locations and other that have fake locations.
> Optimize splits grouping when locality information is not available
> ---
>
> Key: TEZ-3291
> URL: https://issues.apache.org/jira/browse/TEZ-3291
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: TEZ-3291.WIP.patch
>
>
> There are scenarios where splits might not contain the location details. S3
> is an example, where all splits would have "localhost" for the location
> details. In such cases, curent split computation does not go through the
> rack local and allow-small groups optimizations and ends up creating small
> number of splits. Depending on clusters this can end creating long running
> map jobs.
> Example with hive:
> ==
> 1. Inventory table in tpc-ds dataset is partitioned and is relatively a small
> table.
> 2. With query-22, hive requests with the original splits count as 52 and
> overall length of splits themselves is around 12061817 bytes.
> {{tez.grouping.min-size}} was set to 16 MB.
> 3. In tez splits grouping, this ends up creating a single split with 52+
> files be processed in the split. In clusters with split locations, this
> would have landed up with multiple splits since {{allowSmallGroups}} would
> have kicked in.
> But in S3, since everything would have "localhost" all splits get added to
> single group. This makes things a lot worse.
> 4. Depending on the dataset and the format, this can be problematic. For
> instance, file open calls and random seeks can be expensive in S3.
> 5. In this case, 52 files have to be opened and processed by single task in
> sequential fashion. Had it been processed by multiple tasks, response time
> would have drastically reduced.
> E.g log details
> {noformat}
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Grouping splits in Tez
> 2016-06-01 13:48:08,353 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired splits: 110 too large. Desired
> splitLength: 109652 Min splitLength: 16777216 New desired splits: 1 Total
> length: 12061817 Original splits: 52
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Desired numSplits: 1 lengthPerGroup: 12061817
> numLocations: 1 numSplitsPerLocation: 52 numSplitsInGroup: 52 totalLength:
> 12061817 numOriginalSplits: 52 . Grouping by length: true count: false
> 2016-06-01 13:48:08,354 [INFO] [InputInitializer {Map 2} #0]
> |split.TezMapredSplitsGrouper|: Number of splits desired: 1 created: 1
> splitsProcessed: 52
> {noformat}
> Alternate options:
> ==
> 1. Force Hadoop to provide bogus locations for S3. But not sure, if that
> would be accepted anytime soon. Ref: HADOOP-12878
> 2. Set {{tez.grouping.min-size}} to very very low value. But should the end
> user always be doing this on query to query basis?
> 3. When {{(lengthPerGroup < "tez.grouping.min-size")}}, recompute
> desiredNumSplits only when number of distinct locations in the splits is > 1.
> This would force more number of splits to be generated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
