[
https://issues.apache.org/jira/browse/YARN-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767279#comment-13767279
]
Hadoop QA commented on YARN-1130:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12603162/YARN-1130.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.mapreduce.security.ssl.TestEncryptedShuffle
org.apache.hadoop.mapred.TestNetworkedJob
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapred.TestJobCounters
org.apache.hadoop.mapreduce.v2.TestMRAppWithCombiner
org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapred.TestBlockLimits
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
org.apache.hadoop.mapred.TestMiniMRChildTask
org.apache.hadoop.mapreduce.security.TestMRCredentials
org.apache.hadoop.mapreduce.v2.TestNonExistentJob
org.apache.hadoop.mapreduce.v2.TestRMNMInfo
org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser
org.apache.hadoop.mapreduce.v2.TestMROldApiJobs
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution
org.apache.hadoop.mapred.TestJobCleanup
org.apache.hadoop.mapreduce.v2.TestMRAMWithNonNormalizedCapabilities
org.apache.hadoop.mapred.TestReduceFetch
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
org.apache.hadoop.mapred.TestMerge
org.apache.hadoop.mapreduce.v2.TestMRJobs
org.apache.hadoop.mapreduce.TestMRJobClient
org.apache.hadoop.mapred.TestTaskCommit
org.apache.hadoop.mapreduce.TestChild
org.apache.hadoop.mapred.TestJobName
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapreduce.security.TestBinaryTokenFile
org.apache.hadoop.mapreduce.v2.TestUberAM
org.apache.hadoop.mapred.TestMiniMRClientCluster
org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
org.apache.hadoop.ipc.TestSocketFactory
org.apache.hadoop.mapred.TestJobSysDirWithDFS
The following test timeouts occurred in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
org.apache.hadoop.conf.TestNoDefaultsJobConf
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1929//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1929//console
This message is automatically generated.
> Improve the log flushing for tasks when mapred.userlog.limit.kb is set
> ----------------------------------------------------------------------
>
> Key: YARN-1130
> URL: https://issues.apache.org/jira/browse/YARN-1130
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.0.5-alpha
> Reporter: Paul Han
> Fix For: 2.0.5-alpha
>
> Attachments: YARN-1130.patch
>
>
> When userlog limit is set with something like this:
> {code}
> <property>
> <name>mapred.userlog.limit.kb</name>
> <value>2048</value>
> <description>The maximum size of user-logs of each task in KB. 0 disables the
> cap.
> </description>
> </property>
> {code}
> the log entry will be truncated randomly for the jobs.
> The log size is left between 1.2MB to 1.6MB.
> Since the log is already limited, avoid the log truncation is crucial for
> user.
> The other issue with the current
> impl(org.apache.hadoop.yarn.ContainerLogAppender) is that log entries will
> not flush to file until the container shutdown and logmanager close all
> appenders. If user likes to see the log during task execution, it doesn't
> support it.
> Will propose a patch to add a flush mechanism and also flush the log when
> task is done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
