[
https://issues.apache.org/jira/browse/TEZ-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225453#comment-16225453
]
Prasanth Jayachandran commented on TEZ-3861:
--------------------------------------------
[~rajesh.balamohan] please find the relevant log line..
{code}
2017-10-30T11:03:02,864 INFO [TezTR-545720_1_1_2_0_0] impl.ShuffleManager:
Shutting down pending fetchers on sourceMap_1: 0
2017-10-30T11:03:02,864 INFO [TezTR-545720_1_1_2_0_0]
impl.ShuffleInputEventHandlerImpl: Map 4: numDmeEventsSeen=465,
numDmeEventsSeenWithNoData=150, numObsoletionEventsSeen=0, updateOnClose
2017-10-30T11:03:02,864 INFO [TezTR-545720_1_1_2_0_0] impl.ShuffleManager:
Shutting down pending fetchers on sourceMap_4: 0
2017-10-30T11:03:02,864 INFO [TezTR-545720_1_1_2_0_0] impl.PipelinedSorter:
Reducer 3: Starting flush of map output
2017-10-30T11:03:02,865 INFO [TezTR-545720_1_1_2_0_0] impl.PipelinedSorter:
Reducer 3: Span0.length = 125514, perItem = 14
2017-10-30T11:03:02,927 INFO [TezTR-545720_1_1_2_0_0] impl.PipelinedSorter:
Reducer 3: done sorting span=0, length=125514, time=62
2017-10-30T11:03:02,931 INFO [TezTR-545720_1_1_2_0_0] impl.PipelinedSorter:
Reducer 3: Heap = SpanIterator<0:125513> (span=Span[12582912,1879047]),
{code}
> PipelineSorter setting negative progess
> ---------------------------------------
>
> Key: TEZ-3861
> URL: https://issues.apache.org/jira/browse/TEZ-3861
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Prasanth Jayachandran
>
> PipelineSorter is generating too big log mostly coming from setting progress
> to negative value in some cases.
> {code}
> 2017-10-30T01:22:16,466 DEBUG [TezTR-702853_1_1_2_0_0] util.Progress: Illegal
> progress value found, progress is less than 0. Progress will be changed to 0
> 2017-10-30T01:22:16,469 DEBUG [TezTR-702853_1_1_2_0_0] util.Progress: Illegal
> progress value found, progress is less than 0. Progress will be changed to 0
> 2017-10-30T01:22:16,469 DEBUG [TezTR-702853_1_1_2_0_0] util.Progress: Illegal
> progress value found, progress is less than 0. Progress will be changed to 0
> 2017-10-30T01:22:16,470 DEBUG [TezTR-702853_1_1_2_0_0] util.Progress: Illegal
> progress value found, progress is less than 0. Progress will be changed to 0
> 2017-10-30T01:22:16,470 DEBUG [TezTR-702853_1_1_2_0_0] util.Progress: Illegal
> progress value found, progress is less than 0. Progress will be changed to 0
> {code}
> this is emitted from
> https://github.com/apache/tez/blob/87d7c145ffc71707d1d393fddf94efa2a77d8822/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/sort/impl/PipelinedSorter.java#L1126
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
