[
https://issues.apache.org/jira/browse/TEZ-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013646#comment-16013646
]
Sneha Kanekar commented on TEZ-3680:
------------------------------------
The following tests fail intermittently on ppc64le and x86 platform since
TEZ-3680.
-
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testMultipleSpills_WithPipelinedShuffle
-
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testMergeBuffersAndSpill
The error message is as follows:
{code:borderStyle=solid}
java.util.concurrent.RejectedExecutionException: Task
com.google.common.util.concurrent.ListenableFutureTask@272aa26f rejected from
java.util.concurrent.ThreadPoolExecutor@5c508ec9[Running, pool size = 2, active
threads = 2, queued tasks = 0, completed tasks = 1]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
at
com.google.common.util.concurrent.MoreExecutors$ListeningDecorator.execute(MoreExecutors.java:440)
at
com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:56)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.setupNextBuffer(UnorderedPartitionedKVWriter.java:431)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:378)
at
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:329)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.baseTestWithPipelinedTransfer(TestUnorderedPartitionedKVWriter.java:629)
at
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testMultipleSpills_WithPipelinedShuffle(TestUnorderedPartitionedKVWriter.java:565)
{code}
Any update on this issue?
> Optimizations to UnorderedPartitionedKVWriter
> ---------------------------------------------
>
> Key: TEZ-3680
> URL: https://issues.apache.org/jira/browse/TEZ-3680
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Fix For: 0.9.0
>
> Attachments: profiler.png, TEZ-3680.1.patch, TEZ-3680.2.patch,
> TEZ-3680.3.patch, TEZ-3680.4.patch, TEZ-3680.5.patch, TEZ-3680.6.patch,
> TEZ-3680.7.patch
>
>
> 1. Consider increasing the number of threads in spill executor.
> {{TEZ_RUNTIME_UNORDERED_OUTPUT_MAX_PER_BUFFER_SIZE_BYTES}} can be used to
> configure the buffer size. If smaller buffer sizes are provided, there is a
> chance of getting frequent spills; currently the spill executor operates in
> single threaded mode.
> 2. During profiling, things like incrementing the counters, notifying
> progress came up. This may not be common in regular tez jobs. But in
> processes like LLAP (hive based), it is possible to get into such situations.
> I will attach the profiler snapshot showing this. It would be good to
> update/notify less frequently.
> 3. Optimize mergeAll().
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
