[jira] [Commented] (TEZ-3402) SplitGrouper: Integer overflow

2016-08-05 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408998#comment-15408998 ] Tsuyoshi Ozawa commented on TEZ-3402: - [~aplusplus] {quote} If you see List interface, its size()

[jira] [Updated] (TEZ-3402) SplitGrouper: Integer overflow

2016-08-04 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3402: Attachment: TEZ-3402.001.patch > SplitGrouper: Integer overflow > -- > >

[jira] [Updated] (TEZ-3375) Logging with slf4j way instead of "+" to reduce logging overhead

2016-07-25 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3375: Description: Currently, Tez ATSv1.5 uses concatenation of Strings instead using placeholders {} which

[jira] [Updated] (TEZ-3375) Logging with slf4j way instead of "+" to reduce logging overhead

2016-07-25 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3375: Attachment: TEZ-3375.001.patch Attaching a patch. > Logging with slf4j way instead of "+" to reduce

[jira] [Created] (TEZ-3375) Logging with slf4j way instead of "+" to reduce logging overhead

2016-07-25 Thread Tsuyoshi Ozawa (JIRA)
Tsuyoshi Ozawa created TEZ-3375: --- Summary: Logging with slf4j way instead of "+" to reduce logging overhead Key: TEZ-3375 URL: https://issues.apache.org/jira/browse/TEZ-3375 Project: Apache Tez

[jira] [Assigned] (TEZ-3374) Change TEZ_HISTORY_LOGGING_TIMELINE_NUM_DAGS_PER_GROUP conf key name.

2016-07-25 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-3374: --- Assignee: Tsuyoshi Ozawa > Change TEZ_HISTORY_LOGGING_TIMELINE_NUM_DAGS_PER_GROUP conf key

[jira] [Updated] (TEZ-3374) Change TEZ_HISTORY_LOGGING_TIMELINE_NUM_DAGS_PER_GROUP conf key name.

2016-07-25 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3374: Attachment: TEZ-3374.001.patch > Change TEZ_HISTORY_LOGGING_TIMELINE_NUM_DAGS_PER_GROUP conf key

[jira] [Commented] (TEZ-3374) Change TEZ_HISTORY_LOGGING_TIMELINE_NUM_DAGS_PER_GROUP conf key name.

2016-07-25 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393216#comment-15393216 ] Tsuyoshi Ozawa commented on TEZ-3374: - Attaching first patch. > Change

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-12 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374379#comment-15374379 ] Tsuyoshi Ozawa commented on TEZ-3303: - Thanks for your review and your committing :-) > Have

[jira] [Updated] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-12 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3334: Issue Type: New Feature (was: Bug) > Tez Custom Shuffle Handler > -- > >

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-11 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371706#comment-15371706 ] Tsuyoshi Ozawa commented on TEZ-3303: - [~sseth] do you have any comments? > Have ShuffleVertexManager

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-08 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367391#comment-15367391 ] Tsuyoshi Ozawa commented on TEZ-3303: - Thank you, Ming. [~sseth] could you also check the patch? >

[jira] [Updated] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-07 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3303: Attachment: TEZ-3303.003.patch Good catch, thanks [~mingma] for the precise review! Updating the

[jira] [Commented] (TEZ-3313) ATSFileParser : Wrong args passed in VersionInfo

2016-07-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360817#comment-15360817 ] Tsuyoshi Ozawa commented on TEZ-3313: - +1(non-binding), We can implement TEZ-3319 after this issue. >

[jira] [Updated] (TEZ-3313) ATSFileParser : Wrong args passed in VersionInfo

2016-07-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3313: Assignee: Rajesh Balamohan > ATSFileParser : Wrong args passed in VersionInfo >

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360727#comment-15360727 ] Tsuyoshi Ozawa commented on TEZ-3303: - [~sseth] [~mingma] could you take a look? > Have

[jira] [Updated] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3303: Attachment: TEZ-3303.002.patch > Have ShuffleVertexManager consume more precise partition stats >

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-02 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360194#comment-15360194 ] Tsuyoshi Ozawa commented on TEZ-3303: - The test failures don't occur on my local. so kicking the CI

[jira] [Updated] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-01 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3303: Attachment: TEZ-3303.002.patch > Have ShuffleVertexManager consume more precise partition stats >

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-01 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359570#comment-15359570 ] Tsuyoshi Ozawa commented on TEZ-3303: - [~sseth] [~mingma] thanks for your clarification. Make sense.

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358196#comment-15358196 ] Tsuyoshi Ozawa commented on TEZ-3303: - [~sseth] could you take a look? > Have ShuffleVertexManager

[jira] [Comment Edited] (TEZ-3287) Have UnorderedPartitionedKVWriter honor tez.runtime.empty.partitions.info-via-events.enabled

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358068#comment-15358068 ] Tsuyoshi Ozawa edited comment on TEZ-3287 at 6/30/16 11:57 PM: --- [~lmv] thanks

[jira] [Commented] (TEZ-3287) Have UnorderedPartitionedKVWriter honor tez.runtime.empty.partitions.info-via-events.enabled

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358068#comment-15358068 ] Tsuyoshi Ozawa commented on TEZ-3287: - [~lmv] thanks for your taking a look! The patch is targeting

[jira] [Commented] (TEZ-2962) Use per partition stats in shuffle vertex manager auto parallelism

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357961#comment-15357961 ] Tsuyoshi Ozawa commented on TEZ-2962: - This can be done after TEZ-3303. > Use per partition stats in

[jira] [Updated] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3303: Attachment: TEZ-3303.001.patch Attaching a patch to consume more precise partition stats. > Have

[jira] [Assigned] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-3303: --- Assignee: Tsuyoshi Ozawa > Have ShuffleVertexManager consume more precise partition stats >

[jira] [Assigned] (TEZ-3320) Java implementation of bitonic merge sort

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-3320: --- Assignee: Tsuyoshi Ozawa > Java implementation of bitonic merge sort >

[jira] [Created] (TEZ-3320) Java implementation of bitonic merge sort

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
Tsuyoshi Ozawa created TEZ-3320: --- Summary: Java implementation of bitonic merge sort Key: TEZ-3320 URL: https://issues.apache.org/jira/browse/TEZ-3320 Project: Apache Tez Issue Type:

[jira] [Commented] (TEZ-3319) tez-history-parser should not have its own Version class

2016-06-30 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357458#comment-15357458 ] Tsuyoshi Ozawa commented on TEZ-3319: - I agree with the solution :-) > tez-history-parser should not

[jira] [Commented] (TEZ-3313) ATSFileParser : Wrong args passed in VersionInfo

2016-06-27 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352204#comment-15352204 ] Tsuyoshi Ozawa commented on TEZ-3313: - Thanks [~rajesh.balamohan] for your contribution. I have 2

[jira] [Updated] (TEZ-2608) UnorderedPartitionedKVWriter should override write(Object key, Iterable values)

2016-06-27 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-2608: Attachment: TEZ-2608.002.patch Attaching v2 patch based on Sid's comment. >

[jira] [Commented] (TEZ-2608) UnorderedPartitionedKVWriter should override write(Object key, Iterable values)

2016-06-27 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351533#comment-15351533 ] Tsuyoshi Ozawa commented on TEZ-2608: - {quote} Another useful bit to look at is whether the write(Key,

[jira] [Commented] (TEZ-3287) Have UnorderedPartitionedKVWriter honor tez.runtime.empty.partitions.info-via-events.enabled

2016-06-27 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351409#comment-15351409 ] Tsuyoshi Ozawa commented on TEZ-3287: - [~bikassaha] could you review this? > Have

[jira] [Updated] (TEZ-3284) Synchronization for every write in UnorderdKVWriter

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3284: Assignee: Jonathan Eagles > Synchronization for every write in UnorderdKVWriter >

[jira] [Commented] (TEZ-3311) ATSFileParser resource leak: InputStream should be closed

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348961#comment-15348961 ] Tsuyoshi Ozawa commented on TEZ-3311: - [~rajesh.balamohan] thank you for the review. You're right, I

[jira] [Commented] (TEZ-2582) Consider removing DataInputBuffer sync overheads in pipelinedsorter

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348945#comment-15348945 ] Tsuyoshi Ozawa commented on TEZ-2582: - OK, thanks! > Consider removing DataInputBuffer sync overheads

[jira] [Commented] (TEZ-2582) Consider removing DataInputBuffer sync overheads in pipelinedsorter

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348928#comment-15348928 ] Tsuyoshi Ozawa commented on TEZ-2582: - [~rajesh.balamohan] HADOOP-10694 fixes this, but do you mean we

[jira] [Resolved] (TEZ-2146) TaskCounter.OUTPUT_BYTES_PHYSICAL is not populated in PipelinedSorter

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa resolved TEZ-2146. - Resolution: Duplicate Closing this since TEZ-2575 fixes this issue. >

[jira] [Updated] (TEZ-3287) Have UnorderedPartitionedKVWriter honor tez.runtime.empty.partitions.info-via-events.enabled

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3287: Attachment: TEZ-3287.001.patch Attaching a first patch to honor

[jira] [Commented] (TEZ-3284) Synchronization for every write in UnorderdKVWriter

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348807#comment-15348807 ] Tsuyoshi Ozawa commented on TEZ-3284: - [~jeagles] is this jira working in progress? The patch introduces

[jira] [Updated] (TEZ-3311) ATSFileParser resource leak: InputStream should be closed

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3311: Attachment: TEZ-3311.001.patch > ATSFileParser resource leak: InputStream should be closed >

[jira] [Updated] (TEZ-3311) ATSFileParser resource leak: InputStream should be closed

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3311: Attachment: (was: TEZ-3311.001.patch) > ATSFileParser resource leak: InputStream should be closed

[jira] [Updated] (TEZ-3311) ATSFileParser resource leak: InputStream should be closed

2016-06-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3311: Attachment: TEZ-3311.001.patch Attaching a first patch to close the inputStream. > ATSFileParser

[jira] [Commented] (TEZ-3272) Add AMContainerImpl and AMNodeImpl to StateMachine visualization list

2016-06-20 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339094#comment-15339094 ] Tsuyoshi Ozawa commented on TEZ-3272: - This patch only changes pom.xml, so we don't need to add any

[jira] [Commented] (TEZ-3009) Errors that occur during container task acquisition are not logged

2016-06-20 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339049#comment-15339049 ] Tsuyoshi Ozawa commented on TEZ-3009: - [~hitesh] could you check this? Jason's patch LGTM, while my

[jira] [Resolved] (TEZ-3014) OOM during Shuffle in JDK 8

2016-06-20 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa resolved TEZ-3014. - Resolution: Fixed Closing this for now. Feel free to reopen this. > OOM during Shuffle in JDK 8 >

[jira] [Commented] (TEZ-3113) massive increase of run time using PipelinedSorter rather than DefaultSorter

2016-06-20 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339044#comment-15339044 ] Tsuyoshi Ozawa commented on TEZ-3113: - > Observed behaviour of PipelinedSorter is that several hundred

[jira] [Commented] (TEZ-3252) [Umbrella] Enable support for Hadoop-3.x

2016-06-19 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339040#comment-15339040 ] Tsuyoshi Ozawa commented on TEZ-3252: - I think we need to check dependencies here. > [Umbrella] Enable

[jira] [Updated] (TEZ-3272) Add AMContainerImpl and AMNodeImpl to StateMachine visualization list

2016-06-19 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3272: Attachment: TEZ-3272.001.patch Attaching a first patch. I confirmed with this patch that AMContainer

[jira] [Assigned] (TEZ-3272) Add AMContainerImpl and AMNodeImpl to StateMachine visualization list

2016-06-19 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-3272: --- Assignee: Tsuyoshi Ozawa > Add AMContainerImpl and AMNodeImpl to StateMachine visualization

[jira] [Updated] (TEZ-3302) Add a version of processorContext.waitForAllInputsReady and waitForAnyInputReady with a timeout

2016-06-17 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3302: Attachment: TEZ-3302.002.patch Oops, I attached wrong patch. Fixing it and added test. > Add a

[jira] [Updated] (TEZ-3302) Add a version of processorContext.waitForAllInputsReady and waitForAnyInputReady with a timeout

2016-06-17 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3302: Attachment: (was: TEZ-3302.001.patch) > Add a version of processorContext.waitForAllInputsReady

[jira] [Updated] (TEZ-3302) Add a version of processorContext.waitForAllInputsReady and waitForAnyInputReady with a timeout

2016-06-17 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3302: Attachment: TEZ-3302.001.patch Attaching first patch. [~sseth], could you check it? > Add a version

[jira] [Assigned] (TEZ-3302) Add a version of processorContext.waitForAllInputsReady and waitForAnyInputReady with a timeout

2016-06-17 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-3302: --- Assignee: Tsuyoshi Ozawa > Add a version of processorContext.waitForAllInputsReady and >

[jira] [Commented] (TEZ-3237) Corrupted shuffle transfers to disk are not detected during transfer

2016-05-12 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281573#comment-15281573 ] Tsuyoshi Ozawa commented on TEZ-3237: - [~rajesh.balamohan] make sense to me. > Corrupted shuffle

[jira] [Commented] (TEZ-3237) Corrupted shuffle transfers to disk are not detected during transfer

2016-05-11 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281020#comment-15281020 ] Tsuyoshi Ozawa commented on TEZ-3237: - Thanks Jason for taking important change! {quote} Would it be

[jira] [Commented] (TEZ-2607) SIMD-based bitonic merge sorting

2016-03-27 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213712#comment-15213712 ] Tsuyoshi Ozawa commented on TEZ-2607: - I resumed to work on this. Maybe I will be able to post the first

[jira] [Assigned] (TEZ-987) Need a way to measure container reuse

2016-03-10 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-987: -- Assignee: Tsuyoshi Ozawa > Need a way to measure container reuse >

[jira] [Commented] (TEZ-3148) Invalid event TA_TEZ_EVENT_UPDATE on TaskAttempt

2016-03-09 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188803#comment-15188803 ] Tsuyoshi Ozawa commented on TEZ-3148: - [~hitesh] Make sense. Changed to handle TA_TEZ_EVENT_UPDATE only

[jira] [Updated] (TEZ-3148) Invalid event TA_TEZ_EVENT_UPDATE on TaskAttempt

2016-03-09 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3148: Attachment: TEZ-3148.002.patch > Invalid event TA_TEZ_EVENT_UPDATE on TaskAttempt >

[jira] [Commented] (TEZ-3148) Invalid event TA_TEZ_EVENT_UPDATE on TaskAttempt

2016-03-09 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188676#comment-15188676 ] Tsuyoshi Ozawa commented on TEZ-3148: - [~aplusplus] Thanks a lot for the feedback! Yes, I'll add test

[jira] [Commented] (TEZ-3096) Statemachine: TA_TEZ_EVENT_UPDATE at KILL_IN_PROGRESS fails

2016-03-07 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184011#comment-15184011 ] Tsuyoshi Ozawa commented on TEZ-3096: - I've also uploaded the patch, so I appreciate if you take a look.

[jira] [Commented] (TEZ-3096) Statemachine: TA_TEZ_EVENT_UPDATE at KILL_IN_PROGRESS fails

2016-03-07 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184009#comment-15184009 ] Tsuyoshi Ozawa commented on TEZ-3096: - [~gopalv] [~aplusplus] maybe duplicated issue of TEZ-3148? >

[jira] [Updated] (TEZ-3010) Container task acquisition has no retries for errors

2016-03-04 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3010: Attachment: TEZ-3010.001.patch Attaching first patch to add retry semantics to TezChild. This patch

[jira] [Updated] (TEZ-3148) Invalid event TA_TEZ_EVENT_UPDATE on TaskAttempt

2016-03-04 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3148: Attachment: TEZ-3148.001.patch Adding transition after starting task attempt to handle

[jira] [Commented] (TEZ-2756) MergeManager close should not try merging files on close if invoked after a shuffle exception

2016-03-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179517#comment-15179517 ] Tsuyoshi Ozawa commented on TEZ-2756: - [~rajesh.balamohan] thanks Rajesh for the reviewing and

[jira] [Commented] (TEZ-2954) Container launch timeouts should count towards node blacklisting

2016-03-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179515#comment-15179515 ] Tsuyoshi Ozawa commented on TEZ-2954: - I think the patch includes TEZ-925. [~sseth] could you take a

[jira] [Assigned] (TEZ-2954) Container launch timeouts should count towards node blacklisting

2016-03-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-2954: --- Assignee: Tsuyoshi Ozawa > Container launch timeouts should count towards node blacklisting >

[jira] [Updated] (TEZ-2954) Container launch timeouts should count towards node blacklisting

2016-03-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-2954: Attachment: TEZ-2954.001.patch Attaching a first patch to consider container-launch timeout towards

[jira] [Commented] (TEZ-2756) MergeManager close should not try merging files on close if invoked after a shuffle exception

2016-03-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179041#comment-15179041 ] Tsuyoshi Ozawa commented on TEZ-2756: - [~sseth] could you take a look? > MergeManager close should not

[jira] [Commented] (TEZ-2580) Remove VertexManagerPlugin#setVertexParallelism with VertexManagerPlugin#reconfigureVertex

2016-03-03 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179039#comment-15179039 ] Tsuyoshi Ozawa commented on TEZ-2580: - Make sense. Pending this for that it's done. > Remove

[jira] [Updated] (TEZ-2988) DAGAppMaster::shutdownTezAM should return with a no-op if it has been invoked earlier

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-2988: Attachment: TEZ-2988.002.patch [~hitesh] thanks for your review. Completely agree, updating a patch.

[jira] [Updated] (TEZ-2988) DAGAppMaster::shutdownTezAM should return with a no-op if it has been invoked earlier

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-2988: Attachment: TEZ-2988.001.patch Attaching v1 patch. [~hitesh], could you take a look? >

[jira] [Updated] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3120: Attachment: (was: TEZ-3120.001.patch) > Remove TaskCommContext.getCurrentDagName, Identifier >

[jira] [Updated] (TEZ-2580) Remove VertexManagerPlugin#setVertexParallelism with VertexManagerPlugin#reconfigureVertex

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-2580: Attachment: TEZ-2580.001.patch Attaching a patch to remove setVertexParallelism API. > Remove

[jira] [Updated] (TEZ-2756) MergeManager close should not try merging files on close if invoked after a shuffle exception

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-2756: Attachment: TEZ-2756.001.patch Attaching v1 patch. > MergeManager close should not try merging files

[jira] [Commented] (TEZ-1944) OOM when using tez.runtime.shuffle.memory-to-memory.enable=true

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171028#comment-15171028 ] Tsuyoshi Ozawa commented on TEZ-1944: - Can we close this as duplicated problem of TEZ-1911? > OOM when

[jira] [Commented] (TEZ-3009) Errors that occur during container task acquisition are not logged

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171026#comment-15171026 ] Tsuyoshi Ozawa commented on TEZ-3009: - Kicking CI again. [~sseth], the patch looks good to

[jira] [Commented] (TEZ-3014) OOM during Shuffle in JDK 8

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171024#comment-15171024 ] Tsuyoshi Ozawa commented on TEZ-3014: - Linking to TEZ-3114 and TEZ-3115 since they seems to be related.

[jira] [Commented] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171023#comment-15171023 ] Tsuyoshi Ozawa commented on TEZ-3120: - Opps, overlooking that assignee is Sidd. Feel free to use the

[jira] [Updated] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-02-28 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3120: Attachment: TEZ-3120.001.patch Attaching a patch to remove the deprecated method. > Remove

[jira] [Updated] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-26 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3128: Attachment: TEZ-3128.004.patch Attaching v04 patch to remove a needless comment. > Avoid stopping

[jira] [Commented] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-26 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169083#comment-15169083 ] Tsuyoshi Ozawa commented on TEZ-3128: - Attached v03 patch to address the comment. > Avoid stopping

[jira] [Commented] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-25 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167944#comment-15167944 ] Tsuyoshi Ozawa commented on TEZ-3128: - [~sseth] [~hitesh] could you check the patch? > Avoid stopping

[jira] [Updated] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-24 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3128: Attachment: TEZ-3128.002.patch Sidd and Hitesh, thank for the clarification. Created the patch to

[jira] [Commented] (TEZ-3067) Links to tez configs documentation should be bubbled up to top-level release page

2016-02-22 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158007#comment-15158007 ] Tsuyoshi Ozawa commented on TEZ-3067: - Thanks [~hitesh] for your committing and reviewing. > Links to

[jira] [Commented] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-22 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158004#comment-15158004 ] Tsuyoshi Ozawa commented on TEZ-3128: - [~hitesh] [~sseth] Thank you for pointing. {quote} dagappmaster

[jira] [Updated] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-21 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3128: Attachment: TEZ-3128.001.patch Attaching a patch to make the YarnTaskSchedulerService's timeout for

[jira] [Assigned] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-21 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-3128: --- Assignee: Tsuyoshi Ozawa > Avoid stopping containers on the AM shutdown thread >

[jira] [Updated] (TEZ-3067) Links to tez configs documentation should be bubbled up to top-level release page

2016-02-21 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3067: Attachment: TEZ-3067.002.patch [~hitesh] thanks for your review. Updating a patch to add the links to

[jira] [Commented] (TEZ-3067) Links to tez configs documentation should be bubbled up to top-level release page

2016-02-21 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156403#comment-15156403 ] Tsuyoshi Ozawa commented on TEZ-3067: - [~hitesh] could you take a look? > Links to tez configs

[jira] [Updated] (TEZ-3067) Links to tez configs documentation should be bubbled up to top-level release page

2016-02-21 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3067: Attachment: TEZ-3067.001.patch Add a link to the page of configuration. Before 0.8.1-alpha, the links

[jira] [Assigned] (TEZ-3067) Links to tez configs documentation should be bubbled up to top-level release page

2016-02-21 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned TEZ-3067: --- Assignee: Tsuyoshi Ozawa > Links to tez configs documentation should be bubbled up to

[jira] [Commented] (TEZ-1670) Add tests for all converter functions in HistoryEventTimelineConversion

2015-11-10 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1469#comment-1469 ] Tsuyoshi Ozawa commented on TEZ-1670: - Hi Tatsuya, thank you for taking this issue. Submitting a patch

[jira] [Resolved] (TEZ-2932) Демоны Да Винчи 3 сезон 2 серия смотреть сериал онлайн в качестве HD720 на русском [07.11.2015]

2015-11-08 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa resolved TEZ-2932. - Resolution: Invalid > Демоны Да Винчи 3 сезон 2 серия смотреть сериал онлайн в качестве HD720 на >

[jira] [Updated] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093

2015-10-06 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-2849: Attachment: TEZ-2849.2.patch [~gopalv] Attaching a patch rebased on trunk. Unfortunately, it fails

[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093

2015-10-02 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940866#comment-14940866 ] Tsuyoshi Ozawa commented on TEZ-2849: - [~gopalv] thanks for your patch! I'll take a benchmark with your

[jira] [Commented] (TEZ-2607) SIMD-based bitonic merge sorting

2015-08-26 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712747#comment-14712747 ] Tsuyoshi Ozawa commented on TEZ-2607: - I've found that we need a hack in a tricky way to

[jira] [Commented] (TEZ-2607) SIMD-based bitonic merge sorting

2015-08-17 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699070#comment-14699070 ] Tsuyoshi Ozawa commented on TEZ-2607: - Implemented bitonic_algorithm with [~maropu].

[jira] [Commented] (TEZ-2602) Throwing EOFException when launching MR job

2015-07-23 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638342#comment-14638342 ] Tsuyoshi Ozawa commented on TEZ-2602: - Thanks Gopal for review and thanks Rajesh for the

  1   2   3   >