[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for ResourceManager

[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13393510#comment-13393510 ] Tsuyoshi OZAWA commented on MAPREDUCE-4343: --- Harsh, I think this problem

[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for ResourceManager

[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13393519#comment-13393519 ] Tsuyoshi OZAWA commented on MAPREDUCE-4343: --- I'm trying to solve the 2nd

[jira] [Updated] (MAPREDUCE-4343) ZK recovery support for ResourceManager

[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4343: -- Attachment: MR-4343.1.patch Harsh, The attached patch allows ResourceManager to

[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for ResourceManager

[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13393569#comment-13393569 ] Tsuyoshi OZAWA commented on MAPREDUCE-4343: --- s/course/coarse/

[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for ResourceManager

[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396063#comment-13396063 ] Tsuyoshi OZAWA commented on MAPREDUCE-4343: --- Sharad, Bikas marked

[jira] [Updated] (MAPREDUCE-4326) Resurrect RM Restart

[ https://issues.apache.org/jira/browse/MAPREDUCE-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4326: -- Attachment: MR-4343.1.patch Bikas, The attached patch is originally created for

[jira] [Commented] (MAPREDUCE-3642) Remove hardcoded strings from the JC#displayTasks() call.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13397935#comment-13397935 ] Tsuyoshi OZAWA commented on MAPREDUCE-3642: --- Harsh, I'd like to deal with

[jira] [Commented] (MAPREDUCE-4326) Resurrect RM Restart

[ https://issues.apache.org/jira/browse/MAPREDUCE-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13402297#comment-13402297 ] Tsuyoshi OZAWA commented on MAPREDUCE-4326: --- Sharad, MAPREDUCE-2713 is now

[jira] [Commented] (MAPREDUCE-4326) Resurrect RM Restart

[ https://issues.apache.org/jira/browse/MAPREDUCE-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13402323#comment-13402323 ] Tsuyoshi OZAWA commented on MAPREDUCE-4326: --- Bikas, What's going on? I can

[jira] [Commented] (MAPREDUCE-4326) Resurrect RM Restart

[ https://issues.apache.org/jira/browse/MAPREDUCE-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13423770#comment-13423770 ] Tsuyoshi OZAWA commented on MAPREDUCE-4326: --- Yeah, it's not trivial what to

[jira] [Commented] (MAPREDUCE-4326) Resurrect RM Restart

[ https://issues.apache.org/jira/browse/MAPREDUCE-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13424504#comment-13424504 ] Tsuyoshi OZAWA commented on MAPREDUCE-4326: --- I've looked around the code of

[jira] [Commented] (MAPREDUCE-4326) Resurrect RM Restart

[ https://issues.apache.org/jira/browse/MAPREDUCE-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13424552#comment-13424552 ] Tsuyoshi OZAWA commented on MAPREDUCE-4326: --- s/Are the states enough to

[jira] [Created] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

Tsuyoshi OZAWA created MAPREDUCE-4502: - Summary: Multi-level aggregation with combining the result of maps per node/rack Key: MAPREDUCE-4502 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4502

[jira] [Updated] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4502: -- Description: The shuffle costs is expensive in Hadoop in spite of the existence of

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426365#comment-13426365 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- We prototyped the combining the

[jira] [Updated] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-3902: -- Attachment: MAPREDUCE-3902.2.patch As a first step, I fixed the patch by Arun to

[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13428211#comment-13428211 ] Tsuyoshi OZAWA commented on MAPREDUCE-3902: --- IMHO, the 2nd topic(combining

[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13428212#comment-13428212 ] Tsuyoshi OZAWA commented on MAPREDUCE-3902: --- s/should be moved/should be

[jira] [Created] (MAPREDUCE-4525) Combiner per container

Tsuyoshi OZAWA created MAPREDUCE-4525: - Summary: Combiner per container Key: MAPREDUCE-4525 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4525 Project: Hadoop Map/Reduce Issue

[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13430927#comment-13430927 ] Tsuyoshi OZAWA commented on MAPREDUCE-3902: --- @Siddharth, Thank you for your

[jira] [Commented] (MAPREDUCE-4326) Resurrect RM Restart

[ https://issues.apache.org/jira/browse/MAPREDUCE-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13430935#comment-13430935 ] Tsuyoshi OZAWA commented on MAPREDUCE-4326: --- So there may not be the need

[jira] [Updated] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-3902: -- Description: The MR AM is now in a great position to reuse containers across (map)

[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435807#comment-13435807 ] Tsuyoshi OZAWA commented on MAPREDUCE-3902: --- @Siddharth, I have two

[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437662#comment-13437662 ] Tsuyoshi OZAWA commented on MAPREDUCE-3902: --- bq. I think a pull request

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437688#comment-13437688 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I'm working on refactoring

[jira] [Created] (MAPREDUCE-4581) TaskHeartbeatHandler should extends HeartbeatHandlerBase

Tsuyoshi OZAWA created MAPREDUCE-4581: - Summary: TaskHeartbeatHandler should extends HeartbeatHandlerBase Key: MAPREDUCE-4581 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4581 Project:

[jira] [Updated] (MAPREDUCE-4581) TaskHeartbeatHandler should extends HeartbeatHandlerBase

[ https://issues.apache.org/jira/browse/MAPREDUCE-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4581: -- Attachment: MAPREDUCE-4581.patch I rebased code on the new MR-3902 branch and

[jira] [Updated] (MAPREDUCE-4581) TaskHeartbeatHandler should extends HeartbeatHandlerBase

[ https://issues.apache.org/jira/browse/MAPREDUCE-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4581: -- Attachment: MAPREDUCE-4581.2.patch Removed needless packages and lines.

[jira] [Commented] (MAPREDUCE-4581) TaskHeartbeatHandler should extends HeartbeatHandlerBase

[ https://issues.apache.org/jira/browse/MAPREDUCE-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440954#comment-13440954 ] Tsuyoshi OZAWA commented on MAPREDUCE-4581: --- Thank you for review, Sed.

[jira] [Updated] (MAPREDUCE-4581) TaskHeartbeatHandler should extends HeartbeatHandlerBase

[ https://issues.apache.org/jira/browse/MAPREDUCE-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4581: -- Attachment: MAPREDUCE-4581.3.patch Fixed to indent and generate another diagnostic

[jira] [Created] (MAPREDUCE-4599) Ensure not to launch container on blacklisted nodes

Tsuyoshi OZAWA created MAPREDUCE-4599: - Summary: Ensure not to launch container on blacklisted nodes Key: MAPREDUCE-4599 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4599 Project: Hadoop

[jira] [Updated] (MAPREDUCE-4599) Ensure not to launch container on blacklisted nodes

[ https://issues.apache.org/jira/browse/MAPREDUCE-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4599: -- Attachment: MAPREDUCE-4599.patch Implemented AMNodeMap to maintain blacklists of

[jira] [Updated] (MAPREDUCE-4599) Ensure not to launch container on blacklisted nodes

[ https://issues.apache.org/jira/browse/MAPREDUCE-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4599: -- Attachment: MAPREDUCE-4599.2.patch Fixed to dispatch the event type

[jira] [Updated] (MAPREDUCE-4599) Ensure not to launch container on blacklisted nodes

[ https://issues.apache.org/jira/browse/MAPREDUCE-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4599: -- Attachment: MAPREDUCE-4599.3.patch Got rid of the invocation of

[jira] [Updated] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4502: -- Assignee: Tsuyoshi OZAWA Multi-level aggregation with combining the result of

[jira] [Created] (MAPREDUCE-4609) RMContainerAllocator#scheduleInterval should be configurable

Tsuyoshi OZAWA created MAPREDUCE-4609: - Summary: RMContainerAllocator#scheduleInterval should be configurable Key: MAPREDUCE-4609 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4609

[jira] [Updated] (MAPREDUCE-4609) RMContainerAllocator#scheduleInterval should be configurable

[ https://issues.apache.org/jira/browse/MAPREDUCE-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4609: -- Attachment: MR-4609.patch Made scheduleInterval configurable. One concern is that

[jira] [Updated] (MAPREDUCE-4582) [MAPREDUCE-3902] ScheduledRequests#remove should remove the elements from mapsHostMapping and mapsRackMapping

[ https://issues.apache.org/jira/browse/MAPREDUCE-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4582: -- Attachment: MAPREDUCE-4582.2.patch Thanks for your review, Siddharth. I fixed the

[jira] [Updated] (MAPREDUCE-4609) [MAPREDUCE-3902]RMContainerAllocator#scheduleInterval should be configurable

[ https://issues.apache.org/jira/browse/MAPREDUCE-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4609: -- Summary: [MAPREDUCE-3902]RMContainerAllocator#scheduleInterval should be

[jira] [Updated] (MAPREDUCE-4609) [MAPREDUCE-3902] RMContainerAllocator#scheduleInterval should be configurable

[ https://issues.apache.org/jira/browse/MAPREDUCE-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4609: -- Summary: [MAPREDUCE-3902] RMContainerAllocator#scheduleInterval should be

[jira] [Created] (MAPREDUCE-4615) [MAPREDUCE-3902] RMContainerAllocator#assign should be split into functions

Tsuyoshi OZAWA created MAPREDUCE-4615: - Summary: [MAPREDUCE-3902] RMContainerAllocator#assign should be split into functions Key: MAPREDUCE-4615 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4615

[jira] [Updated] (MAPREDUCE-4615) [MAPREDUCE-3902] RMContainerAllocator#assign should be split into functions

[ https://issues.apache.org/jira/browse/MAPREDUCE-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4615: -- Attachment: MAPREDUCE-4615.patch Refactored RMContainerAllocator#assign.

[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13445786#comment-13445786 ] Tsuyoshi OZAWA commented on MAPREDUCE-3902: --- [~sseth], I think that it is

[jira] [Commented] (MAPREDUCE-4615) [MAPREDUCE-3902] RMContainerAllocator#assign should be split into functions

[ https://issues.apache.org/jira/browse/MAPREDUCE-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446423#comment-13446423 ] Tsuyoshi OZAWA commented on MAPREDUCE-4615: --- Sure, NP.

[jira] [Updated] (MAPREDUCE-4619) [MAPREDUCE-3902] Change AMContainerMap to extend AbstractService

[ https://issues.apache.org/jira/browse/MAPREDUCE-4619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4619: -- Assignee: Tsuyoshi OZAWA [MAPREDUCE-3902] Change AMContainerMap to extend

[jira] [Updated] (MAPREDUCE-4619) [MAPREDUCE-3902] Change AMContainerMap to extend AbstractService

[ https://issues.apache.org/jira/browse/MAPREDUCE-4619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4619: -- Attachment: MAPREDUCE-4619.patch Changed AMContainerMap to extend AbstractService.

[jira] [Updated] (MAPREDUCE-4582) [MAPREDUCE-3902] ScheduledRequests#remove should remove the elements from mapsHostMapping and mapsRackMapping

[ https://issues.apache.org/jira/browse/MAPREDUCE-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4582: -- Attachment: MAPREDUCE-4582.3.patch Fixed to remove entries from

[jira] [Commented] (MAPREDUCE-3902) MR AM should reuse containers for map tasks, there-by allowing fine-grained control on num-maps for users without need for CombineFileInputFormat etc.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13447292#comment-13447292 ] Tsuyoshi OZAWA commented on MAPREDUCE-3902: --- Thanks for your enumerating

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13450752#comment-13450752 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- After debating with Siddharsh,

[jira] [Updated] (MAPREDUCE-4525) Combiner per container

[ https://issues.apache.org/jira/browse/MAPREDUCE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4525: -- Assignee: Tsuyoshi OZAWA Combiner per container --

[jira] [Updated] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4502: -- Attachment: speculative_draft.pdf I attached the first speculative draft design of

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452640#comment-13452640 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- Karthik, thanks for your

[jira] [Updated] (MAPREDUCE-4525) Combiner per node

[ https://issues.apache.org/jira/browse/MAPREDUCE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4525: -- Summary: Combiner per node (was: Combiner per container) Combiner per node

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452642#comment-13452642 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- Ah, I'm sorry for confusing

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13455840#comment-13455840 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- Chris and Karthik, Thank you

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13455842#comment-13455842 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- s/Do you have some result to/Do

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458359#comment-13458359 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- bq. For the benchmark, we were

[jira] [Updated] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4502: -- Attachment: design_v2.pdf I redesigned its architecture with the ideas inspired by

[jira] [Commented] (MAPREDUCE-4253) Tests for mapreduce-client-core are lying under mapreduce-client-jobclient

[ https://issues.apache.org/jira/browse/MAPREDUCE-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13463473#comment-13463473 ] Tsuyoshi OZAWA commented on MAPREDUCE-4253: --- I thought this jira was

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465434#comment-13465434 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I've read the overview of the

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465436#comment-13465436 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- BTW, I'm starting to implement

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485781#comment-13485781 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I uploaded the slides about

[jira] [Updated] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4502: -- Attachment: MAPREDUCE-4525-pof.diff I've prototyped proof of concept code of

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13527037#comment-13527037 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I'll create subtasks under

[jira] [Created] (MAPREDUCE-4863) Adding aggregationWaitMap for node-level combiner.

Tsuyoshi OZAWA created MAPREDUCE-4863: - Summary: Adding aggregationWaitMap for node-level combiner. Key: MAPREDUCE-4863 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4863 Project: Hadoop

[jira] [Created] (MAPREDUCE-4864) Extending RPC over umblical protocol getAggregationTargets() for node-level combiner.

Tsuyoshi OZAWA created MAPREDUCE-4864: - Summary: Extending RPC over umblical protocol getAggregationTargets() for node-level combiner. Key: MAPREDUCE-4864 URL:

[jira] [Updated] (MAPREDUCE-4864) Extending RPC over umblical protocol, getAggregationTargets(), for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4864: -- Summary: Extending RPC over umblical protocol, getAggregationTargets(), for

[jira] [Updated] (MAPREDUCE-4864) Adding new umbilical protocol RPC, getAggregationTargets(), for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4864: -- Summary: Adding new umbilical protocol RPC, getAggregationTargets(), for

[jira] [Created] (MAPREDUCE-4865) Launching node-level combiner the end stage of MapTask

Tsuyoshi OZAWA created MAPREDUCE-4865: - Summary: Launching node-level combiner the end stage of MapTask Key: MAPREDUCE-4865 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4865 Project:

[jira] [Updated] (MAPREDUCE-4864) Adding new umbilical protocol RPC, getAggregationTargets(), for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4864: -- Component/s: tasktracker mrv2 Adding new umbilical protocol

[jira] [Updated] (MAPREDUCE-4865) Launching node-level combiner at the end stage of MapTask

[ https://issues.apache.org/jira/browse/MAPREDUCE-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4865: -- Summary: Launching node-level combiner at the end stage of MapTask (was:

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534471#comment-13534471 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I've found a bug when I

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537783#comment-13537783 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I've fixed the bug! Now I start

[jira] [Created] (MAPREDUCE-4910) Adding AggregationWaitMap to some components(MRAppMaster, TaskAttemptListener, JobImpl, MapTaskImpl).

Tsuyoshi OZAWA created MAPREDUCE-4910: - Summary: Adding AggregationWaitMap to some components(MRAppMaster, TaskAttemptListener, JobImpl, MapTaskImpl). Key: MAPREDUCE-4910 URL:

[jira] [Created] (MAPREDUCE-4911) Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

Tsuyoshi OZAWA created MAPREDUCE-4911: - Summary: Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf Key: MAPREDUCE-4911 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4911

[jira] [Updated] (MAPREDUCE-4863) Adding aggregationWaitMap for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4863: -- Attachment: 0002-Adding-aggregationWaitMap-for-node-level-combiner.patch First

[jira] [Updated] (MAPREDUCE-4864) Adding new umbilical protocol RPC, getAggregationTargets(), for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4864: -- Attachment: 0001-Adding-new-umbilical-protocol-RPC-getAggregationTarg.patch

[jira] [Updated] (MAPREDUCE-4865) Launching node-level combiner at the end stage of MapTask and ignoring aggregated inputs at ReduceTask

[ https://issues.apache.org/jira/browse/MAPREDUCE-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4865: -- Summary: Launching node-level combiner at the end stage of MapTask and ignoring

[jira] [Updated] (MAPREDUCE-4865) Launching node-level combiner at the end stage of MapTask and ignoring aggregated inputs at ReduceTask

[ https://issues.apache.org/jira/browse/MAPREDUCE-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4865: -- Attachment: 0004-Changed-Mappers-and-Reducers-to-support-Node-level-a.patch

[jira] [Updated] (MAPREDUCE-4910) Adding AggregationWaitMap to some components(MRAppMaster, TaskAttemptListener, JobImpl, MapTaskImpl).

[ https://issues.apache.org/jira/browse/MAPREDUCE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4910: -- Attachment: 0003-Adding-AggregationWaitMap-to-some-components-MRAppMa.patch

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543699#comment-13543699 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I've spitted this task into

[jira] [Commented] (MAPREDUCE-4910) Adding AggregationWaitMap to some components(MRAppMaster, TaskAttemptListener, JobImpl, MapTaskImpl).

[ https://issues.apache.org/jira/browse/MAPREDUCE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543701#comment-13543701 ] Tsuyoshi OZAWA commented on MAPREDUCE-4910: --- Note that this version supports

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543705#comment-13543705 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- The forked trunk is available

[jira] [Updated] (MAPREDUCE-4910) Adding AggregationWaitMap to some components(MRAppMaster, TaskAttemptListener, JobImpl, MapTaskImpl).

[ https://issues.apache.org/jira/browse/MAPREDUCE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4910: -- Attachment: 0004-Add-AggregationWaitMap-to-some-components-MRAppMaste.patch

[jira] [Updated] (MAPREDUCE-4864) Adding new umbilical protocol RPC, getAggregationTargets(), for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4864: -- Attachment:

[jira] [Updated] (MAPREDUCE-4865) Launching node-level combiner at the end stage of MapTask and ignoring aggregated inputs at ReduceTask

[ https://issues.apache.org/jira/browse/MAPREDUCE-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4865: -- Attachment:

[jira] [Updated] (MAPREDUCE-4911) Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

[ https://issues.apache.org/jira/browse/MAPREDUCE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4911: -- Attachment: MAPREDUCE-4911.patch Add JobConf to configuration about node-level

[jira] [Updated] (MAPREDUCE-4911) Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

[ https://issues.apache.org/jira/browse/MAPREDUCE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4911: -- Target Version/s: trunk Affects Version/s: trunk Add node-level

[jira] [Work started] (MAPREDUCE-4911) Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

[ https://issues.apache.org/jira/browse/MAPREDUCE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-4911 started by Tsuyoshi OZAWA. Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

[jira] [Updated] (MAPREDUCE-4911) Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

[ https://issues.apache.org/jira/browse/MAPREDUCE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4911: -- Status: Patch Available (was: In Progress) Add node-level aggregation flag

[jira] [Work started] (MAPREDUCE-4863) Adding aggregationWaitMap for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-4863 started by Tsuyoshi OZAWA. Adding aggregationWaitMap for node-level combiner. --

[jira] [Work started] (MAPREDUCE-4864) Adding new umbilical protocol RPC, getAggregationTargets(), for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-4864 started by Tsuyoshi OZAWA. Adding new umbilical protocol RPC, getAggregationTargets(), for node-level combiner.

[jira] [Updated] (MAPREDUCE-4864) Adding new umbilical protocol RPC, getAggregationTargets(), for node-level combiner.

[ https://issues.apache.org/jira/browse/MAPREDUCE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4864: -- Affects Version/s: trunk Adding new umbilical protocol RPC,

[jira] [Work started] (MAPREDUCE-4910) Adding AggregationWaitMap to some components(MRAppMaster, TaskAttemptListener, JobImpl, MapTaskImpl).

[ https://issues.apache.org/jira/browse/MAPREDUCE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-4910 started by Tsuyoshi OZAWA. Adding AggregationWaitMap to some components(MRAppMaster, TaskAttemptListener, JobImpl, MapTaskImpl).

[jira] [Work started] (MAPREDUCE-4865) Launching node-level combiner at the end stage of MapTask and ignoring aggregated inputs at ReduceTask

[ https://issues.apache.org/jira/browse/MAPREDUCE-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-4865 started by Tsuyoshi OZAWA. Launching node-level combiner at the end stage of MapTask and ignoring aggregated inputs at ReduceTask

[jira] [Work started] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-4502 started by Tsuyoshi OZAWA. Multi-level aggregation with combining the result of maps per node/rack

[jira] [Updated] (MAPREDUCE-4911) Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

[ https://issues.apache.org/jira/browse/MAPREDUCE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4911: -- Attachment: MAPREDUCE-4911.2.patch Test failure is caused by MAPREDUCE-4948, but

[jira] [Commented] (MAPREDUCE-4502) Multi-level aggregation with combining the result of maps per node/rack

[ https://issues.apache.org/jira/browse/MAPREDUCE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13558532#comment-13558532 ] Tsuyoshi OZAWA commented on MAPREDUCE-4502: --- I've written updated patch sets

[jira] [Updated] (MAPREDUCE-4911) Add node-level aggregation flag feature(setLocalAggregation(boolean)) to JobConf

[ https://issues.apache.org/jira/browse/MAPREDUCE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4911: -- Attachment: MAPREDUCE-4911.3.patch Fixed Javadoc warnings again.

[jira] [Updated] (MAPREDUCE-4911) Add node-level aggregation flag feature(setNodeLevelAggregation(boolean)) to JobConf

[ https://issues.apache.org/jira/browse/MAPREDUCE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4911: -- Summary: Add node-level aggregation flag feature(setNodeLevelAggregation(boolean))

  1   2   3   4   5   6   7   >