[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.004.patch
Thanks [~rajesh.balamohan] for the suggestion. I moved the
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:03 PM:
---
Took a
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:04 PM:
---
Took a
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:07 PM:
---
Took a
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111794#comment-16111794
]
Muhammad Samir Khan commented on TEZ-3809:
--
Took a heap dump on ordered word count before final
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:04 PM:
---
Took a
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Attachment: TEZ-3809.001.patch
Initial patch with the changes mentioned in the description.
Muhammad Samir Khan created TEZ-3809:
Summary: The buffer size allocated for InMemoryMapOutput can be
optimized
Key: TEZ-3809
URL: https://issues.apache.org/jira/browse/TEZ-3809
Project: Apache
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Description:
Related jiras: TEZ-3752 and TEZ-3732.
-When shuffling input to memory, the
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Description:
Related jiras: TEZ-3752 and TEZ-3732.
-When shuffling input to memory, the
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.001.patch
Replace using BoundedByteArrayOutputStream with a byte array
Muhammad Samir Khan created TEZ-3813:
Summary: Reduce Object size of MemoryFetchedInput for large jobs
Key: TEZ-3813
URL: https://issues.apache.org/jira/browse/TEZ-3813
Project: Apache Tez
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113366#comment-16113366
]
Muhammad Samir Khan commented on TEZ-3809:
--
Also tested with memory-to-memory merger to check if
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113451#comment-16113451
]
Muhammad Samir Khan commented on TEZ-3813:
--
*JOL Dump:*
+Before:+
Internals:
{code}
# Running
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113454#comment-16113454
]
Muhammad Samir Khan commented on TEZ-3813:
--
Tested with filterLinesByWord and compared output
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113369#comment-16113369
]
Muhammad Samir Khan commented on TEZ-3809:
--
Request for comments [~rajesh.balamohan]/[~sseth]
>
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.004.patch
Incorporated comments from the review. Removed the alignment
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.005.patch
Fixes to findbugs etc. Added unit test for
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.001.patch
Added a new class that grows buffers via "doubling" until it
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Description: When a single-task vertex is unlucky, it lands on a very slow
node.
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.003.patch
Some changes to ensureExtraCapacity to create a separate
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Attachment: TEZ-3816.002.patch
Thanks [~sseth] for the comments. I updated the patch to put
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Attachment: TEZ-3816.003.patch
Added a couple of comments and speculationValue returns
[
https://issues.apache.org/jira/browse/TEZ-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3753:
-
Attachment: tez-3753.002.patch
Updated patch to include a unit test and some changes based
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3212:
Assignee: Muhammad Samir Khan (was: Jonathan Eagles)
> IFile throws
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096906#comment-16096906
]
Muhammad Samir Khan commented on TEZ-3212:
--
Added unit test and throw an error message when the
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.002.patch
> IFile throws NegativeArraySizeException for value sizes
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reopened TEZ-3212:
--
> IFile throws NegativeArraySizeException for value sizes between 1GB and 2GB
>
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103716#comment-16103716
]
Muhammad Samir Khan commented on TEZ-3752:
--
Ran orderedwordcount with
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.003.patch
1) Added MAX_BUFFER_SIZE not final to findbugs-exclude.xml
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103880#comment-16103880
]
Muhammad Samir Khan commented on TEZ-3752:
--
However, this test doesn't actually hit the RLE case.
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3752:
-
Attachment: TEZ-3752.001.patch
Replace using BoundedByteArrayOutputStream with a byte array
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102046#comment-16102046
]
Muhammad Samir Khan commented on TEZ-3752:
--
JOL dump:
Before:
-internals:
{code}
# Running 64-bit
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105671#comment-16105671
]
Muhammad Samir Khan commented on TEZ-3212:
--
The failing tests pass locally.
> IFile throws
Muhammad Samir Khan created TEZ-3807:
Summary: InMemoryWriter is not tested with RLE enabled
Key: TEZ-3807
URL: https://issues.apache.org/jira/browse/TEZ-3807
Project: Apache Tez
Issue
[
https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3807:
-
Attachment: TEZ-3807.001.patch
Fix testInMemoryWriter() and add test cases to
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.005.patch
Made createLargerArray() static.
> IFile throws
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3159:
Assignee: Muhammad Samir Khan
> Reduce memory utilization while serializing keys and
[
https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108089#comment-16108089
]
Muhammad Samir Khan commented on TEZ-3807:
--
Fixed both.
> InMemoryWriter is not tested with RLE
[
https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3807:
-
Attachment: TEZ-3807.002.patch
> InMemoryWriter is not tested with RLE enabled
>
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.004.patch
Added getBytes() to findbugs exclude file.
> Reduce Object
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.005.patch
Removed int size from MemoryFetchedInput.
*JOL Dump:*
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.002.patch
Fixed findbugs and release audit warnings.
> Reduce memory
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.006.patch
Fixed
> Reduce Object size of MemoryFetchedInput for large
Muhammad Samir Khan created TEZ-3816:
Summary: Ability to automatically speculate single-task vertices
Key: TEZ-3816
URL: https://issues.apache.org/jira/browse/TEZ-3816
Project: Apache Tez
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Attachment: TEZ-3816.001.patch
Added ability to LegacyTaskRuntimeEstimator to speculate
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.002.patch
Some more optimizations related to enums, similar to
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.003.patch
Fixed a NPE when getSize() is called after a free().
Removed
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Attachment: TEZ-3809.002.patch
Removed the unused imports and fixed the names for new
[
https://issues.apache.org/jira/browse/TEZ-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3283:
Assignee: Muhammad Samir Khan
> Add a max cap to the heap computation done by Tez
>
[
https://issues.apache.org/jira/browse/TEZ-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3283:
-
Attachment: tez-3283.001.patch
Added a new parameter to specify the max java non heap
[
https://issues.apache.org/jira/browse/TEZ-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3693:
Assignee: Muhammad Samir Khan
> ControlledClock is not used
>
[
https://issues.apache.org/jira/browse/TEZ-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3693:
-
Attachment: TEZ-3693.001.patch
Removed ControlledClock
> ControlledClock is not used
>
Muhammad Samir Khan created TEZ-3772:
Summary: Allow slowstart for small jobs to be treated differently
Key: TEZ-3772
URL: https://issues.apache.org/jira/browse/TEZ-3772
Project: Apache Tez
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3772:
-
Summary: Allow slowstart for small vertices to be treated differently
(was: Allow
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3772:
-
Attachment: tez-3772.001.patch
Added new configurations to distinguish between
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063760#comment-16063760
]
Muhammad Samir Khan commented on TEZ-3772:
--
For example, when there is a single reducer and some of
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3772:
-
Attachment: tez-3772.002.patch
-Updated defaults for slowstart for small vertices.
-Fixed
Muhammad Samir Khan created TEZ-3753:
Summary: Improve error message in IFile for buffer length overflow
Key: TEZ-3753
URL: https://issues.apache.org/jira/browse/TEZ-3753
Project: Apache Tez
59 matches
Mail list logo
