[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-------------------------------------
Attachment: TEZ-3813.005.patch
Removed int size from MemoryFetchedInput.
*JOL Dump:*
+After:+
Internals:
{code}
# Running 64-bit HotSpot VM.
# Using compressed oop with 3-bit shift.
# Using compressed klass with 3-bit shift.
# Objects are 8 bytes aligned.
# Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
# Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
Instantiated the sample instance via public
org.apache.tez.runtime.library.common.shuffle.MemoryFetchedInput(long,org.apache.tez.runtime.library.common.InputAttemptIdentifier,org.apache.tez.runtime.library.common.shuffle.FetchedInputCallback)
org.apache.tez.runtime.library.common.shuffle.MemoryFetchedInput object
internals:
OFFSET SIZE
TYPE DESCRIPTION VALUE
0 4
(object header) 01 00 00 00 (00000001 00000000
00000000 00000000) (1)
4 4
(object header) 00 00 00 00 (00000000 00000000
00000000 00000000) (0)
8 4
(object header) 7d 12 01 f8 (01111101 00010010
00000001 11111000) (-134147459)
12 4
int FetchedInput.id 0
16 1
byte FetchedInput.state 0
17 3
(alignment/padding gap)
20 4
org.apache.tez.runtime.library.common.InputAttemptIdentifier
FetchedInput.inputAttemptIdentifier null
24 4
org.apache.tez.runtime.library.common.shuffle.FetchedInputCallback
FetchedInput.callback null
28 4
byte[] MemoryFetchedInput.byteArray []
Instance size: 32 bytes
Space losses: 3 bytes internal + 0 bytes external = 3 bytes total
{code}
Footprint:
{code}
# Running 64-bit HotSpot VM.
# Using compressed oop with 3-bit shift.
# Using compressed klass with 3-bit shift.
# Objects are 8 bytes aligned.
# Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
# Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
Instantiated the sample instance via public
org.apache.tez.runtime.library.common.shuffle.MemoryFetchedInput(long,org.apache.tez.runtime.library.common.InputAttemptIdentifier,org.apache.tez.runtime.library.common.shuffle.FetchedInputCallback)
org.apache.tez.runtime.library.common.shuffle.MemoryFetchedInput@215be6bbd
footprint:
COUNT AVG SUM DESCRIPTION
1 16 16 [B
1 32 32
org.apache.tez.runtime.library.common.shuffle.MemoryFetchedInput
2 48 (total)
{code}
> Reduce Object size of MemoryFetchedInput for large jobs
> -------------------------------------------------------
>
> Key: TEZ-3813
> URL: https://issues.apache.org/jira/browse/TEZ-3813
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Muhammad Samir Khan
> Assignee: Muhammad Samir Khan
> Attachments: TEZ-3813.001.patch, TEZ-3813.002.patch,
> TEZ-3813.003.patch, TEZ-3813.004.patch, TEZ-3813.005.patch
>
>
> Same as TEZ-3752 for the unordered case. MemoryFetchedInput has a
> BoundedByteArrayOutputStream that is not used (only the underlying byte[] is
> used).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
