[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Committed to branch-3 and master. > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch, > HIVE-20649.3.patch, HIVE-20649.4.patch, HIVE-20649.5.patch, HIVE-20649.6.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: HIVE-20649.6.patch > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch, > HIVE-20649.3.patch, HIVE-20649.4.patch, HIVE-20649.5.patch, HIVE-20649.6.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: HIVE-20649.5.patch > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch, > HIVE-20649.3.patch, HIVE-20649.4.patch, HIVE-20649.5.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: HIVE-20649.4.patch > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch, > HIVE-20649.3.patch, HIVE-20649.4.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: HIVE-20649.3.patch > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch, > HIVE-20649.3.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: (was: HIVE-20649.2.patch) > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: HIVE-20649.2.patch > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: HIVE-20649.2.patch > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch, HIVE-20649.2.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Status: Patch Available (was: Open) > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20649) LLAP aware memory manager for Orc writers
[ https://issues.apache.org/jira/browse/HIVE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20649: - Attachment: HIVE-20649.1.patch > LLAP aware memory manager for Orc writers > - > > Key: HIVE-20649 > URL: https://issues.apache.org/jira/browse/HIVE-20649 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20649.1.patch > > > ORC writer has its own memory manager that assumes memory usage or memory > available based on JVM heap (MemoryMX bean). This works on tez container mode > execution model but not in LLAP where container sizes (and Xmx) are typically > high and there are multiple executors per LLAP daemon. This custom memory > manager should be aware of memory bounds per executor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
