[
https://issues.apache.org/jira/browse/IMPALA-15187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18096778#comment-18096778
]
ASF subversion and git services commented on IMPALA-15187:
----------------------------------------------------------
Commit 6da4ba8a504ab725aa102cf5d424b1286be06243 in impala's branch
refs/heads/master from David Rorke
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6da4ba8a5 ]
IMPALA-15187: Reduce excessive Trino memory allocation
The impala-minicluster-trino image used for Trino interop tests is attempting to
allocate 50GB for the JVM during startup leading to container startup failures.
The 50GB allocation (on a 64 GB host) is the result of the following settings in
/etc/trino/jvm.config which will allocate 80% of host memory during startup:
-XX:InitialRAMPercentage=80
-XX:MaxRAMPercentage=80
The fix is to reduce these percentages to something more reasonable:
-XX: InitialRAMPercentage=10
-XX: MaxRAMPercentage=30
Change-Id: I692e5363054b045018d3b4cecf24e6b6c4a4c269
Reviewed-on: http://gerrit.cloudera.org:8080/24589
Reviewed-by: Csaba Ringhofer <[email protected]>
Tested-by: Csaba Ringhofer <[email protected]>
> TestIcebergTrinoInterop trino container startup failing from excessive mem
> allocation request
> ---------------------------------------------------------------------------------------------
>
> Key: IMPALA-15187
> URL: https://issues.apache.org/jira/browse/IMPALA-15187
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: David Rorke
> Assignee: David Rorke
> Priority: Major
>
> The Trino container is failing to start during TestIcebergTrinoInterop
> because the JVM is attempting to commit 50GB of memory during startup:
> {noformat}
> OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000fff2f6000000,
> 53552873472, 0) failed; error='Not enough space' (errno=12) # # There is
> insufficient memory for the Java Runtime Environment to continue. # Native
> memory allocation (mmap) failed to map 53552873472 bytes. Error detail:
> committing reserved memory.{noformat}
> The reason appears to be the following in the jvm.config file for the Docker
> image which tells the JVM to allocate 80% of the total container memory at
> startup:
> {noformat}
> -XX:InitialRAMPercentage=80
> -XX:MaxRAMPercentage=80{noformat}
> There are no container memory limits so these percentages will be applied to
> the total host memory (64GB in this case). minicluster_trino should have a
> much lower memory requirement than this and these percentages could be
> reduced to something like 10% initial and 30% max.
> So far we've only seen this failure on ARM (not on Intel). The reason isn't
> clear (some arm instances use a large 64KB page size so maybe more
> fragmentation), but in any case the 50GB allocation is excessive and should
> be reduced on both ARM and Intel.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
