Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20623 )
Change subject: IMPALA-12516: Set HDFS limit based on memlock ...................................................................... IMPALA-12516: Set HDFS limit based on memlock With RHEL 8 on AWS Graviton instances, dfs.datanode.max.locked.memory=64000 is insufficient to run query_test/test_hdfs_caching.py::TestHdfsCaching::test_table_is_cached. Sets dfs.datanode.max.locked.memory based on 'ulimit -l', and sets memlock to 64MB in bootstrap_system.sh to match modern defaults and provide space for future HDFS caching tests. New setting can be seen in admin output like node-1 will use ports DATANODE_PORT=31002, DATANODE_HTTP_PORT=31012, DATANODE_IPC_PORT=31022, DATANODE_HTTPS_PORT=31032, DATANODE_CLIENT_PORT=31042, NODEMANAGER_PORT=31102, NODEMANAGER_LOCALIZER_PORT=31122, NODEMANAGER_WEBUI_PORT=31142, KUDU_TS_RPC_PORT=31202, and KUDU_TS_WEBUI_PORT=31302; DATANODE_LOCKED_MEM=65536000 Change-Id: I7722ddd0c7fbd9bbd1979503952b7522b808194a Reviewed-on: http://gerrit.cloudera.org:8080/20623 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Joe McDonnell <[email protected]> --- M bin/bootstrap_system.sh M testdata/cluster/admin M testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl 3 files changed, 19 insertions(+), 4 deletions(-) Approvals: Impala Public Jenkins: Verified Joe McDonnell: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/20623 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7722ddd0c7fbd9bbd1979503952b7522b808194a Gerrit-Change-Number: 20623 Gerrit-PatchSet: 12 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Laszlo Gaal <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]>
