Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20920
Change subject: IMPALA-12727: Reduce IO threads for non-TARGET_FILESYSTEM filesystems ...................................................................... IMPALA-12727: Reduce IO threads for non-TARGET_FILESYSTEM filesystems The DiskIoMgr starts a large number of threads for each different type of object store, most of which are idle. For development, this slows down processing minidumps and debugging with gdb. This adds an option "reduce_disk_io_threads" to bin/start-impala-cluster.py that sets the thread count startup parameter to one for any filesystem that is not the TARGET_FILESYSTEM. On a typical development setup running against HDFS, this reduces the number of DiskIoMgr threads by 150 and the HDFS monitoring threads by 150 as well. This option is enabled by default. It can disabled by setting --reduce_disk_io_threads=False for bin/start-impala-cluster.py. Separately, DiskIoMgr should be modified to reduce the number of threads it spawns in general. Testing: - Hand tested this on my local development system Change-Id: Ic8ee1fb1f9b9fe65d542d024573562b3bb120b76 --- M bin/start-impala-cluster.py 1 file changed, 33 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/20920/1 -- To view, visit http://gerrit.cloudera.org:8080/20920 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic8ee1fb1f9b9fe65d542d024573562b3bb120b76 Gerrit-Change-Number: 20920 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <[email protected]>
