Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20920
to look at the new patch set (#2).
Change subject: IMPALA-12727: Reduce IO threads for non-TARGET_FILESYSTEM
filesystems
......................................................................
IMPALA-12727: Reduce IO threads for non-TARGET_FILESYSTEM filesystems
The DiskIoMgr starts a large number of threads for each different
type of object store, most of which are idle. For development,
this slows down processing minidumps and debugging with gdb.
This adds an option "reduce_disk_io_threads" to bin/start-impala-cluster.py
that sets the thread count startup parameter to one for any filesystem
that is not the TARGET_FILESYSTEM. On a typical development setup
running against HDFS, this reduces the number of DiskIoMgr threads
by 150 and the HDFS monitoring threads by 150 as well. This option is
enabled by default. It can disabled by setting --reduce_disk_io_threads=False
for bin/start-impala-cluster.py.
Separately, DiskIoMgr should be modified to reduce the number of
threads it spawns in general.
Testing:
- Hand tested this on my local development system
Change-Id: Ic8ee1fb1f9b9fe65d542d024573562b3bb120b76
---
M bin/start-impala-cluster.py
1 file changed, 33 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/20920/2
--
To view, visit http://gerrit.cloudera.org:8080/20920
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic8ee1fb1f9b9fe65d542d024573562b3bb120b76
Gerrit-Change-Number: 20920
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
