Joe McDonnell created IMPALA-12669:
--------------------------------------
Summary: DiskIoMgr should not allocate threads for loop devices
Key: IMPALA-12669
URL: https://issues.apache.org/jira/browse/IMPALA-12669
Project: IMPALA
Issue Type: Improvement
Components: Backend
Affects Versions: Impala 4.4.0
Reporter: Joe McDonnell
On my development machine, Ubuntu's snap package system has various /snap
directories mounted as loop devices:
{noformat}
/dev/loop3 83328 83328 0 100% /snap/gtk-common-themes/1534
/dev/loop2 508928 508928 0 100% /snap/gnome-42-2204/132
/dev/loop5 41856 41856 0 100% /snap/snapd/20290
/dev/loop4 47104 47104 0 100% /snap/snap-store/638
/dev/loop7 508928 508928 0 100% /snap/gnome-42-2204/141
/dev/loop8 75776 75776 0 100% /snap/core22/864
/dev/loop9 65024 65024 0 100% /snap/core20/2015
/dev/loop10 358144 358144 0 100% /snap/gnome-3-38-2004/140
/dev/loop12 12672 12672 0 100% /snap/snap-store/959
{noformat}
DiskIoMgr is allocating 8 threads for each of these devices, so my system has
344 disk-io-mgr threads of which 152 are for loop devices. This also causes
there to be an equivalent number of hdfs-monitor threads. This makes minidumps
larger and take longer to process. It also slows down pstack and other tools.
These loop devices are not real devices and they shouldn't matter in a
production cluster. DiskIoMgr should not maintain a queue and spawn threads for
them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
