[
https://issues.apache.org/jira/browse/IMPALA-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell reassigned IMPALA-12669:
--------------------------------------
Assignee: Joe McDonnell
> DiskIoMgr should not allocate threads for loop devices
> ------------------------------------------------------
>
> Key: IMPALA-12669
> URL: https://issues.apache.org/jira/browse/IMPALA-12669
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 4.4.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
>
> On my development machine, Ubuntu's snap package system has various /snap
> directories mounted as loop devices:
> {noformat}
> /dev/loop3 83328 83328 0 100%
> /snap/gtk-common-themes/1534
> /dev/loop2 508928 508928 0 100% /snap/gnome-42-2204/132
> /dev/loop5 41856 41856 0 100% /snap/snapd/20290
> /dev/loop4 47104 47104 0 100% /snap/snap-store/638
> /dev/loop7 508928 508928 0 100% /snap/gnome-42-2204/141
> /dev/loop8 75776 75776 0 100% /snap/core22/864
> /dev/loop9 65024 65024 0 100% /snap/core20/2015
> /dev/loop10 358144 358144 0 100% /snap/gnome-3-38-2004/140
> /dev/loop12 12672 12672 0 100% /snap/snap-store/959
> {noformat}
> DiskIoMgr is allocating 8 threads for each of these devices, so my system has
> 344 disk-io-mgr threads of which 152 are for loop devices. This also causes
> there to be an equivalent number of hdfs-monitor threads. This makes
> minidumps larger and take longer to process. It also slows down pstack and
> other tools.
> These loop devices are not real devices and they shouldn't matter in a
> production cluster. DiskIoMgr should not maintain a queue and spawn threads
> for them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
