[
https://issues.apache.org/jira/browse/IMPALA-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778712#comment-16778712
]
ASF subversion and git services commented on IMPALA-8188:
---------------------------------------------------------
Commit 7ba6aa64a2455df30524ad42b10d7fd9227efacb in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=7ba6aa6 ]
IMPALA-8188: Fix DiskInfo::GetDeviceNames() for NVME disks
DiskInfo::GetDeviceNames() assumes a device name pattern where
the last digits can be removed to get the main disk device
(i.e. sda3 -> sda). NVME devices have a naming pattern of
nvme{device_id}n{namespace_id}p{partition_id} where the main
disk device removes the p{partition_id} part. This difference
in naming prevents DiskInfo::GetDeviceNames() from looking
up the associated information in /sys/block to find whether
it is a rotational device (and thus use an appropriate
number of DiskIo threads).
This adds a condition to detect an NVME device name and
handle it correctly.
Testing:
- Added unit test for the function used for detection
- Tested this on an AWS m5-4xlarge, which is a Nitro instance
type that exposes the main disk as an NVME device.
Change-Id: I4d5bd93b4b09682a8c8248e7aa123d23d27cfeb4
Reviewed-on: http://gerrit.cloudera.org:8080/12557
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Philip Zeyliger <[email protected]>
> Some SSDs are not properly detected as non-rotational
> -----------------------------------------------------
>
> Key: IMPALA-8188
> URL: https://issues.apache.org/jira/browse/IMPALA-8188
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.2.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
>
> Here is an example Impala log:
>
> {noformat}
> I0211 10:50:40.650727 18344 init.cc:288] Disk Info:
> Num disks 2:
> nvme0n (rotational=true)
> nvme0n1p (rotational=true){noformat}
> I logged into an equivalent machine, and the OS sees these as not rotational:
>
>
> {noformat}
> # cat /sys/block/nvme0n1/queue/rotational
> 0
> {noformat}
> Device names that end in a number get trimmed (i.e. /dev/sda2 becomes
> /dev/sda). See
> [https://github.com/apache/impala/blob/master/be/src/util/disk-info.cc#L73-L74]
> These devices don't follow that pattern, so we don't find the right files.
> Neither /sys/block/nvme0n nor /sys/block/nvme0n1p exist, so both fall back to
> being rotational.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
