Frank Zhao created CRAIL-86:
-------------------------------
Summary: What's GPU memory support status?
Key: CRAIL-86
URL: https://issues.apache.org/jira/browse/CRAIL-86
Project: Apache Crail
Issue Type: Wish
Affects Versions: 1.2
Environment: Linux, Nvidia GPU, data in NVMe
Reporter: Frank Zhao
I just have a few questions on GPU memory support related to Heterogeneity.
1) Currently how does Crail handle or exploit GPU memory as one of tier? I saw
a diagram showing GPU/GPUDirect, however, when exploring source codes including
Crail/DiSNI/jNVMf etc, I didn't find any codes related to GPU/GPUDirect/CUDA
etc. so curious support GPU memory as tier status or any plan in roadmap?
2) a specific question is, Does Crail support P2P from NVMe to GPU (or vise
visa)?
3) in another introduction page
({color:#333333}[https://crail.apache.org/overview/index.html#fs]), it gives an
high level description as below:{color}
"For instance, an application may use the Crail GPU tier to store data. In that
case, sorting can be pushed to the GPU, rather than fetching the data into main
memory and sorting it on the CPU. In other cases, the application may know the
data types in advance and use the information to simplify sorting (e.g. use
Radix sort instead TimSort). "
looks to me, that likely needs a GPU edition of sorting algorithm (such as via
CUDA) to process data at GPU (where data resides), question is who is providing
such GPU edition of sorting? I didn't see Crail did that from current codes, or
is it by Spark or 3rd libraries?
{{spark.crail.shuffle.serializer spark.crail.shuffle.sorter}}
{{thanks a lot!}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
