Hi, Crail this's a very interesting project, I just have a few questions on GPU memory support related to Heterogeneity. 1) Currently how does Crail handle or exploit GPU memory as one of tier? I saw a diagram showing GPU/GPUDirect, however, when exploring source codes including Crail/DiSNI/jNVMf etc, I didn't find any codes related to GPU/GPUDirect/CUDA etc. so curious support GPU memory as tier status or any plan in roadmap? 2) a specific question is, Does Crail support P2P from NVMe to GPU (or vise visa)? 3) in another introduction page ( https://crail.apache.org/overview/index.html#fs), it gives an high level description as below: "For instance, an application may use the Crail GPU tier to store data. In that case, sorting can be pushed to the GPU, rather than fetching the data into main memory and sorting it on the CPU. In other cases, the application may know the data types in advance and use the information to simplify sorting (e.g. use Radix sort instead TimSort). " looks to me, that likely needs a GPU edition of sorting algorithm (such as via CUDA) to process data at GPU (where data resides), question is who is providing such GPU edition of sorting? I didn't see Crail did that from current codes, or is it by Spark or 3rd libraries?
spark.crail.shuffle.serializer spark.crail.shuffle.sorter Thanks a lot ------------------------------------------ zha...@gmail.com
