"So if I am going to use GPU in my job running on the spark , I still need to code the map and reduce function in cuda or in c++ and then invoke them throught jni or something like GPUEnabler , is that right ?"
Sort of. You could go through all of that work yourself, or you could use the plugin that we are going to open source in the next few days. Go to https://nvidia.com/spark and click on the contact us link. You should be able to get the information you want that way. I know from the list of spark summit talks that others are working on similar things too. Intel has a talk about some of their efforts for columnar processing on FPGAs and I think SIMD instructions too, at least going off of their talk last year. It should be an exciting time for accelerated SQL in spark. On Wed, Jun 17, 2020 at 11:17 PM charles_cai <1620075...@qq.com> wrote: > Bobby > > Thanks for your answer, it seems that I have misunderstood this paragraph > in > the website : *"GPU-accelerate your Apache Spark 3.0 data science > pipelines—without code changes—and speed up data processing and model > training while substantially lowering infrastructure costs."* . So if I am > going to use GPU in my job running on the spark , I still need to code the > map and reduce function in cuda or in c++ and then invoke them throught jni > or something like GPUEnabler , is that right ? > > thanks > Charles > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
