Talk info share - "Advancing GPU Analytics with RAPIDS Accelerator for Spark and Alluxio"

2021-04-25 Thread Jasmine Wang
Hi guys, Wanted to share an upcoming free online tech talk on April 27th Tuesday at 10AM PT by NVIDIA on "Advancing GPU Analytics with RAPIDS Accelerator for Spark and Alluxio" There will be a live Q after the talk in case anyone is interested. Registration is here

Re: pyspark sql load with path of special character

2021-04-25 Thread Stephen Coy
It probably does not like the colons in the path name “…20:04:27+00:00/…”, especially if you’re running on a Windows box. On 24 Apr 2021, at 1:29 am, Regin Quinoa mailto:sweatr...@gmail.com>> wrote: Hi, I am using pyspark sql to load files into table following ```LOAD DATA LOCAL INPATH

Re: Is a Hive installation necessary for Spark SQL?

2021-04-25 Thread Dennis Suhari
Hi, you can also load other data source without Hive using spark read format into a spark Dataframe . From there you can also combine the results using the Dataframe world. The use cases of hive is to have a common Abstraction layer when you want to do data tagging, access management under

Is a Hive installation necessary for Spark SQL?

2021-04-25 Thread krchia
Does it make sense to keep a Hive installation when your parquet files come with a transactional metadata layer like Delta Lake / Apache Iceberg? My understanding from this: https://github.com/delta-io/delta/issues/85 is that Hive is no longer necessary other than discovering where the table is