Pytorch support in SPARK 3.0

Hi, I am looking into this: https://github.com/dmmiller612/sparktorch/tree/master/sparktorch Is there any inbuilt support provided in SPARK 3.0 for PyTorch support? Regards, Gourav Sengupta

Cannot read case-sensitive Glue table backed by Parquet

Spark version: 2.4.2 on Amazon EMR 5.24.0 I have a Glue Catalog table backed by S3 Parquet directory. The Parquet files have case-sensitive column names (like /lastModified/). It doesn't matter what I do, I get lowercase column names (/lastmodified/) when reading the Glue Catalog table with

Spark 2.4.4, RPC encryption and Python

Hi everybody, I am currently testing Spark 2.4.4 with the following new settings: spark.authenticate true spark.io.encryption.enabled true spark.io.encryption.keySizeBits 256 spark.io.encryption.keygen.algorithm HmacSHA256 spark.network.crypto.enabled true