arun990 commented on issue #3127: URL: https://github.com/apache/hudi/issues/3127#issuecomment-865442514
Hi @n3nash, Thanks for quick update. Yes, followed these instructions on using pyspark with Hudi -> https://hudi.apache.org/docs/spark_quick-start-guide.html#pyspark-example and its working fine for spark 2.4.7 on google cloud for cow tables. But we have spark 2.4.0 with python 2.7 (and python 3.6) for DEV, UAT, PROPROD and PROD environments. I need to make it work for spark 2.4.0. Following this, provisioned spark 2.4.0 cluster on aws and attempted to save hudi cow tables to hdfs (without hive and set the hive sync as false). In that process of making it work for spark 2.4.0, getting the shared error. spark 2.4.0 is important for us as we have all environments with it and its only next year end we move to spark 2.4.5. Pls help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
