I have loaded docker files into my docker repository on docker hub and it is public.
These are built on Spark 3.1.2 OR 3.1.1, with Scala 2.12 and with Java 11 OR Java 8 on OS jre-slim-buster. The ones built on 3.1.1 with Java 8 should work with GCP No additional packages are added to PySpark in docker. They can be downloaded from here <https://hub.docker.com/repository/docker/michtalebzadeh/spark_dockerfiles/tags?page=1&ordering=last_updated> How to download. The instructions are there. Example: docker pull michtalebzadeh/spark_dockerfiles:spark-py-3.1.1-scala_2.12-8-jre-slim-buster Let me know if any issues HTH view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
