raphaelauv opened a new pull request, #60031: URL: https://github.com/apache/airflow/pull/60031
current spark provider need the package pyspark ( more than 400mb ) where I would like to only use the spark-client ( spark-connect ) to trigger a spark job ( that is 1.5mb ) this is a breaking change but it will make things lighter by default , wdyt ? thanks ( btw spark-client is only available since spark 4.0 and need "grpcio >= 1.44.0" ( that conflict with apache-beam ) ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
