Hi, I see that Spark Context methods are not supported in Spark Connect. There are many common use cases eg. broadcast machine learning model weights to all executors, so no need to fetch individually.
This makes migration of workloads to Spark Connect tougher. I know there are plans to add more & more functionalities to Spark Connect eg. SparkMLlib. My question is: Is there an ETA to support broadcasts in Spark Connect API too? Or it won't be supported due to how Spark Connect is designed, maybe separate JVMs/security etc? I can also consider implementing this based on the above answers and effort. Regards, Deependra
