WeichenXu123 commented on PR #53150: URL: https://github.com/apache/spark/pull/53150#issuecomment-3580698495
> Arrow isn't intended for long term storage it's intended as a wire protocol -- I don't love using it for persisting models. I'm -0.9 on this change for now. Parquet seems like a better choice most likely. @holdenk SparkML model "saveToLocal" is an **internal** API and it is only used in SparkConnect server side to cache model **within one session**, it is not used for **long term storage**. So it should be fine to use it here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
