[GitHub] [spark] WeichenXu123 commented on pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions
WeichenXu123 commented on PR #37734: URL: https://github.com/apache/spark/pull/37734#issuecomment-1384095881 Merged to apache/spark master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions
WeichenXu123 commented on PR #37734: URL: https://github.com/apache/spark/pull/37734#issuecomment-1336407074 @leewyang Does `df.limit(10).cache().withColumn` address the issue ? @HyukjinKwon Looks like https://github.com/apache/spark/pull/37734#issuecomment-1315678614 is a regression, does spark optimizer changed ? @mengxr Could you make a final review ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions
WeichenXu123 commented on PR #37734: URL: https://github.com/apache/spark/pull/37734#issuecomment-1311877149 @mengxr Could you make a final pass ? The PR is LGTM once all my comments addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions
WeichenXu123 commented on PR #37734: URL: https://github.com/apache/spark/pull/37734#issuecomment-1306665478 Some more comments, mainly about doc and adding more data checking. PR is near ready! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions
WeichenXu123 commented on PR #37734: URL: https://github.com/apache/spark/pull/37734#issuecomment-1306594350 > > Pls also fix the linter failure: https://github.com/leewyang/spark/actions/runs/3397174449/jobs/5649073867#step:16:71 > > Updated to latest master, which got rid of the linter error, but it added a new "appveyor" check, which seems to be failing in some SparkR tests. Not sure what to do with that one. The Ci failure has been fixed by https://github.com/apache/spark/commit/9cd55052ccefc1421d30bfc751e2a013973d3ac6 You can merge master to address the CI failure @leewyang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions
WeichenXu123 commented on PR #37734: URL: https://github.com/apache/spark/pull/37734#issuecomment-1305628531 Pls also fix the linter failure: https://github.com/leewyang/spark/actions/runs/3397174449/jobs/5649073867#step:16:71 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions
WeichenXu123 commented on PR #37734: URL: https://github.com/apache/spark/pull/37734#issuecomment-1234139636 But I think we'd better design and discuss the API first. @mengxr Do you have any suggestions ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
