[GitHub] [spark] hvanhovell commented on pull request #39091: [SPARK-41527][CONNECT][PYTHON] Implement `DataFrame.observe`

2023-03-05 Thread via GitHub
hvanhovell commented on PR #39091: URL: https://github.com/apache/spark/pull/39091#issuecomment-1455327845 Merging to master/3.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] hvanhovell commented on pull request #39091: [SPARK-41527][CONNECT][PYTHON] Implement `DataFrame.observe`

2023-02-13 Thread via GitHub
hvanhovell commented on PR #39091: URL: https://github.com/apache/spark/pull/39091#issuecomment-1428036668 @beliefer will take a look today. Thanks for your hard work and patience! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] hvanhovell commented on pull request #39091: [SPARK-41527][CONNECT][PYTHON] Implement `DataFrame.observe`

2022-12-19 Thread GitBox
hvanhovell commented on PR #39091: URL: https://github.com/apache/spark/pull/39091#issuecomment-1358235780 @beliefer can we just send them as part of the `ExecutePlanResponse` at the end of the query? Doing another RPC seems a bit wasteful, and it means we have to track query state in the

[GitHub] [spark] hvanhovell commented on pull request #39091: [SPARK-41527][CONNECT][PYTHON] Implement DataFrame.observe

2022-12-16 Thread GitBox
hvanhovell commented on PR #39091: URL: https://github.com/apache/spark/pull/39091#issuecomment-1355312686 @beliefer thanks for working on this. I have one question how are we going to get the observed metrics to the client? This seems to be missing from the implementation. One of the