Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16664
Basically I see no reason to add some specific parameter to a listener API
that is meant to be generic which already contains reference to QueryExecution.
What are you going to do if next time you want to find some other information
with a take or collect query? Do you go in and add another interface for that?
If the goal is to expose information for writing data out properly, then
just make it work with the existing interface and fix the issue that using
DataFrameWriter doesn't call the callback (and doesn't have the correct
information set in QueryExecution).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
