HyukjinKwon commented on pull request #29067: URL: https://github.com/apache/spark/pull/29067#issuecomment-659306235
Okay, I took a look too. I think the actual API here itself is not very big as it seems. It is a developer API. The idea makes sense to me too. I guess the main concern is that about leveraging DSv2 (?). I remember `ColumnarToRowExec` <> Arrow batches can merged as well around one year ago. It might be great if we have JIRAs and track these with an explicit plan rather than just indefinitely leaving half-baked APIs. With the explicit plans, I think we're good to go without SPIP. Would that alleviate your concern about SPIP @maropu? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
