jinchengchenghh commented on issue #28866: URL: https://github.com/apache/arrow/issues/28866#issuecomment-2095354665
I also have the same requirement. I use dataset scan CSV, but cannot set the parseOptions and readOptions by java ScanOptions, I find ScanBuilder can set CsvFragmentScanOptions in C++ side, so the question is how can we serialize and deserialize CsvFragmentScanOptions. I have 2 proposals: 1. Like the projection and filter serialization, we could use substrait ExtendedExpression::AdvancedExtension to serialize CsvFragmentScanOptions 2. Use protobuf message to serialize directly We can do as this PR does. https://github.com/apache/incubator-gluten/pull/3393/files The first proposal will be ambiguous, because user does not know how to generate the serialized binary, they should be familliar with the AdvancedExtension detail implementation, I would prefer the second. I can help to implement this feature, can you help to confirm the proposal? @westonpace -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
