Apache community, I just want to confirm my understanding of parquet files. I have a streaming set of data that may be produced in realtime. Ideally I would stream it into a parquet file ( and if the process crashes, still be able to read some part of what was streamed ). I can do this with arrow output ( last written batch would be readable ), but as far as I can tell its not possible with parquet writers. Is my understanding correct?
Thanks - Rob DISCLAIMER: This e-mail message and any attachments are intended solely for the use of the individual or entity to which it is addressed and may contain information that is confidential or legally privileged. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, copying or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately and permanently delete this message and any attachments.
