GitHub user SChakravorti21 edited a comment on the discussion: [C++] Footer dumps for IPC Feather/ Parquet
I would also be curious about the end to end workflow! It seems difficult to ensure that readers aren't affected by "dirty writes". I suppose if the filesystem allows swapping the footer file atomically, it would be possible for old readers to be unaffected while new readers pick up new data. But then how do old readers find out that there is new data? Or do they not care? Interesting problem! Edit: come to think of it, this seems a lot like the problem Iceberg solves 🤔 GitHub link: https://github.com/apache/arrow/discussions/46505#discussioncomment-13198805 ---- This is an automatically sent email for user@arrow.apache.org. To unsubscribe, please send an email to: user-unsubscr...@arrow.apache.org
