Hey all, We have quite a bit going on this first sync post holiday. Looking forward to seeing the Java/Python releases (especially writes in PyIceberg) coming soon. The switch to the new docs is about to happen. Also there's a really great discussion at the end on GeoParquet.
Amazing work Iceberg nation! Recording/Transcript: https://www.youtube.com/watch?v=GCHaFrPVbCQ * Highlights * Added Parquet and Avro encryption support (Thanks, Gidon!) * Added Spark support for reading, dropping, renaming views (Thanks, Eduard!) * Added initial setup for Kafka Connect (Thanks, Bryan!) * Added Spark support for delete file granularity (Thanks, Anton!) * Removed support for Spark 3.2 (Thanks, Ajantha!) * Parallelized file footer reads in add_files procedure (Thanks, Manu!) * Docs: Updated new docs to look like the existing docs (Thanks, Bits!) * PyIceberg * Added write support in PyIceberg (Thanks, Fokko!) * Added commit support for PyIceberg Hive (Thanks, Kevin!) * Added commit support for SQL catalog (Thanks, Sung!) * Added Parquet name mapping support in PyIceberg (Thanks, Sung!) * Rust: Added expressions and basic scan planning (Thanks, Renjie!) * [ https://rust.iceberg.apache.org/](https://rust.iceberg.apache.org/) * Releases * Docs refactor (Bits) * Relevant email thread: [ https://lists.apache.org/thread/dv2x9n6pkykmstkhrw1fx934qjtxnofy](https://lists.apache.org/thread/dv2x9n6pkykmstkhrw1fx934qjtxnofy) * Docs: [ https://github.com/apache/iceberg/tree/main/site](https://github.com/apache/iceberg/tree/main/site) * Need votes, will aim to do cutover prior to 1.5.0 and manual release * Java 1.5.0 ([milestone](https://github.com/apache/iceberg/milestone/37 )) * Kafka Connect * CREATE VIEW * Default values in copy-on-write MERGE (Spark 3.4) * View support on JDBC, Hive, REST, … * Python 0.6.0 * [ https://github.com/apache/iceberg-python/milestone/1](https://github.com/apache/iceberg-python/milestone/1) * Write support * Discussion * Spatial extensions to Iceberg * Prior art: [Geoparquet]( https://github.com/opengeospatial/geoparquet), [Geoarrow]( https://github.com/geoarrow/geoarrow), [Havasu]( https://github.com/wherobots/havasu/) * [WKB]( https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry#Well-known_binary) - Well integrated, not the most performant/space considerate * Better performance would be closer to geoarrow, but will be less familiar to developers and anyone trying to integrate with Geo format * How do we actually store these we will dive into in a follow up discussion facilitated on the dev list.
