Sorry, I missed Himadri Pal for rest catalog: 3. Some enhancement of rest catalog, such as oauth, custom headers. Thanks Himadri Pal(himadripal), whynick1, TennyZhuang!
On Thu, Mar 28, 2024 at 10:19 AM Renjie Liu <liurenjie2...@gmail.com> wrote: > Hi: > > Sorry I couldn't join the meeting due to timezone constraints. > > I want to add some summary about progress of iceberg-rust: > > 1. We have added hive catalog support(except update table methods). Thanks > Xuanwo and Marvin Lanhenke(github id: marvinlanhenke)! > 2. The glue catalog is in progress. Thanks Marvin Lanhenke(github > id: marvinlanhenke)! > 3. Some enhancement of rest catalog, such as oauth, custom headers. > Thanks whynick1, TennyZhuang! > 4. The community is actively working on reading support, including reading > to arrow, filter support, please stay tuned! > 5. We have more and more contributors involved in the development, and the > number of contributors has reached 36! > 6. I have created a github project to help schedule the development so > that contributors could easily pick up issues they are interested in! > > cc Brian, I could update the progress to meeting notes if necessary. > > On Thu, Mar 28, 2024 at 6:50 AM Brian Olsen <bitsondata...@gmail.com> > wrote: > >> Hey Iceberg Nation, >> >> Here are the meeting minutes from today's meeting. >> >> Transcription/Recording: https://youtu.be/9TRhgRq5bFk >> Meeting Notes: >> https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit#heading=h.686a3a6fup8m >> >> >> New Issue Template for Spec and Design Proposals: Dan created a new issue >> template and documentation to streamline design docs and spec updates, >> improving visibility into discussions and future planning in the Iceberg >> community. >> Manifest Encryption Support: Manifest encryption support has been added, >> further advancing Iceberg's encryption capabilities, including the ability >> to handle encrypted manifests. >> Function Pushdown in Spark: Anton enhanced function pushdown for >> row-level commands in Spark, refining the handling of system function >> pushdown for commands like merge and update. >> Output Spec Selection in Rewrite Data Files: Himadri introduced a feature >> allowing the selection of different output specs in rewrite data files, >> enhancing the capability for data management and optimization. >> >> Releases: >> 1.5.0: This version has been successfully released after numerous release >> candidates. >> 1.5.1: Potential upcoming release with discussions around schema >> behaviors with time travel reads, Trino integration improvements, and an >> extra HEAD request issue in S3 file IO. >> 1.6.0: Planning to include an Arrow upgrade for CVE fixes. >> PyIceberg 0.6.1: Focus on compatibility and bug fixes, such as addressing >> the creation of version one tables with non-empty partition specs and sort >> orders. >> >> Discussion: >> Spec and Design Proposals: The implementation of a new issue template and >> tagging system to track and manage proposals and discussions. >> REST Protocol Evolution: Discussion about adding capabilities to the REST >> protocol, ensuring client and server compatibility for new features. >> Schema Behavior with Time Travel Reads: Addressing the issue with schema >> behavior changes during time travel reads, aiming for consistency across >> branches. >> Flink Table Maintenance: A deep dive into Flink table maintenance >> including strategies for compaction, orphan file removal, and snapshot >> expiration. Discussion on handling streaming jobs, rolling manifest >> writers, and the RewriteFileGroupPlanner. >> Catalog Discussion: Deliberation on deprecating the DynamoDB catalog. >> How to address challenges related to the Spark 3.3 and 3.5 merge into >> Iceberg, specifically around dynamic partition pruning (DPP) issues. >> >
