Update: The Iceberg community has discussed and agreed on splitting geometry and geography types and focusing on the geometry type [1] at this point. We have also modified the Parquet PR [2] to reflect this. Please take a look and let me know what you think. If there is no objection, we will start a formal vote to accept the new geometry logical type and release it in the parquet-format version 2.11.0.
[1] https://github.com/apache/iceberg/pull/10981 [2] https://github.com/apache/parquet-format/pull/240 Best, Gang On Sun, Sep 8, 2024 at 12:30 AM Gang Wu <ust...@gmail.com> wrote: > Update: > > We are working with Apache Iceberg, Apache Sedona, and GeoParquet > communities to finalize the spec addition of Geometry type [1]. Two PoC > implementations [2][3] are on the way with the help from the Apache Sedona > community. > > Now the spec change has been stable and the implementations are close to > the finish line. In the meantime, Apache Iceberg is waiting for the Parquet > side before adding the geometry support to its v3 spec. I want to encourage > everyone to take a review on the spec and the PoCs. A vote thread will > be sent to finalize the process. > > [1] https://github.com/apache/parquet-format/pull/240 > [2] https://github.com/apache/parquet-java/pull/2971 > [3] https://github.com/apache/arrow/pull/43977 > > Thanks, > Gang > > On Sun, May 12, 2024 at 4:14 PM Gang Wu <ust...@gmail.com> wrote: > >> Hi, >> >> Apache Iceberg community is proposing to add geospatial support [1]. It >> would >> be good if Apache Parquet can support native geometry type to implement >> more >> efficient encoding, statistics and filtering. Therefore, I'd like to >> propose a format >> change to add a new geometry logical type: [2]. It is still in a very >> early stage and >> I'm working with Apache Iceberg community and GeoParquet community to >> polish >> the design. Hopefully this design will be a good starting point to >> support geospatial >> features from the file format level and be interoperable to more table >> formats and >> popular query engines in the future. Any feedback is welcomed! >> >> [1] >> https://docs.google.com/document/d/1iVFbrRNEzZl8tDcZC81GFt01QJkLJsI9E2NBOt21IRI >> [2] https://github.com/apache/parquet-format/pull/240 >> >> Best, >> Gang >> >>
