Re: Parquet sync today July 23rd

2025-07-25 Thread Julien Le Dem
Notes from the meeting:

Notes

   -

   Variant
   -

  Finalize the variant spec
  -

 Aihua: spend time on validation and finalize the spec.
 -

 Java <-> go for shredding
 -

 Rust: shredding is in implementation?
 -

 Commit example to the parquet testing.
 -

https://github.com/apache/parquet-testing/issues/75
-

   Parquet testing fo haskell 
   -

  Pure haskell implementation
  -

  Use apache/parquet-testing for testing
  -

   Time interval
   -

  Yun: agreement on y-m interval
  -

  Duration: nano? Parameter for time unit.
  -

  Will follow up on list and java implementation.
  -

   Encodings
   -

  Jeff/Prateek:
  -

 Doc for process in progress: Parquet new features
 

 -

 Starting a few proposals:
 -

FSST: strings (see paper FSST: Fast Random Access String
Compression  )
-

ALP: floating points (see paper ALP: Adaptive Lossless
floating-Point Compression
)
-

  This paper also has a bunch of good example datasets to test for
  string compression:
  https://www.cs.cit.tum.de/fileadmin/w00cfj/dis/papers/btrblocks.pdf
  -

   Footer
   -

  Micah: to follow up with Alkis
  -

  Rok interested



Action items

   - [image: unchecked]

   Aihua, Michael, Martin, David: to collaborate on to test files, cross
   compatibility tests for finalizing Variant. Can use
   https://github.com/apache/parquet-testing/issues/75 for coordinating
   - [image: unchecked]

   Yun: follow up on the mailing list on time intervals.
   - [image: unchecked]

   Jeff: start email thread on ALP (or a new encoding).


On Wed, Jul 23, 2025 at 9:28 AM Julien Le Dem  wrote:

> The next Parquet sync is today July 23rd at 10am PT - 1pm ET - 7pm CET
> (in 30 mins)
> I'll be there!
>
> To join the invite, join the group:
> https://groups.google.com/g/apache-parquet-community-sync
>
> Everybody is welcome, bring your topic or just listen in.
>
> (Some more details on how the meeting is run:
> https://lists.apache.org/thread/bjdkscmx7zvgfbw0wlfttxy8h6v3f71t )
>


Parquet sync today July 23rd

2025-07-23 Thread Julien Le Dem
The next Parquet sync is today July 23rd at 10am PT - 1pm ET - 7pm CET
(in 30 mins)
I'll be there!

To join the invite, join the group:
https://groups.google.com/g/apache-parquet-community-sync

Everybody is welcome, bring your topic or just listen in.

(Some more details on how the meeting is run:
https://lists.apache.org/thread/bjdkscmx7zvgfbw0wlfttxy8h6v3f71t )