If the specification is not clear, I think we should first clarify what's there and then try to have multiple compatible implementations of the specification.
On Sun, Oct 27, 2019 at 3:35 PM Michael Heuer <heue...@gmail.com> wrote: > > Thank you, Wes. > > What about adding separate LZ4_BLOCK and LZ4_FRAME definitions in > parquet-format, or at least tightening up the specification? > > And is there any motivation in JVM parquet implementations to clear this up? > > michael > > > > On Oct 23, 2019, at 5:28 PM, Wes McKinney <wesmck...@gmail.com> wrote: > > > > We are waiting on a volunteer to come forward and sort out the proper > > implementation in Parquet C++ for LZ4. It is disabled in the meantime > > I think > > > > On Wed, Oct 23, 2019 at 10:11 AM Michael Heuer <heue...@gmail.com> wrote: > >> > >> Hello, > >> > >> There are a few issues related to block-vs-frame LZ4 compression in > >> Parquet/Arrow and related > >> > >> https://issues.apache.org/jira/browse/PARQUET-1241 > >> <https://issues.apache.org/jira/browse/PARQUET-1241> > >> https://issues.apache.org/jira/browse/PARQUET-1515 > >> <https://issues.apache.org/jira/browse/PARQUET-1515> > >> https://issues.apache.org/jira/browse/HADOOP-12990 > >> <https://issues.apache.org/jira/browse/HADOOP-12990> > >> https://github.com/dask/fastparquet/pull/315 > >> <https://github.com/dask/fastparquet/pull/315> > >> https://issues.apache.org/jira/browse/IMPALA-8617 > >> <https://issues.apache.org/jira/browse/IMPALA-8617> > >> > >> Might anyone be able to summarize the current status and take a guess at > >> the way forward? > >> > >> Thank you in advance, > >> > >> michael >
