Parquet sync notes for 2018-12-18

Hi everyone, Here are the notes from Tuesday’s sync. Sorry they’re a bit late! *Attendees and topics*: - Zoltan Ivanfi (Cloudera) - New release candidate - Anna Szonyi (Cloudera) - New release candidate - Gabor Szadovszky (Cloudera) - New release candidate - Gidon Gershinsky (IBM) -

parquet-sync notes December 5 2018

Deepak: encryption, column statistics Zoltan: vote on the release Nandor: Ryan (netflix): release candidate, validation of the release, encryption Gidon (IBM): update encryption Lars (Cloudera Impala): Qinghui (Criteo): PR in parquet-proto, next release. Replace current proto compiler:

Re: parquet sync notes

I would like to attend the next sync. Where do I find instructions to join this meeting? On Tue, Oct 9, 2018 at 10:13 AM Julien Le Dem wrote: > Gabor (Cloudera): column index, benchmark, nested types (filter, indexes) > Anna (Cloudera): process, feature branches, etiquette of waiting for >

parquet sync notes

Gabor (Cloudera): column index, benchmark, nested types (filter, indexes) Anna (Cloudera): process, feature branches, etiquette of waiting for someone? Blocked Zoltan (Cloudera): Feature branches? When to review them? Nandor (Cloudera) parquet file with multiple row groups, schema evolution

Re: parquet sync notes

Hi, I have created the feature branches: - https://github.com/apache/parquet-mr/tree/bloom-filter - https://github.com/apache/parquet-format/tree/bloom-filter - https://github.com/apache/parquet-mr/tree/encryption - https://github.com/apache/parquet-format/tree/encryption I have also

Re: parquet sync notes

Hi Zoltan PR #62 contains some rebase info which is not relate to change itself so I created PR#99. Actually it only contains one file change now, I will add another document file later. Zoltan Ivanfi 于2018年9月26日周三 下午3:19写道： > Hi, > > It seems to me that PR #99 does not supersede PR #62, as

Re: parquet sync notes

Hi, It seems to me that PR #99 does not supersede PR #62, as the latter affects 16 files but the former only modifies a single one. Or has the rest of the changes been already merged to the codebase from another PR? I checked the history and I don't see anything related. Thanks, Zoltan On Wed,

Re: parquet sync notes

Hi the pr28 and pr62 of parquet-format was closed. Will we create a feature branch for bloom filter on parquet-mr as well? Julien Le Dem 于2018年9月26日周三 上午12:48写道： > Lars (Cloudera Impala): listen in. > Zoltan, Gabor and Nandor (Cloudera): > >- feature branch reviewed and merged >-

parquet sync notes

Lars (Cloudera Impala): listen in. Zoltan, Gabor and Nandor (Cloudera): - feature branch reviewed and merged - Parquet-format release - - Define scope Ryan (Netflix) Junjie (tencent): bloom filter Jim Apple (cloud service): bloom filter in parquet-mr? Since they got in parquet-cpp

Re: Parquet sync notes

There are four PRs, each dependent on its predecessor. Please review in this order: 1) #94 in parquet-format: Thrift additions (crypto structures) 2) #95 in parquet-format: encryption/decryption of footer, headers and column metadata - via cipher interfaces 3) #471 in parquet-mr: crypto

Parquet sync notes

QingHui (Criteo): parquet-protobuf Lars (impala), Jim (Cloudera): Bloom filter benchmarks Ryan (Netflix): JunJie (Intel): Bloomfilter and dictionary comparison benchmarks Gidon (IBM): Encryption, feedback Xinli Shang (Uber): Encryption Bloomfilter and dictionary comparison benchmarks: -

Parquet sync notes

Attendees / Agenda: Gidon (IBM): Parquet encryption. Uber, Vertica, Amazon Anna, Gabor, Nandor (Cloudera): Review for column indexing Junjie (tencent): Bloom filter Lars (Cloudera impala) Jim (Cloudera): Bloom filter Deepak (Vertica): Encryption Qinghui, Benoit (Criteo): parquet protobuf. Parquet