[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-08-27 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 15:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/4382/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 15
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Tue, 27 Aug 2019 11:00:01 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-08-27 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#15). ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitPacking, BitWriter and
BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
13 files changed, 4,228 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/15
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 15
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-08-22 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 14:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/4332/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 14
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 22 Aug 2019 17:25:25 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-08-22 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 14:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/12621/14/be/src/util/bit-packing.cc
File be/src/util/bit-packing.cc:

http://gerrit.cloudera.org:8080/#/c/12621/14/be/src/util/bit-packing.cc@63
PS14, Line 63: #define INSTANTIATE_UNPACK_AND_DELTA_DECODE(OUT_TYPE, 
PARQUET_TYPE) \
line too long (109 > 90)


http://gerrit.cloudera.org:8080/#/c/12621/14/be/src/util/bit-packing.cc@65
PS14, Line 65:   BitPacking::UnpackAndDeltaDecodeValues(int 
bit_width,\
line too long (94 > 90)


http://gerrit.cloudera.org:8080/#/c/12621/14/be/src/util/bit-packing.cc@67
PS14, Line 67:   PARQUET_TYPE delta_offset, int64_t num_values, OUT_TYPE* 
__restrict__ out, int64_t stride, \
line too long (98 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 14
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 22 Aug 2019 16:43:23 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-08-22 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#14). ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitPacking, BitWriter and
BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
13 files changed, 4,227 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/14
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 14
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-06-27 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 13:

We decided to split this into multiple subtasks. A part of this change (with 
some modifications) is https://gerrit.cloudera.org/#/c/13737/.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 13
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 27 Jun 2019 12:09:07 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-06-24 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 13:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3720/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 13
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Mon, 24 Jun 2019 10:59:33 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-06-24 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#13). ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
14 files changed, 4,252 insertions(+), 60 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/13
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 13
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-06-22 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 12:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3715/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 12
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Sat, 22 Jun 2019 10:30:27 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-06-22 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#12). ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
14 files changed, 4,247 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/12
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 12
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-05-03 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 11:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3056/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 11
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 03 May 2019 12:49:07 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-05-03 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#11). ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
14 files changed, 4,247 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/11
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 11
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 10:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/2963/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 10
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Mon, 29 Apr 2019 13:35:59 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#10). ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
14 files changed, 4,247 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/10
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 10
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 10:

Rebasing. Adding and correcting licence headers.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 10
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Mon, 29 Apr 2019 12:45:41 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 9:

No Builds Executed


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 9
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Mon, 29 Apr 2019 10:33:38 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 8:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/2953/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 8
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Mon, 29 Apr 2019 10:20:06 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Daniel Becker (Code Review)
Hello Gabor Kaszab, Zoltan Borok-Nagy, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/12621

to look at the new patch set (#9).

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

TODO: Currently the delta coders only support 32-bit integers. For 64
bit integers, we have to extend the functionality of BitWriter and
BatchedBitReader.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
M be/src/exec/parquet/parquet-common.h
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
15 files changed, 4,239 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/9
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 9
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12621/8/be/src/exec/parquet/parquet-delta-coder-test-data.h
File be/src/exec/parquet/parquet-delta-coder-test-data.h:

PS8:
This patch set was added by mistake.



--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 8
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Mon, 29 Apr 2019 10:02:51 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Daniel Becker (Code Review)
Daniel Becker has restored this change. ( http://gerrit.cloudera.org:8080/12621 
)

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Restored
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: restore
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 8
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 8:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/12621/8/be/src/exec/parquet/parquet-delta-coder-test-data.h
File be/src/exec/parquet/parquet-delta-coder-test-data.h:

http://gerrit.cloudera.org:8080/#/c/12621/8/be/src/exec/parquet/parquet-delta-coder-test-data.h@454
PS8, Line 454: const std::vector values_are_the_same_plain = {3, 3, 3, 
3, 3, 3, 3, 3, 3, 3, 3, 3,
line too long (91 > 90)


http://gerrit.cloudera.org:8080/#/c/12621/8/be/src/exec/parquet/parquet-delta-coder-test-data.h@471
PS8, Line 471: const std::vector delta_is_zero_for_each_block_plain = 
{0, 0, 0, 0, 0, 0, 0, 0, 0,
line too long (91 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 8
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Mon, 29 Apr 2019 10:00:10 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Daniel Becker (Code Review)
Hello Gabor Kaszab, Zoltan Borok-Nagy, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/12621

to look at the new patch set (#8).

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

TODO: Currently the delta coders only support 32-bit integers. For 64
bit integers, we have to extend the functionality of BitWriter and
BatchedBitReader.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/exec/parquet/CMakeLists.txt
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
8 files changed, 2,123 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/8
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 8
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-29 Thread Daniel Becker (Code Review)
Daniel Becker has abandoned this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Abandoned
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 8
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-26 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 7:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/2933/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 7
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 26 Apr 2019 13:26:56 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-26 Thread Daniel Becker (Code Review)
Hello Gabor Kaszab, Zoltan Borok-Nagy, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/12621

to look at the new patch set (#7).

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

TODO: Currently the delta coders only support 32-bit integers. For 64
bit integers, we have to extend the functionality of BitWriter and
BatchedBitReader.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
M be/src/exec/parquet/parquet-common.h
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
15 files changed, 4,239 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/7
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 7
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-26 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 6:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/2932/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 6
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 26 Apr 2019 12:21:07 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-26 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/12621


Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..

IMPALA-8253: Draft - Parquet delta encoding and decoding.

Implemented an encoder and decoder for the Parquet delta encoding (see
https://github.com/apache/parquet-format/blob/master/Encodings.md).

The coders are not integrated with Impala yet, they provide an interface
that Impala could use.

TODO: Currently the delta coders only support 32-bit integers. For 64
bit integers, we have to extend the functionality of BitWriter and
BatchedBitReader.

Added new methods to BitWriter and BatchedBitReader handling Uleb and
ZigZag integers for 64 bits.

Testing:
  - Added new tests for the encoder and decoder
  - Tests covering the additions in BitWriter and BatchedBitReader.

Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/parquet-delta-benchmark.cc
M be/src/exec/parquet/CMakeLists.txt
M be/src/exec/parquet/parquet-common.h
A be/src/exec/parquet/parquet-delta-coder-test-data.h
A be/src/exec/parquet/parquet-delta-coder-test.cc
A be/src/exec/parquet/parquet-delta-decoder.h
A be/src/exec/parquet/parquet-delta-encoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-stream-utils.h
M be/src/util/bit-stream-utils.inline.h
15 files changed, 4,238 insertions(+), 56 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12621/6
--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 6
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-8253: Draft - Parquet delta encoding and decoding.

2019-04-26 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12621 )

Change subject: IMPALA-8253: Draft - Parquet delta encoding and decoding.
..


Patch Set 6:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/12621/6/be/src/exec/parquet/parquet-delta-coder-test-data.h
File be/src/exec/parquet/parquet-delta-coder-test-data.h:

http://gerrit.cloudera.org:8080/#/c/12621/6/be/src/exec/parquet/parquet-delta-coder-test-data.h@454
PS6, Line 454: const std::vector values_are_the_same_plain = {3, 3, 3, 
3, 3, 3, 3, 3, 3, 3, 3, 3,
line too long (91 > 90)


http://gerrit.cloudera.org:8080/#/c/12621/6/be/src/exec/parquet/parquet-delta-coder-test-data.h@471
PS6, Line 471: const std::vector delta_is_zero_for_each_block_plain = 
{0, 0, 0, 0, 0, 0, 0, 0, 0,
line too long (91 > 90)


http://gerrit.cloudera.org:8080/#/c/12621/6/be/src/exec/parquet/parquet-delta-encoder.h
File be/src/exec/parquet/parquet-delta-encoder.h:

http://gerrit.cloudera.org:8080/#/c/12621/6/be/src/exec/parquet/parquet-delta-encoder.h@47
PS6, Line 47:
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/12621/6/be/src/exec/parquet/parquet-delta-encoder.h@115
PS6, Line 115:   const int header_size = 
HeaderSize(most_negative_first_value, MAX_TOTAL_VALUE_COUNT);
line too long (91 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/12621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie7378ac1a490a6c89a0a4349aae86cbc0fbc80f8
Gerrit-Change-Number: 12621
Gerrit-PatchSet: 6
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 26 Apr 2019 11:31:40 +
Gerrit-HasComments: Yes