[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. IMPALA-5008: Fix reading stats for TINYINT and SMALLINT TINYINT and SMALLINT types use 1 and 2 byte slots respectively. However, statistics for the corresponding INT_8 and INT_16 Parquet types are encoded using 4 bytes. When reading back we were missing a conversion to the smaller types, thus overwriting the memory behind them. This was caught by the address sanitizer. The fix is to perform the necessary conversion. Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Reviewed-on: http://gerrit.cloudera.org:8080/6226 Reviewed-by: Lars VolkerTested-by: Impala Public Jenkins --- M be/src/exec/parquet-column-stats.cc M be/src/runtime/coordinator.cc 2 files changed, 26 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Verified Lars Volker: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 5: Build started: http://jenkins.impala.io:8080/job/gerrit-verify-dryrun/334/ -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 3: Rebased. Carrying Tim's +2. -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 3: (2 comments) Thanks for your review. I addressed the comments in PS4. Will rebase next. http://gerrit.cloudera.org:8080/#/c/6226/3/be/src/exec/parquet-column-stats.cc File be/src/exec/parquet-column-stats.cc: Line 39: return ret; > return true? Done Line 51: return ret; > return true? Done -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Hello Tim Armstrong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/6226 to look at the new patch set (#4). Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. IMPALA-5008: Fix reading stats for TINYINT and SMALLINT TINYINT and SMALLINT types use 1 and 2 byte slots respectively. However, statistics for the corresponding INT_8 and INT_16 Parquet types are encoded using 4 bytes. When reading back we were missing a conversion to the smaller types, thus overwriting the memory behind them. This was caught by the address sanitizer. The fix is to perform the necessary conversion. Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 --- M be/src/exec/parquet-column-stats.cc M be/src/runtime/coordinator.cc 2 files changed, 26 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/6226/4 -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Tim Armstrong has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 3: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/6226/3/be/src/exec/parquet-column-stats.cc File be/src/exec/parquet-column-stats.cc: Line 39: return ret; return true? Line 51: return ret; return true? -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 2: (3 comments) Thanks for the review. Please see PS 3. http://gerrit.cloudera.org:8080/#/c/6226/2/be/src/exec/parquet-column-stats.cc File be/src/exec/parquet-column-stats.cc: Line 34: if (col_stats < std::numeric_limits::min() || > col_stats isn't valid if ret is false, right? yes, fixed. Line 39: DCHECK(false); > I don't think we should DCHECK here - users do sometimes run DEBUG builds o I wasn't aware. Removed it. Line 50: if (col_stats < std::numeric_limits::min() || > See above comments. Done -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has uploaded a new patch set (#3). Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. IMPALA-5008: Fix reading stats for TINYINT and SMALLINT TINYINT and SMALLINT types use 1 and 2 byte slots respectively. However, statistics for the corresponding INT_8 and INT_16 Parquet types are encoded using 4 bytes. When reading back we were missing a conversion to the smaller types, thus overwriting the memory behind them. This was caught by the address sanitizer. The fix is to perform the necessary conversion. Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 --- M be/src/exec/parquet-column-stats.cc M be/src/runtime/coordinator.cc 2 files changed, 26 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/6226/3 -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Tim Armstrong has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/6226/2/be/src/exec/parquet-column-stats.cc File be/src/exec/parquet-column-stats.cc: Line 34: if (col_stats < std::numeric_limits::min() || col_stats isn't valid if ret is false, right? Line 39: DCHECK(false); I don't think we should DCHECK here - users do sometimes run DEBUG builds on their clusters (often their dev environment). Line 50: if (col_stats < std::numeric_limits::min() || See above comments. -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has posted comments on this change. Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. Patch Set 1: PS1 was missing word in a comment. -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars VolkerGerrit-Reviewer: Lars Volker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has uploaded a new patch set (#2). Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. IMPALA-5008: Fix reading stats for TINYINT and SMALLINT TINYINT and SMALLINT types use 1 and 2 byte slots respectively. However, statistics for the corresponding INT_8 and INT_16 Parquet types are encoded using 4 bytes. When reading back we were missing a conversion to the smaller types, thus overwriting the memory behind them. This was caught by the address sanitizer. The fix is to perform the necessary conversion. Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 --- M be/src/exec/parquet-column-stats.cc M be/src/runtime/coordinator.cc 2 files changed, 34 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/6226/2 -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker
[Impala-ASF-CR] IMPALA-5008: Fix reading stats for TINYINT and SMALLINT
Lars Volker has uploaded a new change for review. http://gerrit.cloudera.org:8080/6226 Change subject: IMPALA-5008: Fix reading stats for TINYINT and SMALLINT .. IMPALA-5008: Fix reading stats for TINYINT and SMALLINT TINYINT and SMALLINT types use 1 and 2 byte slots respectively. However, statistics for the corresponding INT_8 and INT_16 Parquet types are encoded using 4 bytes. When reading back we were missing a conversion to the smaller types, thus overwriting the memory behind them. This was caught by the address sanitizer. The fix is to perform the necessary conversion. Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 --- M be/src/exec/parquet-column-stats.cc M be/src/runtime/coordinator.cc 2 files changed, 34 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/6226/1 -- To view, visit http://gerrit.cloudera.org:8080/6226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9b10508db53747e7b08c8bd9a69c763b82135a78 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker