Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/10483 )
Change subject: IMPALA-7044: Prevent overflow when computing Parquet block size ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/10483/1/be/src/exec/hdfs-parquet-table-writer.cc File be/src/exec/hdfs-parquet-table-writer.cc: http://gerrit.cloudera.org:8080/#/c/10483/1/be/src/exec/hdfs-parquet-table-writer.cc@795 PS1, Line 795: table_desc_->num_cols() - table_desc_->num_clustering_cols() > num_cols Done http://gerrit.cloudera.org:8080/#/c/10483/1/be/src/exec/hdfs-table-sink.cc File be/src/exec/hdfs-table-sink.cc: http://gerrit.cloudera.org:8080/#/c/10483/1/be/src/exec/hdfs-table-sink.cc@376 PS1, Line 376: return Status("HDFS block size must be smaller than 2GB."); > Any reasonable way to test this path? I can't think of one, it's mainly another safe-guard against future mistakes. The parquet_file_size query option does the same check. It is actually OK to allow 2GB block sizes to I loosened the check. Before doing so I was able to trigger this manually by setting parquet_file_size to int_max. -- To view, visit http://gerrit.cloudera.org:8080/10483 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e63420e5a093c0bbc789201771708865b16e138 Gerrit-Change-Number: 10483 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Thomas Marshall <[email protected]> Gerrit-Comment-Date: Wed, 23 May 2018 18:20:16 +0000 Gerrit-HasComments: Yes
