Sailesh Mukil has posted comments on this change. Change subject: IMPALA-3452: S3: Disable Impala staging for INSERTs via flag for speedup ......................................................................
Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/2905/1/be/src/exec/hdfs-table-sink.cc File be/src/exec/hdfs-table-sink.cc: Line 52: DEFINE_bool(s3_skip_insert_staging, false, "Enable to skip the staging step for INSERTs " > why would we not always want to do this (and not even have a flag)? We currently have local staging before we send it to S3, so we can do away with the staging in S3 altogether. However, one reason I can think of is that if in the near future, we choose to stream data to S3 instead of staging it locally, then we wouldn't have any staging step. If that's not a strong enough reason, then we can do away with this option entirely, and make the skipping of S3 staging as default. (And change it back later if we need to) -- To view, visit http://gerrit.cloudera.org:8080/2905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Iff9620d41ba0d5fb1aa0c9f4abb48866fc2b0698 Gerrit-PatchSet: 1 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Sailesh Mukil <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]> Gerrit-HasComments: Yes
