[email protected] has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file ...................................................................... Patch Set 2: (6 comments) http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@12 PS2, Line 12: file://somedir/f1.txt > nit: could you use the more commonly used 'hdfs' path ? Done http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@17 PS2, Line 17: This patch enables to use sfs+ wrappers to show a single file as a single file in a directory. > nit: the wording here is a bit confusing. How about something like: 'This p Done http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@22 PS2, Line 22: > Could you add a comment about the Ranger authorization check that this patc removed that part - it was left there by mistake. http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@22 PS2, Line 22: > Pls add a note about how this patch was tested even if manually. Done http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@23 PS2, Line 23: (cherry picked from commit 998d5a8d72203c9ffead700b2e420049108417c3) > Not sure which commit it is referring to, but this should be removed. Done http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc File be/src/runtime/io/disk-io-mgr.cc: http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc@142 PS2, Line 142: // The maximum number of SFS I/O threads. : DEFINE_int32(num_sfs_io_threads, 16, "Number of SFS I/O threads"); > My understanding is that the file handle cache will be disabled for SFS unl Although it might be possible to do that - I think it might not worth the effort to cover these cases: these are most likely small files so the performance impact might not be serious. I also think that if a customer will run into perfomance issues by using sfs - if the issue is caused by this then he should consider migrating to a real table directory. The main usecase of this stuff is to help people testdrive and validate migration earlier. -- To view, visit http://gerrit.cloudera.org:8080/17878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7 Gerrit-Change-Number: 17878 Gerrit-PatchSet: 2 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Comment-Date: Wed, 03 Nov 2021 18:05:21 +0000 Gerrit-HasComments: Yes
