[
https://issues.apache.org/jira/browse/IMPALA-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361348#comment-17361348
]
ASF subversion and git services commented on IMPALA-10711:
----------------------------------------------------------
Commit ebd3d444117e94226611a7aca0cf11639a7e65f4 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ebd3d44 ]
IMPALA-10711: Allow customizing startup's filesystem check
At startup, Impala runs JniFrontend::checkFileSystem() to verify
that it can list the root directory of the filesystem. Some
deployments would like to restrict Impala to a subdirectory
of the filesystem. In that circumstance, Impala may not have
access to the root of the filesystem and the startup check
will fail.
This adds the startup parameter: startup_filesystem_check_directories
to specify the directories to check in JniFrontend::checkFileSystem()
at startup. For deployments that are restricted to a subdirectory,
startup_filesystem_check_directories=/some/subdirectory would
check /some/subdirectory rather than /. This parameter allows for
a list of directories in case it is useful to verify multiple
locations (such as different s3 buckets). If set to the empty string,
no directories will be checked.
Testing:
- Added test_startup_filesystem_checks.py custom cluster test
- Hand tested locally
Change-Id: I68e6802424f16002c90a37b3ae2490721525e58d
Reviewed-on: http://gerrit.cloudera.org:8080/17570
Reviewed-by: Csaba Ringhofer <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Allow restricting the filesystem permissions check at startup to a particular
> directory
> ---------------------------------------------------------------------------------------
>
> Key: IMPALA-10711
> URL: https://issues.apache.org/jira/browse/IMPALA-10711
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 4.0
> Reporter: Joe McDonnell
> Priority: Critical
>
> At startup, Impala validates that it has access to the filesystem (via
> JniFrontend::checkFileSystem()). It does this by doing a
> getTotalNumVisibleFiles() on the root path. The root path is defined by the
> fs.defaultFS property. On S3 and other object stores, that is always the root
> of the bucket.
> Code:
> https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L887
> For security reasons, a user may want to restrict an Impala instance to a
> subdirectory of the s3 bucket (or other object store). This check at the root
> on startup makes that impossible.
> Impala should provide a way to customize where the check takes place. The
> check is primarily about making sure that Impala has appropriate permissions
> and is configured correctly. Checking a subdirectory of the s3 bucket is just
> as effective for this purpose.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
