This is an automated email from the ASF dual-hosted git repository. jenniferdai pushed a commit to branch doc in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 95ccd28a7b4f4a134f91b2756a26ec6a2a2b241a Author: Jennifer Dai <[email protected]> AuthorDate: Thu Mar 7 15:39:17 2019 -0800 Editing pluggable storage docs --- docs/pluggable_storage.rst | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/docs/pluggable_storage.rst b/docs/pluggable_storage.rst index 5296fae..b9d768f 100644 --- a/docs/pluggable_storage.rst +++ b/docs/pluggable_storage.rst @@ -35,6 +35,26 @@ New Storage Type implementation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In order to add a new type of storage backend (say, Amazon s3) implement the following class: -#. S3FS extends `PinotFS <https://github.com/apache/incubator-pinot/blob/master/pinot-filesystem/src/main/java/org/apache/pinot/filesystem/PinotFS.java>`_ +S3FS extends `PinotFS <https://github.com/apache/incubator-pinot/blob/master/pinot-filesystem/src/main/java/org/apache/pinot/filesystem/PinotFS.java>`_ -The properties for the stream implementation are to be set in your controller and server configurations, `like so <https://github.com/apache/incubator-pinot/wiki/Pluggable-Storage>`_. \ No newline at end of file + +Configurations +^^^^^^^^^^^^^^ +These properties for the stream implementation are to be set in your controller and server configurations. + +Pinot supports different kinds of storage; eg: NFS, HDFS, ADL. In order to get this working with your cluster, you will need to configure the following. + +In your controller and server configs, please set the FS class you would like to support. pinot.controller.storage.factory.class.${YOUR_URI_SCHEME} to the full path of the FS class you would like to include + +You also need to configure pinot.controller.local.temp.dir for the local dir on the controller machine. + +For filesystem specific configs, you can pass in the following with either the pinot.controller prefix or the pinot.server prefix. + +All the following configs need to be prefixed with storage.factory. AzurePinotFS requires the following configs: + +adl.accountId, adl.authEndpoint, adl.clientId, adl.clientSecret segment.fetcher.protocols : adl + + +HadoopPinotFS requires the following configs: + +hadoop.kerberos.principle, hadoop.kerberos.keytab, hadoop.conf.path segment.fetcher.protocols : hdfs \ No newline at end of file --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
