jenniferdai commented on a change in pull request #3932: Editing pluggable storage docs URL: https://github.com/apache/incubator-pinot/pull/3932#discussion_r263880842
########## File path: docs/pluggable_storage.rst ########## @@ -35,6 +35,26 @@ New Storage Type implementation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In order to add a new type of storage backend (say, Amazon s3) implement the following class: -#. S3FS extends `PinotFS <https://github.com/apache/incubator-pinot/blob/master/pinot-filesystem/src/main/java/org/apache/pinot/filesystem/PinotFS.java>`_ +S3FS extends `PinotFS <https://github.com/apache/incubator-pinot/blob/master/pinot-filesystem/src/main/java/org/apache/pinot/filesystem/PinotFS.java>`_ -The properties for the stream implementation are to be set in your controller and server configurations, `like so <https://github.com/apache/incubator-pinot/wiki/Pluggable-Storage>`_. \ No newline at end of file + +Configurations +^^^^^^^^^^^^^^ +These properties for the stream implementation are to be set in your controller and server configurations. + +Pinot supports different kinds of storage; eg: NFS, HDFS, ADL. In order to get this working with your cluster, you will need to configure the following. + +In your controller and server configs, please set the FS class you would like to support. pinot.controller.storage.factory.class.${YOUR_URI_SCHEME} to the full path of the FS class you would like to include + +You also need to configure pinot.controller.local.temp.dir for the local dir on the controller machine. + +For filesystem specific configs, you can pass in the following with either the pinot.controller prefix or the pinot.server prefix. + +All the following configs need to be prefixed with storage.factory. AzurePinotFS requires the following configs: + +adl.accountId, adl.authEndpoint, adl.clientId, adl.clientSecret segment.fetcher.protocols : adl Review comment: That's what users should set if they want to use adl or hdfs, essentially lets us know which segment fetchers to initialize ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
