chenboat commented on a change in pull request #4944: Add documentation for the pinot-gcs-filesystem URL: https://github.com/apache/incubator-pinot/pull/4944#discussion_r364077171
########## File path: docs/pluggable_storage.rst ########## @@ -28,17 +28,65 @@ Some examples of storage backends(other than local storage) currently supported * `HadoopFS <https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html>`_ * `Azure Data Lake <https://azure.microsoft.com/en-us/solutions/data-lake/>`_ +* `Google Cloud Storage <https://cloud.google.com/storage/>`_ -If the above two filesystems do not meet your needs, you can extend the current `PinotFS <https://github.com/apache/incubator-pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/filesystem/PinotFS.java>`_ to customize for your needs. +GcsPinotFS +---------- + + To configure the GcsPinotFS filesystem, the server and controller must be configured as shown below. + The account associated with the ``GCP_KEY_FILE`` must be able to create and drop objects in the gcs bucket, + for example if the account has the ``Storage Admin`` and ``Storage Object Admin`` roles for the bucket that will + be sufficient. + + The ``CLASSPATH_PREFIX`` environment variable must also be set to point to the pinot-gcs-filesystem.jar in the server + and controller. + +Controller Config: + +.. code-block:: none + + controller.helix.cluster.name=pinot + controller.host=<pinot host> + controller.port=9000 + controller.vip.host=<pinot host> Review comment: The vip config is optional and not all installations have vip set up. Some of the example config shown here are not related to pluggable storage: it would be great if you can add comments to highlight the essential settings. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
