yihua commented on code in PR #12923: URL: https://github.com/apache/hudi/pull/12923#discussion_r1983837215
########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. Review Comment: The CLI bundle script is in the bundle folder: ```suggestion Once hudi has been built, the shell can be fired by via `cd packaging/hudi-cli-bundle && ./hudi-cli-with-bundle.sh`. ``` ########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. Review Comment: I think we should remove this section and rename `Hudi CLI Bundle setup` to `Hudi CLI setup` assuming that CLI bundle is the only way now, using local build or existing artifact. ########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. ### Hudi CLI Bundle setup In release `0.13.0` we have now added another way of launching the `hudi cli`, which is using the `hudi-cli-bundle`. +#### Note: The traditional `hudi-cli.sh` script has been deprecated and replaced with `hudi-cli-with-bundle.sh` from this release onwards. Users should migrate to the new bundled CLI script `hudi-cli-with-bundle.sh` for better compatibility and ease of use. + + There are a couple of requirements when using this approach such as having `spark` installed locally on your machine. It is required to use a spark distribution with hadoop dependencies packaged such as `spark-3.3.1-bin-hadoop2.tgz` from https://archive.apache.org/dist/spark/. Review Comment: Let's update the Spark version to Spark 3.5. ########## website/docs/cli.md: ########## @@ -73,7 +76,7 @@ export CLIENT_JAR=/lib/spark-3.2.0-bin-hadoop3.2/jars/aws-java-sdk-bundle-1.12.4 ``` Once these are set, you are good to launch hudi-cli and access S3 dataset. ``` -./hudi-cli/hudi-cli.sh +./packaging/hudi-cli-bundle/hudi-cli-with-bundle.sh Review Comment: Any prep on `SPARK_HOME` and `SPARK_BUNDLE_JAR` on non-EMR environment? ########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. ### Hudi CLI Bundle setup In release `0.13.0` we have now added another way of launching the `hudi cli`, which is using the `hudi-cli-bundle`. +#### Note: The traditional `hudi-cli.sh` script has been deprecated and replaced with `hudi-cli-with-bundle.sh` from this release onwards. Users should migrate to the new bundled CLI script `hudi-cli-with-bundle.sh` for better compatibility and ease of use. + + There are a couple of requirements when using this approach such as having `spark` installed locally on your machine. Review Comment: ```suggestion There are a couple of requirements such as having `spark` installed locally on your machine. ``` ########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. ### Hudi CLI Bundle setup In release `0.13.0` we have now added another way of launching the `hudi cli`, which is using the `hudi-cli-bundle`. +#### Note: The traditional `hudi-cli.sh` script has been deprecated and replaced with `hudi-cli-with-bundle.sh` from this release onwards. Users should migrate to the new bundled CLI script `hudi-cli-with-bundle.sh` for better compatibility and ease of use. Review Comment: ```suggestion :::note The traditional `hudi-cli.sh` script has been deprecated and replaced with `hudi-cli-with-bundle.sh` from `1.1.0` release onwards. Users should migrate to the new bundled CLI script `hudi-cli-with-bundle.sh` for better compatibility and ease of use. ::: ``` ########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. ### Hudi CLI Bundle setup In release `0.13.0` we have now added another way of launching the `hudi cli`, which is using the `hudi-cli-bundle`. Review Comment: Rephrase this sentence because this is the only way, not `another way`. ########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. ### Hudi CLI Bundle setup In release `0.13.0` we have now added another way of launching the `hudi cli`, which is using the `hudi-cli-bundle`. +#### Note: The traditional `hudi-cli.sh` script has been deprecated and replaced with `hudi-cli-with-bundle.sh` from this release onwards. Users should migrate to the new bundled CLI script `hudi-cli-with-bundle.sh` for better compatibility and ease of use. + + There are a couple of requirements when using this approach such as having `spark` installed locally on your machine. It is required to use a spark distribution with hadoop dependencies packaged such as `spark-3.3.1-bin-hadoop2.tgz` from https://archive.apache.org/dist/spark/. Review Comment: Revisit other places too, as I see Spark 3.1 and 3.2 mentioned in different lines. ########## website/docs/cli.md: ########## @@ -73,7 +76,7 @@ export CLIENT_JAR=/lib/spark-3.2.0-bin-hadoop3.2/jars/aws-java-sdk-bundle-1.12.4 ``` Once these are set, you are good to launch hudi-cli and access S3 dataset. ``` -./hudi-cli/hudi-cli.sh +./packaging/hudi-cli-bundle/hudi-cli-with-bundle.sh ``` ### Using hudi-cli on Google Dataproc [Dataproc](https://cloud.google.com/dataproc) is Google's managed service for running Apache Hadoop, Apache Spark, Review Comment: Let's update Hudi version to 1.0.0 in all places. For the docs revision in this file that applies to 1.0.0/1.0.1 docs, let's port the revision to the versioned docs as well (`website/versioned_docs/version-1.0.0/cli.md`, `website/versioned_docs/version-1.0.1/cli.md`). ########## website/docs/cli.md: ########## @@ -5,11 +5,14 @@ last_modified_at: 2021-08-18T15:59:57-04:00 --- ### Local set up -Once hudi has been built, the shell can be fired by via `cd hudi-cli && ./hudi-cli.sh`. +Once hudi has been built, the shell can be fired by via `cd hudi-cli-with-bundle.sh && ./hudi-cli-with-bundle.sh`. Review Comment: Have you verified that this local setup works after build the CLI bundle? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
