[incubator-celeborn] branch main updated: [CELEBORN-139][BUG] Fix read wrong yaml file format when loading config (#1083)

[zhouky]

This is an automated email from the ASF dual-hosted git repository.

zhouky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new 2e37830a [CELEBORN-139][BUG] Fix read wrong yaml file format when 
loading config (#1083)
2e37830a is described below

commit 2e37830a0f4caed5c5d709d23ed9089ca34eb6fc
Author: nafiy <30563796+nafiy...@users.noreply.github.com>
AuthorDate: Wed Dec 14 20:56:04 2022 +0800

    [CELEBORN-139][BUG] Fix read wrong yaml file format when loading config 
(#1083)
---
 common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala | 3 ++-
 common/src/main/scala/org/apache/celeborn/common/util/Utils.scala   | 2 +-
 docs/configuration/quota.md                                         | 2 +-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git 
a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala 
b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
index 64ccfd7c..ec837625 100644
--- a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
+++ b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
@@ -2362,7 +2362,8 @@ object CelebornConf extends Logging {
     buildConf("celeborn.quota.configuration.path")
       .withAlternative("rss.quota.configuration.path")
       .categories("quota")
-      .doc("Quota configuration file path.")
+      .doc("Quota configuration file path. The file format should be yaml. 
Quota configuration file template can be " +
+        "found under conf directory.")
       .version("0.2.0")
       .stringConf
       .createOptional
diff --git a/common/src/main/scala/org/apache/celeborn/common/util/Utils.scala 
b/common/src/main/scala/org/apache/celeborn/common/util/Utils.scala
index b8b6a0a4..a266d873 100644
--- a/common/src/main/scala/org/apache/celeborn/common/util/Utils.scala
+++ b/common/src/main/scala/org/apache/celeborn/common/util/Utils.scala
@@ -561,7 +561,7 @@ object Utils extends Logging {
   def getDefaultQuotaConfigurationFile(env: Map[String, String] = sys.env): 
String = {
     env.get("CELEBORN_CONF_DIR")
       .orElse(env.get("CELEBORN_HOME").map { t => s"$t${File.separator}conf" })
-      .map { t => new File(s"$t${File.separator}quota.yml") }
+      .map { t => new File(s"$t${File.separator}quota.yaml") }
       .filter(_.isFile)
       .map(_.getAbsolutePath)
       .orNull
diff --git a/docs/configuration/quota.md b/docs/configuration/quota.md
index 7d124cec..ae252b19 100644
--- a/docs/configuration/quota.md
+++ b/docs/configuration/quota.md
@@ -19,7 +19,7 @@ license: |
 <!--begin-include-->
 | Key | Default | Description | Since |
 | --- | ------- | ----------- | ----- |
-| celeborn.quota.configuration.path | &lt;undefined&gt; | Quota configuration 
file path. | 0.2.0 | 
+| celeborn.quota.configuration.path | &lt;undefined&gt; | Quota configuration 
file path. The file format should be yaml. Quota configuration file template 
can be found under conf directory. | 0.2.0 | 
 | celeborn.quota.enabled | true | When true, before registering shuffle, 
LifecycleManager should check if current user have enough quota space, if 
cluster don't have enough quota space for current user, fallback to Spark's 
default shuffle | 0.2.0 | 
 | celeborn.quota.identity.provider | 
org.apache.celeborn.common.identity.DefaultIdentityProvider | IdentityProvider 
class name. Default class is 
`org.apache.celeborn.common.identity.DefaultIdentityProvider`, return 
`org.apache.celeborn.common.identity.UserIdentifier` with default tenant id and 
username from `org.apache.hadoop.security.UserGroupInformation`.  | 0.2.0 | 
 | celeborn.quota.manager | 
org.apache.celeborn.common.quota.DefaultQuotaManager | QuotaManger class name. 
Default class is `org.apache.celeborn.common.quota.DefaultQuotaManager`. | 
0.2.0 |