xxjingcd commented on PR #13478:
URL:
https://github.com/apache/dolphinscheduler/pull/13478#issuecomment-1414858501
> > Environment variables HADOOP_CONF_DIR and HADOOP_HOME is very common in
most components, e.g HBASE, Spark、Flink, and so on.
>
> You are right, `HADOOP_CONF_DIR` is a common environment variable,
**HOWEVER**, i think add it into properties is more general, especially when
deploying in k8s.
In practice, users always maintain a set of common global variables， this
always include `HADOOP_CONF_DIR` and `HADOOP_HOME`.
In a word, **using environment variables will make DS deploy faster and
easily**. That is the reason why this pr use environment variables;
Actually, all I know about components are use environment variables to
configure. For Example, a source code fragment of Flink’s
[HadoopUtils](https://github.com/apache/flink/blob/master/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java)
```java
public static String[] possibleHadoopConfPaths(
org.apache.flink.configuration.Configuration flinkConfiguration)
{
String[] possiblePaths = new String[4];
possiblePaths[0] =
flinkConfiguration.getString(ConfigConstants.PATH_HADOOP_CONFIG, null);
possiblePaths[1] = System.getenv("HADOOP_CONF_DIR");
if (System.getenv("HADOOP_HOME") != null) {
possiblePaths[2] = System.getenv("HADOOP_HOME") + "/conf";
possiblePaths[3] = System.getenv("HADOOP_HOME") + "/etc/hadoop";
// hadoop 2.2
}
return
Arrays.stream(possiblePaths).filter(Objects::nonNull).toArray(String[]::new);
}
```
At Last, we should not put all configurations in properties， especially if
there have global common environment variables. That is why DS has file
`dolphinscheduler_env.sh` , similar to `spark-env.sh`;
When there are no global environment variables, there is not much difference
between putting in `dolphinscheduler_env.sh` and properties;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
