[
https://issues.apache.org/jira/browse/FLINK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17303187#comment-17303187
]
hehuiyuan commented on FLINK-21829:
-----------------------------------
Hi [~lirui] , i have some questions.
The method `getHadoopConfiguration` is used by possiable hadoop conf path and
user custom hadoop path .
If possiable path does not exist without user custom hadoop path ， shoud it
continue or throw exception ?
{code:java}
possiblePaths[0] =
flinkConfiguration.getString(ConfigConstants.PATH_HADOOP_CONFIG, null);
possiblePaths[1] = System.getenv("HADOOP_CONF_DIR");
if (System.getenv("HADOOP_HOME") != null) {
possiblePaths[2] = System.getenv("HADOOP_HOME") + "/conf";
possiblePaths[3] = System.getenv("HADOOP_HOME") + "/etc/hadoop"; // hadoop
2.2
}
{code}
> create HiveCatalog with custom hadoopconfdir first
> --------------------------------------------------
>
> Key: FLINK-21829
> URL: https://issues.apache.org/jira/browse/FLINK-21829
> Project: Flink
> Issue Type: Wish
> Components: Connectors / Hive
> Reporter: hehuiyuan
> Priority: Major
> Labels: pull-request-available
>
> here is no prompt when the the path to hadoop conf configured is wrong
> unintentional.
> {code:java}
> private static HiveConf createHiveConf(
> @Nullable String hiveConfDir, @Nullable String hadoopConfDir) {
> // create HiveConf from hadoop configuration with hadoop conf directory
> configured.
> Configuration hadoopConf = null;
> if (isNullOrWhitespaceOnly(hadoopConfDir)) {
> for (String possibleHadoopConfPath :
> HadoopUtils.possibleHadoopConfPaths(
> new org.apache.flink.configuration.Configuration())) {
> hadoopConf = getHadoopConfiguration(possibleHadoopConfPath);
> if (hadoopConf != null) {
> break;
> }
> }
> } else {
> hadoopConf = getHadoopConfiguration(hadoopConfDir);
> }
> if (hadoopConf == null) {
> hadoopConf = new Configuration();
> }
> HiveConf hiveConf = new HiveConf(hadoopConf, HiveConf.class);
> {code}
> It is better to load hadoop conf from possiable hadoop path when the path is
> wrong.
> (1) try to load from the custom hadoop conf path
> (2) try to load from possiable hadoop conf path if {{Configuration
> hadoopConf}} is null.
> (3) new Configuration if {{Configuration hadoopConf}} is null
> {code:java}
> private static HiveConf createHiveConf(
> @Nullable String hiveConfDir, @Nullable String hadoopConfDir) {
> // create HiveConf from hadoop configuration with hadoop conf directory
> configured.
> Configuration hadoopConf = null;
> if (!isNullOrWhitespaceOnly(hadoopConfDir)) {
> hadoopConf = getHadoopConfiguration(hadoopConfDir);
> }
> if (hadoopConf == null) {
> for (String possibleHadoopConfPath :
> HadoopUtils.possibleHadoopConfPaths(
> new org.apache.flink.configuration.Configuration())) {
> hadoopConf = getHadoopConfiguration(possibleHadoopConfPath);
> if (hadoopConf != null) {
> break;
> }
> }
> }
> if (hadoopConf == null) {
> hadoopConf = new Configuration();
> }
> HiveConf hiveConf = new HiveConf(hadoopConf, HiveConf.class);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
