[
https://issues.apache.org/jira/browse/FLINK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17303012#comment-17303012
]
Nicholas Jiang commented on FLINK-21829:
----------------------------------------
[~hehuiyuan], I left the comment for the custom hadoop configuration directory.
IMO, if the users configure the 'hadoopConfDir' and the 'hadoopConf' is null,
this should create new configuration, not find the possible hadoop
configuration path. Because the finding possible path could cause the
inconsistent configuration result from the perspective of users.
[~lirui], what do you think about?
> create HiveCatalog with custom hadoopconfdir first
> --------------------------------------------------
>
> Key: FLINK-21829
> URL: https://issues.apache.org/jira/browse/FLINK-21829
> Project: Flink
> Issue Type: Wish
> Components: Connectors / Hive
> Reporter: hehuiyuan
> Priority: Major
> Labels: pull-request-available
>
> here is no prompt when the the path to hadoop conf configured is wrong
> unintentional.
> {code:java}
> private static HiveConf createHiveConf(
> @Nullable String hiveConfDir, @Nullable String hadoopConfDir) {
> // create HiveConf from hadoop configuration with hadoop conf directory
> configured.
> Configuration hadoopConf = null;
> if (isNullOrWhitespaceOnly(hadoopConfDir)) {
> for (String possibleHadoopConfPath :
> HadoopUtils.possibleHadoopConfPaths(
> new org.apache.flink.configuration.Configuration())) {
> hadoopConf = getHadoopConfiguration(possibleHadoopConfPath);
> if (hadoopConf != null) {
> break;
> }
> }
> } else {
> hadoopConf = getHadoopConfiguration(hadoopConfDir);
> }
> if (hadoopConf == null) {
> hadoopConf = new Configuration();
> }
> HiveConf hiveConf = new HiveConf(hadoopConf, HiveConf.class);
> {code}
> It is better to load hadoop conf from possiable hadoop path when the path is
> wrong.
> (1) try to load from the custom hadoop conf path
> (2) try to load from possiable hadoop conf path if {{Configuration
> hadoopConf}} is null.
> (3) new Configuration if {{Configuration hadoopConf}} is null
> {code:java}
> private static HiveConf createHiveConf(
> @Nullable String hiveConfDir, @Nullable String hadoopConfDir) {
> // create HiveConf from hadoop configuration with hadoop conf directory
> configured.
> Configuration hadoopConf = null;
> if (!isNullOrWhitespaceOnly(hadoopConfDir)) {
> hadoopConf = getHadoopConfiguration(hadoopConfDir);
> }
> if (hadoopConf == null) {
> for (String possibleHadoopConfPath :
> HadoopUtils.possibleHadoopConfPaths(
> new org.apache.flink.configuration.Configuration())) {
> hadoopConf = getHadoopConfiguration(possibleHadoopConfPath);
> if (hadoopConf != null) {
> break;
> }
> }
> }
> if (hadoopConf == null) {
> hadoopConf = new Configuration();
> }
> HiveConf hiveConf = new HiveConf(hadoopConf, HiveConf.class);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
