Re: Flink 连接Hive hiveConf路径配置

Fri, 18 Dec 2020 23:45:34 -0800 

谢谢回复

我想的也是如此，提交到hadoop集群，也应该能读到hive的conf dir

于是询问了DBA hive conf的路径为：/etc/hive/conf

我demo如下

public class FlinkHiveIntegration1 {
    public static void main(String[] args) throws Exception {
        EnvironmentSettings settings =
EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
        TableEnvironment tableEnv = TableEnvironment.create(settings);

        String name = "myhive";
        String defaultDatabase = "nydatabase";
        String hiveConfDir = "/etc/hive/conf";  // hive-site.xml路径
        String version = "1.1.0-cdh5.8.3";

        HiveCatalog hive = new HiveCatalog(name, defaultDatabase,
hiveConfDir, version);
        
        tableEnv.registerCatalog(name, hive);
        tableEnv.useCatalog(name);
        tableEnv.useDatabase(defaultDatabase);
        
        String hql = "select * from flink2hive_test";
        tableEnv.executeSql(hql);
    }
}

提交Job命令为：    
       
./bin/flink run -m yarn-cluster -Djobmanager.memory.process.size=4096m
-Dtaskmanager.memory.process.size=4096m -Dtaskmanager.numberOfTaskSlots=1
-Dyarn.application.name="Flink_to_hive" -c
com.test.flink.function.FlinkHiveIntegration1
/opt/app/Flink/jar/1.11.2/flink-0.0.1-SNAPSHOT.jar --parallelism 1

报错如下：   
      
2020-12-18 23:41:33,188 FATAL org.apache.hadoop.conf.Configuration              
          
[] - error parsing conf file:/etc/hive/conf/hive-site.xml
java.io.FileNotFoundException: /etc/hive/conf/hive-site.xml (No such file or
directory)

.......
.......
......
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error: java.io.FileNotFoundException: /etc/hive/conf/hive-site.xml
(No such file or directory)

提示文件不存在。。。。。





-----
Thanks!
Jacob
--
Sent from: http://apache-flink.147419.n8.nabble.com/

回复