那你在命令行执行：hadoop classpath，有hadoop的classpath输出吗？
Zhou Zach <[email protected]> 于2020年6月16日周二 下午3:22写道： > > > > > > > 在/etc/profile下，目前只加了 > export HADOOP_CLASSPATH=`hadoop classpath` > 我是安装的CDH，没找到sbin这个文件。。 > > > > > > > > > > > > 在 2020-06-16 15:05:12，"王松" <[email protected]> 写道： > >你配置HADOOP_HOME和HADOOP_CLASSPATH这两个环境变量了吗？ > > > >export HADOOP_HOME=/usr/local/hadoop-2.7.2 > >export HADOOP_CLASSPATH=`hadoop classpath` > >export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH > > > >Zhou Zach <[email protected]> 于2020年6月16日周二 下午2:53写道： > > > >> flink/lib/下的jar： > >> flink-connector-hive_2.11-1.10.0.jar > >> flink-dist_2.11-1.10.0.jar > >> flink-jdbc_2.11-1.10.0.jar > >> flink-json-1.10.0.jar > >> flink-shaded-hadoop-2-3.0.0-cdh6.3.0-7.0.jar > >> flink-sql-connector-kafka_2.11-1.10.0.jar > >> flink-table_2.11-1.10.0.jar > >> flink-table-blink_2.11-1.10.0.jar > >> hbase-client-2.1.0.jar > >> hbase-common-2.1.0.jar > >> hive-exec-2.1.1.jar > >> mysql-connector-java-5.1.49.jar > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> 在 2020-06-16 14:48:43，"Zhou Zach" <[email protected]> 写道： > >> > > >> > > >> > > >> > > >> >high-availability.storageDir: hdfs:///flink/ha/ > >> >high-availability.zookeeper.quorum: cdh1:2181,cdh2:2181,cdh3:2181 > >> >state.backend: filesystem > >> >state.checkpoints.dir: > hdfs://nameservice1:8020//user/flink10/checkpoints > >> >state.savepoints.dir: hdfs://nameservice1:8020//user/flink10/savepoints > >> >high-availability.zookeeper.path.root: /flink > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> >在 2020-06-16 14:44:02，"王松" <[email protected]> 写道： > >> >>你的配置文件中ha配置可以贴下吗 > >> >> > >> >>Zhou Zach <[email protected]> 于2020年6月16日周二 下午1:49写道： > >> >> > >> >>> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: > Failed > >> to > >> >>> initialize the cluster entrypoint YarnJobClusterEntrypoint. > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:187) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:518) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint.main(YarnJobClusterEntrypoint.java:119) > >> >>> > >> >>> Caused by: java.io.IOException: Could not create FileSystem for > highly > >> >>> available storage path > (hdfs:/flink/ha/application_1592215995564_0027) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.blob.BlobUtils.createFileSystemBlobStore(BlobUtils.java:103) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.blob.BlobUtils.createBlobStoreFromConfig(BlobUtils.java:89) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:125) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:305) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:263) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:207) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:169) > >> >>> > >> >>> at java.security.AccessController.doPrivileged(Native Method) > >> >>> > >> >>> at javax.security.auth.Subject.doAs(Subject.java:422) > >> >>> > >> >>> at > >> >>> > >> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:168) > >> >>> > >> >>> ... 2 more > >> >>> > >> >>> Caused by: > >> org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: > >> >>> Could not find a file system implementation for scheme 'hdfs'. The > >> scheme > >> >>> is not directly supported by Flink and no Hadoop file system to > support > >> >>> this scheme could be loaded. > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:450) > >> >>> > >> >>> at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:362) > >> >>> > >> >>> at org.apache.flink.core.fs.Path.getFileSystem(Path.java:298) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.blob.BlobUtils.createFileSystemBlobStore(BlobUtils.java:100) > >> >>> > >> >>> ... 13 more > >> >>> > >> >>> Caused by: > >> org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: > >> >>> Cannot support file system for 'hdfs' via Hadoop, because Hadoop is > >> not in > >> >>> the classpath, or some classes are missing from the classpath. > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:184) > >> >>> > >> >>> at > >> >>> > >> > org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:446) > >> >>> > >> >>> ... 16 more > >> >>> > >> >>> Caused by: java.lang.VerifyError: Bad return type > >> >>> > >> >>> Exception Details: > >> >>> > >> >>> Location: > >> >>> > >> >>> > >> >>> > >> > org/apache/hadoop/hdfs/DFSClient.getQuotaUsage(Ljava/lang/String;)Lorg/apache/hadoop/fs/QuotaUsage; > >> >>> @160: areturn > >> >>> > >> >>> Reason: > >> >>> > >> >>> Type 'org/apache/hadoop/fs/ContentSummary' (current frame, > >> stack[0]) > >> >>> is not assignable to 'org/apache/hadoop/fs/QuotaUsage' (from method > >> >>> signature) > >> >>> > >> >>> Current Frame: > >> >>> > >> >>> bci: @160 > >> >>> > >> >>> flags: { } > >> >>> > >> >>> locals: { 'org/apache/hadoop/hdfs/DFSClient', > 'java/lang/String', > >> >>> 'org/apache/hadoop/ipc/RemoteException', 'java/io/IOException' } > >> >>> stack: { 'org/apache/hadoop/fs/ContentSummary' } > >> >>> > >> >>> > >> >>> > >> >>> 在本地intellij idea中可以正常运行，flink job上订阅kafka，sink到mysql和hbase，集群flink > >> lib目录下， > >> >>> > >> >>> > >> >>> > >> >>> > >> >
