lfyee edited a comment on issue #1217: why I cannot push files to hdfs URL: https://github.com/apache/incubator-dolphinscheduler/issues/1217#issuecomment-553356457 If the dolphinscheduler is already running, if you want to enable HDFS in the resource center; (如果dolphinscheduler已经在运行，这时候如果要在资源中心开启HDFS功能) ## 1、Modify configuration file. ## (修改如下的配置文件) ### conf/common/common.properties ### ``` # Users who have permission to create directories under the HDFS root path hdfs.root.user=hdfs # data base dir, resource file will store to this hadoop hdfs path, self configuration, please make sure the directory exists on hdfs and have read write permissions。"/escheduler" is recommended data.store2hdfs.basepath=/escheduler # resource upload startup type : HDFS,S3,NONE res.upload.startup.type=HDFS # whether kerberos starts hadoop.security.authentication.startup.state=false # java.security.krb5.conf path java.security.krb5.conf.path=/opt/krb5.conf # loginUserFromKeytab user [email protected] # loginUserFromKeytab path login.user.keytab.path=/opt/hdfs.headless.keytab ``` Modify the corresponding parameters according to your own cluster environment. Both api-server and worker-server services need to be configured, modified, and restarted. Since the system is already running, it will not help us create the root and tenant directories, so we need to manually create the root and tenant-related directories. (根据自己集群环境修改对应的参数，api-server和worker-server两类服务都需要做配置，修改并重启服务，由于系统已经在运行，因此无法帮助我们创建根目录和租户目录，因此我们需要手动创建与根目录和租户相关的目录) E.g deploy user: dolphinscheduler data.store2hdfs.basepath：/escheduler tenant user：tim ``` # mkdir root hadoop fs -mkdir /escheduler # mkdir tenant hadoop fs -mkdir -p /escheduler/tim/{resources,udfs} # change hadoop fs -chown -R dolphinscheduler:dolphinscheduler /escheduler ``` ### conf/common/hadoop/hadoop.properties ### ``` # ha or single namenode,If namenode ha needs to copy core-site.xml and hdfs-site.xml # to the conf directory，support s3，for example : s3a://dolphinscheduler fs.defaultFS=hdfs://mycluster:8020 ``` ## 2、Copy File ## (复制文件) Copy core-site.xml, hdfs-site.xml from the Hadoop cluster to the conf directory.Both api-server and worker-server services need to be configured, modified, and restarted (从Hadoop集群复制core-site.xml、hdfs-site.xml到conf目录下，api-server和worker-server两类服务都需要做配置，修改并重启服务) After the modification, you can use the HDFS function.(修改完后，就可以使用HDFS功能)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
