moustafaalaa commented on issue #5291: URL: https://github.com/apache/hudi/issues/5291#issuecomment-1107843884
Glue runs in cluster mode. I tried to set the environment variable through spark session, but it doesn't work. Then I changed the below line in [DFSPropertiesConfiguration.java#L60](https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/config/DFSPropertiesConfiguration.java#L60) **From** ```java public static final String DEFAULT_CONF_FILE_DIR = "file:/etc/hudi/conf"; ``` **to** ```java public static final String DEFAULT_CONF_FILE_DIR = "file:/tmp"; ``` Then add the file path in the jar dependencies to be copied to `/tmp` folder, and it worked. ``` s3://glue-development-bucket/scripts/hudi-conf/hudi-defaults.conf ``` I am not sure if there is a better way. But this is the idea which worked with me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
