Amar1404 opened a new issue, #8730: URL: https://github.com/apache/hudi/issues/8730
**Describe the problem you faced** Hi I am trying to use the Enviorment Variable Feature as per the doc. https://hudi.apache.org/docs/next/configurations#ENVIRONMENT_CONFIG I have create the spark-env and yarn-env the enviorment variable i am able to access but this feature is not working the default file is loaded from ClassLoader instead of FILE location first as per the code. public static TypedProperties loadGlobalProps() { DFSPropertiesConfiguration conf = new DFSPropertiesConfiguration(); // First try loading the external config file from class loader URL configFile = Thread.currentThread().getContextClassLoader().getResource(DEFAULT_PROPERTIES_FILE); if (configFile != null) { try (BufferedReader br = new BufferedReader(new InputStreamReader(configFile.openStream()))) { conf.addPropsFromStream(br); return conf.getProps(); } catch (IOException ioe) { throw new HoodieIOException( String.format("Failed to read %s from class loader", DEFAULT_PROPERTIES_FILE), ioe); } } // Try loading the external config file from local file system Option<Path> defaultConfPath = getConfPathFromEnv(); if (defaultConfPath.isPresent()) { conf.addPropsFromFile(defaultConfPath.get()); } else { try { conf.addPropsFromFile(DEFAULT_PATH); } catch (Exception e) { LOG.warn("Cannot load default config file: " + DEFAULT_PATH, e); } } return conf.getProps(); } As per the doc the file name should be hudi-default.conf but the file name used in code is hudi-defaults.conf. **To Reproduce** Steps to reproduce the behavior: 1. Set the Common Properties in the file name hudi-defaults.conf and put in s3 2.Set the enviorment variables of spark and yarn HUDI_CONF_DIR to the path dir of s3 object kept 3. Run the Hudi Deltastream SyncOnce one time. 4. **Expected behavior** A clear and concise description of what you expected to happen. **Environment Description** * Hudi version : 0.12.1 * Spark version : 3.3.3 * Hive version :3.1.2 * Hadoop version : * Storage (HDFS/S3/GCS..) : S3 * Running on Docker? (yes/no) : no **Additional context** Add any other context about the problem here. No Error is shown of print Info about the Using the DEFAULT Config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
