LuciferYang opened a new issue, #13992:
URL: https://github.com/apache/gravitino/issues/13992

   ### Version
   
   main branch
   
   ### Describe what's wrong
   
   For HDFS, `HDFSFileSystemProvider.additionalHDFSConfig` injects 
`ipc.client.connect.timeout=5000` and `ipc.client.ping=true` whenever the plain 
key is absent. It does not check the `gravitino.bypass.` form, so a user who 
sets `gravitino.bypass.ipc.client.connect.timeout` ends up with both keys in 
the map.
   
   `FileSystemUtils.createConfiguration` then writes every entry into one 
Hadoop configuration resource in `HashMap` order. Hadoop keeps the last value 
of a key within a resource, so which of the two values wins depends on the 
map's bucket order, that is, on how many other properties the catalog, schema 
and fileset have. With Hadoop 3.3.6, the user value applies while the map holds 
8 to 88 entries and is silently replaced by 5000 from about 92 entries on. The 
same order dependence decides between any plain key and its `gravitino.bypass.` 
twin, while `toHadoopConfigMap` documents that the plain key wins.
   
   ### Error message and/or stacktrace
   
   _No response_
   
   ### How to reproduce
   
   Create an HDFS fileset catalog with 
`gravitino.bypass.ipc.client.connect.timeout=60000` and more than about 90 
other properties across the catalog, schema and fileset. The HDFS client uses a 
5 s connect timeout. With few properties, the same setting takes effect.
   
   ### Additional context
   
   _No response_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to