dlmarion commented on code in PR #3180:
URL: https://github.com/apache/accumulo/pull/3180#discussion_r1097414905
##########
server/base/src/main/java/org/apache/accumulo/server/fs/VolumeManagerImpl.java:
##########
@@ -353,6 +362,38 @@ public short getDefaultReplication(Path path) {
return getFileSystemByPath(path).getDefaultReplication(path);
}
+ /**
+ * The Hadoop Configuration object does not currently allow for duplicate
properties to be set in
+ * a single Configuration for different FileSystem URIs. Here we will look
for properties in the
+ * Accumulo configuration of the form:
+ *
+ * <pre>
+ * general.custom.<volume-uri>.<hdfs-property>
+ * </pre>
+ *
+ * We will use these properties to return a new Configuration object that
can be used with the
+ * FileSystem URI.
+ *
+ * @param conf AccumuloConfiguration object
+ * @param hadoopConf Hadoop Configuration object
+ * @param filesystemURI Volume Filesystem URI
+ * @return Hadoop Configuration with custom overrides for this FileSystem
+ */
+ private static Configuration
getVolumeManagerConfiguration(AccumuloConfiguration conf,
+ final Configuration hadoopConf, final String filesystemURI) {
+ final Configuration volumeConfig = new Configuration(hadoopConf);
+ final Map<String,String> customProps =
+
conf.getAllPropertiesWithPrefixStripped(Property.GENERAL_ARBITRARY_PROP_PREFIX);
+ customProps.forEach((key, value) -> {
+ if (key.startsWith(filesystemURI)) {
+ String property = key.substring(filesystemURI.length() + 1);
+ log.debug("Overriding property {} to {} for volume {}", property,
value, filesystemURI);
Review Comment:
It should only happen once, at server startup. Increased logging level to
INFO in e0c12c4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
