dlmarion opened a new pull request, #3180: URL: https://github.com/apache/accumulo/pull/3180
When creating Hadoop FileSystem objects the FileSystem class caches FileSystem objects using the filesystem URI scheme and authority fields. Prior to this change, the Accumulo VolumeManager would create FileSystem objects (one per volume) using the standard Hadoop Configuration files and resources. However, user-supplied Hadoop configuration files (core-site.xml and hdfs-site.xml) don't allow the user to specify different properties for different filesystems. For example, a user can create a core-site.xml and hdfs-site.xml file locally and create FileSystem objects that connect to different HDFS clusters. A property specified in the Hadoop configuration will apply to all FileSystem objects. With this change users can specify Hadoop configuration overrides in the Accumulo configuration using the property format: general.custom.<volume-uri>.<property-name>=<property-value> This enables a user with multiple filesystems specified in the instance.volumes property to provide different Hadoop configuration overrides for each volume. For example, the following would allow different hedged read thread sizes for two different volumes: instance.volumes=hdfs://namespace-a:8020/accumulo,hdfs://namespace-b:8020/accumulo general.custom.hdfs://namespace-a:8020/accumulo.dfs.client.hedged.read.threadpool.size=10 general.custom.hdfs://namespace-b:8020/accumulo.dfs.client.hedged.read.threadpool.size=20 Note: When specifying property names that contain colons in the accumulo.properties files, they need to be escaped with a backslash -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
