steveloughran commented on a change in pull request #1925:
URL: https://github.com/apache/hadoop/pull/1925#discussion_r572291660
##########
File path: hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
##########
@@ -877,6 +877,21 @@ enabled for your Azure Storage account."
The directories can be specified as comma separated values. By default the
value
is "/hbase"
+### <a name="singlewriteroptions"></a> Single Writer Options
+`fs.azure.singlewriter.directories`: Directories for single writer support
+can be specified comma separated in this config. By default, multiple
+clients will be able to write to the same file simultaneously. When writing
+to files contained within the directories specified in this config, the
+client will obtain a lease on the file that will prevent any other clients
+from writing to the file. The lease will be renewed by the client until the
+output stream is closed, after which it will be released. To revoke a client's
+write access for a file, the AzureBlobFilesystem breakLease method may be
+ called.
+
+`fs.azure.lease.threads`: This is the size of the thread pool that will be
+used for lease operations for single writer directories. By default the value
+is 0, so it must be set to at least 1 to support single writer directories.
Review comment:
good writeup
1. is there any validation here, that if a path in the local FS is to be
leased then the executor count must be >1?
2. what if I'm working with >1 FS? Will this configuration be per-fs? Or
does it take a list of paths which can be full URIs to paths in a store? That's
what we ended up doing with s3a authoritative paths0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
