[
https://issues.apache.org/jira/browse/HADOOP-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17921285#comment-17921285
]
ASF GitHub Bot commented on HADOOP-19232:
-----------------------------------------
anmolanmol1234 commented on code in PR #7272:
URL: https://github.com/apache/hadoop/pull/7272#discussion_r1930120495
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##########
@@ -377,8 +390,199 @@ public AbfsRestOperation createPath(final String path,
final String eTag,
final ContextEncryptionAdapter contextEncryptionAdapter,
final TracingContext tracingContext) throws AzureBlobFileSystemException
{
- // TODO: [FnsOverBlob][HADOOP-19232] To be implemented as part of ingress
support.
- throw new NotImplementedException("Create Path operation on Blob endpoint
yet to be implemented.");
+ return createPath(path, isFile, overwrite, permissions, isAppendBlob, eTag,
+ contextEncryptionAdapter, tracingContext, false);
+ }
+
+ /**
+ * Get Rest Operation for API
+ * <a
href="https://learn.microsoft.com/en-us/rest/api/storageservices/put-blob";>Put
Blob</a>.
+ * Creates a file or directory (marker file) at the specified path.
+ *
+ * @param path the path of the directory to be created.
+ * @param isFile whether the path is a file.
+ * @param overwrite whether to overwrite if the path already exists.
+ * @param permissions the permissions to set on the path.
+ * @param isAppendBlob whether the path is an append blob.
+ * @param eTag the eTag of the path.
+ * @param contextEncryptionAdapter the context encryption adapter.
+ * @param tracingContext the tracing context.
+ * @param isCreateCalledFromMarkers whether the create is called from
markers.
+ * @return the executed rest operation containing the response from the
server.
+ * @throws AzureBlobFileSystemException if the rest operation fails.
+ */
+ public AbfsRestOperation createPath(final String path,
+ final boolean isFile,
+ final boolean overwrite,
+ final AzureBlobFileSystemStore.Permissions permissions,
+ final boolean isAppendBlob,
+ final String eTag,
+ final ContextEncryptionAdapter contextEncryptionAdapter,
+ final TracingContext tracingContext,
+ boolean isCreateCalledFromMarkers) throws AzureBlobFileSystemException {
+ final List<AbfsHttpHeader> requestHeaders = createDefaultHeaders();
+ if (!getIsNamespaceEnabled() && !isCreateCalledFromMarkers) {
+ AbfsHttpOperation op1Result = null;
+ try {
+ op1Result = getPathStatus(path, tracingContext,
+ null, true).getResult();
+ } catch (AbfsRestOperationException ex) {
+ if (ex.getStatusCode() == HTTP_NOT_FOUND) {
+ LOG.debug("No directory/path found: {}", path);
+ } else {
+ throw ex;
+ }
+ }
+ if (op1Result != null) {
+ boolean isDir = checkIsDir(op1Result);
+ if (isFile == isDir) {
+ throw new AbfsRestOperationException(HTTP_CONFLICT,
+ AzureServiceErrorCode.PATH_CONFLICT.getErrorCode(),
+ PATH_EXISTS,
+ null);
+ }
+ }
+ Path parentPath = new Path(path).getParent();
+ if (parentPath != null && !parentPath.isRoot()) {
+ createMarkers(parentPath, overwrite, permissions, isAppendBlob, eTag,
+ contextEncryptionAdapter, tracingContext);
+ }
+ }
+ if (isFile) {
+ addEncryptionKeyRequestHeaders(path, requestHeaders, true,
+ contextEncryptionAdapter, tracingContext);
+ } else {
+ requestHeaders.add(new AbfsHttpHeader(X_MS_META_HDI_ISFOLDER, TRUE));
+ }
+ requestHeaders.add(new AbfsHttpHeader(CONTENT_LENGTH, ZERO));
+ if (isAppendBlob) {
+ requestHeaders.add(new AbfsHttpHeader(X_MS_BLOB_TYPE, APPEND_BLOB_TYPE));
+ } else {
+ requestHeaders.add(new AbfsHttpHeader(X_MS_BLOB_TYPE, BLOCK_BLOB_TYPE));
+ }
+ if (!overwrite) {
+ requestHeaders.add(new AbfsHttpHeader(IF_NONE_MATCH,
AbfsHttpConstants.STAR));
+ }
+ if (eTag != null && !eTag.isEmpty()) {
+ requestHeaders.add(new AbfsHttpHeader(HttpHeaderConfigurations.IF_MATCH,
eTag));
+ }
+
+ final AbfsUriQueryBuilder abfsUriQueryBuilder =
createDefaultUriQueryBuilder();
+ appendSASTokenToQuery(path, SASTokenProvider.FIXED_SAS_STORE_OPERATION,
abfsUriQueryBuilder);
+
+ final URL url = createRequestUrl(path, abfsUriQueryBuilder.toString());
+ final AbfsRestOperation op = getAbfsRestOperation(
+ AbfsRestOperationType.PutBlob,
+ HTTP_METHOD_PUT, url, requestHeaders);
+ try {
+ op.execute(tracingContext);
+ } catch (AzureBlobFileSystemException ex) {
+ // If we have no HTTP response, throw the original exception.
+ if (!op.hasResult()) {
+ throw ex;
Review Comment:
taken
> ABFS: [FnsOverBlob] Implementing Ingress Support with various Fallback
> Handling
> -------------------------------------------------------------------------------
>
> Key: HADOOP-19232
> URL: https://issues.apache.org/jira/browse/HADOOP-19232
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.0
> Reporter: Anuj Modi
> Assignee: Anmol Asrani
> Priority: Major
> Labels: pull-request-available
>
> Scope of this task is to refactor the AbfsOutputStream class to handle the
> ingress for DFS and Blob endpoint effectively.
> More details will be added soon.
> Perquisites for this Patch:
> 1. [HADOOP-19187] ABFS: [FnsOverBlob]Making AbfsClient Abstract for
> supporting both DFS and Blob Endpoint - ASF JIRA (apache.org)
> 2. [HADOOP-19226] ABFS: [FnsOverBlob]Implementing Azure Rest APIs on Blob
> Endpoint for AbfsBlobClient - ASF JIRA (apache.org)
> 3. [HADOOP-19207] ABFS: [FnsOverBlob]Response Handling of Blob Endpoint APIs
> and Metadata APIs - ASF JIRA (apache.org)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
