[
https://issues.apache.org/jira/browse/HADOOP-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17935707#comment-17935707
]
ASF GitHub Bot commented on HADOOP-19497:
-----------------------------------------
bhattmanish98 opened a new pull request, #7509:
URL: https://github.com/apache/hadoop/pull/7509
We have implemented create and rename recovery using client transaction IDs
over the DFS endpoint
([HADOOP-19450](https://issues.apache.org/jira/browse/HADOOP-19450) [ABFS]
Rename/Create path idempotency client-level resolution - ASF JIRA). Since the
backend changes were not fully rolled out, we initially implemented the changes
with the flag disabled. With this update, we aim to enable the flag, which will
start sending client transaction IDs. In case of a failure, we will attempt to
recover from the failed state if possible. Here are the detailed steps and
considerations for this process:
1. *Implementation Overview*: We introduced a mechanism for create and
rename recovery via client transaction IDs to enhance reliability and data
integrity over the DFS endpoint. This change was initially flagged as disabled
due to incomplete backend rollouts.
2. *Current Update*: With the backend changes now in place, we are ready to
enable the flag. This will activate the sending of client transaction IDs,
allowing us to track and manage transactions more effectively.
3. *Failure Recovery*: The primary advantage of enabling this flag is the
potential for recovery from failed states. If a transaction fails, we can use
the client transaction ID to attempt a recovery, minimizing data loss and
ensuring continuity.
4. *Next Steps*: We will proceed with enabling the flag and closely monitor
the system's performance. Any issues or failures will be documented and
addressed promptly to ensure a smooth transition.
> [ABFS] Enable rename and create recovery from client transaction id over DFS
> endpoint
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-19497
> URL: https://issues.apache.org/jira/browse/HADOOP-19497
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.5.0
> Reporter: Manish Bhatt
> Assignee: Manish Bhatt
> Priority: Major
>
> We have implemented create and rename recovery using client transaction IDs
> over the DFS endpoint ([HADOOP-19450] [ABFS] Rename/Create path idempotency
> client-level resolution - ASF JIRA). Since the backend changes were not fully
> rolled out, we initially implemented the changes with the flag disabled. With
> this update, we aim to enable the flag, which will start sending client
> transaction IDs. In case of a failure, we will attempt to recover from the
> failed state if possible. Here are the detailed steps and considerations for
> this process:
> 1. **Implementation Overview**: We introduced a mechanism for create and
> rename recovery via client transaction IDs to enhance reliability and data
> integrity over the DFS endpoint. This change was initially flagged as
> disabled due to incomplete backend rollouts.
> 2. **Current Update**: With the backend changes now in place, we are ready to
> enable the flag. This will activate the sending of client transaction IDs,
> allowing us to track and manage transactions more effectively.
> 3. **Failure Recovery**: The primary advantage of enabling this flag is the
> potential for recovery from failed states. If a transaction fails, we can use
> the client transaction ID to attempt a recovery, minimizing data loss and
> ensuring continuity.
> 4. **Next Steps**: We will proceed with enabling the flag and closely monitor
> the system's performance. Any issues or failures will be documented and
> addressed promptly to ensure a smooth transition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
