[
https://issues.apache.org/jira/browse/PHOENIX-7750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahul Kumar updated PHOENIX-7750:
---------------------------------
Description:
It is required to have a tool that is scalable and efficient to compare and
synchronize copies of tables within a pair of Phoenix clusters. The current
tools available from HBase are not sufficiently reliable and efficient to be
used in large scale production deployments.
HBase replication has traditionally been used to replicate tables between
Phoenix clusters for increased availability and disaster recovery. However,
this method has encountered issues, including operational problems,
misconfigurations, and replication bugs, which can lead to incomplete file
synchronization between Phoenix clusters.
Existing HBase tools have shortcomings to be used on a regular basis to verify
that tables are replicated correctly or resynchronize them when needed. This is
mainly because these tools are not reliable or sufficiently efficient when
tables are large.
The upcoming re-architected Phoenix HA feature incorporates a new replication
solution. This introduces an even greater need for an enhanced tool to verify
the correctness of this new solution.
Design Doc:
https://docs.google.com/document/d/1OxF4gvj8jRvBvmtGJooO_nbGPrmyyQPRsyCjW3EB9KM/edit?tab=t.0
was:
It is required to have a tool that is scalable and efficient to compare and
synchronize copies of tables within a pair of Phoenix clusters. The current
tools available from HBase are not sufficiently reliable and efficient to be
used in large scale production deployments.
HBase replication has traditionally been used to replicate tables between
Phoenix clusters for increased availability and disaster recovery. However,
this method has encountered issues, including operational problems,
misconfigurations, and replication bugs, which can lead to incomplete file
synchronization between Phoenix clusters.
Existing HBase tools have shortcomings to be used on a regular basis to verify
that tables are replicated correctly or resynchronize them when needed. This is
mainly because these tools are not reliable or sufficiently efficient when
tables are large.
The upcoming re-architected Phoenix HA feature incorporates a new replication
solution. This introduces an even greater need for an enhanced tool to verify
the correctness of this new solution.
> Create PhoenixSyncTable MR Tool to help validate and sync table data b/w
> source and target cluster
> --------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-7750
> URL: https://issues.apache.org/jira/browse/PHOENIX-7750
> Project: Phoenix
> Issue Type: Task
> Affects Versions: 5.2.0, 5.2.1, 5.3.0
> Reporter: Rahul Kumar
> Assignee: Rahul Kumar
> Priority: Major
>
> It is required to have a tool that is scalable and efficient to compare and
> synchronize copies of tables within a pair of Phoenix clusters. The current
> tools available from HBase are not sufficiently reliable and efficient to be
> used in large scale production deployments.
> HBase replication has traditionally been used to replicate tables between
> Phoenix clusters for increased availability and disaster recovery. However,
> this method has encountered issues, including operational problems,
> misconfigurations, and replication bugs, which can lead to incomplete file
> synchronization between Phoenix clusters.
> Existing HBase tools have shortcomings to be used on a regular basis to
> verify that tables are replicated correctly or resynchronize them when
> needed. This is mainly because these tools are not reliable or sufficiently
> efficient when tables are large.
> The upcoming re-architected Phoenix HA feature incorporates a new replication
> solution. This introduces an even greater need for an enhanced tool to verify
> the correctness of this new solution.
> Design Doc:
> https://docs.google.com/document/d/1OxF4gvj8jRvBvmtGJooO_nbGPrmyyQPRsyCjW3EB9KM/edit?tab=t.0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
