Rahul Kumar created PHOENIX-8063:
------------------------------------
Summary: Validate view indexes with PhoenixSyncTableTool,
including the shared _IDX_ table
Key: PHOENIX-8063
URL: https://issues.apache.org/jira/browse/PHOENIX-8063
Project: Phoenix
Issue Type: Improvement
Affects Versions: 5.2.3, 5.4.0, 5.3.3
Reporter: Rahul Kumar
Assignee: Rahul Kumar
Fix For: 5.2.3, 5.4.0, 5.3.3
{{PhoenixSyncTableTool}} can sync a base table or a global index. It cannot
validate tenant view indexes. Those indexes live in one shared HBase table,
{{{}_IDX_<base>{}}}, and a scan of that table without the right tenant scope
either misses rows or reports false drift.
Change
Two ways to validate {{_IDX_<base>}} data:
Logical view index. {{{}--table-name <view_index> --tenant-id <tenant>{}}}.
Phoenix compiles the usual {{SELECT}} and routes it to {{{}_IDX_<base>{}}},
filtered to that view index. Source and target connections are now opened with
the tenant id. Without that, Phoenix TTL scan attributes cannot resolve the
parent view on the target, and the job reports drift that is not there.
Shared view index. {{{}--table-name _IDX_<base>{}}}. The {{_IDX_}} prefix
selects this mode. One job scans the shared physical table and covers every
tenant, every logical view index on that base, and any orphan rows.
{{--tenant-id}} is rejected. The base table must be {{{}MULTI_TENANT{}}}. There
is no Phoenix query plan for a physical name, so input splits are built from
the HBase regions of {{{}_IDX_<base>{}}}. The mapper loads table metadata from
the base table and scans {{_IDX_<base>}} directly. Checkpoint rows are stored
under the physical {{_IDX_<base>}} name, so they do not collide with a
base-table sync of the same table.
Tests
* CLI: auto-detect from the {{_IDX_}} prefix, reject {{{}--tenant-id{}}},
reject a base that is not {{{}MULTI_TENANT{}}}, and round-trip the shared-mode
config.
* Input format: one split per HBase region, and coalescing uses the physical
name as given.
* Mapper: tenant id is set on the connection properties when present.
* End to end: shared mode matches identical data, detects rows missing on the
target, covers two logical indexes in one job, and keeps checkpoints separate
from a base-table sync. Logical mode matches identical data and detects drift
for one tenant view index.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)