Hieu Nguyen created PHOENIX-5295:
------------------------------------
Summary: Local Index data not replicating for older HBase versions
(<= HBase 1.2)
Key: PHOENIX-5295
URL: https://issues.apache.org/jira/browse/PHOENIX-5295
Project: Phoenix
Issue Type: Bug
Environment: Branch 4.14-cdh5.11
Reporter: Hieu Nguyen
Copied from email thread
https://lists.apache.org/thread.html/7ab1d9489eca2ab2b974948fbe60b143fda432ef7dfc603528d460f2@%3Cuser.phoenix.apache.org%3E.
---
We are on Phoenix 4.14-cdh5.11. We are experiencing an issue where local index
data is not being replicated through HBase replication. As suggested in a
previous email thread
(https://lists.apache.org/thread.html/984fba3c8abd944846deefb3ea285195e0436b9181b9779feac39b59@%3Cuser.phoenix.apache.org%3E),
we have enabled replication for the local indexes (the "L#0" column family on
the same table). We wrote an integration test to demonstrate this issue on top
of 4.14-cdh5.11 branch
(https://github.com/hnguyen08/phoenix/commit/3589cb45d941c6909fb3deb5f5abb0f8dfa78dd7).
After some investigation and debugging, we discovered the following:
1. Commit a2f4d7eebec621b58204a9eb78d552f18dcbcf24 (PHOENIX-3827) fixed the
issue, but only in Phoenix for HBase1.3+. It uses the
miniBatchOp.addOperationsFromCP() API introduced in HBase1.3. Unfortunately,
for the time being, we are stuck on cdh5.11 (based on HBase1.2).
2. IndexUtil.writeLocalUpdates() is called in both implementations of
IndexCommitter, both taking skipWAL=true. It seems like we'd actually want to
not skip WAL to ensure that local-index updates are replicated correctly
(since, as mentioned in the above email thread, "HBase-level replication of the
data table will not trigger index updates"). After changing the skipWAL flag
to false, the above integration test passes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
