[
https://issues.apache.org/jira/browse/PHOENIX-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107973#comment-16107973
]
James Taylor commented on PHOENIX-4052:
---------------------------------------
The following test, slightly different than the description above, reproduces
an issue with an extra row with an invalid row key ending up in the index table:
{code}
@Test
public void testOutOfOrderDelete() throws Exception {
String tableName = generateUniqueName();
String indexName = generateUniqueName();
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
long ts = 1000;
props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts));
Connection conn = DriverManager.getConnection(getUrl(), props);
conn.createStatement().execute("CREATE TABLE " + tableName + "(k
CHAR(1) PRIMARY KEY, v VARCHAR) COLUMN_ENCODED_BYTES = 0");
conn.close();
ts = 1010;
props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts));
conn = DriverManager.getConnection(getUrl(), props);
conn.createStatement().execute("CREATE INDEX " + indexName + " ON " +
tableName + "(v)");
conn.close();
ts = 1020;
props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts));
conn = DriverManager.getConnection(getUrl(), props);
conn.createStatement().execute("UPSERT INTO " + tableName + "
VALUES('a','a')");
conn.commit();
conn.close();
ts = 1040;
props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts));
conn = DriverManager.getConnection(getUrl(), props);
conn.createStatement().execute("DELETE FROM " + tableName + " WHERE
k='a'");
conn.commit();
conn.close();
ts = 1030;
props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts));
conn = DriverManager.getConnection(getUrl(), props);
conn.createStatement().execute("UPSERT INTO " + tableName + "
VALUES('a','bbb')");
conn.commit();
conn.close();
TestUtil.dumpTable(conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(Bytes.toBytes(tableName)));
TestUtil.dumpTable(conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(Bytes.toBytes(indexName)));
ts = 1050;
props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts));
conn = DriverManager.getConnection(getUrl(), props);
long count1 = getRowCount(conn, tableName);
long count2 = getRowCount(conn, indexName);
assertTrue("Expected table row count ( " + count1 + ") to match index
row count (" + count2 + ")", count1 == count2);
conn.close();
/**
*
************ dumping T000001;hconnection-0x8f57e4c
**************
a/0:/1040/DeleteFamily/vlen=0/seqid=0
a/0:V/1030/Put/vlen=3/seqid=0
a/0:V/1020/Put/vlen=1/seqid=0
a/0:_0/1030/Put/vlen=1/seqid=0
a/0:_0/1020/Put/vlen=1/seqid=0
-----------------------------------------------
************ dumping T000002;hconnection-0x8f57e4c
**************
\x00a/0:_0/1040/Put/vlen=2/seqid=0
a\x00a/0:/1040/DeleteFamily/vlen=0/seqid=0
a\x00a/0:/1030/DeleteFamily/vlen=0/seqid=0
a\x00a/0:_0/1020/Put/vlen=2/seqid=0
-----------------------------------------------
*/
}
private static long getRowCount(Connection conn, String tableName) throws
SQLException {
ResultSet rs = conn.createStatement().executeQuery("SELECT /*+ NO_INDEX
*/ count(*) FROM " + tableName);
assertTrue(rs.next());
return rs.getLong(1);
}
{code}
Assertion that occurs is:
{code}
java.lang.AssertionError: Expected table row count ( 0) to match index row
count (1)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.phoenix.end2end.index.OutOfOrderMutationsIT.testOutOfOrderDelete(OutOfOrderMutationsIT.java:135)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
{code}
> Create the correct index row mutations for out-of-order data mutations
> ----------------------------------------------------------------------
>
> Key: PHOENIX-4052
> URL: https://issues.apache.org/jira/browse/PHOENIX-4052
> Project: Phoenix
> Issue Type: Bug
> Reporter: James Taylor
>
> Out-of-order processing of data rows during index maintenance causes mutable
> indexes to become out of sync with regard to the data table. Here's a simple
> example to illustrate the issue:
> # Assume table T(K,V) and index X(V,K).
> # Upsert T(A, 1) at t10. Index updates: Put X(1,A) at t10.
> # Upsert T(A, 3) at t30. Index updates: Delete X(1,A) at t29, Put X(3,A) at
> t30.
> # Upsert T(A,2) at t20. Index updates: Delete X(1,A) at t19, Put X(2,A) at
> t20, Delete X(2,A) at t29
> Ideally, we'd want to remove the Delete X(1,A) at t29 since this isn't
> correct in terms of timeline consistency, but we can't do that with HBase
> without support for deleting/undoing Delete markers.
> The above is not what is occurring. Instead, when T(A,2) comes in, the Put
> X(2,A) will occur at t20, but the Delete won't occur. This causes more index
> rows than data rows, essentially making it invalid.
> See PHOENIX-4051 as a workaround for fixing this issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
