[ https://issues.apache.org/jira/browse/PHOENIX-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066338#comment-17066338 ]
Hadoop QA commented on PHOENIX-5795: ------------------------------------ {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12997608/PHOENIX-5795.4.x-HBase-1.5.001.patch against 4.x-HBase-1.5 branch at commit 457ae44cf09231fd0122b3932623d82c3a8b932a. ATTACHMENT ID: 12997608 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 25 new or modified tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: + private long verifyIndexTable(String tableName, String indexName, Connection conn) throws Exception { + // Now we rebuild the entire index table and expect that it is still good after the full rebuild + long actualRowCountAfterCompaction = IndexScrutiny.scrutinizeIndex(conn, tableName, indexName); + + "(k1 INTEGER NOT NULL, k2 INTEGER NOT NULL, a.v1 INTEGER, b.v2 INTEGER, c.v3 INTEGER, d.v4 INTEGER," + + conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + tableName + "(v1) INCLUDE(v2, v3)"); + + (RAND.nextBoolean() ? null : (RAND.nextInt() % nIndexValues)) + ", " + + "(k1 INTEGER NOT NULL, k2 INTEGER NOT NULL, a.v1 INTEGER, b.v2 INTEGER, c.v3 INTEGER, d.v4 INTEGER," + + conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + tableName + "(v1) INCLUDE(v2, v3)"); + "UPSERT INTO " + tableName + " (k1, k2, b.v2, c.v3, d.v4) VALUES (" + + (RAND.nextBoolean() ? null : RAND.nextInt()) + ", " {color:red}-1 core tests{color}. The patch failed these unit tests: ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OnDuplicateKeyIT Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3647//testReport/ Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3647//console This message is automatically generated. > Supporting selective queries for index rows updated concurrently > ---------------------------------------------------------------- > > Key: PHOENIX-5795 > URL: https://issues.apache.org/jira/browse/PHOENIX-5795 > Project: Phoenix > Issue Type: Sub-task > Reporter: Kadir OZDEMIR > Assignee: Kadir OZDEMIR > Priority: Critical > Attachments: PHOENIX-5795.4.x-HBase-1.5.001.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > From the consistent indexing design (PHOENIX-5156) perspective, two or more > pending updates from different batches on the same data row are concurrent if > and only if for all of these updates the data table row state is read from > HBase under the row lock and for none of them the row lock has been acquired > the second time for updating the data table. In other words, all of them are > in the first update phase concurrently. For concurrent updates, the first two > update phases are done but the last update phase is skipped. This means the > data table row will be updated by these updates but the corresponding index > table rows will be left with the unverified status. Then, the read repair > process will repair these unverified index rows during scans. > In addition to leaving index rows unverified, the concurrent updates may > generate index row with incorrect row keys. For example, consider that an > application issues the verify first two upserts on the same row concurrently > and the second update does not include one or more of the indexed columns. > When these updates arrive concurrently to IndexRegionObserver, the existing > row state would be null for both of these updates. This mean the index > updates will be generated solely from the pending updates. The partial upsert > with missing indexed columns will generate an index row by assuming missing > indexed columns have null value, and this assumption may not true as the > other concurrent upsert may have non-null values for indexed columns. After > issuing the concurrent update, if the application attempts to read back the > row using a selective query on the index table and this selective query maps > to an HBase scan that does not scan these unverified rows due to incorrect > row keys on these rows, the application will not get the row content back > correctly. -- This message was sent by Atlassian Jira (v8.3.4#803005)