[ https://issues.apache.org/jira/browse/CASSANDRA-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264264#comment-16264264 ]
Andrés de la Peña commented on CASSANDRA-12245: ----------------------------------------------- bq. I had a final look at the patch and tests, and it mostly looks good, but during the last review I found one edge case still not addressed: if the view build is stopped via {{nodetool stop VIEW_BUILD}}, the view build will be [considered successful|https://github.com/adelapena/cassandra/blob/e1ace2f47be71d48ab1987d0e2c7a07cc9486e97/src/java/org/apache/cassandra/db/view/ViewBuilder.java#L174], so we should probably throw a {{CompactionInterruptedException}} when the view build is stopped and not resubmit a new view build task in this case. Good catch! Done [here|https://github.com/adelapena/cassandra/commit/640d5b70232deab5b6b54b6158b28e86e53f7612]. I have added an overloaded version of [{{ViewBuilderTask.stop}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilderTask.java#L204-L208] to throw the {{CompactionInterruptedException}} only if the stop call comes from a different place than {{ViewBuilder}}. That is, the exception is not thrown in the case of a schema change (such as a drop), when the current build should be stopped without errors and maybe restarted. I've also [extended {{CompactionInterruptedException}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilderTask.java#L215-L249] with {{equals}} and {{hashCode}} methods that consider equals all the compaction interrupted exceptions produced by the same view. Otherwise, the multiple calls to [{{Futures#allAsList}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilder.java#L175]'s [{{FutureCallback#onFailure}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilder.java#L186-L199] with different exceptions make Guava's to log messages of the form {{SEVERE: You called allAsList, and more than one input failed, and the failures were different exceptions}}, which can be quite spamming. bq. Would you mind adding a dtest for this as well to ensure the view will not be marked as built after the view build is stopped? I have added a couple of dtests [here|https://github.com/adelapena/cassandra-dtest/commit/3af871a9daa8cc5aa9dc098f6126ca0a6fe4a015]. [{{test_resume_stopped_build}}|https://github.com/adelapena/cassandra-dtest/blob/3af871a9daa8cc5aa9dc098f6126ca0a6fe4a015/materialized_views_test.py#L1132-L1179] uses `nodetool stop VIEW_BUILD` to interrupt the running task of an ongoing view build and verifies that the unmarked build is resumed after restarting the nodes. [{{test_drop_with_stopped_build}}|https://github.com/adelapena/cassandra-dtest/blob/3af871a9daa8cc5aa9dc098f6126ca0a6fe4a015/materialized_views_test.py#L1074-L1130] verifies that a view with interrupted taks can still be dropped, which is something that has been problematic while writting the patch. CI for utests is clean, the updated dtests are still running: ||[utest|http://jenkins-cassandra.datastax.lan/view/Dev/view/adelapena/job/adelapena-12245-trunk-testall/]||[dtest|http://jenkins-cassandra.datastax.lan/view/Dev/view/adelapena/job/adelapena-12245-trunk-dtest/]|| > initial view build can be parallel > ---------------------------------- > > Key: CASSANDRA-12245 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12245 > Project: Cassandra > Issue Type: Improvement > Components: Materialized Views > Reporter: Tom van der Woerdt > Assignee: Andrés de la Peña > Fix For: 4.x > > > On a node with lots of data (~3TB) building a materialized view takes several > weeks, which is not ideal. It's doing this in a single thread. > There are several potential ways this can be optimized : > * do vnodes in parallel, instead of going through the entire range in one > thread > * just iterate through sstables, not worrying about duplicates, and include > the timestamp of the original write in the MV mutation. since this doesn't > exclude duplicates it does increase the amount of work and could temporarily > surface ghost rows (yikes) but I guess that's why they call it eventual > consistency. doing it this way can avoid holding references to all tables on > disk, allows parallelization, and removes the need to check other sstables > for existing data. this is essentially the 'do a full repair' path -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org