[
https://issues.apache.org/jira/browse/CASSANDRA-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264264#comment-16264264
]
Andrés de la Peña commented on CASSANDRA-12245:
-----------------------------------------------
bq. I had a final look at the patch and tests, and it mostly looks good, but
during the last review I found one edge case still not addressed: if the view
build is stopped via {{nodetool stop VIEW_BUILD}}, the view build will be
[considered
successful|https://github.com/adelapena/cassandra/blob/e1ace2f47be71d48ab1987d0e2c7a07cc9486e97/src/java/org/apache/cassandra/db/view/ViewBuilder.java#L174],
so we should probably throw a {{CompactionInterruptedException}} when the view
build is stopped and not resubmit a new view build task in this case.
Good catch! Done
[here|https://github.com/adelapena/cassandra/commit/640d5b70232deab5b6b54b6158b28e86e53f7612].
I have added an overloaded version of
[{{ViewBuilderTask.stop}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilderTask.java#L204-L208]
to throw the {{CompactionInterruptedException}} only if the stop call comes
from a different place than {{ViewBuilder}}. That is, the exception is not
thrown in the case of a schema change (such as a drop), when the current build
should be stopped without errors and maybe restarted.
I've also [extended
{{CompactionInterruptedException}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilderTask.java#L215-L249]
with {{equals}} and {{hashCode}} methods that consider equals all the
compaction interrupted exceptions produced by the same view. Otherwise, the
multiple calls to
[{{Futures#allAsList}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilder.java#L175]'s
[{{FutureCallback#onFailure}}|https://github.com/adelapena/cassandra/blob/640d5b70232deab5b6b54b6158b28e86e53f7612/src/java/org/apache/cassandra/db/view/ViewBuilder.java#L186-L199]
with different exceptions make Guava's to log messages of the form {{SEVERE:
You called allAsList, and more than one input failed, and the failures were
different exceptions}}, which can be quite spamming.
bq. Would you mind adding a dtest for this as well to ensure the view will not
be marked as built after the view build is stopped?
I have added a couple of dtests
[here|https://github.com/adelapena/cassandra-dtest/commit/3af871a9daa8cc5aa9dc098f6126ca0a6fe4a015].
[{{test_resume_stopped_build}}|https://github.com/adelapena/cassandra-dtest/blob/3af871a9daa8cc5aa9dc098f6126ca0a6fe4a015/materialized_views_test.py#L1132-L1179]
uses `nodetool stop VIEW_BUILD` to interrupt the running task of an ongoing
view build and verifies that the unmarked build is resumed after restarting the
nodes.
[{{test_drop_with_stopped_build}}|https://github.com/adelapena/cassandra-dtest/blob/3af871a9daa8cc5aa9dc098f6126ca0a6fe4a015/materialized_views_test.py#L1074-L1130]
verifies that a view with interrupted taks can still be dropped, which is
something that has been problematic while writting the patch.
CI for utests is clean, the updated dtests are still running:
||[utest|http://jenkins-cassandra.datastax.lan/view/Dev/view/adelapena/job/adelapena-12245-trunk-testall/]||[dtest|http://jenkins-cassandra.datastax.lan/view/Dev/view/adelapena/job/adelapena-12245-trunk-dtest/]||
> initial view build can be parallel
> ----------------------------------
>
> Key: CASSANDRA-12245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12245
> Project: Cassandra
> Issue Type: Improvement
> Components: Materialized Views
> Reporter: Tom van der Woerdt
> Assignee: Andrés de la Peña
> Fix For: 4.x
>
>
> On a node with lots of data (~3TB) building a materialized view takes several
> weeks, which is not ideal. It's doing this in a single thread.
> There are several potential ways this can be optimized :
> * do vnodes in parallel, instead of going through the entire range in one
> thread
> * just iterate through sstables, not worrying about duplicates, and include
> the timestamp of the original write in the MV mutation. since this doesn't
> exclude duplicates it does increase the amount of work and could temporarily
> surface ghost rows (yikes) but I guess that's why they call it eventual
> consistency. doing it this way can avoid holding references to all tables on
> disk, allows parallelization, and removes the need to check other sstables
> for existing data. this is essentially the 'do a full repair' path
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
