[
https://issues.apache.org/jira/browse/CASSANDRA-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284014#comment-17284014
]
Ekaterina Dimitrova commented on CASSANDRA-16441:
-------------------------------------------------
I wasn't able to reproduce it on my own in any way.
But from the CircleCI log it seems as a legit issue. A node is not able to be
brought up because of the following issue:
{code:java}
INFO [main] 2021-02-11 17:25:08,125 StorageService.java:830 - Populating token
metadata from system tables
INFO [main] 2021-02-11 17:25:08,132 StorageService.java:824 - Token metadata:
ERROR [main] 2021-02-11 17:25:08,153 SchemaKeyspace.java:942 - No partition
columns found for table system_auth.network_permissions in
system_schema.columns. This may be due to corruption or concurrent dropping and
altering of a table. If this table is supposed to be dropped, restart cassandra
with -Dcassandra.ignore_corrupted_schema_tables=true and run the following
query to cleanup: "DELETE FROM system_schema.tables WHERE keyspace_name =
'system_auth' AND table_name = 'network_permissions'; DELETE FROM
system_schema.columns WHERE keyspace_name = 'system_auth' AND table_name =
'network_permissions';" If the table is not supposed to be dropped, restore
system_schema.columns sstables from backups.
ERROR [main] 2021-02-11 17:25:08,155 CassandraDaemon.java:271 - Error while
loading schema:
org.apache.cassandra.schema.SchemaKeyspace$MissingColumns: Columns not found in
schema table for system_auth.network_permissions
at
org.apache.cassandra.schema.SchemaKeyspace.fetchColumns(SchemaKeyspace.java:998)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:962)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:921)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:880)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:871)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:859)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:100)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:89)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:267)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:676)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:800)
ERROR [main] 2021-02-11 17:25:08,155 CassandraDaemon.java:822 - Exception
encountered during startup
org.apache.cassandra.schema.SchemaKeyspace$MissingColumns: Columns not found in
schema table for system_auth.network_permissions
at
org.apache.cassandra.schema.SchemaKeyspace.fetchColumns(SchemaKeyspace.java:998)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:962)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:921)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:880)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:871)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:859)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:100)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:89)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:267)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:676)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:800){code}
I just started a multiplexer job in a Jenkins environment.
Considering it is flaky, it seems like some race condition maybe? I didn't have
a lot of time to dig into the log/code for more details today but felt like
leaving here what I have as I will be out almost half next week so in case
someone is interested and has time to look at it too in the meanwhile.
> Snapshot_test.py DTests failing
> -------------------------------
>
> Key: CASSANDRA-16441
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16441
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
> Reporter: Ekaterina Dimitrova
> Priority: Normal
> Fix For: 4.0-beta5, 4.0-beta
>
>
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/623/workflows/1c89bd58-bcd1-45dc-b923-3aba245a78a6/jobs/3451]
> These failures are also observed for other patches in Jenkins dev (weird but
> they are not seen failing in the latest builds and they are failing with
> completely unrelated different patches...)
> [https://jenkins-cm4.apache.org/job/Cassandra-devbranch/375/#showFailuresLink]
> [https://jenkins-cm4.apache.org/job/Cassandra-devbranch/368/]
> Jenkins dev history for one of the tests:
> https://jenkins-cm4.apache.org/job/Cassandra-devbranch/375/testReport/junit/dtest-novnode.snapshot_test/TestArchiveCommitlog/test_archive_commitlog_with_active_commitlog/history/
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
