[
https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884770#comment-16884770
]
Jon Meredith commented on CASSANDRA-15170:
------------------------------------------
Thanks for the review - I forgot to remove a WIP commit on trunk that made the
numClusters unused.
Renaming the log appenders makes it easier to spot left-over logging threads
when instance class loaders do not shut down correctly.
I've seen similar exceptions too - I think they predate this fix. I reran the
trunk tests and am still getting issues with metaspace. I think the internode
Netty listeners aren't shutting down as expected - it all happens in a future
and perhaps is not being combined correctly with other futures to either
execute it or cause shutdown to wait until complete before closing the class
loader.
I'll investigate further, but I don't think the shutdown hooks come into play
when using the in-jvm dtests as the node is initialized without calling
org.apache.cassandra.service.CassandraDaemon#setup.
I'll ping this ticket when I get to the bottom of the listener threads not
shutting down.
> Reduce the time needed to release in-JVM dtest cluster resources after close
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-15170
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15170
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
> Reporter: Jon Meredith
> Assignee: Jon Meredith
> Priority: Normal
>
> There are a few issues that slow the in-JVM dtests from reclaiming metaspace
> once the cluster is closed.
> IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes
> this thread was still running 10s after the dtest cluster was closed.
> Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that
> the thread executing the class loader close executes sooner.
> If an OutboundTcpConnection is waiting to connect() and the endpoint is not
> answering, it has to wait for a timeout before it exits. Instead it should
> check the isShutdown flag and terminate early if shutdown has been requested.
> In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a
> try-with-resources construct and leaks a file handle for the directory. This
> doesn't matter for normal usage, it leaks a file handle for each dtest
> Instance created.
> On trunk, Netty global event executor threads are still running and delay GC
> for the instance class loader.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
