[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17683518#comment-17683518
]
David Capwell edited comment on CASSANDRA-18218 at 2/2/23 6:30 PM:
-------------------------------------------------------------------
patch LGTM, can we attempt to run the simulator in a repeat job so we see if
this doesn't come back?
was (Author: dcapwell):
patch LGTM, can we get circle ci runs? Can we attempt to run the simulator in
a repeat job so we see if this doesn't come back?
> OOM while running ShortAccordSimulationTest
> -------------------------------------------
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
> Reporter: Caleb Rackliffe
> Assignee: Caleb Rackliffe
> Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:3181)
> at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
> at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
> at accord.local.Command.maybeExecute(Command.java:600)
> at accord.local.Command.onChange(Command.java:546)
> at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
> at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
> at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
> at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
> at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
> at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
> at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
> at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
> at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
> at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k6680000gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=10000,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=10000, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
> -XX:Tier4CompileThreshold=1000, -XX:ReservedCodeCacheSize=256M, -Xmx8G,
> -Xmx1024m]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
