[
https://issues.apache.org/jira/browse/IGNITE-24156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladislav Pyatkov updated IGNITE-24156:
---------------------------------------
Description:
h3. Motivation
I got an OOM exception in a node with 1Gb heap:
{noformat}
Thread '%node_3344%resource-vacuum-executor-0' with ID = 282
java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:48)
java.lang.Long.valueOf(Long.java:1206)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.markInitialVacuumObservationTimestamp(VolatileTxStateMetaStorage.java:251)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.lambda$vacuum$2(VolatileTxStateMetaStorage.java:167)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage$$Lambda+0x0000719658bffcf8.apply(Native
method)
java.util.concurrent.ConcurrentHashMap.computeIfPresent(ConcurrentHashMap.java:1828)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.lambda$vacuum$3(VolatileTxStateMetaStorage.java:160)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage$$Lambda+0x0000719658aa8ef0.accept(Native
method)
java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1603)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.vacuum(VolatileTxStateMetaStorage.java:159)
org.apache.ignite.internal.tx.impl.TxManagerImpl.vacuum(TxManagerImpl.java:860)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager.vacuumTxnResources(ResourceVacuumManager.java:184)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager$$Lambda+0x0000719658aa7ce8.run(Native
method)
org.apache.ignite.internal.util.IgniteUtils.inBusyLock(IgniteUtils.java:841)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager.runVacuumOperations(ResourceVacuumManager.java:162)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager$$Lambda+0x0000719658aa78a8.run(Native
method)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
java.lang.Thread.runWith(Thread.java:1596)
java.lang.Thread.run(Thread.java:1583)
{noformat}
because the vacuum thread copies all elements in tx state volatile storage and
stores them more than it is needed.
h3. Implementation notes
The possible way to initiate the initial timestamp on creation of the final
state:
{code:java}
public TxStateMeta(
...
this.initialVacuumObservationTimestamp = TxState.isFinalState(txState) ?
FastTimestamps.coarseCurrentTimeMillis() : null;
...
}
{code}
h3. Definition of done
Remove a TS state on a first vacuum path in the case when it stores longer than
needed.
was:
h3. Motivation
I got an OOM exception in a node with 1Gb heap:
{noformat}
Thread '%node_3344%resource-vacuum-executor-0' with ID = 282
java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:48)
java.lang.Long.valueOf(Long.java:1206)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.markInitialVacuumObservationTimestamp(VolatileTxStateMetaStorage.java:251)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.lambda$vacuum$2(VolatileTxStateMetaStorage.java:167)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage$$Lambda+0x0000719658bffcf8.apply(Native
method)
java.util.concurrent.ConcurrentHashMap.computeIfPresent(ConcurrentHashMap.java:1828)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.lambda$vacuum$3(VolatileTxStateMetaStorage.java:160)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage$$Lambda+0x0000719658aa8ef0.accept(Native
method)
java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1603)
org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.vacuum(VolatileTxStateMetaStorage.java:159)
org.apache.ignite.internal.tx.impl.TxManagerImpl.vacuum(TxManagerImpl.java:860)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager.vacuumTxnResources(ResourceVacuumManager.java:184)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager$$Lambda+0x0000719658aa7ce8.run(Native
method)
org.apache.ignite.internal.util.IgniteUtils.inBusyLock(IgniteUtils.java:841)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager.runVacuumOperations(ResourceVacuumManager.java:162)
org.apache.ignite.internal.tx.impl.ResourceVacuumManager$$Lambda+0x0000719658aa78a8.run(Native
method)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
java.lang.Thread.runWith(Thread.java:1596)
java.lang.Thread.run(Thread.java:1583)
{noformat}
because the vacuum thread copies all elements in tx state volatile storage and
stores them more than it is needed.
h3. Implementation notes
The possible way to initiate the initial timestamp on creation of the final
state:
{code:java}
public TxStateMeta(
...
this.initialVacuumObservationTimestamp = TxState.isFinalState(txState) ?
FastTimestamps.coarseCurrentTimeMillis() : null;
...
}
{code}
h3. Definition of done
Remove a TS state on a first vacuum path in the case when it stores longer than
needed.
> Tx states overconsume heap
> --------------------------
>
> Key: IGNITE-24156
> URL: https://issues.apache.org/jira/browse/IGNITE-24156
> Project: Ignite
> Issue Type: Improvement
> Reporter: Vladislav Pyatkov
> Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> I got an OOM exception in a node with 1Gb heap:
> {noformat}
> Thread '%node_3344%resource-vacuum-executor-0' with ID = 282
> java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:48)
> java.lang.Long.valueOf(Long.java:1206)
>
> org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.markInitialVacuumObservationTimestamp(VolatileTxStateMetaStorage.java:251)
>
> org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.lambda$vacuum$2(VolatileTxStateMetaStorage.java:167)
>
> org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage$$Lambda+0x0000719658bffcf8.apply(Native
> method)
>
> java.util.concurrent.ConcurrentHashMap.computeIfPresent(ConcurrentHashMap.java:1828)
>
> org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.lambda$vacuum$3(VolatileTxStateMetaStorage.java:160)
>
> org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage$$Lambda+0x0000719658aa8ef0.accept(Native
> method)
>
> java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1603)
>
> org.apache.ignite.internal.tx.impl.VolatileTxStateMetaStorage.vacuum(VolatileTxStateMetaStorage.java:159)
>
> org.apache.ignite.internal.tx.impl.TxManagerImpl.vacuum(TxManagerImpl.java:860)
>
> org.apache.ignite.internal.tx.impl.ResourceVacuumManager.vacuumTxnResources(ResourceVacuumManager.java:184)
>
> org.apache.ignite.internal.tx.impl.ResourceVacuumManager$$Lambda+0x0000719658aa7ce8.run(Native
> method)
>
> org.apache.ignite.internal.util.IgniteUtils.inBusyLock(IgniteUtils.java:841)
>
> org.apache.ignite.internal.tx.impl.ResourceVacuumManager.runVacuumOperations(ResourceVacuumManager.java:162)
>
> org.apache.ignite.internal.tx.impl.ResourceVacuumManager$$Lambda+0x0000719658aa78a8.run(Native
> method)
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> java.lang.Thread.runWith(Thread.java:1596)
> java.lang.Thread.run(Thread.java:1583)
> {noformat}
> because the vacuum thread copies all elements in tx state volatile storage
> and stores them more than it is needed.
> h3. Implementation notes
> The possible way to initiate the initial timestamp on creation of the final
> state:
> {code:java}
> public TxStateMeta(
> ...
> this.initialVacuumObservationTimestamp = TxState.isFinalState(txState) ?
> FastTimestamps.coarseCurrentTimeMillis() : null;
> ...
> }
> {code}
> h3. Definition of done
> Remove a TS state on a first vacuum path in the case when it stores longer
> than needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
