[
https://issues.apache.org/jira/browse/SAMZA-439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Riccomini updated SAMZA-439:
----------------------------------
Fix Version/s: 0.9.0
> StackOverflowError in CachedStore
> ---------------------------------
>
> Key: SAMZA-439
> URL: https://issues.apache.org/jira/browse/SAMZA-439
> Project: Samza
> Issue Type: Bug
> Components: kv
> Affects Versions: 0.7.0
> Reporter: Dmitry Bugaychenko
> Fix For: 0.9.0
>
>
> When configured to large cache (100000 records) got stack overflow error on
> scala doublelinkedlist. It looks like reversed method is implemented very
> inefficiently (using recursion). The problem can be fixed by increasing stack
> size, but it would be nice to avoid this inefficiency.
> This could be done by replacing:
> {code}
> val batch = new java.util.ArrayList[Entry[K, V]](this.dirtyCount)
> for (k <- this.dirty.reverse) {
> val entry = this.cache.get(k)
> entry.dirty = null // not dirty any more
> batch.add(new Entry(k, entry.value))
> }
> store.putAll(batch)
> {code}
> with:
> {code}
> val batch = new Array[Entry[K, V]](this.dirtyCount)
> var pos : Int = this.dirtyCount - 1;
> for (k <- this.dirty) {
> val entry = this.cache.get(k)
> entry.dirty = null // not dirty any more
> batch(pos) = new Entry(k, entry.value)
> pos -= 1
> }
> store.putAll(util.Arrays.asList(batch : _*))
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
