[
https://issues.apache.org/jira/browse/APEXMALHAR-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418038#comment-15418038
]
bright chen commented on APEXMALHAR-2182:
-----------------------------------------
The unit test will be failed due to OutOfMemoryError after couple of minutes in
my local machine environment. And the average performance will be decrease
rapid.
Here is the OutOfMemoryError stack trace:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.apex.malhar.lib.utils.serde.SerdeListSlice.serialize(SerdeListSlice.java:70)
at
org.apache.apex.malhar.lib.utils.serde.SerdeListSlice.serialize(SerdeListSlice.java:37)
at
org.apache.apex.malhar.lib.state.spillable.SpillableByteMapImpl.endWindow(SpillableByteMapImpl.java:219)
at
org.apache.apex.malhar.lib.state.spillable.SpillableArrayListImpl.endWindow(SpillableArrayListImpl.java:318)
at
org.apache.apex.malhar.lib.state.spillable.SpillableByteArrayListMultimapImpl.endWindow(SpillableByteArrayListMultimapImpl.java:274)
at
org.apache.apex.malhar.lib.state.spillable.SpillableDSBenchmarkTest.testSpillableMutimap(SpillableDSBenchmarkTest.java:114)
at
org.apache.apex.malhar.lib.state.spillable.SpillableDSBenchmarkTest.testSpillableMutimap(SpillableDSBenchmarkTest.java:78)
Here is the log, which shows performance decreasing:
2016-08-11 15:00:25,946 [main] WARN util.NativeCodeLoader <clinit> - Unable to
load native-hadoop library for your platform... using builtin-java classes
where applicable
2016-08-11 15:00:26,121 [main] INFO spillable.SpillableDSBenchmarkTest
testSpillableMutimap - Spent 67 mills for 0 operation. average: 0
2016-08-11 15:00:26,121 [main] INFO spillable.SpillableDSBenchmarkTest
checkEnvironment - freeMemory: 195M; allocatedMemory: 257M; maxMemory: 3817M
2016-08-11 15:01:26,055 [main] INFO spillable.SpillableDSBenchmarkTest
testSpillableMutimap - Spent 60001 mills for 88948004 operation. average: 1482
2016-08-11 15:01:26,055 [main] INFO spillable.SpillableDSBenchmarkTest
checkEnvironment - freeMemory: 842M; allocatedMemory: 2582M; maxMemory: 3817M
2016-08-11 15:02:26,055 [main] INFO spillable.SpillableDSBenchmarkTest
testSpillableMutimap - Spent 120001 mills for 166005088 operation. average: 1383
2016-08-11 15:02:26,055 [main] INFO spillable.SpillableDSBenchmarkTest
checkEnvironment - freeMemory: 417M; allocatedMemory: 3117M; maxMemory: 3817M
2016-08-11 15:03:26,493 [main] INFO spillable.SpillableDSBenchmarkTest
testSpillableMutimap - Spent 180439 mills for 224000000 operation. average: 1241
2016-08-11 15:03:26,494 [main] INFO spillable.SpillableDSBenchmarkTest
checkEnvironment - freeMemory: 614M; allocatedMemory: 3816M; maxMemory: 3817M
2016-08-11 15:04:26,572 [main] INFO spillable.SpillableDSBenchmarkTest
testSpillableMutimap - Spent 240518 mills for 239125628 operation. average: 994
2016-08-11 15:04:26,572 [main] INFO spillable.SpillableDSBenchmarkTest
checkEnvironment - freeMemory: 450M; allocatedMemory: 3816M; maxMemory: 3817M
2016-08-11 15:05:26,289 [main] INFO spillable.SpillableDSBenchmarkTest
testSpillableMutimap - Spent 300235 mills for 241102872 operation. average: 803
2016-08-11 15:05:26,289 [main] INFO spillable.SpillableDSBenchmarkTest
checkEnvironment - freeMemory: 423M; allocatedMemory: 3816M; maxMemory: 3817M
> benchmark for spillable data structure and storage
> ---------------------------------------------------
>
> Key: APEXMALHAR-2182
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2182
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: bright chen
> Assignee: bright chen
> Original Estimate: 96h
> Remaining Estimate: 96h
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
