[
https://issues.apache.org/jira/browse/SAMZA-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Kumarasubramanian updated SAMZA-2481:
---------------------------------------------
Fix Version/s: 1.5
> Improve memory usage in samza-core unit tests
> ---------------------------------------------
>
> Key: SAMZA-2481
> URL: https://issues.apache.org/jira/browse/SAMZA-2481
> Project: Samza
> Issue Type: Improvement
> Reporter: Cameron Lee
> Assignee: Cameron Lee
> Priority: Major
> Fix For: 1.5
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Gradle 5 reduced the default memory allocation for test executors. Before
> Gradle 5, our unit tests were given plenty of memory, so they ran fine. When
> upgrading to Gradle 5, we noticed some issues with running tests. Setting the
> memory (to 3GB heap) explicitly seems to have resolved the issues, but it
> seems odd that unit tests need so much. It would be good to investigate this
> possible inefficiency. It might also help build times, since heavy memory
> usage could mean a unnecessarily heavy unit test.
> One potential example is TestZkMetadataStore. It tests large zookeeper values
> (10MB), and that seems to cause some issues. If heap usage is set to 1GB or
> 1.5GB, then that test takes a long time to execute (seems like it might be
> running out of memory). This can be reproduced by changing the heap usage for
> "test" for the samza-core module in build.gradle, and then running "./gradlew
> :samza-core:test". By setting the zookeeper value to 1MB (and using 1GB of
> heap), the tests will execute fine. It is odd that the test works on the
> scale of MB but seems to have memory requirements on the scale of GB.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
