[
https://issues.apache.org/jira/browse/HDDS-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-4678:
-----------------------------
Description:
Original comment:
https://github.com/apache/ozone/pull/1680#pullrequestreview-549375494
h2. Summary
Currently we don't have explicit control over how much events Recon will
receive on each batch of events from OM. This can lead to Recon OOM like in
HDDS-4570 when the max heap is too small and/or OM is too busy and/or [sync
interval|https://github.com/apache/ozone/blob/efaa4fcb46d70cbce40451402bfb56dcc098802a/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java#L219-L230]
is too long.
A good start point for looking into this might be at
[{{OzoneManagerServiceProviderImpl#syncDataFromOM}}|https://github.com/apache/ozone/blob/efaa4fcb46d70cbce40451402bfb56dcc098802a/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java#L402].
was:
Original comment:
https://github.com/apache/ozone/pull/1680#pullrequestreview-549375494
h2. Summary
Currently we don't have explicit control over how much events Recon will
receive on each batch of events from OM. This can lead to Recon OOM like in
HDDS-4570 when the max heap is too small and/or OM is too busy and/or [sync
interval|https://github.com/apache/ozone/blob/efaa4fcb46d70cbce40451402bfb56dcc098802a/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java#L219-L230]
is too long.
A good start point for looking into this might be at
[{{OzoneManagerServiceProviderImpl#syncDataFromOM}}|https://github.com/apache/ozone/blob/efaa4fcb46d70cbce40451402bfb56dcc098802a/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java#L402].
> Add an option to limit the number of OM events Recon retrieves in each batch
> ----------------------------------------------------------------------------
>
> Key: HDDS-4678
> URL: https://issues.apache.org/jira/browse/HDDS-4678
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Recon
> Affects Versions: 1.0.0
> Reporter: Siyao Meng
> Priority: Major
>
> Original comment:
> https://github.com/apache/ozone/pull/1680#pullrequestreview-549375494
> h2. Summary
> Currently we don't have explicit control over how much events Recon will
> receive on each batch of events from OM. This can lead to Recon OOM like in
> HDDS-4570 when the max heap is too small and/or OM is too busy and/or [sync
> interval|https://github.com/apache/ozone/blob/efaa4fcb46d70cbce40451402bfb56dcc098802a/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java#L219-L230]
> is too long.
> A good start point for looking into this might be at
> [{{OzoneManagerServiceProviderImpl#syncDataFromOM}}|https://github.com/apache/ozone/blob/efaa4fcb46d70cbce40451402bfb56dcc098802a/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java#L402].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
