[
https://issues.apache.org/jira/browse/BEAM-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17202862#comment-17202862
]
Beam JIRA Bot commented on BEAM-10787:
--------------------------------------
This issue was marked "stale-assigned" and has not received a public comment in
7 days. It is now automatically unassigned. If you are still working on it, you
can assign it to yourself again. Please also give an update about the status of
the work.
> External sorter 2048MB memory limit should be Hadoop only
> ---------------------------------------------------------
>
> Key: BEAM-10787
> URL: https://issues.apache.org/jira/browse/BEAM-10787
> Project: Beam
> Issue Type: Bug
> Components: extensions-java-sorter
> Affects Versions: 2.23.0
> Reporter: Neville Li
> Priority: P4
>
> The 2048MB limit only applies when using Hadoop sequence file for external
> sort.
> It doesn't apply when sorting is entirely in memory since we're using a
> {{List<KV<byte[], byte[]>>}} to buffer data and {{long}} to keep track of
> total bytes. Same with native external sorter.
> https://github.com/apache/beam/pull/12659
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
