[
https://issues.apache.org/jira/browse/FLINK-17192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-17192:
-----------------------------------
Labels: auto-deprioritized-critical stale-major (was:
auto-deprioritized-critical)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Major but is unassigned and neither itself nor its Sub-Tasks have been updated
for 30 days. I have gone ahead and added a "stale-major" to the issue". If this
ticket is a Major, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> java.lang.RuntimeException: Error obtaining the sorted input: Thread
> 'SortMerger Reading Thread' terminated due to an exception: The record
> exceeds the maximum size of a sort buffer (current maximum: 72351744 bytes)
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-17192
> URL: https://issues.apache.org/jira/browse/FLINK-17192
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
> Affects Versions: 1.8.2, 1.9.0, 1.10.0, 1.11.0, 1.12.0
> Environment: flink 1.8.2
> Reporter: Nikola
> Priority: Major
> Labels: auto-deprioritized-critical, stale-major
>
> We are trying to sort our data and we get to the following exception:
> {code:java}
> java.lang.RuntimeException: Error obtaining the sorted input: Thread
> 'SortMerger Reading Thread' terminated due to an exception: The record
> exceeds the maximum size of a sort buffer (current maximum: 72351744 bytes).
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650)
>
> at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108)
> at org.apache.flink.runtime.operators.NoOpDriver.run(NoOpDriver.java:82)
> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
> at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated
> due to an exception: The record exceeds the maximum size of a sort buffer
> (current maximum: 72351744 bytes).
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831)
> Caused by: java.io.IOException: The record exceeds the maximum size of a sort
> buffer (current maximum: 72351744 bytes).
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986)
>
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827){code}
>
> The sort buffer says to be "72351744 bytes" which is about 70MB. Our TM has
> 30GB memory set for flink.
> I tried to find the same issue, but nothing really gave me an answer. I tried
> finding the same issue on JIRA, but I couldn't.
> [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Any-way-to-increase-sort-buffer-size-td10152.html]
> This one is from 2016 so I am not sure if I can trust it anymore since Flink
> has evolved since
>
> Is it possible to increase this sort buffer?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
