erenavsarogullari commented on code in PR #20226:
URL: https://github.com/apache/datafusion/pull/20226#discussion_r2796833803
##########
datafusion/physical-plan/src/sorts/sort.rs:
##########
@@ -1739,6 +1740,48 @@ mod tests {
Ok(())
}
+ #[tokio::test]
+ async fn test_external_sorter_when_resource_exhausted() -> Result<()> {
+ let session_config = SessionConfig::new().with_batch_size(100);
+ // Memory allocation is being set lower than input data memory
footprint
+ // to cause ResourcesExhausted error
+ let runtime = RuntimeEnvBuilder::new()
+ .with_memory_limit(1024, 1.0)
+ .build_arc()?;
+ let task_ctx = Arc::new(
+ TaskContext::default()
+ .with_session_config(session_config)
+ .with_runtime(runtime),
+ );
+
+ // The input has 200 partitions, each partition has a batch containing
100 rows.
+ // Each row has a single Utf8 column, the Utf8 string values are
roughly 42 bytes.
+ // The total size of the input is roughly 8.4 KB.
Review Comment:
Updated legacy test case in terms of (200 partitions × 100 rows × ~42 bytes)
/ 1024 = 820 KB.
Thanks for the catch :smiley:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
