gruuya commented on code in PR #7180:
URL: https://github.com/apache/arrow-datafusion/pull/7180#discussion_r1286391361
##########
datafusion/core/tests/memory_limit.rs:
##########
@@ -45,17 +47,38 @@ fn init() {
let _ = env_logger::try_init();
}
+#[rstest]
+#[case::cant_grow_reservation(vec!["Resources exhausted: Failed to allocate
additional", "ExternalSorter"], 100_000)]
+#[case::cant_spill_to_disk(vec!["Resources exhausted: Memory Exhausted while
Sorting (DiskManager is disabled)"], 200_000)]
+#[case::no_oom(vec![], 600_000)]
#[tokio::test]
-async fn oom_sort() {
+async fn sort(#[case] expected_errors: Vec<&str>, #[case] memory_limit: usize)
{
TestCase::new(
"select * from t order by host DESC",
- vec![
- "Resources exhausted: Memory Exhausted while Sorting (DiskManager
is disabled)",
- ],
- 200_000,
+ expected_errors,
+ memory_limit,
+ )
+ .run()
+ .await
+}
+
+// We expect to see lower memory thresholds in general when applying a `LIMIT`
clause due to eager sorting
+#[rstest]
+#[case::cant_grow_reservation(vec!["Resources exhausted: Failed to allocate
additional", "ExternalSorter"], 20_000)]
+#[case::cant_spill_to_disk(vec!["Memory Exhausted while Sorting (DiskManager
is disabled)"], 40_000)]
+//#[case::no_oom(vec![], 80_000)]
Review Comment:
I don't quite understand why atm, but this test case now fails. In fact, the
no-oom threshold is like 10 times that value which is even weirder. I did
double-check with bytehound and the memory profiles are still looking good
outside of this test. Will need to think about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
