MohamedAbdeen21 commented on PR #11453:
URL: https://github.com/apache/datafusion/pull/11453#issuecomment-2227415236
I've been running some experiments for the past few hours, here are some
observations:
- Main branch tests pass with default tokio stack size (2 MB for all
non-windows systems), but overflows when slightly reducing the stack size to
1.9 MB.
- Also, slightly pumping the stack size to 2.1 MB in this branch causes the
tests to pass.
It doesn't look like it's a single struct/enum filling the stack, but
multiple small structs/enums being copied, which kinda explains why the release
build passes, but not the debug build.
Maybe we've been slightly increasing the stack usage with every change and
this PR caused it to finally overflow.
I think we should temporarily increase the stack size to something > 2 MB
(I'm thinking 2.5 MB) for now and try to find the problematic test in a
follow-up, just to keep things moving.
To replicate, change
`datafusion/sqllogictest/bin/sqllogictests.rs:main.rs:54` from
```rs
#[tokio::main]
#[cfg(not(target_family = "windows"))]
pub async fn main() -> Result<()> {
run_tests().await
}
```
to
```rs
#[cfg(not(target_family = "windows"))]
fn main() -> Result<()> {
tokio::runtime::Builder::new_multi_thread()
.thread_stack_size(2 * 1024 * 1024) // 2 MB
// .thread_stack_size(2 * 1024 * 1024 - 128 * 1024) // 1.9 MB
// .thread_stack_size(2 * 1024 * 1024 + 128 * 1024) // 2.1 MB
.enable_all()
.build()
.unwrap()
.block_on(run_tests())
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
