brayanjuls commented on issue #9814:
URL:
https://github.com/apache/arrow-datafusion/issues/9814#issuecomment-2058185423
@alamb I did three test, the results:
1. The program I did for testing was running forever for this query in my
computer, I did not experience OOM contrary the memory consumption was really
small 15mb and the cpu consumption was high. I will attach a flame graph for
this query.
Code:
```rust
fn main(){
bomb_table();
}
#[tokio::main]
async fn bomb_table(){
let ctx = SessionContext::new();
ctx.register_parquet("bomb", "path/to/fortytwodotparquet/42.parquet",
ParquetReadOptions::default().skip_metadata(false)).await.unwrap();
let df = ctx.sql("select sum(b) from bomb").await.unwrap();
df.show().await.unwrap();
}
```
Flame Graph of sum query:
2. I was able to successfully cancel the query and the resources in use
where clean.
3. I run the following query `SELECT * FROM bomb` to bring everything into
memory and the program was killed by OS when it reached the limits of my
computer memory.
Flame Graph of * query:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
