haohuaijin commented on issue #6363: URL: https://github.com/apache/arrow-rs/issues/6363#issuecomment-2360048743
Sorry for the delay, @alamb. Recently, I have had some travels. I set the paraments of `generate_record_batch` to `(1000, 100)` and use `cargo build --release` to compile https://github.com/haohuaijin/ipc-bug and run `heaptrack ./target/release/ipc-bug` to get the data. ``` heaptrack output will be written to "/home/hhj/ipc-bug/heaptrack.ipc-bug.1013097.zst" /usr/lib/heaptrack/libheaptrack_preload.so starting application, this might take some time... num_rows: 100, original batch size: 2616000 ipc buffer size: 1120446 num rows: 100, after decode batch size: 1025627000, batches len: 1 heaptrack stats: allocations: 137148 leaked allocations: 0 temporary allocations: 3034 Heaptrack finished! Now run the following to investigate the data: heaptrack --analyze "/home/hhj/ipc-bug/heaptrack.ipc-bug.1013097.zst" ``` Then, I used heaptrack to visualize the data and get the following pictures. <img width="1445" alt="image" src="https://github.com/user-attachments/assets/331b7b21-3b1b-4574-b326-6333384245d2";> <img width="1446" alt="image" src="https://github.com/user-attachments/assets/abb36322-6782-4faa-9a2e-47b591c27e92";> <img width="1447" alt="image" src="https://github.com/user-attachments/assets/91dbffc2-a25f-4f04-9519-a6d13aa55883";> <img width="1446" alt="image" src="https://github.com/user-attachments/assets/90b8758d-053c-4b1f-b838-17e17f1bf097";> the file in https://github.com/haohuaijin/ipc-bug/blob/main/heaptrack.ipc-bug.1013097.zst. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
