datbth commented on issue #40881:
URL: https://github.com/apache/arrow/issues/40881#issuecomment-2038828252
### Question
Hi, may I ask if there is any planning/estimation for this yet? What does
the effort look like? Would you need any help?
<details>
<summary><h3>Background</h3></summary>
I'm facing this while trying to fetch data from an Arrow Flight server into
Ruby.
The testing script looks like this:
```ruby
client = ArrowFlight::Client.new(location)
reader = client.do_get(payload) # server arrow has 9 cols, 512205 rows,
50000 records per chunk/batch
GC.disable
data = []
t = Benchmark.realtime do
while (chunk = reader.read_next)
chunk_records = chunk.data.raw_records
data += chunk_records
end
end
```
Result of `t` (tested on Ruby 2.7.4, 3.2.3; 4 runs per case)
* With `GC.disable`: 0.31s
* Without `GC.disable`: 0.66s
* Running in 4 Threads (each Thread runs the whole codes above):
* With `GC.disable`: ranging between 0.61s - 1.1s each
* WIthout `GC.disable`: ranging between 2.1s - 3.5s each
Sorry if I'm bloating this Issue. But I want to put a bit more clarity on
the importance of this enhancement.
I'm considering this a blocker for putting my Arrow integration into
production.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
