collimarco opened a new issue, #36291:
URL: https://github.com/apache/arrow/issues/36291
### Describe the enhancement requested
Currently you can filter the rows in a Parquet file using a code like this:
```ruby
Arrow::FileInputStream.open(file) do |input|
reader = Parquet::ArrowFileReader.new(input)
reader.n_row_groups.times do |i|
table = reader.read_row_group(i)
result = table.slice { |slicer| (slicer['status'] == 200) &
(slicer['message'].match_substring? 'test') }
puts result if result.n_rows > 0
end
end
```
The problem is that the filter (`slice` condition) is statically defined /
it's defined at "compile" time.
If you receive a user input you cannot build a query.
For example, with SQL you would build a query string dynamically based on
the inputs from user.
The only solutions that I see, to allow the building of queries at runtime,
is to make `slice` accept:
1. a string of conditions (e.g. `status = 200 AND message MATCHES 'test'`)
2. OR, an array of conditions (e.g. `[[:equal, :status, 200], [:matches,
:message, 'test']]`)
Building a string or an array of conditions dynamically would be easy and
that would allow to build query dynamically at runtime. This is not possible
right now.
### Component(s)
Ruby
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
