cfrancois7 commented on issue #953:
URL: https://github.com/apache/iceberg-python/issues/953#issuecomment-2252468758
I get the right code for pyarrow to works.
With the same data as previously shared:
```
import pyarrow as pa
import pyarrow.compute as pc
table_pa = table.scan().to_arrow() # to parse pyiceberg table into pyarrow
one
# Extract the field using struct_field
employment_status = pc.field('employment', 'status')
# Query the table
status_employed = pc.equal(employment_status, pa.scalar('Employed'))
# Filter the table based on the employment status
filtered_table = table_pa.filter(status_employed)
# Convert the filtered table back to a list of dictionaries
filtered_data = filtered_table.to_pylist()
pd.DataFrame(filtered_data) # success
```
For three levels:
```
import pyarrow as pa
import pyarrow.compute as pc
employment_status = pc.field('employment', 'company', 'name')
status_employed = pc.equal(employment_status, pa.scalar('Freelance'))
filtered_table = table_pa.filter(status_employed)
filtered_data = filtered_table.to_pylist()
pd.DataFrame(filtered_data)
```
The right pc.field instanciation is ` pc.field('employment', 'status')` and
not ` pc.field('employment.status')` for pyarrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
