martin-g opened a new issue, #2367:
URL: https://github.com/apache/datafusion-ballista/issues/2367
**Describe the bug**
tpch.py prints an error after registering the tables for the Parquet files:
```
Registering table part at path tpch-data/part.parquet
Registering table supplier at path tpch-data/supplier.parquet
Registering table partsupp at path tpch-data/partsupp.parquet
Registering table customer at path tpch-data/customer.parquet
Registering table orders at path tpch-data/orders.parquet
Registering table lineitem at path tpch-data/lineitem.parquet
Registering table nation at path tpch-data/nation.parquet
Registering table region at path tpch-data/region.parquet
grpc client failed to fetch partition: PartitionId { job_id:
"0381QN35XP000", stage_id: 3, partition_id: 0 } , message: "Status { code:
Internal, message: \"Ballista Error: General(\\\"Failed to open partition file
at
\\\\\\\"/var/folders/0g/qbtl2t512f5bh4sp79cvfy_00000gn/T/.tmpIzsco9/0381QN35XP000/3/0/data.arrow\\\\\\\":
Os { code: 2, kind: NotFound, message: \\\\\\\"No such file or
directory\\\\\\\" }\\\")\", metadata: MetadataMap { headers: {\"content-type\":
\"application/grpc\", \"date\": \"Tue, 25 Aug 2026 06:43:25 GMT\"} }, source:
None }"
Traceback (most recent call last):
File
"/Users/mgrigorov/git/apache/datafusion/datafusion-ballista/benchmarks/tpch.py",
line 55, in <module>
df.show()
File
"/Users/mgrigorov/git/apache/datafusion/datafusion-ballista/python/python/ballista/extension.py",
line 58, in method_wrapper
return getattr(df, func)(*argz, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception: DataFusion error: Arrow error: External error: External error:
Shuffle fetch partition error from Executor
9bbf0041-d0d6-43bc-a452-d35d92f0e627, map_stage 3, map_partition 0, error desc:
Status { code: Internal, message: "Ballista Error: General(\"Failed to open
partition file at
\\\"/var/folders/0g/qbtl2t512f5bh4sp79cvfy_00000gn/T/.tmpIzsco9/0381QN35XP000/3/0/data.arrow\\\":
Os { code: 2, kind: NotFound, message: \\\"No such file or directory\\\"
}\")", metadata: MetadataMap { headers: {"content-type": "application/grpc",
"date": "Tue, 25 Aug 2026 06:43:25 GMT"} }, source: None }
```
@andygrove
[guesses](https://discord.com/channels/885562378132000778/1179822705525141605/1539686066909352067)
that it might be related to the new DataFusion + old Ballista Python bindings
state in current HEAD (50d732299c9129947aa41949a709c5aa1697cf3e)
**To Reproduce**
1. Start a Scheduler
2. Start at least one Executor
3. `cd datafusion-ballista/benchmarks`
4. `tpchgen-cli -s 1 --format parquet --output-dir tpch-data`
5. `python3 tpch.py --path tpch-data --query q1`
**Expected behavior**
Only the `Registering table ...` messages should be printed on the stdout.
**Additional context**
Ballista v54.0.0
DataFusion v55.0.0
Git HEAD: 50d732299c9129947aa41949a709c5aa1697cf3e perf(core): reuse shuffle
partition-index buffers across batches
The job is properly executed and `Completed` according to the TUI app
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
