devoopsman45 commented on PR #1881:
URL:
https://github.com/apache/datafusion-ballista/pull/1881#issuecomment-4754007506
> For some reason tpch.py says that no rows have been returned when executed
against the Docker-Compose setup:
>
> ```
> datafusion-ballista/benchmarks ❯ python3 tpch.py --path tpch-data --query
q1
> Registering table part at path tpch-data/part.parquet
> Registering table supplier at path tpch-data/supplier.parquet
> Registering table partsupp at path tpch-data/partsupp.parquet
> Registering table customer at path tpch-data/customer.parquet
> Registering table orders at path tpch-data/orders.parquet
> Registering table lineitem at path tpch-data/lineitem.parquet
> Registering table nation at path tpch-data/nation.parquet
> Registering table region at path tpch-data/region.parquet
> DataFrame has no rows
> Query q1 took 0.4933938980102539 second(s)
> ```
>
> Usually it prints something like:
>
> ```
> DataFrame()
>
+--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+
> | l_returnflag | l_linestatus | sum_qty | sum_base_price |
sum_disc_price | sum_charge | avg_qty | avg_price | avg_disc |
count_order |
>
+--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+
> | A | F | 37734107.00 | 56586554400.73 |
53758257134.8700 | 55909065222.827692 | 25.522005 | 38273.129734 | 0.049985 |
1478493 |
> | N | F | 991417.00 | 1487504710.38 |
1413082168.0541 | 1469649223.194375 | 25.516471 | 38284.467760 | 0.050093 |
38854 |
> | N | O | 74476040.00 | 111701729697.74 |
106118230307.6056 | 110367043872.497010 | 25.502226 | 38249.117988 | 0.049996 |
2920374 |
> | R | F | 37719753.00 | 56568041380.90 |
53741292684.6040 | 55889619119.831932 | 25.505793 | 38250.854626 | 0.050009 |
1478870 |
>
+--------------+--------------+-------------+-----------------+-------------------+---------------------+-----------+--------------+----------+-------------+
> Query q1 took 1.125335693359375 second(s)
> ```
>
> The TUI app lists the job and it is successful. I am not sure where is the
difference.
Thanks for testing this @martin-g . You are not seeing any results because
the data directory is not mounted to the executor container. Right now it is
commented out in _docker-compose.quick.yml_ :
ballista-executor:
# Uncomment to mount local data for queries:
# volumes:
# - /absolute/path/to/your/data:/data:ro
It should work if mounted. I tested it. Although, a minor improvement to the
comment is needed. The mount should be exact path. The scheduler would not be
able to find inside data. The below should work.
_/absolute/path/to/your/data:/absolute/path/to/your/data:ro_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
