jiayuasu commented on PR #124: URL: https://github.com/apache/sedona-spatialbench/pull/124#issuecomment-5011568406
## Benchmark CI: `main` vs this PR (per engine) Comparing the scheduled baseline on `main` (run `29395354874`) with this PR at `82c44a7` (run `29635115880`). Cells are query time; `TIMEOUT` = hit the 600s limit, `ERROR` = query error, `—` = no result recorded (runner killed / OOM / not attempted). Only the affected queries (Q1, Q5, Q7, Q8, Q9, Q10, Q12) are shown. ### 🦆 DuckDB | Query | SF1 main | SF1 PR | SF10 main | SF10 PR | |:--|--:|--:|--:|--:| | Q1 | 0.21s | 0.16s | 2.21s | 2.14s | | Q5 | 1.84s | 1.58s | 367.72s | **TIMEOUT** | | **Q7** | 5.85s | **0.54s** | **TIMEOUT** | **5.48s** | | Q8 | 0.87s | 0.90s | 10.44s | 6.50s | | Q9 | 0.04s | 0.03s | 0.22s | 0.16s | | Q10 | 240.41s | 243.84s | TIMEOUT | TIMEOUT | | Q12 | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT | ### 🐼 GeoPandas | Query | SF1 main | SF1 PR | SF10 main | SF10 PR | |:--|--:|--:|--:|--:| | Q1 | 14.75s | 11.18s | — | TIMEOUT | | Q5 | 55.00s | 60.40s | — | TIMEOUT | | Q7 | 202.22s | 203.78s | — | — | | Q8 | 15.80s | 16.35s | — | — | | Q9 | 0.09s | 0.11s | 0.30s | 0.37s | | Q10 | 41.98s | 34.52s | — | — | | Q12 | TIMEOUT | TIMEOUT | TIMEOUT | — | ### 🌴 PyCanopy | Query | SF1 main | SF1 PR | SF10 main | SF10 PR | |:--|--:|--:|--:|--:| | Q1 | 0.39s | 0.38s | 4.69s | 5.14s | | Q5 | 1.26s | 1.09s | 23.21s | 24.94s | | Q7 | 1.65s | 1.50s | 26.48s | 27.13s | | Q8 | 0.66s | 0.74s | 9.01s | 8.61s | | Q9 | 0.02s | 0.03s | 0.05s | 0.05s | | Q10 | 6.45s | 6.03s | 42.39s | 44.29s | | Q12 | 28.07s | 21.70s | — | — | ### 🌵 SedonaDB | Query | SF1 main | SF1 PR | SF10 main | SF10 PR | |:--|--:|--:|--:|--:| | Q1 | 0.32s | 0.37s | 2.80s | 2.62s | | Q5 | 7.46s | 6.51s | ERROR | ERROR | | **Q7** | 4.27s | **1.60s** | 46.15s | **9.34s** | | Q8 | 0.85s | 0.88s | 9.87s | 9.32s | | Q9 | 0.04s | 0.04s | 0.11s | 0.10s | | Q10 | 23.51s | 23.43s | 119.94s | 120.95s | | **Q12** | **ERROR** | **24.05s** | **ERROR** | **277.12s** | ### 🐻❄️ Spatial Polars | Query | SF1 main | SF1 PR | SF10 main | SF10 PR | |:--|--:|--:|--:|--:| | Q1 | 3.21s | 2.70s | 33.67s | 37.33s | | Q5 | 10.07s | 12.80s | 129.00s | 125.75s | | Q7 | 8.26s | 7.31s | 73.25s | 78.99s | | Q8 | 6.36s | 6.40s | — | — | | Q9 | 0.07s | 0.07s | 0.30s | 0.31s | | Q10 | 17.67s | 19.00s | — | — | | Q12 | 68.58s | 62.79s | — | — | ### Takeaways - **Q7 is the clear win**: DuckDB SF1 5.85s → 0.54s and SF10 `TIMEOUT` → 5.48s; SedonaDB SF10 46s → 9.3s. Bounding stops it materializing 6M/60M rows. - **SedonaDB Q12: `ERROR` → completes** (24s at SF1, 277s at SF10) after the redesign. - **Everything else is within run-to-run noise** — bounding preserves the compute, so timings are stable rather than faster/slower. - **Row counts** (engine-independent, not shown): every affected query now returns ≤100 rows vs up to 60M on `main` — the objective of this PR. - DuckDB Q5/Q10 and SedonaDB Q5 at SF10 are runner resource limits (timeout / `No space left on device`), not this change; DuckDB Q10 times out in both runs (join-bound — `LIMIT` cannot reduce it, by design). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
