adriangb opened a new issue, #27485:
URL: https://github.com/apache/datafusion/issues/27485

   ### Is your feature request related to a problem or challenge?
   
   The SQL unparser (`datafusion/sql/src/unparser`) is tested by hand-written 
cases in `unparser/expr.rs` and `datafusion/sql/tests/cases/plan_to_sql.rs`. 
Those cases cover the shapes their authors thought of. Gaps show up later, one 
at a time, when a downstream user unparses a real plan and the SQL does not 
plan again. For example, #27477 reports that a non-finite float literal 
unparses as a bare `inf` or `NaN` token.
   
   The sqllogictest suite already has a few thousand queries with checked 
results. Today it runs them through the unparser only by accident. Nothing in 
`datafusion/sqllogictest` calls `Unparser` or `plan_to_sql`.
   
   ### Describe the solution you'd like
   
   Add a round-trip engine, like the existing 
`datafusion_substrait_roundtrip_engine`, behind a new runner flag (for example 
`--unparser-round-trip`). For each query, the engine:
   
   1. Plans the SQL (optimized or unoptimized, like `--substrait-optimize`).
   2. Unparses the logical plan with `plan_to_sql` and the default dialect.
   3. Plans the unparsed SQL again in the same `SessionContext`.
   4. Runs the re-planned query and returns its rows for the normal comparison.
   
   Like the Substrait engine, it passes through plans that SQL cannot express 
(`Ddl`, `Dml`, `Copy`, `Explain`, `Show`, `Statement`) unchanged.
   
   A failure in step 2 or 3 is a bug in the unparser or the planner. A row 
difference in step 4 is a semantic bug in the unparser, which is harder to find 
by hand.
   
   ### Describe alternatives you've considered
   
   - Keep adding cases to `plan_to_sql.rs` per bug report. That works, but it 
finds each gap only after a user hits it.
   - Compare the re-planned logical plan with the original plan instead of the 
rows. That is stricter, but many correct unparses do not give an identical plan 
(aliases, cast placement). Comparing the rows matches what the Substrait mode 
does.
   
   ### Additional context
   
   - Many files will fail at first. So a first version can be opt-in only, like 
`--substrait-round-trip`, which I do not see in CI today. A later step can run 
it in CI with a list of known-failing files that shrinks over time.
   - The known-failing list doubles as an inventory of unparser gaps.
   - I can work on this if maintainers agree with the approach.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to