jacques-n commented on pull request #3188:
URL: https://github.com/apache/iceberg/pull/3188#issuecomment-933769038
A couple of thoughts:
- Was there a discussion of resolved SQL? Most systems will want to store
this one way or another. The schema field can potentially help with this but it
feels somewhat backwards. It's actually strange that `field-aliases` is
populated for a view statement that has that part of the statement but the
schema is used to resolve field names in the case that a * or arbitrary
expression (e.g. `select a+b from foo` without field `as` clause) is used. I'm
inclined to think that the we should include both resolved and unresolved sql
to be explicit about such things. Really, the only reason to have unresolved is
so people can get back to what they typed. Engines need resolved to be reliably
consistent.
- In the context of Substrait: I think there are two distinct use cases. The
first is SQL view with substrait plan for reliable consumption cross engine.
The second is a substrait plan-based view that doesn't have associated SQL. The
latter exists when the FE that produces the plan isn't sql-based (for example,
Pandas could be the producer of the Substrait plan). It seems like there
probably should be some kind of union such as the following (using pseudo
protobuf since I'm in that world right now):
```
message ProcessingInstruction {
oneof processing_type {
Sql sql = 1;
SubstraitPlan plan = 2; // to be added in future.
}
message Sql {
string text = 1;
string dialect = 2;
SubstraitPlan plan = 3; // to be added in future.
}
// to be added in future.
message SubstraitPlan {
int32 substrait_version = 1;
binary plan = 2;
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
