I'm looking at adding system fields to certain tables in Druid: they should be referenceable by name but shouldn't show up through star-expansion. For example, we'd like to expose the name of the data file currently being read as "__filename".
I'm trying to figure out if this is something we can do with Calcite right now or if we need to do some Calcite changes first. I did find a couple of things, but AFAICT they are not exactly what I'm looking for: - There is discussion in CALCITE-2755 about treating Elasticsearch's _id field as a system field, but it looks like the _id field is a key in a map rather than a top-level field, and its visibility is managed at runtime by ElasticsearchProject rather than being handled at the validator. - In SqlValidatorImpl expandStar, there's a variable "includeSystemVars" that is set to false when called by SqlToRelConverter. Seems promisingly named, although it doesn't seem to do anything as far as I can tell. I'm wondering if I'm missing something, or if this is an incomplete feature. It's also not clear to me how the validator would figure out which fields are system fields anyway, since I don't see an obvious place to mark them in a RelDataType, which I think is how the validator learns about available fields from its tables. Happy to do a contribution to Calcite if one is needed. Thanks, Gian
