Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/12156#issuecomment-205672374
@bomeng `DISTRIBUTE` `UNSET` have been defined as one rule, so we cannot
use them as identifiers. This will fail for instance: `select unset from x`
whereas `select TBLPROPERTIES from x` would work.
The `OPTIONS` keyword is part of the spark native `CREATE [TEMPORARY] TABLE
table USING codec [OPTIONS (opt1 'foo', opt2 'bar')] [AS SELECT ...]` syntax;
we still support that.
You have a point regarding the missing non-reserved keywords. This is
however not as black and white as you would expect. There are a few things to
consider:
A lot of non-reserved keywords can cause code blowups. This is much better
in ANTLR4 than in ANTLR3 but is still a thing we need to be careful about:
- There is a difference between keywords that are non-reserved and SQL2003
keywords. Personally I think the grammar we are using shouldn't be to
restrictive, and we should allow as much as sanely possible.
- We shouldn't have to many (non-reserved) keywords at all. Case and point
of this are DataTypes. In this case you would have to do the parsing twice,
once syntactically and once semantically. So now we allow any identifier as a
`DataType` and have the `AstBuilder` throw exceptions when we cannot figure out
which `DataType` should be used.
- There is a minor regression. Hive allowed us to use `LEFT`/`RIGHT`/`FULL`
as identifiers, this causes some trouble with joins; so we made them reserved.
I have created https://issues.apache.org/jira/browse/SPARK-14398 to track
this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
