[GitHub] spark pull request: Add toDataFrame to PySpark SQL

2015-02-10 Thread dwmclary
Github user dwmclary closed the pull request at: https://github.com/apache/spark/pull/4421 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Add toDataFrame to PySpark SQL

2015-02-10 Thread dwmclary
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73783731 I've been thinking of it as equivalent to a CREATE TABLE, in which case I think it's dialect-specific. Perhaps ANSI and pgSQL allow it, but, for example, Oracle

[GitHub] spark pull request: Add toDataFrame to PySpark SQL

2015-02-10 Thread dwmclary
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73770007 OK, I've updated this to use as a reference. One thing we may want to take from this PR is that toDataFrame and createDataFrame absolutely need to check reserved words

[GitHub] spark pull request: Add toDataFrame to PySpark SQL

2015-02-10 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73771328 @dwmclary It's almost ready: https://github.com/apache/spark/pull/4498 --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: Add toDataFrame to PySpark SQL

2015-02-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73772116 Believe it or not that is valid SQL ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: Add toDataFrame to PySpark SQL

2015-02-10 Thread dwmclary
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73771478 So, we'll allow a column named SELECT regardless of whether it's been called out as `SELECT`? It just seems to me that it invites a lot of potentially erroneous

[GitHub] spark pull request: Add toDataFrame to PySpark SQL

2015-02-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73770871 Why do you need to check reserved words. In SQL you can use backticks to access columns that are named after reserved words. --- If your project is set up for it, you