[GitHub] spark pull request: Add toDataFrame to PySpark SQL
Github user dwmclary closed the pull request at: https://github.com/apache/spark/pull/4421 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Add toDataFrame to PySpark SQL
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73783731 I've been thinking of it as equivalent to a CREATE TABLE, in which case I think it's dialect-specific. Perhaps ANSI and pgSQL allow it, but, for example, Oracle disallows: SQL create table dumb_name (select varchar2(10), from varchar2(10)); create table dumb_name (select varchar2(10), from varchar2(10)) * ERROR at line 1: ORA-00904: : invalid identifier SQL create table dumb_name (select varchar2(10), from varchar2(10)); Table created. Either way, I'm fine to just close out this PR. We should close SPARK-2789 too. Cheers, Dan On Tue, Feb 10, 2015 at 11:59 AM, Reynold Xin notificati...@github.com wrote: Believe it or not that is valid SQL ... â Reply to this email directly or view it on GitHub https://github.com/apache/spark/pull/4421#issuecomment-73772116. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Add toDataFrame to PySpark SQL
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73770007 OK, I've updated this to use as a reference. One thing we may want to take from this PR is that toDataFrame and createDataFrame absolutely need to check reserved words in column names. I've added the behavior in scala and in the DataFrame Suite. Perhaps I should just open a new PR with the reserved words checking? I'll take a look at @davies PR when it shows up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Add toDataFrame to PySpark SQL
Github user davies commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73771328 @dwmclary It's almost ready: https://github.com/apache/spark/pull/4498 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Add toDataFrame to PySpark SQL
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73772116 Believe it or not that is valid SQL ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Add toDataFrame to PySpark SQL
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73771478 So, we'll allow a column named SELECT regardless of whether it's been called out as `SELECT`? It just seems to me that it invites a lot of potentially erroneous user behavior at DDL time. On Tue, Feb 10, 2015 at 11:53 AM, Michael Armbrust notificati...@github.com wrote: Why do you need to check reserved words. In SQL you can use backticks to access columns that are named after reserved words. â Reply to this email directly or view it on GitHub https://github.com/apache/spark/pull/4421#issuecomment-73770871. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Add toDataFrame to PySpark SQL
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73770871 Why do you need to check reserved words. In SQL you can use backticks to access columns that are named after reserved words. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
