[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype
dongjoon-hyun edited a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-659086163 I guess we can forbid that too consistently as a continuation of this approach. BTW, until now, it's beyond of the scope because this PR was designed to prevent Hive void type. Since Apache Spark doesn't talk to Apache Hive Metastore in case of `in-memory` catalog, other PMC member may has a different opinion. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype
dongjoon-hyun edited a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-657383898 Hi, @ulysses-you . We already choose the plan. This is a step to forbid that gracefully. For `create view v1 as select null as col`, we can add an `AnalysisException` if you want. Could you file a JIRA for that? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype
dongjoon-hyun edited a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-655295006 Could you make a follow-up(full revert or partial revert) as what you suggest, @HyukjinKwon ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype
dongjoon-hyun edited a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-655291242 First of all, it's not a good idea to add `NullType` into a new Spark datatype officially. Not only the exposure causes more complexity, but also what can we do in Spark SQL world (https://spark.apache.org/docs/latest/sql-ref-datatypes.html) if that is an official type? > If we'll still care and have NullType, let's make it a proper type in Spark. Previously, this was supported until Apache Spark 2.0.0. After that, Apache Spark didn't support void. This PR also tried to forbid `VOID`. `AstBuilder` provides a way for graceful warning. Currently, we are very careful even in the error message, we didn't mention `void type`. We called it `unknown type`. I believe this PR is one way to implement your idea, too. Of course, we can add more messages, too. > If we're going to treat void as Hive legacy, let's don't support it at all and make the direction to deprecate and remove NullType away. In any way, since this is a legitimate suggestion from @HyukjinKwon , cc @gatorsmile , too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype
dongjoon-hyun edited a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-655234096 Merged to master. Thank you for your patience, @LantaoJin . (The last commit is only about HiveDDLSuite. I tested it locally.) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
