[
https://issues.apache.org/jira/browse/SPARK-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yin Huai updated SPARK-14118:
-----------------------------
Description:
Right now, we have many DDL/DML commands that are passed to Hive, which may
cause missing functionality, failures with bad error messages, or inconsistent
behaviors (e.g. a command that works with some cases but fails for other
cases). For Spark 2.0, it will be great to not ask Hive to process those
DDL/DML commands.
You can find the doc at
https://issues.apache.org/jira/secure/attachment/12793435/Implementing%20native%20DDL%20and%20DML%20statements%20for%20Spark%202.pdf
(under SPARK-13879).
There are mainly two kinds of commands, (1) Native, i.e. we want to have native
implementation in Spark; (2) Exception, i.e. we should throw an exception. That
doc has a few commands that are marked as TBD. We should first throw exceptions
for them.
Sub-tasks are created based on the doc. A command is represented by its
corresponding Token.
was:
Right now, we have many DDL/DML commands that are passed to Hive, which may
cause missing functionality, failures with bad error messages, or inconsistent
behaviors (e.g. a command that works with some cases but fails for other
cases). For Spark 2.0, it will be great to not ask Hive to process those
DDL/DML commands.
You can find the doc at
https://issues.apache.org/jira/secure/attachment/12793435/Implementing%20native%20DDL%20and%20DML%20statements%20for%20Spark%202.pdf
(under SPARK-13879).
There are mainly two kinds of commands, (1) Native, i.e. we want to have native
implementation in Spark; (2) Exception, i.e. we should throw an exception. That
doc has a few commands that are marked as TBD. We should first throw exceptions
for them.
> Implement DDL/DML commands for Spark 2.0
> ----------------------------------------
>
> Key: SPARK-14118
> URL: https://issues.apache.org/jira/browse/SPARK-14118
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Yin Huai
> Priority: Blocker
>
> Right now, we have many DDL/DML commands that are passed to Hive, which may
> cause missing functionality, failures with bad error messages, or
> inconsistent behaviors (e.g. a command that works with some cases but fails
> for other cases). For Spark 2.0, it will be great to not ask Hive to process
> those DDL/DML commands.
> You can find the doc at
> https://issues.apache.org/jira/secure/attachment/12793435/Implementing%20native%20DDL%20and%20DML%20statements%20for%20Spark%202.pdf
> (under SPARK-13879).
> There are mainly two kinds of commands, (1) Native, i.e. we want to have
> native implementation in Spark; (2) Exception, i.e. we should throw an
> exception. That doc has a few commands that are marked as TBD. We should
> first throw exceptions for them.
> Sub-tasks are created based on the doc. A command is represented by its
> corresponding Token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
