[
https://issues.apache.org/jira/browse/SPARK-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028245#comment-14028245
]
Cheng Lian edited comment on SPARK-2081 at 6/11/14 7:12 PM:
------------------------------------------------------------
To [~marmbrus]: Just found out that I happened to solve this issue while
working on [SPARK-2094 (exactly once semantics for DDL and
commands)|https://issues.apache.org/jira/browse/SPARK-2094], please assign this
issue to me.
A tricky part here is how to deal with {{NativeCommand}}. Currently, we split
each output text line around {{'\t'}}, and make each of the component a field
of the resulting rows. This makes the schema of a {{NativeCommand}}
unpredictable (we don't know how many field are there) unless we create
concrete logical/physical plan classes for each kind of command. To simplify
things for now, I propose that the schema of a {{NativeCommand}} only owns 1
"result" field whose data type is {{StringType}}, and stop splitting the output
text lines.
was (Author: lian cheng):
[~marmbrus] Just found out that I happened to solve this issue while working on
[SPARK-2094 (exactly once semantics for DDL and
commands)|https://issues.apache.org/jira/browse/SPARK-2094], please assign this
issue to me.
A tricky part here is how to deal with {{NativeCommand}}. Currently, we split
each output text line around {{'\t'}}, and make each of the component a field
of the resulting rows. This makes the schema of a {{NativeCommand}}
unpredictable (we don't know how many field are there) unless we create
concrete logical/physical plan classes for each kind of command. To simplify
things for now, I propose that the schema of a {{NativeCommand}} only owns 1
"result" field whose data type is {{StringType}}, and stop splitting the output
text lines.
> Undefine output() from the abstract class Command and implement it in
> concrete subclasses
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-2081
> URL: https://issues.apache.org/jira/browse/SPARK-2081
> Project: Spark
> Issue Type: Improvement
> Reporter: Zongheng Yang
> Assignee: Zongheng Yang
> Priority: Minor
>
> It doesn't make too much sense to have that method in the abstract class.
> Relevant discussions / cases where this issue comes up:
> https://github.com/apache/spark/pull/956
> https://github.com/apache/spark/pull/1003
--
This message was sent by Atlassian JIRA
(v6.2#6252)
