[ https://issues.apache.org/jira/browse/SPARK-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028245#comment-14028245 ]
Cheng Lian edited comment on SPARK-2081 at 6/11/14 7:12 PM: ------------------------------------------------------------ To [~marmbrus]: Just found out that I happened to solve this issue while working on [SPARK-2094 (exactly once semantics for DDL and commands)|https://issues.apache.org/jira/browse/SPARK-2094], please assign this issue to me. A tricky part here is how to deal with {{NativeCommand}}. Currently, we split each output text line around {{'\t'}}, and make each of the component a field of the resulting rows. This makes the schema of a {{NativeCommand}} unpredictable (we don't know how many field are there) unless we create concrete logical/physical plan classes for each kind of command. To simplify things for now, I propose that the schema of a {{NativeCommand}} only owns 1 "result" field whose data type is {{StringType}}, and stop splitting the output text lines. was (Author: lian cheng): [~marmbrus] Just found out that I happened to solve this issue while working on [SPARK-2094 (exactly once semantics for DDL and commands)|https://issues.apache.org/jira/browse/SPARK-2094], please assign this issue to me. A tricky part here is how to deal with {{NativeCommand}}. Currently, we split each output text line around {{'\t'}}, and make each of the component a field of the resulting rows. This makes the schema of a {{NativeCommand}} unpredictable (we don't know how many field are there) unless we create concrete logical/physical plan classes for each kind of command. To simplify things for now, I propose that the schema of a {{NativeCommand}} only owns 1 "result" field whose data type is {{StringType}}, and stop splitting the output text lines. > Undefine output() from the abstract class Command and implement it in > concrete subclasses > ----------------------------------------------------------------------------------------- > > Key: SPARK-2081 > URL: https://issues.apache.org/jira/browse/SPARK-2081 > Project: Spark > Issue Type: Improvement > Reporter: Zongheng Yang > Assignee: Zongheng Yang > Priority: Minor > > It doesn't make too much sense to have that method in the abstract class. > Relevant discussions / cases where this issue comes up: > https://github.com/apache/spark/pull/956 > https://github.com/apache/spark/pull/1003 -- This message was sent by Atlassian JIRA (v6.2#6252)