Github user concretevitamin commented on a diff in the pull request:
https://github.com/apache/spark/pull/1071#discussion_r13728707
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala ---
@@ -22,45 +22,69 @@ import org.apache.spark.rdd.RDD
import org.apache.spark.sql.{SQLContext, Row}
import org.apache.spark.sql.catalyst.expressions.{GenericRow, Attribute}
+trait PhysicalCommand {
+ /**
+ * A concrete command should override this lazy field to wrap up any
side effects caused by the
+ * command or any other computation that should be evaluated exactly
once. The value of this field
+ * can be used as the contents of the corresponding RDD generated from
the physical plan of this
+ * command.
+ *
+ * The `execute()` method of all the physical command classes should
reference `sideEffect` so
+ * that the command can be executed eagerly right after the command
query is created.
+ */
+ protected[sql] lazy val sideEffectResult: Seq[Any] = Seq.empty[Any]
+}
+
/**
* :: DeveloperApi ::
*/
@DeveloperApi
-case class SetCommandPhysical(key: Option[String], value: Option[String],
output: Seq[Attribute])
- (@transient context: SQLContext) extends
LeafNode {
- def execute(): RDD[Row] = (key, value) match {
- // Set value for key k; the action itself would
- // have been performed in QueryExecution eagerly.
- case (Some(k), Some(v)) => context.emptyResult
+case class SetCommandPhysical(
+ key: Option[String], value: Option[String], output: Seq[Attribute])(
+ @transient context: SQLContext)
+ extends LeafNode with PhysicalCommand {
+
+ override protected[sql] lazy val sideEffectResult: Seq[(String, String)]
= (key, value) match {
+ // Set value for key k.
+ case (Some(k), Some(v)) =>
+ context.set(k, v)
+ Array.empty[(String, String)]
--- End diff --
Have we decided to not echo the newly set key-val pair?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
