[spark] branch master updated: [SPARK-26696][SQL] Makes Dataset encoder public

[wenchen]

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new b34b4c5  [SPARK-26696][SQL] Makes Dataset encoder public
b34b4c5 is described below

commit b34b4c59b4bcfa2c6007aa36e705d55af961d13f
Author: Simeon Simeonov <s...@fastignite.com>
AuthorDate: Tue Feb 12 11:04:26 2019 +0800

    [SPARK-26696][SQL] Makes Dataset encoder public
    
    ## What changes were proposed in this pull request?
    
    Implements the solution proposed in 
[SPARK-26696](https://issues.apache.org/jira/browse/SPARK-26696), a minor 
refactoring that allows frameworks to perform advanced type-preserving dataset 
transformations without carrying `Encoder` implicits from user code.
    
    The change allows
    
    ```scala
    def foo[A](ds: Dataset[A]): Dataset[A] =
      ds.toDF().as[A](ds.encoder)
    ```
    
    instead of
    
    ```scala
    def foo[A: Encoder](ds: Dataset[A]): Dataset[A] =
      ds.toDF().as[A](implicitly[Encoder[A]])
    ```
    
    ## How was this patch tested?
    
    This patch was tested with an automated test that was later removed as it 
was deemed unnecessary per the discussion in this PR.
    
    Closes #23620 from ssimeonov/ss_SPARK-26696.
    
    Authored-by: Simeon Simeonov <s...@fastignite.com>
    Signed-off-by: Wenchen Fan <wenc...@databricks.com>
---
 sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala 
b/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
index 32f6234..8a26152 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
@@ -178,7 +178,7 @@ private[sql] object Dataset {
 class Dataset[T] private[sql](
     @transient val sparkSession: SparkSession,
     @DeveloperApi @Unstable @transient val queryExecution: QueryExecution,
-    encoder: Encoder[T])
+    @DeveloperApi @Unstable @transient val encoder: Encoder[T])
   extends Serializable {
 
   queryExecution.assertAnalyzed()


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org