[GitHub] spark pull request #22499: [SPARK-25489][ML][TEST] Refactor UDTSerialization...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22499 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22499: [SPARK-25489][ML][TEST] Refactor UDTSerialization...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22499#discussion_r219709959
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/linalg/UDTSerializationBenchmark.scala
---
@@ -18,52 +18,52 @@
package org.apache.spark.mllib.linalg
import org.apache.spark.sql.catalyst.encoders.ExpressionEncoder
-import org.apache.spark.util.Benchmark
+import org.apache.spark.util.{Benchmark, BenchmarkBase =>
FileBenchmarkBase}
/**
* Serialization benchmark for VectorUDT.
+ * To run this benchmark:
+ * 1. without sbt: bin/spark-submit --class
--- End diff --
Thank you, @seancxmao . Happy festival! :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22499: [SPARK-25489][ML][TEST] Refactor UDTSerialization...
Github user seancxmao commented on a diff in the pull request:
https://github.com/apache/spark/pull/22499#discussion_r219698800
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/linalg/UDTSerializationBenchmark.scala
---
@@ -18,52 +18,52 @@
package org.apache.spark.mllib.linalg
import org.apache.spark.sql.catalyst.encoders.ExpressionEncoder
-import org.apache.spark.util.Benchmark
+import org.apache.spark.util.{Benchmark, BenchmarkBase =>
FileBenchmarkBase}
/**
* Serialization benchmark for VectorUDT.
+ * To run this benchmark:
+ * 1. without sbt: bin/spark-submit --class
--- End diff --
I have rebased this PR to the latest master and also fixed the docs. I am
outing for mid-autumn festival, sorry for late reply. BTW, happy mid-autumn
festival.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22499: [SPARK-25489][ML][TEST] Refactor UDTSerialization...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22499#discussion_r219685155
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/linalg/UDTSerializationBenchmark.scala
---
@@ -18,52 +18,52 @@
package org.apache.spark.mllib.linalg
import org.apache.spark.sql.catalyst.encoders.ExpressionEncoder
-import org.apache.spark.util.Benchmark
+import org.apache.spark.util.{Benchmark, BenchmarkBase =>
FileBenchmarkBase}
/**
* Serialization benchmark for VectorUDT.
+ * To run this benchmark:
+ * 1. without sbt: bin/spark-submit --class
--- End diff --
+1 for fix the docs to pass Jenkins.
Also, could you rebase this PR to resolve conflicts, @seancxmao ?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22499: [SPARK-25489][ML][TEST] Refactor UDTSerialization...
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/22499#discussion_r219366799
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/linalg/UDTSerializationBenchmark.scala
---
@@ -18,52 +18,52 @@
package org.apache.spark.mllib.linalg
import org.apache.spark.sql.catalyst.encoders.ExpressionEncoder
-import org.apache.spark.util.Benchmark
+import org.apache.spark.util.{Benchmark, BenchmarkBase =>
FileBenchmarkBase}
/**
* Serialization benchmark for VectorUDT.
+ * To run this benchmark:
+ * 1. without sbt: bin/spark-submit --class
--- End diff --
I think `<` should replaced to `[`:
```scala
[error]
/home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/target/java/org/apache/spark/sql/DatasetBenchmark.java:5:
error: unknown tag: this
[error] * 1. without sbt: bin/spark-submit --class
[error] ^
[error]
/home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/target/java/org/apache/spark/sql/DatasetBenchmark.java:5:
error: unknown tag: spark
[error] * 1. without sbt: bin/spark-submit --class
[error] ^
[error]
/home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/target/java/org/apache/spark/sql/DatasetBenchmark.java:6:
error: unknown tag: this
[error] * 2. build/sbt "sql/test:runMain "
[error] ^
[error]
/home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/target/java/org/apache/spark/sql/DatasetBenchmark.java:7:
error: unknown tag: this
[error] * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
[error]
```
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22499: [SPARK-25489][ML][TEST] Refactor UDTSerialization...
GitHub user seancxmao opened a pull request: https://github.com/apache/spark/pull/22499 [SPARK-25489][ML][TEST] Refactor UDTSerializationBenchmark ## What changes were proposed in this pull request? Refactor `UDTSerializationBenchmark` to use main method and print the output as a separate file. Run blow command to generate benchmark results: ``` SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "mllib/test:runMain org.apache.spark.mllib.linalg.UDTSerializationBenchmark" ``` ## How was this patch tested? Manual tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/seancxmao/spark SPARK-25489 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22499.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22499 commit e43dda9c279fdffa3293e3ef40897cf06fb7cbfa Author: seancxmao Date: 2018-09-20T15:32:17Z [SPARK-25489] Refactor UDTSerializationBenchmark --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
