Github user goungoun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20800#discussion_r188019259
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -511,6 +511,14 @@ class Dataset[T] private[sql](
*/
def isLocal
Github user goungoun commented on the issue:
https://github.com/apache/spark/pull/20800
Thanks!!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user goungoun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20800#discussion_r176728379
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -511,6 +511,14 @@ class Dataset[T] private[sql](
*/
def isLocal
Github user goungoun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19876#discussion_r176911201
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -710,15 +711,58 @@ class LinearRegressionModel private[ml
Github user goungoun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19876#discussion_r178432400
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -710,15 +711,58 @@ class LinearRegressionModel private[ml
GitHub user goungoun opened a pull request:
https://github.com/apache/spark/pull/20782
[SPARK-SPARK-23627][SQL] Provide isEmpty in DataSet
## What changes were proposed in this pull request?
This PR adds a isEmpty in DataSet
## How was this patch tested
Github user goungoun closed the pull request at:
https://github.com/apache/spark/pull/20782
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
GitHub user goungoun opened a pull request:
https://github.com/apache/spark/pull/20800
isEmpty in Dataset and its testSuite
## What changes were proposed in this pull request?
This PR adds isEmpty() in DataSet
## How was this patch tested?
Unit tests added
Github user goungoun commented on the issue:
https://github.com/apache/spark/pull/20782
As unnecessary information is included, I closed this pull request. Please
refer request #20800 instead of #20782. I am sorry for your inconvenience
Github user goungoun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20800#discussion_r174002184
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -511,6 +511,12 @@ class Dataset[T] private[sql](
*/
def isLocal
Github user goungoun commented on the issue:
https://github.com/apache/spark/pull/20800
@HyukjinKwon, @maropu
Just a gentle reminder. Jenkins is waiting for a comment like 'ok to test'.
---
-
To unsu
Github user goungoun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20800#discussion_r174673621
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -511,6 +511,14 @@ class Dataset[T] private[sql](
*/
def isLocal
Github user goungoun commented on the issue:
https://github.com/apache/spark/pull/20800
@rxin, checking empty is likely to be a common process in every ETL batch
job. I think it is the right place to provide that functionality. When a basic
function is missing already supposed to be
Github user goungoun commented on the issue:
https://github.com/apache/spark/pull/20800
For additional check that I mentioned. The following code shows that Spark
users does not need to add take(1). ds.rdd.take(1).isEmpty is redundant.
[RDD.scala](https://github.com/apache
Github user goungoun commented on the issue:
https://github.com/apache/spark/pull/22428
Awesome! @HyukjinKwon , @gatorsmile thanks for good information. Let me
look into it further. By the way, I still hope this conversation is open to
users' voice, not limited with devel
GitHub user goungoun opened a pull request:
https://github.com/apache/spark/pull/22428
[SPARK-25430][SQL] Add map parameter for withColumnRenamed
## What changes were proposed in this pull request?
This PR allows withColumnRenamed with a map input argument
## How was
Github user goungoun commented on the issue:
https://github.com/apache/spark/pull/22428
@HyukjinKwon , thanks for your review. Actually, that is the reason that I
open this pull request. I think it is better to giving reusable option to users
than repeating too much of same code in
17 matches
Mail list logo
