Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22375
**[Test build #95861 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95861/testReport)**
for PR 22375 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22347
**[Test build #95859 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95859/testReport)**
for PR 22347 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22370#discussion_r216217235
--- Diff: R/pkg/R/catalog.R ---
@@ -69,7 +69,6 @@ createExternalTable <- function(x, ...) {
#' @param ... additional named parameters as options
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21968
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22378
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user MichaelChirico commented on the issue:
https://github.com/apache/spark/pull/22370
@felixcheung I disagree... what's the point of deprecation if it's going to
keep being considered as a co-equal function in the eyes of documentation?
If the function is being
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22378
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21596
**[Test build #95866 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95866/testReport)**
for PR 21596 at commit
Github user Fokko commented on the issue:
https://github.com/apache/spark/pull/21596
@HyukjinKwon I've rebased onto master since the Spark 2.4 branch has been
cut.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22237
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22237
Will take a look soon.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22237
> Will take a look soon.
@HyukjinKwon Thank you. Waiting for you feedback.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22357
**[Test build #95868 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95868/testReport)**
for PR 22357 at commit
Github user Dooyoung-Hwang commented on the issue:
https://github.com/apache/spark/pull/22347
I tested in my local PC. 3.3 GHz Intel Core i5, and selected 400,000 rows x
25 times.
I took a total execution time between decodeUnsafeRows.
My tested data is skewed, so gathered
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/22348
Could I ask you to review this PR if you have the time? cc @dongjoon-hyun
@cloud-fan
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21893
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22373
I think we should allow `struct` function to take empty arguments.
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18516#discussion_r216292382
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -292,14 +296,17 @@ trait
Github user onursatici commented on the issue:
https://github.com/apache/spark/pull/22146
@mccheah @erikerlandson @skonto should we keep the container selection
logic here, or should we revert back to having a configuration value for
pinning spark container names, either by name of
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/18142
@gatorsmile btw, we've already file a lira to track these kinds of all the
SQL-compiliant issues? not yet?
---
-
To unsubscribe,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22377
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/20999#discussion_r216299331
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf)
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22373#discussion_r216326459
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala ---
@@ -256,4 +256,9 @@ class VectorAssemblerSuite
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/22381
@felixcheung psl review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22357
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22237
**[Test build #95867 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95867/testReport)**
for PR 22237 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22237
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22237
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95867/
Test FAILed.
---
Github user Dooyoung-Hwang commented on the issue:
https://github.com/apache/spark/pull/22347
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22373
> I think we should allow `struct` function to take empty arguments.
Yes, I agree @cloud-fan. At least until 3.0 IMHO. But since the change was
already released in 3.0, I was not sure
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/20999#discussion_r216307940
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf)
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/20999#discussion_r216322634
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf)
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22378
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22367#discussion_r216324646
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala
---
@@ -79,7 +79,8 @@ private[csv] object
Github user wangyum closed the pull request at:
https://github.com/apache/spark/pull/22372
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22343
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22343
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95864/
Test PASSed.
---
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22379
jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18516#discussion_r216292172
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -292,14 +296,17 @@ trait
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/18516#discussion_r216296697
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -292,14 +296,17 @@ trait
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22373
ya, sorry for bothering your all. thansk, @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user Fokko commented on the issue:
https://github.com/apache/spark/pull/21596
Thanks @robert3005 But also API calls are failing, so it is not isolated to
the `RDDOperationScope`. I'm compiling Spark locally to check what the API
calls are returning.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22380
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
GitHub user skonto opened a pull request:
https://github.com/apache/spark/pull/22381
[SPARK-25394][Core] Add application status metrics source
## What changes were proposed in this pull request?
- Exposes several metrics regarding application status as a source.
-
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22373
cc @cloud-fan @jerryshao despite a very minor one, this can be considered a
regression, so may be considered as a blocker for 2.4/2.3.2?
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22343
**[Test build #95864 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95864/testReport)**
for PR 22343 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22284
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22364#discussion_r216294749
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala
---
@@ -39,10 +41,15 @@ object AttributeSet {
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22364#discussion_r216298504
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala
---
@@ -39,10 +41,15 @@ object AttributeSet {
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22364#discussion_r216303304
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala
---
@@ -39,10 +41,15 @@ object AttributeSet {
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21649
Right - I think we could inline it or simplify it further.
---
-
To
Github user ConeyLiu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22371#discussion_r216304910
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -51,6 +52,8 @@ private[spark] class
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/20999#discussion_r216305161
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -261,6 +261,14 @@ partitionVal
: identifier (EQ
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/21968
@cloud-fan thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95868/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22377
**[Test build #95863 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95863/testReport)**
for PR 22377 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21596
**[Test build #95866 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95866/testReport)**
for PR 21596 at commit
Github user robert3005 commented on the issue:
https://github.com/apache/spark/pull/21596
FYI - I have found nondeterministic flakes with RDDOperationScope in newer
jackson, you can see fix at https://github.com/palantir/spark/pull/379. What
happens is that jackson object mapper
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22373
Can you also update the PR description? thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22373#discussion_r216323311
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala ---
@@ -256,4 +256,9 @@ class VectorAssemblerSuite
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22377
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22377
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95863/
Test PASSed.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22284
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22380
cc @tdas @zsxwing @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/22380
[SPARK-25278][SQL][followup] remove the hack in ProgressReporter
## What changes were proposed in this pull request?
It turns out it's a bug that a `DataSourceV2ScanExec` instance may be
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/22370
I donât feel strongly either way.
I do think this is very minor since there are still many other ways to the
doc page for createExternalTable (eg the index page) or via ? search
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20999#discussion_r216304811
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf)
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21596
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user michaelmior commented on the issue:
https://github.com/apache/spark/pull/21433
@gatorsmile @cloud-fan I'll just go with a boolean config as there really
is no need for more than two options and this simplifies things quite a bit.
---
Github user michaelmior commented on a diff in the pull request:
https://github.com/apache/spark/pull/21433#discussion_r216285120
--- Diff: core/src/main/scala/org/apache/spark/storage/RDDInfo.scala ---
@@ -53,10 +55,16 @@ class RDDInfo(
}
private[spark] object
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21596
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95866/
Test FAILed.
---
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22377
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user ConeyLiu commented on the issue:
https://github.com/apache/spark/pull/22371
Thanks @felixcheung, @srowen, @cloud-fan for your time. There is only one
instance of `IndexShuffleBlockResolver` per executor, and the synchronize is
used to protect the modify safely when there
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22373
@maropu says it's OK to revert that part, @mgaido91 can you do that? thanks!
---
-
To unsubscribe, e-mail:
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22373
thanks @cloud-fan @maropu , I'll update this accordingly ASAP, thanks
---
-
To unsubscribe, e-mail:
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22364#discussion_r216315044
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala
---
@@ -39,10 +41,15 @@ object AttributeSet {
Github user ifilonenko commented on the issue:
https://github.com/apache/spark/pull/22376
the unit test flakiness is unrelated
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ifilonenko commented on the issue:
https://github.com/apache/spark/pull/22376
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22378
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22378
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22364
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22380
**[Test build #95878 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95878/testReport)**
for PR 22380 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22380
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
> @mallman It will be great that we can have this fix in 2.4 release as
this can dramatically reduce the data being read in many applications which is
the purpose of the original work.
I
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22383
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22375
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22375
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20673
What's the status of this PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user mmolimar commented on a diff in the pull request:
https://github.com/apache/spark/pull/22234#discussion_r216337792
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -91,9 +91,10 @@ abstract class CSVDataSource
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22357
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18544
what's the status here?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22357
Btw, this PR isn't intended to address filter push down for schema pruning.
I do think it should be another one topic.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22376
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2979/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22376
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
101 - 200 of 591 matches
Mail list logo
