Github user httfighter commented on the issue:
https://github.com/apache/spark/pull/22683
@srowen OK. Thank you very much for your advice.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22683
I believe most of the Spark code really uses units like KiB, multiples of
1024, as you see here. However it's often referred to as KB, incorrectly, and
so on. It seems like the UI is the only place
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23128
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23128
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23128
@gatorsmile Thanks Xiao! Conflicts resolve done, as Reynold comments in
https://github.com/apache/spark/pull/23105#discussion_r235950427, when the
ShuffleMetricsReporter move to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23128
**[Test build #99223 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99223/testReport)**
for PR 23128 at commit
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/23128#discussion_r236032855
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLShuffleMetricsReporter.scala
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed
Github user httfighter commented on the issue:
https://github.com/apache/spark/pull/22683
@srowen @ajbozarth I am not sure about some things, can you give me some
advice? In the process of modification, I have a question. In Spark, whether M
and MB represent MiB. Spark does not use
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21732
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21732
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99222/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21732
**[Test build #99222 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99222/testReport)**
for PR 21732 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23052
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99221/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23052
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23052
**[Test build #99221 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99221/testReport)**
for PR 23052 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23128
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99220/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23128
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23128
**[Test build #99220 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99220/testReport)**
for PR 23128 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21732
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21732
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21732
**[Test build #99222 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99222/testReport)**
for PR 21732 at commit
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/23128#discussion_r236025838
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLShuffleMetricsReporter.scala
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/23128#discussion_r236025817
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLShuffleMetricsReporter.scala
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23052
**[Test build #99221 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99221/testReport)**
for PR 23052 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23105
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23128
@xuanyuanking Could you address the conflicts? Thanks for you fast work!
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99216/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23125
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23125
**[Test build #99216 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99216/testReport)**
for PR 23125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23128
**[Test build #99220 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99220/testReport)**
for PR 23128 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23128
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23128
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/23105#discussion_r236020103
--- Diff: core/src/main/scala/org/apache/spark/shuffle/metrics.scala ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/23128
[SPARK-26139][SQL] Support passing shuffle metrics to exchange operator
## What changes were proposed in this pull request?
Implement `SQLShuffleMetricsReporter` on the sql side as
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23103
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23103
**[Test build #99219 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99219/testReport)**
for PR 23103 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23103
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99219/
Test PASSed.
---
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/23127
Looks good. One more higher level question that can also be addressed in a
follow-up.
---
-
To unsubscribe, e-mail:
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/23127#discussion_r236017398
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala
---
@@ -406,14 +415,39 @@ trait
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23080
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99215/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23080
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23080
**[Test build #99215 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99215/testReport)**
for PR 23080 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23127
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23127
**[Test build #99218 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99218/testReport)**
for PR 23127 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23127
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99218/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23103
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23103
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23103
**[Test build #99219 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99219/testReport)**
for PR 23103 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23022
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99217/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22512
**[Test build #99217 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99217/testReport)**
for PR 22512 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23127
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23127
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/23022
Merging to master. Thank!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user juliuszsompolski commented on the issue:
https://github.com/apache/spark/pull/23127
cc @hvanhovell @rednaxelafx
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23127
**[Test build #99218 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99218/testReport)**
for PR 23127 at commit
GitHub user juliuszsompolski opened a pull request:
https://github.com/apache/spark/pull/23127
[SPARK-26159] Codegen for LocalTableScanExec and ExistingRDDExec
## What changes were proposed in this pull request?
Implement codegen for LocalTableScanExec and ExistingRDDExec.
Github user juliuszsompolski commented on the issue:
https://github.com/apache/spark/pull/23022
ping @hvanhovell
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22466
Regarding the potentially high cost of file listing, `CREATE DATABASE` is
not a frequent operation. The cost is high only if the target directory is
non-empty with many many files. We are
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22466#discussion_r236005686
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -2370,4 +2370,17 @@ class HiveDDLSuite
))
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
Thanks, @gaborgsomogyi and @HyukjinKwon .
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235999222
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235999040
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23125
**[Test build #99216 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99216/testReport)**
for PR 23125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22512
**[Test build #99217 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99217/testReport)**
for PR 22512 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23125
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/23125#discussion_r235995724
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -79,6 +79,9 @@ private[ui] class StagePage(parent: StagesTab, store:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22512
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/23022
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/22824
@tdas or @jose-torres any opinion on whether it's worth refactoring these
checks as suggested by @arunmahadevan
---
-
To
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/23084
@cloud-fan,thanks
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/18304
@danielvdende now the newlines are automatically derected. should be not an
issue anymore.
---
-
To unsubscribe, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23080
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23080
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23080
**[Test build #99215 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99215/testReport)**
for PR 23080 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23080
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23080
@MaxGekk, thanks for working on this one.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23080
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23080
Last changes were only doc changes. Let me get this in.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23080
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23111
Hey all, I will merge this in few days if there's no more comments. It's
going to speed up the tests roughly 12 ~ 15 mins.
---
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23102#discussion_r235975268
--- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala
---
@@ -61,11 +62,12 @@ private[deploy] object DependencyUtils extends Logging {
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/23102
@MaxGekk This PR may change a separator for `userJar` that has `\` on
Windows. `resolveGlobPaths` is not applied to `userJar`.
---
Github user KyleLi1985 commented on the issue:
https://github.com/apache/spark/pull/23126
Plug do some more test on real data after add this commit
we use data from
http://archive.ics.uci.edu/ml/datasets/EEG+Steady-State+Visual+Evoked+Potential+Signals
and data
Github user KyleLi1985 commented on the issue:
https://github.com/apache/spark/pull/23126
After add this commit
We get the result for RowMatrix computeCovariance function:
For the input data
1.0,2.0,3.0,4.0,5.0
2.0,3.0,1.0,2.0,6.0
RowMatrix function
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23126
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23126
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23126
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user KyleLi1985 commented on the issue:
https://github.com/apache/spark/pull/23126
Compare Spark computeCovariance function in RowMatrix for DenseVector and
Numpy's function cov,
Find two problem, below is the result:
1)The Spark function computeCovariance in
GitHub user KyleLi1985 opened a pull request:
https://github.com/apache/spark/pull/23126
[SPARK-26158] [MLLIB] fix covariance accuracy problem for DenseVector
## What changes were proposed in this pull request?
Enhance accuracy of the covariance logic in RowMatrix for function
Github user advancedxy commented on a diff in the pull request:
https://github.com/apache/spark/pull/23083#discussion_r235955118
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala ---
@@ -727,9 +727,10 @@ private[spark] class ExternalSorter[K, V,
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235952965
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235950666
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/23105#discussion_r235950427
--- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleManager.scala
---
@@ -48,7 +48,8 @@ private[spark] trait ShuffleManager {
handle:
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235950148
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235947044
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23125
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
1 - 100 of 156 matches
Mail list logo
