Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15286
Thank you very much. @tgravescs @JoshRosen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15246
Hi, @srowen Yes, you are right. I am searching the code base to see if we
can fix more.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs
[SPARK-17714](https://issues.apache.org/jira/browse/SPARK-17714) has been
created for further investigation.
---
If your project is set up for it, you can reply to this email
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15286
@JoshRosen Yes. The title has been changed.
@tgravescs Yes. Now I am creating a separate jira to investigate this more.
---
If your project is set up for it, you can reply to this email
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15286
Yes. The title has been changed. Thanks. @tgravescs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Thanks @tgravescs yes. I have created a PR
[15286](https://github.com/apache/spark/pull/15286).
---
If your project is set up for it, you can reply to this email and have your
reply appear
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/15286
[SPARK-16757][Follow UP] Fix ClassCircularityError in ReplSuite tests in
Maven build: use 'Class.forName' instead of 'Utils.classForName'
## What changes were proposed in this pull request
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
@tgravescs @srowen Thanks. Using `Class.forName` which uses
`this.getClass().getClassLoader()` by default makes all the tests passed (both
sbt and maven). However there must be some reason we
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs @srowen
Give an intermediate update, If using `Class.forName `instead of
Utils.classForName`, Maven build and all of the tests will be passed.
```
def
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
@tgravescs @srowen Sorry for the failure. I am looking into it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs Should we also commit this PR to Branch-2? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15246
Hi, @srowen Thanks a lot for the comments. Yes, setting the working dir can
work. However, working dir varies from machine to machine. It would be a little
tricky to maintain and troubleshoot
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Thanks a lot for the review. @tgravescs @cnauroth @steveloughran @srowen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r80579059
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2421,6 +2421,69 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs Thanks a lot for the comments. I have updated the PR to
rename local vals and remove the `@since` in `Utils.scala`.
---
If your project is set up for it, you can reply
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r80579032
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2421,6 +2421,69 @@ private[spark] object Utils extends Logging
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/15246
[MINOR][SQL] Use resource path for test_script.sh
## What changes were proposed in this pull request?
This PR modified the test case `test("script")` to use resource path for
`test
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r80323261
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -54,7 +54,10 @@ private[spark] abstract class Task[T](
val partitionId
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r80161383
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -54,7 +54,10 @@ private[spark] abstract class Task[T](
val partitionId
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs Thank you very much. Yes. I have updated the PR to make the
string of the caller context shorter.
---
If your project is set up for it, you can reply to this email and have your
Github user Sherry302 closed the pull request at:
https://github.com/apache/spark/pull/15175
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15175
Thanks you, @srowen .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15175
@gatorsmile Title has been updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15170
@lresende @rxin Thank for the review. I have created a
[PR](https://github.com/apache/spark/pull/15175) to branch-2.0.
---
If your project is set up for it, you can reply to this email and have
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/15175
[MINOR][BUILD] Fix CheckStyle Error
## What changes were proposed in this pull request?
This PR is to fix the code style errors.
## How was this patch tested?
Manual
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15170
Retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/15170
There is 0 failures (±0) , 1 skipped (±0) in the Test Result page. I'll
re-trigger again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/15170
[MINOR][BUILD] Fix CheckStyle Error
## What changes were proposed in this pull request?
This PR is to fix the code style errors before 2.0.1 release.
## How was this patch tested
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs Thank you so much for the review. I have updated the PR
based on your every comment.
The only one question left is this one (in `Task`) "are these params all
optional
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r79695565
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r79695449
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r79693044
--- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala
---
@@ -42,7 +42,10 @@ import org.apache.spark.rdd.RDD
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r79692960
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala ---
@@ -196,8 +198,13 @@ private[spark] class ApplicationMaster
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r79693000
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r79692475
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r79692046
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -54,7 +54,10 @@ private[spark] abstract class Task[T](
val partitionId
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs Could you please review this again? I have updated the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
There is "0 failures (±0)" in the Test Result page. All tests passed. I'll
re-trigger again.
---
If your project is set up for it, you can reply to this email and have your
re
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78898595
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2418,6 +2418,21 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78898257
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2418,6 +2418,21 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78897068
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -79,6 +82,13 @@ private[spark] abstract class Task[T](
metrics
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78896928
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -79,6 +82,13 @@ private[spark] abstract class Task[T](
metrics
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78896972
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -79,6 +82,13 @@ private[spark] abstract class Task[T](
metrics
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78896863
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala ---
@@ -184,6 +184,9 @@ private[spark] class ApplicationMaster
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78896816
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2418,6 +2418,21 @@ private[spark] object Utils extends Logging
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78896707
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -54,7 +54,10 @@ private[spark] abstract class Task[T](
val partitionId
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78896692
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -51,8 +51,12 @@ private[spark] class ShuffleMapTask
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14659#discussion_r78896681
--- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala
---
@@ -51,8 +51,12 @@ private[spark] class ResultTask[T, U](
locs: Seq
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs Thank you very much for the review. I have updated the PR
based on your every comment, including adding a CallerContext class, updating
java doc, and made the caller context string
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
@tgravescs Sure. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @tgravescs Could you please review this PR? Thank you very much.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
@steveloughran Thank you very much. I have updated the PR based on your
comments. Also, I have added an unit test.
---
If your project is set up for it, you can reply to this email and have
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
@srowen Thanks all the same.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @srowen Could you please review this PR again?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @srowen Could you please review this PR? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
The only failure is 'basic functionality', but it passed locally. I'll
re-trigger again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14769
Yes. Are they ok now? @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14768
@srowen Thanks for the review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14768
For this piece of code:
I think lines in` if `and `else` blocks should
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14768#discussion_r75908543
--- Diff:
examples/src/main/java/org/apache/spark/examples/sql/streaming/JavaStructuredNetworkWordCount.java
---
@@ -61,7 +61,8 @@ public static void
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14768
Thanks, @jerryshao. I have updated the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14769
[MINOR][SQL] Remove implemented functions from comments of 'HiveSessiâ¦
## What changes were proposed in this pull request?
This PR removes implemented functions from comments
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14768
[MINOR][BUILD] Fix Java CheckStyle Error
## What changes were proposed in this pull request?
As Spark 2.0.1 will be released soon (mentioned in the spark dev mailing
list), besides
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Thanks a lot for adding me as âcontributorâ in Hadoop :) @steveloughran
@cnauroth
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @steveloughran Thanks a lot for the comments.
In the audit log, if users set some configuration in spark-defaults.conf
like `spark.eventLog.dir hdfs://localhost:9000/spark-history
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @cnauroth Thank you very much for the review and suggestion. I have
removed the spaces in the value of the caller context, and prepended "Spark"
instead (refer to the commit
[3b9a
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @steveloughran Thank you very much for the comments. I have created an
Hadoop jira [HADOOP-13527 ](https://issues.apache.org/jira/browse/HADOOP-13527)
and attached the patch, could you please
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14577
Hi, @srowen Thanks a lot for the comments. Sorry for the late reply. You
are right. I will check how other pages format the date.
---
If your project is set up for it, you can reply
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14659
Hi, @srowen . Thank you so much for the review. Sorry for the test
failure and late update. The failure reasons are that âjobIDâ were
none or there was no âspark.app.nameâ
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14659
[SPARK-16757] Set up Spark caller context to HDFS
## What changes were proposed in this pull request?
1. Pass `jobId` to Task.
2. Invoke Hadoop APIs.
A new function
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14556
@srowen Thanks for the new PR and the review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14577
Hi, @rxin . Thanks for the quick feedback. This PR is to remove time
inconsistency between webpages. Right now the times in history page is
inconsistent
with the times in other pages like
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14577
[SPARK-16986][WEB UI] Make 'Started' time, 'Completed' time and 'Lastâ¦
## What changes were proposed in this pull request?
In historypage.js, format 'Started' time, 'Completed' time
Github user Sherry302 closed the pull request at:
https://github.com/apache/spark/pull/14556
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14556
[SPARK-16966][Core] Make App Name to the valid name instead of a randâ¦
## What changes were proposed in this pull request?
In the SparkSession, before setting "spark.app
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14532
SPARK-16945: Fix Java Lint errors
## What changes were proposed in this pull request?
This PR is to fix the minor Java linter errors as following:
[ERROR]
src/main/java/org/apache/spark
Github user Sherry302 closed the pull request at:
https://github.com/apache/spark/pull/14312
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14312#discussion_r72536143
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -66,6 +66,9 @@ private[spark] class Client(
import Client
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14312
Thanks the feedback, Jerry. I am going to update the patch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14312
[SPARK-15857]Add caller context in Spark: invoke YARN/HDFS API to setâ¦
## What changes were proposed in this pull request?
1. Pass 'jobId' to Task.
2. Add a new function
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/14163
Updated the doc based on the feedback.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user Sherry302 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14163#discussion_r71425048
--- Diff: docs/monitoring.md ---
@@ -224,10 +224,12 @@ both running applications, and in the history server.
The endpoints are mounted
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14163
[SPARK-15923][YARN] Spark Application rest api returns 'no such app: â¦
## What changes were proposed in this pull request?
Update monitoring.md.
â¦'
You can merge this pull
Github user Sherry302 closed the pull request at:
https://github.com/apache/spark/pull/14024
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/14024
[SPARK-15923][YARN] Spark Application rest api returns 'no such app: â¦
## What changes were proposed in this pull request?
1. Updated the monitoring.md doc.
2
Github user Sherry302 commented on the issue:
https://github.com/apache/spark/pull/13448
Merged to master/2.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
GitHub user Sherry302 opened a pull request:
https://github.com/apache/spark/pull/13448
[SPARK-15707][SQL] Make Code Neat - Use map instead of if check.
## What changes were proposed in this pull request?
In forType function of object RandomDataGenerator, the code following
89 matches
Mail list logo
