Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/14433
Merged into master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/14433
LGTM. Thanks @wangmiao1981 -- I'll keep this open for a bit to see if there
are any more comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on G
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64839/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #64839 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64839/consoleFull)**
for PR 14433 at commit
[`ef8887d`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #64839 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64839/consoleFull)**
for PR 14433 at commit
[`ef8887d`](https://github.com/apache/spark/commit/e
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@shivaram I simplified the solution by only changing the message as we
discussed. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
Any further comments? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64493/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #64493 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64493/consoleFull)**
for PR 14433 at commit
[`fb16375`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64491/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #64491 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64491/consoleFull)**
for PR 14433 at commit
[`6e66b5d`](https://github.com/apache/spark/commit/
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled a
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #64493 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64493/consoleFull)**
for PR 14433 at commit
[`fb16375`](https://github.com/apache/spark/commit/f
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64492/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #64491 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64491/consoleFull)**
for PR 14433 at commit
[`6e66b5d`](https://github.com/apache/spark/commit/6
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
`args.primaryResource` is good for this purpose. I can make change similar
to my initial commit but checking against `args.primaryResource`.
---
If your project is set up for it, you can repl
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
that's a good point actually - how about we use `args.primaryResource` or
`args.isR` that already exists in SparkSubmit?
---
If your project is set up for it, you can reply to this email and ha
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14433
It feels like some overkill unless there are going to be more uses for
changing logic based on whether it's running a shell. It seems not so bad to
define `setRootLevel` in Scala as an alias when in
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
@srowen @vanzin what do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enab
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung, Any comments on the new change?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63805/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63805 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63805/consoleFull)**
for PR 14433 at commit
[`80914e2`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63805 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63805/consoleFull)**
for PR 14433 at commit
[`80914e2`](https://github.com/apache/spark/commit/8
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63802/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63802 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63802/consoleFull)**
for PR 14433 at commit
[`ad88977`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63802 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63802/consoleFull)**
for PR 14433 at commit
[`ad88977`](https://github.com/apache/spark/commit/a
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung I think SparkSubmitAction is good for this purpose. We might
have to make the scope as [spark] instead of [deploy]. So it can be used in
other components if needed. I will create
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
Can we have this like
[SparkSubmitAction](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L55)
that extends `Enumeration`?
---
If your
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung I have an idea of creating an Enum object in scala, like the
example below:
object WeekDay {
sealed trait EnumVal
case object Mon extends EnumVal
case object T
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung I don't find a better way other than checking the string
either passed by resource or by the class name. Do you have any good ideas?
---
If your project is set up for it, you ca
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung Since all shell instances (i.e., spark-shell, pyspark and
sparkR) are initialized through spark-submit script, I think a possible
solution is to set a status flag in SparkSubmit ob
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
I am investigating a solution now. Either today or tomorrow, I can give an
update. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHu
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
Sure - but one is an subset of another (ie. knowing it's the sparkR shell
means it is running a shell)
I don't feel strongly about and we could come back later on, but thought
less fragmenta
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung Checking whether it is running from shell is not exactly the
same as checking which shell is calling it. My approach is depends on the fact
that the Logging trait is used in three
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
I think both checks are a bit fragile and would be great to a single way to
check if running as shell that is shared, and that could be what
SparkSubmit.scala call as well.
Would be bet
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung "I think we might have a need to create a helper for "am I
running in the SparkR shell" function?" Do you mean for #14258 ? Not for this
PR, right?
---
If your project is set up
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
awesome!
I think we might have a need to create a helper for "am I running in the
SparkR shell" function?
https://github.com/apache/spark/pull/14258#issuecomment-236336042
---
If
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung I checked which object calls the log and print message
accordingly.
./bin/sparkR
R version 3.3.0 (2016-05-03) -- "Supposedly Educational"
Copyright (C) 2016 Th
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63137/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63137 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63137/consoleFull)**
for PR 14433 at commit
[`a78d354`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63137 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63137/consoleFull)**
for PR 14433 at commit
[`a78d354`](https://github.com/apache/spark/commit/a
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
Jenkins, re-test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
The failure seems unrelated to the change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63132/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63132 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63132/consoleFull)**
for PR 14433 at commit
[`a78d354`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63132 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63132/consoleFull)**
for PR 14433 at commit
[`a78d354`](https://github.com/apache/spark/commit/a
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung I will try to retrieve terminal/shell type before printing out
the message. I will update the PR if I can find a way of doing that. Thanks!
---
If your project is set up for it, y
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/internal/Logging.scala#L138
---
If your project is set up for it, you can reply to this email and have your
repl
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@felixcheung Let me check the code of launching the sparkR shell. Can you
point me the code?
Thanks!
---
If your project is set up for it, you can reply to this email and have your
re
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14433
Correct, I don't think should name the function `sc.setLogLevel`
The message is coming from Scala - I changed the default logging level for
python and R recently so I think I know where it is
Github user wangmiao1981 commented on the issue:
https://github.com/apache/spark/pull/14433
@shivaram I found spark-shell and pyspark using the same message:
Python 2.7.11 |Anaconda 2.4.0 (x86_64)| (default, Dec 6 2015, 18:57:58)
[GCC 4.2.1 (Apple Inc. build 5577)] on da
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/14433
I think a better change might be to change that message if we are launching
SparkR ? cc @felixcheung
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63069/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63069 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63069/consoleFull)**
for PR 14433 at commit
[`9d3d8e5`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14433
**[Test build #63069 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63069/consoleFull)**
for PR 14433 at commit
[`9d3d8e5`](https://github.com/apache/spark/commit/9
64 matches
Mail list logo
