[jira] [Commented] (SPARK-17944) sbin/start-* scripts use of `hostname -f` fail with Solaris
[
https://issues.apache.org/jira/browse/SPARK-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590009#comment-15590009
]
Apache Spark commented on SPARK-17944:
--
User 'JnyJny' has created a pull request for this issue:
https://github.com/apache/spark/pull/15557
> sbin/start-* scripts use of `hostname -f` fail with Solaris
>
>
> Key: SPARK-17944
> URL: https://issues.apache.org/jira/browse/SPARK-17944
> Project: Spark
> Issue Type: Bug
> Components: Deploy
>Affects Versions: 2.0.1
> Environment: Solaris 10, Solaris 11
>Reporter: Erik O'Shaughnessy
>Priority: Trivial
>
> {{$SPARK_HOME/sbin/start-master.sh}} fails:
> {noformat}
> $ ./start-master.sh
> usage: hostname [[-t] system_name]
>hostname [-D]
> starting org.apache.spark.deploy.master.Master, logging to
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> failed to launch org.apache.spark.deploy.master.Master:
> --properties-file FILE Path to a custom Spark properties file.
>Default is conf/spark-defaults.conf.
> full log in
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> {noformat}
> I found SPARK-17546 which changed the invocation of hostname in
> sbin/start-master.sh, sbin/start-slaves.sh and sbin/start-mesos-dispatcher.sh
> to include the flag {{-f}}, which is not a valid command line option for the
> Solaris hostname implementation.
> As a workaround, Solaris users can substitute:
> {noformat}
> `/usr/sbin/check-hostname | awk '{print $NF}'`
> {noformat}
> Admittedly not an obvious fix, but it provides equivalent functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17944) sbin/start-* scripts use of `hostname -f` fail with Solaris
[
https://issues.apache.org/jira/browse/SPARK-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15582898#comment-15582898
]
Sean Owen commented on SPARK-17944:
---
To solve this problem -- if it's really the only script-based issue -- just
replace these 3 instances with some if-else bash construct to call
check-hostname if it exists. I wouldn't introduce a bigger abstraction for now.
> sbin/start-* scripts use of `hostname -f` fail with Solaris
>
>
> Key: SPARK-17944
> URL: https://issues.apache.org/jira/browse/SPARK-17944
> Project: Spark
> Issue Type: Bug
>Affects Versions: 2.0.1
> Environment: Solaris 10, Solaris 11
>Reporter: Erik O'Shaughnessy
>Priority: Trivial
>
> {{$SPARK_HOME/sbin/start-master.sh}} fails:
> {noformat}
> $ ./start-master.sh
> usage: hostname [[-t] system_name]
>hostname [-D]
> starting org.apache.spark.deploy.master.Master, logging to
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> failed to launch org.apache.spark.deploy.master.Master:
> --properties-file FILE Path to a custom Spark properties file.
>Default is conf/spark-defaults.conf.
> full log in
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> {noformat}
> I found SPARK-17546 which changed the invocation of hostname in
> sbin/start-master.sh, sbin/start-slaves.sh and sbin/start-mesos-dispatcher.sh
> to include the flag {{-f}}, which is not a valid command line option for the
> Solaris hostname implementation.
> As a workaround, Solaris users can substitute:
> {noformat}
> `/usr/sbin/check-hostname | awk '{print $NF}'`
> {noformat}
> Admittedly not an obvious fix, but it provides equivalent functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17944) sbin/start-* scripts use of `hostname -f` fail with Solaris
[
https://issues.apache.org/jira/browse/SPARK-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15582817#comment-15582817
]
Erik O'Shaughnessy commented on SPARK-17944:
AFAIK, check-hostname is a Solaris specific script written to provide a fully
qualified host name to sendmail.
I am fairly certain that before SPARK-16962 there were no Apache Spark
installations on Solaris SPARC platforms external to Oracle. My management
assures me that there is significant interest in having Apache Spark work
"out-of-the-box" on Solaris SPARC, thus my (admittedly recent) participation in
Apache Spark.
As far as your second question, whether "we know it otherwise works" on
Solaris, the only evidence I can offer is the paucity of issues raised as a
consequence of Oracle running Apache Spark on Solaris; SPARK-13776 which
tickled a Jetty bug and SPARK-16962 which addresses unaligned memory accesses.
The only other major problem we have encountered is with an Apache Spark
dependency, snappy-java, that does not support 64-bit Solaris SPARC (yet). We
have patches in-house which are under legal review.
I'll be happy to write a {{sbin/spark-funcs.sh}} so we have something concrete
to discuss.
> sbin/start-* scripts use of `hostname -f` fail with Solaris
>
>
> Key: SPARK-17944
> URL: https://issues.apache.org/jira/browse/SPARK-17944
> Project: Spark
> Issue Type: Bug
>Affects Versions: 2.0.1
> Environment: Solaris 10, Solaris 11
>Reporter: Erik O'Shaughnessy
>Priority: Trivial
>
> {{$SPARK_HOME/sbin/start-master.sh}} fails:
> {noformat}
> $ ./start-master.sh
> usage: hostname [[-t] system_name]
>hostname [-D]
> starting org.apache.spark.deploy.master.Master, logging to
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> failed to launch org.apache.spark.deploy.master.Master:
> --properties-file FILE Path to a custom Spark properties file.
>Default is conf/spark-defaults.conf.
> full log in
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> {noformat}
> I found SPARK-17546 which changed the invocation of hostname in
> sbin/start-master.sh, sbin/start-slaves.sh and sbin/start-mesos-dispatcher.sh
> to include the flag {{-f}}, which is not a valid command line option for the
> Solaris hostname implementation.
> As a workaround, Solaris users can substitute:
> {noformat}
> `/usr/sbin/check-hostname | awk '{print $NF}'`
> {noformat}
> Admittedly not an obvious fix, but it provides equivalent functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17944) sbin/start-* scripts use of `hostname -f` fail with Solaris
[
https://issues.apache.org/jira/browse/SPARK-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15577826#comment-15577826
]
Sean Owen commented on SPARK-17944:
---
Yes, for example `hostname` differs on OS X / Linux but the behavior of `-f`
happens to give the same desired output on both. Is `check-hostname` present
only on Solaris (or at least, does it do the right thing everywhere we think it
exists)? If so I could imagine just checking if that command exists and using
it if present.
There are 3 usages, and all could have the same treatment. That's not so bad
IMHO. A central script to switch out commands seems appealing but the
indirection non-trivially complicates the scripts. I'm not sure how far to go
to try to support Solaris, not because there's anything wrong with it, but
because a) I don't know how many Spark users are on Solaris, and b) I am not
sure we know it otherwise works, that there aren't other problems like this
> sbin/start-* scripts use of `hostname -f` fail with Solaris
>
>
> Key: SPARK-17944
> URL: https://issues.apache.org/jira/browse/SPARK-17944
> Project: Spark
> Issue Type: Bug
>Affects Versions: 2.0.1
> Environment: Solaris 10, Solaris 11
>Reporter: Erik O'Shaughnessy
>Priority: Trivial
>
> {{$SPARK_HOME/sbin/start-master.sh}} fails:
> {noformat}
> $ ./start-master.sh
> usage: hostname [[-t] system_name]
>hostname [-D]
> starting org.apache.spark.deploy.master.Master, logging to
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> failed to launch org.apache.spark.deploy.master.Master:
> --properties-file FILE Path to a custom Spark properties file.
>Default is conf/spark-defaults.conf.
> full log in
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> {noformat}
> I found SPARK-17546 which changed the invocation of hostname in
> sbin/start-master.sh, sbin/start-slaves.sh and sbin/start-mesos-dispatcher.sh
> to include the flag {{-f}}, which is not a valid command line option for the
> Solaris hostname implementation.
> As a workaround, Solaris users can substitute:
> {noformat}
> `/usr/sbin/check-hostname | awk '{print $NF}'`
> {noformat}
> Admittedly not an obvious fix, but it provides equivalent functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17944) sbin/start-* scripts use of `hostname -f` fail for Solaris
[
https://issues.apache.org/jira/browse/SPARK-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576367#comment-15576367
]
Erik O'Shaughnessy commented on SPARK-17944:
I'm sure there are situations where Linux and OS X disagree with how to get
things done, I was hoping for more inclusivity despite Solaris being a less
favored development platform.
A simple change might be to add a file similar to {{sbin/spark-config.sh}},
call it {{sbin/spark-funcs.sh}}, and populate it with a set of bash functions
which can attempt OS specific invocations until it finds one that is successful
or returns an error after exhausting all options. This could provide a
framework for resolving future conflicts between OS implementation details in
addition to addressing this particular problem.
> sbin/start-* scripts use of `hostname -f` fail for Solaris
> ---
>
> Key: SPARK-17944
> URL: https://issues.apache.org/jira/browse/SPARK-17944
> Project: Spark
> Issue Type: Bug
>Affects Versions: 2.0.1
> Environment: Solaris 10, Solaris 11
>Reporter: Erik O'Shaughnessy
>Priority: Trivial
>
> {{$SPARK_HOME/sbin/start-master.sh}} fails:
> {noformat}
> $ ./start-master.sh
> usage: hostname [[-t] system_name]
>hostname [-D]
> starting org.apache.spark.deploy.master.Master, logging to
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> failed to launch org.apache.spark.deploy.master.Master:
> --properties-file FILE Path to a custom Spark properties file.
>Default is conf/spark-defaults.conf.
> full log in
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> {noformat}
> I found SPARK-17546 which changed the invocation of hostname in
> sbin/start-master.sh, sbin/start-slaves.sh and sbin/start-mesos-dispatcher.sh
> to include the flag {{-f}}, which is not a valid command line option for the
> Solaris hostname implementation.
> As a workaround, Solaris users can substitute:
> {noformat}
> `/usr/sbin/check-hostname | awk '{print $NF}'`
> {noformat}
> Admittedly not an obvious fix, but it provides equivalent functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17944) sbin/start-* scripts use of `hostname -f` fail for Solaris
[
https://issues.apache.org/jira/browse/SPARK-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576323#comment-15576323
]
Sean Owen commented on SPARK-17944:
---
Yeah, I think Solaris is the odd man out here then. Linux and OS X support
this. I'm not sure Solaris is something Spark intends to support in general,
for reasons like this. If there's a simple change that makes it work, OK, but
otherwise I'd close this.
> sbin/start-* scripts use of `hostname -f` fail for Solaris
> ---
>
> Key: SPARK-17944
> URL: https://issues.apache.org/jira/browse/SPARK-17944
> Project: Spark
> Issue Type: Bug
>Affects Versions: 2.0.1
> Environment: Solaris 10, Solaris 11
>Reporter: Erik O'Shaughnessy
>Priority: Trivial
>
> {{$SPARK_HOME/sbin/start-master.sh}} fails:
> {noformat}
> $ ./start-master.sh
> usage: hostname [[-t] system_name]
>hostname [-D]
> starting org.apache.spark.deploy.master.Master, logging to
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> failed to launch org.apache.spark.deploy.master.Master:
> --properties-file FILE Path to a custom Spark properties file.
>Default is conf/spark-defaults.conf.
> full log in
> /home/eoshaugh/local/spark/logs/spark-eoshaugh-org.apache.spark.deploy.master.Master-1-m7-16-002-ld1.out
> {noformat}
> I found SPARK-17546 which changed the invocation of hostname in
> sbin/start-master.sh, sbin/start-slaves.sh and sbin/start-mesos-dispatcher.sh
> to include the flag {{-f}}, which is not a valid command line option for the
> Solaris hostname implementation.
> As a workaround, Solaris users can substitute:
> {noformat}
> `/usr/sbin/check-hostname | awk '{print $NF}'`
> {noformat}
> Admittedly not an obvious fix, but it provides equivalent functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
