Hey Stephen, SSH actually supports creating a HTTP proxy through the -D flag. Take a look at the -D option on our spark-ec2 script for example, which just exposes the -D option of ssh. With this feature you can do stuff like ssh -D 8088 <host> and then configure localhost:8088 as a proxy in your web browser, and browse any page accessible from the machine you've SSHed into.
Matei On Oct 25, 2013, at 7:40 PM, Stephen Haberman <[email protected]> wrote: > Hey, > > The new Spark UI looks awesome. Unfortunately, it's hard for me to use > in a real browser (on my desktop, vs. lynx on the cluster), since our > (EMR) clusters are basically only accessible via SSH. > > I can port forward the master:8080 web UI port locally, and it looks > great, but I can't follow any links through to the slaves. The links > technically use the public AWS host names, but we don't have those > ports opened in the security group. > > Two things come to mind: > > 1) Maybe the entire UI could be rendered on the master? Either based on > the master's data or making actor/JSON/whatever calls to the slaves. > > 2) Somehow proxy links through the master to each slave. Like > master:8080/slave/<id>/<some-path> ends up proxied to the > slave:8081/<some-path>, and the master just sends the HTML as-is back > to the client. > > I'm not very familiar with the web UI, so I'm not sure which would be > easier. I also don't mean to bikeshed on the really great UI. > > But, at least with our current setup, this would demonstrably increase > the usability for me. > > Is this something that you guys have considered? Do you have any > thoughts on it? WIP? Open to contributions? > > Thanks! > > - Stephen
