Minor nitpick: a proxy created using the "-D" flag should be configured as a SOCKS proxy, not an HTTP proxy.
On Fri, Oct 25, 2013 at 9:42 PM, Matei Zaharia <[email protected]> wrote: > Hey Stephen, > > SSH actually supports creating a HTTP proxy through the -D flag. Take a look > at the -D option on our spark-ec2 script for example, which just exposes the > -D option of ssh. With this feature you can do stuff like ssh -D 8088 <host> > and then configure localhost:8088 as a proxy in your web browser, and browse > any page accessible from the machine you've SSHed into. > > Matei > > On Oct 25, 2013, at 7:40 PM, Stephen Haberman <[email protected]> > wrote: > >> Hey, >> >> The new Spark UI looks awesome. Unfortunately, it's hard for me to use >> in a real browser (on my desktop, vs. lynx on the cluster), since our >> (EMR) clusters are basically only accessible via SSH. >> >> I can port forward the master:8080 web UI port locally, and it looks >> great, but I can't follow any links through to the slaves. The links >> technically use the public AWS host names, but we don't have those >> ports opened in the security group. >> >> Two things come to mind: >> >> 1) Maybe the entire UI could be rendered on the master? Either based on >> the master's data or making actor/JSON/whatever calls to the slaves. >> >> 2) Somehow proxy links through the master to each slave. Like >> master:8080/slave/<id>/<some-path> ends up proxied to the >> slave:8081/<some-path>, and the master just sends the HTML as-is back >> to the client. >> >> I'm not very familiar with the web UI, so I'm not sure which would be >> easier. I also don't mean to bikeshed on the really great UI. >> >> But, at least with our current setup, this would demonstrably increase >> the usability for me. >> >> Is this something that you guys have considered? Do you have any >> thoughts on it? WIP? Open to contributions? >> >> Thanks! >> >> - Stephen >
