bq. getServletHandlers is not intended for public use >From MetricsSystem.scala :
private[spark] class MetricsSystem private ( Looks like there is no easy way to extend REST API. On Thu, Mar 24, 2016 at 1:09 PM, Sebastian Kochman < sebastian.koch...@outlook.com> wrote: > Hello, > I have a question: is there a way to extend Spark REST API with > higher-level, application-specific handlers? (exposing additional > information, like app-specific metrics, but also taking some actions within > an app) > > Yes, I could host my own REST endpoint within an app, but then I have to > solve the problem of advertising that endpoint to the client. The app will > be deployed on YARN, so the port cannot be fixed (since multiple AMs can > run on a single machine). Spark's REST API endpoint is already "advertised" > by YARN RM to the client, so I was hoping I could just add to it. > > I tried calling sparkContext.metricsSystem.getServletHandlers, but the > Jetty dependencies are shaded (I guess so the application could use a > different version of Jetty), so its de-facto signature is as follows: > > public org.spark-project.jetty.servlet.ServletContextHandler[] > getServletHandlers(); > > (note the org.spark-project package) > ...so when I add the Jetty dependency (org.eclipse.jetty...) to my > project, it compiles but during runtime it throws: > > java.lang.NoSuchMethodError: > org.apache.spark.metrics.MetricsSystem.getServletHandlers()[Lorg/eclipse/jetty/servlet/ServletContextHandler; > > So it seems to me that getServletHandlers is not intended for public use - > correct? > > Is there any other way to extended Spark's REST API? > > I would appreciate your help! > Thanks, > Sebastian >
