Hi, > > Is there a way to provide minThreads and maxThreds for > Threadpool through jetty.xml for the jetty that is used by spark Web > UI? > > I am hitting an issue very similar to the issue described in > http://lifelongprogrammer.blogspot.com/2014/10/jetty-insufficient-threads-configured.html > >
I am unable to connect to the web UI after I bring up the master daemon using sbin/start-master.sh It looks like this is an issue due to the threadpool settings used by the embedded in the master daemon. > > I am unable to figure out the location where I can provide jetty.xml , so > that it is picked up by spark master daemon. > > Thanks > Vasanth > > > > On Mon, Feb 22, 2016 at 7:09 PM, Vasanth Bhat <vasb...@gmail.com> wrote: > >> Thanks a lot robin. >> >> Doing a search on Goolge, seems to indicate that I need to control the >> minThreads and maxThreds for Threadpool through jetty.xml >> >> But I am not able to find the jetty.xml in the spark installation. >> >> Thanks >> Vasanth >> >> >> On Mon, Feb 22, 2016 at 5:43 PM, Robin East <robin.e...@xense.co.uk> >> wrote: >> >>> I suspect that it is related to the warning in your master startup log: >>> >>> WARN AbstractConnector: insufficient threads configured >>> >>> I don’t know much about that area of Spark - maybe someone else on the >>> mailing list can comment - but it looks like the embedded Jetty web server >>> doesn’t have enough resources to do its job >>> >>> >>> >>> >>> >>> On 22 Feb 2016, at 12:04, Vasanth Bhat <vasb...@gmail.com> wrote: >>> >>> The port 4040 is *not* used. No process is listening on 4040. >>> >>> As per the logs, 8080 is used for WebUI. The log mentions the below >>> >>> 16/02/19 03:07:32 INFO Utils: Successfully started service 'MasterUI' on >>> port 8080. >>> 16/02/19 03:07:32 INFO MasterWebUI: Started MasterWebUI at >>> http://127.0.0.1:8080 >>> >>> However I am not able to see any UI, when I point the browser to >>> http://127.0.0.1:8080 >>> >>> >>> The browser just hangs. >>> >>> Thanks >>> Vasanth >>> >>> On Mon, Feb 22, 2016 at 4:53 PM, Robin East <robin.e...@xense.co.uk> >>> wrote: >>> >>>> port 4040 is the application UI but I think the OP is looking for the >>>> UI presented by the Spark master usually this would be 8080 >>>> >>>> >>>> >>>> >>>> >>>> On 22 Feb 2016, at 11:00, Kayode Odeyemi <drey...@gmail.com> wrote: >>>> >>>> Try http://localhost:4040 >>>> >>>> On Mon, Feb 22, 2016 at 8:23 AM, Vasanth Bhat <vasb...@gmail.com> >>>> wrote: >>>> >>>>> Thanks Gourav, Eduardo >>>>> >>>>> I tried http://localhost:8080 and http://OAhtvJ5MCA:8080/ >>>>> <http://oahtvj5mca:8080/> . Both cases the forefox just hangs. >>>>> >>>>> Also I tried with lynx text based browser. I get the message "HTTP >>>>> request sent; waiting for response." and it hangs as well. >>>>> >>>>> Is there way to enable debug logs in spark master service, to >>>>> understand what's going wrong? >>>>> >>>>> >>>>> Thanks >>>>> Vasanth >>>>> >>>>> >>>>> >>>>> On Fri, Feb 19, 2016 at 5:46 PM, Gourav Sengupta < >>>>> gourav.sengu...@gmail.com> wrote: >>>>> >>>>>> can you please try localhost:8080? >>>>>> >>>>>> Regards, >>>>>> Gourav Sengupta >>>>>> >>>>>> On Fri, Feb 19, 2016 at 11:18 AM, vasbhat <vasb...@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I have installed the spark1.6 and trying to start the master >>>>>>> (start-master.sh) and access the webUI. >>>>>>> >>>>>>> I get the following logs on running the start-master.sh >>>>>>> >>>>>>> Spark Command: /usr/jdk/instances/jdk1.8.0/jre/bin/java -cp >>>>>>> >>>>>>> /usr/local/spark-1.6.0-bin-hadoop2.6/conf/:/usr/local/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar:/usr/local/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/usr/local/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/usr/local/spark-1.6.0-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar >>>>>>> -Xms4g -Xmx4g org.apache.spark.deploy.master.Master --ip OAhtvJ5MCA >>>>>>> --port >>>>>>> 7077 --webui-port 8080 >>>>>>> ======================================== >>>>>>> Using Spark's default log4j profile: >>>>>>> org/apache/spark/log4j-defaults.properties >>>>>>> 16/02/19 03:07:30 INFO Master: Registered signal handlers for [TERM, >>>>>>> HUP, >>>>>>> INT] >>>>>>> 16/02/19 03:07:30 WARN NativeCodeLoader: Unable to load native-hadoop >>>>>>> library for your platform... using builtin-java classes where >>>>>>> applicable >>>>>>> 16/02/19 03:07:31 INFO SecurityManager: Changing view acls to: sluser >>>>>>> 16/02/19 03:07:31 INFO SecurityManager: Changing modify acls to: >>>>>>> sluser >>>>>>> 16/02/19 03:07:31 INFO SecurityManager: SecurityManager: >>>>>>> authentication >>>>>>> disabled; ui acls disabled; users with view permissions: >>>>>>> Set(sluser); users >>>>>>> with modify permissions: Set(sluser) >>>>>>> 16/02/19 03:07:32 INFO Utils: Successfully started service >>>>>>> 'sparkMaster' on >>>>>>> port 7077. >>>>>>> 16/02/19 03:07:32 INFO Master: Starting Spark master at >>>>>>> spark://OAhtvJ5MCA:7077 >>>>>>> 16/02/19 03:07:32 INFO Master: Running Spark version 1.6.0 >>>>>>> 16/02/19 03:07:32 WARN AbstractConnector: insufficient threads >>>>>>> configured >>>>>>> for SelectChannelConnector@0.0.0.0:8080 >>>>>>> 16/02/19 03:07:32 INFO Utils: Successfully started service >>>>>>> 'MasterUI' on >>>>>>> port 8080. >>>>>>> 16/02/19 03:07:32 INFO MasterWebUI: Started MasterWebUI at >>>>>>> http://127.0.0.1:8080 >>>>>>> 16/02/19 03:07:32 WARN AbstractConnector: insufficient threads >>>>>>> configured >>>>>>> for SelectChannelConnector@OAhtvJ5MCA:6066 >>>>>>> 16/02/19 03:07:32 INFO Utils: Successfully started service on port >>>>>>> 6066. >>>>>>> 16/02/19 03:07:32 INFO StandaloneRestServer: Started REST server for >>>>>>> submitting applications on port 6066 >>>>>>> 16/02/19 03:07:33 INFO Master: I have been elected leader! New >>>>>>> state: ALIVE >>>>>>> >>>>>>> ------------------------------ >>>>>>> Through netstat I can see that port 8080 is Listening >>>>>>> Now when I start firefox and access http://127.0.0.1:8080 , >>>>>>> firefox just >>>>>>> hangs with the message >>>>>>> >>>>>>> Waiting for "127.0.0.1" and does not connect to UI. >>>>>>> >>>>>>> How do I enable debug for the spark master daemon, to understand >>>>>>> what's >>>>>>> happening. >>>>>>> >>>>>>> Thanks >>>>>>> Vasanth >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> View this message in context: >>>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Re-Accessing-Web-UI-tp23029p26276.html >>>>>>> Sent from the Apache Spark User List mailing list archive at >>>>>>> Nabble.com <http://nabble.com/>. >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>> >>> >> >