[galaxy-dev] Galaxy cannot start last [server:handlerxx]
It worked like a charm Thanks --R On Fri, Aug 9, 2013 at 12:32 PM, Nate Coraor wrote: > > On Aug 9, 2013, at 1:16 PM, Rodolfo Aramayo wrote: > >> Dear Nate, >> >> Thanks >> >> Just to be sure I understand this correctly >> >> I now have in my 'universe' file the following: >> >> # >> # HTTP Server >> -- >> >> # Configuration of the internal HTTP server. >> >> [server:main] >> >> # job_manager = manager > errors when it is not commented >> >> >> >> [server:manager] >> use = egg:Paste#http >> port = 8200 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> >> >> [server:handler0] >> use = egg:Paste#http >> port = 8300 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> [server:handler1] >> use = egg:Paste#http >> port = 8301 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> ...snip... >> >> [server:handler19] >> use = egg:Paste#http >> port = 8319 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> # [server:handler20] >> # use = egg:Paste#http >> # port = 8320 >> # host = 127.0.0.1 >> # use_threadpool = true >> # threadpool_workers = 5 >> >> >> # The internal HTTP server to use. Currently only Paste is provided. >> This >> # option is required. >> use = egg:Paste#http >> >> # The port on which to listen. >> #port = 8080 >> >> # The address on which to listen. By default, only listen to >> localhost (Galaxy >> # will not be accessible over the network). Use '0.0.0.0' to listen >> on all >> # available network interfaces. >> #host = 127.0.0.1 >> host = 0.0.0.0 >> >> # Use a threadpool for the web server instead of creating a thread for >> each >> # request. >> use_threadpool = True >> >> # Number of threads in the web server thread pool. >> threadpool_workers = 200 >> >> >> [server:web0] >> use = egg:Paste#http >> port = 8400 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 10 >> >> [server:web1] >> use = egg:Paste#http >> port = 8401 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 10 >> >> ...snip... >> >> [server:web11] >> use = egg:Paste#http >> port = 8411 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 10 >> >> [server:web12] >> use = egg:Paste#http >> port = 8412 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 10 >> >> >> # Filters -- >> >> # Filters sit between Galaxy and the HTTP server. >> ... >> >> # >> >> And in my job_conf.xml: >> >> >> >> ...snip... >> >> >> >> # >> >> Is this correct? > > Rodolfo, > > Some of this confusion might be because you have the (documented) options > from the original [server:main] section orphaned underneath > [server:handler19], which already has those options set. In an ini file, > sections begin with [name] and all key/value pairs underneath that heading > are assigned to [name] until another section heading is encountered. > Removing all comments, that means your config would look like this: > > [server:main] > > [server:manager] > use = egg:Paste#http > port = 8200 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler0] > use = egg:Paste#http > port = 8300 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler1] > use = egg:Paste#http > port = 8301 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler19] > use = egg:Paste#http > port = 8319 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > use = egg:Paste#http > host = 0.0.0.0 > use_threadpool = True > threadpool_workers = 200 > > [server:web0] > use = egg:Paste#http > port = 8400 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > [server:web1] > use = egg:Paste#http > port = 8401 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > [server:web11] > use = egg:Paste#http > port = 8411 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > [server:web12] > use = egg:Paste#http > port = 8412 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > - > > Remove [server:main], the entire [server:manager] section, and those extra > options under [server:handler19], and you should be good: > > [server:handler0] > use = egg:Paste#http > port = 8300 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler1] > us
Re: [galaxy-dev] Galaxy cannot start last [server:handlerxx]
On Aug 9, 2013, at 1:16 PM, Rodolfo Aramayo wrote: > Dear Nate, > > Thanks > > Just to be sure I understand this correctly > > I now have in my 'universe' file the following: > > # > # HTTP Server > -- > > # Configuration of the internal HTTP server. > > [server:main] > > # job_manager = manager errors when it is not commented > > > > [server:manager] > use = egg:Paste#http > port = 8200 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > > > [server:handler0] > use = egg:Paste#http > port = 8300 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler1] > use = egg:Paste#http > port = 8301 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > ...snip... > > [server:handler19] > use = egg:Paste#http > port = 8319 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > # [server:handler20] > # use = egg:Paste#http > # port = 8320 > # host = 127.0.0.1 > # use_threadpool = true > # threadpool_workers = 5 > > > # The internal HTTP server to use. Currently only Paste is provided. > This > # option is required. > use = egg:Paste#http > > # The port on which to listen. > #port = 8080 > > # The address on which to listen. By default, only listen to > localhost (Galaxy > # will not be accessible over the network). Use '0.0.0.0' to listen > on all > # available network interfaces. > #host = 127.0.0.1 > host = 0.0.0.0 > > # Use a threadpool for the web server instead of creating a thread for > each > # request. > use_threadpool = True > > # Number of threads in the web server thread pool. > threadpool_workers = 200 > > > [server:web0] > use = egg:Paste#http > port = 8400 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > [server:web1] > use = egg:Paste#http > port = 8401 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > ...snip... > > [server:web11] > use = egg:Paste#http > port = 8411 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > [server:web12] > use = egg:Paste#http > port = 8412 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 > > > # Filters -- > > # Filters sit between Galaxy and the HTTP server. > ... > > # > > And in my job_conf.xml: > > > > ...snip... > > > > # > > Is this correct? Rodolfo, Some of this confusion might be because you have the (documented) options from the original [server:main] section orphaned underneath [server:handler19], which already has those options set. In an ini file, sections begin with [name] and all key/value pairs underneath that heading are assigned to [name] until another section heading is encountered. Removing all comments, that means your config would look like this: [server:main] [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 use = egg:Paste#http host = 0.0.0.0 use_threadpool = True threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 - Remove [server:main], the entire [server:manager] section, and those extra options under [server:handler19], and you should be good: [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http
[galaxy-dev] Galaxy cannot start last [server:handlerxx]
On Thu, Aug 8, 2013 at 1:31 PM, Nate Coraor wrote: > On Aug 5, 2013, at 6:23 PM, Rodolfo Aramayo wrote: > >> Hi, >> >> I have a 80 core/1TB RAM Dell server. >> In order improve Galaxy performance I modified the universe_wsgi.ini >> file so as to improve the Scaling and Load Balancing >> as follows: >> >> # Use a threadpool for the web server instead of creating a thread for each >> # request. >> use_threadpool = True >> >> # Number of threads in the web server thread pool. >> threadpool_workers = 200 >> >> >> [server:web0] >> use = egg:Paste#http >> port = 8090 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 10 > > ... snip ... > >> [server:handler18] >> use = egg:Paste#http >> port = 8318 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> # [server:handler19] >> # use = egg:Paste#http >> # port = 8319 >> # host = 127.0.0.1 >> # use_threadpool = true >> # threadpool_workers = 5 >> >> # [server:handler20] >> # use = egg:Paste#http >> # port = 8320 >> # host = 127.0.0.1 >> # use_threadpool = true >> # threadpool_workers = 5 >> >> >> job_manager = manager >> job_handlers = >> handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18 > > Hi Rodolfo, > > The job_manager and job_handlers options go under the [server:main] section, > it looks like they are probably underneath the last [server:...] section > right now. > > Please note that all of these features should now be configured via the job > configuration file (job_conf.xml), and I've just updated the documentation > you referred to accordingly. > > --nate Dear Nate, Thanks Just to be sure I understand this correctly I now have in my 'universe' file the following: # # HTTP Server -- # Configuration of the internal HTTP server. [server:main] # job_manager = manager ...snip... # Is this correct? Also, I guess the part I do not understand is what is the magic formula between the number of threadpool_workers as configured here: # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 and those configured on the web servers and on the server handlers? Thanks --Rodolfo ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy cannot start last [server:handlerxx]
On Aug 5, 2013, at 6:23 PM, Rodolfo Aramayo wrote: > Hi, > > I have a 80 core/1TB RAM Dell server. > In order improve Galaxy performance I modified the universe_wsgi.ini > file so as to improve the Scaling and Load Balancing > as follows: > > # Use a threadpool for the web server instead of creating a thread for each > # request. > use_threadpool = True > > # Number of threads in the web server thread pool. > threadpool_workers = 200 > > > [server:web0] > use = egg:Paste#http > port = 8090 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 10 ... snip ... > [server:handler18] > use = egg:Paste#http > port = 8318 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > # [server:handler19] > # use = egg:Paste#http > # port = 8319 > # host = 127.0.0.1 > # use_threadpool = true > # threadpool_workers = 5 > > # [server:handler20] > # use = egg:Paste#http > # port = 8320 > # host = 127.0.0.1 > # use_threadpool = true > # threadpool_workers = 5 > > > job_manager = manager > job_handlers = > handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18 Hi Rodolfo, The job_manager and job_handlers options go under the [server:main] section, it looks like they are probably underneath the last [server:...] section right now. Please note that all of these features should now be configured via the job configuration file (job_conf.xml), and I've just updated the documentation you referred to accordingly. --nate > > > > Now the problem I have is that when I start galaxy I cannot get the > last [server:handler18] to load correclt > I get the following error: > > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling > 'library_contents' API controller, class: LibraryContentsController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling > 'group_users' API controller, class: GroupUsersAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling > 'configuration' API controller, class: ConfigurationController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,237 Enabling > 'requests' API controller, class: RequestsAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling > 'folders' API controller, class: FoldersController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling > 'quotas' API controller, class: QuotaAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling > 'folder_contents' API controller, class: FolderContentsController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling > 'roles' API controller, class: RoleAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling > 'request_types' API controller, class: RequestTypeAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling > 'users' API controller, class: UserAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling > 'samples' API controller, class: SamplesAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling > 'forms' API controller, class: FormDefinitionAPIController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling > 'permissions' API controller, class: PermissionsController > galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,242 Enabling > 'workflows' API controller, class: WorkflowsAPIController > galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,242 > app.config.api_folders: False > galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling > 'httpexceptions' middleware > galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling > 'recursive' middleware > galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,700 Enabling > 'error' middleware > galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling > 'trans logger' middleware > galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling > 'x-forwarded-host' middleware > galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling > 'Request ID' middleware > Starting server in PID 3899. > Traceback (most recent call last): > File "./scripts/paster.py", line 33, in >serve.run() > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", > line 1049, in run >invoke(command, command_name, options, args[1:]) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", > line 1055, in invoke >exit_code = runner.run(args) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", > line 220, in run >result = self.command() > File
[galaxy-dev] Galaxy cannot start last [server:handlerxx]
Hi, I have a 80 core/1TB RAM Dell server. In order improve Galaxy performance I modified the universe_wsgi.ini file so as to improve the Scaling and Load Balancing as follows: # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web2] use = egg:Paste#http port = 8092 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web3] use = egg:Paste#http port = 8093 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web4] use = egg:Paste#http port = 8094 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web5] use = egg:Paste#http port = 8095 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web6] use = egg:Paste#http port = 8096 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web7] use = egg:Paste#http port = 8097 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web8] use = egg:Paste#http port = 8098 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web9] use = egg:Paste#http port = 8099 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web10] use = egg:Paste#http port = 8100 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler2] use = egg:Paste#http port = 8302 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler3] use = egg:Paste#http port = 8303 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler4] use = egg:Paste#http port = 8304 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler5] use = egg:Paste#http port = 8305 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler6] use = egg:Paste#http port = 8306 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler7] use = egg:Paste#http port = 8307 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler8] use = egg:Paste#http port = 8308 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler9] use = egg:Paste#http port = 8309 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler10] use = egg:Paste#http port = 8310 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler11] use = egg:Paste#http port = 8311 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler12] use = egg:Paste#http port = 8312 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler13] use = egg:Paste#http port = 8313 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler14] use = egg:Paste#http port = 8314 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler15] use = egg:Paste#http port = 8315 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler16] use = egg:Paste#http port = 8316 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler17] use = egg:Paste#http port = 8317 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler18] use = egg:Paste#http port = 8318 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler19] # use = egg:Paste#http # port = 8319 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 job_manager = manager job_handlers = handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18 Now the problem I have is that when I start galaxy I cannot get the last [server:handler18] to load correclt I get the following error: galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'library_contents' API controller, class:
