[galaxy-dev] Galaxy cannot start last [server:handlerxx]
On Thu, Aug 8, 2013 at 1:31 PM, Nate Coraor n...@bx.psu.edu wrote: On Aug 5, 2013, at 6:23 PM, Rodolfo Aramayo wrote: Hi, I have a 80 core/1TB RAM Dell server. In order improve Galaxy performance I modified the universe_wsgi.ini file so as to improve the Scaling and Load Balancing as follows: # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 ... snip ... [server:handler18] use = egg:Paste#http port = 8318 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler19] # use = egg:Paste#http # port = 8319 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 job_manager = manager job_handlers = handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18 Hi Rodolfo, The job_manager and job_handlers options go under the [server:main] section, it looks like they are probably underneath the last [server:...] section right now. Please note that all of these features should now be configured via the job configuration file (job_conf.xml), and I've just updated the documentation you referred to accordingly. --nate Dear Nate, Thanks Just to be sure I understand this correctly I now have in my 'universe' file the following: # # HTTP Server -- # Configuration of the internal HTTP server. [server:main] # job_manager = manager I ASSUME THIS IS CORRECT because I get errors when it is not commented [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 ...snip... [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # The internal HTTP server to use. Currently only Paste is provided. This # option is required. use = egg:Paste#http # The port on which to listen. #port = 8080 # The address on which to listen. By default, only listen to localhost (Galaxy # will not be accessible over the network). Use '0.0.0.0' to listen on all # available network interfaces. #host = 127.0.0.1 host = 0.0.0.0 # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 ...snip... [server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 # Filters -- # Filters sit between Galaxy and the HTTP server. ... # And in my job_conf.xml: handler id=handler0 tags=handlers/ handler id=handler1 tags=handlers/ ...snip... handler id=handler18 tags=handlers/ handler id=handler19 tags=handlers/ # Is this correct? Also, I guess the part I do not understand is what is the magic formula between the number of threadpool_workers as configured here: # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 and those configured on the web servers and on the server handlers? Thanks --Rodolfo
Re: [galaxy-dev] Galaxy cannot start last [server:handlerxx]
On Aug 9, 2013, at 1:16 PM, Rodolfo Aramayo wrote: Dear Nate, Thanks Just to be sure I understand this correctly I now have in my 'universe' file the following: # # HTTP Server -- # Configuration of the internal HTTP server. [server:main] # job_manager = manager I ASSUME THIS IS CORRECT because I get errors when it is not commented [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 ...snip... [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # The internal HTTP server to use. Currently only Paste is provided. This # option is required. use = egg:Paste#http # The port on which to listen. #port = 8080 # The address on which to listen. By default, only listen to localhost (Galaxy # will not be accessible over the network). Use '0.0.0.0' to listen on all # available network interfaces. #host = 127.0.0.1 host = 0.0.0.0 # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 ...snip... [server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 # Filters -- # Filters sit between Galaxy and the HTTP server. ... # And in my job_conf.xml: handler id=handler0 tags=handlers/ handler id=handler1 tags=handlers/ ...snip... handler id=handler18 tags=handlers/ handler id=handler19 tags=handlers/ # Is this correct? Rodolfo, Some of this confusion might be because you have the (documented) options from the original [server:main] section orphaned underneath [server:handler19], which already has those options set. In an ini file, sections begin with [name] and all key/value pairs underneath that heading are assigned to [name] until another section heading is encountered. Removing all comments, that means your config would look like this: [server:main] [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 use = egg:Paste#http host = 0.0.0.0 use_threadpool = True threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 - Remove [server:main], the entire [server:manager] section, and those extra options under [server:handler19], and you should be good: [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true
[galaxy-dev] Galaxy cannot start last [server:handlerxx]
It worked like a charm Thanks --R On Fri, Aug 9, 2013 at 12:32 PM, Nate Coraor n...@bx.psu.edu wrote: On Aug 9, 2013, at 1:16 PM, Rodolfo Aramayo wrote: Dear Nate, Thanks Just to be sure I understand this correctly I now have in my 'universe' file the following: # # HTTP Server -- # Configuration of the internal HTTP server. [server:main] # job_manager = manager I ASSUME THIS IS CORRECT because I get errors when it is not commented [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 ...snip... [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # The internal HTTP server to use. Currently only Paste is provided. This # option is required. use = egg:Paste#http # The port on which to listen. #port = 8080 # The address on which to listen. By default, only listen to localhost (Galaxy # will not be accessible over the network). Use '0.0.0.0' to listen on all # available network interfaces. #host = 127.0.0.1 host = 0.0.0.0 # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 ...snip... [server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 # Filters -- # Filters sit between Galaxy and the HTTP server. ... # And in my job_conf.xml: handler id=handler0 tags=handlers/ handler id=handler1 tags=handlers/ ...snip... handler id=handler18 tags=handlers/ handler id=handler19 tags=handlers/ # Is this correct? Rodolfo, Some of this confusion might be because you have the (documented) options from the original [server:main] section orphaned underneath [server:handler19], which already has those options set. In an ini file, sections begin with [name] and all key/value pairs underneath that heading are assigned to [name] until another section heading is encountered. Removing all comments, that means your config would look like this: [server:main] [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 use = egg:Paste#http host = 0.0.0.0 use_threadpool = True threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 - Remove [server:main], the entire [server:manager] section, and those extra options under [server:handler19], and you should be good: [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler19] use = egg:Paste#http port = 8319 host =
Re: [galaxy-dev] Galaxy cannot start last [server:handlerxx]
On Aug 5, 2013, at 6:23 PM, Rodolfo Aramayo wrote: Hi, I have a 80 core/1TB RAM Dell server. In order improve Galaxy performance I modified the universe_wsgi.ini file so as to improve the Scaling and Load Balancing as follows: # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 ... snip ... [server:handler18] use = egg:Paste#http port = 8318 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler19] # use = egg:Paste#http # port = 8319 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 job_manager = manager job_handlers = handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18 Hi Rodolfo, The job_manager and job_handlers options go under the [server:main] section, it looks like they are probably underneath the last [server:...] section right now. Please note that all of these features should now be configured via the job configuration file (job_conf.xml), and I've just updated the documentation you referred to accordingly. --nate Now the problem I have is that when I start galaxy I cannot get the last [server:handler18] to load correclt I get the following error: galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'library_contents' API controller, class: LibraryContentsController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'group_users' API controller, class: GroupUsersAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'configuration' API controller, class: ConfigurationController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,237 Enabling 'requests' API controller, class: RequestsAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling 'folders' API controller, class: FoldersController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling 'quotas' API controller, class: QuotaAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling 'folder_contents' API controller, class: FolderContentsController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling 'roles' API controller, class: RoleAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling 'request_types' API controller, class: RequestTypeAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling 'users' API controller, class: UserAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling 'samples' API controller, class: SamplesAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling 'forms' API controller, class: FormDefinitionAPIController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling 'permissions' API controller, class: PermissionsController galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,242 Enabling 'workflows' API controller, class: WorkflowsAPIController galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,242 app.config.api_folders: False galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling 'httpexceptions' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling 'recursive' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,700 Enabling 'error' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling 'trans logger' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling 'x-forwarded-host' middleware galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling 'Request ID' middleware Starting server in PID 3899. Traceback (most recent call last): File ./scripts/paster.py, line 33, in module serve.run() File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1049, in run invoke(command, command_name, options, args[1:]) File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1055, in invoke exit_code = runner.run(args) File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 220, in run result = self.command() File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 670, in command serve() File
[galaxy-dev] Galaxy cannot start last [server:handlerxx]
Hi, I have a 80 core/1TB RAM Dell server. In order improve Galaxy performance I modified the universe_wsgi.ini file so as to improve the Scaling and Load Balancing as follows: # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 200 [server:web0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web2] use = egg:Paste#http port = 8092 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web3] use = egg:Paste#http port = 8093 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web4] use = egg:Paste#http port = 8094 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web5] use = egg:Paste#http port = 8095 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web6] use = egg:Paste#http port = 8096 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web7] use = egg:Paste#http port = 8097 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web8] use = egg:Paste#http port = 8098 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web9] use = egg:Paste#http port = 8099 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web10] use = egg:Paste#http port = 8100 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler2] use = egg:Paste#http port = 8302 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler3] use = egg:Paste#http port = 8303 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler4] use = egg:Paste#http port = 8304 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler5] use = egg:Paste#http port = 8305 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler6] use = egg:Paste#http port = 8306 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler7] use = egg:Paste#http port = 8307 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler8] use = egg:Paste#http port = 8308 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler9] use = egg:Paste#http port = 8309 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler10] use = egg:Paste#http port = 8310 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler11] use = egg:Paste#http port = 8311 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler12] use = egg:Paste#http port = 8312 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler13] use = egg:Paste#http port = 8313 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler14] use = egg:Paste#http port = 8314 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler15] use = egg:Paste#http port = 8315 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler16] use = egg:Paste#http port = 8316 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler17] use = egg:Paste#http port = 8317 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler18] use = egg:Paste#http port = 8318 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 # [server:handler19] # use = egg:Paste#http # port = 8319 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 # [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5 job_manager = manager job_handlers = handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18 Now the problem I have is that when I start galaxy I cannot get the last [server:handler18] to load correclt I get the following error: galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling 'library_contents' API controller, class:
