Re: [galaxy-dev] Galaxy cannot start last [server:handlerxx]

[Nate Coraor]

On Aug 9, 2013, at 1:16 PM, Rodolfo Aramayo wrote:

 Dear Nate,
 
 Thanks
 
 Just to be sure I understand this correctly
 
 I now have in my 'universe' file the following:
 
 #
 #  HTTP Server
 --
 
 # Configuration of the internal HTTP server.
 
 [server:main]
 
 # job_manager = manager  I ASSUME THIS IS CORRECT because I get
 errors when it is not  commented
 
 
 
 [server:manager]
 use = egg:Paste#http
 port = 8200
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5
 
 
 
 [server:handler0]
 use = egg:Paste#http
 port = 8300
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5
 
 [server:handler1]
 use = egg:Paste#http
 port = 8301
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5
 
 ...snip...
 
 [server:handler19]
 use = egg:Paste#http
 port = 8319
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5
 
 # [server:handler20]
 # use = egg:Paste#http
 # port = 8320
 # host = 127.0.0.1
 # use_threadpool = true
 # threadpool_workers = 5
 
 
 # The internal HTTP server to use.  Currently only Paste is provided.
 This
 # option is required.
 use = egg:Paste#http
 
 # The port on which to listen.
 #port = 8080
 
 # The address on which to listen.  By default, only listen to
 localhost (Galaxy
 # will not be accessible over the network).  Use '0.0.0.0' to listen
 on all
 # available network interfaces.
 #host = 127.0.0.1
 host = 0.0.0.0
 
 # Use a threadpool for the web server instead of creating a thread for
 each
 # request.
 use_threadpool = True
 
 # Number of threads in the web server thread pool.
 threadpool_workers = 200
 
 
 [server:web0]
 use = egg:Paste#http
 port = 8400
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 10
 
 [server:web1]
 use = egg:Paste#http
 port = 8401
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 10
 
 ...snip...
 
 [server:web11]
 use = egg:Paste#http
 port = 8411
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 10
 
 [server:web12]
 use = egg:Paste#http
 port = 8412
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 10
 
 
 #  Filters --
 
 # Filters sit between Galaxy and the HTTP server.
 ...
 
 #
 
 And in my job_conf.xml:
 
handler id=handler0 tags=handlers/
handler id=handler1 tags=handlers/
 ...snip...
   handler id=handler18 tags=handlers/
handler id=handler19 tags=handlers/
 
 #
 
 Is this correct?

Rodolfo,

Some of this confusion might be because you have the (documented) options from 
the original [server:main] section orphaned underneath [server:handler19], 
which already has those options set.  In an ini file, sections begin with 
[name] and all key/value pairs underneath that heading are assigned to [name] 
until another section heading is encountered.  Removing all comments, that 
means your config would look like this:

[server:main]

[server:manager]
use = egg:Paste#http
port = 8200
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5

[server:handler0]
use = egg:Paste#http
port = 8300
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5

[server:handler1]
use = egg:Paste#http
port = 8301
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5

[server:handler19]
use = egg:Paste#http
port = 8319
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5
use = egg:Paste#http
host = 0.0.0.0
use_threadpool = True
threadpool_workers = 200

[server:web0]
use = egg:Paste#http
port = 8400
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10

[server:web1]
use = egg:Paste#http
port = 8401
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10

[server:web11]
use = egg:Paste#http
port = 8411
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10

[server:web12]
use = egg:Paste#http
port = 8412
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 10

-

Remove [server:main], the entire [server:manager] section, and those extra 
options under [server:handler19], and you should be good:

[server:handler0]
use = egg:Paste#http
port = 8300
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5

[server:handler1]
use = egg:Paste#http
port = 8301
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5

[server:handler19]
use = egg:Paste#http
port = 8319
host = 127.0.0.1
use_threadpool = true
threadpool_workers = 5

[server:web0]
use = egg:Paste#http
port = 8400
host = 127.0.0.1
use_threadpool = true

Re: [galaxy-dev] Galaxy cannot start last [server:handlerxx]

[Nate Coraor]

On Aug 5, 2013, at 6:23 PM, Rodolfo Aramayo wrote:

 Hi,
 
 I have a 80 core/1TB RAM Dell server.
 In order improve Galaxy performance I modified the universe_wsgi.ini
 file so as to improve the Scaling and Load Balancing
 as follows:
 
 # Use a threadpool for the web server instead of creating a thread for each
 # request.
 use_threadpool = True
 
 # Number of threads in the web server thread pool.
 threadpool_workers = 200
 
 
 [server:web0]
 use = egg:Paste#http
 port = 8090
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 10

... snip ...

 [server:handler18]
 use = egg:Paste#http
 port = 8318
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5
 
 # [server:handler19]
 # use = egg:Paste#http
 # port = 8319
 # host = 127.0.0.1
 # use_threadpool = true
 # threadpool_workers = 5
 
 # [server:handler20]
 # use = egg:Paste#http
 # port = 8320
 # host = 127.0.0.1
 # use_threadpool = true
 # threadpool_workers = 5
 
 
 job_manager = manager
 job_handlers = 
 handler0,handler1,,handler2,handler3,handler4,handler5,handler6,handler7,handler8,handler9,handler10,handler11,handler12,handler13,handler14,handler15,handler16,handler17,handler18

Hi Rodolfo,

The job_manager and job_handlers options go under the [server:main] section, it 
looks like they are probably underneath the last [server:...] section right now.

Please note that all of these features should now be configured via the job 
configuration file (job_conf.xml), and I've just updated the documentation you 
referred to accordingly.

--nate

 
 
 
 Now the problem I have is that when I start galaxy I cannot get the
 last [server:handler18] to load correclt
 I get the following error:
 
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling
 'library_contents' API controller, class: LibraryContentsController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling
 'group_users' API controller, class: GroupUsersAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,236 Enabling
 'configuration' API controller, class: ConfigurationController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,237 Enabling
 'requests' API controller, class: RequestsAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling
 'folders' API controller, class: FoldersController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,238 Enabling
 'quotas' API controller, class: QuotaAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling
 'folder_contents' API controller, class: FolderContentsController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,239 Enabling
 'roles' API controller, class: RoleAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling
 'request_types' API controller, class: RequestTypeAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,240 Enabling
 'users' API controller, class: UserAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling
 'samples' API controller, class: SamplesAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling
 'forms' API controller, class: FormDefinitionAPIController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,241 Enabling
 'permissions' API controller, class: PermissionsController
 galaxy.web.framework.base DEBUG 2013-08-05 17:08:24,242 Enabling
 'workflows' API controller, class: WorkflowsAPIController
 galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,242
 app.config.api_folders: False
 galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling
 'httpexceptions' middleware
 galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,450 Enabling
 'recursive' middleware
 galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,700 Enabling
 'error' middleware
 galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling
 'trans logger' middleware
 galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling
 'x-forwarded-host' middleware
 galaxy.webapps.galaxy.buildapp DEBUG 2013-08-05 17:08:24,701 Enabling
 'Request ID' middleware
 Starting server in PID 3899.
 Traceback (most recent call last):
  File ./scripts/paster.py, line 33, in module
serve.run()
  File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py,
 line 1049, in run
invoke(command, command_name, options, args[1:])
  File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py,
 line 1055, in invoke
exit_code = runner.run(args)
  File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py,
 line 220, in run
result = self.command()
  File /usr/local/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py,
 line 670, in command
serve()
  File