Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
So I finally figured it out (thanks to the help from the IRC folks, bag and natefoo). I incorrectly had my job_conf.xml setup where no job runners were set. All is working now. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 14, 2013 at 1:54 PM, Adam Brenner wrote: > Anyone have any ideas? Still have not sorted this out > > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Thu, Apr 11, 2013 at 2:24 PM, Adam Brenner wrote: > >> FYI: Here are the entries in paster.log. If you notice no job_handlers >> are being sent. I performed tests on both nginx reverse proxy and direct >> localhost:8080 (limited to our private network on campus) >> >> 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] "GET >> /tool_runner?tool_id=upload1 HTTP/1.0" 200 - " >> http://ghtf-hpc.oit.uci.edu/root/tool_menu"; "Mozilla/5.0 (Macintosh; >> Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) >> Version/6.0.3 Safari/536.28.10" >> galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool >> upload1 created job id 8 >> 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] "POST /tool_runner/index >> HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1"; >> "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 >> (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" >> 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /history HTTP/1.0" >> 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner/index"; "Mozilla/5.0 >> (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like >> Gecko) Version/6.0.3 Safari/536.28.10" >> 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET >> /api/histories/56959efe847bb360 HTTP/1.0" 200 - " >> http://ghtf-hpc.oit.uci.edu/history"; "Mozilla/5.0 (Macintosh; Intel Mac >> OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 >> Safari/536.28.10" >> >> >> -- >> Adam Brenner >> Computer Science, Undergraduate Student >> Donald Bren School of Information and Computer Sciences >> >> Research Computing Support >> Office of Information Technology >> http://www.oit.uci.edu/rcs/ >> >> University of California, Irvine >> www.ics.uci.edu/~aebrenne/ >> aebre...@uci.edu >> >> >> On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner wrote: >> >>> Howdy Galaxy Admins, >>> >>> I am installing Galaxy on our HPC cluster and running into a strange >>> issue that I can no longer debug further. I am running the latest galaxy >>> build as of Tuesday April 9th. >>> hg clone https://bitbucket.org/galaxy/galaxy-dist/ >>> hg update stable >>> >>> When a user tries to upload a file via the web interface, nginx reverse >>> proxy setup as described here >>> http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, >>> the upload stays in a state of "Job is waiting to run." >>> >>> The job stays like this for both FTP uploads, and uploads via browser. >>> The dataset is a 319 bytes (both FTP and upload via browser) >>> >>> My universe_wsgi.ini variables are like so: >>> .. under [app:main] >>> new_file_path = /som/galaxy/tmp >>> file_path = /som/galaxy/datasets/files >>> track_jobs_in_database = True >>> job_manager = manager >>> job_handlers = handler0,handler1 >>> ... before [app:main] in [server:main] >>> >>> [server:main] >>> use = egg:Paste#http >>> port = 8080 >>> host = 127.0.0.1 >>> use_threadpool = true >>> threadpool_workers = 5 >>> >>> [server:manager] >>> use = egg:Paste#http >>> port = 8079 >>> host = 127.0.0.1 >>> use_threadpool = true >>> threadpool_workers = 5 >>> >>> [server:handler0] >>> use = egg:Paste#http >>> port = 8090 >>> host = 127.0.0.1 >>> use_threadpool = true >>> threadpool_workers = 5 >>> >>> [server:handler1] >>> use = egg:Paste#http >>> port = 8091 >>> host = 127.0.0.1 >>> use_threadpool = true >>> threadpool_workers = 5 >>> >>> >>> Galaxy, for now, is running under the root user and is installed in >>> /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS >>> system under /som/galaxy/. >>> >>> I have tried removing the extra job managers and handlers so I just have >>> server:main and this still continues to happen. The paster.log files (with >>> DEBUG = True) does not show any errors from python or issues writing to >>> file. All POST and GET responses are recorded and nothing else. >>> >>> What I find strange is that if I look in ne
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
Anyone have any ideas? Still have not sorted this out -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 2:24 PM, Adam Brenner wrote: > FYI: Here are the entries in paster.log. If you notice no job_handlers are > being sent. I performed tests on both nginx reverse proxy and direct > localhost:8080 (limited to our private network on campus) > > 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] "GET > /tool_runner?tool_id=upload1 HTTP/1.0" 200 - " > http://ghtf-hpc.oit.uci.edu/root/tool_menu"; "Mozilla/5.0 (Macintosh; > Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) > Version/6.0.3 Safari/536.28.10" > galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool > upload1 created job id 8 > 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] "POST /tool_runner/index > HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1"; > "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 > (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" > 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /history HTTP/1.0" > 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner/index"; "Mozilla/5.0 > (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like > Gecko) Version/6.0.3 Safari/536.28.10" > 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET > /api/histories/56959efe847bb360 HTTP/1.0" 200 - " > http://ghtf-hpc.oit.uci.edu/history"; "Mozilla/5.0 (Macintosh; Intel Mac > OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 > Safari/536.28.10" > > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > > > On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner wrote: > >> Howdy Galaxy Admins, >> >> I am installing Galaxy on our HPC cluster and running into a strange >> issue that I can no longer debug further. I am running the latest galaxy >> build as of Tuesday April 9th. >> hg clone https://bitbucket.org/galaxy/galaxy-dist/ >> hg update stable >> >> When a user tries to upload a file via the web interface, nginx reverse >> proxy setup as described here >> http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, >> the upload stays in a state of "Job is waiting to run." >> >> The job stays like this for both FTP uploads, and uploads via browser. >> The dataset is a 319 bytes (both FTP and upload via browser) >> >> My universe_wsgi.ini variables are like so: >> .. under [app:main] >> new_file_path = /som/galaxy/tmp >> file_path = /som/galaxy/datasets/files >> track_jobs_in_database = True >> job_manager = manager >> job_handlers = handler0,handler1 >> ... before [app:main] in [server:main] >> >> [server:main] >> use = egg:Paste#http >> port = 8080 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> [server:manager] >> use = egg:Paste#http >> port = 8079 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> [server:handler0] >> use = egg:Paste#http >> port = 8090 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> [server:handler1] >> use = egg:Paste#http >> port = 8091 >> host = 127.0.0.1 >> use_threadpool = true >> threadpool_workers = 5 >> >> >> Galaxy, for now, is running under the root user and is installed in >> /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS >> system under /som/galaxy/. >> >> I have tried removing the extra job managers and handlers so I just have >> server:main and this still continues to happen. The paster.log files (with >> DEBUG = True) does not show any errors from python or issues writing to >> file. All POST and GET responses are recorded and nothing else. >> >> What I find strange is that if I look in new_file_path I do see my >> dataset as >> upload_file_data_Zx_1rK >> and cat'ing the file shows the correct contents. The items listed in >> my file_path (the dataset_*.dat) files are empty/blank..but are created. >> >> Any ideas as to why its failing? Is it nginx? I only have it redirect >> 8080, and none of the other job managers / handlers...is that an issue? Do >> you think its permissions on the my file_path or new_file_path directories? >> >> A very stumped Adam. >> >> -- >> Adam Brenner >> Computer Science, Undergraduate Student >> Donald Bren School of Information and Computer Sciences >> >> Research Computing Support >> Office of
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
FYI: Here are the entries in paster.log. If you notice no job_handlers are being sent. I performed tests on both nginx reverse proxy and direct localhost:8080 (limited to our private network on campus) 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.0" 200 - " http://ghtf-hpc.oit.uci.edu/root/tool_menu"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool upload1 created job id 8 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] "POST /tool_runner/index HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /history HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner/index"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /api/histories/56959efe847bb360 HTTP/1.0" 200 - " http://ghtf-hpc.oit.uci.edu/history"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner wrote: > Howdy Galaxy Admins, > > I am installing Galaxy on our HPC cluster and running into a strange issue > that I can no longer debug further. I am running the latest galaxy build as > of Tuesday April 9th. > hg clone https://bitbucket.org/galaxy/galaxy-dist/ > hg update stable > > When a user tries to upload a file via the web interface, nginx reverse > proxy setup as described here > http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the > upload stays in a state of "Job is waiting to run." > > The job stays like this for both FTP uploads, and uploads via browser. The > dataset is a 319 bytes (both FTP and upload via browser) > > My universe_wsgi.ini variables are like so: > .. under [app:main] > new_file_path = /som/galaxy/tmp > file_path = /som/galaxy/datasets/files > track_jobs_in_database = True > job_manager = manager > job_handlers = handler0,handler1 > ... before [app:main] in [server:main] > > [server:main] > use = egg:Paste#http > port = 8080 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:manager] > use = egg:Paste#http > port = 8079 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler0] > use = egg:Paste#http > port = 8090 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > [server:handler1] > use = egg:Paste#http > port = 8091 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > > Galaxy, for now, is running under the root user and is installed in > /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS > system under /som/galaxy/. > > I have tried removing the extra job managers and handlers so I just have > server:main and this still continues to happen. The paster.log files (with > DEBUG = True) does not show any errors from python or issues writing to > file. All POST and GET responses are recorded and nothing else. > > What I find strange is that if I look in new_file_path I do see my dataset > as > upload_file_data_Zx_1rK > and cat'ing the file shows the correct contents. The items listed in > my file_path (the dataset_*.dat) files are empty/blank..but are created. > > Any ideas as to why its failing? Is it nginx? I only have it redirect > 8080, and none of the other job managers / handlers...is that an issue? Do > you think its permissions on the my file_path or new_file_path directories? > > A very stumped Adam. > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
Howdy Galaxy Admins, I am installing Galaxy on our HPC cluster and running into a strange issue that I can no longer debug further. I am running the latest galaxy build as of Tuesday April 9th. hg clone https://bitbucket.org/galaxy/galaxy-dist/ hg update stable When a user tries to upload a file via the web interface, nginx reverse proxy setup as described here http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the upload stays in a state of "Job is waiting to run." The job stays like this for both FTP uploads, and uploads via browser. The dataset is a 319 bytes (both FTP and upload via browser) My universe_wsgi.ini variables are like so: .. under [app:main] new_file_path = /som/galaxy/tmp file_path = /som/galaxy/datasets/files track_jobs_in_database = True job_manager = manager job_handlers = handler0,handler1 ... before [app:main] in [server:main] [server:main] use = egg:Paste#http port = 8080 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Galaxy, for now, is running under the root user and is installed in /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS system under /som/galaxy/. I have tried removing the extra job managers and handlers so I just have server:main and this still continues to happen. The paster.log files (with DEBUG = True) does not show any errors from python or issues writing to file. All POST and GET responses are recorded and nothing else. What I find strange is that if I look in new_file_path I do see my dataset as upload_file_data_Zx_1rK and cat'ing the file shows the correct contents. The items listed in my file_path (the dataset_*.dat) files are empty/blank..but are created. Any ideas as to why its failing? Is it nginx? I only have it redirect 8080, and none of the other job managers / handlers...is that an issue? Do you think its permissions on the my file_path or new_file_path directories? A very stumped Adam. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
