[galaxy-dev] Error in matplotlib
Could anyone shed some light on this error please? (Information from the Galaxy operator here who is having trouble posting) The error is for the dependency “package_matplotlib1_2” The full error information is Name: matplotlib Version: 1.2.1 Type: Package Status: Error Error: File "/home/little_galaxy/galaxy/lib/tool_shed/galaxy_install/install_manager.py", line 128, in install_and_build_package_via_fabric tool_dependency = self.install_and_build_package( install_environment, tool_dependency, actions_dict ) File "/home/little_galaxy/galaxy/lib/tool_shed/galaxy_install/install_manager.py", line 86, in install_and_build_package initial_download=True ) File "/home/little_galaxy/galaxy/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 31, in execute_step initial_download=initial_download ) File "/home/little_galaxy/galaxy/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 694, in execute_step dir = self.url_download( work_dir, downloaded_filename, url, extract=True, checksums=checksums ) File "/home/little_galaxy/galaxy/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 187, in url_download raise Exception( err_msg ) Error downloading from URL https://github.com/matplotlib/matplotlib/archive/v1.2.1.tar.gz#md5#47ee56f51200b8760b98d97ee86f57c6: HTTP Error 400: Bad Request ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Does anybody know if ts (a task scheduler) is supported under Galaxy.
Hi Hans-Rudolph, I am grateful to you (and all the other people who share their knowledge on this list) for pointing out a ready made, in-built solution to what I perceived as a problem. Many thanks, Shane >Hi Shane, > > >please allow me to jump in late - since you are looking for a simple >solution: why not rely just on the built in job scheduler? Have a look >at ~/config/job_conf.xml.sample_advanced > >Even on a single box, you can set up different queues with individual >restrictions. > >Hans-Rudolf > > > >On 11/24/2015 06:48 PM, Eric Rasche wrote: >> I'd also recommend HTCondor. >> Very easy to set up on debian/ubuntu, apt-get has a quick >> questionnaire that gets you up and running. >> >> Cheers, >> Eric >> >> On 11/24/2015 11:46 AM, Björn Grüning wrote: >>> Hi Shane, >>> >>> would Slurm work for you? >>> >>> http://slurm.schedmd.com/ >>> >>> Cheers, >>> Bjoern >>> >>> Am 24.11.2015 um 13:12 schrieb Shane Kelly: >>>> Hi, >>>> I am looking for a simple task scheduler to use with Galaxy >>>> and initially picked Torque, but since my server is a single box, >>>> Torque would appear to be overkill? >>>> >>>> Can I have any input from anyone who may have used it? >>>> >>>> Thanks for any response >>>> ___ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>>https://lists.galaxyproject.org/ >>>> >>>> To search Galaxy mailing lists use the unified search at: >>>>http://galaxyproject.org/search/mailinglists/ >>>> >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>>https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>>http://galaxyproject.org/search/mailinglists/ >>> >> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Does anybody know if ts (a task scheduler) is supported under Galaxy.
Hi, I am looking for a simple task scheduler to use with Galaxy and initially picked Torque, but since my server is a single box, Torque would appear to be overkill? Can I have any input from anyone who may have used it? Thanks for any response ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] I need some advice on what type of Galaxy server to implement
Hi Iyad, thanks for those links, and the sound advice. I'll read and absorb, and see what kind of a plan I can come up with. So good of you to take the time to answer, many thanks. Regards, Shane >What you should do (in my opinion) is install a grid scheduler (SGE, >torque, etc) on the big server. If you run Galaxy on a separate >server, it can be configured to submit jobs to the scheduler. Galaxy >also has the concept of a Web App and Handler components. >Essentially, handlers take care of talking with the scheduler while >the Web App will serve pages to users. By default, the Web App and >Handlers are combined in the same process. you can configure galaxy >to start up multiple handler processes and multiple web app >processes. Then, you can use Apache or Nginx to load balance user >requests between the various Galaxy Web Apps. > >My recommendation is that you start small to accommodate about 5 >concurrent users without noticeable performance issues: 1 Galaxy web >app 1 Galaxy handler > >Recommended reading: >https://wiki.galaxyproject.org/Admin/Config/Performance/Scaling >https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster > > >-Original Message- >From: Shane Kelly [mailto:s...@shanek54.co.uk] >Sent: July-29-15 7:32 AM >To: Kandalaft, Iyad >Cc: galaxy-dev@lists.galaxyproject.org >Subject: Re: [galaxy-dev] I need some advice on what type of Galaxy >server to implement > >Hi Iyad, > Thanks for taking the time to get back to me. >I am an IT guy, and would not know how to answer these questions >except to say that the server is to be used to facilitate biological >and medical research in the areas of genomics, transcriptomics, >epigenomics and metagenomics. (straight from the mission statement :-) >) > > I think that they would say that overall performance is less > important than being able to run large datasets (1/2-3TB). > > I like the idea of a separate box for the web-server, but I am > not sure how the web server would communicate with the > pipeline box - is that ability built into galaxy, or is it a > well worn path with plenty of examples that I could plagiarize? > > Sorry to be such a newb, but I don't know much about galaxy at > all. Luckily I have 2-3 months to put this in place... > >Again, thank you for your time. > >Regards, >Shane > > > > >>When you say NGS, is it genome assembly? If so, what type of genomes >>and do you have experience with its memory and cpu requirements. We >>noted that servers with large amount of memory and cores have a >>memory bus bottleneck. The other aspect is high processing on the >>server will impact the performance of Galaxy unless it is given >>higher priority. Note that if you overcommit the server, it can >>destabilize and bring down the Galaxy web app and database. >> >>My general approach is Galaxy web app + proxy on a separate machine >>from the handlers. The analysis server is either running a grid or >>the handlers. >> >>I recommend multiple smaller servers if you can get away with it as >>long as you have one that can accommodate your LARGE workloads. If >>you don't care about overall performance, large servers are the way >>to go as they are more "versatile". >> >>Regards, >> >>Iyad Kandalaft >> >>Acting Chief Bioinformatician in Biodiversity, STB Agriculture and >>Agri-Food Canada / Government of Canada iyad.kandal...@agr.gc.ca / >>Tel: 613-759-1228 / TTY: 613-773-2600 >> >>Bioinformaticien chef de la biodiversite interim, Direction générale >>des Science et de la technologie Agriculture et Agroalimentaire >>Canada / Gouvernement du Canada iyad.kandal...@agr.gc.ca / Tel: >>613-759-1228 / TTY: 613-773-2600 >> >> >> >> >>-Original Message- >>From: galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] >>On Behalf Of Shane Kelly Sent: July-28-15 8:23 AM >>To: galaxy-dev@lists.galaxyproject.org >>Subject: [galaxy-dev] I need some advice on what type of Galaxy >>server to implement >> >>Hi >> I have been tasked with getting a Galaxy server up and running >> for a group at work. >> >> 1. No-one can tell me how many users (concurrent or otherwise) >> there will be 2. Most of the analyses will be NGS. >> 3. Tools will be developed in-house but we will use public >> domain tools also. 4. There will be a guy running the >> server/developing tools pretty much full time. >> >> I have two favoured solutions at the m
Re: [galaxy-dev] I need some advice on what type of Galaxy server to implement
Hi Iyad, Thanks for taking the time to get back to me. I am an IT guy, and would not know how to answer these questions except to say that the server is to be used to facilitate biological and medical research in the areas of genomics, transcriptomics, epigenomics and metagenomics. (straight from the mission statement :-) ) I think that they would say that overall performance is less important than being able to run large datasets (1/2-3TB). I like the idea of a separate box for the web-server, but I am not sure how the web server would communicate with the pipeline box - is that ability built into galaxy, or is it a well worn path with plenty of examples that I could plagiarize? Sorry to be such a newb, but I don't know much about galaxy at all. Luckily I have 2-3 months to put this in place... Again, thank you for your time. Regards, Shane >When you say NGS, is it genome assembly? If so, what type of genomes >and do you have experience with its memory and cpu requirements. We >noted that servers with large amount of memory and cores have a memory >bus bottleneck. The other aspect is high processing on the server will >impact the performance of Galaxy unless it is given higher priority. >Note that if you overcommit the server, it can destabilize and bring >down the Galaxy web app and database. > >My general approach is Galaxy web app + proxy on a separate machine >from the handlers. The analysis server is either running a grid or >the handlers. > >I recommend multiple smaller servers if you can get away with it as >long as you have one that can accommodate your LARGE workloads. If >you don't care about overall performance, large servers are the way to >go as they are more "versatile". > >Regards, > >Iyad Kandalaft > >Acting Chief Bioinformatician in Biodiversity, STB >Agriculture and Agri-Food Canada / Government of Canada >iyad.kandal...@agr.gc.ca / Tel: 613-759-1228 / TTY: 613-773-2600 > >Bioinformaticien chef de la biodiversite interim, Direction générale >des Science et de la technologie Agriculture et Agroalimentaire >Canada / Gouvernement du Canada iyad.kandal...@agr.gc.ca / Tel: >613-759-1228 / TTY: 613-773-2600 > > > > >-Original Message- >From: galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] >On Behalf Of Shane Kelly Sent: July-28-15 8:23 AM >To: galaxy-dev@lists.galaxyproject.org >Subject: [galaxy-dev] I need some advice on what type of Galaxy server >to implement > >Hi > I have been tasked with getting a Galaxy server up and running > for a group at work. > > 1. No-one can tell me how many users (concurrent or otherwise) > there will be 2. Most of the analyses will be NGS. > 3. Tools will be developed in-house but we will use public > domain tools also. 4. There will be a guy running the > server/developing tools pretty much full time. > > I have two favoured solutions at the moment: > >1. A pipeline processor ( 64 Core, 512G Ram, with DAS of about 150TB >), and a Web server to act as frontend and database server, and >another, smaller box for a total install of galaxy, but doing only the >development work. > >2. An all-in-one server with 128 Cores, 1TB ram, DAS storage of 150TB, >and development work done on a VM. > >Any input would be hepfull. > >Thanks >Shane >___ >Please keep all replies on the list by using "reply all" >in your mail client. To manage your subscriptions to this and other >Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > >To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >___ >Please keep all replies on the list by using "reply all" >in your mail client. To manage your subscriptions to this >and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > >To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] I need some advice on what type of Galaxy server to implement
Hi I have been tasked with getting a Galaxy server up and running for a group at work. 1. No-one can tell me how many users (concurrent or otherwise) there will be 2. Most of the analyses will be NGS. 3. Tools will be developed in-house but we will use public domain tools also. 4. There will be a guy running the server/developing tools pretty much full time. I have two favoured solutions at the moment: 1. A pipeline processor ( 64 Core, 512G Ram, with DAS of about 150TB ), and a Web server to act as frontend and database server, and another, smaller box for a total install of galaxy, but doing only the development work. 2. An all-in-one server with 128 Cores, 1TB ram, DAS storage of 150TB, and development work done on a VM. Any input would be hepfull. Thanks Shane ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
