[galaxy-dev] [GSoC 2013] Introduction
Hi Everyone! I have been selected to work on the eQTL pipeline and visualisation for the Galaxy Project as part of the Google Summer of Code 2013 ( http://www.google-melange.com/gsoc/homepage/google/gsoc2013) I am pursuing my fourth year of undergraduate studies at Indian Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of interest are broadly Bioinformatics and of lately I have been working on NGS problems. I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . The first post should be up soon. My proposal is here : http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001. I would love to know your suggestions/comments. I am relatively new to Galaxy, and some of my questions in the process might be elementary, so kindly bear with it :) Thanks Saket Choudhary ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [GSoC 2013] Introduction
Welcome Saket! Interesting topic ... wish you all the best and don't hesitate to ask. There is also a galaxyproject channel on freenode. Cheers, Björn Hi Everyone! I have been selected to work on the eQTL pipeline and visualisation for the Galaxy Project as part of the Google Summer of Code 2013 (http://www.google-melange.com/gsoc/homepage/google/gsoc2013) I am pursuing my fourth year of undergraduate studies at Indian Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of interest are broadly Bioinformatics and of lately I have been working on NGS problems. I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . The first post should be up soon. My proposal is here : http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001 . I would love to know your suggestions/comments. I am relatively new to Galaxy, and some of my questions in the process might be elementary, so kindly bear with it :) Thanks Saket Choudhary ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [GSoC 2013] Introduction
Welcome Saket! Interesting topic ... wish you all the best and don't hesitate to ask. There is also a #galaxyproject channel on freenode. Cheers, Björn Hi Everyone! I have been selected to work on the eQTL pipeline and visualisation for the Galaxy Project as part of the Google Summer of Code 2013 (http://www.google-melange.com/gsoc/homepage/google/gsoc2013) I am pursuing my fourth year of undergraduate studies at Indian Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of interest are broadly Bioinformatics and of lately I have been working on NGS problems. I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . The first post should be up soon. My proposal is here : http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001 . I would love to know your suggestions/comments. I am relatively new to Galaxy, and some of my questions in the process might be elementary, so kindly bear with it :) Thanks Saket Choudhary ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [GSoC 2013] Introduction
Hi, Saket. Welcome aboard - help is always available here or irc if you're in the right timezone. https://chgr.mc.vanderbilt.edu/plato/https://chgr.mc.vanderbilt.edu/plato/steps/3 is a brand new eQTL option from Marylyn Ritchie's group I stumbled across at http://gettinggeneticsdone.blogspot.com.au/ today - the comments about plink are apt - it can be a pain to work with in my experience. On Fri, May 31, 2013 at 4:35 PM, Saket Choudhary sake...@gmail.com wrote: Hi Everyone! I have been selected to work on the eQTL pipeline and visualisation for the Galaxy Project as part of the Google Summer of Code 2013 ( http://www.google-melange.com/gsoc/homepage/google/gsoc2013) I am pursuing my fourth year of undergraduate studies at Indian Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of interest are broadly Bioinformatics and of lately I have been working on NGS problems. I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . The first post should be up soon. My proposal is here : http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001. I would love to know your suggestions/comments. I am relatively new to Galaxy, and some of my questions in the process might be elementary, so kindly bear with it :) Thanks ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Internal Server Error during installation of ncbi_blast_plus
Hi Greg,
thanks for your comment. Some days ago, I updated Galaxy following:
http://wiki.galaxyproject.org/News/2013_04_08_Galaxy_Security_Release
Here is what hg sum said
parent: 9292:2cc8d10988e0 security_2013.04.08
controllers/history: use get_history in switch_to_history
Zweig: stable
Übernehme: 10 unbekannt (clean)
Aktualisiere: 11 neue Änderungssätze (Aktualisierung)
(Sorry for the German terms.) I then updated Galaxy by
hg update
obtaining:
parent: 9320:47ddf167c9f1 tip
Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response.
Fixes downloading tarballs from the Tool Shed when use_debug = false.
Zweig: stable
Übernehme: 10 unbekannt (clean)
Aktualisiere: (aktuell)
I then restarted Galaxy and tried to install again, but it fails again. Even
trying for a second time doesn't change the behaviour. Here is what paster.log
says:
172.31.30.183 - - [31/May/2013:07:47:32 +0200] GET
/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=1d92ebdf7e8d466cchangeset_revisions=9dabbfd73c8a
HTTP/1.0 500 -
http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?repository_id=1d92ebdf7e8d466cchangeset_revision=9dabbfd73c8a;
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0
Error - type 'exceptions.ValueError': too many values to unpack
URL:
http://galaxy.jki.bund.de/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=1d92ebdf7e8d466cchangeset_revisions=9dabbfd73c8a
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py',
line 149 in __call__
app_iter = self.application(environ, sr_checker)
File '/opt/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py',
line 84 in __call__
return self.application(environ, start_response)
File
'/opt/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
line 633 in __call__
return self.application(environ, start_response)
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 128 in
__call__
return self.handle_request( environ, start_response )
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 184 in
handle_request
body = method( trans, **kwargs )
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 231
in decorator
return func( self, trans, *args, **kwargs )
File
'/opt/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py',
line 894 in prepare_for_install
common_install_util.get_dependencies_for_repository( trans, tool_shed_url,
repo_info_dict, includes_tool_dependencies )
File '/opt/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line
88 in get_dependencies_for_repository
installed_rd, missing_rd =
get_installed_and_missing_repository_dependencies_for_new_install( trans,
repo_info_tuple )
File '/opt/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line
192 in get_installed_and_missing_repository_dependencies_for_new_install
tool_shed, name, owner, changeset_revision = rd_tup
ValueError: too many values to unpack
CGI Variables
-
CONTENT_LENGTH: '0'
HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
HTTP_ACCEPT_ENCODING: 'gzip, deflate'
HTTP_ACCEPT_LANGUAGE: 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3'
HTTP_CACHE_CONTROL: 'max-age=0'
HTTP_CONNECTION: 'keep-alive'
HTTP_COOKIE:
'galaxysession=c6ca0ddb55be603a88fa41dbba7680819dc3ac10c92a6efa6da917af7060700e4360e4afb5987baa'
HTTP_HOST: 'galaxy.jki.bund.de'
HTTP_REFERER:
'http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?repository_id=1d92ebdf7e8d466cchangeset_revision=9dabbfd73c8a'
HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101
Firefox/21.0'
HTTP_VIA: '1.1 proxy.jki.bund.de:8000 (squid/2.7.STABLE9)'
HTTP_X_FORWARDED_FOR: '172.31.30.183'
ORGINAL_REMOTE_ADDR: '172.16.20.52'
PATH_INFO: '/admin_toolshed/prepare_for_install'
QUERY_STRING:
'tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=1d92ebdf7e8d466cchangeset_revisions=9dabbfd73c8a'
REMOTE_ADDR: '172.31.30.183'
REQUEST_METHOD: 'GET'
SERVER_NAME: '0.0.0.0'
SERVER_PORT: '8080'
SERVER_PROTOCOL: 'HTTP/1.0'
WSGI Variables
--
application: paste.recursive.RecursiveMiddleware object at 0x7c3ed50
paste.cookies: (SimpleCookie:
galaxysession='c6ca0ddb55be603a88fa41dbba7680819dc3ac10c92a6efa6da917af7060700e4360e4afb5987baa',
'galaxysession=c6ca0ddb55be603a88fa41dbba7680819dc3ac10c92a6efa6da917af7060700e4360e4afb5987baa')
paste.expected_exceptions: [class 'paste.httpexceptions.HTTPException']
paste.httpexceptions: paste.httpexceptions.HTTPExceptionHandler object at
0x7c3e710
paste.httpserver.thread_pool: paste.httpserver.ThreadPool object at
0x7c9ff50
paste.parsed_querystring: ([('tool_shed_url',
'http://toolshed.g2.bx.psu.edu/'), ('repository_ids', '1d92ebdf7e8d466c'),
('changeset_revisions',
[galaxy-dev] Status of Galaxy Cloud Instance dev
Hi all, I am planning to launch a Galaxy Cloud Instance, to run for about one month. But I believe I've read somewhere a new image of Galaxy on AWS is being prepared right now. Can somebody update me on the status? Perhaps it is smart to wait a few days with launching through CloudLaunch until the new release is there? Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Status of Galaxy Cloud Instance dev
No firm release set for the new cloudman yet, but you can always follow development here: https://trello.com/board/galaxy-cloudman/5136209e9beefacb77004e6d There are a few final issues we're wrapping up before we can release, but we're hoping it'll be soon. On Fri, May 31, 2013 at 7:31 AM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Hi all, I am planning to launch a Galaxy Cloud Instance, to run for about one month. But I believe I've read somewhere a new image of Galaxy on AWS is being prepared right now. Can somebody update me on the status? Perhaps it is smart to wait a few days with launching through CloudLaunch until the new release is there? Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.**php/about/80-teamhttp://www.bits.vib.be/index.php/about/80-team __**_ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/**search/mailinglists/http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Status of Galaxy Cloud Instance dev
Hey Dannon, Thanks for the link, I saw that you had moved some cards to a CloudMan board but I was never able to find it. Could you enable comments for non-board members like the Galaxy Development Trello allows? You know I have a compulsion to comment on everything Galaxy related anywhere on the Internet. -John On Fri, May 31, 2013 at 8:34 AM, Dannon Baker dannon.ba...@gmail.com wrote: No firm release set for the new cloudman yet, but you can always follow development here: https://trello.com/board/galaxy-cloudman/5136209e9beefacb77004e6d There are a few final issues we're wrapping up before we can release, but we're hoping it'll be soon. On Fri, May 31, 2013 at 7:31 AM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Hi all, I am planning to launch a Galaxy Cloud Instance, to run for about one month. But I believe I've read somewhere a new image of Galaxy on AWS is being prepared right now. Can somebody update me on the status? Perhaps it is smart to wait a few days with launching through CloudLaunch until the new release is there? Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [GSoC 2013] Introduction
Cool! :) Feel free to join the Google+ group as well, and link your blog posts there, to notify users (and make it easy for us to share further): http://gplus.to/galaxyportal Cheers // Samuel On 05/31/2013 08:35 AM, Saket Choudhary wrote: Hi Everyone! I have been selected to work on the eQTL pipeline and visualisation for the Galaxy Project as part of the Google Summer of Code 2013 (http://www.google-melange.com/gsoc/homepage/google/gsoc2013) I am pursuing my fourth year of undergraduate studies at Indian Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of interest are broadly Bioinformatics and of lately I have been working on NGS problems. I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . The first post should be up soon. snip ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Status of Galaxy Cloud Instance dev
On Fri, May 31, 2013 at 9:51 AM, John Chilton chil...@msi.umn.edu wrote: Could you enable comments for non-board members like the Galaxy Development Trello allows? Of course, actually thought I'd already done that. Fixed! ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Multi File upload api
Hi guys, I am sure I will be looking into this shortly but now I want to get the one file case working first :). Do you guys have any idea why the example_watch_folder.pl will copy the file but not its contents? The file it copies into the library folder is empty. How can I go about resolving this issue? I appreciate any help. Regards, -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of alex.khassa...@csiro.au Sent: Friday, May 31, 2013 1:22 AM To: chil...@msi.umn.edu Cc: galaxy-...@bx.psu.edu; neil.burd...@csiro.au Subject: Re: [galaxy-dev] Multi File upload api Thanks John, We do have pretty old code, so I applied your patches. One minor problem, in multi_upload tool Dataset Name is ignored, just picks the first file name. Do we need to update upoald.py or multi_upload.xml? -Alex -Original Message- From: jmchil...@gmail.com [mailto:jmchil...@gmail.com] On Behalf Of John Chilton Sent: Friday, 31 May 2013 5:50 AM To: Khassapov, Alex (CSIRO IMT, Clayton) Cc: Burdett, Neil (ICT Centre, Herston - RBWH); galaxy-...@bx.psu.edu Subject: Re: Multi File upload api Alex, if I am not mistaken you guys are using a fairly old version of the multiple file dataset code. I think on the server side you would need to apply the following three changesets to update the library backend to support the creation of multifile datasets (I guess if you are doing path pasting only the first changeset is really needed): https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=25e137130d32at=stable https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=a15ce4c00716at=stable https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=d2ae2265bc96at=stable If your version of Galaxy is likewise pretty old you will need pull request 123 as first (https://bitbucket.org/galaxy/galaxy-central/pull-request/123/refactor-library_commonpy-toward-more/diff). Once those are in place updating example_watch_folder.py should be pretty doable (Hagai Cohen had it working at some point): 1) You will need to replace: data['upload_option'] = 'upload_paths' with data['upload_option'] = 'upload_paths_multifile' 2) You will need to specify a name (e.g. add something like: data['NAME'] = os.path.basename(fullpath)) 3) Multiple file datasets do not support data['ext'] = 'auto', so you will need to specify an explicit format there (the script will hopefully be able to determine this from the inputs). The most updated version of the multiple file dataset stuff https://bitbucket.org/msiappdev/galaxy-extras/ paired with the most updated version of blend4j (https://github.com/jmchilton/blend4j) are compatible and if you do enough digging can provide insights on how to create multiple file datasets via the API (library path pastes, library server directory uploads, and history uploads). Hope this helps, -John On Thu, May 30, 2013 at 1:40 AM, alex.khassa...@csiro.au wrote: Hi John, Can you please have a look at Neil's question. Thank you, -Alex From: Burdett, Neil (ICT Centre, Herston - RBWH) Sent: Thursday, 30 May 2013 4:30 PM To: Khassapov, Alex (CSIRO IMT, Clayton) Subject: Multi File upload api Hi Alex, The file galaxy-dist/scripts/api/example_watch_folder.py allows us to watch a folder then upload files that arrive in a specified input folder into the database and history. Can you ask your friend (who implemented the multi file upload tool), what changes are necessary to this file so we can upload multiple files as we do from the GUI. I assume he would know quite quickly what to do and hopefully quite simple Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] June 2013 Galaxy Update
Hello all, The June 2013 Galaxy Update is now availablehttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06 . *Highlights:* - *Register http://wiki.galaxyproject.org/Events/GCC2013/Register for GCC2013 http://wiki.galaxyproject.org/GalaxyUpdates/2013_06#GCC2013 by 7 June to avoid late registration rates* - Two new public servershttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06#New_Public_Servers ODoSE and CBiB - 39 new papershttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06#New_Papers - Open Positionshttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06#Who.27s_Hiring at four different institutions - Other Upcoming Eventshttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06#Other_Upcoming_Events - Duplicate Accounts on Mainhttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06#Duplicate_Accounts_on_Main - Tool Shed Contributionshttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06#Tool_Shed_Contributions - Other Newshttp://wiki.galaxyproject.org/GalaxyUpdates/2013_06#Other_News If you have anything you would like to see in the July *Galaxy Updatehttp://wiki.galaxyproject.org/GalaxyUpdates *, please let us know. Dave Clements and the Galaxy Team http://wiki.galaxyproject.org/GalaxyTeam -- http://galaxyproject.org/GCC2013 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Multi File upload api
This changeset is where I implemented collecting and assigning names for multiple file datasets: https://bitbucket.org/msiappdev/galaxy-extras/commits/bfb07a8c8b7b4c404339cd1efe50f9b2becb3b62 -John On Fri, May 31, 2013 at 1:21 AM, alex.khassa...@csiro.au wrote: Thanks John, We do have pretty old code, so I applied your patches. One minor problem, in multi_upload tool Dataset Name is ignored, just picks the first file name. Do we need to update upoald.py or multi_upload.xml? -Alex -Original Message- From: jmchil...@gmail.com [mailto:jmchil...@gmail.com] On Behalf Of John Chilton Sent: Friday, 31 May 2013 5:50 AM To: Khassapov, Alex (CSIRO IMT, Clayton) Cc: Burdett, Neil (ICT Centre, Herston - RBWH); galaxy-...@bx.psu.edu Subject: Re: Multi File upload api Alex, if I am not mistaken you guys are using a fairly old version of the multiple file dataset code. I think on the server side you would need to apply the following three changesets to update the library backend to support the creation of multifile datasets (I guess if you are doing path pasting only the first changeset is really needed): https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=25e137130d32at=stable https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=a15ce4c00716at=stable https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=d2ae2265bc96at=stable If your version of Galaxy is likewise pretty old you will need pull request 123 as first (https://bitbucket.org/galaxy/galaxy-central/pull-request/123/refactor-library_commonpy-toward-more/diff). Once those are in place updating example_watch_folder.py should be pretty doable (Hagai Cohen had it working at some point): 1) You will need to replace: data['upload_option'] = 'upload_paths' with data['upload_option'] = 'upload_paths_multifile' 2) You will need to specify a name (e.g. add something like: data['NAME'] = os.path.basename(fullpath)) 3) Multiple file datasets do not support data['ext'] = 'auto', so you will need to specify an explicit format there (the script will hopefully be able to determine this from the inputs). The most updated version of the multiple file dataset stuff https://bitbucket.org/msiappdev/galaxy-extras/ paired with the most updated version of blend4j (https://github.com/jmchilton/blend4j) are compatible and if you do enough digging can provide insights on how to create multiple file datasets via the API (library path pastes, library server directory uploads, and history uploads). Hope this helps, -John On Thu, May 30, 2013 at 1:40 AM, alex.khassa...@csiro.au wrote: Hi John, Can you please have a look at Neil's question. Thank you, -Alex From: Burdett, Neil (ICT Centre, Herston - RBWH) Sent: Thursday, 30 May 2013 4:30 PM To: Khassapov, Alex (CSIRO IMT, Clayton) Subject: Multi File upload api Hi Alex, The file galaxy-dist/scripts/api/example_watch_folder.py allows us to watch a folder then upload files that arrive in a specified input folder into the database and history. Can you ask your friend (who implemented the multi file upload tool), what changes are necessary to this file so we can upload multiple files as we do from the GUI. I assume he would know quite quickly what to do and hopefully quite simple Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] login to ftp server failed or Problem with ftp connexion to Galaxy
To whom this may concern, This is Danni Yu, a Galaxy user starting to use Galaxy since 2011. I was able to upload large files through ftp in 2011 and 2012. Now, I am facing the same problem as the other users (such as Barbara Sixt on May 27, 2013 and Fabrice Besnard on May 22, 2013). The error message is 530 login incorrect. Ms. Jennifer Hillman-Jackson in Galaxy Support and Training team very kindly mentioned that Please feel free to contact us for an update if this is still a problem by next Friday on May 24, 2013. Since it is already Friday but the problem still exists, I am hoping to learn whether a solution has been provided. Thanks very much and Sincerely, Danni ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] login to ftp server failed or Problem with ftp connexion to Galaxy
Hi Danni, the following is copied from a Mail from Jennifer Jackson 2 days ago. For some reason this mail is not indexed from google, yet. Hope that helps, Björn Dear Galaxy Main Community, We apologize for recent FTP login issues, but are glad to report that a solution is now in place. May 29th 2013 FTP login resolution Issue: FTP login failed for user accounts newly created or that had passwords updated during the time frame May 20th - May 29th. Solution: Change your Galaxy password using the form on the Main instance at http://usegalaxy.org User - Preferences - Manage your information On the Manage User Information form, change your password Wait ~20 minutes for the server to update, then use FTP with the new credentials How to use FTP Upload Share notice: http://bit.ly/ftpmay29 Thanks for using Galaxy, The Galaxy Team --- To whom this may concern, This is Danni Yu, a Galaxy user starting to use Galaxy since 2011. I was able to upload large files through ftp in 2011 and 2012. Now, I am facing the same problem as the other users (such as Barbara Sixt on May 27, 2013 and Fabrice Besnard on May 22, 2013). The error message is 530 login incorrect. Ms. Jennifer Hillman-Jackson in Galaxy Support and Training team very kindly mentioned that Please feel free to contact us for an update if this is still a problem by next Friday on May 24, 2013. Since it is already Friday but the problem still exists, I am hoping to learn whether a solution has been provided. Thanks very much and Sincerely, Danni ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Getting example_watch_folder.py to work...
In case someone has the same issue,
This is resolved by extending the sleep time found here
http://gmod.827538.n3.nabble.com/Trouble-Shooting-example-watch-folder-py-td4030355.html
From: galaxy-dev-boun...@lists.bx.psu.edu
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Hakeem Almabrazi
Sent: Thursday, May 30, 2013 4:47 PM
To: Dannon Baker; neil.burd...@csiro.au
Cc: Galaxy Dev
Subject: Re: [galaxy-dev] Getting example_watch_folder.py to work...
Sorry for bringing this back but I am having an issue with the
example_watch_folder.pl script. After running in and put a fasta file, it gets
copied over but it is empty. Any idea how to fix this issue.
Uploaded by: galaxy@
Date uploaded: 2013-05-30
File size: 0 bytes
Data type: txt
Build: ?
Miscellaneous information: Uploaded temporary file (/tmp/API/myseq.fasta) does
not exist.
empty
Database/Build: ?
Number of data lines: None
Disk file: /usr/local/galaxy/galaxy-dist/database/files/000/dataset_299.dat
Thank you for your help,
Regards
From: galaxy-dev-boun...@lists.bx.psu.edu
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Dannon Baker
Sent: Thursday, May 09, 2013 6:48 AM
To: neil.burd...@csiro.au
Cc: Galaxy Dev
Subject: Re: [galaxy-dev] Getting example_watch_folder.py to work...
Hey Neil,
A user browsing that history wouldn't automatically get the updates (unless the
history was already refreshing for some reason), but they definitely would upon
refresh. Glad you got the rest working!
-Dannon
On Thu, May 9, 2013 at 2:02 AM,
neil.burd...@csiro.aumailto:neil.burd...@csiro.au wrote:
Hi all,
I finally resolved the issue I'll mention it here to help anyone else
who has the same issue. In Edit mode on the workflow you add the Workflow
control-Inputs to use the api.
Final question
Everything gets created and stored in the database, but the history column on
the left of the browser doesn't get updated. Can user see if the task has
completed successfully and all files uploaded from the browser?
Thanks
Neil
From: Burdett, Neil (ICT Centre, Herston - RBWH)
Sent: Thursday, 9 May 2013 2:56 PM
To: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu
Subject: Getting example_watch_folder.py to work...
Hi,
I'm still struggling to get the example_watch_folder.py to work. So any
help much appreciated.
I've created a simple workflow, which essentially opens a text file and then
writes out the data to a html file.
My xml file:
tool id=CopyTool name=Copy Tool
descriptionTest/description
command interpreter=perl$__root_dir__/tools/copy/copy.plhttp://copy.pl
--input_image $inputImage
--output_html $output
/command
inputs
param format=txt name=inputImage type=data label=Input Image /
/inputs
outputs
data format=html name=output label=${tool.namehttp://tool.name} on
#echo os.path.basename( str ( $inputImage.name ) )# /
/outputs
help
Copy Tool
/help
/tool
And copy.plhttp://copy.pl:
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long;
my $input_image ;
my $output_html ;
# Get options from command line (i.e. galaxy)
my $result = GetOptions ( input_image=s = \$input_image,
output_html=s = \$output_html )
or die Bad options;
print input_image is: $input_image\n;
open FILE,$input_image or die $!;
my @lines = FILE;
close FILE;
my $numOfLines = scalar @lines;
# Create the output HTML, with links to the files and the square gif.
#
open HTML, , $output_html
or die Failed to create output HTML file '$output_html': $! ;
print HTMLEOF;
html
head
style
iframe {
border: 0px;
background: #ee ;
}
/style
/head
body
h1Copy Files tool/h1
h2Generated $numOfLines files/h2
EOF
# Put direct links to each output file
foreach my $sub_filename ( @lines )
{
print HTML Direct link to the $sub_filename file.br/\n;
}
print HTML br/br/br/\n;
print HTML /body/html\n ;
close HTML ;
So pretty basic.
I run:
./example_watch_folder.py cce1b01926646d548f6ddc32ff01aa2e
http://140.253.78.234/galaxy/api/ /home/galaxy/data_input
/home/galaxy/data_output My API Import f2db41e1fa331b3e
and get the following output:
{'outputs': ['a799d38679e985db'], 'history': '33b43b4e7093c91f'}
I can see the sample.txt file I placed in the /home/galaxy/data_input has been
put into the database:
/home/galaxy/galaxy-dist/database/files/000/dataset_8.dat
and:
http://140.253.78.234/galaxy/api/libraries/f2db41e1fa331b3e/contents/1cd8e2f6b131e891
shows the file has been uploaded:
{
data_type: txt,
date_uploaded: 2013-05-09T04:41:10.579602,
file_name: /home/galaxy/galaxy-dist/database/files/000/dataset_8.dat,
file_size: 76,
genome_build: ?,
id: 1cd8e2f6b131e891,
ldda_id: 1cd8e2f6b131e891,
message: ,
metadata_data_lines: 8,
metadata_dbkey: ?,
misc_blurb: 8 lines,
misc_info: uploaded txt file,
model_class: LibraryDataset,
name: sam2.txt,
[galaxy-dev] uploading large datasets into a new instance of Galaxy on a Mac Pro
Hello, I installed Galaxy on a new Mac Pro with 64gb of ram and a solid state first drive and 3 , 2tb normal drives. I can upload from my desktop a smaller file like a reference list or a cuffdiff file but when I try to load a 16gb fastq file it just does not load. Given that I have my files on the same computer that has Galaxy, and given that I am not well versed in unix, would anyone have a suggestion as to ways I could easily upload these large files. I would like to think I am just naive and that this has a simple solution. Hopefully this is the correct list to pose this question. Thank you for any help, el linney For those in the no, I did see this appear on the terminal after I tried to upload the file through the browser mechanism for upload: Overflow Error: signed integer is greater than maximum ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Storage question for Galaxy-Admins
Raj, I agree with the points Alex made. It would be much easier to start small and then work your way up. I recently setup galaxy for our HPC cluster for the entire school of medicine. Its only a few ~month into our setup but we are currently using ~300GB of storage. This is part of our larger cluster storage pool of 600TB. We use the gluster filesystem across 6 storage nodes. We are trying fhgfs and other file systems as well (lustre, ceph, etc). If you have any other questions, feel free to ask. -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, May 29, 2013 at 10:34 PM, Raj Ayyampalayam ra...@uga.edu wrote: Hello, I am trying to come up with a number for our local galaxy storage requirements when we go live (This is an installation for research at our university). I checked out the survey that was done a while back for the GalaxyAdmins. It seems that the bigger installations have allocated about 200TB for their installation. It would be of great help to me if I can get some feedback on storage used by local galaxy installations. If possible please use the following format: Total capacity: Used capacity: Storage Technology: Comments: Thank you all very much. -Raj ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] uploading large datasets into a new instance of Galaxy on a Mac Pro
How are you trying to upload the data to Galaxy? Are you using the web interface? Or...? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 31, 2013 at 1:59 PM, Elwood Linney ellin...@gmail.com wrote: Hello, I installed Galaxy on a new Mac Pro with 64gb of ram and a solid state first drive and 3 , 2tb normal drives. I can upload from my desktop a smaller file like a reference list or a cuffdiff file but when I try to load a 16gb fastq file it just does not load. Given that I have my files on the same computer that has Galaxy, and given that I am not well versed in unix, would anyone have a suggestion as to ways I could easily upload these large files. I would like to think I am just naive and that this has a simple solution. Hopefully this is the correct list to pose this question. Thank you for any help, el linney For those in the no, I did see this appear on the terminal after I tried to upload the file through the browser mechanism for upload: Overflow Error: signed integer is greater than maximum ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
