Re: [galaxy-dev] [GSoC 2013] Introduction

2013-05-31 Thread Björn Grüning 
Welcome Saket!

Interesting topic ... wish you all the best and don't hesitate to ask.
There is also a galaxyproject channel on freenode.

Cheers,
Björn


> Hi Everyone!
> 
> 
> I have been selected to work on the eQTL pipeline and visualisation
> for the Galaxy Project as part of the Google Summer of Code 2013
> (http://www.google-melange.com/gsoc/homepage/google/gsoc2013)
> 
> 
> I am pursuing my fourth year of undergraduate studies at Indian
> Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of
> interest are broadly Bioinformatics and of lately I have been working
> on NGS problems.
> 
> I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ .
> The first post should be up soon.
> 
> 
> My proposal is  here :
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001
>  . I would love to know your suggestions/comments.
> 
> I am relatively new to Galaxy, and some of my questions in the process
> might be elementary, so kindly bear with it :)
> 
> Thanks
> 
> Saket Choudhary
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [GSoC 2013] Introduction

2013-05-31 Thread Björn Grüning 
Welcome Saket!

Interesting topic ... wish you all the best and don't hesitate to ask.
There is also a #galaxyproject channel on freenode.

Cheers,
Björn
> Hi Everyone!
> 
> 
> I have been selected to work on the eQTL pipeline and visualisation
> for the Galaxy Project as part of the Google Summer of Code 2013
> (http://www.google-melange.com/gsoc/homepage/google/gsoc2013)
> 
> 
> I am pursuing my fourth year of undergraduate studies at Indian
> Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of
> interest are broadly Bioinformatics and of lately I have been working
> on NGS problems.
> 
> 
> 
> I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ .
> The first post should be up soon.
> 
> 
> My proposal is  here :
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001
>  . I would love to know your suggestions/comments.
> 
> 
> 
> 
> 
> I am relatively new to Galaxy, and some of my questions in the process
> might be elementary, so kindly bear with it :)
> 
> 
> 
> Thanks
> 
> Saket Choudhary
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [GSoC 2013] Introduction

2013-05-31 Thread Ross 
Hi, Saket.
Welcome aboard - help is always available here or irc if you're in the
right timezone.
https://chgr.mc.vanderbilt.edu/plato/
is
a brand new eQTL option from Marylyn Ritchie's group I stumbled across at
http://gettinggeneticsdone.blogspot.com.au/ today - the comments about
plink are apt - it can be a pain to work with in my experience.




On Fri, May 31, 2013 at 4:35 PM, Saket Choudhary  wrote:

> Hi Everyone!
>
> I have been selected to work on the eQTL pipeline and visualisation for
> the Galaxy Project as part of the Google Summer of Code 2013 (
> http://www.google-melange.com/gsoc/homepage/google/gsoc2013)
>
> I am pursuing my fourth year of undergraduate studies at Indian Institute
> of Technology Bombay(http://www.iitb.ac.in/). My areas of interest are
> broadly Bioinformatics and of lately I have been working on NGS problems.
>
>
> I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . The
> first post should be up soon.
>
> My proposal is  here :
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001.
>  I would love to know your suggestions/comments.
>
>
> I am relatively new to Galaxy, and some of my questions in the process
> might be elementary, so kindly bear with it :)
>
>
> Thanks
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Internal Server Error during installation of ncbi_blast_plus

2013-05-31 Thread Keilwagen, Jens 
Hi Greg,

thanks for your comment. Some days ago, I updated Galaxy following:

http://wiki.galaxyproject.org/News/2013_04_08_Galaxy_Security_Release

Here is what "hg sum" said

parent: 9292:2cc8d10988e0 security_2013.04.08
 controllers/history: use get_history in switch_to_history
Zweig: stable
Übernehme: 10 unbekannt (clean)
Aktualisiere: 11 neue Änderungssätze (Aktualisierung)

(Sorry for the German terms.) I then updated Galaxy by

hg update

obtaining:

parent: 9320:47ddf167c9f1 tip
 Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response.  
Fixes downloading tarballs from the Tool Shed when use_debug = false.
Zweig: stable
Übernehme: 10 unbekannt (clean)
Aktualisiere: (aktuell)

I then restarted Galaxy and tried to install again, but it fails again. Even 
trying for a second time doesn't change the behaviour. Here is what paster.log 
says:

172.31.30.183 - - [31/May/2013:07:47:32 +0200] "GET 
/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/&repository_ids=1d92ebdf7e8d466c&changeset_revisions=9dabbfd73c8a
 HTTP/1.0" 500 - 
"http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?repository_id=1d92ebdf7e8d466c&changeset_revision=9dabbfd73c8a";
 "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0"
Error - : too many values to unpack
URL: 
http://galaxy.jki.bund.de/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/&repository_ids=1d92ebdf7e8d466c&changeset_revisions=9dabbfd73c8a
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', 
line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File '/opt/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
  return self.application(environ, start_response)
File 
'/opt/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', 
line 633 in __call__
  return self.application(environ, start_response)
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 128 in 
__call__
  return self.handle_request( environ, start_response )
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 184 in 
handle_request
  body = method( trans, **kwargs )
File '/opt/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 231 
in decorator
  return func( self, trans, *args, **kwargs )
File 
'/opt/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py',
 line 894 in prepare_for_install
  common_install_util.get_dependencies_for_repository( trans, tool_shed_url, 
repo_info_dict, includes_tool_dependencies )
File '/opt/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 
88 in get_dependencies_for_repository
  installed_rd, missing_rd = 
get_installed_and_missing_repository_dependencies_for_new_install( trans, 
repo_info_tuple )
File '/opt/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 
192 in get_installed_and_missing_repository_dependencies_for_new_install
  tool_shed, name, owner, changeset_revision = rd_tup
ValueError: too many values to unpack


CGI Variables
-
  CONTENT_LENGTH: '0'
  HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3'
  HTTP_CACHE_CONTROL: 'max-age=0'
  HTTP_CONNECTION: 'keep-alive'
  HTTP_COOKIE: 
'galaxysession=c6ca0ddb55be603a88fa41dbba7680819dc3ac10c92a6efa6da917af7060700e4360e4afb5987baa'
  HTTP_HOST: 'galaxy.jki.bund.de'
  HTTP_REFERER: 
'http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?repository_id=1d92ebdf7e8d466c&changeset_revision=9dabbfd73c8a'
  HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 
Firefox/21.0'
  HTTP_VIA: '1.1 proxy.jki.bund.de:8000 (squid/2.7.STABLE9)'
  HTTP_X_FORWARDED_FOR: '172.31.30.183'
  ORGINAL_REMOTE_ADDR: '172.16.20.52'
  PATH_INFO: '/admin_toolshed/prepare_for_install'
  QUERY_STRING: 
'tool_shed_url=http://toolshed.g2.bx.psu.edu/&repository_ids=1d92ebdf7e8d466c&changeset_revisions=9dabbfd73c8a'
  REMOTE_ADDR: '172.31.30.183'
  REQUEST_METHOD: 'GET'
  SERVER_NAME: '0.0.0.0'
  SERVER_PORT: '8080'
  SERVER_PROTOCOL: 'HTTP/1.0'


WSGI Variables
--
  application: 
  paste.cookies: (,
 
'galaxysession=c6ca0ddb55be603a88fa41dbba7680819dc3ac10c92a6efa6da917af7060700e4360e4afb5987baa')
  paste.expected_exceptions: []
  paste.httpexceptions: 
  paste.httpserver.thread_pool: 
  paste.parsed_querystring: ([('tool_shed_url', 
'http://toolshed.g2.bx.psu.edu/'), ('repository_ids', '1d92ebdf7e8d466c'), 
('changeset_revisions', '9dabbfd73c8a')], 
'tool_shed_url=http://toolshed.g2.bx.psu.edu/&repository_ids=1d92ebdf7e8d466c&changeset_revisions=9dabbfd73c8a')
  paste.recursive.forward: 
  paste.recursive.include: 
  paste.recursive.include_app_iter: 
  paste.recursive.script_name: ''
  paste.throw_errors: True
  request_id: '964a9788c9b511e2bb080050568d0b

[galaxy-dev] Status of Galaxy Cloud Instance dev

2013-05-31 Thread Joachim Jacob | VIB | 

Hi all,


I am planning to launch a Galaxy Cloud Instance, to run for about one 
month. But I believe I've read somewhere a new image of Galaxy on AWS is 
being prepared right now.


Can somebody update me on the status? Perhaps it is smart to wait a few 
days with launching through CloudLaunch until the new release is there?



Thanks,
Joachim

--
Joachim Jacob
Contact details: http://www.bits.vib.be/index.php/about/80-team


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Status of Galaxy Cloud Instance dev

2013-05-31 Thread Dannon Baker 
No firm release set for the new cloudman yet, but you can always follow
development here:
https://trello.com/board/galaxy-cloudman/5136209e9beefacb77004e6d

There are a few final issues we're wrapping up before we can release, but
we're hoping it'll be soon.



On Fri, May 31, 2013 at 7:31 AM, Joachim Jacob | VIB |  wrote:

> Hi all,
>
>
> I am planning to launch a Galaxy Cloud Instance, to run for about one
> month. But I believe I've read somewhere a new image of Galaxy on AWS is
> being prepared right now.
>
> Can somebody update me on the status? Perhaps it is smart to wait a few
> days with launching through CloudLaunch until the new release is there?
>
>
> Thanks,
> Joachim
>
> --
> Joachim Jacob
> Contact details: 
> http://www.bits.vib.be/index.**php/about/80-team
>
>
> __**_
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>  
> http://galaxyproject.org/**search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Status of Galaxy Cloud Instance dev

2013-05-31 Thread John Chilton 
Hey Dannon,

Thanks for the link, I saw that you had moved some cards to a CloudMan
board but I was never able to find it. Could you enable comments for
non-board members like the Galaxy Development Trello allows? You know
I have a compulsion to  comment on everything Galaxy related anywhere
on the Internet.

-John

On Fri, May 31, 2013 at 8:34 AM, Dannon Baker  wrote:
> No firm release set for the new cloudman yet, but you can always follow
> development here:
> https://trello.com/board/galaxy-cloudman/5136209e9beefacb77004e6d
>
> There are a few final issues we're wrapping up before we can release, but
> we're hoping it'll be soon.
>
>
>
> On Fri, May 31, 2013 at 7:31 AM, Joachim Jacob | VIB |
>  wrote:
>>
>> Hi all,
>>
>>
>> I am planning to launch a Galaxy Cloud Instance, to run for about one
>> month. But I believe I've read somewhere a new image of Galaxy on AWS is
>> being prepared right now.
>>
>> Can somebody update me on the status? Perhaps it is smart to wait a few
>> days with launching through CloudLaunch until the new release is there?
>>
>>
>> Thanks,
>> Joachim
>>
>> --
>> Joachim Jacob
>> Contact details: http://www.bits.vib.be/index.php/about/80-team
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [GSoC 2013] Introduction

2013-05-31 Thread Samuel Lampa 

Cool! :)

Feel free to join the Google+ group as well, and link your blog posts 
there, to notify users (and make it easy for us to share further):

http://gplus.to/galaxyportal

Cheers
// Samuel

On 05/31/2013 08:35 AM, Saket Choudhary wrote:

Hi Everyone!

I have been selected to work on the eQTL pipeline and visualisation 
for the Galaxy Project as part of the Google Summer of Code 2013 
(http://www.google-melange.com/gsoc/homepage/google/gsoc2013)


I am pursuing my fourth year of undergraduate studies at Indian 
Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of 
interest are broadly Bioinformatics and of lately I have been working 
on NGS problems.



I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . 
The first post should be up soon.




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Status of Galaxy Cloud Instance dev

2013-05-31 Thread Dannon Baker 
On Fri, May 31, 2013 at 9:51 AM, John Chilton  wrote:

> Could you enable comments for non-board members like the Galaxy
> Development Trello allows?


Of course, actually thought I'd already done that.  Fixed!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Multi File upload api

2013-05-31 Thread Hakeem Almabrazi 
Hi guys,

I am sure I will be looking into this shortly but now I want to get the one 
file case working first :).  Do you guys have any idea why the 
example_watch_folder.pl will copy the file but not its contents?  The file it 
copies into the library folder is empty.  How can I go about resolving this 
issue?

I appreciate any help.

Regards,


-Original Message-
From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of 
alex.khassa...@csiro.au
Sent: Friday, May 31, 2013 1:22 AM
To: chil...@msi.umn.edu
Cc: galaxy-...@bx.psu.edu; neil.burd...@csiro.au
Subject: Re: [galaxy-dev] Multi File upload api

Thanks John,

We do have pretty old code, so I applied your patches.

One minor problem, in multi_upload tool "Dataset Name" is ignored, just picks 
the first file name. Do we need to update upoald.py or multi_upload.xml?

-Alex

-Original Message-
From: jmchil...@gmail.com [mailto:jmchil...@gmail.com] On Behalf Of John Chilton
Sent: Friday, 31 May 2013 5:50 AM
To: Khassapov, Alex (CSIRO IM&T, Clayton)
Cc: Burdett, Neil (ICT Centre, Herston - RBWH); 
Subject: Re: Multi File upload api

Alex, if I am not mistaken you guys are using a fairly old version of the 
multiple file dataset code. I think on the server side you would need to apply 
the following three changesets to update the library backend to support the 
creation of multifile datasets (I guess if you are doing path pasting only the 
first changeset is really needed):

https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=25e137130d32&at=stable
https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=a15ce4c00716&at=stable
https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=d2ae2265bc96&at=stable

If your version of Galaxy is likewise pretty old you will need pull request 123 
as first 
(https://bitbucket.org/galaxy/galaxy-central/pull-request/123/refactor-library_commonpy-toward-more/diff).

Once those are in place updating example_watch_folder.py should be pretty 
doable (Hagai Cohen had it working at some point):

1) You will need to replace:

data['upload_option'] = 'upload_paths'

with

data['upload_option'] = 'upload_paths_multifile'

2) You will need to specify a name (e.g. add something like:
data['NAME'] = os.path.basename(fullpath))

3) Multiple file datasets do not support data['ext'] = 'auto', so you will need 
to specify an explicit format there (the script will hopefully be able to 
determine this from the inputs).

The most updated version of the multiple file dataset stuff 
https://bitbucket.org/msiappdev/galaxy-extras/ paired with the most updated 
version of blend4j (https://github.com/jmchilton/blend4j) are compatible and if 
you do enough digging can provide insights on how to create multiple file 
datasets via the API (library path pastes, library server directory uploads, 
and history uploads).

Hope this helps,
-John

On Thu, May 30, 2013 at 1:40 AM,   wrote:
> Hi John,
>
>
>
> Can you please have a look at Neil's question.
>
>
>
> Thank you,
>
>
>
> -Alex
>
>
>
> From: Burdett, Neil (ICT Centre, Herston - RBWH)
> Sent: Thursday, 30 May 2013 4:30 PM
> To: Khassapov, Alex (CSIRO IM&T, Clayton)
> Subject: Multi File upload api
>
>
>
> Hi Alex,
>
>   The file "galaxy-dist/scripts/api/example_watch_folder.py" 
> allows us to watch a folder then upload files that arrive in a 
> specified input folder into the database and history.
>
>
>
> Can you ask your friend (who implemented the multi file upload tool), 
> what changes are necessary to this file so we can upload multiple 
> files as we do from the GUI. I assume he would know quite quickly what 
> to do and hopefully quite simple
>
>
>
> Thanks
>
> Neil

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] June 2013 Galaxy Update

2013-05-31 Thread Dave Clements 
Hello all,

The June 2013 Galaxy Update is now
available
.

*Highlights:*

   -

   *Register  for
   GCC2013  by
   7 June to avoid late registration rates*
   -

   Two new public
servers
ODoSE
   and CBiB
   -

   39 new papers
   -

   Open 
Positions
at
   four different institutions
   -

   Other Upcoming
Events
   -

   Duplicate Accounts on
Main
   -

   Tool Shed 
Contributions
   -

   Other News

If you have anything you would like to see in the July *Galaxy
Update
*, please let us know.

Dave Clements and the Galaxy Team 
-- 
http://galaxyproject.org/GCC2013
http://galaxyproject.org/
http://getgalaxy.org/
http://usegalaxy.org/
http://wiki.galaxyproject.org/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Multi File upload api

2013-05-31 Thread John Chilton 
This changeset is where I implemented collecting and assigning names
for multiple file datasets:

https://bitbucket.org/msiappdev/galaxy-extras/commits/bfb07a8c8b7b4c404339cd1efe50f9b2becb3b62

-John


On Fri, May 31, 2013 at 1:21 AM,   wrote:
> Thanks John,
>
> We do have pretty old code, so I applied your patches.
>
> One minor problem, in multi_upload tool "Dataset Name" is ignored, just picks 
> the first file name. Do we need to update upoald.py or multi_upload.xml?
>
> -Alex
>
> -Original Message-
> From: jmchil...@gmail.com [mailto:jmchil...@gmail.com] On Behalf Of John 
> Chilton
> Sent: Friday, 31 May 2013 5:50 AM
> To: Khassapov, Alex (CSIRO IM&T, Clayton)
> Cc: Burdett, Neil (ICT Centre, Herston - RBWH); 
> Subject: Re: Multi File upload api
>
> Alex, if I am not mistaken you guys are using a fairly old version of the 
> multiple file dataset code. I think on the server side you would need to 
> apply the following three changesets to update the library backend to support 
> the creation of multifile datasets (I guess if you are doing path pasting 
> only the first changeset is really needed):
>
> https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=25e137130d32&at=stable
> https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=a15ce4c00716&at=stable
> https://bitbucket.org/msiappdev/galaxy-extras/diff/lib/galaxy/webapps/galaxy/controllers/library_common.py?diff2=d2ae2265bc96&at=stable
>
> If your version of Galaxy is likewise pretty old you will need pull request 
> 123 as first 
> (https://bitbucket.org/galaxy/galaxy-central/pull-request/123/refactor-library_commonpy-toward-more/diff).
>
> Once those are in place updating example_watch_folder.py should be pretty 
> doable (Hagai Cohen had it working at some point):
>
> 1) You will need to replace:
>
> data['upload_option'] = 'upload_paths'
>
> with
>
> data['upload_option'] = 'upload_paths_multifile'
>
> 2) You will need to specify a name (e.g. add something like:
> data['NAME'] = os.path.basename(fullpath))
>
> 3) Multiple file datasets do not support data['ext'] = 'auto', so you will 
> need to specify an explicit format there (the script will hopefully be able 
> to determine this from the inputs).
>
> The most updated version of the multiple file dataset stuff 
> https://bitbucket.org/msiappdev/galaxy-extras/ paired with the most updated 
> version of blend4j (https://github.com/jmchilton/blend4j) are compatible and 
> if you do enough digging can provide insights on how to create multiple file 
> datasets via the API (library path pastes, library server directory uploads, 
> and history uploads).
>
> Hope this helps,
> -John
>
> On Thu, May 30, 2013 at 1:40 AM,   wrote:
>> Hi John,
>>
>>
>>
>> Can you please have a look at Neil's question.
>>
>>
>>
>> Thank you,
>>
>>
>>
>> -Alex
>>
>>
>>
>> From: Burdett, Neil (ICT Centre, Herston - RBWH)
>> Sent: Thursday, 30 May 2013 4:30 PM
>> To: Khassapov, Alex (CSIRO IM&T, Clayton)
>> Subject: Multi File upload api
>>
>>
>>
>> Hi Alex,
>>
>>   The file "galaxy-dist/scripts/api/example_watch_folder.py"
>> allows us to watch a folder then upload files that arrive in a
>> specified input folder into the database and history.
>>
>>
>>
>> Can you ask your friend (who implemented the multi file upload tool),
>> what changes are necessary to this file so we can upload multiple
>> files as we do from the GUI. I assume he would know quite quickly what
>> to do and hopefully quite simple
>>
>>
>>
>> Thanks
>>
>> Neil

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] "login to ftp server failed" or "Problem with ftp connexion to Galaxy"

2013-05-31 Thread Danni Yu 
To whom this may concern, 

This is Danni Yu, a Galaxy user starting to use Galaxy since 2011. I was able 
to upload large files through ftp in 2011 and 2012. Now, I am facing the same 
problem as the other users (such as Barbara Sixt on May 27, 2013 and Fabrice 
Besnard on May 22, 2013). The error message is "530 login incorrect".

Ms. Jennifer Hillman-Jackson in Galaxy Support and Training team very kindly 
mentioned that "Please feel free to contact us for an update if this is still a 
problem by next Friday" on May 24, 2013. 

Since it is already Friday but the problem still exists, I am hoping to learn 
whether a solution has been provided. 

Thanks very much and Sincerely,

Danni
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] "login to ftp server failed" or "Problem with ftp connexion to Galaxy"

2013-05-31 Thread Björn Grüning 
Hi Danni,

the following is copied from a Mail from Jennifer Jackson 2 days ago.
For some reason this mail is not indexed from google, yet.

Hope that helps,
Björn


Dear Galaxy Main Community,

We apologize for recent FTP login issues, but are glad to report that a
solution is now in place.

May 29th 2013 FTP login resolution

Issue: FTP login failed for user accounts newly created or that had
passwords 
updated during the time frame May 20th - May 29th.

Solution: Change your Galaxy password using the form on the Main
instance 
at http://usegalaxy.org

User -> Preferences -> Manage your information

On the "Manage User Information" form, change your password

Wait ~20 minutes for the server to update, then use FTP with the new
credentials
How to use FTP Upload

Share notice: http://bit.ly/ftpmay29


Thanks for using Galaxy,

The Galaxy Team
---

> To whom this may concern, 
> 
> This is Danni Yu, a Galaxy user starting to use Galaxy since 2011. I was able 
> to upload large files through ftp in 2011 and 2012. Now, I am facing the same 
> problem as the other users (such as Barbara Sixt on May 27, 2013 and Fabrice 
> Besnard on May 22, 2013). The error message is "530 login incorrect".
> 
> Ms. Jennifer Hillman-Jackson in Galaxy Support and Training team very kindly 
> mentioned that "Please feel free to contact us for an update if this is still 
> a problem by next Friday" on May 24, 2013. 
> 
> Since it is already Friday but the problem still exists, I am hoping to learn 
> whether a solution has been provided. 
> 
> Thanks very much and Sincerely,
> 
> Danni
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Getting example_watch_folder.py to work...

2013-05-31 Thread Hakeem Almabrazi 
In case someone has the same issue,

This is resolved by extending the sleep time found here 
http://gmod.827538.n3.nabble.com/Trouble-Shooting-example-watch-folder-py-td4030355.html

From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Hakeem Almabrazi
Sent: Thursday, May 30, 2013 4:47 PM
To: Dannon Baker; neil.burd...@csiro.au
Cc: Galaxy Dev
Subject: Re: [galaxy-dev] Getting example_watch_folder.py to work...

Sorry for bringing this back but I am having an issue with the 
example_watch_folder.pl script.  After running in and put a fasta file, it gets 
copied over but it is empty.  Any idea how to fix this issue.

Uploaded by: galaxy@
Date uploaded: 2013-05-30
File size: 0 bytes
Data type: txt
Build: ?
Miscellaneous information: Uploaded temporary file (/tmp/API/myseq.fasta) does 
not exist.
empty
Database/Build: ?
Number of data lines: None
Disk file: /usr/local/galaxy/galaxy-dist/database/files/000/dataset_299.dat


Thank you for your help,

Regards
From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Dannon Baker
Sent: Thursday, May 09, 2013 6:48 AM
To: neil.burd...@csiro.au
Cc: Galaxy Dev
Subject: Re: [galaxy-dev] Getting example_watch_folder.py to work...

Hey Neil,

A user browsing that history wouldn't automatically get the updates (unless the 
history was already refreshing for some reason), but they definitely would upon 
refresh.  Glad you got the rest working!

-Dannon

On Thu, May 9, 2013 at 2:02 AM, 
mailto:neil.burd...@csiro.au>> wrote:
Hi all,
  I finally resolved the issue I'll mention it here to help anyone else 
who has the same issue. In "Edit" mode on the workflow you add the "Workflow 
control->Inputs" to use the api.

Final question
Everything gets created and stored in the database, but the history column on 
the left of the browser doesn't get updated. Can user see if the task has 
completed successfully and all files uploaded from the browser?

Thanks
Neil

From: Burdett, Neil (ICT Centre, Herston - RBWH)
Sent: Thursday, 9 May 2013 2:56 PM

To: galaxy-dev@lists.bx.psu.edu
Subject: Getting example_watch_folder.py to work...

Hi,

   I'm still struggling to get the example_watch_folder.py to work. So any 
help much appreciated.

I've created a simple workflow, which essentially opens a text file and then 
writes out the data to a html file.

My xml file:

  Test
  $__root_dir__/tools/copy/copy.pl
  --input_image "$inputImage"
  --output_html "$output"
  

  

  
  
http://tool.name>} on 
#echo os.path.basename( str ( $inputImage.name ) )#" />
  
  
Copy Tool

  


And copy.pl:
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long;
my $input_image ;
my $output_html ;

# Get options from command line (i.e. galaxy)
my $result = GetOptions ( "input_image=s" => \$input_image,
  "output_html=s" => \$output_html )
  or die "Bad options";

print "input_image is: $input_image\n";

open FILE,"$input_image" or die $!;
my @lines = ;
close FILE;
my $numOfLines = scalar @lines;


# Create the output HTML, with links to the files and the square gif.
#
open HTML, ">", $output_html
or die "Failed to create output HTML file '$output_html': $! ";

print HTML<


iframe {
border: 0px;
background: #ee ;
}



Copy Files tool
Generated $numOfLines files
EOF

# Put direct links to each output file
foreach my $sub_filename ( @lines )
{
  print HTML "Direct link to the $sub_filename file.\n";
}

print HTML "\n";

print HTML "\n" ;

close HTML ;

So pretty basic.

I run:
./example_watch_folder.py cce1b01926646d548f6ddc32ff01aa2e 
http://140.253.78.234/galaxy/api/ /home/galaxy/data_input 
/home/galaxy/data_output "My API Import" f2db41e1fa331b3e

and get the following output:
{'outputs': ['a799d38679e985db'], 'history': '33b43b4e7093c91f'}

I can see the sample.txt file I placed in the /home/galaxy/data_input has been 
put into the database:
/home/galaxy/galaxy-dist/database/files/000/dataset_8.dat

and:
http://140.253.78.234/galaxy/api/libraries/f2db41e1fa331b3e/contents/1cd8e2f6b131e891
shows the file has been uploaded:

{

"data_type": "txt",

"date_uploaded": "2013-05-09T04:41:10.579602",

"file_name": "/home/galaxy/galaxy-dist/database/files/000/dataset_8.dat",

"file_size": 76,

"genome_build": "?",

"id": "1cd8e2f6b131e891",

"ldda_id": "1cd8e2f6b131e891",

"message": "",

"metadata_data_lines": 8,

"metadata_dbkey": "?",

"misc_blurb": "8 lines",

"misc_info": "uploaded txt file",

"model_class": "LibraryDataset",

"name": "sam2.txt",

"template_data": {},

"uploaded_by": "t...@test.com",

"uuid": null

}










However, looking in the histories:





http://140.253.78.234/galaxy/api/histories/33b43b4e7093c91f/contents/a799d38679e9

[galaxy-dev] uploading large datasets into a new instance of Galaxy on a Mac Pro

2013-05-31 Thread Elwood Linney 
Hello,
I installed Galaxy on a new Mac Pro with 64gb of ram and a solid state
first drive and 3 , 2tb normal drives.

I can upload from my desktop a smaller file like a reference list or a
cuffdiff file but when I try to load a 16gb fastq file it just does not
load.

Given that I have my files on the same computer that has Galaxy, and given
that I am not well versed in unix, would anyone have a suggestion as to
ways I could easily upload these large files.

I would like to think I am just naive and that this has a simple solution.
Hopefully this is the correct list to pose this question.

Thank you for any help,

el linney

For those in the no, I did see this appear on the terminal after I tried to
upload the file through the browser mechanism for upload:

"Overflow Error:  signed integer is greater than maximum"
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Storage question for Galaxy-Admins

2013-05-31 Thread Adam Brenner 
Raj,

I agree with the points Alex made. It would be much easier to start
small and then work your way up.

I recently setup galaxy for our HPC cluster for the entire school of
medicine. Its only a few ~month into our setup but we are currently
using ~300GB of storage. This is part of our larger cluster storage
pool of 600TB.

We use the gluster filesystem across 6 storage nodes. We are trying
fhgfs and other file systems as well (lustre, ceph, etc).

If you have any other questions, feel free to ask.
-Adam

--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences

Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/

University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu

On Wed, May 29, 2013 at 10:34 PM, Raj Ayyampalayam  wrote:
> Hello,
>
> I am trying to come up with a number for our local galaxy storage
> requirements when we go live (This is an installation for research at our
> university).
>
> I checked out the survey that was done a while back for the GalaxyAdmins. It
> seems that the bigger installations have allocated about 200TB for their
> installation.
>
> It would be of great help to me if I can get some feedback on storage used
> by local galaxy installations.
> If possible please use the following format:
>
> Total capacity:
> Used capacity:
> Storage Technology:
> Comments:
>
> Thank you all very much.
> -Raj
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] uploading large datasets into a new instance of Galaxy on a Mac Pro

2013-05-31 Thread Adam Brenner 
How are you trying to upload the data to Galaxy? Are you using the web
interface? Or...?

--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences

Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/

University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu

On Fri, May 31, 2013 at 1:59 PM, Elwood Linney  wrote:
>
> Hello,
> I installed Galaxy on a new Mac Pro with 64gb of ram and a solid state first
> drive and 3 , 2tb normal drives.
>
> I can upload from my desktop a smaller file like a reference list or a
> cuffdiff file but when I try to load a 16gb fastq file it just does not
> load.
>
> Given that I have my files on the same computer that has Galaxy, and given
> that I am not well versed in unix, would anyone have a suggestion as to ways
> I could easily upload these large files.
>
> I would like to think I am just naive and that this has a simple solution.
> Hopefully this is the correct list to pose this question.
>
> Thank you for any help,
>
> el linney
>
> For those in the no, I did see this appear on the terminal after I tried to
> upload the file through the browser mechanism for upload:
>
> "Overflow Error:  signed integer is greater than maximum"
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/