Re: [galaxy-dev] In which Encryption form does Galaxy store password?

2013-01-07 Thread Ido Tamir
Its hashed (sha1) and you can not reverse the hash. Storing passwords in a 
reversible manner is bad practice. 
http://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/util/hash_util.html

You could maybe authenticate your users from the other program with the same 
method against the hashed passwords.

best,
ido

On Jan 7, 2013, at 12:43 PM, Sachit Adhikari wrote:

 Hi,
 
 In which encryption galaxy store the password? It's really urgent. I want to 
 decode the password and use in the other program. Thanks
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] fastq_combiner

2013-01-07 Thread Jennifer Hillman-Jackson

Hello,

The tool you are looking for is:

 galaxy-dist / tools / fastq / fastq_combiner.py
 galaxy-dist / tools / fastq / fastq_combiner.xml


Hopefully this helps. Next time, please be sure to send questions 
directly to the galaxy-...@bx.psu.edu email address. Your question went 
to galaxy-dev-ow...@lists.bx.psu.edu originally, which is not the 
mailing list, but a way to reach the list moderators if there is a 
technical problem.


Take care,

Jen
Galaxy team


On 1/7/13 3:09 AM, wei liu wrote: Dear all,

 I have used the local galaxy, and I want to find out which script is
 used to Combine FASTA and QUAL
 https://main.g2.bx.psu.edu/tool_runner?tool_id=fastq_combiner into
 FASTQ. I have tried the scripts in
 galaxy-dist/tools/next_gen_conversion/solid2fastq.py, but it can not
 merge the several lines which belong to the same sequence.

 Looking forward for your reply!

 Best Regards,
 Wei Liu.

--
Jennifer Hillman-Jackson
Galaxy Support and Training
http://galaxyproject.org
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


Re: [galaxy-dev] (no subject)

2013-01-07 Thread Jennifer Hillman-Jackson

Security warning: DO NOT CLICK on the link in this thread.

Rehan Saleem - it is likely that your original sending hotmail email 
account has been compromised. We have removed it from the 
galaxy-...@bx.psu.edu mailing list to prevent further unmoderated posts 
until the problem is cleared. Contacting your local system administrator 
for help is one place to start. Anyone who did click on the link already 
should do the same, as this is usually how such viruses spread, although 
much depends on OS and other factors.


Good luck!

Jen
Galaxy team

--
Jennifer Hillman-Jackson
Galaxy Support and Training
http://galaxyproject.org
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


Re: [galaxy-dev] Stumped by GridOperation

2013-01-07 Thread James Taylor
Ted, have you considered if you can do what you need with the 'allowed'
method of GridOperation? It let's you define a callable condition that
determines if that operation is allowed on a given item. It does not have
access to trans but it would be reasonable to add that.


--
James Taylor, Assistant Professor, Biology/CS, Emory University


On Mon, Jan 7, 2013 at 2:55 AM, Ted Goldstein t...@soe.ucsc.edu wrote:

 HI Greg,
 I thought of this. But there is potential bug in it which has security
 implications.  Though it is a low probability event, I have seen this sort
 of leakage happen in other environments where user data was stuck into
 globally shared objects.   The grid object is shared by all requests.  You
 are storing user specific data in the GridOperation. So if there is an
 unfortunate context switch in the middle of the setting the ids int the
 grid operation but before they are used,  the user specific data of one
 user will be visible to the other user.   Of course the solution is to make
 sure the grid is not shared (e.g. it is instantiated or  deep-copied) in
 the function or at least in the request thread that uses it.

 Thanks,  I will do this with on a deepcopy of the Grid to ensure that it
 doesn't et shared between threads.   You might want to chsnge the code in
 repository.py

 Best regards,

 Ted


 On Jan 6, 2013, at 4:12 PM, Greg Von Kuster wrote:

 Hi Ted,

 If I understand what your trying to do, I believe I've implemented grid
 operations similar to what your attempting in several places in the Galaxy
 tool shed, so checking the various grid classes and associated methods in
 /lib/galaxy/webapps/community/controllers will give you some examples.

 The trick is that you need to define the grid operations outside of the
 grid class itself, but in the method that calls the grid (but just before
 the call to display it) so the request parameters will include the selected
 page ids.

 For an example of what I'm talking about, see the following method on
 about line #740 in my
 ~/lib/galaxy/webapps/community/controllers/repository.py code file.

 Here is the relevant snippet of code.

 def browse_valid_repositories( self, trans, **kwd ):
 ...irrelevant code not displayed...
 repository_id = None
 for k, v in kwd.items():
 changset_revision_str = 'changeset_revision_'
 if k.startswith( changset_revision_str ):
 repository_id = trans.security.encode_id( int( k.lstrip(
 changset_revision_str ) ) )
 repository = suc.get_repository_in_tool_shed( trans,
 repository_id )
 if repository.tip( trans.app ) != v:
 return trans.response.send_redirect( web.url_for(
 controller='repository',

 action='preview_tools_in_changeset',

 repository_id=trans.security.encode_id( repository.id ),

 changeset_revision=v ) )
 url_args = dict( action='browse_valid_repositories',
  operation='preview_tools_in_changeset',
  repository_id=repository_id )
 self.valid_repository_grid.operations = [ grids.GridOperation( 
 Preview
 and install,

 url_args=url_args,

 allow_multiple=False,

 async_compatible=False ) ]
 return self.valid_repository_grid( trans, **kwd )

 Sorry if I misunderstood your dilemma and this does not help you.

 Greg Von Kuster

 On Jan 6, 2013, at 4:52 PM, Ted Goldstein wrote:

 Hi,
 Based on a suggestion by James Taylor, I am working on extending the
 security model so that pages can use the same RBAC permission system used
 in libraries. In fact, many of my pages have assets (images, etc) which are
 stored in a library. So this is a natural model. Therefore I want to make a
 connection between Pages and Libraries. that is visible to the user and
 extends the security model of page access. The owner of the page
 essentially grants anyone with access to a library access to the page as
 well. This part is going well.  It is few lines of code in the page access
 security code.

 The implementation is to that I created a many-to-many relationship table
 LibraryPageAssociation, that  Connects Page objects to Library Objects.
  Grid objects provide a convenient interface to selecting multiple of each.
 In fact, I just extend the  regular PageListGrid  instantiated  by the
 /page/list  action  to select the pages by adding a  new GridOperation
 labeled Select Libraries  which then brings up new Grid LibrarySelectGrid
 (similar to the one visualization.py, but simpler) it has one GridOperation
 labeled Connect Pages to Libraries   (allows_multi=True).

 Here is the problem:  I can pass the page ids (enoded) into the mako page
 that carries the LibrarySelectGrid.  But I don't see a way to
 parametriize  Connect Pages to Libraries  GridOperation to pass anything
 out of the page except the Grid Sttate itself.   Do I have to extend
 GridOperation or am I missing something?  I have tried a number of things
 with url_args and 

Re: [galaxy-dev] user creation using API

2013-01-07 Thread John Chilton
I am pretty confident there is no way to do this via the API
currently, though I imagine it wouldn't be terribly complicated to add
and the community would appreciate the contribution. If API-based
access isn't a strong requirement and you have direct access to the
Galaxy server here is a script-template I have used to automatically
create users:

https://gist.github.com/4475646

The above gist is probably enough to get going, but here is the full
context of how I create such a file if that is of interest:
https://github.com/jmchilton/cloudbiolinux/blob/deploy/cloudbio/deploy/galaxy.py

-John

On Mon, Jan 7, 2013 at 3:37 AM, Hagai Cohen haga...@gmail.com wrote:
 Hi,
 I would like to auto create users on my local Galaxy instance.

 I saw that in the create user on the API, I should enable use_remote_user
 (which as I understand means that I have to use Apache or nginx).
 Is there another way besides using Apache to create users automatically?

 Thanks,
 Hagai


 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] backtrace when attempting to view a shared workflow

2013-01-07 Thread Daniel Blankenberg
Hi Brad,

Thanks for reporting this issue. I've committed what I hope is a fix for your 
issue in changeset 8527:eb4dac39eed5 in galaxy-central.  Once you are able to 
apply these changes, would you mind letting us know if this resolves your 
broken workflows? 


Thanks for using Galaxy,

Dan


On Jan 7, 2013, at 9:11 AM, Langhorst, Brad wrote:

 I've recently updated to the december 20th revision of galaxy and still have 
 a problem with a few workflows.
 You can find the backtraces below.
 
 based on these I'm starting to look in 
 /mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/workflow/modules.py - but any 
 hints would be appreciated.
 
 
 thanks
 
 Brad
 
 here's a backtrace from an attempt to edit the problem workflow 
 
 Error - type 'exceptions.TypeError': 'NoneType' object is unsubscriptable
 URL: 
 http://galaxy.neb.com/workflow/load_workflow?id=4280bfc7d7999071_=1357567007122
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
  line 143 in __call__
   app_iter = self.application(environ, start_response)
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
  line 80 in __call__
   return self.application(environ, start_response)
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py',
  line 91 in __call__
   return self.app( environ, start_response )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
  line 632 in __call__
   return self.application(environ, start_response)
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 
 160 in __call__
   body = method( trans, **kwargs )
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', 
 line 73 in decorator
   return simplejson.dumps( func( self, trans, *args, **kwargs ) )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/workflow.py',
  line 676 in load_workflow
   module = module_factory.from_workflow_step( trans, step )
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/workflow/modules.py', line 
 402 in from_workflow_step
   return self.module_types[type].from_workflow_step( trans, step )
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/workflow/modules.py', line 
 217 in from_workflow_step
   module.state.inputs = module.tool.params_from_strings( step.tool_inputs, 
 trans.app, ignore_errors=True )
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py', line 
 2130 in params_from_strings
   return params_from_strings( self.inputs, params, app, ignore_errors )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/tools/parameters/__init__.py', 
 line 94 in params_from_strings
   value = params[key].value_from_basic( value, app, ignore_errors )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/tools/parameters/grouping.py', 
 line 444 in value_from_basic
   current_case = rval['__current_case__'] = value['__current_case__']
 TypeError: 'NoneType' object is unsubscriptable
 
 
 here's one attempting to run it
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
  line 143 in __call__
   app_iter = self.application(environ, start_response)
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
  line 80 in __call__
   return self.application(environ, start_response)
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py',
  line 91 in __call__
   return self.app( environ, start_response )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
  line 632 in __call__
   return self.application(environ, start_response)
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 
 160 in __call__
   body = method( trans, **kwargs )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/workflow.py',
  line 1473 in run
   step.module = module_factory.from_workflow_step( trans, step )
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/workflow/modules.py', line 
 402 in from_workflow_step
   return self.module_types[type].from_workflow_step( trans, step )
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/workflow/modules.py', line 
 217 in from_workflow_step
   module.state.inputs = module.tool.params_from_strings( step.tool_inputs, 
 trans.app, ignore_errors=True )
 File '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py', line 
 2130 in params_from_strings
   return params_from_strings( self.inputs, params, app, ignore_errors )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/tools/parameters/__init__.py', 
 line 94 in params_from_strings
   value = params[key].value_from_basic( value, app, ignore_errors )
 File 
 '/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/tools/parameters/grouping.py', 
 line 444 in value_from_basic
   current_case = rval['__current_case__'] = value['__current_case__']
 

Re: [galaxy-dev] Uploading large files to history through API (some random thoughts)

2013-01-07 Thread Kyle Ellrott
The pull request for this patch is still on the queue. Is anything
happening with this?

Kyle


On Tue, Nov 27, 2012 at 8:56 AM, John Chilton chil...@msi.umn.edu wrote:

 I went down something of a rabbit hole last night, I thought that
 uploading files through the API using multipart/form-data worked. It
 is only at the very end of my adventure that I realized it only works
 for libraries uploads. I think it would be great to get that working
 to histories as well. I cannot spend much more time on this right now
 (I thought I only needed to spend an hour to implement the client side
 stuff), but I thought I would post the progress I made here in case
 someone wants to take up the fight someday:

 The first thing is that when issuing multipart/form-data requests,
 inputs are not currently being deserialized from JSON (so for instance
 in the tools api code the inputs variable would be the string
 containing the json, not the python dictionary).

 To bring multipart/form-data requests in line with other content
 requests I modified this in ilb/galaxy/web/framework/__init__.py

 payload = kwargs.copy()
 named_args, _, _, _ = inspect.getargspec(func)
 for arg in named_args:
 payload.pop(arg, None)

 with this:

 payload = kwargs.copy()
 named_args, _, _, _ = inspect.getargspec(func)
 for arg in named_args:
 payload.pop(arg, None)
 for k, v in payload.iteritems():
 if isinstance(v, (str, unicode)):
 try:
 payload[k] = simplejson.loads(v)
 except:
 # may not actually be json, just continue
 pass
 payload =
 util.recursively_stringify_dictionary_keys( payload )

 One could also imagine doing this replacement in the tools api
 controller directly on payload['input'] instead.

 After that change, files still weren't being matched up with inputs
 properly:

 This debug statement I added to upload_common.py, demonstrates that
 file_data is None.

 galaxy.tools.actions.upload_common INFO 2012-11-27 00:22:17,585
 Uploaded datasets is {'NAME': u'galxtest694734465387762969.txt',
 'file_data': None, 'space_to_tab': None, 'url_paste': None,
 '__index__': 0, 'ftp_files': None}

 To address this, the files_*|file_data parameters need to be moved
 inside of the inputs dict.

 That means in lib/galaxy/webapps/galaxy/api/tools.py, changing this:

 inputs = payload[ 'inputs' ]

 To this:

 inputs = payload[ 'inputs' ]
 for k, v in payload.iteritems():
 if k.startswith(files_):
 inputs[k] = v

 Then the debug line becomes this:

 galaxy.tools.actions.upload_common INFO 2012-11-27 00:33:15,168
 Uploaded datasets is {'NAME': u'galxtest2484272839208214846.txt',
 'file_data': FieldStorage('files_0|file_data',
 u'galxtest2484272839208214846.txt'), 'space_to_tab': None,
 'url_paste': None, '__index__': 0, 'ftp_files': None}

 Which matches pretty well with the same line coming from a web browser
 request:

 galaxy.tools.actions.upload_common INFO 2012-11-27 00:23:57,590
 Uploaded datasets is {'NAME': u'', 'file_data':
 FieldStorage('files_0|file_data', u'second_step.png'), 'space_to_tab':
 None, 'url_paste': u'', '__index__': 0, 'ftp_files': None}

 These changes aside I still wasn't it sill didn't work, that is why
 this is a rambling e-mail and not a pull request. I got this exception
 http://pastebin.com/hhs1pjtP. I guess something to do with the session
 handling stuff being different between API calls and normal web calls.

 Anyway, I've inspected the actual requests I was generating and I
 think they are reasonable, Galaxy just needs to be augmented to accept
 them :). If someone does land up taking a look at this, I have
 committed my test case to blend4j so it can be used to really quickly
 test such a client request (requires git, Java, and maven):

 % git checkout g...@github.com:jmchilton/blend4j.git
 % cd blend4j
 % mvn test -Dtest=ToolsTest
 -Dtest.galaxy.instance=http://localhost:8080/
 -Dtest.galaxy.key=testapikey

 Thanks,
 -John
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Transfer Manager

2013-01-07 Thread Kyle Ellrott
I'm trying to figure out if I can do this all through the API (so I can
skip setting up FTP servers and sharing database servers).
I can scan one system, and initiate downloads on the destination system
(using upload1). So as far as moving files from one machine to another, it
should be fine. I could push all the data, with correct names, annotations
and tags.
But then it becomes a matter of the pushing the metadata onto the object on
the destination system. But there are two problems,
1) No way to push tool info and input parameter data, everything would be a
product of 'upload1'.
2) No global IDs. If I try to sync a second time, it may be difficult to
figure out which elements have been previously pushed. Comparing names
could lead to problems...

Kyle



On Fri, Jan 4, 2013 at 4:54 AM, Rémy Dernat remy...@gmail.com wrote:

 Hi,

 A good practise is to create a ftp server
 http://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP and use a
 tool to send / retrieve informations to this ftp server :
 http://toolshed.g2.bx.psu.edu/
 - Data Source / data_nfs

 Then export your ftp directory by NFS to your galaxy installations.

 For databases, it is a little bit complex. If you have a database server,
 you could share access to a single database, but your installation between
 your server should be the same, and all working directories must be shared
 with NFS on all galaxy servers...

 Regards


 2012/12/6 Kyle Ellrott kellr...@soe.ucsc.edu

 Is there any documentation on the transfer manager?
 Is this a mechanism that I could use to synchronize data libraries
 between two different Galaxy installations?

 Kyle

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/



___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Stumped by GridOperation

2013-01-07 Thread Ted Goldstein
Hi  Greg and James,
This has become a plumbing project.  What starts off as fixing a leaky faucet 
ends being a $200 bill replacing the entire sink ;-).  But I don't see anyway 
around it.

Yes, there is a lot of power in the allowed method since it uses a function or 
a lambda. There are two problems with this: idea (1) it feels wrong to have a 
predicate method called allowed modify the state of the object.
and worse,(2)  the Python context that the lambda is capturing is at global 
instantiation time.  But the data that I need captured and saved in the page 
exist only after the user has made a set of choices. Thus it only exists in a 
previous call to thePage.list() method.

Another problem. 
Since this is a multi operation, the state that needs to be passed cannot go 
into the  form URL itself of the form. I see that the call to get the URL  of 
the form in grid_base.mako at line 702 does not take any operation arguments.  
Which is correct.  Normally the submit button on an HTML form does not modify 
the form's URL  The usual solution from an HTML perspective is to have hidden 
input fields on an HTML form.   So the solution would be to extend the Grid 
class to generate and use hidden input fields.  This would be done on  a copy 
of the grid object;. So there needs to be a new Grid method.

(my changes in bold)

class Grid
...
def clone_with_pass_through_data(self, data_dict):
# make a copy of myself for use in passing though data
clone = deepcopy(self)
clone.pass_through_data = data_dict
return clone


my  example use in Page.list()
   # Clone a copy of grid to have the pageids.
grid = 
self._libraries_selection_grid.clone_with_pass_through_data({pageids: 
pageidsraw});
return trans.fill_template( 
page/library_datasets_select_grid.mako, pagetitles=pagetitles, grid=grid, 
**kwargs)

pass_through_data   has a three line implementation in grid_base.mako
   form action=${url()} method=post onsubmit=return false;
input type=hidden name=webapp value=${webapp}/

  %for pass_name, pass_value in grid.pass_through_data.items():
input type=hidden name=${pass_name} value=${pass_value}/
%endfor
 ..

But  nothing is simple, form parameters  arguments are more complex than they 
sem at first.


  function go_to_URL() {
// Not async request.
url_args['async'] = false;

// Build argument string.
var arg_str = ;
for (var arg in url_args) {
arg_str = arg_str + arg + = + url_args[arg] + ;
}

// add in hidden fields
$('input[type=hidden]').each(function() {
if (this.name == webapp || this.name == select_all_checkbox)
   return
else
arg_str = arg_str + this.name + = + this.value + ;
})


// Go.
window.location = encodeURI( ${h.url_for()}? + arg_str );
}


What do you think of this idea?

Ted
   


On Jan 7, 2013, at 7:00 AM, James Taylor wrote:

 Ted, have you considered if you can do what you need with the 'allowed' 
 method of GridOperation? It let's you define a callable condition that 
 determines if that operation is allowed on a given item. It does not have 
 access to trans but it would be reasonable to add that. 
 
 
 --
 James Taylor, Assistant Professor, Biology/CS, Emory University
 
 
 On Mon, Jan 7, 2013 at 2:55 AM, Ted Goldstein t...@soe.ucsc.edu wrote:
 HI Greg,
 I thought of this. But there is potential bug in it which has security 
 implications.  Though it is a low probability event, I have seen this sort of 
 leakage happen in other environments where user data was stuck into globally 
 shared objects.   The grid object is shared by all requests.  You are storing 
 user specific data in the GridOperation. So if there is an unfortunate 
 context switch in the middle of the setting the ids int the grid operation 
 but before they are used,  the user specific data of one user will be visible 
 to the other user.   Of course the solution is to make sure the grid is not 
 shared (e.g. it is instantiated or  deep-copied) in the function or at least 
 in the request thread that uses it.   
 
 Thanks,  I will do this with on a deepcopy of the Grid to ensure that it 
 doesn't et shared between threads.   You might want to chsnge the code in 
 repository.py
 
 Best regards,
 
 Ted
 
 
 On Jan 6, 2013, at 4:12 PM, Greg Von Kuster wrote:
 
 Hi Ted,
 
 If I understand what your trying to do, I believe I've implemented grid 
 operations similar to what your attempting in several places in the Galaxy 
 tool shed, so checking the various grid classes and associated methods in 
 /lib/galaxy/webapps/community/controllers will give you some examples.
 
 The trick is that you need to define the grid operations outside of the grid 
 class itself, but in the method that calls the grid 

[galaxy-dev] Galaxy submitting jobs to remote cluster

2013-01-07 Thread Thomas Uram
Hello:

I am trying to configure Galaxy to submit jobs to a remote cluster 
(specifically, an XSEDE resource to which I don't have administrative access). 
I'm familiar with LWR and have read another response about configuring the 
Galaxy host as a PBS submit host. What is the best approach to accomplish what 
I'm after? 

I have experimented with the cli job runner with the SecureShell and Torque 
plugins. With some minimal modifications, I can run jobs via this mechanism, 
but file staging fails. I haven't looked into the details of the staging 
failure yet. Is this a viable approach? Is the cli job runner supported? (the 
universe_wsgi.ini file says available job runners include pbs and drmaa)

I've found a similar question on the galaxy-dev mailing list, with mention of 
some relevant recent work:

http://dev.list.galaxyproject.org/Remote-Job-Submission-from-Local-Install-td4655144.html

What is the status of the work referenced in that post?


Thanks, 
Tom Uram

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] File import recovery

2013-01-07 Thread Kyle Ellrott
Editing the DB and manually fixing all the errors in the UI doesn't seem
like a long term solution.
Is there a way to do this through the API? If not, where should I start
working to add it in?

Kyle


On Wed, Jan 2, 2013 at 7:41 AM, Nate Coraor n...@bx.psu.edu wrote:

 On Dec 31, 2012, at 8:18 PM, Kyle Ellrott wrote:

  I'm currently adding a large number of files into my Galaxy instance's
 dataset library. During the import some of the files (a small percentage)
 failed with:
 
  /inside/depot4/galaxy/set_metadata.sh: line 4: 14790 Segmentation fault
  (core dumped) python ./scripts/set_metadata.py $@
 
  I think it's probably standard cluster shenanigans, and may work just
 fine if run again. But there doesn't seem to be a way retry. Is there a way
 to deal with this that is easier than manually deleting and re-uploading
 the offending files?

 Hi Kyle,

 Unfortunately, there's not going to be a way to do this entirely in the
 UI.  Your best shot is to change the state of the datasets in the database
 from 'error' to 'ok' and then try using the metadata auto-detect button in
 the UI.

 --nate

 
  Kyle
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
 
   http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Tool tests

2013-01-07 Thread Cameron Jack
Hi, I hope I'm writing to the right area (if not, please point me in the 
right direction and I'll go on my way).


I have a set of tools that our team have built and been using for 
functional genomics analysis, which I would like to integrate into Galaxy.


My first concern is with the tests. Our code is unit tested but one 
script is tasked with downloading gene sets from Ensembl. We have a 
local copy of much of the Ensembl DB (5TB worth) and even this is fairly 
slow to pull from (5-10 mins). Pulling from the publicly available 
Ensembl site would be horrifically slow. Since it's stated here: 
http://wiki.galaxyproject.org/Admin/Tools/Writing%20Tests that 
functional tests are required, I'm looking for suggestions as to how to 
meet this requirement without taking up an hour of the test runner's 
time (for one tool).


My second question is if a file is produced by a script and given a 
specific name based on its inputs, how should I (if at all) define the 
output of this script.


My last question is how much functional testing is required?. I have a 
script here whose job is to produce plots of data. There are about 30 
command-line options and many of these have multiple choices available 
to them. Do I need to provide functional tests for every one of these? I 
can imagine having to provide 100 PNG files to compare my output again


I have searched for answers to the above (made much more tricky thanks 
to Samsung), I don't wish to waste anyone's time.


Many thanks in advance,
Cameron


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] variables not defined

2013-01-07 Thread Jeremy Goecks
These bugs have been fixed in our development repository (galaxy-central) and 
will be available in the next distribution, which should occur in the coming 
week.

Thanks,
J.


On Jan 7, 2013, at 6:23 AM, MONJEAUD wrote:

 Hello all,
 
 I am using my own Galaxy instance and I had a problem with page and 
 visualization tabs.
 For example, when I want to edit attributes of a page, I've got this error 
 traceback : 
 
 NameError: global name 'VALID_SLUG_RE' is not defined
 
 and when I want to remove or share a page or a visualization, I've got  this 
 one :
 
 NameError: global name 'util' is not defined
 
 To fix these errors, I added in the 
 lib/galaxy/webapps/galaxy/controllers/page.py and 
 lib/galaxy/webapps/galaxy/controllers/visualization.py these lines :
 
 from galaxy import util
 import re
 VALID_SLUG_RE = re.compile( ^[a-z0-9\-]+$ )
 
 So, I want to know if you know these problems or if it's only for my instance 
 of galaxy. Maybe some files (where these variables are defined) are not 
 present in my own galaxy.
 
 Cheers,
 Cyril Monjeaud
 -- 
 
 Cyril Monjeaud
 Equipe Symbiose / Plate-forme GenOuest
 Bureau D156
 IRISA-INRIA, Campus de Beaulieu
 35042 Rennes cedex, France
 Tél: +33 (0) 2 99 84 74 17
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] anybody seen 403s after a recent upgrade?

2013-01-07 Thread Langhorst, Brad
galaxy.web.framework WARNING 2013-01-07 22:06:30,044 User logged in as '(null)' 
externally, but has a cookie as 'langho...@neb.commailto:langho...@neb.com' 
invalidating session
10.254.254.86 - - [07/Jan/2013:22:06:30 -0400] GET 
/api/histories/4c8cd68e0b9ed4a7 HTTP/1.1 403 - http://galaxy.neb.com/history; 
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20100101 
Firefox/17.0

i did not observe this before today… but maybe I've screwed things up in my hg 
fiddling today.
Did anybody else see this?

manual refreshes work, but automatic ones don't


Brad
--
Brad Langhorst
langho...@neb.commailto:langho...@neb.com





___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/