Re: [galaxy-dev] Re Acutal user code

[Chorny, Ilya]

Everything seems to be working correctly, including splitting.

-Original Message-
From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, Ilya
Sent: Wednesday, November 02, 2011 9:25 AM
To: Nate Coraor (n...@bx.psu.edu)
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Re Acutal user code

I am pulling and testing as we speak. I did a preliminary test on my VM but it 
does not have enough memory to run BWA. We pull into my dev version on the 
cluster and do further testing. Also need to test how it integrates with the 
splitting code. Should have some results by EOD.

Ilya


-Original Message-
From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
Sent: Wednesday, November 02, 2011 9:15 AM
To: Chorny, Ilya
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Re Acutal user code

Hi Ilya,

I made a few more commits this morning.  Could you test it out in the present 
state?  If everything works properly in your environment, I think it's pretty 
close to being ready to integrate.  The major outstanding issue is to try to 
avoid chmods in the upload action, which I have some ideas about.

--nate

Nate Coraor (n...@bx.psu.edu) wrote:
> Nate Coraor (n...@bx.psu.edu) wrote:
> > Chorny, Ilya wrote:
> > > So it looks like for some reason galaxy is not picking up the information 
> > > in the .e file. Not sure why. I even switched it back to writing the 
> > > files to the pbs directory and it still did not pick up the .e files. BWA 
> > > was core dumping but I was not given the error message in galaxy. 
> > 
> > Okay, this is pretty odd, I'll try to see what's going on here.  If 
> > you figure anything out, please let me know.
> 
> Whoops, variable name collision in finish().  I've just committed a fix.
> 
> > 
> > > BTW, I find it nice to be able to see the .o and .e files. The way you 
> > > have it coded the .o and .e are written out to the job_working_directory 
> > > and then disappear at the end of the run.  But the way I had it coded the 
> > > pbs directory is world read writable. Is there an issue with having the 
> > > pbs directory be world read write?
> > 
> > I imagine for some environments, it's preferable to avoid 
> > world-writable directories.  Also, going forward, containing all of 
> > a job's filesystem data to a single directory will be important for 
> > running on multiple clusters without access to a single shared 
> > filesystem mounted at the same path.
> > 
> > There's an issue in bitbucket somewhere regarding saving standard 
> > out and standard error for all jobs, which would solve the problem 
> > of it being in the working directory.
> > 
> > For debugging, if you comment out shutil.rmtree( 
> > self.working_directory ) in jobs.cleanup(), it'll leave everything behind.
> > 
> > --nate
> > 
> > > 
> > > Thanks,
> > > 
> > > Ilya
> > > 
> > > 
> > > -Original Message-
> > > From: galaxy-dev-boun...@lists.bx.psu.edu
> > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > Ilya
> > > Sent: Monday, October 31, 2011 8:37 PM
> > > To: Nate Coraor (n...@bx.psu.edu)
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > 
> > > It seems to be working but when I run BWA or Bowtie it finishes green but 
> > > the files are empty. Doesn't give me much information. I also tried 
> > > fastqc and it ran just fine.
> > > 
> > > -Original Message-
> > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > Sent: Monday, October 31, 2011 1:21 PM
> > > To: Chorny, Ilya
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: Re Acutal user code
> > > 
> > > Chorny, Ilya wrote:
> > > > Sweet!! Can't wait to see it.
> > > 
> > > Have a look:
> > > 
> > > https://bitbucket.org/natefoo/galaxy-central-ichorny/
> > > 
> > > Caveats:
> > > 
> > > * job_working_directory has to be in a shared filesystem
> > > * outputs_to_working_directory must be True
> > > * set_metadata_externally must be True
> > > 
> > > One thing that would be good from a security standpoint would be if the 
> > > external chown script took a Galaxy Job ID for its argument instead of a 
> > > path, and then looked up the working directory path from the ID.  That 
> > > way the script couldn't be used maliciously to chown any file.
> > > 
> > > --nate
> > > 
> > > > 
> > > > Thanks,
> > > > 
> > > > Ilya
> > > > 
> > > > -Original Message-
> > > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > > Sent: Thursday, October 27, 2011 1:24 PM
> > > > To: Chorny, Ilya
> > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > Subject: Re: Re Acutal user code
> > > > 
> > > > Yup, cleaning up some things that were accidentally lost in merges.  I 
> > > > added another bitbucket repo so as not to mess up yours.
> > > > 
> > > > Chorny, Ilya wrote:
> > > > > Hi Nate,
> > > > > 
> > > > > Any updates?
> > > > > 
> > > > > Thanks,
> > > 

Re: [galaxy-dev] Local Galaxy Instance MarkupSafe error

[Jerico Nico De Leon Revote]

Hi Nate,

Yes, that was the issue. I just made sure that the cluster nodes are using
the same Python as the application server and it works now.
Thanks for the support.

Regards,

Jerico

On 3 November 2011 04:18, Nate Coraor  wrote:

> Jerico Nico De Leon Revote wrote:
> > It's the same case as what I'm getting. I can see the output via "eye"
> icon
> > on the history panel and able to download the files as well.
>
> Do your cluster nodes have internet access?  If so, log into a node and
> run the command again from there.  Your nodes may have a different
> Python version or Unicode byte order encoding scheme than your Galaxy
> application server.
>
> --nate
>
> >
> > On 1 November 2011 03:37, Mickael ESCUDERO  >wrote:
> >
> > > Hi there,
> > >
> > > I'm getting exactly the same problem with any job running on a
> TORQUE/PBS
> > > cluster. The jobs actually run fine as I can see the output and
> download
> > > it, but it's marked as failed in the galaxy history, with the following
> > > message:
> > >
> > > WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako)
> cannot be fetched
> > >
> > > The command `python -ES ./scripts/fetch_eggs.py` gives nothing as
> output.
> > > If I run the same tools locally there is no problem.
> > >
> > > Cheers
> > > Micka
> > >
> > >
> > >> Message: 5
> > >> Date: Thu, 27 Oct 2011 15:08:17 +1100
> > >> From: Jerico Nico De Leon Revote 
> > >> To: galaxy-dev@lists.bx.psu.edu
> > >> Subject: [galaxy-dev] Local Galaxy Instance MarkupSafe error
> > >> Message-ID:
> > >><
> > >> cap9ulhyipxyb2cwtqosm3mquuu55arxy5por1dbahej1uco...@mail.gmail.com>
> > >> Content-Type: text/plain; charset="iso-8859-1"
> > >>
> > >> Hi,
> > >>
> > >> I'm just doing a simple get->data from UCSC on our local Galaxy
> > >> instance and got the following error:
> > >>
> > >> WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako)
> > >> cannot be fetched
> > >>
> > >> The job box then is displayed as red on the history panel.
> > >> The job runner states that the job finished normally on the cluster.
> > >> Galaxy is checkout from galaxy-central (changeset: 6176:34fffbf01183).
> > >>
> > >> Thanks,
> > >>
> > >> Jerico
> > >> -- next part --
> > >> An HTML attachment was scrubbed...
> > >> URL: <
> > >>
> http://lists.bx.psu.edu/pipermail/galaxy-dev/attachments/20111027/cd777ff3/attachment-0001.html
> > >> >
> > >>
> > >> --
> > >>
> > >> Message: 6
> > >> Date: Thu, 27 Oct 2011 16:58:13 +1100
> > >> From: Jerico Nico De Leon Revote 
> > >> To: galaxy-dev@lists.bx.psu.edu
> > >> Subject: Re: [galaxy-dev] Local Galaxy Instance MarkupSafe error
> > >> Message-ID:
> > >><
> > >> cap9ulhbenenkqc50uzyxoca4w-zucdy8obs4i6b3e2qyfmy...@mail.gmail.com>
> > >> Content-Type: text/plain; charset="iso-8859-1"
> > >>
> > >> Just to follow-up on this. The MarkupSafe egg is definitely present
> on the
> > >> eggs directory and the servers are ran through virtualenv.
> > >>
> > >> On 27 October 2011 15:08, Jerico Nico De Leon Revote <
> > >> jerico.rev...@monash.edu> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I'm just doing a simple get->data from UCSC on our local Galaxy
> > >> instance and got the following error:
> > >> >
> > >> > WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako)
> > >> cannot be fetched
> > >> >
> > >> > The job box then is displayed as red on the history panel.
> > >> > The job runner states that the job finished normally on the cluster.
> > >> > Galaxy is checkout from galaxy-central (changeset:
> 6176:34fffbf01183).
> > >> >
> > >> > Thanks,
> > >> >
> > >> > Jerico
> > >> >
> > >> >
> > >> -- next part --
> > >> An HTML attachment was scrubbed...
> > >> URL: <
> > >>
> http://lists.bx.psu.edu/pipermail/galaxy-dev/attachments/20111027/f8baadf6/attachment-0001.html
> > >> >
> > >>
> > >> --
> > >>
> > >> Message: 7
> > >> Date: Thu, 27 Oct 2011 02:40:51 -0400
> > >> From: Nate Coraor 
> > >> To: Jerico Nico De Leon Revote 
> > >> Cc: galaxy-dev@lists.bx.psu.edu
> > >> Subject: Re: [galaxy-dev] Local Galaxy Instance MarkupSafe error
> > >> Message-ID: <20111027064051.gg2...@bx.psu.edu>
> > >> Content-Type: text/plain; charset=us-ascii
> > >>
> > >> Jerico Nico De Leon Revote wrote:
> > >> > Hi,
> > >> >
> > >> > I'm just doing a simple get->data from UCSC on our local Galaxy
> > >> > instance and got the following error:
> > >> >
> > >> > WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako)
> > >> > cannot be fetched
> > >> >
> > >> > The job box then is displayed as red on the history panel.
> > >> > The job runner states that the job finished normally on the cluster.
> > >> > Galaxy is checkout from galaxy-central (changeset:
> 6176:34fffbf01183).
> > >>
> > >> Hi Jerico,
> > >>
> > >> Are you using a cluster?  If not, could you run:
> > >>
> > >>% python -ES ./scripts/fetch_eggs.py
> > >>
> > >> --nate
> > >>
> > >> >
> >

[galaxy-dev] Ucsc bigbed display tool does not show up on local installation

[Jeremy Villalobos]

Hello:

The display tool for UCSC that comes with the local installation package
does not work.  uploading a bigbed file (from the test-data that comes with
the package) does not trigger the webapp to show an option to view the file
using UCSC's viewer.

How do I enable this ?

To reproduce the issue, simply download galaxy using mercurial.  change the
host address to 0.0.0.0.  and load the bigbed file from the test-data
directory.
Once the file uploads, the user should expect to see a display tool added
to the files description (on the right), but nothing is shown.

Thanks

Jeremy
UNCC
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] galaxy citation_url setting

[Shantanu Pavgi]

On Nov 2, 2011, at 3:39 PM, Nate Coraor wrote:

> Shantanu Pavgi wrote:
>> 
>> Hi,
>> 
>> It seems like modification of 'citation_url'  in the universe_wsgi.ini 
>> config file has no effect in the UI (Help --> How to Cite Galaxy). Is it 
>> something hard-coded in the source? 
> 
> Hi Shantanu,
> 
> Whoops.  This has been fixed in 6205:9a9479f7e53f.
> 


Thanks Nate. I was wondering if 'lib/galaxy/config.py' file needs to be 
modified as well. 

{{{
$ hg diff ./lib/galaxy/config.py
diff -r 9e90faf2cb1c lib/galaxy/config.py
--- a/lib/galaxy/config.py  Wed Nov 02 14:15:08 2011 -0700
+++ b/lib/galaxy/config.py  Wed Nov 02 16:46:08 2011 -0500
@@ -113,6 +113,7 @@
 self.gbrowse_display_sites = kwargs.get( 'gbrowse_display_sites', 
"wormbase,tair,modencode_worm,modencode_fly,sgd_yeast" ).lower().split(",")
 self.genetrack_display_sites = kwargs.get( 'genetrack_display_sites', 
"main,test" ).lower().split(",")
 self.brand = kwargs.get( 'brand', None )
+   self.citation_url = kwargs.get('citation_url', 
'http://wiki.g2.bx.psu.edu/Citing%20Galaxy')
 self.support_url = kwargs.get( 'support_url', 
'http://wiki.g2.bx.psu.edu/Support' )
 self.wiki_url = kwargs.get( 'wiki_url', 'http://g2.trac.bx.psu.edu/' )
 self.blog_url = kwargs.get( 'blog_url', None )

}}}

Also, it seems like the default values are being passed twice here - in the 
mako templates and initialization method of Configuration class. I was 
wondering if default values can be passed only once during initialization and 
then other get methods would only query the necessary configuration option.   I 
don't know all the code in detail, so I might be wrong here. 

--
Shantanu. 



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Clusters, Runners, and user credentials

[Glen Beane]

On Nov 2, 2011, at 4:57 PM, Nate Coraor wrote:

> Glen Beane wrote:
>> Many of us are using the PBS job runner (for TORQUE) and would definitely be 
>> interested in a port. 
> 
> Technically, using the drmaa runner with TORQUE is supposed to work.  I
> just tried it here to test Ilya's code, and Galaxy was segfaulting when
> trying to interact with TORQUE's libdrmaa on setting up the job
> template.  I didn't look into it any further, I'm using a fairly old
> TORQUE client here, so I suspect it may just be due to that.

I don't believe TORQUE's DRMAA support receives that much attention these days

--
Glen L. Beane
Senior Software Engineer
The Jackson Laboratory
(207) 288-6153


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Looks like actual user breaks splitting

[Duddy, John]

I'll submit a pull request shortly...

John Duddy
Sr. Staff Software Engineer
Illumina, Inc.
9885 Towne Centre Drive
San Diego, CA 92121
Tel: 858-736-3584
E-mail: jdu...@illumina.com


-Original Message-
From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu] 
Sent: Wednesday, November 02, 2011 12:24 PM
To: Duddy, John
Cc: Chorny, Ilya; galaxy-dev@lists.bx.psu.edu
Subject: Re: Looks like actual user breaks splitting

John, Ilya,

I get further with sequence type inputs but it looks like
JobWrapper.get_output_datasets_and_fnames() is not returning the right
thing when outputs_to_working_directory = True

BTW, the base Data.split() method is broken after the updates to
Sequence.split() since it wasn't updated to expect
HistoryDatasetAssociations rather than filenames.  Could you take a look
at that when you get a chance?

--nate

Duddy, John wrote:
> The datatype you are using does not define a split method. Are you working 
> with our in-progress gz type or fastqillumina?
> 
> John Duddy
> Sr. Staff Software Engineer
> Illumina, Inc.
> 9885 Towne Centre Drive
> San Diego, CA 92121
> Tel: 858-736-3584
> E-mail: jdu...@illumina.com
> 
> From: Chorny, Ilya
> Sent: Wednesday, November 02, 2011 11:50 AM
> To: Duddy, John
> Cc: Nate Coraor (n...@bx.psu.edu); galaxy-dev@lists.bx.psu.edu
> Subject: Looks like actual user breaks splitting
> 
> Hey John,
> 
> Any thoughts?
> 
> Ilya
> 
> Traceback (most recent call last):
>   File 
> "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/runners/tasks.py", line 
> 73, in run_job
> tasks = splitter.do_split(job_wrapper)
>   File 
> "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/splitters/multi.py", 
> line 73, in do_split
> input_type.split(input_datasets, get_new_working_directory_name, 
> parallel_settings)
>   File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/datatypes/data.py", 
> line 473, in split
> raise Exception("Text file splitting does not support multiple files")
> Exception: Text file splitting does not support multiple files
> 
> Ilya Chorny Ph.D.
> Bioinformatics Scientist I
> Illumina, Inc.
> 9885 Towne Centre Drive
> San Diego, CA 92121
> Work: 858.202.4582
> Email: icho...@illumina.com
> Website: www.illumina.com
> 
> 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Clusters, Runners, and user credentials

[Nate Coraor]

Glen Beane wrote:
> Many of us are using the PBS job runner (for TORQUE) and would definitely be 
> interested in a port. 

Technically, using the drmaa runner with TORQUE is supposed to work.  I
just tried it here to test Ilya's code, and Galaxy was segfaulting when
trying to interact with TORQUE's libdrmaa on setting up the job
template.  I didn't look into it any further, I'm using a fairly old
TORQUE client here, so I suspect it may just be due to that.

--nate

> 
> How do you deal with making sure the user's environment is configured 
> properly? We use a python virtualenv and load specific module files with 
> tested tool versions in our galaxy users startup scripts on our cluster. 
> 
> Sent from my iPhone
> 
> On Oct 31, 2011, at 6:29 PM, "Chorny, Ilya"  wrote:
> 
> > BTW, I am not sure if PBS works with drmaa. If not then the code will need 
> > to be ported to work with pbs.
> > 
> > Ilya
> > 
> > 
> > -Original Message-
> > From: galaxy-dev-boun...@lists.bx.psu.edu 
> > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, Ilya
> > Sent: Monday, October 31, 2011 3:27 PM
> > To: Lloyd Brown; Galaxy Dev List
> > Subject: Re: [galaxy-dev] Clusters, Runners, and user credentials
> > 
> > Lyod,
> > 
> > See Nate's email below Title: Actual user code. We have been working on 
> > implementing this feature in galaxy. The code is still in development but 
> > feel free to test it out and let us know how it works for you.
> > 
> > Best,
> > 
> > Ilya 
> > 
> > -Original Message-
> > From: galaxy-dev-boun...@lists.bx.psu.edu 
> > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Lloyd Brown
> > Sent: Monday, October 31, 2011 2:35 PM
> > To: Galaxy Dev List
> > Subject: [galaxy-dev] Clusters, Runners, and user credentials
> > 
> > I'm a systems administrator for an HPC cluster, and have been asked by a 
> > faculty member here to try to get galaxy to work on our cluster.
> > Unfortunately, there are one or two outstanding questions that I can't seem 
> > to find the answer to, and I'm hoping someone here can help me out.
> > 
> > In particular, is galaxy, and the PBS runner specifically, capable of 
> > submitting jobs under specific user names?  Essentially, if I set up galaxy 
> > to push jobs to our cluster, will they all show up under one user 
> > credential (eg. the "galaxy" user), or can we set it up so that the user 
> > logged into galaxy, is used to submit the job?
> > 
> > This one is kindof a show-stopper, since our internal policies require that 
> > all jobs have a specific user credential, with one person per username.
> > 
> > Thanks,
> > Lloyd
> > 
> > 
> > --
> > Lloyd Brown
> > Systems Administrator
> > Fulton Supercomputing Lab
> > Brigham Young University
> > http://marylou.byu.edu
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this and other Galaxy 
> > lists, please use the interface at:
> > 
> >  http://lists.bx.psu.edu/
> > 
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this and other Galaxy 
> > lists, please use the interface at:
> > 
> >  http://lists.bx.psu.edu/
> > 
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > 
> >  http://lists.bx.psu.edu/
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] getting job's index position in the history from database

[Nate Coraor]

Shantanu Pavgi wrote:
> 
> I am trying to get job's index in the history panel using SQL query (index 
> number shown in the history when a job is submitted). I am unable to locate 
> the necessary table-column for it. Any pointers in which table-column it is 
> located will be really helpful. 

Hi Shantanu,

Do you mean the dataset's sequence number (the first dataset in a
history is 1, next is 2...)?  That's accessable on a
HistoryDatasetAssociation object as .hid

--nate

> 
> --
> Thanks,
> Shantanu. 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] ImportError: No module named galaxy

[Nate Coraor]

Oren Livne wrote:
> Dear All,
> 
> I am running the example_watcher_folder.py API script and it
> intermittently fails to run a workflow on a file dropped into the
> import directory (the file is imported and the workflow starts
> running, but its jobs hang or fail). Galaxy uses a condor dramaa job
> runner. When I look at the galaxy script being run by condor, it
> sets the PYTHONPATH and then executes the following command, which
> also fails at the command line.

Hi Oren,

Is the intermittent failure the same error was below?

> galaxy@vm-debian:~/galaxy-dist$ python
> /home/galaxy/galaxy-dist/tools/filters/sorter.py
> --input=/tmp/galaxy/datafiles/000/dataset_105.dat
> --out_file1=/tmp/galaxy/datafiles/000/dataset_106.dat
> --column=7   --style=num   --order=DESC
> Traceback (most recent call last):
>   File "/home/galaxy/galaxy-dist/tools/filters/sorter.py", line 15,
> in 
> from galaxy import eggs
> ImportError: No module named galaxy

This won't work from the command line unless you set PYTHONPATH the same
way that the framework does (i.e. PYTHONPATH=~/galaxy-dist/lib).

--nate

> 
> How can I resolve this error? I would like to be able to test such
> commands at the command line, as well as ensure that galaxy
> workflows run reliably.
> 
> Thanks,
> Oren
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] galaxy citation_url setting

[Nate Coraor]

Shantanu Pavgi wrote:
> 
> Hi,
> 
> It seems like modification of 'citation_url'  in the universe_wsgi.ini config 
> file has no effect in the UI (Help --> How to Cite Galaxy). Is it something 
> hard-coded in the source? 

Hi Shantanu,

Whoops.  This has been fixed in 6205:9a9479f7e53f.

--nate

> 
> --
> Thanks,
> Shantanu. 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-user] Queuepool limit problem

[Nate Coraor]

Cinto Queky wrote:
> 
> Hi All,
> We encountered queuepool limit problem after running the workflows for a few 
> times.
> We have tried to increase these 2 parameters: 
> database_engine_option_pool_size and database_engine_option_max_overflow to 
> 20 and 40 respectively.
> Are there other parameters that we can change?

Hi Cinto,

I've moved this over to galaxy-dev since it concerns a local install.

Are you running the workflows concurrently, or sequentially?  This
shouldn't really happen with only a couple workflows.

You could also add:

database_engine_option_strategy = threadlocal

Thanks,
--nate

> 
> Thanks,Cinto

> ___
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org.  Please keep all replies on the list by
> using "reply all" in your mail client.  For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>   http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
> 
>   http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] cufflinks version

[Jeremy Goecks]

David,

Wrapper supports v1.0.0 to v1.1.0

I'll update the wiki momentarily.

J.

On Nov 2, 2011, at 2:04 PM, David Hoover wrote:

> What version of cufflinks do the current Galaxy wrappers support?  The wiki 
> (http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies) says 1.0.1, but 
> our local instance only works with 1.1.0.  Can the wiki be updated?
> 
> David Hoover
> Helix Systems Staff
> http://helix.nih.gov
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-user] Cannot get rid of custom build error

[Jeremy Goecks]

Hi Jim,

You'll need to update your datatypes_conf.xml file to get custom builds + 
converters to work. This file is located in the root of your galaxy directory. 

If you haven't modified your datatypes_conf.xml file previously, you can just 
copy datatypes_conf.xml.sample to datatypes_conf.xml If you have modifed this 
file, please add the following lines to your datatypes_conf.xml:

(a)   
  


to the fasta definition

and

(b) 



to the len definition

See datatypes_conf.xml.sample for examples.

Also, you'll need to download and add the UCSC program fastaToTwoBit to your 
path as well. You can find UCSC executables here:

hgwdev.cse.ucsc.edu/~kent/exe/

Good luck,
J.


> Hello,
> 
> I've tried to create a custom build (starting from the trackster page)
> and clearly made some mistake. I get a NoConverterException error
> which is almost certainly my fault putting something wrong in.
> However, now the Custom Builds page shows this error and I cannot make
> it go away.
> 
> I have a local install, have killed and restarted, logged in and out,
> but still this error stays. I cannot try to enter another custom build
> because there is nothing on the page except this error. Does anyone
> kow how to get rid of this? Any help in getting around this error are
> appreciated.
> 
> Jim
> ___
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org.  Please keep all replies on the list by
> using "reply all" in your mail client.  For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>  http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-user] Cannot get rid of custom build error

[James Taylor]

James, do you have an entry for the "fasta_to_len" convertor in your 
datatypes_conf.xml? If not, you might want to update your config to include 
those entries from the sample. Thanks!

-- jt 

(composed on my phone)

On Nov 2, 2011, at 11:58 AM, James Vincent  wrote:

> Hello Devs,
> 
> I can confirm I am using the latest build:
> 
> galaxy@monolith:~/galaxy-dist$ hg pull -u -r 338ead4737ba
> pulling from https://bitbucket.org/galaxy/galaxy-dist/
> searching for changes
> no changes found
> galaxy@monolith:~/galaxy-dist$
> 
> Here is a copy of the error message that shows when I click on Custom
> Builds under the User main menu. Oddly, when on this page I cannot
> kill the galaxy main process. I start with ./run.sh --reload. Giving a
> control-c won't kill it until I leave that error page. Hope that
> helps.
> 
> 
> Error Traceback:
> View as:   Interactive  |  Text  |  XML (full)
> ⇝ NoConverterException: "Conversion from 'fasta' to 'len' not possible"
> URL: http://monolith.uvm.edu:8080/user/dbkeys
> Module weberror.evalexception.middleware:364 in respond  view
>>> app_iter = self.application(environ, detect_start_response)
> Module paste.debug.prints:98 in __call__  view
>>> environ, self.app)
> Module paste.wsgilib:539 in intercept_output  view
>>> app_iter = application(environ, replacement_start_response)
> Module paste.recursive:80 in __call__  view
>>> return self.application(environ, start_response)
> Module paste.httpexceptions:632 in __call__  view
>>> return self.application(environ, start_response)
> Module galaxy.web.framework.base:160 in __call__  view
>>> body = method( trans, **kwargs )
> Module galaxy.web.framework:86 in decorator  view
>>> return func( self, trans, *args, **kwargs )
> Module galaxy.web.controllers.user:1217 in dbkeys  view
>>> len_dataset = fasta_dataset.get_converted_dataset( trans, "len" )
> Module galaxy.model:913 in get_converted_dataset  view
>>> raise NoConverterException("Conversion from '%s' to '%s' not possible" % 
>>> (self.extension, target_ext) )
> NoConverterException: "Conversion from 'fasta' to 'len' not possible"
> 
> 
> 
> Extra Features
>>>Display the lines of code near each part of the traceback
>Show a debug prompt to allow you to directly debug the code at the 
> traceback
> Re-GET Page
> test build.
> 
> Here is a copy of the error message that shows when I click on Custom
> Builds under the User main menu:
> 
> 
> On Wed, Nov 2, 2011 at 10:35 AM, Jennifer Jackson  wrote:
>> Hi Jim,
>> 
>> I am going to post your question over to the galaxy-dev mailing list (is
>> best for local/cloud instance questions).
>> 
>> Would you be able to confirm which version of Galaxy you are running? The
>> latest version of the distribution?
>> 
>>http://wiki.g2.bx.psu.edu/News%20Briefs/2011_10_25
>>"upgrade: % hg pull -u -r 338ead4737ba"
>> 
>> Please send your reply as a "reply-all" that includes the galaxy-dev mailing
>> list, but omits the galaxy-user mailing list.
>> 
>> Thanks for the additional information,
>> 
>> Jen
>> Galaxy team
>> 
>> On 11/2/11 4:04 AM, James Vincent wrote:
>>> 
>>> Hello,
>>> 
>>> I've tried to create a custom build (starting from the trackster page)
>>> and clearly made some mistake. I get a NoConverterException error
>>> which is almost certainly my fault putting something wrong in.
>>> However, now the Custom Builds page shows this error and I cannot make
>>> it go away.
>>> 
>>> I have a local install, have killed and restarted, logged in and out,
>>> but still this error stays. I cannot try to enter another custom build
>>> because there is nothing on the page except this error. Does anyone
>>> kow how to get rid of this? Any help in getting around this error are
>>> appreciated.
>>> 
>>> Jim
>>> ___
>>> The Galaxy User list should be used for the discussion of
>>> Galaxy analysis and other features on the public server
>>> at usegalaxy.org.  Please keep all replies on the list by
>>> using "reply all" in your mail client.  For discussion of
>>> local Galaxy instances and the Galaxy source code, please
>>> use the Galaxy Development list:
>>> 
>>>   http://lists.bx.psu.edu/listinfo/galaxy-dev
>>> 
>>> To manage your subscriptions to this and other Galaxy lists,
>>> please use the interface at:
>>> 
>>>   http://lists.bx.psu.edu/
>> 
>> --
>> Jennifer Jackson
>> http://usegalaxy.org
>> http://galaxyproject.org/wiki/Support
>> 
>> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lis

Re: [galaxy-dev] using nginx instead of paste as Galaxy web server

[James Taylor]

Dean, nginx integration works by proxying to the Paste server, but offloading 
some data intensive parts. So you still need that configuration as it is. 
Thanks!

-- jt 

(composed on my phone)

On Nov 2, 2011, at 12:35 PM, Dean Snyder  wrote:

> I switched from using the default Python Paste web server for my local Galaxy 
> installation to nginx. I have verified that it works fine so far, but 
> shouldn't the following option in the universe_wsgi.ini file be changed? (I 
> don't see anything in the documentation about this.)
> 
> 
> # The internal HTTP server to use.  Currently only Paste is provided.  This
> # option is required.
> use = egg:Paste#http
> 
> 
> Dean A. Snyder
> Senior Programmer/Analyst
> Johns Hopkins University, School of Medicine
> Center for Inherited Disease Research (CIDR)
> 333 Cassell Dr, Triad Bldg, Suite 2000
> Baltimore, MD 21224
> cell:717 668-3048 office:410-550-4629
> www.cidr.jhmi.edu
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Looks like actual user breaks splitting

[Nate Coraor (n...@bx.psu.edu)]

John, Ilya,

I get further with sequence type inputs but it looks like
JobWrapper.get_output_datasets_and_fnames() is not returning the right
thing when outputs_to_working_directory = True

BTW, the base Data.split() method is broken after the updates to
Sequence.split() since it wasn't updated to expect
HistoryDatasetAssociations rather than filenames.  Could you take a look
at that when you get a chance?

--nate

Duddy, John wrote:
> The datatype you are using does not define a split method. Are you working 
> with our in-progress gz type or fastqillumina?
> 
> John Duddy
> Sr. Staff Software Engineer
> Illumina, Inc.
> 9885 Towne Centre Drive
> San Diego, CA 92121
> Tel: 858-736-3584
> E-mail: jdu...@illumina.com
> 
> From: Chorny, Ilya
> Sent: Wednesday, November 02, 2011 11:50 AM
> To: Duddy, John
> Cc: Nate Coraor (n...@bx.psu.edu); galaxy-dev@lists.bx.psu.edu
> Subject: Looks like actual user breaks splitting
> 
> Hey John,
> 
> Any thoughts?
> 
> Ilya
> 
> Traceback (most recent call last):
>   File 
> "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/runners/tasks.py", line 
> 73, in run_job
> tasks = splitter.do_split(job_wrapper)
>   File 
> "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/splitters/multi.py", 
> line 73, in do_split
> input_type.split(input_datasets, get_new_working_directory_name, 
> parallel_settings)
>   File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/datatypes/data.py", 
> line 473, in split
> raise Exception("Text file splitting does not support multiple files")
> Exception: Text file splitting does not support multiple files
> 
> Ilya Chorny Ph.D.
> Bioinformatics Scientist I
> Illumina, Inc.
> 9885 Towne Centre Drive
> San Diego, CA 92121
> Work: 858.202.4582
> Email: icho...@illumina.com
> Website: www.illumina.com
> 
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Looks like actual user breaks splitting

[Duddy, John]

The datatype you are using does not define a split method. Are you working with 
our in-progress gz type or fastqillumina?

John Duddy
Sr. Staff Software Engineer
Illumina, Inc.
9885 Towne Centre Drive
San Diego, CA 92121
Tel: 858-736-3584
E-mail: jdu...@illumina.com

From: Chorny, Ilya
Sent: Wednesday, November 02, 2011 11:50 AM
To: Duddy, John
Cc: Nate Coraor (n...@bx.psu.edu); galaxy-dev@lists.bx.psu.edu
Subject: Looks like actual user breaks splitting

Hey John,

Any thoughts?

Ilya

Traceback (most recent call last):
  File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/runners/tasks.py", 
line 73, in run_job
tasks = splitter.do_split(job_wrapper)
  File 
"/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/splitters/multi.py", line 
73, in do_split
input_type.split(input_datasets, get_new_working_directory_name, 
parallel_settings)
  File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/datatypes/data.py", line 
473, in split
raise Exception("Text file splitting does not support multiple files")
Exception: Text file splitting does not support multiple files

Ilya Chorny Ph.D.
Bioinformatics Scientist I
Illumina, Inc.
9885 Towne Centre Drive
San Diego, CA 92121
Work: 858.202.4582
Email: icho...@illumina.com
Website: www.illumina.com


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Looks like actual user breaks splitting

[Chorny, Ilya]

Hey John,

Any thoughts?

Ilya

Traceback (most recent call last):
  File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/runners/tasks.py", 
line 73, in run_job
tasks = splitter.do_split(job_wrapper)
  File 
"/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/splitters/multi.py", line 
73, in do_split
input_type.split(input_datasets, get_new_working_directory_name, 
parallel_settings)
  File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/datatypes/data.py", line 
473, in split
raise Exception("Text file splitting does not support multiple files")
Exception: Text file splitting does not support multiple files

Ilya Chorny Ph.D.
Bioinformatics Scientist I
Illumina, Inc.
9885 Towne Centre Drive
San Diego, CA 92121
Work: 858.202.4582
Email: icho...@illumina.com
Website: www.illumina.com


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] cufflinks version

[David Hoover]

What version of cufflinks do the current Galaxy wrappers support?  The wiki 
(http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies) says 1.0.1, but our 
local instance only works with 1.1.0.  Can the wiki be updated?

David Hoover
Helix Systems Staff
http://helix.nih.gov
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Local Galaxy Instance MarkupSafe error

[Nate Coraor]

Jerico Nico De Leon Revote wrote:
> It's the same case as what I'm getting. I can see the output via "eye" icon
> on the history panel and able to download the files as well.

Do your cluster nodes have internet access?  If so, log into a node and
run the command again from there.  Your nodes may have a different
Python version or Unicode byte order encoding scheme than your Galaxy
application server.

--nate

> 
> On 1 November 2011 03:37, Mickael ESCUDERO wrote:
> 
> > Hi there,
> >
> > I'm getting exactly the same problem with any job running on a TORQUE/PBS
> > cluster. The jobs actually run fine as I can see the output and download
> > it, but it's marked as failed in the galaxy history, with the following
> > message:
> >
> > WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako) cannot be 
> > fetched
> >
> > The command `python -ES ./scripts/fetch_eggs.py` gives nothing as output.
> > If I run the same tools locally there is no problem.
> >
> > Cheers
> > Micka
> >
> >
> >> Message: 5
> >> Date: Thu, 27 Oct 2011 15:08:17 +1100
> >> From: Jerico Nico De Leon Revote 
> >> To: galaxy-dev@lists.bx.psu.edu
> >> Subject: [galaxy-dev] Local Galaxy Instance MarkupSafe error
> >> Message-ID:
> >><
> >> cap9ulhyipxyb2cwtqosm3mquuu55arxy5por1dbahej1uco...@mail.gmail.com>
> >> Content-Type: text/plain; charset="iso-8859-1"
> >>
> >> Hi,
> >>
> >> I'm just doing a simple get->data from UCSC on our local Galaxy
> >> instance and got the following error:
> >>
> >> WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako)
> >> cannot be fetched
> >>
> >> The job box then is displayed as red on the history panel.
> >> The job runner states that the job finished normally on the cluster.
> >> Galaxy is checkout from galaxy-central (changeset: 6176:34fffbf01183).
> >>
> >> Thanks,
> >>
> >> Jerico
> >> -- next part --
> >> An HTML attachment was scrubbed...
> >> URL: <
> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/attachments/20111027/cd777ff3/attachment-0001.html
> >> >
> >>
> >> --
> >>
> >> Message: 6
> >> Date: Thu, 27 Oct 2011 16:58:13 +1100
> >> From: Jerico Nico De Leon Revote 
> >> To: galaxy-dev@lists.bx.psu.edu
> >> Subject: Re: [galaxy-dev] Local Galaxy Instance MarkupSafe error
> >> Message-ID:
> >><
> >> cap9ulhbenenkqc50uzyxoca4w-zucdy8obs4i6b3e2qyfmy...@mail.gmail.com>
> >> Content-Type: text/plain; charset="iso-8859-1"
> >>
> >> Just to follow-up on this. The MarkupSafe egg is definitely present on the
> >> eggs directory and the servers are ran through virtualenv.
> >>
> >> On 27 October 2011 15:08, Jerico Nico De Leon Revote <
> >> jerico.rev...@monash.edu> wrote:
> >>
> >> > Hi,
> >> >
> >> > I'm just doing a simple get->data from UCSC on our local Galaxy
> >> instance and got the following error:
> >> >
> >> > WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako)
> >> cannot be fetched
> >> >
> >> > The job box then is displayed as red on the history panel.
> >> > The job runner states that the job finished normally on the cluster.
> >> > Galaxy is checkout from galaxy-central (changeset: 6176:34fffbf01183).
> >> >
> >> > Thanks,
> >> >
> >> > Jerico
> >> >
> >> >
> >> -- next part --
> >> An HTML attachment was scrubbed...
> >> URL: <
> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/attachments/20111027/f8baadf6/attachment-0001.html
> >> >
> >>
> >> --
> >>
> >> Message: 7
> >> Date: Thu, 27 Oct 2011 02:40:51 -0400
> >> From: Nate Coraor 
> >> To: Jerico Nico De Leon Revote 
> >> Cc: galaxy-dev@lists.bx.psu.edu
> >> Subject: Re: [galaxy-dev] Local Galaxy Instance MarkupSafe error
> >> Message-ID: <20111027064051.gg2...@bx.psu.edu>
> >> Content-Type: text/plain; charset=us-ascii
> >>
> >> Jerico Nico De Leon Revote wrote:
> >> > Hi,
> >> >
> >> > I'm just doing a simple get->data from UCSC on our local Galaxy
> >> > instance and got the following error:
> >> >
> >> > WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako)
> >> > cannot be fetched
> >> >
> >> > The job box then is displayed as red on the history panel.
> >> > The job runner states that the job finished normally on the cluster.
> >> > Galaxy is checkout from galaxy-central (changeset: 6176:34fffbf01183).
> >>
> >> Hi Jerico,
> >>
> >> Are you using a cluster?  If not, could you run:
> >>
> >>% python -ES ./scripts/fetch_eggs.py
> >>
> >> --nate
> >>
> >> >
> >> > Thanks,
> >> >
> >> > Jerico
> >>
> >> > ___
> >> > Please keep all replies on the list by using "reply all"
> >> > in your mail client.  To manage your subscriptions to this
> >> > and other Galaxy lists, please use the interface at:
> >> >
> >> >   http://lists.bx.psu.edu/
> >>
> >> --
> > Mickael Escudero, Bioinformatics Officer
> >
> >
> > ___
> > Please keep all replies on the list by using "r

Re: [galaxy-dev] Providing Data to Galaxy Users

[Ostell, Jim (NIH/NLM/NCBI) [E]]

Thanks Dan,

We will read the docs and follow up. I cc'd 5 folks who are looking into this 
at NCBI by way of email introduction, in case you get some questions from one 
of them.

   Regards,
   Jim Ostell

From: Daniel Blankenberg [mailto:d...@bx.psu.edu]
Sent: Wednesday, October 26, 2011 3:56 PM
To: Ostell, Jim (NIH/NLM/NCBI) [E]
Cc: galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] Providing Data to Galaxy Users

Hi Jim,

The preferred method to add external datasources is the URL-based callback 
mechanism as described in http://www.ncbi.nlm.nih.gov/pubmed/21531983 and 
http://wiki.g2.bx.psu.edu/Admin/Internals/Data%20Sources. It has proven 
effective for importing data from various data resources using their native 
interfaces into any of the many public or private Galaxy instances which exist, 
based upon the user's starting location.

Please let us know if we can provide additional assistance.


Thanks for your interest in Galaxy,

Dan


On Oct 26, 2011, at 9:00 AM, Ostell, Jim (NIH/NLM/NCBI) [E] wrote:


Hello Galaxy Folks,

NCBI has received a few requests to provide a mechanism for people browsing the 
NCBI databases to download the data they have selected into Galaxy, so we are 
looking into it. I know some other sites have done this, it looks like through 
a URL based callback mechanism.

However, I know things can change over time, and I'd like to be sure that we 
are assessing an approach that reflects your current best thinking, whether it 
turns out to be something well established you are happy with, or if you might 
be considering moving toward something new, we'd like you to tell us what you 
think would work best for your users in your environment.

If you can just point me at a URL with the appropriate documentation, that's 
great. If you'd prefer to dialog a little, we're happy to talk. If there is a 
particular person we should be talking to, let me know.

Thanks for your help.

  Regards,
  Jim Ostell


James M. Ostell, Ph.D.
Chief, Information Engineering Branch
National Center for Biotechnology Information, NLM, NIH
Bldg 45, Rm 5AN.44A
45 Center Drive
Bethesda, MD 20892-6510


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-user] Cannot get rid of custom build error

[James Vincent]

Hello Devs,

I can confirm I am using the latest build:

galaxy@monolith:~/galaxy-dist$ hg pull -u -r 338ead4737ba
pulling from https://bitbucket.org/galaxy/galaxy-dist/
searching for changes
no changes found
galaxy@monolith:~/galaxy-dist$

Here is a copy of the error message that shows when I click on Custom
Builds under the User main menu. Oddly, when on this page I cannot
kill the galaxy main process. I start with ./run.sh --reload. Giving a
control-c won't kill it until I leave that error page. Hope that
helps.


Error Traceback:
View as:   Interactive  |  Text  |  XML (full)
⇝ NoConverterException: "Conversion from 'fasta' to 'len' not possible"
URL: http://monolith.uvm.edu:8080/user/dbkeys
Module weberror.evalexception.middleware:364 in respond  view
>>  app_iter = self.application(environ, detect_start_response)
Module paste.debug.prints:98 in __call__  view
>>  environ, self.app)
Module paste.wsgilib:539 in intercept_output  view
>>  app_iter = application(environ, replacement_start_response)
Module paste.recursive:80 in __call__  view
>>  return self.application(environ, start_response)
Module paste.httpexceptions:632 in __call__  view
>>  return self.application(environ, start_response)
Module galaxy.web.framework.base:160 in __call__  view
>>  body = method( trans, **kwargs )
Module galaxy.web.framework:86 in decorator  view
>>  return func( self, trans, *args, **kwargs )
Module galaxy.web.controllers.user:1217 in dbkeys  view
>>  len_dataset = fasta_dataset.get_converted_dataset( trans, "len" )
Module galaxy.model:913 in get_converted_dataset  view
>>  raise NoConverterException("Conversion from '%s' to '%s' not possible" % 
>> (self.extension, target_ext) )
NoConverterException: "Conversion from 'fasta' to 'len' not possible"



Extra Features
>>  Display the lines of code near each part of the traceback
Show a debug prompt to allow you to directly debug the code at the 
traceback
Re-GET Page
test build.

Here is a copy of the error message that shows when I click on Custom
Builds under the User main menu:


On Wed, Nov 2, 2011 at 10:35 AM, Jennifer Jackson  wrote:
> Hi Jim,
>
> I am going to post your question over to the galaxy-dev mailing list (is
> best for local/cloud instance questions).
>
> Would you be able to confirm which version of Galaxy you are running? The
> latest version of the distribution?
>
>    http://wiki.g2.bx.psu.edu/News%20Briefs/2011_10_25
>    "upgrade: % hg pull -u -r 338ead4737ba"
>
> Please send your reply as a "reply-all" that includes the galaxy-dev mailing
> list, but omits the galaxy-user mailing list.
>
> Thanks for the additional information,
>
> Jen
> Galaxy team
>
> On 11/2/11 4:04 AM, James Vincent wrote:
>>
>> Hello,
>>
>> I've tried to create a custom build (starting from the trackster page)
>> and clearly made some mistake. I get a NoConverterException error
>> which is almost certainly my fault putting something wrong in.
>> However, now the Custom Builds page shows this error and I cannot make
>> it go away.
>>
>> I have a local install, have killed and restarted, logged in and out,
>> but still this error stays. I cannot try to enter another custom build
>> because there is nothing on the page except this error. Does anyone
>> kow how to get rid of this? Any help in getting around this error are
>> appreciated.
>>
>> Jim
>> ___
>> The Galaxy User list should be used for the discussion of
>> Galaxy analysis and other features on the public server
>> at usegalaxy.org.  Please keep all replies on the list by
>> using "reply all" in your mail client.  For discussion of
>> local Galaxy instances and the Galaxy source code, please
>> use the Galaxy Development list:
>>
>>   http://lists.bx.psu.edu/listinfo/galaxy-dev
>>
>> To manage your subscriptions to this and other Galaxy lists,
>> please use the interface at:
>>
>>   http://lists.bx.psu.edu/
>
> --
> Jennifer Jackson
> http://usegalaxy.org
> http://galaxyproject.org/wiki/Support
>
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] using nginx instead of paste as Galaxy web server

[Dean Snyder]

I switched from using the default Python Paste web server for my local Galaxy 
installation to nginx. I have verified that it works fine so far, but shouldn't 
the following option in the universe_wsgi.ini file be changed? (I don't see 
anything in the documentation about this.)


# The internal HTTP server to use.  Currently only Paste is provided.  This
# option is required.
use = egg:Paste#http


Dean A. Snyder
Senior Programmer/Analyst
Johns Hopkins University, School of Medicine
Center for Inherited Disease Research (CIDR)
333 Cassell Dr, Triad Bldg, Suite 2000
Baltimore, MD 21224
cell:717 668-3048 office:410-550-4629
www.cidr.jhmi.edu


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Re Acutal user code

[Nate Coraor (n...@bx.psu.edu)]

Chorny, Ilya wrote:

> I am pulling and testing as we speak. I did a preliminary test on my VM but 
> it does not have enough memory to run BWA. We pull into my dev version on the 
> cluster and do further testing. Also need to test how it integrates with the 
> splitting code. Should have some results by EOD.

Yeah, there is probably a problem with splitting, I was about to test
that myself.

> 
> Ilya
> 
> 
> -Original Message-
> From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu] 
> Sent: Wednesday, November 02, 2011 9:15 AM
> To: Chorny, Ilya
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] Re Acutal user code
> 
> Hi Ilya,
> 
> I made a few more commits this morning.  Could you test it out in the present 
> state?  If everything works properly in your environment, I think it's pretty 
> close to being ready to integrate.  The major outstanding issue is to try to 
> avoid chmods in the upload action, which I have some ideas about.
> 
> --nate
> 
> Nate Coraor (n...@bx.psu.edu) wrote:
> > Nate Coraor (n...@bx.psu.edu) wrote:
> > > Chorny, Ilya wrote:
> > > > So it looks like for some reason galaxy is not picking up the 
> > > > information in the .e file. Not sure why. I even switched it back to 
> > > > writing the files to the pbs directory and it still did not pick up the 
> > > > .e files. BWA was core dumping but I was not given the error message in 
> > > > galaxy. 
> > > 
> > > Okay, this is pretty odd, I'll try to see what's going on here.  If 
> > > you figure anything out, please let me know.
> > 
> > Whoops, variable name collision in finish().  I've just committed a fix.
> > 
> > > 
> > > > BTW, I find it nice to be able to see the .o and .e files. The way you 
> > > > have it coded the .o and .e are written out to the 
> > > > job_working_directory and then disappear at the end of the run.  But 
> > > > the way I had it coded the pbs directory is world read writable. Is 
> > > > there an issue with having the pbs directory be world read write?
> > > 
> > > I imagine for some environments, it's preferable to avoid 
> > > world-writable directories.  Also, going forward, containing all of 
> > > a job's filesystem data to a single directory will be important for 
> > > running on multiple clusters without access to a single shared 
> > > filesystem mounted at the same path.
> > > 
> > > There's an issue in bitbucket somewhere regarding saving standard 
> > > out and standard error for all jobs, which would solve the problem 
> > > of it being in the working directory.
> > > 
> > > For debugging, if you comment out shutil.rmtree( 
> > > self.working_directory ) in jobs.cleanup(), it'll leave everything behind.
> > > 
> > > --nate
> > > 
> > > > 
> > > > Thanks,
> > > > 
> > > > Ilya
> > > > 
> > > > 
> > > > -Original Message-
> > > > From: galaxy-dev-boun...@lists.bx.psu.edu 
> > > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > > Ilya
> > > > Sent: Monday, October 31, 2011 8:37 PM
> > > > To: Nate Coraor (n...@bx.psu.edu)
> > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > > 
> > > > It seems to be working but when I run BWA or Bowtie it finishes green 
> > > > but the files are empty. Doesn't give me much information. I also tried 
> > > > fastqc and it ran just fine.
> > > > 
> > > > -Original Message-
> > > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > > Sent: Monday, October 31, 2011 1:21 PM
> > > > To: Chorny, Ilya
> > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > Subject: Re: Re Acutal user code
> > > > 
> > > > Chorny, Ilya wrote:
> > > > > Sweet!! Can't wait to see it.
> > > > 
> > > > Have a look:
> > > > 
> > > > https://bitbucket.org/natefoo/galaxy-central-ichorny/
> > > > 
> > > > Caveats:
> > > > 
> > > > * job_working_directory has to be in a shared filesystem
> > > > * outputs_to_working_directory must be True
> > > > * set_metadata_externally must be True
> > > > 
> > > > One thing that would be good from a security standpoint would be if the 
> > > > external chown script took a Galaxy Job ID for its argument instead of 
> > > > a path, and then looked up the working directory path from the ID.  
> > > > That way the script couldn't be used maliciously to chown any file.
> > > > 
> > > > --nate
> > > > 
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > Ilya
> > > > > 
> > > > > -Original Message-
> > > > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > > > Sent: Thursday, October 27, 2011 1:24 PM
> > > > > To: Chorny, Ilya
> > > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > > Subject: Re: Re Acutal user code
> > > > > 
> > > > > Yup, cleaning up some things that were accidentally lost in merges.  
> > > > > I added another bitbucket repo so as not to mess up yours.
> > > > > 
> > > > > Chorny, Ilya wrote:
> > > > > > Hi Nate,
> > > > > > 
> > > > > > Any updates?
> > > > > > 
> > > > > > Thanks

Re: [galaxy-dev] Re Acutal user code

[Chorny, Ilya]

I am pulling and testing as we speak. I did a preliminary test on my VM but it 
does not have enough memory to run BWA. We pull into my dev version on the 
cluster and do further testing. Also need to test how it integrates with the 
splitting code. Should have some results by EOD.

Ilya


-Original Message-
From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu] 
Sent: Wednesday, November 02, 2011 9:15 AM
To: Chorny, Ilya
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Re Acutal user code

Hi Ilya,

I made a few more commits this morning.  Could you test it out in the present 
state?  If everything works properly in your environment, I think it's pretty 
close to being ready to integrate.  The major outstanding issue is to try to 
avoid chmods in the upload action, which I have some ideas about.

--nate

Nate Coraor (n...@bx.psu.edu) wrote:
> Nate Coraor (n...@bx.psu.edu) wrote:
> > Chorny, Ilya wrote:
> > > So it looks like for some reason galaxy is not picking up the information 
> > > in the .e file. Not sure why. I even switched it back to writing the 
> > > files to the pbs directory and it still did not pick up the .e files. BWA 
> > > was core dumping but I was not given the error message in galaxy. 
> > 
> > Okay, this is pretty odd, I'll try to see what's going on here.  If 
> > you figure anything out, please let me know.
> 
> Whoops, variable name collision in finish().  I've just committed a fix.
> 
> > 
> > > BTW, I find it nice to be able to see the .o and .e files. The way you 
> > > have it coded the .o and .e are written out to the job_working_directory 
> > > and then disappear at the end of the run.  But the way I had it coded the 
> > > pbs directory is world read writable. Is there an issue with having the 
> > > pbs directory be world read write?
> > 
> > I imagine for some environments, it's preferable to avoid 
> > world-writable directories.  Also, going forward, containing all of 
> > a job's filesystem data to a single directory will be important for 
> > running on multiple clusters without access to a single shared 
> > filesystem mounted at the same path.
> > 
> > There's an issue in bitbucket somewhere regarding saving standard 
> > out and standard error for all jobs, which would solve the problem 
> > of it being in the working directory.
> > 
> > For debugging, if you comment out shutil.rmtree( 
> > self.working_directory ) in jobs.cleanup(), it'll leave everything behind.
> > 
> > --nate
> > 
> > > 
> > > Thanks,
> > > 
> > > Ilya
> > > 
> > > 
> > > -Original Message-
> > > From: galaxy-dev-boun...@lists.bx.psu.edu 
> > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > Ilya
> > > Sent: Monday, October 31, 2011 8:37 PM
> > > To: Nate Coraor (n...@bx.psu.edu)
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > 
> > > It seems to be working but when I run BWA or Bowtie it finishes green but 
> > > the files are empty. Doesn't give me much information. I also tried 
> > > fastqc and it ran just fine.
> > > 
> > > -Original Message-
> > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > Sent: Monday, October 31, 2011 1:21 PM
> > > To: Chorny, Ilya
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: Re Acutal user code
> > > 
> > > Chorny, Ilya wrote:
> > > > Sweet!! Can't wait to see it.
> > > 
> > > Have a look:
> > > 
> > > https://bitbucket.org/natefoo/galaxy-central-ichorny/
> > > 
> > > Caveats:
> > > 
> > > * job_working_directory has to be in a shared filesystem
> > > * outputs_to_working_directory must be True
> > > * set_metadata_externally must be True
> > > 
> > > One thing that would be good from a security standpoint would be if the 
> > > external chown script took a Galaxy Job ID for its argument instead of a 
> > > path, and then looked up the working directory path from the ID.  That 
> > > way the script couldn't be used maliciously to chown any file.
> > > 
> > > --nate
> > > 
> > > > 
> > > > Thanks,
> > > > 
> > > > Ilya
> > > > 
> > > > -Original Message-
> > > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > > Sent: Thursday, October 27, 2011 1:24 PM
> > > > To: Chorny, Ilya
> > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > Subject: Re: Re Acutal user code
> > > > 
> > > > Yup, cleaning up some things that were accidentally lost in merges.  I 
> > > > added another bitbucket repo so as not to mess up yours.
> > > > 
> > > > Chorny, Ilya wrote:
> > > > > Hi Nate,
> > > > > 
> > > > > Any updates?
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > Ilya
> > > > > 
> > > > > 
> > > > > -Original Message-
> > > > > From: galaxy-dev-boun...@lists.bx.psu.edu
> > > > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of 
> > > > > Chorny, Ilya
> > > > > Sent: Friday, October 21, 2011 2:24 PM
> > > > > To: Nate Coraor (n...@bx.psu.edu)
> > > > > Cc: galaxy-dev@lists.bx.psu.edu
>

Re: [galaxy-dev] Re Acutal user code

[Nate Coraor (n...@bx.psu.edu)]

Hi Ilya,

I made a few more commits this morning.  Could you test it out in the
present state?  If everything works properly in your environment, I
think it's pretty close to being ready to integrate.  The major
outstanding issue is to try to avoid chmods in the upload action, which
I have some ideas about.

--nate

Nate Coraor (n...@bx.psu.edu) wrote:
> Nate Coraor (n...@bx.psu.edu) wrote:
> > Chorny, Ilya wrote:
> > > So it looks like for some reason galaxy is not picking up the information 
> > > in the .e file. Not sure why. I even switched it back to writing the 
> > > files to the pbs directory and it still did not pick up the .e files. BWA 
> > > was core dumping but I was not given the error message in galaxy. 
> > 
> > Okay, this is pretty odd, I'll try to see what's going on here.  If you
> > figure anything out, please let me know.
> 
> Whoops, variable name collision in finish().  I've just committed a fix.
> 
> > 
> > > BTW, I find it nice to be able to see the .o and .e files. The way you 
> > > have it coded the .o and .e are written out to the job_working_directory 
> > > and then disappear at the end of the run.  But the way I had it coded the 
> > > pbs directory is world read writable. Is there an issue with having the 
> > > pbs directory be world read write?
> > 
> > I imagine for some environments, it's preferable to avoid world-writable
> > directories.  Also, going forward, containing all of a job's filesystem
> > data to a single directory will be important for running on multiple
> > clusters without access to a single shared filesystem mounted at the
> > same path.
> > 
> > There's an issue in bitbucket somewhere regarding saving standard out
> > and standard error for all jobs, which would solve the problem of it
> > being in the working directory.
> > 
> > For debugging, if you comment out shutil.rmtree( self.working_directory ) 
> > in jobs.cleanup(), it'll leave everything behind.
> > 
> > --nate
> > 
> > > 
> > > Thanks,
> > > 
> > > Ilya
> > > 
> > > 
> > > -Original Message-
> > > From: galaxy-dev-boun...@lists.bx.psu.edu 
> > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, Ilya
> > > Sent: Monday, October 31, 2011 8:37 PM
> > > To: Nate Coraor (n...@bx.psu.edu)
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > 
> > > It seems to be working but when I run BWA or Bowtie it finishes green but 
> > > the files are empty. Doesn't give me much information. I also tried 
> > > fastqc and it ran just fine.
> > > 
> > > -Original Message-
> > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > Sent: Monday, October 31, 2011 1:21 PM
> > > To: Chorny, Ilya
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: Re Acutal user code
> > > 
> > > Chorny, Ilya wrote:
> > > > Sweet!! Can't wait to see it.
> > > 
> > > Have a look:
> > > 
> > > https://bitbucket.org/natefoo/galaxy-central-ichorny/
> > > 
> > > Caveats:
> > > 
> > > * job_working_directory has to be in a shared filesystem
> > > * outputs_to_working_directory must be True
> > > * set_metadata_externally must be True
> > > 
> > > One thing that would be good from a security standpoint would be if the 
> > > external chown script took a Galaxy Job ID for its argument instead of a 
> > > path, and then looked up the working directory path from the ID.  That 
> > > way the script couldn't be used maliciously to chown any file.
> > > 
> > > --nate
> > > 
> > > > 
> > > > Thanks,
> > > > 
> > > > Ilya
> > > > 
> > > > -Original Message-
> > > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > > Sent: Thursday, October 27, 2011 1:24 PM
> > > > To: Chorny, Ilya
> > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > Subject: Re: Re Acutal user code
> > > > 
> > > > Yup, cleaning up some things that were accidentally lost in merges.  I 
> > > > added another bitbucket repo so as not to mess up yours.
> > > > 
> > > > Chorny, Ilya wrote:
> > > > > Hi Nate,
> > > > > 
> > > > > Any updates?
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > Ilya
> > > > > 
> > > > > 
> > > > > -Original Message-
> > > > > From: galaxy-dev-boun...@lists.bx.psu.edu
> > > > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > > > Ilya
> > > > > Sent: Friday, October 21, 2011 2:24 PM
> > > > > To: Nate Coraor (n...@bx.psu.edu)
> > > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > > > 
> > > > > One more thing there are some changes in lib/galaxy/tools/__init__.py 
> > > > > and lib/galaxy/model/__init__.py that are not related to actual user. 
> > > > > They look like they are toolshed related and may have been changed 
> > > > > because to version of the file that I copied over was older then 
> > > > > galaxy-central. Probably should have done a merge first. I can redo 
> > > > > it if you like. Let me know.
> > > > > 
> > > > > Ilya
> > > > > 

Re: [galaxy-dev] [galaxy-user] Cannot get rid of custom build error

[Jennifer Jackson]

Hi Jim,

I am going to post your question over to the galaxy-dev mailing list (is 
best for local/cloud instance questions).


Would you be able to confirm which version of Galaxy you are running? 
The latest version of the distribution?


http://wiki.g2.bx.psu.edu/News%20Briefs/2011_10_25
"upgrade: % hg pull -u -r 338ead4737ba"

Please send your reply as a "reply-all" that includes the galaxy-dev 
mailing list, but omits the galaxy-user mailing list.


Thanks for the additional information,

Jen
Galaxy team

On 11/2/11 4:04 AM, James Vincent wrote:

Hello,

I've tried to create a custom build (starting from the trackster page)
and clearly made some mistake. I get a NoConverterException error
which is almost certainly my fault putting something wrong in.
However, now the Custom Builds page shows this error and I cannot make
it go away.

I have a local install, have killed and restarted, logged in and out,
but still this error stays. I cannot try to enter another custom build
because there is nothing on the page except this error. Does anyone
kow how to get rid of this? Any help in getting around this error are
appreciated.

Jim
___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

   http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

   http://lists.bx.psu.edu/


--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] Re Acutal user code

[Nate Coraor (n...@bx.psu.edu)]

Nate Coraor (n...@bx.psu.edu) wrote:
> Chorny, Ilya wrote:
> > So it looks like for some reason galaxy is not picking up the information 
> > in the .e file. Not sure why. I even switched it back to writing the files 
> > to the pbs directory and it still did not pick up the .e files. BWA was 
> > core dumping but I was not given the error message in galaxy. 
> 
> Okay, this is pretty odd, I'll try to see what's going on here.  If you
> figure anything out, please let me know.

Whoops, variable name collision in finish().  I've just committed a fix.

> 
> > BTW, I find it nice to be able to see the .o and .e files. The way you have 
> > it coded the .o and .e are written out to the job_working_directory and 
> > then disappear at the end of the run.  But the way I had it coded the pbs 
> > directory is world read writable. Is there an issue with having the pbs 
> > directory be world read write?
> 
> I imagine for some environments, it's preferable to avoid world-writable
> directories.  Also, going forward, containing all of a job's filesystem
> data to a single directory will be important for running on multiple
> clusters without access to a single shared filesystem mounted at the
> same path.
> 
> There's an issue in bitbucket somewhere regarding saving standard out
> and standard error for all jobs, which would solve the problem of it
> being in the working directory.
> 
> For debugging, if you comment out shutil.rmtree( self.working_directory ) 
> in jobs.cleanup(), it'll leave everything behind.
> 
> --nate
> 
> > 
> > Thanks,
> > 
> > Ilya
> > 
> > 
> > -Original Message-
> > From: galaxy-dev-boun...@lists.bx.psu.edu 
> > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, Ilya
> > Sent: Monday, October 31, 2011 8:37 PM
> > To: Nate Coraor (n...@bx.psu.edu)
> > Cc: galaxy-dev@lists.bx.psu.edu
> > Subject: Re: [galaxy-dev] Re Acutal user code
> > 
> > It seems to be working but when I run BWA or Bowtie it finishes green but 
> > the files are empty. Doesn't give me much information. I also tried fastqc 
> > and it ran just fine.
> > 
> > -Original Message-
> > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > Sent: Monday, October 31, 2011 1:21 PM
> > To: Chorny, Ilya
> > Cc: galaxy-dev@lists.bx.psu.edu
> > Subject: Re: Re Acutal user code
> > 
> > Chorny, Ilya wrote:
> > > Sweet!! Can't wait to see it.
> > 
> > Have a look:
> > 
> > https://bitbucket.org/natefoo/galaxy-central-ichorny/
> > 
> > Caveats:
> > 
> > * job_working_directory has to be in a shared filesystem
> > * outputs_to_working_directory must be True
> > * set_metadata_externally must be True
> > 
> > One thing that would be good from a security standpoint would be if the 
> > external chown script took a Galaxy Job ID for its argument instead of a 
> > path, and then looked up the working directory path from the ID.  That way 
> > the script couldn't be used maliciously to chown any file.
> > 
> > --nate
> > 
> > > 
> > > Thanks,
> > > 
> > > Ilya
> > > 
> > > -Original Message-
> > > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > > Sent: Thursday, October 27, 2011 1:24 PM
> > > To: Chorny, Ilya
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: Re Acutal user code
> > > 
> > > Yup, cleaning up some things that were accidentally lost in merges.  I 
> > > added another bitbucket repo so as not to mess up yours.
> > > 
> > > Chorny, Ilya wrote:
> > > > Hi Nate,
> > > > 
> > > > Any updates?
> > > > 
> > > > Thanks,
> > > > 
> > > > Ilya
> > > > 
> > > > 
> > > > -Original Message-
> > > > From: galaxy-dev-boun...@lists.bx.psu.edu
> > > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > > Ilya
> > > > Sent: Friday, October 21, 2011 2:24 PM
> > > > To: Nate Coraor (n...@bx.psu.edu)
> > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > > 
> > > > One more thing there are some changes in lib/galaxy/tools/__init__.py 
> > > > and lib/galaxy/model/__init__.py that are not related to actual user. 
> > > > They look like they are toolshed related and may have been changed 
> > > > because to version of the file that I copied over was older then 
> > > > galaxy-central. Probably should have done a merge first. I can redo it 
> > > > if you like. Let me know.
> > > > 
> > > > Ilya
> > > > 
> > > > 
> > > > -Original Message-
> > > > From: Chorny, Ilya
> > > > Sent: Friday, October 21, 2011 2:11 PM
> > > > To: Chorny, Ilya; Nate Coraor (n...@bx.psu.edu)
> > > > Cc: galaxy-dev@lists.bx.psu.edu
> > > > Subject: RE: Re Acutal user code
> > > > 
> > > > Sorry, the link was wrong.
> > > > 
> > > > https://bitbucket.org/ichorny/galaxy-central-ilya2/
> > > > 
> > > > -Original Message-
> > > > From: galaxy-dev-boun...@lists.bx.psu.edu
> > > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > > Ilya
> > > > Sent: Friday, October 21, 2011 2:07 PM
>

Re: [galaxy-dev] data set import into current history

[Dannon Baker]

Not currently, though I can put it on my list to try and add when I have a 
chance.

-Dannon

On Nov 1, 2011, at 5:30 PM, Oren Livne wrote:

> Dear Dannon,
> 
> Is there a web API method for deleting a data library? If not, what are the 
> chances of it being added soon?
> 
> Thanks so much,
> Oren
> 
> On 10/31/2011 6:19 PM, Dannon Baker wrote:
>> Oren,
>> 
>> Not yet.  Right now, the only path to accomplish this would be to upload via 
>> the API to a library, and then copy the contents to a history.  This is a 
>> feature that definitely needs to be implemented, we simply haven't had a 
>> chance yet.  If you're interested in writing it, I'm sure we'd welcome the 
>> contribution ;)
>> 
>> -Dannon
>> 
>> 
>> On Oct 31, 2011, at 12:57 PM, Oren Livne wrote:
>> 
>>> Dear All,
>>> 
>>> Is there an API call for importing data sets into the current history (or a 
>>> new history)? This feature exists in the web UI.
>>> 
>>> Thanks so much,
>>> Oren
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> 
>>> http://lists.bx.psu.edu/
> 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Re Acutal user code

[Nate Coraor (n...@bx.psu.edu)]

Chorny, Ilya wrote:
> So it looks like for some reason galaxy is not picking up the information in 
> the .e file. Not sure why. I even switched it back to writing the files to 
> the pbs directory and it still did not pick up the .e files. BWA was core 
> dumping but I was not given the error message in galaxy. 

Okay, this is pretty odd, I'll try to see what's going on here.  If you
figure anything out, please let me know.

> BTW, I find it nice to be able to see the .o and .e files. The way you have 
> it coded the .o and .e are written out to the job_working_directory and then 
> disappear at the end of the run.  But the way I had it coded the pbs 
> directory is world read writable. Is there an issue with having the pbs 
> directory be world read write?

I imagine for some environments, it's preferable to avoid world-writable
directories.  Also, going forward, containing all of a job's filesystem
data to a single directory will be important for running on multiple
clusters without access to a single shared filesystem mounted at the
same path.

There's an issue in bitbucket somewhere regarding saving standard out
and standard error for all jobs, which would solve the problem of it
being in the working directory.

For debugging, if you comment out shutil.rmtree( self.working_directory ) 
in jobs.cleanup(), it'll leave everything behind.

--nate

> 
> Thanks,
> 
> Ilya
> 
> 
> -Original Message-
> From: galaxy-dev-boun...@lists.bx.psu.edu 
> [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, Ilya
> Sent: Monday, October 31, 2011 8:37 PM
> To: Nate Coraor (n...@bx.psu.edu)
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] Re Acutal user code
> 
> It seems to be working but when I run BWA or Bowtie it finishes green but the 
> files are empty. Doesn't give me much information. I also tried fastqc and it 
> ran just fine.
> 
> -Original Message-
> From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> Sent: Monday, October 31, 2011 1:21 PM
> To: Chorny, Ilya
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: Re Acutal user code
> 
> Chorny, Ilya wrote:
> > Sweet!! Can't wait to see it.
> 
> Have a look:
> 
> https://bitbucket.org/natefoo/galaxy-central-ichorny/
> 
> Caveats:
> 
> * job_working_directory has to be in a shared filesystem
> * outputs_to_working_directory must be True
> * set_metadata_externally must be True
> 
> One thing that would be good from a security standpoint would be if the 
> external chown script took a Galaxy Job ID for its argument instead of a 
> path, and then looked up the working directory path from the ID.  That way 
> the script couldn't be used maliciously to chown any file.
> 
> --nate
> 
> > 
> > Thanks,
> > 
> > Ilya
> > 
> > -Original Message-
> > From: Nate Coraor (n...@bx.psu.edu) [mailto:n...@bx.psu.edu]
> > Sent: Thursday, October 27, 2011 1:24 PM
> > To: Chorny, Ilya
> > Cc: galaxy-dev@lists.bx.psu.edu
> > Subject: Re: Re Acutal user code
> > 
> > Yup, cleaning up some things that were accidentally lost in merges.  I 
> > added another bitbucket repo so as not to mess up yours.
> > 
> > Chorny, Ilya wrote:
> > > Hi Nate,
> > > 
> > > Any updates?
> > > 
> > > Thanks,
> > > 
> > > Ilya
> > > 
> > > 
> > > -Original Message-
> > > From: galaxy-dev-boun...@lists.bx.psu.edu
> > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > Ilya
> > > Sent: Friday, October 21, 2011 2:24 PM
> > > To: Nate Coraor (n...@bx.psu.edu)
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > 
> > > One more thing there are some changes in lib/galaxy/tools/__init__.py and 
> > > lib/galaxy/model/__init__.py that are not related to actual user. They 
> > > look like they are toolshed related and may have been changed because to 
> > > version of the file that I copied over was older then galaxy-central. 
> > > Probably should have done a merge first. I can redo it if you like. Let 
> > > me know.
> > > 
> > > Ilya
> > > 
> > > 
> > > -Original Message-
> > > From: Chorny, Ilya
> > > Sent: Friday, October 21, 2011 2:11 PM
> > > To: Chorny, Ilya; Nate Coraor (n...@bx.psu.edu)
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: RE: Re Acutal user code
> > > 
> > > Sorry, the link was wrong.
> > > 
> > > https://bitbucket.org/ichorny/galaxy-central-ilya2/
> > > 
> > > -Original Message-
> > > From: galaxy-dev-boun...@lists.bx.psu.edu
> > > [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Chorny, 
> > > Ilya
> > > Sent: Friday, October 21, 2011 2:07 PM
> > > To: Nate Coraor (n...@bx.psu.edu)
> > > Cc: galaxy-dev@lists.bx.psu.edu
> > > Subject: Re: [galaxy-dev] Re Acutal user code
> > > 
> > > https://bitbucket.org/ichorny/galaxy-central-ilya2/pull-request/new
> > > 
> > > everything but the tophat/cufflinks wrapper.
> > > 
> > > Thanks,
> > > 
> > > Ilya
> > > 
> > > 
> > > -Original Message-
> > > From: Nate Coraor (n..

Re: [galaxy-dev] data set import into current history

[Oren Livne]

Dear Dannon,

Is there a web API method for deleting a data library? If not, what are 
the chances of it being added soon?


Thanks so much,
Oren

On 10/31/2011 6:19 PM, Dannon Baker wrote:

Oren,

Not yet.  Right now, the only path to accomplish this would be to upload via 
the API to a library, and then copy the contents to a history.  This is a 
feature that definitely needs to be implemented, we simply haven't had a chance 
yet.  If you're interested in writing it, I'm sure we'd welcome the 
contribution ;)

-Dannon


On Oct 31, 2011, at 12:57 PM, Oren Livne wrote:


Dear All,

Is there an API call for importing data sets into the current history (or a new 
history)? This feature exists in the web UI.

Thanks so much,
Oren
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/