Re: [galaxy-dev] User list and disk space?
On Thu, Jun 9, 2011 at 4:20 PM, Nate Coraor n...@bx.psu.edu wrote: Peter Cock wrote: Hi Nate, Since you're looking at this kind of thing, would the Saved Histories page be worth updating to show the size on disk of each history? As a Galaxy user that seems moderately useful - although perhaps more of interest to Galaxy admins? This is already done in development, I was going to finish up the code and commit it within the next couple of weeks. This has now been committed to galaxy-central, and looks very useful. I notice the Saved Histories page now has buttons Rename, Delete, Delete and remove datasets from disk (new) and Undelete. That distinction between hide it away and really delete should make sense to users from the Recycle Bin/Trash idea for files on Windows/MacOSX. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] The new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 3:00 AM, Ravi Madduri madd...@mcs.anl.gov wrote: I apologize for jumping on to this thread a bit late. I read below that there is a plan to pull tools into a galaxy installation automagically. I wonder if you plan on providing some kind of API to query the tool registry and discover the tools and install them into an existing galaxy installation. Yes, have a look at Greg's slides from the Galaxy Community Conference http://wiki.g2.bx.psu.edu/GCC2011 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
Hi all, I just tried updating one of my tools on the new hg based Galaxy Tool Shed and ran into some issues. I guess I'm the first person to try this... First of all, there is no clear update action like there used to me. Could that be restored please? https://bitbucket.org/galaxy/galaxy-central/issue/585/hg-tool-shed-lacks-update-tool-action I tried using the upload files to repository option, and picked my tarball. I did not pick a location since I wanted the tar ball unpacked at the repository root, but this isn't what happened: https://bitbucket.org/galaxy/galaxy-central/issue/586/hg-tool-shed-upload-files-to-repository Finally, it seems there is currently no delete files action, so I can't fix this (delete the misplaced files) via the web interface. https://bitbucket.org/galaxy/galaxy-central/issue/584/cant-delete-files-in-new-hg-tool-shed Are you expecting tool authors to work primarily at the hg level? Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Display names accessible from Tool XML?
I'm building a tool that needs the display name of a file inside the
output of the tool.
To clarify, I have a tool that merges gene expression result from cufflinks
for many samples. It generates a matrix tab-delimited file that provides
the results with genes or transcript down one axis and samples across the
other with intensity values in the cells of the matrix.
I have already built into my galaxy server the ability to carry meaningful
sample names from the original input file all the way through to the output
of the final output of the cufflinks step of my workflow (this is
functionality I've already offered up to the galaxy team to include, and can
explain at a later date).
Okay, so the problem I have is from the XML, I want to be able to pass my
command-line script both the physical file names so the files can be
parsed, but I also want to be able to pass the script the display name to
use as the sample names for the columns in the file.
Is there a way using the metadata to grab this from the input? I see that I
can do something like input.metadata.dbkey and get the database key, but I
can't find a way to do anything like input.metadata.name or
input.metadata.display_name to give me what I want.
My example XML file is below.
Thanks,
Dave
tool id=cuff_exp_2_matrix name=Merge Cufflinks Exp Files to Matrix(files
as sample names) version=1.1.1
descriptionmerges multiple cufflinks expression output files (gene or
transcript) into a single matrix formatted file (id by sample, with
intensity values)/description
command interpreter=python
cuff_exp_2_matrix_new.py
$input1.metadata.display_name
$input1
$output1
#for $i in $inputs
${i.input.metadata.display_name}
${i.input}
#end for
/command
inputs
param name=input1 label=First file type=data format=tabular
help=Need to add more files? Use controls below./
repeat name=inputs title=Input Files
param name=input label=Add file type=data format=tabular /
/repeat
/inputs
outputs
data format=tabular name=output1 label=${tool.name} on
${on_string}: Expression matrix /
/outputs
help
**What it does**
This is a Jackson Laboratory custom tool for taking the output for several
runs of
cufflinks, where each run represents a sample, and merging these results
into a file
that contains a matrix of gene/transcript id by sample, with the cells of
the matrix
filled with the intensity values. The end result is a file that is easily
imported
into R or other tools for expression analysis.
/help
/tool
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Conditional parameters regression: KeyError: '__current_case__'
Hi Peter, If you were curious it was fixed about 7 hours later in 5681:0886ed0a8c9f Thanks for using Galaxy, Dan On Jun 16, 2011, at 4:32 AM, Peter Cock wrote: On Wed, Jun 15, 2011 at 4:56 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Dan, I just started work on updating my MIRA wrapper, and fell over a regression present of 6079:5e6254f2b9e3, steps to reproduce on Linux:- 1. Install latest Galaxy 2. Install my MIRA wrapper v0.0.1 from the tool shed (don't need MIRA for this) 3. Run Galaxy 4. Select the MIRA tool 5. Change Backbones/reference chromosomes? or Sanger/Capillary reads? or 454 reads? or Solexa/Illumina reads? to yes 6. Instead of new file parameter appearing, get a KeyError: '__current_case__' exception (details below). Shorter example without a 3rd party tool: 1. Install latest Galaxy 2. Run Galaxy 3. Select the Convert SOLiD output to fastq tool 4. Change Is this a mate-pair run? to yes 5. Instead of new file parameters appearing, get a KeyError: '__current_case__' exception. By going through recent commits and retesting, I found the commit which caused this regression: changeset: 6078:894112e0b9d5 user:Daniel Blankenberg d...@bx.psu.edu date:Thu Jun 09 11:26:41 2011 -0400 summary: Some fixes for interplay between DataToolParameter, implicit datatype conversion and dynamic select lists. Closes #491. I guess there are downsides to running my development environment off galaxy-central rather than galaxy-dist ... Peter This has been fixed as of 6116:124f8bcea0ff but I didn't bother to work out which change solved it. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Conditional parameters regression: KeyError: '__current_case__'
On Thu, Jun 16, 2011 at 2:23 PM, Daniel Blankenberg d...@bx.psu.edu wrote: Hi Peter, If you were curious it was fixed about 7 hours later in 5681:0886ed0a8c9f Thanks for using Galaxy, Dan Thanks - I know this kind of unexpected breakage is inevitable sometimes, but would recommend tool developers in general run with galaxy-dist or galaxy-central? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Conditional parameters regression: KeyError: '__current_case__'
Hi Peter, Production servers, or any server where you have users, should always track Galaxy-dist. Galaxy-central is a development repo (you get access to the latest and greatest, but also should Expect more bugs to pop up at any given time), from the Galaxy-central header on bitbucket: Main development repository for Galaxy. Active development happens here, and this repository is thus intended for those working on Galaxy development. See http://bitbucket.org/galaxy/galaxy-dist/ for a more stable repository intended for end-users. Its really a personal choice that each (tool) developer will have to make based upon whether they consider themselves to be a pure end-user (just adding tools or running a Galaxy server) who only wants to work on a stable branch; or if they want to contribute to Core development or gain early access to development features (and bugs). I would say that finalized tools and e.g. tools submitted to the tool shed should be vetted against galaxy-dist. Thanks for using Galaxy, Dan On Jun 16, 2011, at 9:26 AM, Peter Cock wrote: On Thu, Jun 16, 2011 at 2:23 PM, Daniel Blankenberg d...@bx.psu.edu wrote: Hi Peter, If you were curious it was fixed about 7 hours later in 5681:0886ed0a8c9f Thanks for using Galaxy, Dan Thanks - I know this kind of unexpected breakage is inevitable sometimes, but would recommend tool developers in general run with galaxy-dist or galaxy-central? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Conditional parameters regression: KeyError: '__current_case__'
On Thu, Jun 16, 2011 at 3:01 PM, Daniel Blankenberg d...@bx.psu.edu wrote: Hi Peter, Production servers, or any server where you have users, should always track Galaxy-dist. Yes, that's very clear. Galaxy-central is a development repo (you get access to the latest and greatest, but also should Expect more bugs to pop up at any given time), from the Galaxy-central header on bitbucket: Main development repository for Galaxy. Active development happens here, and this repository is thus intended for those working on Galaxy development. See http://bitbucket.org/galaxy/galaxy-dist/ for a more stable repository intended for end-users. Its really a personal choice that each (tool) developer will have to make based upon whether they consider themselves to be a pure end-user (just adding tools or running a Galaxy server) who only wants to work on a stable branch; or if they want to contribute to Core development or gain early access to development features (and bugs). I would say that finalized tools and e.g. tools submitted to the tool shed should be vetted against galaxy-dist. OK, that's basically how I've been working - quite often during tool development I've hit bugs in Galaxy and wanted to make branches with fixes etc, or needed new functionality for a tool that isn't yet in galaxy-dist. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-user] Error running latest Galaxy Distribution: numpy.dtype exception
Any other ideas? I can run galaxy-central but I'd like to work with the stable galaxy if possible. Because earlier revs of galaxy-dist startup fine for me I went through the exercise of using hg bisect to mark my last pull rev of 5355 as good and 5585 as bad I went through some revs, each time attempting to start the server. The bad revs failed to startup with ValueError 'numpy.dtype' exception where the good revs started fine. So according to hg bisect: The first bad revision is: changeset: 5410:0194ab30a730 user:Nate Coraor n...@bx.psu.edu date:Mon Apr 18 16:42:12 2011 -0400 summary: Update bx-python to the tip. I'm not sure from the diff what change in that revision could cause this error. What am I missing? Is anyone else having trouble running the latest galaxy-dist? ~Mike On Jun 15, 2011, at 12:18 PM, Nate Coraor wrote: Michael Steder wrote: I'm just following the documentation here: https://bitbucket.org/galaxy/galaxy-central/wiki/GetGalaxy And I ran into the following error: $ hg clone https://bitbucket.org/galaxy/galaxy-dist $ cd galaxy-dist $ sh run.sh Fetch successful. Traceback (most recent call last): File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/buildapp.py, line 81, in app_factory from galaxy.app import UniverseApplication File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/app.py, line 11, in module from galaxy.tools.imp_exp import load_history_imp_exp_tools File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py, line 6, in module from galaxy.web.base.controller import UsesHistory File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/base/controller.py, line 12, in module from galaxy.visualization.tracks.data_providers import get_data_provider File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py, line 16, in module from bx.arrays.array_tree import FileArrayTreeDict File numpy.pxd, line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) ValueError: numpy.dtype does not appear to be the correct type object I was able to run the previous stable version of Galaxy. Is there something I can do to get galaxy-dist running? Hi Mike, I'm moving this over to galaxy-dev since it deals with a local installation. It should be working, this would have been a problem for a short time using galaxy-central, but should not have affected -dist. Could you send the output of: python ./scripts/get_platforms.py python -V Should I be running galaxy-central instead? No, galaxy-dist is fine (and should be less prone to issues due to cloning/updating at the wrong moment). Thanks, --nate ~Mike ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
Hi Peter, On Jun 16, 2011, at 10:10 AM, Peter Cock wrote: On Thu, Jun 16, 2011 at 2:45 PM, Greg Von Kuster g...@bx.psu.edu wrote: Peter, I've added comments to each of these issues in bitbucket, and have pated them inline below as well. On Jun 16, 2011, at 5:03 AM, Peter Cock wrote: Hi all, I just tried updating one of my tools on the new hg based Galaxy Tool Shed and ran into some issues. I guess I'm the first person to try this... First of all, there is no clear update action like there used to me. Could that be restored please? https://bitbucket.org/galaxy/galaxy-central/issue/585/hg-tool-shed-lacks-update-tool-action The requested functionality is currently possible using normal mercurial features ( clone - make changes to the cloned repo - commit changes to the cloned repo - push changes to the tool shed repo ). Right, but not everyone wants to use hg for this. I understand, and that is why I am working feverishly to add features that do not force the use of hg form the command line (I've got several features functional, but not everything yet). The current tool shed requires some use of hg if you want to delete files from the repo, but other than that, hg is not really required. The current tool shed also provides some very useful features (e.g., browsing tool code files) that were not available in the old tool shed. In addition, tool developers are no longer forced to to the extra work that was necessary in the old tool shed to add a tool (i.e., there is no longer a requirement for tool suite config definitions ). My hope is that the current features make everyone happy enough that they will be able to wait for those messing featuers that are wanted, using the current features in the meantime. It is also currently possible to upload a new tarball to any upload point (position) in the tool shed repository by selecting the upload point in the built-in repository file browser. - The hierarchy that the uploaded tarball contains will be added to that upload point in the tool shed repository. - Any files in the uploaded tarball that do not currently exist in the tool shed repository will be added. - Any files that exist in the hierarchy of the uploaded tarball that also exist in the same location of the hierarchy of the repository relative to the upload point will be replaced in the repository. Yes, I tried that and its broken (bug 586 below). This is because you uploaded a gzipped tarball, which mercurial treats like a single file. The current tool shed is based on mercurial, so mercurial's behavior is basically the tool shed's behavior. It is on my list to determine if a file is compressed and if so uncompress it upon upload. A potential problem with this is that some developers may not want their uploaded file uncompressed. It is on my list to: Delete any files that do not exist in the uploaded tarball's hierarchy but that do existin the same position of the tool shed repository's hierarchy relative to the upload point. This feature will require clear documented communication to the user, which I've not yet had a chance to do, but will soon. That would be a change from upload files to replace all the files, in line with the missing update tool functionality (#585). To me mixing these two is quite strange. This should become more clear when the feature to delete files in the hierarchy is added. I tried using the upload files to repository option, and picked my tarball. I did not pick a location since I wanted the tar ball unpacked at the repository root, but this isn't what happened: https://bitbucket.org/galaxy/galaxy-central/issue/586/hg-tool-shed-upload-files-to-repository Peter, I believe this is a duplicate of # 584, correct? Not really, 584 is about deleting files via the web interface. Perhaps you regard 585 (missing tool update) and 586 (broken upload at root) as the same basic issue? See above explanation. Finally, it seems there is currently no delete files action, so I can't fix this (delete the misplaced files) via the web interface. https://bitbucket.org/galaxy/galaxy-central/issue/584/cant-delete-files-in-new-hg-tool-shed You can currently delete specific files by cloning the repository, making any changes you want (including deleting), and committing and pushing the changes using mercurial's command line. Again, I'd prefer not to have to do this. Understood... I've added it to my list to provide a feature allowing a user to select a specific file in the repository (hopefully using the built-in file browser) and deleting the file in some way that doesn't require mercurial's command line, but in the meantime, use the above existing feature. Are you expecting tool authors to work primarily at the hg level? You didn't answer that one ;) Not necessarily, but hg is the basis for uploading and downloading tools. I'm not sure if it will be possible
Re: [galaxy-dev] [galaxy-user] Error running latest Galaxy Distribution: numpy.dtype exception
Michael Steder wrote: Any other ideas? I can run galaxy-central but I'd like to work with the stable galaxy if possible. I can rebuild the bx_python egg, but it's odd this is happening - the old numpy eggs haven't changed in over a year. Could you remove ~/.python-eggs, and the bx_python and numpy eggs from Galaxy's eggs/ subdirectory, and then try again? Thanks, --nate Because earlier revs of galaxy-dist startup fine for me I went through the exercise of using hg bisect to mark my last pull rev of 5355 as good and 5585 as bad I went through some revs, each time attempting to start the server. The bad revs failed to startup with ValueError 'numpy.dtype' exception where the good revs started fine. So according to hg bisect: The first bad revision is: changeset: 5410:0194ab30a730 user:Nate Coraor n...@bx.psu.edu date:Mon Apr 18 16:42:12 2011 -0400 summary: Update bx-python to the tip. I'm not sure from the diff what change in that revision could cause this error. What am I missing? Is anyone else having trouble running the latest galaxy-dist? ~Mike On Jun 15, 2011, at 12:18 PM, Nate Coraor wrote: Michael Steder wrote: I'm just following the documentation here: https://bitbucket.org/galaxy/galaxy-central/wiki/GetGalaxy And I ran into the following error: $ hg clone https://bitbucket.org/galaxy/galaxy-dist $ cd galaxy-dist $ sh run.sh Fetch successful. Traceback (most recent call last): File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/buildapp.py, line 81, in app_factory from galaxy.app import UniverseApplication File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/app.py, line 11, in module from galaxy.tools.imp_exp import load_history_imp_exp_tools File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py, line 6, in module from galaxy.web.base.controller import UsesHistory File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/base/controller.py, line 12, in module from galaxy.visualization.tracks.data_providers import get_data_provider File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py, line 16, in module from bx.arrays.array_tree import FileArrayTreeDict File numpy.pxd, line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) ValueError: numpy.dtype does not appear to be the correct type object I was able to run the previous stable version of Galaxy. Is there something I can do to get galaxy-dist running? Hi Mike, I'm moving this over to galaxy-dev since it deals with a local installation. It should be working, this would have been a problem for a short time using galaxy-central, but should not have affected -dist. Could you send the output of: python ./scripts/get_platforms.py python -V Should I be running galaxy-central instead? No, galaxy-dist is fine (and should be less prone to issues due to cloning/updating at the wrong moment). Thanks, --nate ~Mike ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Setting up Galaxy with SGE
Yes. But right now it is kind of a hack to do sudo in the condor runner. We are working on integrating this with Globus online's (www.globusonline.org) identity and group management capability to make it more flexible. On Jun 15, 2011, at 8:40 PM, Chorny, Ilya wrote: Ravi, Do your galaxy jobs submit to the cluster as the galaxy user or as the user? Thanks, Ilya Sent from my iPhone On Jun 15, 2011, at 2:43 PM, Ravi Madduri madd...@mcs.anl.govmailto:madd...@mcs.anl.gov wrote: Hi We have been able to get the following setup working pretty reliably: * Automated deployment of Galaxy clusters on EC2. These clusters have the following setup: * A NFS server providing global directories for home directories, software, and scratch space. We are also experimenting with glusterfs * A NIS server providing an authentication domain within the cluster. The list of users must be specified when the cluster is created * A GridFTP server, which is automatically registered as a Globus Online (http://www.globusonline.orgwww.globusonline.orghttp://www.globusonline.org) endpoint for high speed and reliable data transfer. * Ability to script transfer tasks as part of the workflow using Globus Online Galaxy tools (To get large quantities of data in and out of galaxy EC2 cluster reliably) * A Condor pool with jobs running using user's credentials (using users accounts that are provisioned in the EC2 cluster) * A Galaxy server with the modifications outlined above, which we plan to contribute back to the galaxy community soon I gave a talk at the recent Galaxy users conference about this setup. You can find slides here: PowerPointhttp://wiki.g2.bx.psu.edu/GCC2011?action=AttachFiledo=gettarget=GalaxyGridFTPCondorAndGlobusOnline.pptx Please let me know if you are interested and would like more information. Regards On Jun 15, 2011, at 5:28 AM, Marina Gourtovaia wrote: There are two slightly different problems here. One with the files ystem and the second is whether the machine Galaxy is running on can submit jobs to the cluster (ssh is mentioned in the original e-mail, suggesting that it's impossible to communicate with the cluster from the machine the job is running on). The Galaxy instance is constantly communicating with the cluster job scheduling system (SGE or other) in order to get updates on the status of the jobs. It should be possible to do it over ssh, but, in my experience, this slows down the code. We run the Galaxy server on one of the cluster nodes. I imagine that other people using Galaxy with the cluster do the same. Are they? Marina On 15/06/2011 08:51, Peter Cock wrote: On Wed, Jun 15, 2011 at 2:30 AM, Ka Ming Nipmailto:km...@bcgsc.cakm...@bcgsc.camailto:km...@bcgsc.ca wrote: Hello, I have been trying to set up Galaxy to run its jobs on my SGE cluster using the Unified Method, based on the steps described in the wiki: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/Clusterhttps://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster My local instance of Galaxy is installed under my home directory, but my cluster is on a different file system. I need to ssh to the head node before I can submit jobs. Is there any way to set up Galaxy with a SGE cluster on a different file system? I haven't looked into it in great detail, but we currently have a similar network setup, and would also like to link Galaxy to SGE. The main problem is the lack of a shared file system - in order to run a job on the cluster we (Galaxy) has to get the data onto the cluster, run the job, and get the results back. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ http://lists.bx.psu.edu/ -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/http://lists.bx.psu.edu/ -- Ravi K Madduri The Globus Alliance | Argonne National Laboratory | University of Chicago http://www.mcs.anl.gov/~maddurihttp://www.mcs.anl.gov/~madduri ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ http://lists.bx.psu.edu/ --
Re: [galaxy-dev] [galaxy-user] Error running latest Galaxy Distribution: numpy.dtype exception
I tried the following: $ rm -rf ~/.python-eggs/ $ cd galaxy-dist $ rm -f ~/eggs/* $ sh run.sh Some eggs are out of date, attempting to fetch... Fetched http://eggs.g2.bx.psu.edu/Mako/Mako-0.2.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pysam/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/Babel/Babel-0.9.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Whoosh/Whoosh-0.3.18-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Tempita/Tempita-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Cheetah/Cheetah-2.2.2-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/lrucache/lrucache-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/NoseHTML/NoseHTML-0.4.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pexpect/pexpect-2.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/amqplib/amqplib-0.6.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/bx_python/bx_python-0.7.0_494c2d1d68b3-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/PasteDeploy/PasteDeploy-1.3.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebHelpers/WebHelpers-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/docutils/docutils-0.7-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/numpy/numpy-1.3.0-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/pysqlite/pysqlite-2.5.6_3.6.17_static-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/Beaker/Beaker-1.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SVGFig/SVGFig-1.1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SQLAlchemy/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/simplejson/simplejson-2.1.1-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/NoseTestDiff/NoseTestDiff-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_lzo/python_lzo-1.08_2.03_static-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/wchartype/wchartype-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/MarkupSafe/MarkupSafe-0.12-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/twill/twill-0.9-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Routes/Routes-1.12.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/elementtree/elementtree-1.2.6_20050316-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/decorator/decorator-3.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/GeneTrack/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pycrypto/pycrypto-2.0.1-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/WebError/WebError-0.8a-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Paste/Paste-1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/wsgiref/wsgiref-0.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_daemon/python_daemon-1.5.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/nose/nose-0.11.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/sqlalchemy_migrate/sqlalchemy_migrate-0.5.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebOb/WebOb-0.8.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/PasteScript/PasteScript-1.7.3-py2.6.egg Fetch successful. Traceback (most recent call last): File /Users/steder/galaxy-dist/lib/galaxy/web/buildapp.py, line 81, in app_factory from galaxy.app import UniverseApplication File /Users/steder/galaxy-dist/lib/galaxy/app.py, line 11, in module from galaxy.tools.imp_exp import load_history_imp_exp_tools File /Users/steder/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py, line 6, in module from galaxy.web.base.controller import UsesHistory File /Users/steder/galaxy-dist/lib/galaxy/web/base/controller.py, line 12, in module from galaxy.visualization.tracks.data_providers import get_data_provider File /Users/steder/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py, line 16, in module from bx.arrays.array_tree import FileArrayTreeDict File numpy.pxd, line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) ValueError: numpy.dtype does not appear to be the correct type object :-/ ~Mike On Jun 16, 2011, at 9:29 AM, Nate Coraor wrote: Michael Steder wrote: Any other ideas? I can run galaxy-central but I'd like to work with the stable galaxy if possible. I can rebuild the bx_python egg, but it's odd this is happening - the old numpy eggs haven't changed in over a year. Could you remove ~/.python-eggs, and the bx_python and numpy eggs from Galaxy's eggs/ subdirectory, and then try again? Thanks, --nate Because earlier revs of galaxy-dist startup fine for me I went through the exercise of using hg bisect to mark my last pull rev of 5355 as good and 5585 as bad I went through some revs, each time attempting to start the server. The bad revs failed to startup with ValueError 'numpy.dtype' exception where the good revs started fine. So according to hg bisect: The first bad revision is: changeset: 5410:0194ab30a730 user:Nate Coraor n...@bx.psu.edu date:
Re: [galaxy-dev] CSV import of Sample information Error
Hello Joe, I've enhanced the feature for defining sample form rows in change set 5716:6e1576c83456 (currently available only in our development repo, but will be available in the distribution shortly) to include the ability to include field values (in addition to field names) when defining sample form rows by importing from a csv file. The csv file must be in the following format: The [:FieldValue] is optional, the named form field will contain the value after the ':' if included. SampleName,DataLibraryName,FolderName,HistoryName,WorkflowName,Field1Name:Field1Value,Field2Name:Field2Value... Let me know if you run into additional issues when using this new feature. Thanks for all of your patience on this. On Jun 10, 2011, at 3:40 PM, Joe Cruz wrote: Hey Greg, Sorry for not being clear. I'm trying to create a sample information form where users can import csv files which would contain DIFFERENT values for each field. So, while one sample may have value 'agctac' for the last field, another sample will have a different value, say '123. So, these can't be hardcoded in the sample form definition. Since these are form fields, of course the field NAMES should remain standard and field VALUES should will keep changing. From what I understand, the sample form definition will have the field NAMES and the csv file will have different field VALUES. Galaxy should map the field VALUES to the field NAMES. My form has the following field names: (change this accordingly) SampleName LibraryName LibraryFolder History Workflow Barcode A user may submit a csv file such as: Sample1,Library1,Folder1,History1,Workflow1,BarcodeAC Sample10,Library10,Folder10,History10,Workflow10,BarcodeDE Galaxy should take this csv file, parse it, create two samples (Sample1 and Sample10) and assign the rest of the values to the corresponding fields for each sample. Thanks, joe On Thu, Jun 9, 2011 at 8:50 PM, Greg Von Kuster g...@bx.psu.edu wrote: Joe, Try editing your Sample Form Definition, changing the Field name value from the default of 1_field_name to be your desired agctac. Then, your csv file with the line Sample1,Library1,Lib1Folder1,History,Workflow,agctac should work. Greg On Jun 9, 2011, at 6:03 PM, Joe Cruz wrote: Hey Greg, I got it to work by filling the field names with their default names, but I thought you could fill out the entire sample forms, including the field names, with the csv file. For example: Sample1,Library1,Lib1Folder1,History,Workflow,1_field_name The above works when imported, filling out the sample name, library, folder, history, and workflow fields but leaving 1_field_name (barcode field in this case) empty. We wish this next example to work,which I currently get the '1_field_name' error: Sample1,Library1,Lib1Folder1,History,Workflow,agctac Currently we have clients fill out a spreadsheet for the samples they intend to submit, since many submit multiple samples and most of the time have the same parameters for all of them. We wish to convert their completed spreadsheet into a csv file and then import it into galaxy so they dont' have to fill out all the fields repetitively. joe On Thu, Jun 9, 2011 at 3:20 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Joe, Sorry you've bumped into problems - we know we're lacking on documentation for the sample tracking stuff currently. I believe this new issue is due to a field named differently in your Sample FormDefinition vs your csf file. In looking at this, I found a slightly incorrect description of the csv file layout on our Add Samples page, which I've corrected. In the Sample FormDefinition, the default name for a field is 1_field_name, 2_field_name, etc. Below is a screen capture that show this. However, the csv file you sent me previously includes the following: SampleName1,library1,library1_folder1,history1,workflow1,sampleType1,conc,runType,readLength The csv layout is as follows ( I've correct the incorrect FieldValue1, etc to be Field1Name, etc. The csv file must be in the following format: SampleName,DataLibraryName,DataLibraryFolderName,HistoryName,WorkflowName,Field1Name,Field2name.. The Field1Name, Field2Name values in your csv file must match the value in the Field name form field in the screen shot below. After making this change, let me know if you bump into any other problems. Thanks for you patience on this, Greg sample_form.tiff On Jun 9, 2011, at 12:32 PM, Joe Cruz wrote: Hey Greg, Unfortunately after pulling your changeset I got a new error related to '1_field_name'. Here is the error traceback: URL: http://localhost:8080/requests_common/add_samples?cntrller=requestsid=f2db41e1fa331b3e File '/home/jcruz/galaxy-central/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter =
Re: [galaxy-dev] [galaxy-user] Error running latest Galaxy Distribution: numpy.dtype exception
Michael Steder wrote: I tried the following: $ rm -rf ~/.python-eggs/ $ cd galaxy-dist $ rm -f ~/eggs/* $ sh run.sh Some eggs are out of date, attempting to fetch... Fetched http://eggs.g2.bx.psu.edu/Mako/Mako-0.2.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pysam/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/Babel/Babel-0.9.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Whoosh/Whoosh-0.3.18-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Tempita/Tempita-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Cheetah/Cheetah-2.2.2-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/lrucache/lrucache-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/NoseHTML/NoseHTML-0.4.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pexpect/pexpect-2.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/amqplib/amqplib-0.6.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/bx_python/bx_python-0.7.0_494c2d1d68b3-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/PasteDeploy/PasteDeploy-1.3.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebHelpers/WebHelpers-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/docutils/docutils-0.7-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/numpy/numpy-1.3.0-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/pysqlite/pysqlite-2.5.6_3.6.17_static-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/Beaker/Beaker-1.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SVGFig/SVGFig-1.1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SQLAlchemy/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/simplejson/simplejson-2.1.1-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/NoseTestDiff/NoseTestDiff-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_lzo/python_lzo-1.08_2.03_static-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/wchartype/wchartype-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/MarkupSafe/MarkupSafe-0.12-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/twill/twill-0.9-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Routes/Routes-1.12.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/elementtree/elementtree-1.2.6_20050316-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/decorator/decorator-3.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/GeneTrack/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pycrypto/pycrypto-2.0.1-py2.6-macosx-10.6-universal-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/WebError/WebError-0.8a-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Paste/Paste-1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/wsgiref/wsgiref-0.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_daemon/python_daemon-1.5.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/nose/nose-0.11.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/sqlalchemy_migrate/sqlalchemy_migrate-0.5.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebOb/WebOb-0.8.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/PasteScript/PasteScript-1.7.3-py2.6.egg Fetch successful. Traceback (most recent call last): File /Users/steder/galaxy-dist/lib/galaxy/web/buildapp.py, line 81, in app_factory from galaxy.app import UniverseApplication File /Users/steder/galaxy-dist/lib/galaxy/app.py, line 11, in module from galaxy.tools.imp_exp import load_history_imp_exp_tools File /Users/steder/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py, line 6, in module from galaxy.web.base.controller import UsesHistory File /Users/steder/galaxy-dist/lib/galaxy/web/base/controller.py, line 12, in module from galaxy.visualization.tracks.data_providers import get_data_provider File /Users/steder/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py, line 16, in module from bx.arrays.array_tree import FileArrayTreeDict File numpy.pxd, line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) ValueError: numpy.dtype does not appear to be the correct type object :-/ Arright, I'm running out of ideas, the same platform and python version on a fresh checkout works fine for me and the other developers. Could you remove just your bx_python egg and then scramble it locally: % rm -rf eggs/bx_python* % python ./scripts/scramble.py bx_python Sorry for all the hassle, --nate ~Mike On Jun 16, 2011, at 9:29 AM, Nate Coraor wrote: Michael Steder wrote: Any other ideas? I can run galaxy-central but I'd like to work with the stable galaxy if possible. I can rebuild the bx_python egg, but it's odd this is happening - the old numpy eggs haven't changed in over a year. Could you remove ~/.python-eggs, and the bx_python and numpy eggs from Galaxy's eggs/ subdirectory, and then try again? Thanks, --nate Because earlier revs of galaxy-dist startup fine
Re: [galaxy-dev] [galaxy-user] Error running latest Galaxy Distribution: numpy.dtype exception
On Jun 16, 2011, at 9:55 AM, Nate Coraor wrote: Arright, I'm running out of ideas, the same platform and python version on a fresh checkout works fine for me and the other developers. Could you remove just your bx_python egg and then scramble it locally: % rm -rf eggs/bx_python* % python ./scripts/scramble.py bx_python Sorry for all the hassle, --nate It's not too much hassle, I just wanted to be sure I could run galaxy-dist before upgrading my working repo. Nate, that worked. Thanks for the help! ~Mike ~Mike On Jun 16, 2011, at 9:29 AM, Nate Coraor wrote: Michael Steder wrote: Any other ideas? I can run galaxy-central but I'd like to work with the stable galaxy if possible. I can rebuild the bx_python egg, but it's odd this is happening - the old numpy eggs haven't changed in over a year. Could you remove ~/.python-eggs, and the bx_python and numpy eggs from Galaxy's eggs/ subdirectory, and then try again? Thanks, --nate Because earlier revs of galaxy-dist startup fine for me I went through the exercise of using hg bisect to mark my last pull rev of 5355 as good and 5585 as bad I went through some revs, each time attempting to start the server. The bad revs failed to startup with ValueError 'numpy.dtype' exception where the good revs started fine. So according to hg bisect: The first bad revision is: changeset: 5410:0194ab30a730 user:Nate Coraor n...@bx.psu.edu date:Mon Apr 18 16:42:12 2011 -0400 summary: Update bx-python to the tip. I'm not sure from the diff what change in that revision could cause this error. What am I missing? Is anyone else having trouble running the latest galaxy-dist? ~Mike On Jun 15, 2011, at 12:18 PM, Nate Coraor wrote: Michael Steder wrote: I'm just following the documentation here: https://bitbucket.org/galaxy/galaxy-central/wiki/GetGalaxy And I ran into the following error: $ hg clone https://bitbucket.org/galaxy/galaxy-dist $ cd galaxy-dist $ sh run.sh Fetch successful. Traceback (most recent call last): File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/buildapp.py, line 81, in app_factory from galaxy.app import UniverseApplication File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/app.py, line 11, in module from galaxy.tools.imp_exp import load_history_imp_exp_tools File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py, line 6, in module from galaxy.web.base.controller import UsesHistory File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/base/controller.py, line 12, in module from galaxy.visualization.tracks.data_providers import get_data_provider File /Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py, line 16, in module from bx.arrays.array_tree import FileArrayTreeDict File numpy.pxd, line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) ValueError: numpy.dtype does not appear to be the correct type object I was able to run the previous stable version of Galaxy. Is there something I can do to get galaxy-dist running? Hi Mike, I'm moving this over to galaxy-dev since it deals with a local installation. It should be working, this would have been a problem for a short time using galaxy-central, but should not have affected -dist. Could you send the output of: python ./scripts/get_platforms.py python -V Should I be running galaxy-central instead? No, galaxy-dist is fine (and should be less prone to issues due to cloning/updating at the wrong moment). Thanks, --nate ~Mike ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 3:26 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, I understand, and that is why I am working feverishly to add features that do not force the use of hg form the command line (I've got several features functional, but not everything yet). The current tool shed requires some use of hg if you want to delete files from the repo, but other than that, hg is not really required. That's great, but I do feel the new hg went live a bit prematurely. The current tool shed also provides some very useful features (e.g., browsing tool code files) that were not available in the old tool shed. Really? I find the browsing the tool code files has regressed. In old tool shed had a simple non-interactive tree of the files visible on the main page for a tool or suite, where the individual files could be clicked on to view/download. It was simple and effective (at least when there weren't many files which seems typical). The new tool shed makes you click on tool actions, browse repo, and then gives an interactive tree which is fully collapsed by default, forcing lots more clicking to drill down to the files. All that clicking makes it slow and fiddly to use. In addition, tool developers are no longer forced to to the extra work that was necessary in the old tool shed to add a tool (i.e., there is no longer a requirement for tool suite config definitions ). For tool suites you have a point, that was a bit of a hassle. My hope is that the current features make everyone happy enough that they will be able to wait for those messing featuers that are wanted, using the current features in the meantime. It'll get there :) Another big regression I would prioritise is the complete lack of any user provided text on a tool's main page. The old tool shed had the minimal text box where you could summarise the tool, its requirements, recent changes etc. The new tool shed has nothing to replace this as far as I can see. A simple wiki like, ReST, or just plain text details box would do. Automatic detection of a readme text file in the repo, to be displayed on the tools' main page would be an elegant solution (are you familiar with how github.com does this?). It also keeps the description in the hg repo which is a plus point. The mock up idea would be another (more complex) way to tackle this, https://bitbucket.org/galaxy/galaxy-central/issue/565 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Cluster Usage
Does anyone have practical information for installation and set up for the galaxy distribution on a cluster?? Robert C. Jackson Software Systems Specialist III The University of Texas Pan-American Computer Support Services Division of Information Technology Phone: 956-665-2455 Fax: 956-665-8777 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Cluster Usage
On Thu, Jun 16, 2011 at 4:31 PM, Robert Jackson rj...@utpa.edu wrote: Does anyone have practical information for installation and set up for the galaxy distribution on a cluster?? Did you not get these replies? http://lists.bx.psu.edu/pipermail/galaxy-user/2011-June/002715.html http://lists.bx.psu.edu/pipermail/galaxy-user/2011-June/002715.html http://lists.bx.psu.edu/pipermail/galaxy-user/2011-June/002716.html On the plus side, the galaxy-dev list is more appropriate for this question than galaxy-user list. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Jun 16, 2011, at 10:23 AM, Peter Cock wrote: On Thu, Jun 16, 2011 at 3:26 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, On Jun 16, 2011, at 10:10 AM, Peter Cock wrote: Are you expecting tool authors to work primarily at the hg level? You didn't answer that one ;) Not necessarily, but hg is the basis for uploading and downloading tools. I'm not sure if it will be possible to completely eliminate the requirement of a tool developer using hg, but we're making every attempt to do so. Good. Why are you so averse to using hg? Because git suits me better? ;) Seriously, I have no strong aversion to hg - what puts me off a little is the need to have one repo per tool or tool-suite, compared to my current setup where all by tools are in a branch from the main Galaxy repo. There would be a significant time and effort cost in switching, made worse by having multiple tools on the Tool Shed. I'm actually thinking of this (requiring hg knowledge) as a more general issue, namely a potential impediment to new Tool Shed contributors. Peter I'm of the opposite mind; I'm not sure there is an absolute need to have one's tools be a branch or fork of the main tool shed, though I agree there is also utility in allowing that (having a customized 'main' repo, or optimizing tools already present). Having completely separate repos for tools seems cleaner, focusing development on those tools alone (not any of the others present in a branch) and pushes maintenance of the tool back to the tool developer themselves (e.g. there is no need for the galaxy devs to 'pull' in changes at any point into a main repo). This might also allow the galaxy devs to designate a set of 'blessed' or supported tools in various repositories. Re: git: as Peter knows I'm also primarily a git/github user. It is feasible at some future point to allow git/github repos. For instance, one could possibly integrate github usage via this: http://hg-git.github.com/ Of course, I think it's much more important that any additional vcs integration wait until the new tool shed interface stabilizes somewhat, but (at least from the github perspective) seems like it shouldn't be terribly hard to do. chris ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 5:40 PM, Chris Fields cjfie...@illinois.edu wrote: I'm of the opposite mind; I'm not sure there is an absolute need to have one's tools be a branch or fork of the main tool shed, though I agree there is also utility in allowing that (having a customized 'main' repo, or optimizing tools already present). To clarify, I have been using branches on an hg fork of the main Galaxy repository up until now, then preparing tarballs for upload to the Galaxy Tool Shed. I could equally have tracked my local changes under git, svn or cvs. The point is under this model, the Tool Shed has no connection to whatever repository (or repositories) the tool author uses on their machine. Barring teething issues like Bug 584/585/586 tool authors could continue to use this development model, where the fact that the Galaxy Tool Shed uses an hg repository internally is an irrelevant internal implementation detail. As far as I know, there are no plans to make the Tool Shed work directly with a full Galaxy hg repo - only mini-repositories, one for each tool or tool suite. Having completely separate repos for tools seems cleaner, focusing development on those tools alone (not any of the others present in a branch) and pushes maintenance of the tool back to the tool developer themselves Yes, and that is the model the new Galaxy Tool Shed is following. Although I'm a little hazy on the details of tool suites in the new model (and there are lots of situations where it makes sense to bundle several tools). (e.g. there is no need for the galaxy devs to 'pull' in changes at any point into a main repo). Well, there still is for general bug fixes, adding new file formats, and merging any tools which they decide to include in the core set. This might also allow the galaxy devs to designate a set of 'blessed' or supported tools in various repositories. Indeed. Re: git: as Peter knows I'm also primarily a git/github user. It is feasible at some future point to allow git/github repos. For instance, one could possibly integrate github usage via this: http://hg-git.github.com/ Of course, I think it's much more important that any additional vcs integration wait until the new tool shed interface stabilizes somewhat, but (at least from the github perspective) seems like it shouldn't be terribly hard to do. True - if there were sufficient demand from Tool Shed contributors. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Fwd: a Trackster Question
Matt, I'm moving your Trackster question onto the galaxy-dev list: one issue I keep running into is that it's not super clear how to go about configuring the dynamic tracks you showed off in your talk. Is there some guidance somewhere that I am missing on this? I confess my own motive is to be able to dynamically interact with a sliding window-based algorithm for measuring DNA replication status from sequencing data. This would help us parameterize the algorithm before running it on the entire data set. This (very new) galaxy-central changeset: https://bitbucket.org/galaxy/galaxy-central/changeset/a7e9af183746 formalizes tool integration into Trackster. To enable the Show Tool option in Trackster for a particular tool, two things need to happen: (a) add the trackster_conf/ tag to the tool XML file; (b) ensure that the tool has parameters supported by Trackster; only integer, float, and static select parameters are currently supported in a tool's interface, but it is straightforward to add support for other parameter types as well, and we'll do so as needed. We'll create a wiki page soon so that this information is easier to find. Let us know if you have more questions. Best, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-user] SRMA on a local galaxy instance?
Henry, I'm moving this over to galaxy-dev since it has to do with a local instance. It definitely sounds like you've got things set up right. You don't say what the result of your run is--is it successful (green) but an empty file? Or is there an error? If it's empty, it's possible your data just didn't yield any results. If there's an error, please share that with us. There are a couple of other things you could do to start figure out where the problem is. Have you tried running the functional tests for SRMA? There are two and you could comment out the first one in the xml since it relies on hg18chr21 being in the loc file, but if the second one passes then we'd know SRMA is installed correctly but there is something with your data (the command is: sh run_functional_tests.sh - id srma_wrapper). You also could try running your data through on our test server's (test.g2.bx.psu.edu) version of SRMA to see if if produces any results. Let us know if you need further assistance. Thanks, Kelly On Thu Jun 16, at 12:49 PM, Gong, Henry wrote: I may have just sent an empty reply email; if so, my apologies. Yes, I have the srma jar in that folder, without the version number exactly like that. Cheers, Henry -Original Message- From: Nate Coraor [mailto:n...@bx.psu.edu] Sent: Thu 6/16/2011 7:35 AM To: Gong, Henry Cc: galaxy-u...@lists.bx.psu.edu; Shieh, Joseph Subject: Re: [galaxy-user] SRMA on a local galaxy instance? Gong, Henry wrote: Does anyone know how to get SRMA to work properly in galaxy? I've used the pre-built 0.1.15 jar as well as a 0.1.16 jar I built from source, but the result of an SRMA re-alignment job in both instances is a python process which only takes a few percent CPU (or less, since it only seems to be outputting timestamps to the terminal). Here are my system's specs if they're important: 2.8Ghz Intel i5, 4 core; 4GB 1333MHz DDR3; 1TB HDD (430GB free). The OS is OS X 10.6.7. I've also built the required indices and placed them in path/to/galaxy-dist/hg19/srma_path/hg19.dict, hg19.fa.fai, and hg19.fa; the hg19 is a concatenated version of the various chromosome and contig files, and I've followed the sample files in adding these locations to the .loc files (both srma and picard tools). The NGS installation page seems to suggest that SRMA does work, so I'm not sure if I'm overreacting, but I'd really appreciate any advice on this, since I imagine others have had similar problems. Hi Henry, Where is your srma JAR file located? It should be /path/to/galaxy-dist/tool-data/shared/jars/srma.jar --nate ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-user] SRMA on a local galaxy instance?
Hi Kelly and others, Thanks for the move. The SRMA run doesn't seem to go to completion; instead, it stays yellow and produces no output (the corresponding dataset is 0KB). I did try the second test in srma_wrapper, and here is the error I got: Traceback (most recent call last): File /Users/user/galaxy-dist/eggs/nose-0.11.1-py2.4.egg/nose/failure.py, line 39, in runTest raise self.exc_class(self.exc_val) OSError: No such file /Users/user/galaxy-dist/- I'm not sure what that file is supposed to be. Here's the whole output on pastebin, just in case it helps: http://pastebin.com/yUE0LmGL Thanks! Henry -Original Message- From: Kelly Vincent [mailto:kpvinc...@bx.psu.edu] Sent: Thu 6/16/2011 10:20 AM To: Gong, Henry Cc: Nate Coraor; Shieh, Joseph; Galaxy Dev Subject: Re: [galaxy-user] SRMA on a local galaxy instance? Henry, I'm moving this over to galaxy-dev since it has to do with a local instance. It definitely sounds like you've got things set up right. You don't say what the result of your run is--is it successful (green) but an empty file? Or is there an error? If it's empty, it's possible your data just didn't yield any results. If there's an error, please share that with us. There are a couple of other things you could do to start figure out where the problem is. Have you tried running the functional tests for SRMA? There are two and you could comment out the first one in the xml since it relies on hg18chr21 being in the loc file, but if the second one passes then we'd know SRMA is installed correctly but there is something with your data (the command is: sh run_functional_tests.sh - id srma_wrapper). You also could try running your data through on our test server's (test.g2.bx.psu.edu) version of SRMA to see if if produces any results. Let us know if you need further assistance. Thanks, Kelly On Thu Jun 16, at 12:49 PM, Gong, Henry wrote: I may have just sent an empty reply email; if so, my apologies. Yes, I have the srma jar in that folder, without the version number exactly like that. Cheers, Henry -Original Message- From: Nate Coraor [mailto:n...@bx.psu.edu] Sent: Thu 6/16/2011 7:35 AM To: Gong, Henry Cc: galaxy-u...@lists.bx.psu.edu; Shieh, Joseph Subject: Re: [galaxy-user] SRMA on a local galaxy instance? Gong, Henry wrote: Does anyone know how to get SRMA to work properly in galaxy? I've used the pre-built 0.1.15 jar as well as a 0.1.16 jar I built from source, but the result of an SRMA re-alignment job in both instances is a python process which only takes a few percent CPU (or less, since it only seems to be outputting timestamps to the terminal). Here are my system's specs if they're important: 2.8Ghz Intel i5, 4 core; 4GB 1333MHz DDR3; 1TB HDD (430GB free). The OS is OS X 10.6.7. I've also built the required indices and placed them in path/to/galaxy-dist/hg19/srma_path/hg19.dict, hg19.fa.fai, and hg19.fa; the hg19 is a concatenated version of the various chromosome and contig files, and I've followed the sample files in adding these locations to the .loc files (both srma and picard tools). The NGS installation page seems to suggest that SRMA does work, so I'm not sure if I'm overreacting, but I'd really appreciate any advice on this, since I imagine others have had similar problems. Hi Henry, Where is your srma JAR file located? It should be /path/to/galaxy-dist/tool-data/shared/jars/srma.jar --nate ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
