[galaxy-dev] Manually installing rsem_datatypes package from toolshed
I'm trying to manually install rsem_datatypes package into my local Galaxy install. I can't use the Toolshed for various reasons. Anyway, I downloaded the package using hg clone https://toolshed.g2.bx.psu.edu/repos/jjohnson/rsem_datatypes I see two files: 1) datatypes_conf.xml, and 2) rsem.py I copied the datatype extension ... lines into my config/datatypes_conf.xml file. I also copied rsem.py to lib/galaxy/datatypes/. When I restart Galaxy, it doesn't seem that Galaxy knows how to handle rsem datatypes, as I get this error: galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File /data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py, line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File /data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py, line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,520 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File /data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py, line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' I'm obviously missing something but not sure what. I suspect maybe the rsem.py file is not in the right place? Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py
On 21.01.2015 16:11, John Chilton wrote: So the tool dependency stuff didn't change - but the local job runner now behaves like the other job runners (DRMAA, PBS, etc...) so you have uncovered a bug for all of them I think. Is this a tool shed tool or a locally created dependency? This is a tool shed tool (package_mimodd_0_1_5), but one I uploaded myself (I discovered the bug when I did a test install after updating our local Galaxy). I need to think about how to solve this more generally but if you want a quick work around - I think it would work to just replace python in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's python or perhaps just python2. Since I can change the tool shed tool a more global solution could be to simply remove the python symlink from the python3 virtualenv after its created and have the tool use python3 explicitly at runtime. I'll try whether that works, but I think it should. Sorry for the inconvenience and thanks for bringing this to our attention. No problem. I'm glad if I could help with something instead of just asking questions all the time. Wolfgang On Tue, Jan 20, 2015 at 12:52 PM, John Chilton jmchil...@gmail.com wrote: Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)? -John On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de wrote: On 01/20/2015 06:20 PM, Wolfgang Maier wrote: I have not seen this error before (I believe not with latest_2014.10.06) update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again. The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately. Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] dataset collections
Hi all, I’m toying around a little in galaxy-dist with the dataset collections feature. Since I know this is work in progress, I was wondering about some things I haven’t really found online. It seems to work really well to run a tool on a list of datasets, and a new job is run for each list item. But when I want to reduce to a smaller amount of list items, I understand I need to write some sort of merge tool myself, dependent on the data (all proteomics data here currently). This works well for reducing a dataset to a single file, but I am not sure about how to reduce to a new smaller collection. In the tool I’m writing, I let the user choose the size of the collection. Is there some way to tell galaxy dynamically how many outputs to expect AND put them in a collection? Something like: outputs output type=“data_collection” amount_of_files=“3”/ /outputs Where 3 is set by the user in a param also. Also, when running with two or more lists as input, is there some sort of correlation between the lists? It seems like it takes the files in dataset no order, so just checking. By the way, thanks very much John and everyone else involved in collections for doing and pushing this stuff. If there are smaller issues I can help with, I’d be thrilled. Can’t stress enough how much this feature means for galaxy adoption in our lab and possibly field. cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py
Hi, Could this be related to the fact that galaxy is not seeing the tools that are installed in /usr/local/bin?? I'm on latest_2015.01.13 and since the two last upgrades I saw that interproscan and bsmap were not found by galaxy. Both are installed in /usr/local and when invoked on the command line, as the galaxy user, they work perfectly well. Before the last two upgrades both worked from within galaxy. Cristian Date: Wed, 21 Jan 2015 10:11:43 -0500 From: jmchil...@gmail.com To: wolfgang.ma...@biologie.uni-freiburg.de CC: galaxy-...@lists.bx.psu.edu Subject: Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py So the tool dependency stuff didn't change - but the local job runner now behaves like the other job runners (DRMAA, PBS, etc...) so you have uncovered a bug for all of them I think. Is this a tool shed tool or a locally created dependency? I need to think about how to solve this more generally but if you want a quick work around - I think it would work to just replace python in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's python or perhaps just python2. Sorry for the inconvenience and thanks for bringing this to our attention. -John On Tue, Jan 20, 2015 at 12:52 PM, John Chilton jmchil...@gmail.com wrote: Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)? -John On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de wrote: On 01/20/2015 06:20 PM, Wolfgang Maier wrote: I have not seen this error before (I believe not with latest_2014.10.06) update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again. The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately. Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Manually installing rsem_datatypes package from toolshed
I believe this is an ancient bug that I has been fixed in Galaxy's central branch but is not in the latest release: https://bitbucket.org/galaxy/galaxy-central/commits/1422966f1ca88472b8685015f5a8cdd3dd0f1db9 The previous workaround for this bug was that one would need to import the datatype at the top of registry.py (import rsem_datatypes) which is another option for addressing this problem instead of applying the patch. Hope this helps, -John On Wed, Jan 21, 2015 at 2:16 PM, Ryan G ngsbioinformat...@gmail.com wrote: I'm trying to manually install rsem_datatypes package into my local Galaxy install. I can't use the Toolshed for various reasons. Anyway, I downloaded the package using hg clone https://toolshed.g2.bx.psu.edu/repos/jjohnson/rsem_datatypes I see two files: 1) datatypes_conf.xml, and 2) rsem.py I copied the datatype extension ... lines into my config/datatypes_conf.xml file. I also copied rsem.py to lib/galaxy/datatypes/. When I restart Galaxy, it doesn't seem that Galaxy knows how to handle rsem datatypes, as I get this error: galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File /data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py, line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File /data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py, line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,520 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File /data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py, line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' I'm obviously missing something but not sure what. I suspect maybe the rsem.py file is not in the right place? Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Setting a file download link in Galaxy help
The help text is interpreted as reStructuredText ( http://docutils.sourceforge.net/rst.html) not HTML. Here is an example link for the citation used in Galaxy's MAF tools - it demonstrates one way to create links in tools using restructuredtext: If you use this tool, please cite `Blankenberg D, Taylor J, Nekrutenko A; The Galaxy Team. Making whole genome multiple alignments usable for biologists. Bioinformatics. 2011 Sep 1;27(17):2426-2428. lt; http://www.ncbi.nlm.nih.gov/pubmed/21775304gt;`_ Hope this helps, -John On Mon, Jan 19, 2015 at 10:49 PM, XiaoTao Jiang biofuture.ji...@gmail.com wrote: Dear all I have installed Galaxy in local and I want to set a file download link in galaxy help. I want the user to download a small data-set from the help with a link. However, the general HTML download syntax does not take effect, look the figure I set. Anyone knows how to set the download link? [image: 内嵌图片 1] Thanks for all your attention. Regards Jimmy -- Department of Engineering The University of HongKong ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py
On Wed, Jan 21, 2015 at 6:54 PM, John Chilton jmchil...@gmail.com wrote: On Wed, Jan 21, 2015 at 10:35 AM, Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de wrote: On 21.01.2015 16:11, John Chilton wrote: So the tool dependency stuff didn't change - but the local job runner now behaves like the other job runners (DRMAA, PBS, etc...) so you have uncovered a bug for all of them I think. Is this a tool shed tool or a locally created dependency? This is a tool shed tool (package_mimodd_0_1_5), but one I uploaded myself (I discovered the bug when I did a test install after updating our local Galaxy). I need to think about how to solve this more generally but if you want a quick work around - I think it would work to just replace python in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's python or perhaps just python2. Since I can change the tool shed tool a more global solution could be to simply remove the python symlink from the python3 virtualenv after its created and have the tool use python3 explicitly at runtime. I'll try whether that works, but I think it should. This sounds like a better short-term solution to me - this way older versions of Galaxy will still be able to use the recipe. I created a Trello issue to track the longer term fix of isolating metadata generation from the tool environment - which is needed but could be a little disruptive so I would rather push it off until after the next release. Opps - I forgot the link and forgot to cc the mailing list. https://trello.com/c/aMCwYZsN -John Thanks! -John Sorry for the inconvenience and thanks for bringing this to our attention. No problem. I'm glad if I could help with something instead of just asking questions all the time. Wolfgang On Tue, Jan 20, 2015 at 12:52 PM, John Chilton jmchil...@gmail.com wrote: Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)? -John On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de wrote: On 01/20/2015 06:20 PM, Wolfgang Maier wrote: I have not seen this error before (I believe not with latest_2014.10.06) update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again. The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately. Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy and torque - resource allocation
Was hoping someone with an actual torque cluster would respond. I think the way you would configure this might be for instance might be setting native_specification like -l nodes=1:ppn=n. Depending on how things are configured or simply because I am ignorant - this may be wrong - but I think what you really want should look a lot like your arguments to qsub. So let say you have defined a job runner plugin called pbs_drmaa at the top of your job_conf.xml file. Then you could default everything to a single core by default and send big jobs to a destination with 8 cores with destinations and tool sections that look something like this... ... destinations default=singlecore destination id=singlecore runner=pbs_drmaa param id=native_specification-l nodes=1:ppn=1/param /destination destination id=multicore runner=pbs_drmaa param id=native_specification-l nodes=1:ppn=8/param /destination /destinations and ... tools tool id=bowtie2 destination=multicore / tool id=bowtie destination=multicore / tool id=deeptools destination=multicore / tool id=cufflinksdestination=multicore / tool id=cuffdiff destination=multicore / tool id=cuffmerge destination=multicore / tool id=tophat2 destination=multicore / /tools Again - that native specification could be wrong - probably best to test it out locally (or maybe Nate or JJ will step in and correct me). Hope this helps, -John On Mon, Jan 19, 2015 at 8:05 AM, Fernandez Edgar edgar.fernan...@umontreal.ca wrote: Hello gents, Once again I would like to convey my most sincere appreciation for your help!!! And yes I would like to hear your elaboration on my DRMAA runner which is what I'm using. So my installation looks like: galaxy -- pbs_drmaa -- torque Cordialement / Regards, Edgar Fernandez -Message d'origine- De : John Chilton [mailto:jmchil...@gmail.com] Envoyé : January-18-15 8:59 PM À : Fernandez Edgar Cc : galaxy-...@bx.psu.edu Objet : Re: [galaxy-dev] galaxy and torque - resource allocation Galaxy generally defers to the DRM (torque in your case) for dealing with these things. In your job_conf.xml you can specify limits for memory or CPUs and Galaxy will pass these along to the DRM at which point it is up to the DRM to enforce these - details depend on if you are using the PBS runner or the DRMAA runner (let me know which and I can try to elobrate if that would be useful). In your particular case - I don't believe torque schedules RAM so things will generally only be... throttled... by CPUs counts. This is what I was told by the admins at MSI when I worked there anyway. If you want to place hard limits on RAM I think you need to upgrade to the MOAB scheduler or switch over to a different DRM entirely like SLURM. Even for DRMs that deal more directly with memory - Galaxy doesn't provide a general mechanism for passing this along to the tool (https://trello.com/c/3RkTDnIn) - so it would be up to the tool to interact with the environment variables your DRM sets . This sounds really terrible in the abstract - but it reality it usually isn't an issue - most of Galaxy's multi-core mappers say have relatively low memory per CPU usage - and for things like assemblers where this is more important - one can usually just assign them to their own CPU or something like that to ensure they get all the memory available. Unlike memory - Galaxy will attempt to pass the number of slots allocated to a job to the tools by setting the GALAXY_SLOTS environment variable. All of the multiple core devteam Galaxy tools at this point use this and so should work - as should probably most multi-core tools from the tool shed - at least the most popular ones. Hope this helps, -John On Tue, Jan 13, 2015 at 11:52 AM, Fernandez Edgar edgar.fernan...@umontreal.ca wrote: Hello gents, Hope everyone had a great holiday break! Wish you guys all the best for 2015! I have a couple of questions about how resources (CPU and memory) is allocated when you have a galaxy and torque installation. So I’ve setup torque with some default and maximum amount of CPU and memory allocations. However, I have some worries when it comes to application (like tophat for example). By default, it takes half the CPU of a server unless specified otherwise. How is the CPU allocation is specified to application like tophat through galaxy? Also, how does galaxy react if a job needs more memory than the limit set by torque? Any information would help me a lot! My sincere salutations to you all!!! Cordialement / Regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication ( Bur. : 1-514-343-6111 poste 16568 Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-218 ___ Please
Re: [galaxy-dev] dataset collections
On Wed, Jan 21, 2015 at 11:02 AM, Jorrit Boekel jorrit.boe...@scilifelab.se wrote: Hi all, I’m toying around a little in galaxy-dist with the dataset collections feature. Since I know this is work in progress, I was wondering about some things I haven’t really found online. It seems to work really well to run a tool on a list of datasets, and a new job is run for each list item. But when I want to reduce to a smaller amount of list items, I understand I need to write some sort of merge tool myself, dependent on the data (all proteomics data here currently). This works well for reducing a dataset to a single file, but I am not sure about how to reduce to a new smaller collection. In the tool I’m writing, I let the user choose the size of the collection. Is there some way to tell galaxy dynamically how many outputs to expect AND put them in a collection? Something like: outputs output type=“data_collection” amount_of_files=“3”/ /outputs Where 3 is set by the user in a param also. Through the January 2015 release - this is not possible. It is now possible in central for tools to explicitly produce collections and this functionality will be in the next release (I think it is still an open question as to whether the team is aiming for February or March 2015 for that). There are a lot of details and examples linked to in the following pull request that I merged last week: https://bitbucket.org/galaxy/galaxy-central/pull-request/634/allow-tools-to-explicitly-produce-dataset There are three style of outputs of increasing complexity - tools that produce static collection (pairs or lists of fixed size), N-N operations (like normalization), and finally fully dynamic collections. Since one can pre-determine the size this would ideally fall under the first type but I had not foreseen this use case so there is currently no syntax like you described so you have to use the third most complicated style. I am going to assume this is some sort of binning operation? Lets imagine - the user selects 3 bins and your tools creates a directory and populates it with mzml files : output/bin1.mzml output/bin2.mzml output/bin3.mzml Then you could create an output collection like this: outputs collection name=binned_output type=list label=Binned Outputs discover_datasets pattern=__name_and_ext__ directory=outputs / /collection /outputs After the job is complete a dataset collection will be populated with three elements of type mzml and element identifiers bin1, bin2, and bin3 (inferred from the name). The syntax of the discover_datasets thing can be quite complex and you can variously hard code properties like datatype, name, etc... or infer them from the name on the file system. Also, when running with two or more lists as input, is there some sort of correlation between the lists? It seems like it takes the files in dataset no order, so just checking. Yes definitely. The UI for creating lists is pretty limited and needs to be updated to look a lot more like the UI for creating lists of paired datasets and this would become a lot more clear I think. So lists are ordered data structures and element identifiers are preserved across executions. So if you start with a list of raw files with identifiers sample1, sample2, and sample3 and map and operation like peak picking over them you would get a new list with the same order and identifiers (sample1, sample2, and sample3 in that order), then if you map and operation like peptide identification on the picked files again you would get a list with identifiers (sample1, sample2, and sample3). Then if you have some sort of summary operation that takes in a raw file and identification and you pass it the original list and the result of the identifications - Galaxy should match everything up and assign the create a resulting list with sample1, sample2, and sample3. (The API lets you do more complicated things like cross-products over subsets of inputs - but this isn't exposed in the GUI yet). If you have two ordered lists and identifiers don't match up - Galaxy's behavior should be considered undefined but it will assign the resulting list the identifiers from one or the other inputs. By the way, thanks very much John and everyone else involved in collections for doing and pushing this stuff. Thanks - and it would be just a playground for me to write API tests against without all the excellent work Carl has put into the UI to make everything actually usable and useful. If there are smaller issues I can help with, I’d be thrilled. The number one thing I encourage everyone to do to help is to build awesome tools and workflows and put them in the tool shed. Can’t stress enough how much this feature means for galaxy adoption in our lab and possibly field. Shhh... don't tell them I am secretly wasting money they would like to put into building a platform for sequencing data analysis to address mass spec use cases - they will
Re: [galaxy-dev] Unable to get data from UCSC
Hi John, Unfortunately swapping out this line for the one in the previous version does not work, but get this error when trying to use ucsc rather than just when transferring back: Blocked loading mixed active content http://genome.ucsc.edu/cgi-bin/hgTables; Also, we are serving galaxy over https, I'm not sure why it is linking back to http (although it would work because it is would be redirected to https). Is there anywhere I can change the address it links back to so it is https? Many thanks Anna On 20 January 2015 at 01:41, John Chilton jmchil...@gmail.com wrote: Ugh - hit send too soon accidentally. I was trying to say - that I believe this was recently changed to allow Galaxy served over HTTPS to access the UCSC in the linked changeset. I was wondering if you could try to just undo that change and see if things work for you again. https://bitbucket.org/galaxy/galaxy-central/commits/579d211c2b0fa8ef4195930cb999edbb9a0cf8eb If that does work - it might work to set that XML block to something like inputs action=//genome.ucsc.edu/cgi-bin/hgTables check_values=false method=get which may correctly tell the web browser to infer HTTP or HTTPS and that would be a change I would be happy to make in -central to fix this problem. Sorry again for all the e-mails. -John P.S. I would definitely serve Galaxy over HTTPS if at all possible - so that might be something you just want to do now to side this problem. On Mon, Jan 19, 2015 at 8:36 PM, John Chilton jmchil...@gmail.com wrote: Hello Anna, Can you try something for me - https://bitbucket.org/galaxy/galaxy-central/commits/579d211c2b0fa8ef4195930cb999edbb9a0cf8eb On Mon, Jan 19, 2015 at 9:12 AM, Anna Terry iseegl...@googlemail.com wrote: Update, Looking at the browser developer tools error logs in firefox I can see the following error: Blocked loading mixed active content http://galaxy.hpc.qmul.ac.uk/tool_runner; similar error in chrome: (index):1 The page at 'https://galaxy.hpc.qmul.ac.uk/' was loaded over HTTPS, but displayed insecure content from 'http://galaxy.hpc.qmul.ac.uk/tool_runner': this content should also be loaded over HTTPS. genome.ucsc.edu/cgi-bin/hgTables:1 The page at 'https://genome.ucsc.edu/cgi-bin/hgTables' was loaded over HTTPS, but is submitting data to an insecure location at 'http://galaxy.hpc.qmul.ac.uk/tool_runner': this content should also be submitted over HTTPS. (index):1 Mixed Content: The page at 'https://galaxy.hpc.qmul.ac.uk/' was loaded over HTTPS, but requested an insecure form action 'http://galaxy.hpc.qmul.ac.uk/tool_runner'. This request has been blocked; the content must be served over HTTPS. Is there any way to change settings so that it points back to https? The server should redirect to https but I guess the browser doesn't know this? TIA Anna On 13 January 2015 at 16:39, Anna Terry iseegl...@googlemail.com wrote: Hi, I am having lots of problems with the get data tools installed on our local galaxy server, I'll start with just one. With the UCSC Main one, I am unable to send data back to galaxy, when I click Send query to Galaxy nothing happens, the button is unresponsive. This is in contrast to when I try the same query from the main galaxy server, on the same browser and the content in correctly transferred back to galaxy. I have no idea how to start debugging this, there is nothing in paster.log Does anyone have any ideas? TIA, Anna ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unable to get data from UCSC
Hi John, we just figured out the problem. Had to follow the instructions at https://wiki.galaxyproject.org/Admin/Config/ApacheProxy: SSL If you place Galaxy behind a proxy address that uses SSL (e.g. https:// URLs), set the following in your Apache config: Toggle line numbers https://wiki.galaxyproject.org/Admin/Config/ApacheProxy# Location /RequestHeader set X-URL-SCHEME https/Location Many thanks for your help Anna On 21 January 2015 at 11:04, Anna Terry iseegl...@googlemail.com wrote: Hi John, Unfortunately swapping out this line for the one in the previous version does not work, but get this error when trying to use ucsc rather than just when transferring back: Blocked loading mixed active content http://genome.ucsc.edu/cgi-bin/hgTables; Also, we are serving galaxy over https, I'm not sure why it is linking back to http (although it would work because it is would be redirected to https). Is there anywhere I can change the address it links back to so it is https? Many thanks Anna On 20 January 2015 at 01:41, John Chilton jmchil...@gmail.com wrote: Ugh - hit send too soon accidentally. I was trying to say - that I believe this was recently changed to allow Galaxy served over HTTPS to access the UCSC in the linked changeset. I was wondering if you could try to just undo that change and see if things work for you again. https://bitbucket.org/galaxy/galaxy-central/commits/579d211c2b0fa8ef4195930cb999edbb9a0cf8eb If that does work - it might work to set that XML block to something like inputs action=//genome.ucsc.edu/cgi-bin/hgTables check_values=false method=get which may correctly tell the web browser to infer HTTP or HTTPS and that would be a change I would be happy to make in -central to fix this problem. Sorry again for all the e-mails. -John P.S. I would definitely serve Galaxy over HTTPS if at all possible - so that might be something you just want to do now to side this problem. On Mon, Jan 19, 2015 at 8:36 PM, John Chilton jmchil...@gmail.com wrote: Hello Anna, Can you try something for me - https://bitbucket.org/galaxy/galaxy-central/commits/579d211c2b0fa8ef4195930cb999edbb9a0cf8eb On Mon, Jan 19, 2015 at 9:12 AM, Anna Terry iseegl...@googlemail.com wrote: Update, Looking at the browser developer tools error logs in firefox I can see the following error: Blocked loading mixed active content http://galaxy.hpc.qmul.ac.uk/tool_runner; similar error in chrome: (index):1 The page at 'https://galaxy.hpc.qmul.ac.uk/' was loaded over HTTPS, but displayed insecure content from 'http://galaxy.hpc.qmul.ac.uk/tool_runner': this content should also be loaded over HTTPS. genome.ucsc.edu/cgi-bin/hgTables:1 The page at 'https://genome.ucsc.edu/cgi-bin/hgTables' was loaded over HTTPS, but is submitting data to an insecure location at 'http://galaxy.hpc.qmul.ac.uk/tool_runner': this content should also be submitted over HTTPS. (index):1 Mixed Content: The page at 'https://galaxy.hpc.qmul.ac.uk/' was loaded over HTTPS, but requested an insecure form action 'http://galaxy.hpc.qmul.ac.uk/tool_runner'. This request has been blocked; the content must be served over HTTPS. Is there any way to change settings so that it points back to https? The server should redirect to https but I guess the browser doesn't know this? TIA Anna On 13 January 2015 at 16:39, Anna Terry iseegl...@googlemail.com wrote: Hi, I am having lots of problems with the get data tools installed on our local galaxy server, I'll start with just one. With the UCSC Main one, I am unable to send data back to galaxy, when I click Send query to Galaxy nothing happens, the button is unresponsive. This is in contrast to when I try the same query from the main galaxy server, on the same browser and the content in correctly transferred back to galaxy. I have no idea how to start debugging this, there is nothing in paster.log Does anyone have any ideas? TIA, Anna ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Oct '14 and Jan'15 release, trouble installing tools from the toolshed, resetting metadata for installed repositories
Just an update on that.
I found out that if I put back the migrated_tool_conf.xml and
shed_tool_conf.xml back in the galaxy_dist root, I can now reset metadata
on most of the repositories (but not all)
Same error as before pops up in paster.log for those not working.
Cheers!
Yves Gagnon
From: Yves Gagnon yves.gag...@dnalandmarks.ca
To: galaxy-dev@lists.galaxyproject.org
Date: 20/01/2015 03:26 PM
Subject:[galaxy-dev] Oct '14 and Jan'15 release, trouble
installing tools from the toolshed, resetting metadata for installed
repositories
Sent by:galaxy-dev galaxy-dev-boun...@lists.galaxyproject.org
Hello fellow galaxy users :
I noticed that trying to install some tools from the main toolshed,
sometimes it works, others it does not. In a specific case, trying to
install clustalw and its dependency, I get this error message :
URL: (redactedURL)/admin_toolshed/prepare_for_install
File '(redacted
path)/galaxy_dist_dev/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py',
line 364 in respond
app_iter = self.application(environ, detect_start_response)
File '(redacted
path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py',
line 84 in __call__
return self.application(environ, start_response)
File '(redacted
path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
line 633 in __call__
return self.application(environ, start_response)
File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py',
line 132 in __call__
return self.handle_request( environ, start_response )
File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py',
line 190 in handle_request
body = method( trans, **kwargs )
File '(redacted
path)/galaxy_dist_dev/lib/galaxy/web/framework/decorators.py', line 87 in
decorator
return func( self, trans, *args, **kwargs )
File '(redacted
path)/galaxy_dist_dev/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py',
line 1095 in prepare_for_install
install_repository_manager.handle_tool_shed_repositories(
installation_dict, using_api=False )
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/install_manager.py',
line 636 in handle_tool_shed_repositories
new_tool_panel_section_label=new_tool_panel_section_label )
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py',
line 200 in create_repository_dependency_objects
self.app.installed_repository_manager.activate_repository(
repository_db_record )
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/installed_repository_manager.py',
line 90 in activate_repository
repository_tools_tups = irmm.get_repository_tools_tups()
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py',
line 69 in get_repository_tools_tups
index, shed_conf_dict = self.tpm.get_shed_tool_conf_dict(
self.metadata_dict.get( 'shed_config_filename' ) )
TypeError: 'NoneType' object is not iterable
Same goes for resetting all metadata on installed repositories after
update, 23 out of ~200 tools work, for the rest I get this error in
paster.log :
tool_shed.galaxy_install.metadata.installed_repository_metadata_manager
ERROR 2015-01-20 15:18:35,357 Error attempting to reset metadata on
repository tabular_to_fasta
Traceback (most recent call last):
File (redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py,
line 114, in reset_metadata_on_selected_repositories
self.set_repository( repository )
File (redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py,
line 142, in set_repository
super( InstalledRepositoryMetadataManager, self ).set_repository(
repository )
File (redacted
path)/galaxy_dist_dev/lib/tool_shed/metadata/metadata_generator.py, line
1066, in set_repository
self.metadata_dict = { 'shed_config_filename' :
self.shed_config_dict.get( 'config_filename', None ) }
AttributeError: 'NoneType' object has no attribute 'get'
My galaxy contains this line for tool_conf files, and I moved all config
files from the distrib root to the config folder:
tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml
Did I do something wrong while updating to the latest releases???
Thanks for your help!
Yves Gagnon___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your
Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py
So the tool dependency stuff didn't change - but the local job runner now behaves like the other job runners (DRMAA, PBS, etc...) so you have uncovered a bug for all of them I think. Is this a tool shed tool or a locally created dependency? I need to think about how to solve this more generally but if you want a quick work around - I think it would work to just replace python in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's python or perhaps just python2. Sorry for the inconvenience and thanks for bringing this to our attention. -John On Tue, Jan 20, 2015 at 12:52 PM, John Chilton jmchil...@gmail.com wrote: Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)? -John On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de wrote: On 01/20/2015 06:20 PM, Wolfgang Maier wrote: I have not seen this error before (I believe not with latest_2014.10.06) update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again. The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately. Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
