Re: [galaxy-dev] Bug report: within
Philipp, Pages are seriously deprecated -- do not use them. Many things will not work, they are not supported in workflows, the API, et cetera. They only still exist for backward compatibility with ANCIENT tools. Thanks, James -- jt On Tue, Aug 12, 2014 at 4:59 AM, Hans-Philipp Brachvogel wrote: > Hello, > > I found that it is possible for tool-xmls defined with tags to also > have multipage helps, but this feature is not really documented and also does > not work because of a tiny bug. You can have multipage helps by writing > something like this (if your tool has two s): > > > > Helptext for page 1 > > > > Helptext for page 2 > > > > > But for it to work one has to fix the following: > rev 13771:7a4d321c0e38 > file lib/galaxy/tools/__init__.py > expression starting at line 1439: > > self.help_by_page = [ Template( rst_to_html( help_header + x + > help_footer, > input_encoding='utf-8', > output_encoding='utf-8', > default_filters=[ 'decode.utf8' ], > encoding_errors='replace' ) ) > for x in self.help_by_page ] > > The bracket is misplaced and the whole thing should instead be: > > self.help_by_page = [ Template( rst_to_html( help_header + x + > help_footer), > input_encoding='utf-8', > output_encoding='utf-8', > default_filters=[ 'decode.utf8' ], > encoding_errors='replace' ) > for x in self.help_by_page ] > > One question: is deprecated, does this mean there will be no multipage > tools in the future? Or will there be (or is already?) an alternative system? > > Best, > Philipp > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Citations for tools
Eric, I'm very much in favor of this feature, and particularly the
idea of generating a list of citations from a history or workflow. I
imagine the only thing to quibble about will be the syntax. There are
already some efforts to represent bibtex in xml (e.g.
https://github.com/Zearin/BibTeXML), however they have always struck
me as overly verbose.
-- jt
On Tue, May 27, 2014 at 12:45 PM, Eric Rasche wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> I'd want to open up discussion on a feature I'd like to see. I'll try
> and implement it if I can find time this summer. I didn't see a trello
> card for anything like this yet, but please feel free to direct me there
> if I missed it.
>
>
> I'd like to see citations as a part of every tool.
>
> This would happen in the form of a block in the XML, which
> would contain sub-elements with text. These sub-elements could be based
> off of BibTeX, since they have existing specs for citing things:
> https://en.wikipedia.org/wiki/BibTeX
>
> These citations would then be accessible in the HTML generated tool
> pages, or via a View/Download button somewhere on the tool page. By
> storing as an XML tree, we could render these citations as BibTeX
> entries for the LaTeX users, and I believe there are ways to convert
> BibTeX to EndNote XML and so on.
>
> This could be extended for use in workflows so that when you run
> workflows, somehow a list of citations for all tools used could be
> generated.
>
> Anyone have thoughts or opinions on this?
>
>
>
>
> Using the example bibtex entry from the wikipedia page:
>
> @Book{abramowitz+stegun,
> author= "Milton {Abramowitz} and Irene A. {Stegun}",
> title = "Handbook of Mathematical Functions with
> Formulas, Graphs, and Mathematical Tables",
> publisher = "Dover",
> year = 1964,
> address = "New York",
> edition = "ninth Dover printing, tenth GPO printing"
> }
>
> I imagine it'd look like the following in a real-life tool:
>
>
> ...
>
> Milton Abramowitz and Irene A. Stegun"
> Handbook of Mathematical Functions with Formulas, Graphs, and
> Mathematical Tables
> Dover
> 1964
> New York
> ninth Dover printing, tenth GPO printing
>
>
>
>
> Cheers,
> Eric
> - --
> Eric Rasche
> Programmer II
> Center for Phage Technology
> Texas A&M University
> College Station, TX 77843
> 404-692-2048
> e...@tamu.edu
> rasche.e...@yandex.ru
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.17 (GNU/Linux)
>
> iQIcBAEBAgAGBQJThMEUAAoJEMqDXdrsMcpVe9AP/1BjPbP5JdK6KDybOeV2ElvC
> jvmUGetAjdQzkKO1Ikeuxb46yp0j4abAGGG92AccxlBYALsT3jsPv5dYjm505vcU
> IwlfTBE7gc5y19x3zx1CuAd7PB11tryODz2LwXNKI75f39bNdX6Qe5aA74Vn/k8a
> FBeFL/EvkZwI8Z6CvuTGtZrEHvDXVvTFoxMX875f59g2vNy7W8Fh5wQCQ8qgiogi
> 0SWGIGZ6OJdqX60h2hOjJ06NhzBcTsjQea+AwTo5wAkyGPFJy4DV78jKOvsd5PMG
> qkk4U6gBcmcFolCeItlLPa/C4uiyOK2wVWHyTnEeKjdpD6dwLuemRmwtZrModbQu
> PPPFkKzQhUnh5Wgq8TIP4kSc2t1/jSpdV8MHqM0piqOFnR52fExPrwLn82WEFcp8
> tXfHHTaZj+6gL+IGswdAgqvxk1V+6QTjH57igWmYtcENpIDbhZVBh+HOD8SS2qdv
> ohwFg/Vn8VrCeTGGehNSBvkam0HXjXXq7mZ8W4xrOS8vxv8HPAFZHze2sPlUG/Ob
> WeOdiLgD2HCH95SXYDDBNFG6HwFkLETA3DjpfvgHqPdaLmEfIZW+ks1WF9RG+Pmo
> jhbWH8OmtTk5A/Zh63uRowMaDe7QSCruPIfIwlrgqLozOj8htMyJPBIW9oZGGkqC
> 7Vu5UFeL8gcigRmQv5NA
> =ceAo
> -END PGP SIGNATURE-
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Citing Galaxy + Toolshed in an app note of a small tool
Hey Assaf, For Cite1 (Galaxy): doi:10.1186/gb-2010-11-8-r86 For Cite2 (ToolShed): doi:10.1186/gb4161 Thanks for asking! -- jt On Tue, Mar 25, 2014 at 1:15 PM, Assaf Gordon wrote: > Hello Galaxy People! > > (it's been a while since I've last been here... a pleasure to be back). > > I intend to publish a small command-line utility, which will also be > available through Galaxy Toolshed. > It'll be a small application note, so not a lot of space for many citations. > > The relevant sentence would read something like: > "The tool is also available for the Galaxy Bioinformatics Platform [Cite1], > with automatic installation provided though the Galaxy Tool Shed [Cite2]". > > What should I use for [cite1 (galaxy) and [cite2 (toolshed)], out of this > impressive long list of publications: > https://wiki.galaxyproject.org/CitingGalaxy > > > Thanks! > -gordon > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ProFFTPd-1.3.4d no longer able to authenticate users from Postgresql database
> 'PBKDF2$sha256$1$1xx',got Œyy¹
> I am not using either PBKDF2 or sha256 see below why are these prefixes
> appended? Xxx and are NOT the same alphanumeric strings
Are you sure? Unless you speficically disable it Galaxy now uses
PBKDF2 + SHA256 by default.
I'm not sure if anybody has managed to get it working with proftp
(though the support is there and it should be possible). The easy
solution is to disable using PBKDF2 in universe_wsgi.ini.
On Wed, Jan 8, 2014 at 4:56 PM, Hazard, E. Starr wrote:
> I was able to query my Postgresql Db for user authentication at one time.
> I deleted and then rebuilt the postgresql DB and rebuilt the entire Galaxy
> instance in early December. Since then I cannot get ProFFTPd-1.3.4d to
> authenticate users from Postgresql database. At that time my existing
> universe_wsgi.ini
>
> I changed the following from postgres to postgresql
>
>
> #database_connection =
> postgres://galaxy:Galaxy2013@localhost:5434/galaxy_prod
> database_connection =
> postgresql://galaxy:galaxy2013@localhost:5434/galaxy_prod
>
> What ftp interaction behavior changes were included in this
> database_connectio method renaming? (And how can I fix them?)
>
>
>
> My Galaxy users can login to Galaxy just fine and execute programs if I
> manually load files into the designated FTP dir and then move these to
> database/files .
>
> Screen session from interactive test mode
> FTP session opened.
> dispatching PRE_CMD command 'USER [valid email]' to mod_core
> .
> .
> .
> dispatching CMD command 'PASS (hidden)' to mod_auth
> no supplemental groups found for user '[valid email]'
> ROOT PRIVS at mod_auth_pam.c:338
> RELINQUISH PRIVS at mod_auth_pam.c:508
> mod_sql_passwd/0.4: expected
> 'PBKDF2$sha256$1$1xx',got Œyy¹
>
> I am not using either PBKDF2 or sha256 see below why are these prefixes
> appended? Xxx and are NOT the same alphanumeric strings
>
>
> SQLLogFile.txt excerpt:
>
>
> Jan 06 17:27:49 mod_sql/4.3[29179]: query "SELECT
> email,password,' [valid email]','{UNKNOWN
> TAG}',¹/PATH/galaxy-dist/database/files/
> [vali
> d email]','/bin/
> bash' FROM galaxy_user WHERE email=' [valid
> email]'"
> Jan 06 17:27:49 mod_sql/4.3[29179]: enteringpostgres cmd_close
> Jan 06 17:27:49 mod_sql/4.3[29179]: connection 'default' count is now 1
> Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_close
> Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_select
> Jan 06 17:27:49 mod_sql/4.3[29179]: <<< process_named_query
> 'LookupGalaxyUser'
> Jan 06 17:27:49 mod_sql/4.3[29179]: <<< sql_lookup
> Jan 06 17:27:49 mod_sql/4.3[29179]: user UID 0 below SQLMinUserUID 999,
> using SQLDefaultUID 65533
> Jan 06 17:27:49 mod_sql/4.3[29179]: user GID 0 below SQLMinUserGID 999,
> using SQLDefaultGID 65533
> Jan 06 17:27:49 mod_sql/4.3[29179]: cache miss for user '
> [valid email]'
> Jan 06 17:27:49 mod_sql/4.3[29179]: user ' [valid
> email]' cached
> Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_name : [valid email]
> Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_uid : 65533
> Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_gid : 65533
> Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_dir :
> /PATH/galaxy-dist/database/files/
> [valid
> email]
> Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_shell : /bin/bash
> Jan 06 17:27:49 mod_sql/4.3[29179]: <<< cmd_getpwnam
> Jan 06 17:27:49 mod_sql/4.3[29179]: >>> cmd_auth
> Jan 06 17:27:49 mod_sql/4.3[29179]: enteringpostgres cmd_escapestring
> Jan 06 17:27:49 mod_sql/4.3[29179]: enteringpostgres cmd_open
> Jan 06 17:27:49 mod_sql/4.3[29179]: connection 'default' count is now 2
> Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_open
> Jan 06 17:27:49 mod_sql/4.3[29179]: enteringpostgres cmd_close
> Jan 06 17:27:49 mod_sql/4.3[29179]: connection 'default' count is now 1
> Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_close
> Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_escapestring
> Jan 06 17:27:49 mod_sql/4.3[29179]: cache hit for user '[valid email]'
> Jan 06 17:27:49 mod_sql/4.3[29179]: >>> cmd_check
> Jan 06 17:27:49 mod_sql/4.3[29179]: checking password using SQLAuthType
> 'sha1'
> Jan 06 17:27:49 mod_sql/4.3[29179]: 'sha1' SQLAuthType handler reports
> failure
>
> ProFTPd conf file excerpt:
>
> (based on https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP)
>
> # Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1
> SQLPasswordEngine on
> SQLPasswordEncoding hex
>
> # Set up SQLLogfile
> SQLLogFile /PATH/ProFFTPd-1.3.4d/etc/SQLLogFile.txt
>
> # Set up mod_sql to authenticate against the Galaxy database
> SQLEngine on
> SQLBackend postgres
> SQLConnectInfo galaxy_prod@:5xxx galaxy galpwd # port
> and password are changed here; they a
Re: [galaxy-dev] Is there a way yet to more easily determine name of inputs for renaming in workflows
Would something as simple as showing the input parameter name in a tooltip
inside the workflow editor accomplish what you need?
--
James Taylor
On Fri, Dec 27, 2013 at 4:24 PM, Thon deBoer wrote:
> Hi,
>
> In order to make Galaxy workflows even more useful, it is going to be
> important for data files to be renamed in a workflow.
> I know we can use the "#{input1 | base name}-BWA.sam" trick which is
> useful IF you know what the actual parameter name is in the XML file.
>
> There was some discussions some years ago (
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-September/011288.html)
> but it is is now very time consuming to figure out from the XML file what
> the name of the input parameter is you would like to use (and it is often
> very tricky, due to the fact that the name of the actual input parameter
> may actually change based on some selection in the UI….
>
> I often had to resort to actually chaining the XML file to make the
> output something like:
>
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/
>
<>___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tiff files in galaxy
VisiGene only sends gif and jpeg over the wire. -- James Taylor, Associate Professor, Biology/CS, Emory University On Tue, Dec 3, 2013 at 12:59 PM, Robert Baertsch wrote: > I was planning to use server code to tile the large images and then use one > of a number of JavaScript libraries to render. Any browser should work > without extensions . Jim did this with visigene in the genome browser years > ago. > - Robert > > On Dec 3, 2013, at 9:16 AM, Carl Eberhard wrote: > > Hi, Robert > > As far as I know, all browsers besides Safari aren't set up to render tiff > files by default. Are you using Safari to display the files in Galaxy? > > There are extensions for Firefox and Chrome that will allow you to display > these files. Would that be an option? > > As a side note: creating, saving, and manipulating images and thumbnails > would require Galaxy including a server side image processing library as a > dependency. I'd like to do this at some point in the future as it would > allow us other advantages too. > > Thanks, > Carl > > > On Thu, Nov 21, 2013 at 4:17 PM, Robert Baertsch > wrote: >> >> Do you know a fast way to open large tiff files in galaxy ? The sniffer >> recognizes the type, but clicking on the eyeball on a large 82mb file just >> hangs. >> Downloading the file and opening in preview does work but it is clunky. >> Perhaps storing a low res version that would be display when clicking the >> eyeball would speed access. >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Shed packages for BLAST+ binaries
This is not an objection, but do we need bash? Can we live with posix sh? We should ask this question about every requirement we introduce. (bash is not part of the default installation of FreeBSD or OpenBSD for example. bash is unfortunately licensed under GPLV3, so if you are trying to create an OS not polluted by viral licensing you don't get bash). On Mon, Oct 7, 2013 at 11:36 AM, John Chilton wrote: > My own preference is that we specify at least /bin/sh and /bin/bash > are available before utilizing the tool shed. Is there an objection to > this from any corner? Is there realistically a system that Galaxy > should support that will not have /bin/bash available? -- James Taylor, Associate Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Using input dataset names in output dataset names
I have not tested the patch, just read it, but won't this result in dataset names like: "Some operation on data 27 (Some operation on data 26 (Some other operation on data 25 (...(...(..." (avoiding this is why we came up with HIDs in the first place). -- James Taylor, Associate Professor, Biology/CS, Emory University On Mon, Nov 11, 2013 at 5:42 AM, Peter Cock wrote: > On Thu, Nov 7, 2013 at 3:50 PM, Peter Cock wrote: >> >> Getting back to my motivating example, since fasta_to_tabular.xml >> does not give the output a label and depends on the default, the >> small change to $on_string should result in the conversion of a >> file named "My Genes" as "FASTA-to-Tabular on My Genes", >> rather than "FASTA-to-Tabular on data 1" as now. >> >> Here's another variant to keep the "data 1" text in $on_string, >> if people are attached to this functionality. That would result in >> "FASTA-to-Tabular on data 1 (My Genes)". >> >> ... >> >> -- >> >> $ hg diff lib/galaxy/tools/actions/__init__.py >> diff -r 77d58fdd1c2e lib/galaxy/tools/actions/__init__.py >> --- a/lib/galaxy/tools/actions/__init__.pyTue Oct 29 14:21:48 2013 -0400 >> +++ b/lib/galaxy/tools/actions/__init__.pyThu Nov 07 15:49:15 2013 + >> @@ -181,6 +181,7 @@ >> input_names = [] >> input_ext = 'data' >> input_dbkey = incoming.get( "dbkey", "?" ) >> +on_text = '' >> for name, data in inp_data.items(): >> if not data: >> data = NoneDataset( datatypes_registry = >> trans.app.datatypes_registry ) >> @@ -194,6 +195,8 @@ >> else: # HDA >> if data.hid: >> input_names.append( 'data %s' % data.hid ) >> +#Will use this on_text if only one input dataset: >> +on_text = "data %s (%s)" % (data.id, data.name) >> input_ext = data.ext >> >> if data.dbkey not in [None, '?']: >> @@ -230,7 +233,10 @@ >> output_permissions = >> trans.app.security_agent.history_get_default_permissions( history ) >> # Build name for output datasets based on tool name and input names >> if len( input_names ) == 1: >> -on_text = input_names[0] >> +#We recorded the dataset name as on_text earlier... >> +if not on_text: >> +#Fall back on the shorter 'data %i' style: >> +on_text = input_names[0] >> elif len( input_names ) == 2: >> on_text = '%s and %s' % tuple(input_names[0:2]) >> elif len( input_names ) == 3: >> > > Would this patch be welcomed as a pull request? (Expanding > $on_string to include the name as well as dataset number > when there is only one input dataset) > > How about renaming the outputs of the conversion tools? > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers
> So far $GALAXY_SLOTS seems to be working nicely for me. > > However, I am wondering if it would be possible to use it inside > the section? Is that run at the time of job creation > on the Galaxy server (where determining the number of threads > may be hard) or as part of job execution (e.g. on the cluster, > when $GALAXY_SLOTS would be known). Unfortunately, the former, and there is no easy way to change this. One could do some simple variable substitution on the config file when the job runs though (sed would do the job here, not pretty but it works). ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Streamlined Uploading
As part of this card: https://trello.com/c/TnSlPz1j/282-55-improvements-to-file-upload Sam has been completely redoing the UI for upload. Particularly with the focus on multiple file upload and progress displays. However, it will be a dedicated UI for upload not using standard tool forms, so I think what you want can be accommodated well. Hopefully he can elaborate (maybe that card should have some screenshots of the working version?). > 3) We then create several news tools, which would appear before > "Advanced Upload Tool" in the "Get Data" section: > > "Upload file from your computer" > "Import files from the web via URLs" > "Import FTP uploaded files" > "Paste file contents" I think each of these could be tabs under a single upload tool. > 4) To further streamline things, we could replace "Your history is > empty. Click 'Get Data' on the left pane to start" in the empty > history section with something with links: "Your history is empty. To > get started, import data from [your computer], [web links], [your > clipboard], [FTP], or by choosing an external data source from the > 'Get Data' section in the left panel". Hmmm... calling it "Upload or import data" might be clarifying. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] worflow editor is slow with a large amount of steps
On Mon, Oct 21, 2013 at 4:08 AM, Bjoern Gruening wrote: > one of our users has created a large workflow with > 300 steps. Yes he > really loves Galaxy :) > Unfortunately, its not possible to use the graphical user interface > anymore with so many steps. You can't connect boxes and scrolling is > also slow. Can you do any javascript profiling to see where the time is being spent. If I had to guess, I would say the inability to connect is due to the search through all possible inputs to highlight the connectable ones in green. You could just disable that if the workflow has more than say 50 steps, and switch to just checking on hover. (but that's just a guess) The workflow editor doesn't use any library beyond jquery, all the relevant code to this problem should be in: static/scripts/galaxy.workflow_editor.canvas.js -- James Taylor, Associate Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [galaxy-user] Need your helps about Galaxy
They are already running a local instance. I didn't realize that bowtie required a different index for colorspace alignment. So Lei, you will have to build the index using bowtie-build -C. -- James Taylor, Associate Professor, Biology/CS, Emory University On Wed, Oct 16, 2013 at 3:31 PM, Jennifer Jackson wrote: > Hello Lei, > > If genomes are not listed, that means that they are not indexed for use with > the tool. The test server is primarily for demonstration or test use > besides, and there could be other unexpected issues even if genomes are > listed (we really do test here). Also, the quotas are very small (10G). If > you want to use this tool, a local, cloud, or slipstream Galaxy is > recommended. Full choices with details are listed here: > http://wiki.galaxyproject.org/BigPicture/Choices > http://usegalaxy.org/toolshed > > Help for setup is here, with the galaxy-...@bx.psu.edu mailing list > available for further support. Tools will need to be installed, and indexes > created. You can rsync the genome, but most genomes will not have loc file > entries and indexes for SOLiD already created - see the Tophat manual for > the command to create these: > http://wiki.galaxyproject.org/Tool%20Shed#Installing.2C_maintaining_and_uninstalling_tool_shed_repositories_within_a_Galaxy_instance > http://wiki.galaxyproject.org/Admin/Data%20Integration > http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup > > Hopefully this helps! > > Jen > Galaxy team > > > On 10/16/13 11:57 AM, Lei Yan wrote: >> >> Hi all, >> >> We are trying to use “Tophat for SOLiD”. >> But this tool (Tophat for SOLiD) does not seem to be linked to the >> reference genomes that are installed. I can see those genomes on the Tophat >> for illumina tool and the other tools that require a reference genome. >> Please see attachments. >> Does anybody have any ideas for this? Thanks a lot. >> >> >> Lei Yan >> Center for Integrative and Translational Genomics >> The University of Tennessee Health Science Center > > > -- > Jennifer Hillman-Jackson > http://galaxyproject.org > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error when running paster directly
The paster script distributed with Galaxy is a stripped down Galaxy specific version. It only supports the command serve and the arguments --daemon or --reload. -- James Taylor, Associate Professor, Biology/CS, Emory University On Fri, Oct 11, 2013 at 8:36 AM, Renato Alves wrote: > Hi everyone, > > When trying to adapt the init script for debian machines (provided in > contrib/) I started playing with paster directly. I was trying to get > some sort of help or list of commands/params that can be passed but > instead I was presented with the following output: > > $ python ./scripts/paster.py > > Usage: ./scripts/paster.py COMMAND > Command 'help' not known (you may need to run setup.py egg_info) > Traceback (most recent call last): > File "./scripts/paster.py", line 33, in > serve.run() > File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py", > line 1049, in run > invoke(command, command_name, options, args[1:]) > File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py", > line 1055, in invoke > exit_code = runner.run(args) > File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py", > line 367, in run > commands = get_commands().items() > NameError: global name 'get_commands' is not defined > > After inspection of the file, the get_commands function doesn't seem to > be defined nor imported. > > I'm not sure how to proceed at this point. > > Thanks, > Renato > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Migrating database from development back to stable?
On Tue, Oct 8, 2013 at 4:32 PM, Guest, Simon wrote: > What I am wondering is whether it is guaranteed that I will be able to get > back onto stable at a later date, i.e. there will be a database migration > path from whatever development database version I end up on, onto a future > stable database version. Yes, this should not be a problem, eventually the revision you are running will be an ancestor of stable and from there you can just "upgrade" to the new stable. -- James Taylor, Associate Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
Make the precedence a config option. Otherwise I agree. In addition, I still like the idea I suggested earlier of dependency provider plugins. Then you could (for example) have one that uses 'modules' and skips env.sh entirely. On Sep 26, 2013, at 9:15 PM, John Chilton wrote: > I was not even thinking we needed to modify the tool shed to implement > this. I was hoping (?) you could just modify: > > lib/galaxy/tools/deps/__init__.py > > to implement this. If some tool contains the tag > > numpy > > then if there is a manually installed tool_dependency in > `tool_dependency_dir/numpy/1.7.1/env.sh` that would take precedence > over the tool shed installed version (would that be something like > `tool_dependency_dir/numpy/1.7.1/owner/name/changeset/env.sh`)? Let me > know if this is way off base. > > There is a lot you could do to make this more complicated of course - > an interface for mapping exact tool shed dependencies to manually > installed ones, the ability to auto-compile tool shed dependencies > against manually installed libraries, etc..., but I am not sure those > complexities are buying you anything really. > > Thoughts? > > -John > > > > On Thu, Sep 26, 2013 at 5:47 PM, Greg Von Kuster wrote: >> Hi John, >> >> On Sep 26, 2013, at 5:27 PM, John Chilton wrote: >> >>> My recommendation would be make the tool dependency install work on as >>> many platforms as you can and not try to optimize in such a way that >>> it is not going to work - i.e. favor reproduciblity over performance. >>> If a system administrator or institution want to sacrifice >>> reproduciblity and optimize specific packages they should be able to >>> do so manually. Its not just Atlas and CPU throttling right? Its >>> vendor versions of MPI, GPGPU variants of code, variants of OpenMP, >>> etc Even if the tool shed provided some mechanism for determining >>> if some particular package optimization is going to work, perhaps its >>> better to just not enable it by default because frequently these cause >>> slightly different results than the unoptimized version. >>> >>> The problem with this recommendation that is Galaxy currently provides >>> no mechanism for doing so. Luckily this is easy to solve and the >>> solution solves other problems. If the tool dependency resolution code >>> would grab the manually configured dependency instead of the tool shed >>> variant when available, instead of favoring the opposite, then it >>> would be really easy to add in an optimized version of numpy or an MPI >>> version of software X. >> >> How would you like this to happen? Would it work to provide an admin the >> ability to create a ToolDependency object and point it to a "manually >> configured dependency" in whatever location on disk the admin chooses via a >> new UI feature? Or do you have a different idea? >> >> Thanks, >> >> Greg Von Kuster >> >> >>> >>> Whats great is this solves other problems as well. For instance, our >>> genomics Galaxy web server runs Debian but the worker nodes run >>> CentOS. This means many tool shed installed dependencies do not work. >>> JJ being the patient guy he is goes in and manually updates the tool >>> shed installed env.sh files to load modules. Even if you think not >>> running the same version of the OS on your server and worker nodes is >>> a bit crazy, there is the much more reasonable (common) case of just >>> wanting to submit to multiple different clusters. When I was talking >>> with the guys at NCGAS they were unsure how to do this, this one >>> change would make that a lot more tenable. >>> >>> -John >>> >>> On Thu, Sep 26, 2013 at 1:29 PM, Björn Grüning >>> wrote: Hi, > Hi Bjoern, > > Is there anything else we (the Galaxy community) can do to help > sort out the ATLAS installation problems? Thanks for asking. I have indeed a few things I would like some comments. > Another choice might be to use OpenBLAS instead of ATLAS, e.g. > http://stackoverflow.com/questions/11443302/compiling-numpy-with-openblas-integration I have no experience with it. Does it also need to turn off CPU throttling? I would assume so, or how is it optimizing itself? > However, I think we build NumPy without using ATLAS or any > BLAS library. That seems like the most pragmatic solution > in the short term - which I think is what Dan tried here: > http://testtoolshed.g2.bx.psu.edu/view/blankenberg/package_numpy_1_7 I can remove them if that is the consensus. A few points: - fixing the atlas issue can speed up numpy, scipy, R considerably (by 400% in some cases) - as far as I understand that performance gain is due to optimizing itself on specific hardware, for atlas there is no way around to disable CPU throttling (How about OpenBlas?) - it seems to be complicated to deactivate CPU throttling on OS-X - binary installation does not m
Re: [galaxy-dev] ImportError: No module named paramiko
This should happen automatically at startup though, indicating a bug, -- James Taylor, Associate Professor, Biology/CS, Emory University On Tue, Sep 24, 2013 at 12:08 PM, Dave Bouvier wrote: > Peter, > > We recently upgraded the Fabric egg, which now depends on paramiko. You > should be able to resolve this issue by running > > python ./scripts/fetch_eggs.py > >--Dave B. > > > On 09/24/2013 11:42 AM, Peter Cock wrote: >> >> Hi all, >> >> Has something changed recently on galaxy-central with the fabric egg, >> and a possible missing dependency? >> >> $ ./run.sh >> ... >> galaxy.jobs.handler INFO 2013-09-24 16:40:21,849 job handler stop queue >> started >> galaxy.jobs.handler INFO 2013-09-24 16:40:21,867 job handler queue started >> galaxy.sample_tracking.external_service_types DEBUG 2013-09-24 >> 16:40:21,870 Loaded external_service_type: Simple unknown sequencer >> 1.0.0 >> galaxy.sample_tracking.external_service_types DEBUG 2013-09-24 >> 16:40:21,873 Loaded external_service_type: Applied Biosystems SOLiD >> 1.0.0 >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,899 Enabling >> 'data_admin' controller, class: DataAdmin >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,903 Enabling >> 'requests_common' controller, class: RequestsCommon >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,905 Enabling >> 'forms' controller, class: Forms >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,974 Enabling >> 'workflow' controller, class: WorkflowController >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,975 Enabling >> 'async' controller, class: ASync >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,975 Enabling >> 'library' controller, class: Library >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,982 Enabling >> 'data_manager' controller, class: DataManager >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:21,983 Enabling >> 'error' controller, class: Error >> galaxy.web.framework.base DEBUG 2013-09-24 16:40:22,213 Enabling >> 'cloudlaunch' controller, class: CloudController >> galaxy.eggs WARNING 2013-09-24 16:40:22,480 Warning: paramiko (a >> dependent egg of Fabric) cannot be fetched >> Traceback (most recent call last): >>File >> "/mnt/galaxy/galaxy-central/eggs/Fabric-1.7.0-py2.6.egg/fabric/network.py", >> line 24, in >> import paramiko as ssh >> ImportError: No module named paramiko >> >> There was a problem importing our SSH library (see traceback above). >> Please make sure all dependencies are installed and importable >> ... >> >> Also what is this CloudController and can I turn it off instead? >> >> Thanks, >> >> Peter >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >>http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >>http://galaxyproject.org/search/mailinglists/ >> > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] workflow issues in local Galaxy....
Sqlite is absolutely not intended for production use. With multiple users or complex components you will definitely see database locked. Use Postgres. On Sep 16, 2013, at 9:48 PM, Nikhil Joshi wrote: > I am not using CloudMan, but we do use a sqlite database. It seems to have > that behavior when there is an output that connects to two inputs... but I > think I need to do more investigating. I suppose we could try postgres and > see if that works... > > - Nik. > > > On Mon, Sep 16, 2013 at 2:04 PM, Björn Grüning wrote: >> Hi Nikhil, >> >> do you use the CloudMan?http://wiki.galaxyproject.org/CloudMan >> >> I have such a behaviour when I used the sqlite database and not >> postgresql. >> >> Cheers, >> Bjoern >> >> > Hi all, >> > >> > >> > So we use Galaxy to teach our bioinformatics courses and we have an >> > install of Galaxy in the Amazon cloud that we have customized with >> > various tools. However, we don't touch the Galaxy code base itself. >> > Recently, we've been getting errors in creating and running workflows. >> > We are getting "locked database" errors, sometimes the steps seem to >> > run out of order, and sometimes the entire workflow just hangs. Also, >> > the workflow editor seems to be having issues, like it allows >> > connections between outputs and inputs of entirely different >> > datatypes. Has anyone seen any issues like this? >> > >> > >> > - Nik. >> > >> > -- >> > Nikhil Joshi >> > Bioinformatics Analyst/Programmer >> > UC Davis Bioinformatics Core >> > http://bioinformatics.ucdavis.edu/ >> > najoshi -at- ucdavis -dot- edu >> > 530.752.2698 (w) >> > ___ >> > Please keep all replies on the list by using "reply all" >> > in your mail client. To manage your subscriptions to this >> > and other Galaxy lists, please use the interface at: >> > http://lists.bx.psu.edu/ >> > >> > To search Galaxy mailing lists use the unified search at: >> > http://galaxyproject.org/search/mailinglists/ > > > > -- > Nikhil Joshi > Bioinformatics Analyst/Programmer > UC Davis Bioinformatics Core > http://bioinformatics.ucdavis.edu/ > najoshi -at- ucdavis -dot- edu > 530.752.2698 (w) > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
I'd like to propose a slightly different approach. I don't like the idea of introducing a different type of requirement for this because it really isn't a different type of requirement. You are still requiring a package, you just want to use a different mechanism to provide it. So, instead, what about creating a plugin for tool requirement handling? You could replace the default dependency injection approach with one based on modules. So, the tool configurations would not need to change, you would just change the configuration of your instance to inject module loads instead of source env.shs. And yes, we would definitely accept something like this as long as it is a plugin that is not enabled by default. -- James Taylor, Associate Professor, Biology/CS, Emory University On Thu, Sep 12, 2013 at 7:03 PM, Guest, Simon wrote: >> > Another vote for the excellent modules system from us at AgResearch >> >> Galaxy's dependency injection works in a similar way, without requiring >> all the external dependencies. We don't need unload since we compose a >> unique environment for every tool execution. This is much cleaner that >> loading everything into the environment before running Galaxy, we want >> tools to be executed with as clean an environment as possible. > > Setting the environment for each tool just prior to running the tool is > clearly better. > > What I'm doing currently is loading an environment module just before running > a tool in Galaxy. So far, I have done this in the tool XML file by prefixing > the command with a module load command, e.g. > > . /etc/profile; module load EMBOSS/5.0.0; infoseq -sequence > [snipped] > > The ugly ". /etc/profile" is needed for technical reasons. I don't like > this, as it's a bit hacky, but what it achieves is useful and flexible. > > I envisage an extension to the tool XML syntax which would let me write it > like this, say: > >version="5.0.0">EMBOSS > infoseq -sequence [snipped] > > This will enable my native EMBOSS wrapper to run the defined version of the > (system-installed) EMBOSS suite. However, it would only be worth me writing > this extension if there's a likelihood of getting it into Galaxy. Is there? > > Here's where I'm coming from. I already have most of the applications I want > installed on my system. I am working on updating the RPMs to support > side-by-side installs of multiple versions, with version selection by the > environment modules facility. This is a small change (and improvement) on > what I have today. It's a work in progress, but it's a manageable amount of > work. It's looking likely that this multi-version packaging approach may be > adopted by an official CentOS Scientific Repo, and this could change the > landscape in terms of packaging of scientific applications for major Linux > distributions. Who knows, that's all in the future. > > In the meantime, there's work going on in Galaxy land to package every > application wanted in Galaxy, by writing install scripts, encapsulating > environment variables in tool XML files, etc. (There is difficulty here in > writing install scripts which work on every platform, which I have commented > on previously.) > > The big question is, will the Galaxy team accept contributions to Galaxy to > support different ways of solving this packaging problem? In particular, > allowing for environment modules to be loaded as part of running a tool, and > having this functionality in the toolshed? > > cheers, > Simon > > > === > Attention: The information contained in this message and/or attachments > from AgResearch Limited is intended only for the persons or entities > to which it is addressed and may contain confidential and/or privileged > material. Any review, retransmission, dissemination or other use of, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipients is prohibited by AgResearch > Limited. If you have received this message in error, please notify the > sender immediately. > === ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] PATH and local jobs
> Another vote for the excellent modules system from us at AgResearch. I think > it's the only sane way to manage PATH, etc, when you have multiple versions > of tools installed. > http://modules.sourceforge.net/ > I know I've said this before here. Tell all your friends. ;-) Galaxy's dependency injection works in a similar way, without requiring all the external dependencies. We don't need unload since we compose a unique environment for every tool execution. This is much cleaner that loading everything into the environment before running Galaxy, we want tools to be executed with as clean an environment as possible. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Better packaging for toolshed binaries
On Thu, Aug 29, 2013 at 3:36 AM, Peter Cock wrote: > There is a similar but probably larger set of Debian packages > available via Debian-Med and Bio-Linux too. The catch here > is can you install arbitrary versions of a tool in parallel? And > I think the answer sadly is no. This is the crucial concern for us. The standard OS packaging approaches (RPM and DEB) do not support this except very poorly. This is something we absolutely need. There are other package managers that do a better job (I'm quite fond of Homebrew on OS X, NIX also looks nice) but would add more dependencies. > The idea of standard recipe templates (e.g. typical Python > install) James outlined here might help: > http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-August/016273.html I (as always) think a lot can be solved through abstraction. I'm envisioning a very high level description of what it takes to install a package, and then have different adapters to take that and install it for a given OS. -- James Taylor, Associate Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool_dependencies.xml format
On Mon, Aug 26, 2013 at 11:48 AM, John Chilton wrote: > I think it is interesting that there was push back on providing > infrastructure (tool actions) for obtaining CBL from github and > performing installs based on it because it was not in the tool shed > and therefore less reproducible, but the team believes infrastructure > should be put in place to support pypi. Well, first, I'm not sure what "the team" believes, I'm stating what I believe and engaging in a discussion with "the community". At some point this should evolve into what we are actually going to do and be codified in a spec as a Trello card, which is even then not set in stone. Second, I'm not suggesting we depend on PyPI. The nice thing about the second format I proposed on galaxy-dev is that we can easily parse out the URL and archive that file. Then someday we could provide a fallback repository where if the PyPI URL no longer works we still have it stored. > I think we all value reproduciblity here, but we make different > calculations on what is reproducible. I think in terms of implementing > the ideas James has laid out or similar things I have proposed, it > might be beneficial to have some final answers on what external > resources are allowed - both for obtaining a Galaxy IUC gold star and > for the tool shed providing infrastructure to support their usage. My focus is ensuring that we can archive things that pass through the toolshed. Tarballs from *anywhere* are easy enough to deal with. External version control repositories are a bit more challenging, especially when you are pulling just a particular file out, so that's where things got a little hinky for me. Since we don't have the archival mechanism in place yet anyway, this is more a philosophical discussion and setting the right precedent. And yes, keeping an archive of all the software in the world is a scary prospect, though compared to the amount of data we currently keep for people it is a blip. And I'm not sure how else we can really achieve the level of reproducibility we desire. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] tool_dependencies.xml format
All,
I've been seeing some examples of tool_depedencies.xml come across of
the list, and I'm wondering if there are ways that it can be
simplified. When we were first defining these features, we talked
about having high level recipes for certain types of installs. This
could greatly simplify things. For example, can this:
http://pypi.python.org/packages/source/r/requests/re
quests-1.2.3.tar.gz
$INSTALL_DIR/lib/python
export
PYTHONPATH=$PYTHONPATH:$INSTALL_DIR/lib/python &&
python setup.py install --home $INSTALL_DIR
--install-scripts $INSTALL_DIR/bin
$INSTALL_DIR/lib/python
$INSTALL_DIR/bin
Be simplified to:
http://pypi.python.org/packages/source/r/requests/requests-1.2.3.tar.gz";
/>
The assumptions are: when version is not provided, it is 1.0 (we've
always maintained compatibility in the past for config files so
hopefully this never changes), when installing a python package the
install directories and environment variables that need to be set are
always the same.
Similar recipes could be:
autoconf: default to configure; make; make install, allow providing
configuration options
make_install: just make; make install; allow providing make options
python_virtualenv
ruby_rbenv
r_package
...
Basically, most of the times the steps to install a particular package
are boilerplate, this would remove a ton of duplication in the recipe
files. Also, a likely less popular proposal would be to go one step
further, tool_dependencies.yaml:
recipe: python_package_setuptools
name: requests
version: 1.2.3
url:
http://pypi.python.org/packages/source/r/requests/requests-${version}.tar.gz
-- jt
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Note about running functional tests.
On Mon, Aug 19, 2013 at 1:35 PM, Peter Cock wrote: >> Additionally, it is not recommended to run functional tests on a production >> instance, since the test suite alters a number of configuration settings on >> the fly, with the result being that it is not a reliable test of the exact >> production environment, which may give misleading results. > > Hang on - is that a short term warning? We *want* to run the > functional tests on our production Galaxy precisely to check > the tools are installed and working there. How are you doing this currently? I can understand running the tests in your production Galaxy environment, but they should always be run with a separate database and files directory. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] disable PBKDF2, revert to SHA1
It should be as simple as adding "use_pbkdf2=False" to the universe_wsgi.ini file. This commit made it configurable: https://bitbucket.org/galaxy/galaxy-central/commits/a6f688dade808b0e16903fbb50186ad9656c0d1c -- James Taylor, Assistant Professor, Biology/CS, Emory University On Mon, Aug 19, 2013 at 12:25 PM, Ulf Schaefer wrote: > Hello all > > It was indicated in several recent posts (e.g. there > http://dev.list.galaxyproject.org/support-pbkdf2-in-proftpd-1-3-5rc3-td4660836.html) > that for Galaxy to reliably work with ProFTPD, it is recommended to > disable the new PBKDF2 encryption of passwords and revert back to SHA1. > > Unfortunately it appears to be beyond me to figure out how to do this. > Could somebody point me in the right direction, please? > > Thanks a lot > Ulf > > ** > The information contained in the EMail and any attachments is confidential > and intended solely and for the attention and use of the named addressee(s). > It may not be disclosed to any other person without the express authority of > Public Health England, or the intended recipient, or both. If you are not the > intended recipient, you must not disclose, copy, distribute or retain this > message or any part of it. This footnote also confirms that this EMail has > been swept for computer viruses by Symantec.Cloud, but please re-sweep any > attachments before opening or saving. http://www.gov.uk/PHE > ** > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] support pbkdf2 in proftpd 1.3.5rc3
For the moment, the best solution is to disable PBKDF2 in Galaxy if you are also using FTP. That is how Galaxy main is currently configured. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Fri, Aug 9, 2013 at 10:01 AM, Nate Coraor wrote: > On Aug 9, 2013, at 2:38 AM, Leon Mei wrote: > >> Hi Nate, >> >> Thanks for the suggestion! Unfortunately, it still failed :( >> >> I got the following error message in proftp log: >> >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: enteringpostgres >> cmd_escapestring >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: enteringpostgres cmd_open >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: connection 'default' count is >> now 2 >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: exiting postgres cmd_open >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: enteringpostgres cmd_close >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: connection 'default' count is >> now 1 >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: exiting postgres cmd_close >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: exiting postgres >> cmd_escapestring >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: cache hit for user >> 'hailiang.m...@nbic.nl' >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: >>> cmd_check >> 2013-08-09 08:32:41,777 mod_sql/4.3[32384]: checking password using >> SQLAuthType 'sha1' >> 2013-08-09 08:32:41,781 mod_sql/4.3[32384]: 'sha1' SQLAuthType handler >> reports failure >> 2013-08-09 08:32:41,781 mod_sql/4.3[32384]: checking password using >> SQLAuthType 'sha256' >> 2013-08-09 08:32:41,781 mod_sql/4.3[32384]: 'sha256' SQLAuthType handler >> reports failure >> 2013-08-09 08:32:41,781 mod_sql/4.3[32384]: checking password using >> SQLAuthType 'pbkdf2' >> 2013-08-09 08:32:41,841 mod_sql/4.3[32384]: 'pbkdf2' SQLAuthType handler >> reports failure >> 2013-08-09 08:32:41,841 mod_sql/4.3[32384]: <<< cmd_check >> 2013-08-09 08:32:41,841 mod_sql/4.3[32384]: <<< cmd_auth >> >> The old user account generated before our code update still works. >> >> I wonder how it is configured at the Galaxy main server? >> >> Thanks, >> Leon > > It isn't in use on the Main server, but now that I'm aware that ProFTPD has > PBKDF2 support, I will put this on my to-do list for next week to test. > > --nate > > >> >> >> >> On Thu, Aug 8, 2013 at 8:45 PM, Nate Coraor wrote: >> On Jul 26, 2013, at 3:51 PM, Leon Mei wrote: >> >> > Dear galaxy developers, >> > >> > We have tried today to upgrade our proftpd configuration to make uploading >> > for our galaxy users possible again, both for users with "old" as well as >> > "new" style hashed passwords. We upgraded proftpd on the server to >> > 1.3.5rc3 and have the following SQL part in our configuration file based >> > on the post of >> > http://dev.list.galaxyproject.org/ProFTPD-integration-with-Galaxy-td4660295.html >> > >> > SQLEngine on >> > SQLLogFile /var/log/proftpd-sql.log >> > SQLBackend postgres >> > SQLConnectInfo galaxy@localhost:5840 galaxyftp <[ourpassword]> >> > SQLAuthTypesSHA1 SHA256 PBKDF2 >> > SQLPasswordPBKDF2 SHA256 1000 24 >> > SQLPasswordUserSalt sql:/GetUserSalt >> > SQLAuthenticate users >> > SQLDefaultUID 108 >> > SQLDefaultGID 116 >> > SQLDefaultHomedir /opt/cloudman/pkg/proftpd/var >> > SQLUserInfo custom:/LookupGalaxyUser >> > SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN >> > substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from >> > 38 for 32) ELSE password END) AS >> > password2,'108','116','/mnt/galaxyData/tmp/ftp/%U','/bin/bash' FROM >> > galaxy_user WHERE email='%U'" >> > SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING (password from 1 >> > for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16) END) AS salt >> > FROM galaxy_user WHERE email='%U'" >> > >> > We have executed the LookupGalaxyUser and GetUserSalt commands manually, >> > and the results look good. Now, old users can login via ftp, but for a new >> > user, the authentication still fails: >> >
Re: [galaxy-dev] Defining $GALAXY_CPUS for use in tool wrappers
I would actually prefer we use $GALAXY_SLOTS. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jul 30, 2013 at 11:18 AM, Peter Cock wrote: > Hello all, > > Re: > http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html > http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011557.html > > Something I raised during the GCC2013, and we talked about > via Twitter as well was a Galaxy environment variable for use > within Tool Wrappers setting the number of threads/CPUs to > use. > > The idea is that you can configure a default value, and then > override this per runner or per tool etc. James Taylor had > suggested calling this environment variable $GALAXY_CPUS > which seem fine to me (personally I'd say threads not CPUs > but I don't really mind). e.g. > > my_tool --threads "\$GALAXY_CPUS" > --input "$input" --output "$output" > > Everyone I spoke to about this seemed positive about the idea. > > This would/should be integrated into the various cluster back > ends, for example for SGE/OGE the number of threads is > already configurable via the DRMAA settings and available > as the environment variable $NSLOTS for non-MPI jobs, so > my guess is all Galaxy needs to do is something like this: > > $ hg diff > diff -r ce0d758bb995 lib/galaxy/jobs/runners/drmaa.py > --- a/lib/galaxy/jobs/runners/drmaa.py Tue Jul 30 12:30:30 2013 +0100 > +++ b/lib/galaxy/jobs/runners/drmaa.py Tue Jul 30 16:10:40 2013 +0100 > @@ -43,6 +43,7 @@ > # - execute the command > # - take the command's exit code ($?) and write it to a file. > drm_template = """#!/bin/sh > +export GALAXY_CPUS="$NSLOTS" > GALAXY_LIB="%s" > if [ "$GALAXY_LIB" != "None" ]; then > if [ -n "$PYTHONPATH" ]; then > > Is there an open Trello card for this? > > Thanks, > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] job_conf.xml workers
use_threadpool and threadpool_workers specify the number of threads for handling web requests and have nothing to do with job running. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, Jul 24, 2013 at 11:02 AM, Ido Tamir wrote: > Hi, > > what is the relationship between workers defined in universe and > job_conf.xml? e.g.: > > [server:handler0] > use = egg:Paste#http > port = 9092 > host = 127.0.0.1 > use_threadpool = true > threadpool_workers = 5 > > > > > load="galaxy.jobs.runners.local:LocalJobRunner"/> > load="galaxy.jobs.runners.drmaa:DRMAAJobRunner" workers="10"/> > > > a) do I have to specify the sum of the workers in job_conf.xml in universe? > e.g. 4 local, 10 drmaa => 14 threadpool_workers per handler? Or is universe > overridden by job_conf.xml? > > b) > How many concurrent jobs can I then expect to have concurrently? 14 per > handler? > > > thank you very much, > ido > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bug report
What format are the files you are trying to process with MACS in? -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jul 23, 2013 at 3:05 PM, Jianxin Wang wrote: > Hi Galaxy dev team, > > I'm trying to the you galaxy server to do NGS peak calling: Tools -> NGS: > Peak Calling -> > MACS Model-based Analysis of ChIP-Seq. However, the page won't allow me > enter the input files: the drop-down selection box won't show anything when > clicked. Same problem for the ChIP-Seq Control File selection box. Please > take a look and email me back if problem is fixed. > > > Jianxin Wang, PhD > Computational Biologist > jianxin.w...@oicr.on.ca > > Ontario Institute for Cancer Research > MaRS Centre, South Tower > 101 College Street, Suite 800 > Toronto, Ontario, Canada M5G 0A3 > > Tel: 416-673-8569 > www.oicr.on.ca > > This message and any attachments may contain confidential and/or privileged > information for the sole use of the intended recipient. Any review or > distribution by anyone other than the person for whom it was originally > intended is strictly prohibited. If you have received this message in error, > please contact the sender and delete all copies. Opinions, conclusions or > other information contained in this message may not be that of the > organization. > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] workflow: default parameters
Not currently possible, but it shouldn't be too difficult to make work. Please file a Trello card. On Tue, Jul 9, 2013 at 11:36 AM, Keilwagen, Jens wrote: > when building workflows we can choose for each parameter whether to 'set at > runtime' or to 'set in advance' (when creating or editing the workflow). We > are looking for a possiblity to set a default value that is used (only) for > the workflow, i.e., we like to specify a value that might be used by most > users, but might be altered by expert users. This value might be different > from the default value of the corresponding tool. So far, we did not find any > possiblity. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy-less Tool Installing
On Mon, Jul 15, 2013 at 7:27 PM, John Chilton wrote: > One of my goals for the GCC was to sell the idea that tool shed > repositories need to be installable without a database present. I John, as I've mentioned in the past I'm very strongly in favor of this. There is absolutely no reason why the ToolShed needs to be tied to Galaxy. For dependency installation we've now got a nice system that does version isolation well, and I've had several groups express interest in using the wrappers from the ToolShed in environments other than Galaxy. > from Greg and others (Dave, Bjorn, Peter, Nate) has gone into building a > modular > dependency system that could very easily be leveraged by applications other > than > Galaxy, so the extra steps that could be taken to make this possible should to > make the codebase as broadly useful and to encourage adoption. Exactly! > 1. Rework installing tool shed repositories to not require a database. A > kind > of messy way to do this might be adding a use_database flag throughout. A > cleaner way might be to use allow the core functionality to work with > callbacks > or plugins that performed the database interactions. Personally I would definitely prefer a plugin model. Basically, a layer isolating the underlying datastore used during tool installation, dependency resolution, et cetera. However, it occurs to me that even if the tool installation component is isolation from Galaxy, it could still use the same model and database. All package management systems need some way to maintain state, a sqlite database is not an unreasonable choice. > 2. Separate the core functionality out of the Galaxy code base entirely into > a reusable, stand-alone library. This is a long term goal for us. I think at first we'd like to at least see it all happen in the same repository so that we can still support a checkout and run scenario. But decoupling the various web applications and share components that make up the Galaxy ecosystem is very important. > include building a little tool shed version of the module command - > http://linux.die.net/man/1/module to demonstrate this work and have something > immediately useful produced. YES! Absolutely. One of the reasons for the way tool dependency injection was implemented was to support a command line module system. Coupling this with the toolshed to also enable dependency installation and management completes the puzzle. (and yes to all the various commands, through I figured it would be gx_module ;) > This would be different from using the API scripts because there would be no > API, Galaxy instance, or Galaxy database involved - just the Galaxy code. If > this was able to split into its own Python library, one could imagine even > allowing something like tsmodule to be installable right from pip and > recursively fetch a toolshed_client library or something like that. Absolutely. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Plugins
Michael, Galaxy is all about plugins! Tools, datatypes, batch systems, visualizations, et cetera. Additional ways to extend Galaxy through plugins are very welcome. We just need to make sure that whatever solution is chosen is extremely general. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jul 16, 2013 at 1:57 PM, Michael E. Cotterell wrote: > Are there any plans in the works to support plugins within Galaxy? One of the > tools I'm working on is actually more of an extension of the workflow > canvas/editor's user interface. Right now, I've minimized the modifications > required to the editor.mako file to simply adding a mako include near the > bottom of the file. However, like I just said, this requires editing a Galaxy > file. It would be a lot better if Galaxy supported plugins with hooks, thus > allowing me to extend the user interface (or any other part of the > application) by registering hooks in a plugin. > > To demonstrate how easy it would be to integrate plugins into a python > application, I've written up a simple plugin manager script (~100 loc) an > placed it in a gist: https://gist.github.com/mepcotterell/6004997. Using this > (or something similar to this) would make Galaxy extremely extensible. > > The only hard part would be figuring out where you want to place the hooks > and documenting them for plugin developers. > > Sincerely, > Michael E. Cotterell > > Ph.D. Student in Computer Science, University of Georgia > Instructor of Record, Graduate RA & TA, University of Georgia > Faculty Liaison, CS Graduate Student Association, University of Georgia > mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) > mepc...@uga.edu (mailto:mepc...@uga.edu) > m...@cs.uga.edu (mailto:m...@cs.uga.edu) > http://michaelcotterell.com/ > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] repeat tag enhancement?
Rather than adding additional increments to add/remove, what about allowing the user to directly set the number of elements in the repeat? -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jul 16, 2013 at 7:34 AM, Berner, Thomas wrote: > Hey guys, > > > > i was searching the mailing list and wiki for some hints if there is a > possibility to enhance the repeat tag. > > We want to give the option to our users to add more than one additional set > of the contained parameters at one go e.g. 10 or 20 or 50, because it can be > very frustrating to click the “add new dataset” button 50 times. > > Is there any option to do this with a text field or preset like “add 20 new > datasets” or something like that? > > I would be very thankful for an advice. > > > > Greetings, > > > > Thomas > > > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Lessen load on server
Can you be more specific about what resources Galaxy is consuming and how you are measuring them? Galaxy should consume very few resources when idle (other than perhaps memory). On Fri, Jul 12, 2013 at 3:20 PM, Auerbach, Kenneth R. wrote: > Hello, > > Can you tell me if there is any way to lessen the load of Galaxy on our > server. Even when it's idle it's consuming a lot of resources. > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically decide where a Galaxy tool runs
On Fri, Jul 12, 2013 at 2:42 PM, Saliya Ekanayake wrote: > Hi, > > I see in documentation that Galaxy tools may be configured to run on a > cluster (http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster). > This is something our research group is interested in, but it's not possible > to specify which cluster the particular tool should run on ahead of time. This is definitely possible, see "Mapping tools to destinations" on this page: http://wiki.galaxyproject.org/Admin/Config/Jobs ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Config specific tools that auto hide other panels to get more room
You can use the minwidth attribute in the tool config for this purpose, it will hide the history if the main panel is less than a certain width. See the UCSC Get Data tool as an example. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, Jul 11, 2013 at 12:00 AM, Peng Li wrote: > Dear All > > I develop one data display tool for my local galaxy instance, I want to make > the history panel and tools panel auto hidden while I load this tools since > I need more room to show the data, does anyone has any clue for this? Like > change the tools xml or add a JS function to operate father web-frame? > > > > Many thanks > > > > wbflame > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] gzipped fastq reader
open_compressed in bx-python does this already (for bz2 as well). On Jul 8, 2013, at 5:58 PM, Peter Cock wrote: > On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch > wrote: >> Peter and Dan, >> I like the idea of replacing all open() with galaxy_open() in all tools. You >> can tell the format by looking at the first 4 byes (see C code below from >> the UCSC browser team). Is there some pythonic way of overriding open? > > There is monkey patching (replace the current 'open' function with > your modified version), but that is not a good idea in general. > > In any case, this would only affect the small number of Python > tools which happen to use the Galaxy parsing libraries - which > is a very small fraction of the tools in Galaxy. Most of the tools > in Galaxy are compiled programs and are entirely separate. > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How were liftover and nib files generated?
liftOver files come from the UCSC genome browser, derived from pairwise alignments. A variety of tools can create nib files from fasta, however we prefer the use of the twoBit format now, which can be created using faToTwoBit from the UCSC utilities. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Sat, Jul 6, 2013 at 9:19 PM, Perez, Ricardo wrote: > Dear all, > > When I rsync files from let say the following command: > rsync -avzP rsync://datacache.g2.bx.psu.edu/indexes/dm3 . > > The system will sync files that are related to genome aligments, liftover, > and nibs. > How did the Galaxy team generated this files? > The reason I ask is because I would lather will like to generate similar > files to custom genomes to be used in galaxy. > > Thank you for your time, > --Ricardo Perez > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] RFC: Cheetah template in tool_dependencies.xml
It probably makes sense to specify the template type in this case.
Maybe an additional attribute, language='cheetah' ?
On Sun, Jul 7, 2013 at 1:19 PM, Björn Grüning
wrote:
>
> #if env.get('X11_LIB_DIR', False) and env.get('X11_INCLUDE_DIR', False):
>./configure --prefix=$env.INSTALL_DIR
> --x-includes=$env.X11_INCLUDE_DIR --x-libraries=$env.X11_LIB_DIR
>#else:
> ./configure --prefix=$env.INSTALL_DIR
>#end if
>
--
James Taylor, Assistant Professor, Biology/CS, Emory University
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Scripts interacting with the database
> 1) Is there any kind of existing functionality that is close to what I want? Depending on whether you are using stock Galaxy or nglims, and how you are getting data into Galaxy, there are various ways to do it. For Galaxy's stock sample tracking, you can setup workflows to trigger automatically when data arrives. However, what I suspect is that you want to use the Galaxy API for this. There should never be a need to interact directly with the database, you should be able to do everything you want using the REST APIs. There should be an example script included with Galaxy that watches a directory for datasets, loads them into Galaxy, and runs a workflow using the API. > 2) If not, > 2a) Is there a way to trigger some scripts for assembly and metrics (and > maybe more) once a MiSeq dumps all the data into Galaxy? > 2b) Is there a certain convention on interaction with the database? And > documentation on that? > 2c) Is there an existing convention on reading the Galaxy database, so > that I can produce the spreadsheet? Documentation? > 2d) Our lab would call this functionality Rover, since it will roam Galaxy > looking for and retrieving data (not really a question, but I thought they > came up with a good name!) > > -- > Lee Katz, Ph.D. > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Dependency Check for eXpress - requirements syntax incorrect?
Hi Adam, yes, it should be package, the "binary" type has never been supported but keeps getting propagated for some reason. At some point it might be nice to have a configurable mapping from the other dependency types to packages. However, changing it to package *should* work. Can you send the relevant part of the paster log, it should log information about dependency resolution and the command line. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, Jun 27, 2013 at 1:38 PM, Adam Brenner wrote: > Howdy, > > One of our users is running the eXpress package under NGS: RNA > Analysis -> eXpress -- galaxy/dist/tools/ngs_rna/express_wrapper.* > > I noticed the syntax for express_wrapper.xml has the following: > eXpress > any reason why it was set to that, as opposed to: > eXpress > > In our setup, we need to source the env.sh file for express to setup > the correct environmental variables. Changing from binary to package > has not solved this and re-running the job never sources our env.sh > file. > > What am I doing wrong? > > My tool dependency directory looks like so: >galaxy/dist/tool_dependency/eXpress/ >|---default --> 1.3.0 >|1.3.0/env.sh > > Thanks, > -Adam > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebre...@uci.edu > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Styling the Galaxy tool config xml
Graham, this is not supported. The goal with the tool config is not to specify any particular user interface format so that we are not locked into a particular UI technology in the future. It is an abstract description from which we can generate HTML, but also potentially other interfaces. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Fri, Jun 21, 2013 at 11:11 AM, graham etherington (TSL) wrote: > Hi, > I'm trying to see if I can style Galaxy tools, so instead of all the input > fields being a long list one after the other, I can align a few of the > fields side-by-side. > Has anyone any suggestions on how I might accomplish this? > Best wishes, > Graham > > > Dr. Graham Etherington > Bioinformatics Support Officer, > The Sainsbury Laboratory, > Norwich Research Park, > Norwich NR4 7UH. > UK > Tel: +44 (0)1603 450601 > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visulaization tool
Display applications are extensively documented here: http://wiki.galaxyproject.org/Admin/Tools/External%20Display%20Applications%20Tutorial -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, Jun 20, 2013 at 2:20 AM, shashi shekhar wrote: > Hi james, > > can you share some examples of display application . and in which tool you > use for display application in galaxy ? > > > Regards > shashi > > ________ > From: James Taylor > To: shashi shekhar > Cc: Ross ; "galaxy-dev@lists.bx.psu.edu" > > Sent: Wednesday, 19 June 2013 7:03 PM > > Subject: Re: [galaxy-dev] visulaization tool > > Tightly integrated interactive visualizations like Trackster need to be web > based (implemented in Javascript). See our paper that just came out in BMC > genomics. Note there is already a phylogenetic tree visualization tool > (phyloviz) that was built by Tomithy Too. Extensions to that for additional > features would be welcome. > > For a Java based visualization tool you need to take a completely different > approach. Look at Display Applications support, specifically examples like > IGB and IGV. > > On Jun 19, 2013, at 7:56 AM, shashi shekhar wrote: > > Hi Ross, > Actually i have to add the visualization which will be interactive . it > will take dataset from galaxy .it will be in the same way as tracker works > . > in stead of tracker i want to add my own visualization tool. from wehere > should i start to replace tracker with my own visulization tool. > Regards > shashi > > > From: Ross > To: shashi shekhar > Cc: Hans-Rudolf Hotz ; "galaxy-dev@lists.bx.psu.edu" > > Sent: Wednesday, 19 June 2013 5:02 PM > Subject: Re: [galaxy-dev] visulaization tool > > Shashi, > Thanks for the additional information. > AFAIK: If it's command line driven and spews out pdf's, adapt any similar > tool that spews out pdfs like fastqc. > Galaxy tool execution is isolated from the user and from Galaxy itself, so > communication is just not possible if it's interactive - unless you hack the > tool into Galaxy itself or something. There is no documentation for that. > > > > On Wed, Jun 19, 2013 at 9:01 PM, shashi shekhar > wrote: > > Hi Ross, > thanks for the reply . actually i want to use TreeView as visulaization > tool. > > TreeView X is an open source and multi-platform program to display > phylogenetic trees. It can read and display NEXUS and Newick format tree > files (such as those output by PAUP*, ClustalX, TREE-PUZZLE, and other > programs). It allows one to order the branches of the trees, and to export > the trees in SVG format. > > i have to add treeviewx in galaxy.. > > Regards > shashi > > > > From: Ross > To: shashi shekhar > Cc: Hans-Rudolf Hotz ; "galaxy-dev@lists.bx.psu.edu" > > Sent: Wednesday, 19 June 2013 3:28 PM > > Subject: Re: [galaxy-dev] visulaization tool > > Shashi, > Hans made a brave guess based on your vague question, but you'll probably > get better answers if you do us all a favour and make an effort to provide > unambiguous detail about exactly what you want to achieve. > > If there is a working Galaxy tool that does what you need but using a > different "visualisation program" (whatever you mean by that vague term), > substituting your executable will probably be relatively trivial. eg do you > consider the Html object produced by the fastqc or the image from the > scatterplot tool to be "visualisation program" outputs? If so, clone a > working tool. > > > On Wed, Jun 19, 2013 at 5:37 PM, shashi shekhar > wrote: > > > Hi, > > I want to add the the complete separate tool in galaxy . how will i do it > ..i don't want to use existing visulaization tool. > > Regards > shashi > > > From: Hans-Rudolf Hotz > To: shashi shekhar > Cc: "galaxy-dev@lists.bx.psu.edu" > Sent: Wednesday, 19 June 2013 12:29 PM > Subject: Re: [galaxy-dev] visulaization tool > > Hi Shashi > > Have a look at: > > http://wiki.galaxyproject.org/Learn/Visualization > http://wiki.galaxyproject.org/Visualization%20Setup > > > Regards, Hans-Rudolf > > > > On 06/19/2013 08:07 AM, shashi shekhar wrote: >> Hi, >> >> i want to add new visualization tool in galaxy . can i get the required >> documents ? >> >> can u send me the details how can i add the visualization tool in galaxy >> . >> >> >> >> Regards >> shashi > > > > > >
Re: [galaxy-dev] Default history
On Wed, Jun 19, 2013 at 3:32 PM, Jordi Vaquero wrote: > the shared and publish option of history but they don't work as I wanted. Published histories should be accessible to anonymous users, under the Shared Data tab. Can you provide more information on why this is not working for you? (Data libraries are also a good solution) -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visulaization tool
Tightly integrated interactive visualizations like Trackster need to be web based (implemented in Javascript). See our paper that just came out in BMC genomics. Note there is already a phylogenetic tree visualization tool (phyloviz) that was built by Tomithy Too. Extensions to that for additional features would be welcome. For a Java based visualization tool you need to take a completely different approach. Look at Display Applications support, specifically examples like IGB and IGV. On Jun 19, 2013, at 7:56 AM, shashi shekhar wrote: > Hi Ross, > Actually i have to add the visualization which will be interactive . it > will take dataset from galaxy .it will be in the same way as tracker works . > in stead of tracker i want to add my own visualization tool. from wehere > should i start to replace tracker with my own visulization tool. > Regards > shashi > > From: Ross > To: shashi shekhar > Cc: Hans-Rudolf Hotz ; "galaxy-dev@lists.bx.psu.edu" > > Sent: Wednesday, 19 June 2013 5:02 PM > Subject: Re: [galaxy-dev] visulaization tool > > Shashi, > Thanks for the additional information. > AFAIK: If it's command line driven and spews out pdf's, adapt any similar > tool that spews out pdfs like fastqc. > Galaxy tool execution is isolated from the user and from Galaxy itself, so > communication is just not possible if it's interactive - unless you hack the > tool into Galaxy itself or something. There is no documentation for that. > > > > On Wed, Jun 19, 2013 at 9:01 PM, shashi shekhar > wrote: > Hi Ross, > thanks for the reply . actually i want to use TreeView as visulaization tool. > > TreeView X is an open source and multi-platform program to display > phylogenetic trees. It can read and display NEXUS and Newick format tree > files (such as those output by PAUP*, ClustalX, TREE-PUZZLE, and other > programs). It allows one to order the branches of the trees, and to export > the trees in SVG format. > > i have to add treeviewx in galaxy.. > > Regards > shashi > > > From: Ross > To: shashi shekhar > Cc: Hans-Rudolf Hotz ; "galaxy-dev@lists.bx.psu.edu" > > Sent: Wednesday, 19 June 2013 3:28 PM > > Subject: Re: [galaxy-dev] visulaization tool > > Shashi, > Hans made a brave guess based on your vague question, but you'll probably get > better answers if you do us all a favour and make an effort to provide > unambiguous detail about exactly what you want to achieve. > > If there is a working Galaxy tool that does what you need but using a > different "visualisation program" (whatever you mean by that vague term), > substituting your executable will probably be relatively trivial. eg do you > consider the Html object produced by the fastqc or the image from the > scatterplot tool to be "visualisation program" outputs? If so, clone a > working tool. > > > On Wed, Jun 19, 2013 at 5:37 PM, shashi shekhar > wrote: > > Hi, > > I want to add the the complete separate tool in galaxy . how will i do it ..i > don't want to use existing visulaization tool. > > Regards > shashi > > From: Hans-Rudolf Hotz > To: shashi shekhar > Cc: "galaxy-dev@lists.bx.psu.edu" > Sent: Wednesday, 19 June 2013 12:29 PM > Subject: Re: [galaxy-dev] visulaization tool > > Hi Shashi > > Have a look at: > > http://wiki.galaxyproject.org/Learn/Visualization > http://wiki.galaxyproject.org/Visualization%20Setup > > > Regards, Hans-Rudolf > > > > On 06/19/2013 08:07 AM, shashi shekhar wrote: > > Hi, > > > > i want to add new visualization tool in galaxy . can i get the required > > documents ? > > > > can u send me the details how can i add the visualization tool in galaxy . > > > > > > > > Regards > > shashi > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] setting parameters for uploading a file for a java program
The upload_dataset parameter type is very specific to the upload tool.
You may be able to make this work but it is not a supported feature
(meaning, we can't tell you how to do it). You probably want to use a
normal data input (" wrote:
> Hi all,
>
>I have a tool written in java for which I want to upload a file as a
> parameter. I have trouble figuring out how to do it. I tried writing the xml
> file similar to 'tools/data-source/upload.xml' but it is not working.
>
> correlation.xml
>
>
>
>for the given BAM file
>
>
>
> java -jar Correlation.jar $input_bam_file $output
>
>
>
>
>
> file_type_name="file_type" metadata_ref="files_metadata">
>
> not ( ( isinstance(
> value, unicode ) or isinstance( value, str ) ) and value != "" )
>
>
>
>
> value_from="self:app.datatypes_registry.get_upload_metadata_params"
> value_ref="file_type" value_ref_in_group="False" />
>
>
>
>
>
>
>
>
>
>
> I am not sure where I am making a mistake... Is the procedure to upload a
> file for java different from python?
>
> Thanks,
> VJ.
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] problem integrating new jar file
>> >> sample.xml >> >> >>java -jar sample.jar >> Your tool config does not define any output elements, so no dataset is created. You need to define at least one output, and then use it's name in the command line. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy cuffdiff running too long
> [galaxy@q ~]$ /bin/cuffdiff > /bin/cuffdiff: /usr/lib64/libz.so.1: no version information available > (required by /bin/cuffdiff) > cuffdiff v2.0.2 (3522) This definitely indicates a problem with your cuffdiff install. Did you build cuffdiff on that system or install in some other way? (very odd to have it installed in /bin). ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Fwd: Failed runtime validation of To
This message should be associated with a particular step in the workflow. If you can tell us what tool it is that will help to diagnose. The message means that a parameter that could not be checked until the workflow ran had an invalid value. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Mon, Jun 10, 2013 at 5:19 PM, Sameer Dixit wrote: > > > Hey all, > > I've been trying to run a workflow on Galaxy, which basically attempts to > establish a correlation between the Chip-Chip, Polymerase binding and Dnase > Hypersensitivity data. The input would be these 3 files, I am initially > performing a liftover on the Chip-Chip data to convert from hg18 to hg19 and > then using the output to establish the correlation. > > The following error keeps coming up > > 'Failed runtime validation of To (An invalid option was selected, please > verify)' > > > > Can anyone please tell me what this might mean? > > This came up while attempting to run the following tool > > Traceback (most recent call last): > File > "/galaxy/home/g2main/galaxy_main/lib/galaxy/jobs/runners/__init__.py", line > 121, in prepare_job > job_wrapper.prepare() > File "/galaxy/home/g2main/galaxy_main/lib/galaxy/jobs/__init__.py", line > 656, in prepare > self.tool.handle_unvalidated_param_values( incoming, self.app ) > File "/galaxy/home/g2main/galaxy_main/lib/galaxy/tools/__init__.py", line > 2343, in handle_unvalidated_param_values > self.handle_unvalidated_param_values_helper( self.inputs, input_values, > app ) > File "/galaxy/home/g2main/galaxy_main/lib/galaxy/tools/__init__.py", line > 2382, in handle_unvalidated_param_values_helper > raise LateValidationError( message ) > LateValidationError > > Any assistance would be highly appreciated!! > > Thanks > > -Sameer > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tag in xml file for a C++ tool
You might find it instructive to look at the existing bedtools wrappers here: http://toolshed.g2.bx.psu.edu/view/aaronquinlan/bedtools -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, May 22, 2013 at 12:21 PM, vijayalakshmi wrote: > Hi all, > > I want to include a new tool to galaxy. The tool is written in C++. How > do I write the xml file to execute a cpp program? The tool that I am trying > to integrate ishttps://code.google.com/p/bedtools/ > > I have the class file to be executed and the command to execute it. For > example, > > 'IntersectBed' is my class file. The command to execute it is > $intersectBed -a reads.bed -b genes.bed (reads.bed and genes.bed are input > files provided by the user) > > How should I write the tag in the xml file so as to execute the > above command? Since C++ is complied as opposed to Python which is > interpreted, I am not sure how to write the tag. > > > I am a newbie. I appreciate any guidance offered! > > Thanks, > VJ. > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error in installing Galaxy.
On Tue, May 21, 2013 at 7:36 AM, sridhar srinivasan wrote: > I use the command you mentioned in reply.. > and the error is "psql: FATAL: Ident authentication failed for user > "galaxy" " Your problem is then not in Galaxy. Your postgresql server either does not have a galaxy user or does not allow ident authentication for that user. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] memdump
It is a debugging feature that dumps an enormous amount of information about memory usage. Unless you are debugging a memory leak there is absolutely no reason you would want to use this. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, May 16, 2013 at 8:51 AM, Nikos Sidiropoulos wrote: > Hi everybody > > Has anybody used the "profile memory usage" feature, and if so, what are the > benefits and the compromises of it? > > I'm trying to find documentation about it, without any luck. > > Thanks > > Nikos > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Embedding a help text in a Workflow?
On Fri, May 10, 2013 at 12:28 PM, Peter Cock wrote: > Why HTML and not the same markup as the tool help? Either way, > there are likely plugin editors that might be integrated... but for now > I would be content to edit the *.ga file by hand if need be. I don't believe there is any good javascript editor for reStructuredText which is used in the tool help. For non-html readable markup these days I'd prefer markdown since there seems to be so much consensus around it. Or a fairly minimal subset of HTML. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How is the paster.py running to start Galaxy Framework ?
Recent versions of Galaxy no longer use PasteScript at all, you seem to be looking at a fairly old version. Currently, scripts/paster.py calls lib/galaxy/util/pastescript/serve.py The "run" method in that module users PasteDeploy to load everything that is specified in the universe_wsgi.ini. Specifically, calling buildapp for the web application which starts all of the other Galaxy components. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Sun, May 12, 2013 at 3:11 AM, 王聃Don wrote: > Hello, Galaxy developers: > I'm a galaxy user and have customized a Galaxy instance locally. I'm > very interested in Galaxy framework and the way it's running. I've > search over the doc from readthedocs and look through some source code > files. But I'm confused at first: How to run paster.py to start all > process to build a Galaxy server? I've no idea why the last line of code > "command.run()" can start the whole Galaxy server and thanks for someone > to point it out. > Thanks for your help! > wangdan > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] history page not opening
How is your Galaxy configured? Are you using any kind of proxy server in front of it? -- James Taylor, Assistant Professor, Biology/CS, Emory University On Mon, May 13, 2013 at 2:53 AM, sridhar srinivasan wrote: > Dear Developers, > > After configuring the galaxy.. i could'nt see the history column in the web > page.. > it shows the error as > > "Content Encoding Error > The page you are trying to view cannot be shown because it uses an invalid > or unsupported form of compression." > > > Best > Sridhar > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem with Tools Relying on R
Adam, there are two problems here. We have some kind of bug regarding
putting the stderr string in the Galaxy database when it contains unicode
strings.
However, in both cases you can still read the message it is trying to save.
In the first case, "Error in hclustfun(distfun(x))" -- this might just be a
warning and the tool might still work -- but in the second case, Error in
library("cummeRbund"), is definitely fatal.
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Sun, May 12, 2013 at 12:34 AM, Panzer, Adam wrote:
> Hello All,
>
> I have recently deployed a local instance of Galaxy at work and while I've
> been able to troubleshoot problems with all other tool types, I have a
> persistent issue with using any tool that relies on R.
>
> For deseq:
>
>
> Traceback (most recent call last):
> File "/home/adam/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 109,
> in queue_job
> job_wrapper.finish( stdout, stderr, exit_code )
> File "/home/adam/galaxy-dist/lib/galaxy/jobs/__init__.py", line 999, in
> finish
> self.sa_session.flush()
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/scoping.py",
> line 127, in do
> return getattr(self.registry(), name)(*args, **kwargs)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py",
> line 1356, in flush
> self._flush(objects)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py",
> line 1434, in _flush
> flush_context.execute()
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py",
> line 261, in execute
> UOWExecutor().execute(self, tasks)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py",
> line 753, in execute
> self.execute_save_steps(trans, task)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py",
> line 768, in execute_save_steps
> self.save_objects(trans, task)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py",
> line 759, in save_objects
> task.mapper._save_obj(task.polymorphic_tosave_objects, trans)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/mapper.py",
> line 1413, in _save_obj
> c = connection.execute(statement.values(value_params), params)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py",
> line 824, in execute
> return Connection.executors[c](self, object, multiparams, params)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py",
> line 874, in _execute_clauseelement
> return self.__execute_context(context)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py",
> line 896, in __execute_context
> self._cursor_execute(context.cursor, context.statement,
> context.parameters[0], context=context)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py",
> line 950, in _cursor_execute
> self._handle_dbapi_exception(e, statement, parameters, cursor, context)
> File
> "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py",
> line 931, in _handle_dbapi_exception
> raise exc.DBAPIError.instance(statement, parameters, e,
> connection_invalidated=is_disconnect)
> ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings
> unless you use a text_factory that can interpret 8-bit bytestrings (like
> text_factory = str). It is highly recommended that you instead just switch
> your application to Unicode strings. u'UPDATE job SET update_time=?,
> stdout=?, stderr=? WHERE job.id = ?' ['2013-05-10 06:12:01.564757', '',
> 'Error: Loading required package: Biobase\nLoading required package:
> BiocGenerics\nLoading required package: methods\nLoading required package:
> parallel\n\nAttaching package: \xe2\x80\x98BiocGenerics\xe2\x80\x99\n\nThe
> following objects are masked from
> \xe2\x80\x98package:parallel\xe2\x80\x99:\n\nclusterApply,
> clusterApplyLB, clusterCall, clusterEvalQ,\nclusterExport, clusterMap,
> parApply, parCapply, parLapply,\nparLapplyLB, parRapply, parSapply,
> parSapplyLB\n\nThe following object
Re: [galaxy-dev] Track Job Runtime
For what it is worth, I think the right way to handle this is to add a new
table that tracks job state changes. Whenever the state of a job changes a
row would be written with the job id, new state, and timestamp.
(You could then try to populate it for existing jobs in exactly the way you
propose as part of a migration).
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Wed, May 8, 2013 at 11:38 AM, Geert Vandeweyer <
geert.vandewey...@ua.ac.be> wrote:
> Hi all,
>
> I'm fiddling with this, and I have a proof of principle working for PBS
> jobs using very ugly sql-alchemy hack.
>
> Idea is:
> - if job goes from queued to running : store seconds since epoch in
> 'runtime'
> - if job goes from running to finished : compare time, store difference as
> runtime.
>
> I've created an extra field 'runtime' for holding this info, using seconds
> since epoch.
> When querying afterwards, one should filter for 'OK' jobs, and discard
> jobs that are still running.
>
> right now, i have these statements to add timestamps to the database
> (somewhere in the check_watched_items function in pbs.py) :
>
> self.sa_session.execute('**UPDATE job SET runtime = :runtime WHERE id =
> :id',{'runtime':runtime,'id':**galaxy_job_id})
>
> Does anybody know how to translate this to a proper sqlalchemy statement
> such as (which does not work):
>
> self.sa_session.query(self.**model.Job).filter_by(id=**
> galaxy_job_id).update({"**runtime":runtime},synchronize_**session=False)
> or
> sa_session.execute(self.sa_**session.Table('job').update().**
> values(runtime=runtime).where(**id=galaxy_job_id))
>
> If I can figure this out, I'll try to polish it and create a pull request.
>
>
> Best,
>
> Geert
>
>
> On 05/08/2013 03:58 PM, Bossers, Alex wrote:
>
>> +1 for me!
>> Alex
>>
>>
>> __**__
>> Van:
>> galaxy-dev-boun...@lists.bx.**psu.edu[
>> galaxy-dev-boun...@lists.bx.**psu.edu]
>> namens Peter Cock [p.j.a.c...@googlemail.com]
>> Verzonden: woensdag 8 mei 2013 12:06
>> To: Geert Vandeweyer
>> Cc: galaxy-dev@lists.bx.psu.edu
>> Onderwerp: Re: [galaxy-dev] Track Job Runtime
>>
>> On Wed, May 8, 2013 at 10:08 AM, Geert Vandeweyer
>> wrote:
>>
>>> Hi,
>>>
>>> Are there options available to track the actual runtime of jobs on a
>>> cluster
>>> and store them in the database?
>>>
>> Not yet, but I'd really like to have that information too.
>>
>> Peter
>> __**_
>> Please keep all replies on the list by using "reply all"
>> in your mail client. To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>
>> http://galaxyproject.org/**search/mailinglists/<http://galaxyproject.org/search/mailinglists/>
>>
>
>
> --
>
> Geert Vandeweyer, Ph.D.
> Department of Medical Genetics
> University of Antwerp
> Prins Boudewijnlaan 43
> 2650 Edegem
> Belgium
> Tel: +32 (0)3 275 97 56
> E-mail: geert.vandewe...@ua.ac.be
> http://ua.ac.be/**cognitivegenetics <http://ua.ac.be/cognitivegenetics>
> http://www.linkedin.com/pub/**geert-vandeweyer/26/457/726<http://www.linkedin.com/pub/geert-vandeweyer/26/457/726>
>
> __**_
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>
> http://galaxyproject.org/**search/mailinglists/<http://galaxyproject.org/search/mailinglists/>
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user password different type encoding
Rather than committing this directly I created the following pull request:
https://bitbucket.org/galaxy/galaxy-central/pull-request/165/password-security-use-pbkdf2-scheme-with
It would be great if a couple of people could sign-off on it before
merging. I don't think I'm doing anything stupid, but a sanity check
is appreciated.
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Sun, May 5, 2013 at 12:12 PM, James Taylor wrote:
> Vipin, I think the main problem here is that you cannot treat PBKDF2
> as a hash in this way. Every time you hash the same password you get a
> different result because you are generating a new random salt.
> Instead, you need to decode the in database representation to extract
> the salt and then do a comparison on the hashed part.
>
> I have this working in a backward compatible way, and I think it is a
> good idea so I will be committing it to central shortly.
>
> --
> James Taylor, Assistant Professor, Biology/CS, Emory University
>
>
> On Thu, May 2, 2013 at 2:34 PM, Vipin TS wrote:
>>
>> Thanks James, I have updated the password of one user in galaxy_user table
>> with the new algorithm,
>> I also adjusted the function "new_secure_hash" in
>> /lib/galaxy/util/hash_util.py in such a way that it returns
>> the new hash instead of sha1. Now I tried to login, it fails to get the
>> account, I think there is something going
>> wrong in the password hash comparison. Can you please assit here.
>>
>> +++ b/lib/galaxy/util/hash_util.py Thu May 02 14:33:07 2013 -0400
>> @@ -25,13 +25,60 @@
>> Returns either a sha1 hash object (if called with no arguments), or a
>> hexdigest of the sha1 hash of the argument `text_type`.
>> """
>> +import hashlib
>> +from os import urandom
>> +from base64 import b64encode, b64decode
>> +from itertools import izip
>> +from pbkdf2 import pbkdf2_bin
>> +
>> +SALT_LENGTH = 12
>> +KEY_LENGTH = 24
>> +HASH_FUNCTION = 'sha256'
>> +COST_FACTOR = 1
>> +
>> if text_type:
>> +#return sha1( text_type ).hexdigest()
>> +
>> +sec_hash_1 = sha1( text_type ).hexdigest()
>> +
>> +if isinstance(sec_hash_1, unicode):
>> +sec_hash_1 = sec_hash_1.encode('utf-8')
>> +salt = b64encode(urandom(SALT_LENGTH))
>> +
>> +return 'PBKDF2${0}${1}${2}${3}'.format(
>> +HASH_FUNCTION,
>> +COST_FACTOR,
>> +salt,
>> +b64encode(pbkdf2_bin(sec_hash_1, salt, COST_FACTOR, KEY_LENGTH,
>> getattr(hashlib, HASH_FUNCTION
>>
>>
>> thanks, Vipin
>>
>>
>>> That should be the only place, it is called from the some methods of
>>> the User model object. So you could modify it to always hash new
>>> passwords in a different way, but check old passwords with sha1 first,
>>> then something else.
>>>
>>> Although it might be nice to move the functionality into
>>> security.validate_user_input since it is really specific to user
>>> passwords, especially with those changes.
>>>
>>> I'd be happy to see this go into main with sha256 or something
>>> similar. Also, we could consider adding a random per-user salt field
>>> if you are really concerned about this.
>>>
>>> --
>>> James Taylor, Assistant Professor, Biology/CS, Emory University
>>>
>>>
>>> On Thu, May 2, 2013 at 10:21 AM, Vipin TS wrote:
>>> > Hello dev-team,
>>> > I would like to add the different type of password encryption to the
>>> > users
>>> > in my galaxy instance. I started working with the current password
>>> > encoding
>>> > script:
>>> > /home/apps/galaxy-dist/lib/galaxy/util/hash_util.py
>>> >
>>> > I will keep the current sha1 and add another layer of encryption to the
>>> > sha1
>>> > hash, otherwise I need to force all my users to change the password and
>>> > follow the new hashing method.
>>> >
>>> > Can anyone please point me any other place/script which I missed
>>> > regarding
>>> > the encryption/decryption of user authentication.
>>> >
>>> > thanks in advance,
>>> > --/Vipin
>>> >
>>> >
>>> > ___
>>> > Please keep all replies on the list by using "reply all"
>>> > in your mail client. To manage your subscriptions to this
>>> > and other Galaxy lists, please use the interface at:
>>> > http://lists.bx.psu.edu/
>>> >
>>> > To search Galaxy mailing lists use the unified search at:
>>> > http://galaxyproject.org/search/mailinglists/
>>
>>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user password different type encoding
Vipin, I think the main problem here is that you cannot treat PBKDF2
as a hash in this way. Every time you hash the same password you get a
different result because you are generating a new random salt.
Instead, you need to decode the in database representation to extract
the salt and then do a comparison on the hashed part.
I have this working in a backward compatible way, and I think it is a
good idea so I will be committing it to central shortly.
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Thu, May 2, 2013 at 2:34 PM, Vipin TS wrote:
>
> Thanks James, I have updated the password of one user in galaxy_user table
> with the new algorithm,
> I also adjusted the function "new_secure_hash" in
> /lib/galaxy/util/hash_util.py in such a way that it returns
> the new hash instead of sha1. Now I tried to login, it fails to get the
> account, I think there is something going
> wrong in the password hash comparison. Can you please assit here.
>
> +++ b/lib/galaxy/util/hash_util.py Thu May 02 14:33:07 2013 -0400
> @@ -25,13 +25,60 @@
> Returns either a sha1 hash object (if called with no arguments), or a
> hexdigest of the sha1 hash of the argument `text_type`.
> """
> +import hashlib
> +from os import urandom
> +from base64 import b64encode, b64decode
> +from itertools import izip
> +from pbkdf2 import pbkdf2_bin
> +
> +SALT_LENGTH = 12
> +KEY_LENGTH = 24
> +HASH_FUNCTION = 'sha256'
> +COST_FACTOR = 1
> +
> if text_type:
> +#return sha1( text_type ).hexdigest()
> +
> +sec_hash_1 = sha1( text_type ).hexdigest()
> +
> +if isinstance(sec_hash_1, unicode):
> +sec_hash_1 = sec_hash_1.encode('utf-8')
> +salt = b64encode(urandom(SALT_LENGTH))
> +
> +return 'PBKDF2${0}${1}${2}${3}'.format(
> +HASH_FUNCTION,
> +COST_FACTOR,
> +salt,
> +b64encode(pbkdf2_bin(sec_hash_1, salt, COST_FACTOR, KEY_LENGTH,
> getattr(hashlib, HASH_FUNCTION
>
>
> thanks, Vipin
>
>
>> That should be the only place, it is called from the some methods of
>> the User model object. So you could modify it to always hash new
>> passwords in a different way, but check old passwords with sha1 first,
>> then something else.
>>
>> Although it might be nice to move the functionality into
>> security.validate_user_input since it is really specific to user
>> passwords, especially with those changes.
>>
>> I'd be happy to see this go into main with sha256 or something
>> similar. Also, we could consider adding a random per-user salt field
>> if you are really concerned about this.
>>
>> --
>> James Taylor, Assistant Professor, Biology/CS, Emory University
>>
>>
>> On Thu, May 2, 2013 at 10:21 AM, Vipin TS wrote:
>> > Hello dev-team,
>> > I would like to add the different type of password encryption to the
>> > users
>> > in my galaxy instance. I started working with the current password
>> > encoding
>> > script:
>> > /home/apps/galaxy-dist/lib/galaxy/util/hash_util.py
>> >
>> > I will keep the current sha1 and add another layer of encryption to the
>> > sha1
>> > hash, otherwise I need to force all my users to change the password and
>> > follow the new hashing method.
>> >
>> > Can anyone please point me any other place/script which I missed
>> > regarding
>> > the encryption/decryption of user authentication.
>> >
>> > thanks in advance,
>> > --/Vipin
>> >
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client. To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> > http://lists.bx.psu.edu/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> > http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user password different type encoding
The only other relevant place is the User object in model/__init__.py
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Thu, May 2, 2013 at 6:46 PM, Vipin TS wrote:
> I have updated the table schema from the script to adjust the column length
> from the following script:
> lib/galaxy/model/mapping.py
>
> Now my new registration passwords are encrypted with second layer of
> authentication using PBKDF2
> new entry from the database table:
>
> galaxy=# select username,email,password from galaxy_user where email =
> 'vi...@mail.com';
> username | email | password
> --++---
> | vi...@mail.com |
> PBKDF2$sha256$1$lv0RfxbU3SymKvEA$l4RH9f9xHrH4pcf9n6ELP1MWjG+hooEW
>
> BUT now I am experiencing the problem with authenticating the newly
> registered user name. once I log out,
> I can’t log back in again – password invalid. This tells me there is
> something going on with the password hash/compare
> function.
>
> Can you please guide through the right module to look for this,
>
> Will be quite helpful,
> --/Vipin
>
>
>
>> I have started testing with creating a new user and the password hash
>> created using new algorithm,
>>
>> galaxy=# select username,email,password from galaxy_user where email =
>> 'fml...@gmail.com';
>> username | email | password
>> --+--+--
>> | fml...@gmail.com | PBKDF2$sha256$1$e0DVCuGEua3ebxxU$Bh6
>>
>> I have updated the length of password column to 80 characters in my table
>> and still the stored password
>> seems to be in 40 char long, I print the hash after creating the second
>> hash (password -> sha1 hash 40 char long-> pbdkf2 hash 69 char long)
>>
>> before storing into the database table, I believing the hash has been
>> truncated, any idea what is happening here.
>> I am not seeing any clue in the code.
>>
>> thanks, Vipin
>>
>>>
>>> Thanks James, I have updated the password of one user in galaxy_user
>>> table with the new algorithm,
>>> I also adjusted the function "new_secure_hash" in
>>> /lib/galaxy/util/hash_util.py in such a way that it returns
>>> the new hash instead of sha1. Now I tried to login, it fails to get the
>>> account, I think there is something going
>>> wrong in the password hash comparison. Can you please assit here.
>>>
>>> +++ b/lib/galaxy/util/hash_util.py Thu May 02 14:33:07 2013 -0400
>>> @@ -25,13 +25,60 @@
>>> Returns either a sha1 hash object (if called with no arguments), or
>>> a
>>> hexdigest of the sha1 hash of the argument `text_type`.
>>> """
>>> +import hashlib
>>> +from os import urandom
>>> +from base64 import b64encode, b64decode
>>> +from itertools import izip
>>> +from pbkdf2 import pbkdf2_bin
>>> +
>>> +SALT_LENGTH = 12
>>> +KEY_LENGTH = 24
>>> +HASH_FUNCTION = 'sha256'
>>> +COST_FACTOR = 1
>>> +
>>> if text_type:
>>> +#return sha1( text_type ).hexdigest()
>>> +
>>> +sec_hash_1 = sha1( text_type ).hexdigest()
>>> +
>>> +if isinstance(sec_hash_1, unicode):
>>> +sec_hash_1 = sec_hash_1.encode('utf-8')
>>> +salt = b64encode(urandom(SALT_LENGTH))
>>> +
>>> +return 'PBKDF2${0}${1}${2}${3}'.format(
>>> +HASH_FUNCTION,
>>> +COST_FACTOR,
>>> +salt,
>>> +b64encode(pbkdf2_bin(sec_hash_1, salt, COST_FACTOR,
>>> KEY_LENGTH, getattr(hashlib, HASH_FUNCTION
>>>
>>>
>>> thanks, Vipin
>>>
>>>
>>>> That should be the only place, it is called from the some methods of
>>>> the User model object. So you could modify it to always hash new
>>>> passwords in a different way, but check old passwords with sha1 first,
>>>> then something else.
>>>>
>>>> Although it might be nice to move the functionality into
>>>> security.validate_user_input since it is really specific to user
>>>> passwords, especially with those changes.
>>>>
>>>> I'd be happy to see this go into main with sha256 or something
>>>> s
Re: [galaxy-dev] user password different type encoding
That should be the only place, it is called from the some methods of the User model object. So you could modify it to always hash new passwords in a different way, but check old passwords with sha1 first, then something else. Although it might be nice to move the functionality into security.validate_user_input since it is really specific to user passwords, especially with those changes. I'd be happy to see this go into main with sha256 or something similar. Also, we could consider adding a random per-user salt field if you are really concerned about this. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, May 2, 2013 at 10:21 AM, Vipin TS wrote: > Hello dev-team, > I would like to add the different type of password encryption to the users > in my galaxy instance. I started working with the current password encoding > script: > /home/apps/galaxy-dist/lib/galaxy/util/hash_util.py > > I will keep the current sha1 and add another layer of encryption to the sha1 > hash, otherwise I need to force all my users to change the password and > follow the new hashing method. > > Can anyone please point me any other place/script which I missed regarding > the encryption/decryption of user authentication. > > thanks in advance, > --/Vipin > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Using conditional parameters in unit tests
> I regard this as a bug or limitation of the test framework, but if not, > things are confusingly inconsistent. If you think parameter names > should be unique, it would make the Cheetah commands a bit > shorter as the wouldn't need the conditional-name-dot prefixes. Definitely a Bug/Limitation of the testing framework. Parameter names need only be unique within a given context. The testing framework should be extended to address this. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tools declaring Python dependencies
On Wed, Apr 17, 2013 at 2:43 PM, Peter Cock wrote: > Does > that mean installing Python modules locally and then > manipulating the PYTHONPATH is the best plan? That's what I had in mind, there may be better ways though. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tools declaring Python dependencies
No it does not have root, and it is very important that we not install anything at the system level since we need to maintain version isolation. We should always be able to inject things into the environment so that Python/Perl/R look in the right place. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Apr 16, 2013 at 2:05 PM, Peter Cock wrote: > On Tue, Apr 16, 2013 at 5:23 PM, Peter Cock wrote: >> Hi all, >> >> Galaxy has long had two very simple (but largely unused) >> dependency mechanisms in a tool's XML (pre-dating the >> tool shed), for binaries and Python modules, e.g. >> >> >> numpy >> taxBuilder >> >> >> http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Crequirement.3E_tag_set >> >> This has since been supplemented by the Tool Shed centric >> package requirement mechanism. >> >> How can a tool declare a Python dependency, and have >> it installed automatically via the Tool Shed? Presumably >> it would have to end up as an egg in the Galaxy eggs >> folder, and not be installed under the default Python's >> PYTHONPATH? > > Actually thinking about this on the way home, I had > that back to front - we do want to install the dependency > so that is it available to the system's default Python > (and that will not look at the Galaxy eggs folder). > > The Galaxy Tool Shed dependency installer doesn't > have root does it? That would help for many install > issues for libraries etc at system level, not just for > Python but similarly for Perl, R, etc. > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] the release_2013.04.01 update crashes our galaxy server
The default value for static_enabled is True, and I don't think this
has changed recently, so I'm not sure how your config ever worked.
Uncommenting static_enabled and changing it to False *should* solve
the problem.
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Tue, Apr 16, 2013 at 3:33 AM, Christophe Antoniewski
wrote:
> Hi James & Ross,
>
> here is the static part of our universe_wsgi.in...
>
> # all commented by chris on 30-12-2011 as pages served by apache2
> #static_enabled = True
> #static_cache_time = 360
> #static_dir = %(here)s/static/
> #static_images_dir = %(here)s/static/images
> #static_favicon_dir = %(here)s/static/favicon.ico
> #static_scripts_dir = %(here)s/static/scripts/
> #static_style_dir = %(here)s/static/june_2007_style/blue
>
> Yes, it is all commented, but worked fine since December 2011 when we
> proxied Galaxy by apache.
>
> Chris
>
>
>
>
> 2013/4/16 James Taylor
>>
>> This looks like just a configuration file problem. What is the value
>> of the various static_ options (like static_dir =) in your
>> universe_wsgi.ini?
>>
>>
>> On Mon, Apr 15, 2013 at 7:25 PM, Christophe Antoniewski
>> wrote:
>>
>> > webapp = wrap_in_static( webapp, global_conf, **kwargs )
>> > File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py",
>> > line 327, in wrap_in_static
>> > urlmap["/static"] = Static( conf.get( "static_dir" ), cache_time )
>> > File
>> >
>> > "/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/static.py",
>> > line 16, in __init__
>> > StaticURLParser.__init__( self, directory )
>> > File
>> >
>> > "/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py",
>> > line 430, in __init__
>> > self.directory = self.normpath(directory)
>> > File
>> >
>> > "/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py",
>> > line 435, in normpath
>> > return os.path.normcase(os.path.abspath(path))
>> > File "/usr/lib/python2.7/posixpath.py", line 343, in abspath
>> > if not isabs(path):
>> > File "/usr/lib/python2.7/posixpath.py", line 53, in isabs
>> > return s.startswith('/')
>> > AttributeError: 'NoneType' object has no attribute 'startswith'
>>
>
>
>
> --
>
> Christophe Antoniewski
>
>
> Drosophila Genetics and Epigenetics
> Laboratoire de Biolologie du Développement
> 9, Quai St Bernard, Boîte courrier 24
> 75252 Paris Cedex 05
>
> Tel +33 1 44 27 34 39
> Fax +33 1 44 27 34 45
> Mobile +33 6 68 60 51 50
>
> http://drosophile.org
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] where the log in information saved?
On Tue, Apr 16, 2013 at 5:34 AM, Björn Grüning wrote: > In the table 'galaxy_user' you will find your mail and password. Importantly, you will find a hash of the password, Galaxy does not store user's plaintext passwords. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] the release_2013.04.01 update crashes our galaxy server
This looks like just a configuration file problem. What is the value
of the various static_ options (like static_dir =) in your
universe_wsgi.ini?
On Mon, Apr 15, 2013 at 7:25 PM, Christophe Antoniewski
wrote:
> webapp = wrap_in_static( webapp, global_conf, **kwargs )
> File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py",
> line 327, in wrap_in_static
> urlmap["/static"] = Static( conf.get( "static_dir" ), cache_time )
> File
> "/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/static.py",
> line 16, in __init__
> StaticURLParser.__init__( self, directory )
> File
> "/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py",
> line 430, in __init__
> self.directory = self.normpath(directory)
> File
> "/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py",
> line 435, in normpath
> return os.path.normcase(os.path.abspath(path))
> File "/usr/lib/python2.7/posixpath.py", line 343, in abspath
> if not isabs(path):
> File "/usr/lib/python2.7/posixpath.py", line 53, in isabs
> return s.startswith('/')
> AttributeError: 'NoneType' object has no attribute 'startswith'
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trinity wrapper not working on local installation
I haven't looked at this directly, but it should be using None, in which case the correct comparison is "library_type is None" and "library_type is not None". -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, Mar 28, 2013 at 6:35 AM, Peter Cock wrote: > On Thu, Mar 28, 2013 at 4:36 AM, Raj Ayyampalayam wrote: >> Hello, >> >> I am trying to get the trinity wrapper to work on our local galaxy >> installation using the latest trinity version. >> >> The main issue is that the Trinity.pl call has --SS_lib_type None in the >> script file. The data I am using is unstranded paired end reads and I am >> selecting None in the tool parameters. >> I looked at the trinity_all.xml file and it seems like the following code is >> not working: >> >> #if $inputs.library_type != 'None': >> --SS_lib_type $inputs.library_type >> #end if >> >> I am new to cheeta and python and I am not sure why this code is not >> working. Any suggestion on how to go about debugging it? > > This is almost certainly a type comparison error, obscured by > the cheetah template language and the parameter proxy classes. > In Python there is a special object None, which is probably what > the library type is using. I would try making this an explicit > comparison of strings (a pattern used in many other wrappers, > e.g. tools/gatk/*.xml): > > #if str($inputs.library_type) != 'None': > --SS_lib_type $inputs.library_type > #end if > > Or, this might work too: > > #if $inputs.library_type != None: > --SS_lib_type $inputs.library_type > #end if > > (This does seem to be a bug in the trinity wrapper) > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Circster Visualization
I think if you save your trackster visualization first then switching over to circster should work fine. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, Mar 27, 2013 at 10:04 AM, Merella Stefania wrote: > Hi Jeremy, > thanks for replying. > > When I click on Circster visualization the browser ask me "Are you sure you > want to leave this page?" and when I click on "Leave this page" I receive > this error: > Visualization not found > > And this is from paster.log > > 172.25.46.193 - - [27/Mar/2013:15:02:57 +0200] "GET > /visualization/circster?id=undefined HTTP/1.1" 200 - > "http://172.21.156.12/visualization/trackster"; "Mozilla/5.0 (Macintosh; Intel > Mac OS X 10_8_3) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 > Safari/537.31" > > Thanks! > > On 27/mar/2013, at 14:53, Jeremy Goecks wrote: > >>> But if I click on it I receive an error. Obviously I get this error because >>> I didn't change any file to activate it. >> >> Unfortunately, this is not the case. If Trackster works, Circster should >> work. Can you share the error that you're getting when you click on the >> Circster icon? >> >> Thanks, >> J. >> > > -- > LA TUA CURA E' SCRITTA NEL TUO DNA. AL SAN RAFFAELE LA STIAMO REALIZZANDO. > AIUTA LA RICERCA, DAI IL TUO 5XMILLE - CF: 07636600962 > info:www.5xmi...@hsr.it - www.5xmille.org > > Disclaimer added by CodeTwo Exchange Rules 2007 > http://www.codetwo.com > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] slow history panel?
There have been some performance regressions in history display, we're working on this actively: https://trello.com/card/ui-refactoring-increase-speed-load-render-of-history-panel/506338ce32ae458f6d15e4b3/150 On Mon, Mar 25, 2013 at 5:06 PM, Langhorst, Brad wrote: > Is it my imagination or has the history panel become much slower in the > latest release? > > Maybe I have something misconfigured? > -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool config for accordian-like subpanels?
Hi Ted, this would fit well as another feature for the new section grouping construct I'm going to implement one of these days. There is a trello car if you want to add a comment, I'll update the spec. -- jt (composed on my phone) On Mar 13, 2013, at 3:01 PM, Ted Goldstein wrote: > I have a single tool with several different major modes. Each mode has its > own uniques group of options that can be managed separately. Rathan than make > five different tools, I would like to put it together and simply have the > user interface show just the options for the selected mode. People do this > in web pages sometimes using Jquery "Accordian" tabs. > > Is there any support to have something like Jquery accordion grouping of > options? see that the drill down , but it is only used for setting binary > options. > > Thanks,Ted > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Adding data sources in the main galaxy instance
Hi Ian, We've tried to make it as simple as possible to support communication with Galaxy. The protocols are described here: http://wiki.galaxyproject.org/Admin/Internals/Data%20Sources Note that page also links to a paper in the journal DATABASE which is entirely about data source integration. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Mar 12, 2013 at 8:11 AM, Fingerman, Ian (NIH/NLM/NCBI) [E] < ian.finger...@nih.gov> wrote: > I have a question regarding data sources currently available on the main > instance of Galaxy (https://main.g2.bx.psu.edu). How can one get added > as a new data source? I work at the NCBI on the Epigenomics database ( > http://www.ncbi.nlm.nih.gov/epigenomics) . We currently hold a large > volume of NGS data in the form of wig files, and some users have expressed > interest in using Galaxy for data analysis.We provide an easy to use > user interface for examining these data as seen here ( > http://www.ncbi.nlm.nih.gov/epigenomics/browse/) and currently host over > 4200 data tracks. > > ** ** > > I think it would useful if we could integrate Galaxy functionality into > our resource. I also thing providing our resource as a “Data Source” would > also be convenient for users less familiar with our database but are > current Galaxy users. > > ** ** > > I have been looking through the Galaxy wiki and I am struggling to find > documentation that details step-by-step, what exactly needs to be done. > One thing to note, I am not a developer, I’m the scientific lead for the > project and my programming/developing skills are lacking. I was hoping > someone could point me to thorough documentation mainly to pass on to > developers on my team. I guess also understanding my options with regards > to integrating or interfacing with Galaxy would be very valuable to me too. > > > ** ** > > Thank you for any help/suggestions you may have. > > ** ** > > Ian > > ** ** > > ** ** > > Ian Fingerman, Ph.D. > > Staff Scientist > > NIH/NLM/NCBI > > Building 45, Room 4AN28D-29 > > 45 Center Drive MSC-6510 > > Bethesda, MD 20894 > > ** ** > > Phone: (301) 496-6806 > > finge...@ncbi.nlm.nih.gov > > ** ** > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy on Cluster - how to set -a flag with username
Greg, I don't think there is currently any support for variable substitution of this sort in job runner configuration, but it could definitely be added. Worth filing as an issue. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, Mar 13, 2013 at 8:12 AM, greg wrote: > Hi guys, > > I just wanted to check in and see if anyone had any ideas on this. > (It's actually the -A flag I'm after.) > > I did try putting this in universe.wsgi: > > default_cluster_job_runner = drmaa://-V -pe batch 8 -A Account_Name/ > > I was thinking galaxy might substitute the user name for > "Account_Name" but no luck :-( > > -Greg > > On Tue, Mar 12, 2013 at 1:42 PM, greg wrote: > > In our local galaxy install we want the cluster jobs to be run from > > the galaxy user but we want to include a -a [account name] to our grid > > software bills properly. > > > > Here's what I currently have in universe.wsgi: > > > > default_cluster_job_runner = drmaa://-V -pe batch 8/ > > > > What I want is something like this: > > > > default_cluster_job_runner = drmaa://-V -pe batch 8 -a [logged in user > name]/ > > > > Is this possible? > > > > Thanks, > > > > Greg > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Composite datatype output for Cuffdiff
> I understand that instead of having one dataset with multiple files you are > planning to use existing datasets and combine them in a ‘collection’. My > concerns are: This needs to be fleshed out much more, but this is not exactly what we are thinking. The main change is to make it possible for a history to contain items other than datasets. Groups of datasets would be one such thing. Multifile datasets would be another. Workflow invocations a third (needed to support extensions to the workflow system we are proposing). > 1. Our data consists of 200-8000 files, can you imagine how many datasets > we’ll end up with? It will be a mess. Yes, it would, which is why there does need to be the concept of a homogenous dataset collection to support this. > 5. We are already using the “m:xxx” type datasets (thanks John) in our > project, I guess you don’t even have a timeframe for implementing the > “collection” concept? I’m sure that for many projects using multi file > datasets is a requirement now, not in ‘years’ time. We recognize the need, but implementing these using the existing datasets with a prefix on the extension, and then special casing all over the place, is not a maintainable solution going forward. They should be implemented as their own entity. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Validating dynamic inputs stream of consciousness
On Wed, Mar 6, 2013 at 12:22 PM, John Chilton wrote: > This whole concept puts a lot of onus on the tool developer. A > biologist who has taken a two week course on perl could probably write > a Galaxy tool, they probably couldn't write a secure tool for a public > LWR. I think some experience in thinking about how to secure web > accessible applications and prevent injection style attacks is needed. > I will update the documentation urging additional caution with respect > to this. What I'm trying to understand is whether this model for a public LWR makes sense. It appears that your LWR will take a command line, and then apply a series of validations to it. This set of validations would need to be very comprehensive -- perhaps impossibly comprehensive -- to be secure. To me it would make more sense that the LWR takes an input values dict and constructs the command line itself after validating everything (it already has the toolbox, so this should be possible). > That said, there have been in the recent past multiple tools on public > Galaxy servers (main included) that were developed by serious > programmers that allowed arbitrary code execution. This is something > the whole community (or at least that subset hosting public servers) > needs to address and take more seriously. I could not agree more, though I see this as a somewhat different issue. There are a number of things that would be really helpful here: - Some automatic validation of command line construction to look for common exploits (again, impossible to do comprehensively) - Some kind of sandboxing, through support for chroot, zones, jails, or (dare I dream) running under native client. > there is some malicious command that could get through. I cannot > guarantee that it is secure, but I would be eager for counter examples > or specific issues I can address. I think this is what concerned me as well. I'm always worried about security through comprehensive screening, someone almost always finds a way around it. This is why the original python sandbox failed. Constructing the command line from validated inputs seems safer (as long as you trust the template that builds the command line). > If we are honest and accept that there are going to security problems > with the tools we wrap, one idea that might be worth pursuing for both > the LWR and Galaxy itself is running tools in chrooted environments or > at least as a different user then the webapp. On this we completely agree. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Broken pipe in server
Broken pipe normally occurs when the browser drops the connection
(e.g. if you quit your browser while downloading a dataset you might
see this). Can you describe in more detail how the "frequent hiccups"
manifest?
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Mon, Mar 4, 2013 at 12:36 PM, Thomas van Gurp
wrote:
> Dear all,
>
> Our galaxy suffers from frequent hickups, which apparently are due to broken
> pipes.
>
> It will give these errors in paster.log
>
>
> Exception happened during processing of request from ('127.0.0.1', 53234)
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
> line 1068, in process_request_in_thread
> self.finish_request(request, client_address)
> File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
> self.RequestHandlerClass(request, client_address, self)
> File "/usr/lib/python2.7/SocketServer.py", line 641, in __init__
> self.finish()
> File "/usr/lib/python2.7/SocketServer.py", line 694, in finish
> self.wfile.flush()
> File "/usr/lib/python2.7/socket.py", line 303, in flush
> self._sock.sendall(view[write_offset:write_offset+buffer_size])
> error: [Errno 32] Broken pipe
>
>
>
> Any idea why this might be an what I would need to check?
>
> Cheers,
> THomas
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Authentication: email addresses are not validated prior to account creation
repoze.who would seem like the best candidate these days, it would be great to see that integrated, but I worry it would also cause lots of unintentional breakage in the corner cases. On Mar 5, 2013 12:27 PM, "Paul Boddie" wrote: > On 05/03/13 17:09, James Taylor wrote: > >> On Mar 1, 2013, at 10:39 AM, Vipin TS wrote: >> >> Hello members, >>> >>> I believe currently there is no process to validate email address >>> provided during user account creation. We are experiencing a huge fake >>> account creation attack on our public facing galaxy instance. >>> >>> Does anybody who has been managing a public instance, implemented an >>> on-demand account creation activation by sending an email containing a >>> link, which when clicked, validate the account creation request. Or any >>> plans from dev-team to add this in future release? >>> >> How about some kind of captcha support? >> > > Recently, there has been increased awareness of some of the pitfalls > involved in managing identity and authentication-related information in > Python-based applications - not specifically to do with Python itself, but > more to do with the community and the perceived best practices - and I'd > really like to see a bit more collaboration around those things as well as > around anti-spam mechanisms. Having looked at the authentication aspects of > Galaxy, I can't help wondering if there shouldn't be some kind of generic > "shell" for such functionality that is separate from the core functionality > of Galaxy and would be used for other systems as well. Certainly, using > Apache is one solution, but people do seem to want a more controlled kind > of integration between that and the underlying applications. > > At the very least, one would hope to reuse and integrate existing > components, perhaps at the WSGI level. Failing that, there might be some > generic libraries that could support such reusable components. Perhaps the > most significant challenge would be to cleanly integrate the user interface > aspects of such components with the Galaxy output. > > Certainly, one could just extend the registration mechanism with captcha > support, but I'd be worried about the maintainability of the code. Unless > things have progressed fairly recently, there was already a lot of > special-cased stuff in the area of authentication, and I'd be worried about > unintentional breakage. > > Paul > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Authentication: email addresses are not validated prior to account creation
How about some kind of captcha support? -- jt (composed on my phone) On Mar 1, 2013, at 10:39 AM, Vipin TS wrote: > Hello members, > > I believe currently there is no process to validate email address provided > during user account creation. We are experiencing a huge fake account > creation attack on our public facing galaxy instance. > > Does anybody who has been managing a public instance, implemented an > on-demand account creation activation by sending an email containing a link, > which when clicked, validate the account creation request. Or any plans from > dev-team to add this in future release? > > thanks in advance, > --/Vipin > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool wrappers - Triggering warnings on specific options?
Peter, I quite like this idea, and I think it might be possible to do as a simple extension to the validator framework. Basically, a flag indicating that failing a certain validation would be able to be considered a warning rather than an error. Or, allow a validator to return a sentinel indicating warning vs error. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, Feb 20, 2013 at 11:08 AM, Peter Cock wrote: > Hello all, > > I was wondering if anyone else writing Galaxy tools has > wanted to be able to show warning messages conditional > on tool options? > > For example, setting a parameter to a deprecated value > could trigger a warning message (with an OK/Cancel > dialogue to allow continuation or returning to the tool > options). More generally, a warning might be desirable > only if certain options are used together. > > Even stricter, for some tools it would be useful to show > an error message with particular option combinations, > and refuse to submit the job. > > Any thoughts? > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] ASN.1 (text and binary) formats in Galaxy & Tool Shed
Clarity in this case, definitely needs the 1. On Feb 19, 2013, at 3:38 PM, Peter Cock wrote: > On Tue, Feb 19, 2013 at 6:45 PM, Nicola Soranzo wrote: >> Il giorno mar, 19/02/2013 alle 14.15 +, Peter Cock ha scritto: >>> On Tue, Feb 19, 2013 at 2:00 PM, James Taylor wrote: >>>> On Tue, Feb 19, 2013 at 6:32 AM, Peter Cock wrote: >>>>> I think it could make sense to define generic 'asn1' and >>>>> 'asn1-binary' formats in the Galaxy core (name suggestions >>>>> welcome) >> >> What about >> >> extension="asn" type="galaxy.datatypes.data:GenericAsn1" >> >> and >> >> extension="asnb" type="galaxy.datatypes.binary:GenericAsn1Binary" >> >> like GenericXml class? > > Those seem sensible to me as the class names, although I'm > not so sure about the format names (aka 'extensions' in Galaxy > terms). I'd prefer to see the '1' in the name for clarity. My > suggestions of 'asn1' and 'asn1-binary' were based on NCBI usage. > > Perhaps the Galaxy team could comment on their views here > for conciseness versus clarity in file format names for Galaxy? > >>>>> Would a pull request implementing this be acceptable? >>>> >>>> Yes. My understanding is that ASN is a completely flexible metaformat, >>>> like XML, and so should be under either Text or Data, with appropriate >>>> subtypes defined for blast, et cetera. >>> >>> Thank James, >>> >>> Yes - very like XML, but with the subtlety that ASN.1 comes in text and >>> binary favours (which I presume applies to all the variants, although >>> the binary versions may not be as commonly used for the smaller files). >>> >>> Nicola - do you want to make a pull request to galaxy-central defining >>> ASN.1 text and binary formats (which we can then subclass for the >>> NCBI BLAST+ wrappers)? Or should I? >> >> If you mean a minimal implementation, I can surely do that. If something >> more elaborated is needed, then probably you are more qualified than me! > > The current minimal implementation you sent me for BLAST+ > would be an excellent start. Things like data type sniffers etc > would be a nice to have feature, but can be added later I think. > And the sooner this gets into the Galaxy core, the sooner we > can use it in the BLAST+ wrappers :) > >>> I think the mime-type for the base ASN.1 text format should probably >>> be text/plain based on the NCBI usage patterns. >> >> Ok. >> >>> I'm not sure what the mime-type for the base ASN.1 binary format >>> should be (but I don't think it should be chemical/ncbi-asn1-binary). >> >> application/octet-stream ? > > Probably OK - we can/should test this by uploading a test > binary ASN.1 file into Galaxy and downloading it out again. > > Thanks, > > Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool wrapper XSD
On Tue, Feb 19, 2013 at 3:39 AM, Pierre Pericard wrote: > Is there a Galaxy XML tool wrapper XSD ? No, but if anyone would like to make such a schema we would be happy to include it. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] ASN.1 (text and binary) formats in Galaxy & Tool Shed
On Tue, Feb 19, 2013 at 6:32 AM, Peter Cock wrote: > Would a pull request implementing this be acceptable? Yes. My understanding is that ASN is a completely flexible metaformat, like XML, and so should be under either Text or Data, with appropriate subtypes defined for blast, et cetera. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problem using NGS: Peak Calling - MACS
You likely want the newest version (3) On Feb 15, 2013 9:37 AM, "greg" wrote: > Thanks! So one question, when I go to install it from the toolshed I > see two choices for a revision to select: > > 3:cdd9791c0afa > 0:c1eeccec29d1 > > Which would I choose and why? > > Thanks, > > Greg > > On Fri, Feb 15, 2013 at 9:30 AM, James Taylor > wrote: > > The wrapper you are using is for MACS 1.3, there is a wrapper that is > > compatible with 1.4 in the toolshed. > > > > -- > > James Taylor, Assistant Professor, Biology/CS, Emory University > > > > > > On Fri, Feb 15, 2013 at 9:26 AM, greg wrote: > >> Hi guys, > >> > >> I'm trying to run MACS in my local galaxy install (see screenshot: > >> http://snag.gy/5QGUH.jpg) > >> > >> And I'm getting this error: > >> > >> Usage: macs <-t tfile> [-n name] [-g genomesize] [options] > >> > >> Example: macs -t ChIP.bam -c Control.bam -f BAM -g h -n test -w > --call-subpeaks > >> > >> macs: error: no such option: --lambdaset > >> > >> Thanks in advance for any help. > >> > >> -Greg > >> > >> Here's how I installed Macs (I guess it doesn't come built in?) > >> > >>> cd /usr/local/galaxy/downloads/ > >>> wget https://github.com/downloads/taoliu/MACS/MACS-1.4.2-1.tar.gz > >>> tar -xzvf MACS-1.4.2-1.tar.gz > >>> cd MACS-1.4.2 > >>> source /usr/local/galaxy/job_environment_setup_file > >>> python setup.py install > >>> deactivate > >>> cd /misc/local/galaxy/galaxy_python/bin > >>> ln -s macs14 macs > >> > >> (and made sure /misc/local/galaxy/galaxy_python/bin is on the galaxy > >> user's path) > >> ___ > >> Please keep all replies on the list by using "reply all" > >> in your mail client. To manage your subscriptions to this > >> and other Galaxy lists, please use the interface at: > >> > >> http://lists.bx.psu.edu/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problem using NGS: Peak Calling - MACS
The wrapper you are using is for MACS 1.3, there is a wrapper that is compatible with 1.4 in the toolshed. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Fri, Feb 15, 2013 at 9:26 AM, greg wrote: > Hi guys, > > I'm trying to run MACS in my local galaxy install (see screenshot: > http://snag.gy/5QGUH.jpg) > > And I'm getting this error: > > Usage: macs <-t tfile> [-n name] [-g genomesize] [options] > > Example: macs -t ChIP.bam -c Control.bam -f BAM -g h -n test -w > --call-subpeaks > > macs: error: no such option: --lambdaset > > Thanks in advance for any help. > > -Greg > > Here's how I installed Macs (I guess it doesn't come built in?) > >> cd /usr/local/galaxy/downloads/ >> wget https://github.com/downloads/taoliu/MACS/MACS-1.4.2-1.tar.gz >> tar -xzvf MACS-1.4.2-1.tar.gz >> cd MACS-1.4.2 >> source /usr/local/galaxy/job_environment_setup_file >> python setup.py install >> deactivate >> cd /misc/local/galaxy/galaxy_python/bin >> ln -s macs14 macs > > (and made sure /misc/local/galaxy/galaxy_python/bin is on the galaxy > user's path) > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Reloading a tools configuration does not seem to actually work
On Thu, Feb 14, 2013 at 10:40 AM, Nate Coraor wrote: >> What I am facing, is that I have 100 FASTQ pairs or so, for a single >> flowcell, I can start the analysis of that set from the UI, but it will just >> crank through them and takes about 2-4 minutues for each pair to be >> processed, so with 100 pairs or so, you are looking 3-4 hours of an >> hourglass before control is given back to the user... > > Is this in the context of starting a workflow with hundreds of inputs? I believe this is entirely in the web process, creating a workflow requires creating tons of datasets, which requires tons of database flushes. I have a patch to fix this from a long time ago but it would need substantial testing. I think we should also background the process, but this depends on having a way to attach the workflow invocation to the history, hence dataset groups. All the pieces are coming together on this. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Upload file : auto-detect based on file extension ?
I don't believe you can, although it should be possible to extend
upload to provide that information. However, is there no header you
can use in your filetype to detect it?
--
James Taylor, Assistant Professor, Biology/CS, Emory University
On Mon, Feb 11, 2013 at 10:51 PM, David Angot wrote:
> Hi,
>
> We are using a proprietary file format in some of our tools.
> I successfully added a new data type, but what I would like to do is to use
> the auto-detect when uploading the file, just based on the extension of the
> file.
>
> My guess is I have to override the sniff() in the datatype class, and test
> for the extension ? Somethinkg like that :
>
> if file.endswith('.extension123'):
> ...
>
> But how do I get the original filename when it was uploaded ?
>
> Thanks,
>
> --
> David
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Can Galaxy run on Hadoop system?
Hi, Because hadoop is a different computing approach, all the underlying tools would need to be written for hadoop as well, so currently Galaxy would not be very useful with only a hadoop cluster as a backend. However work is underway to allow hadoop jobs to be run alongside other jobs from a Galaxy frontend. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, Feb 6, 2013 at 9:51 AM, lukeyoyo wrote: > Hi, > > I am a computer engineer and have to deploy a cloud computing system, like > Hadoop. I want to deploy a local Galaxy system on the Hadoop platform. I > know that Galaxy can run on Cloud such as EC2 from Amazon. > > My question is: can Galaxy run on Hadoop? (Not running within a virtual > mechine, but interacting with multiple slave nodes in Hadoop) > If NOT, do you have any plan to migrate it to Hadoop platform? > > Best regards. > > > lukeyoyo > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy assumed linux shell
Command lines inside tool configs are assumed to be 'sh' (posix bourne shell). If you are extending one of the runners, definitely stick to pure 'sh'. However, once your command is run, there is no reason you can't use any shell of your choice. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Sun, Feb 3, 2013 at 11:56 AM, Hagai Cohen wrote: > Hi, > There are couple of Linux shells: bash, sh, zsh, csh. etc > I would like to know if Galaxy is written to be supported on a specific one > or it is written to support any linux shell? > > I'm asking this, because I want to add some features and I want to know what > shell should I support? > Or should write cross-shell code? > > In Dramaa runner, for example, it uses sh shell (inside drm_template inside > drmaa.py), so any job that is run through this runner should support sh. > > Thanks, > Hagai > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] How do I allow a large group of users to RUN my workflow without them having to make a local copy
> It seems strange not to be able to share workflows with groups if you can > share them with individual users.. The group/role system for data libraries and the sharing system for other assets were developed concurrently. It would be reasonable (and probably not that difficult) to add groups to the sharing system. Feel free to create an issue for this, a pull request would certainly be welcome. > I also have to know the exact username (email) for each user, rather than > being able to choose it from a list... This is deliberate, we only expose the full user list to administrators, we would consider it a privacy issue to expose the email address of every Galaxy user to every other user. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error loading a BAM file -- dataset needs grooming
I believe Galaxy requires samtools to be installed to work with bam data. If you install samtools in a location Galaxy can see it then this error should no longer occur. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Fri, Jan 25, 2013 at 11:11 AM, Maureen J. Donlin wrote: > Hi, > > I tried to load a BAM file into our local instance of Galaxy but get the > following error. I had preloaded the genome to which it was aligned and the > bai file. Do I have to load these using a different method and the file > upload? > > Thanks, > Maureen > > Traceback error: > > Traceback (most recent call last): > File "/work/apps/galaxy/tools/data_source/upload.py", line 384, in > > __main__() > File "/work/apps/galaxy/tools/data_source/upload.py", line 373, in > __main__ > add_file( dataset, registry, json_file, output_path ) > File "/work/apps/galaxy/tools/data_source/upload.py", line 312, in > add_file > if link_data_only == 'copy_files' and > datatype.dataset_content_needs_grooming( output_path ): > File "/work/apps/galaxy/lib/galaxy/datatypes/binary.py", line 113, in > dataset_content_needs_grooming > version = self._get_samtools_version() > File "/work/apps/galaxy/lib/galaxy/datatypes/binary.py", line 97, in > _get_samtools_version > output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, > stdout=subprocess.PIPE ).communicate()[1] > File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__ > errread, errwrite) > File "/usr/lib64/python2.6/subprocess.py", line 1228, in _execute_child > raise child_exception > OSError: [Errno 2] No such file or directory > > > > -- > Maureen J. Donlin, Ph.D. > Research Associate Professor > Dept. of Biochemistry & Molecular Biology > Dept. of Molecular Microbiology & Immunology > Saint Louis University School of Medicine > 507 Doisy Research Center > Office: 314-977-8858 > Cell: 314-750-2345 > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] maximum recursion depth exceeded while calling a Python object
Can you send some additional lines of the error? The important part is to see what part of the stack is repeating. Based on what you have it suggests recursion in trying to do a datatype conversion. Have you modified datatypes_conf.xml and is it possible there is a problem there? Thanks! -- James Taylor, Assistant Professor, Biology/CS, Emory University On Fri, Jan 25, 2013 at 2:47 PM, Liisa Koski wrote: > Hi, > I'm running a local instance of Galaxy and no matter what tool I run I get > the following error: > > Error executing tool: maximum recursion depth exceeded while calling a > Python object > > In the log file: > > galaxy.tools ERROR 2013-01-25 14:06:54,375 Exception caught while attempting > tool execution: > Traceback (most recent call last): > File "/Galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 1776, in > handle_input > _, out_data = self.execute( trans, incoming=params, history=history ) > File "/Galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 2103, in > execute > return self.tool_action.execute( self, trans, incoming=incoming, > set_output_hid=set_output_hid, history=history, **kwargs ) > File "/Galaxy/galaxy_dist/lib/galaxy/tools/actions/__init__.py", line 203, > in execute > chrom_info = build_fasta_dataset.get_converted_dataset( trans, 'len' > ).file_name > File "/Galaxy/galaxy_dist/lib/galaxy/model/__init__.py", line 1161, in > get_converted_dataset > new_dataset = self.datatype.convert_dataset( trans, self, target_ext, > return_output=True, visible=False, deps=deps, set_output_history=False > ).values()[0] > File "/Galaxy/galaxy_dist/lib/galaxy/datatypes/data.py", line 467, in > convert_dataset > converted_dataset = converter.execute( trans, incoming=params, > set_output_hid=visible, set_output_history=set_output_history)[1] > File "/Galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 2103, in > execute > return self.tool_action.execute( self, trans, incoming=incoming, > set_output_hid=set_output_hid, history=history, **kwargs ) > File "/Galaxy/galaxy_dist/lib/galaxy/tools/actions/__init__.py", line 203, > in execute > chrom_info = build_fasta_dataset.get_converted_dataset( trans, 'len' > ).file_name > File "/Galaxy/galaxy_dist/lib/galaxy/model/__init__.py", line 1161, in > get_converted_dataset > new_dataset = self.datatype.convert_dataset( trans, self, target_ext, > return_output=True, visible=False, deps=deps, set_output_history=False > ).values()[0] > File "/Galaxy/galaxy_dist/lib/galaxy/datatypes/data.py", line 467, in > convert_dataset > converted_dataset = converter.execute( trans, incoming=params, > set_output_hid=visible, set_output_history=set_output_history)[1] > File "/Galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 2103, in > execute > return self.tool_action.execute( self, trans, incoming=incoming, > set_output_hid=set_output_hid, history=history, **kwargs ) > File "/Galaxy/galaxy_dist/lib/galaxy/tools/actions/__init__.py", line 203, > in execute > chrom_info = build_fasta_dataset.get_converted_dataset( trans, 'len' > ).file_name > File "/Galaxy/galaxy_dist/lib/galaxy/model/__init__.py", line 1161, in > get_converted_dataset > new_dataset = self.datatype.convert_dataset( trans, self, target_ext, > return_output=True, visible=False, deps=deps, set_output_history=False > ).values()[0] > File "/Galaxy/galaxy_dist/lib/galaxy/datatypes/data.py", line 467, in > convert_dataset > converted_dataset = converter.execute( trans, incoming=params, > set_output_hid=visible, set_output_history=set_output_history)[1] > File "/Galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 2103, in > execute > return self.tool_action.execute( self, trans, incoming=incoming, > set_output_hid=set_output_hid, history=history, **kwargs ) > > thousands of lines here...ending with... > > File > "/Galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/schema.py", > line 760, in _make_proxy > selectable.columns.add(c) > File > "/Galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/sql/expression.py", > line 1668, in add > self[column.key] = column > File > "/Galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/sql/expression.py", > line 1671, in __setitem__ > if key in self: > File > "/Galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/sql/expression.py", > line 1702, in __contains__ > return util.OrderedProperties.__cont
Re: [galaxy-dev] rerun tool in a workflow
Not currently, however we are planning to implement this in the future. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Sun, Jan 27, 2013 at 10:20 AM, Hagai Cohen wrote: > Hi, > is it possible that when I rerun a tool it will also rerun the tools that > depends on it? > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] How to call a cgi web page in a galaxy folder from a script?
I'm not sure I fully understand the question, so it may help to clarify. However, while Galaxy is a web application which provides an HTTP interface, it is not a general web server and cannot serve CGI scripts. You might be able to this either with a proxy server in front of Galaxy or by making your script a Galaxy tool, but it depends on what you mean by "how will I call it inside a script in galaxy?". -- James Taylor, Assistant Professor, Biology/CS, Emory University On Sun, Jan 27, 2013 at 2:23 PM, Zinonas Antoniou wrote: > Hello, > > Now I have a web page (cgi) in a server like www.example.com/myexample.py > > I want to have the myexample.py file under a galaxy folder. Then, how will I > call it inside a script in galaxy? > > > Thank you in advance, > Zinon > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] show urls in csv (in eye mode)
You may want to be careful with this solution because it allows script injection, any HTML including javascript in the uploaded tabular file will be executed by the .html call, with complete access to everything in the Galaxy browser window (like your session cookies). -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jan 29, 2013 at 7:18 AM, Hagai Cohen wrote: > Hi, > I found a simple solution. > > In the file: > templates/dataset/tabular_chunked.mako > inside the method: renderCell - replace all jQuery .text(..) calls with > .html(..) calls. > > Note: > Any entry in the csv with html syntax will be shown like html (links, tags. > etc). > > > > > > > On Tue, Jan 29, 2013 at 1:39 PM, Hagai Cohen wrote: >> >> Hi, >> I have a tool which outputs a csv file with links. >> When I press the eye icon - I would like these links to be shown as links >> in the browser. >> It is possible? >> >> Thanks, >> Hagai > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Nice 'citable' URLs for Galaxy Tool Shed repositories
On Tue, Jan 29, 2013 at 11:44 AM, Peter Cock wrote: > Presumably at this point changing the name in the code would > break too many things? Actually, changing the package name would probably be a pretty clean refactoring, just some changes in buildapp. But given how many people ar actively working on that code it might be best to wait, merging lots of code on top of renames can get messy. -- James Taylor, Assistant Professor, Biology/CS, Emory University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Different behavior of select tag in the tool configuration file when the number of options is different.
I have a patch to replace the searchable select implementation with select2 which seems to work well, but still needs comprehensive testing. I think it may address this problem. If anyone is interested in trying it: https://bitbucket.org/james_taylor/james-galaxy-central-patches/src/b70fa48058f60c88dff0f29a084ee8b0876d033a/select2-for-big-selects -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, Jan 30, 2013 at 4:45 AM, Peter Cock wrote: > On Wed, Jan 30, 2013 at 3:01 AM, Luobin Yang wrote: >> Hi, >> >> I have a 'select' tag in a tool's XML file, ... when the number of options >> exceeds 20, then drop down menu of the web interface displays "Click to >> Select or Search" ... Why the behavior changed when there are 20 options >> or more for a drop down menu? >> >> Thanks, >> Luobin > > The drop down list becomes very hard to use with lots of entries (for > example we have lots of local BLAST databases offered to the user), > and in that situation the search works really well. > > So for most cases, the search system is much more user friendly. > > You have a special case wanting a blank entry in the list, there may > be a better way to handle that. One idea might be to use a caption > like instead of an empty string? > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error trying to add tools from tool shed respository
Wasn't sure of the order, those are definitely after running the update. Dannon's advice makes a good point, upgrading to the latest mercurial should help. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jan 29, 2013 at 11:10 AM, Waldron, Michael H wrote: > Already in the email thread. > >> [galaxy@galaxy galaxy-dist]$ hg heads >> changeset: 8677:22788c1262a2 >> tag: tip >> user:jeremy goecks >> date:Mon Jan 28 16:36:02 2013 -0500 >> summary: Language improvements for tophat wrappers. >> >> changeset: 7987:966a9b393b85 >> branch: add_requirement_tags_for_plink >> parent: 7472:06b31a61793c >> user:Dannon Baker >> date:Mon Aug 13 13:23:54 2012 -0400 >> summary: Branch close >> >> changeset: 7152:ffc20445b35e >> branch: split_blast2 >> parent: 7058:22f545c0d412 >> user:Dannon Baker >> date:Wed Apr 18 07:59:38 2012 -0400 >> summary: Branch close. >> >> changeset: 7151:c94957ac5454 >> branch: workflow_import_menu_api >> parent: 6834:1a3d01b09f7a >> user:Dannon Baker >> date:Wed Mar 21 09:50:44 2012 -0400 >> summary: Branch close for pull request. >> >> changeset: 7150:314a10ae112d >> branch: split_blast >> parent: 6757:ebf65c0b1e26 >> user:Dannon Baker >> date:Tue Feb 28 12:32:54 2012 -0500 >> summary: Close split_blast branch >> >> changeset: 7149:1327f2219423 >> branch: meme_requirement_on_fimo >> parent: 6662:8ba74c17e809 >> user:Daniel Blankenberg >> date:Fri Feb 03 11:15:29 2012 -0500 >> summary: close branch >> > > Mike Waldron > Systems Specialist > ITS - Research Computing Center > University of North Carolina at Chapel Hill > > > > From: ja...@taylorlab.org [ja...@taylorlab.org] on behalf of James Taylor > [ja...@jamestaylor.org] > Sent: Tuesday, January 29, 2013 11:04 AM > To: Waldron, Michael H > Cc: Greg Von Kuster; galaxy-...@bx.psu.edu > Subject: Re: [galaxy-dev] Error trying to add tools from tool shed respository > > Can you send the output of hg heads after having run the pull -u ? > > Thanks! > > -- > James Taylor, Assistant Professor, Biology/CS, Emory University > > > On Tue, Jan 29, 2013 at 9:57 AM, Waldron, Michael H > wrote: >> To be honest, I barely understand this enough to just follow the wiki >> instructions. When it doesn't work as expected, I'm lost. >> >> I did go back to the output from when I ran the update, and just noticed >> something about not updating because of new heads. What does this mean? >> >> [galaxy@galaxy galaxy-dist]$ hg pull -u >> pulling from https://bitbucket.org/galaxy/galaxy-central >> searching for changes >> adding changesets >> adding manifests >> adding file changes >> added 691 changesets with 2002 changes to 603 files (+1 heads) >> not updating, since new heads added >> (run 'hg heads' to see heads, 'hg merge' to merge) >> >> >> >> Mike Waldron >> Systems Specialist >> ITS - Research Computing Center >> University of North Carolina at Chapel Hill >> >> >> From: Greg Von Kuster [g...@bx.psu.edu] >> Sent: Tuesday, January 29, 2013 9:52 AM >> >> To: Waldron, Michael H >> Cc: galaxy-...@bx.psu.edu >> Subject: Re: [galaxy-dev] Error trying to add tools from tool shed >> respository >> >> Hi Michael, >> >> Your paster log still is showing the older code. Any idea how this happens >> in your environment? >> >> On Jan 29, 2013, at 9:46 AM, Waldron, Michael H wrote: >> >> >> I had stopped the Galaxy server before running the update, then started it >> afterward. >> >> >> Mike Waldron >> Systems Specialist >> ITS - Research Computing Center >> University of North Carolina at Chapel Hill >> >> >> From: Greg Von Kuster [g...@bx.psu.edu] >> Sent: Tuesday, January 29, 2013 9:12 AM >> To: Waldron, Michael H >> Cc: galaxy-...@bx.psu.edu >> Subject: Re: [galaxy-dev] Error trying to add tools from tool shed >> respository >> >> Hi Michael, >> >> Did you restart your Galaxy server after updating from Galaxy central? Your >> paster log shows the following: >> >> File >> '/nas02/apps/galaxy-pr
Re: [galaxy-dev] Error trying to add tools from tool shed respository
Can you send the output of hg heads after having run the pull -u ? Thanks! -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jan 29, 2013 at 9:57 AM, Waldron, Michael H wrote: > To be honest, I barely understand this enough to just follow the wiki > instructions. When it doesn't work as expected, I'm lost. > > I did go back to the output from when I ran the update, and just noticed > something about not updating because of new heads. What does this mean? > > [galaxy@galaxy galaxy-dist]$ hg pull -u > pulling from https://bitbucket.org/galaxy/galaxy-central > searching for changes > adding changesets > adding manifests > adding file changes > added 691 changesets with 2002 changes to 603 files (+1 heads) > not updating, since new heads added > (run 'hg heads' to see heads, 'hg merge' to merge) > > > > Mike Waldron > Systems Specialist > ITS - Research Computing Center > University of North Carolina at Chapel Hill > > > From: Greg Von Kuster [g...@bx.psu.edu] > Sent: Tuesday, January 29, 2013 9:52 AM > > To: Waldron, Michael H > Cc: galaxy-...@bx.psu.edu > Subject: Re: [galaxy-dev] Error trying to add tools from tool shed > respository > > Hi Michael, > > Your paster log still is showing the older code. Any idea how this happens > in your environment? > > On Jan 29, 2013, at 9:46 AM, Waldron, Michael H wrote: > > > I had stopped the Galaxy server before running the update, then started it > afterward. > > > Mike Waldron > Systems Specialist > ITS - Research Computing Center > University of North Carolina at Chapel Hill > > > From: Greg Von Kuster [g...@bx.psu.edu] > Sent: Tuesday, January 29, 2013 9:12 AM > To: Waldron, Michael H > Cc: galaxy-...@bx.psu.edu > Subject: Re: [galaxy-dev] Error trying to add tools from tool shed > respository > > Hi Michael, > > Did you restart your Galaxy server after updating from Galaxy central? Your > paster log shows the following: > > File > '/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', > line 1201 in prepare_for_install > > description, repository_clone_url, changeset_revision, ctx_rev, > repository_owner, tool_dependencies = repo_info_tuple > > However, that line no longer exists in the revision to which you've updated, > and line 1201 of the admin_toolshed.py file is now: > > status = kwd.get( 'status', 'done' ) > > Greg Von Kuster > > > On Jan 29, 2013, at 9:03 AM, Waldron, Michael H wrote: > > Here is the output from hg heads. > > [galaxy@galaxy galaxy-dist]$ hg heads > changeset: 8677:22788c1262a2 > tag: tip > user:jeremy goecks > date:Mon Jan 28 16:36:02 2013 -0500 > summary: Language improvements for tophat wrappers. > > changeset: 7987:966a9b393b85 > branch: add_requirement_tags_for_plink > parent: 7472:06b31a61793c > user:Dannon Baker > date:Mon Aug 13 13:23:54 2012 -0400 > summary: Branch close > > changeset: 7152:ffc20445b35e > branch: split_blast2 > parent: 7058:22f545c0d412 > user:Dannon Baker > date:Wed Apr 18 07:59:38 2012 -0400 > summary: Branch close. > > changeset: 7151:c94957ac5454 > branch: workflow_import_menu_api > parent: 6834:1a3d01b09f7a > user:Dannon Baker > date:Wed Mar 21 09:50:44 2012 -0400 > summary: Branch close for pull request. > > changeset: 7150:314a10ae112d > branch: split_blast > parent: 6757:ebf65c0b1e26 > user:Dannon Baker > date:Tue Feb 28 12:32:54 2012 -0500 > summary: Close split_blast branch > > changeset: 7149:1327f2219423 > branch: meme_requirement_on_fimo > parent: 6662:8ba74c17e809 > user:Daniel Blankenberg > date:Fri Feb 03 11:15:29 2012 -0500 > summary: close branch > > > Mike Waldron > Systems Specialist > ITS - Research Computing Center > University of North Carolina at Chapel Hill > > > From: Greg Von Kuster [g...@bx.psu.edu] > Sent: Tuesday, January 29, 2013 9:00 AM > To: Waldron, Michael H > Cc: galaxy-...@bx.psu.edu > Subject: Re: [galaxy-dev] Error trying to add tools from tool shed > respository > > Hi Michael, > > The previous post you mention is outdated. The main Galaxy tool shed from > which you are pulling has been tracking Galaxy dist for some time now, so > you should only have to update to that. > > However, the communication layer between Galaxy
Re: [galaxy-dev] workflow Editor
The main things you want to look at are: static/scripts/galaxy.workflow_editor.canvas.js templates/workflow/editor.mako -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jan 29, 2013 at 9:37 AM, sshashi wrote: > Hi James, > if i want to create editor only and tools are not required , can i get > some documents related with galaxy editor . you can send me module name > in galaxy . i can go through for creating editor . > Regards > shashi > > On Mon, 2013-01-28 at 10:55 -0500, James Taylor wrote: >> The client side of the workflow editor is in javascript, using a mix >> of HTML5 Canvas and DOM elements. It communicates with the server >> using JSON. The server side is written in Python. The editor could be >> decoupled and reused, however there are some details that are fairly >> Galaxy specific right now (particularly how tool states -- the values >> of the various input parameters -- are encoded). >> >> -- >> James Taylor, Assistant Professor, Biology/CS, Emory University >> >> >> On Mon, Jan 28, 2013 at 5:17 AM, wrote: >> > Hi, >> > >> > I want to know about workflow editor and how it works internally? what >> > type of canvas editor are you using ? doest it support only python >> > environment ? >> > if you have any documents related with ,can i get it . >> > >> > Regards >> > shashi >> > >> > --- >> > >> > This e-mail is for the sole use of the intended recipient(s) and may >> > contain confidential and privileged information. If you are not the >> > intended recipient, please contact the sender by reply e-mail and destroy >> > all copies and the original message. Any unauthorized review, use, >> > disclosure, dissemination, forwarding, printing or copying of this email >> > is strictly prohibited and appropriate legal action will be taken. >> > --- >> > >> > ___ >> > Please keep all replies on the list by using "reply all" >> > in your mail client. To manage your subscriptions to this >> > and other Galaxy lists, please use the interface at: >> > >> > http://lists.bx.psu.edu/ >> >> Email secured by Check Point >> > > > > --- > > This e-mail is for the sole use of the intended recipient(s) and may > contain confidential and privileged information. If you are not the > intended recipient, please contact the sender by reply e-mail and destroy > all copies and the original message. Any unauthorized review, use, > disclosure, dissemination, forwarding, printing or copying of this email > is strictly prohibited and appropriate legal action will be taken. > --- > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] workflow Editor
The client side of the workflow editor is in javascript, using a mix of HTML5 Canvas and DOM elements. It communicates with the server using JSON. The server side is written in Python. The editor could be decoupled and reused, however there are some details that are fairly Galaxy specific right now (particularly how tool states -- the values of the various input parameters -- are encoded). -- James Taylor, Assistant Professor, Biology/CS, Emory University On Mon, Jan 28, 2013 at 5:17 AM, wrote: > Hi, > > I want to know about workflow editor and how it works internally? what > type of canvas editor are you using ? doest it support only python > environment ? > if you have any documents related with ,can i get it . > > Regards > shashi > > --- > > This e-mail is for the sole use of the intended recipient(s) and may > contain confidential and privileged information. If you are not the > intended recipient, please contact the sender by reply e-mail and destroy > all copies and the original message. Any unauthorized review, use, > disclosure, dissemination, forwarding, printing or copying of this email > is strictly prohibited and appropriate legal action will be taken. > --- > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Selecting Intermediary Files in a Workflow
You can work around this using the from_work_dir attribute when specifying the output file in your tool config. -- jt (composed on my phone) On Jan 27, 2013, at 6:13 PM, Peter Cock wrote: > On Sun, Jan 27, 2013 at 10:26 PM, Kolby Chien wrote: >> Hello, >> >> I've been working with a local installation of Galaxy and my question is >> about how Galaxy chooses which file to use in the next step of the workflow. >> >> I have created a custom file type called "cub" which is required for my >> custom tools and I have added this custom file type into the extension list. >> When I run the first step of my workflow, Galaxy will create an intermediate >> cub file called dataset_51.dat.cub containing my processed data, but also an >> empty intermediate dat file as well of the same name sans cub extension >> (dataset_51.dat) . The next tool in the workflow requires a cub file type >> input, but instead of using the cub file, it chooses to process the empty >> .dat file instead, causing the tool to process incorrectly. >> >> I've looked over some settings and double checked that my tool xml files >> specify cub as the output file type and as the input file type for the next >> tool, but now I am at a loss. Is there anyway to specify to Galaxy that the >> cub file should be used in the next step of the workflow instead of the dat >> file? > > Galaxy wants all the files to use the names Galaxy tells the tool, > and those filenames all end with .dat - the problem is probably > Galaxy has told you (i.e. your tool) to use dataset_51.dat as the > output file, but your tool has instead used dataset_51.dat.cub > (which Galaxy will ignore). > > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
