[galaxy-dev] Variable multiple output files not appearing in history
Hi, I'm developing a barcode splitter, which will split barcodes of variable length and then put them into the history for downstream analysis (currently FASTX barcode splitter only splits barcodes of the same length and the user has to download the data through an html link). The number of output files can't be determined until the tool has executed as it will depend on the number of barcodes presented. Here is an abridged version of the tool xml file: split_var_length_barcodes_wrapper.py $input $barcodes $output.extra_files_path $output When I run the tool, the $output file to which I write various tool progress stuff is written and is displayed in the history. Also, the spit barcode files are written to a sub-directory of the same name as the output file. In the Galaxy database, if the $output file is named dataset_11940.dat then an example barcode file name and path is: /home/galaxy/software/galaxy-central/database/files/011/dataset_11940_files /AACAGAGT.fastq Unfortunately, the split barcode files are not being displayed in the history, so I was wondering if anyone could see any problems in what I'm doing. Many thanks, Graham Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Variable multiple output files not appearing in history
Graham, How are the output files being handled in split_var_length_barcodes_wrapper.py? See http://wiki.g2.bx.psu.edu/Admin/Tools/Multiple%20Output%20Files for a reference-- you're going to want to follow the bottom example there involving $__new_file_path__ and the naming convention specified. On a side note, this is something I've brushed up against recently myself, I'd be very interested in seeing what you come up with. -Dannon On Dec 5, 2011, at 6:47 AM, graham etherington (TSL) wrote: > Hi, > I'm developing a barcode splitter, which will split barcodes of variable > length and then put them into the history for downstream analysis > (currently FASTX barcode splitter only splits barcodes of the same length > and the user has to download the data through an html link). The number of > output files can't be determined until the tool has executed as it will > depend on the number of barcodes presented. > Here is an abridged version of the tool xml file: > > version="1.0.0" force_history_refresh="True"> > split_var_length_barcodes_wrapper.py > $input > $barcodes $output.extra_files_path $output >label="Barcodes > to use" /> >name="input" > type="data" label="Library to split" /> > > > > > > > > When I run the tool, the $output file to which I write various tool > progress stuff is written and is displayed in the history. Also, the spit > barcode files are written to a sub-directory of the same name as the > output file. > In the Galaxy database, if the $output file is named dataset_11940.dat > then an example barcode file name and path is: > /home/galaxy/software/galaxy-central/database/files/011/dataset_11940_files > /AACAGAGT.fastq > > Unfortunately, the split barcode files are not being displayed in the > history, so I was wondering if anyone could see any problems in what I'm > doing. > > Many thanks, > Graham > > > > > Dr. Graham Etherington > Bioinformatics Support Officer, > The Sainsbury Laboratory, > Norwich Research Park, > Norwich NR4 7UH. > UK > Tel: +44 (0)1603 450601 > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Variable multiple output files not appearing in history
Hi Graham, I've created something similar myself. I've not put it on the toolshed yet, as I have to test it further, but it seems to work as expected. See code in attachment. Best regards, geert vandeweyer University of Antwerp On 12/05/2011 12:47 PM, graham etherington (TSL) wrote: Hi, I'm developing a barcode splitter, which will split barcodes of variable length and then put them into the history for downstream analysis (currently FASTX barcode splitter only splits barcodes of the same length and the user has to download the data through an html link). The number of output files can't be determined until the tool has executed as it will depend on the number of barcodes presented. Here is an abridged version of the tool xml file: split_var_length_barcodes_wrapper.py $input $barcodes $output.extra_files_path $output When I run the tool, the $output file to which I write various tool progress stuff is written and is displayed in the history. Also, the spit barcode files are written to a sub-directory of the same name as the output file. In the Galaxy database, if the $output file is named dataset_11940.dat then an example barcode file name and path is: /home/galaxy/software/galaxy-central/database/files/011/dataset_11940_files /AACAGAGT.fastq Unfortunately, the split barcode files are not being displayed in the history, so I was wondering if anyone could see any problems in what I'm doing. Many thanks, Graham Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ fastx_barcode_splitter.tar.gz Description: GNU Zip compressed data ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Urgent help needed
Hi Everybody,
I am running a tool which is not defining the output file name as parameter.
the tool is running and I can see the output file in the history directory but
not in the history frame.
how could I solve the problem? I should see and download the output file from
the history but I see an error message instead. the error message not a real
error but rather galaxy cannot figure out which file to display?
your help is appreciated,
From: Jennifer Jackson [j...@bx.psu.edu]
Sent: Tuesday, November 08, 2011 8:22 AM
To: Toqa Manasrah
Cc: Galaxy Dev
Subject: Re: [galaxy-dev] Installing Galaxy on local server
Hello Tuqa,
Galaxy will not stop by itself. That said, you will want to monitor the
instance and any associated cluster, just as you would any other
important processes.
Please send all follow-up with a cc to the mailing list ("reply-all").
The development community is a great resource for administrative advice
and they may have more to add to this reply, i.e. how stable they have
found Galaxy to be under various conditions.
Best,
Jen
Galaxy team
On 11/8/11 4:01 AM, Toqa Manasrah wrote:
> Thank you so much,
>
> I did the installation and I ran it under my server.
> I have one thing concerning about now. if u can help plz.
>
> how to run Galaxy in such way that the html interface will continuously
> working and available for all users visiting the page?
>
> i mean how to guarantee that the "sh run.sh" will keep running without any
> interrupt?
>
> Thank you,
> Tuqa
>
>
> From: Jennifer Jackson [j...@bx.psu.edu]
> Sent: Thursday, November 03, 2011 6:47 PM
> To: Toqa Manasrah
> Cc: galaxy-...@bx.psu.edu
> Subject: Re: [galaxy-dev] Installing Galaxy on local server
>
> Hello Toqa,
>
> From your description, this sounds like a local installation for the
> Galaxy instance itself. Then you can add on a cluster (local or cloud)
> as needed.
>
> Help is under "Deploy Galaxy" at http://wiki.g2.bx.psu.edu/
> Also see the "Tool Shed" at http://wiki.g2.bx.psu.edu/Tool%20Shed
>
> Hopefully the documentation is helpful,
>
> Best,
>
> Jen
> Galaxy team
>
> On 11/3/11 9:40 AM, Toqa Manasrah wrote:
>> Hello,
>>
>> I am a graduate student at GSU. I am looking for installing GALAXY on
>> our local server http://alla.cs.gsu.edu/~software. as a result I like my
>> server homepage start with the GALAXY interface, like this one:
>> http://rna1.engr.uconn.edu:7474/ . after that I wish to integrate some
>> software tolls that we developed in ourdepartment. my primary question
>> is which option Ihave to choose: Local or cloud?
>>
>> looking forward for your help and directions.
>>
>> Thank you.
>>
>> Tuqa
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client. To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>
>> http://lists.bx.psu.edu/
>
> --
> Jennifer Jackson
> http://usegalaxy.org
> http://galaxyproject.org/wiki/Support
>
--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Urgent help needed
Toqa,
Just to make sure I've understood your question: the problem is that a tool
that you're trying to wrap doesn't provide a way to specify a particular output
filename? Take a look at the from_work_dir attribute of a element.
Or are you asking how to define outputs in general? For that, see the wiki at
http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Coutputs.3E_tag_set
Lastly, for new questions please start new email threads, instead of replying
to old ones. This makes it much easier for us to track our responses and make
sure people get appropriate help.
-Dannon
On Dec 5, 2011, at 1:26 PM, Toqa Manasrah wrote:
> Hi Everybody,
> I am running a tool which is not defining the output file name as parameter.
> the tool is running and I can see the output file in the history directory
> but not in the history frame.
>
> how could I solve the problem? I should see and download the output file from
> the history but I see an error message instead. the error message not a real
> error but rather galaxy cannot figure out which file to display?
>
> your help is appreciated,
>
>
>
> From: Jennifer Jackson [j...@bx.psu.edu]
> Sent: Tuesday, November 08, 2011 8:22 AM
> To: Toqa Manasrah
> Cc: Galaxy Dev
> Subject: Re: [galaxy-dev] Installing Galaxy on local server
>
> Hello Tuqa,
>
> Galaxy will not stop by itself. That said, you will want to monitor the
> instance and any associated cluster, just as you would any other
> important processes.
>
> Please send all follow-up with a cc to the mailing list ("reply-all").
> The development community is a great resource for administrative advice
> and they may have more to add to this reply, i.e. how stable they have
> found Galaxy to be under various conditions.
>
> Best,
>
> Jen
> Galaxy team
>
> On 11/8/11 4:01 AM, Toqa Manasrah wrote:
>> Thank you so much,
>>
>> I did the installation and I ran it under my server.
>> I have one thing concerning about now. if u can help plz.
>>
>> how to run Galaxy in such way that the html interface will continuously
>> working and available for all users visiting the page?
>>
>> i mean how to guarantee that the "sh run.sh" will keep running without any
>> interrupt?
>>
>> Thank you,
>> Tuqa
>>
>>
>> From: Jennifer Jackson [j...@bx.psu.edu]
>> Sent: Thursday, November 03, 2011 6:47 PM
>> To: Toqa Manasrah
>> Cc: galaxy-...@bx.psu.edu
>> Subject: Re: [galaxy-dev] Installing Galaxy on local server
>>
>> Hello Toqa,
>>
>> From your description, this sounds like a local installation for the
>> Galaxy instance itself. Then you can add on a cluster (local or cloud)
>> as needed.
>>
>> Help is under "Deploy Galaxy" at http://wiki.g2.bx.psu.edu/
>> Also see the "Tool Shed" at http://wiki.g2.bx.psu.edu/Tool%20Shed
>>
>> Hopefully the documentation is helpful,
>>
>> Best,
>>
>> Jen
>> Galaxy team
>>
>> On 11/3/11 9:40 AM, Toqa Manasrah wrote:
>>> Hello,
>>>
>>> I am a graduate student at GSU. I am looking for installing GALAXY on
>>> our local server http://alla.cs.gsu.edu/~software. as a result I like my
>>> server homepage start with the GALAXY interface, like this one:
>>> http://rna1.engr.uconn.edu:7474/ . after that I wish to integrate some
>>> software tolls that we developed in ourdepartment. my primary question
>>> is which option Ihave to choose: Local or cloud?
>>>
>>> looking forward for your help and directions.
>>>
>>> Thank you.
>>>
>>> Tuqa
>>>
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client. To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>
>>>http://lists.bx.psu.edu/
>>
>> --
>> Jennifer Jackson
>> http://usegalaxy.org
>> http://galaxyproject.org/wiki/Support
>>
>
> --
> Jennifer Jackson
> http://usegalaxy.org
> http://galaxyproject.org/wiki/Support
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-user] Upload executable in galaxy
Hi Giota, The executable should be placed in a directory on $PATH. You can see what directories those are with: % echo $PATH You can add to $PATH with: % export PATH="/new/dir:$PATH" --nate On Dec 4, 2011, at 3:55 PM, Giota Kottara wrote: > Hi, > > I've put the executable in the directory where the xml file and wrapper is. > The message I'm getting when I run the tool is Unable to run tool: No such > file or directory. > I guess that I'm forgetting something > > > Subject: Re: [galaxy-user] Upload executable in galaxy > > From: n...@bx.psu.edu > > Date: Mon, 28 Nov 2011 13:40:24 -0500 > > CC: galaxy-...@bx.psu.edu > > To: giota_kott...@hotmail.com > > > > On Nov 23, 2011, at 11:10 AM, Giota Kottara wrote: > > > > > Dear galaxy developers, > > > > > > I am trying to integrate an executable tool in a private instance of > > > galaxy. I stored the configuration .xml file and the executable in tools > > > folder of Galaxy and the tool appears in the homepage. Nevertheless when > > > i am running the tool it doesn't seem to find the executable. Am I > > > forgetting something? > > > > Hi Giota, > > > > I've moved this over to the galaxy-dev list since it pertains to tool > > development. > > > > Executables will need to be on the $PATH of the user running Galaxy. Could > > you share the error messages you receive when you run the tool? > > > > --nate > > > > > > > > Thank you in advance > > > Giota Kottara > > > ___ > > > The Galaxy User list should be used for the discussion of > > > Galaxy analysis and other features on the public server > > > at usegalaxy.org. Please keep all replies on the list by > > > using "reply all" in your mail client. For discussion of > > > local Galaxy instances and the Galaxy source code, please > > > use the Galaxy Development list: > > > > > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > > > > > To manage your subscriptions to this and other Galaxy lists, > > > please use the interface at: > > > > > > http://lists.bx.psu.edu/ > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] possibly weird config requests...
On Dec 4, 2011, at 5:39 PM, Ross wrote:
> Hi Russell,
>
> Just addressing AD/LDAP authentication - authentication is trivially and best
> (IMHO) left to an external (eg apache) proxy - save yourself a lot of effort
> - it's known to work well.
> Lock down the paste process so it only talks to your apache and provide pass
> through authentication.
> Doing that deals with your first and second questions since galaxy will
> create new users as they appear if appropriately configured - although you
> are stuck with user@yourconfigured.domain as the user ids inside Galaxy even
> though the user authenticates with the sAMAccountName in the example below so
> it's the same login for all our AD logins.
>
> The recipe has been posted a few times and should be easy to find using a
> search.
>
> Something like:
>
>
> Options FollowSymLinks
> AuthType Basic
> AuthBasicProvider ldap
> AuthzLDAPAuthoritative Off
> AuthName "BHRI LDAP"
> Order Allow,Deny
> Allow From All
> Require valid-user
> AuthLDAPBindDN "user@domain"
> AuthLDAPBindPassword "passwordforuser@domain"
> AuthLDAPURL
> "ldap://172.16.1.245:389/OU=Baker,DC=bhri,DC=internal?sAMAccountName?sub?(objectClass=*)"
> RequestHeader set REMOTE_USER %{AUTHENTICATE_sAMAccountName}e
>
>
> works for us on a local AD server
The documentation for the feature explained by Ross is here:
http://usegalaxy.org/production
Responses to other questions follow:
>
>
> On Mon, Dec 5, 2011 at 9:21 AM, Smithies, Russell
> wrote:
> This may be a list of weird requests but our Galaxy server runs internally
> and is only accessible to our users so I’d like to make it a bit easier to
> manage.
>
> I could customize a copy but hopefully some of the “features” I’m after are
> already available. If not, I’m happy to work on them and integrate if others
> think they’ll be useful.
>
>
>
> · Rather than email address as usernames, we’d rather just use our
> corporate logins (lastname, first initial, eg.mine is smithiesr) Or can I
> have an alias list (like samba)?
I know this has been hacked in by another person in their instance, but you
should also not really need to do this if you use the authentication via proxy
as explained above.
> · Is there a bulk user creation script? Rather than have everyone
> create their own account, I’d rather do them all at once. It’s not that I
> don’t trust users but…
Accounts are automatically created when authentication is delegated to the
upstream proxy server.
> · Can I have every user’s home dir automagiclly added as a data dir?
> eg. I’d like to have /home/smithiesr available. Would be nice if this was
> part of a bulk useradd script
Sort of. You could create a directory of symlinks named as the user's Galaxy
email address that link to their home directories and then set this directory
as the value of ftp_upload_dir. The user would then see the contents of his or
her home directory on the upload form (but I imagine this would be quite slow
for large directories). Instead of their entire directory, I'd suggest using a
subdirectory or another designated space on the filesystem.
Importing to a history via the Upload tool may fail if the Galaxy user is
unable to write to the file and/or directory since this is not what the FTP
upload functionality was originally designed for. Documentation is here (not
all of it is relevant to this non-FTP usage):
http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP
> · Any chance of AD integration? I have no idea if Python plays well
> with Active Directory and/or LDAP.
>
> · Is it possible to add new tools without restarting Galaxy? I know
> tool configs can be reloaded from the admin console but I’d like our users to
> be able to incorporate their existing scripts as tools. Would a local
> tool-shed make this easier?
The public tool shed or a local tool shed would allow for this. Note that
users still need administrative privileges to install tools from a tool shed.
--nate
> · We use NTLM for authentication on most of our internal sites – am I
> going to have problems getting Galaxy/Python to authenticate? A few years ago
> we ran into the same requirement with a Java app so ended up hacking together
> a script that passed SSO creds to Internet Exploiter for authentication.
>
>
>
> Sorry if some of these sound dumb or are already implemented, I’m still
> reading thru the docs.
>
>
>
> Thanx,
>
> Russell Smithies
> Infrastructure Team
>
> T 03 489 9085
> M 0274 734 600
> E russell.smith...@agresearch.co.nz
>
> Invermay Agricultural Centre
> Puddle Alley, Private Bag 50034, Mosgiel 9053, New Zealand
>
> T +64 3 489 3809 F +64 3 489 3739 www.agresearch.co.nz
>
>
>
>
>
>
> Attention: The information contained in this message and/or attachments from
> AgResearch Limited is intended only for the persons or entities to which it
> is addressed and may
Re: [galaxy-dev] GATK / R local install configuration
Hi Carlos,
Are you using a version of GATK that is 1.3?
Another possibility is that you are missing an R library (e.g. 'ggplot2'), that
is used by the VariantRecalibrator for building the plots. Start up an
interactive R session and type: "library('ggplot2')"
If you are missing the library, then you will have to install it (it is
available in CRAN). If you have the library, then that is not the problem, but
you can get the actual reason for the Rscript failure by running the Rscript
directly, in this case:
"Rscript
/Volumes/Data/Users/cjavier/galaxy_central/database/files/000/dataset_393.dat"
Please let us know what you find out and if we can provide additional
assistance.
Thanks for using Galaxy,
Dan
dan$ R
R version 2.13.1 (2011-07-08)
Copyright (C) 2011 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library('ggplot2')
Loading required package: reshape
Loading required package: plyr
Attaching package: 'reshape'
The following object(s) are masked from 'package:plyr':
rename, round_any
Loading required package: grid
Loading required package: proto
>
On Dec 1, 2011, at 10:37 AM, Carlos Borroto wrote:
> Hi,
>
> Any chances someone could give me hint on this issue?
>
> Thanks in advance,
> Carlos
>
> On Tue, Nov 29, 2011 at 2:02 PM, Carlos Borroto
> wrote:
>> Hi,
>>
>> I'm testing the GATK pipeline and I ran into a problem with "Variant
>> Recalibrator" tool. It seems I don't have correctly configure R and
>> GATK on my instance, as this tool is failing with this error:
>> mv:
>> /Volumes/Data/Users/cjavier/galaxy_central/database/files/000/dataset_393.dat.pdf:
>> No such file or directory
>>
>> I see this PDF is builded with R and I also see this in the log file:
>> INFO 17:11:44,260 VariantRecalibrator - Executing: Rscript
>> /Volumes/Data/Users/cjavier/galaxy_central/database/files/000/dataset_393.dat
>> WARN 17:11:48,407 RScriptExecutor - RScript exited with 1. Run with
>> -l DEBUG for more info.
>>
>> In the testing server this tool does work:
>> http://test.g2.bx.psu.edu/u/cjav/h/variant-recalibrator---tutorial
>>
>> At least after making sure the input VCF file has matching annotations
>> string to the one you want to select in Galaxy.
>>
>> Any help on what need to be done to get this configuration right. BTW
>> I already have R and Rpy correctly configured and I can run tools like
>> "Statistics/Summary Statistics".
>>
>> Thanks!
>> Carlos
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] DB schema upgrade problem (79 -> 85)
Dear all, Today I upgraded our NBIC server from an older release (fetched on July 5th, 2011) to the latest version in galaxy-dist. After execute "hg update" and "sh manage_db upgrade" and merge some local configurations, I successfully restarted the server. However once I access the server from a web browser, Galaxy crashed. Log file complains that some tables don't exist: == ProgrammingError: (ProgrammingError) (1146, "Table 'galaxy.user_quota_association' doesn't exist") u'SELECT user_quota_association.id AS user_quota_association_id, user_quota_association.user_id AS user_quota_association_user_id, user_quota_association.quota_id AS user_quota_association_quota_id, user_quota_association.create_time AS user_quota_association_create_time, user_quota_association.update_time AS user_quota_association_update_time \nFROM user_quota_association \nWHERE %s = user_quota_association.user_id' [1L] == A check in mysql confirms that only the following 3 tables were added during the migration from schema 79 to 85: * default_quota_association * quota * tool_shed_repository The following 9 tables are missing: cloud_*, user_quota_association, group_quota_association Has anyone experienced a similar problem? Any hint on how to solve this is highly appreciate! Thanks, Leon -- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing Skype: leon_meiMobile: +31 6 41709231 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] DB schema upgrade problem (79 -> 85)
On Dec 5, 2011, at 3:02 PM, Leon Mei wrote: > Dear all, > > Today I upgraded our NBIC server from an older release (fetched on July 5th, > 2011) to the latest version in galaxy-dist. After execute "hg update" and "sh > manage_db upgrade" and merge some local configurations, I successfully > restarted the server. However once I access the server from a web browser, > Galaxy crashed. > > Log file complains that some tables don't exist: > == > ProgrammingError: (ProgrammingError) (1146, "Table > 'galaxy.user_quota_association' doesn't exist") u'SELECT > user_quota_association.id AS user_quota_association_id, > user_quota_association.user_id AS user_quota_association_user_id, > user_quota_association.quota_id AS user_quota_association_quota_id, > user_quota_association.create_time AS user_quota_association_create_time, > user_quota_association.update_time AS user_quota_association_update_time > \nFROM user_quota_association \nWHERE %s = user_quota_association.user_id' > [1L] > == > > A check in mysql confirms that only the following 3 tables were added during > the migration from schema 79 to 85: > * default_quota_association > * quota > * tool_shed_repository > > The following 9 tables are missing: > cloud_*, user_quota_association, group_quota_association Hi Leon, Were there any errors output when you ran mange_db.sh upgrade? --nate > > Has anyone experienced a similar problem? Any hint on how to solve this is > highly appreciate! > > Thanks, > Leon > > > -- > Hailiang (Leon) Mei > Netherlands Bioinformatics Center > BioAssist NGS Taskforce > - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing > Skype: leon_meiMobile: +31 6 41709231 > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] DB schema upgrade problem (79 -> 85)
Hi Nate, Here's the output of the migration: == galaxy@galaxy2:~/prog/galaxy-2011-12-5$ sh manage_db.sh upgrade 79 -> 80... Migration script to create tables for disk quotas. done 80 -> 81... Migration script to add a 'tool_version' column to the hda/ldda tables. done 81 -> 82... Migration script to add the tool_shed_repository table. done 82 -> 83... Migration script to add 'prepare_input_files_cmd' column to the task table and to rename a column. done 83 -> 84... Migration script to add 'ldda_id' column to the implicitly_converted_dataset_association table. done 84 -> 85... Migration script to add 'info' column to the task table. done == Thanks, Leon On Mon, Dec 5, 2011 at 9:11 PM, Nate Coraor wrote: > On Dec 5, 2011, at 3:02 PM, Leon Mei wrote: > > > Dear all, > > > > Today I upgraded our NBIC server from an older release (fetched on July > 5th, 2011) to the latest version in galaxy-dist. After execute "hg update" > and "sh manage_db upgrade" and merge some local configurations, I > successfully restarted the server. However once I access the server from a > web browser, Galaxy crashed. > > > > Log file complains that some tables don't exist: > > == > > ProgrammingError: (ProgrammingError) (1146, "Table > 'galaxy.user_quota_association' doesn't exist") u'SELECT > user_quota_association.id AS user_quota_association_id, > user_quota_association.user_id AS user_quota_association_user_id, > user_quota_association.quota_id AS user_quota_association_quota_id, > user_quota_association.create_time AS user_quota_association_create_time, > user_quota_association.update_time AS user_quota_association_update_time > \nFROM user_quota_association \nWHERE %s = user_quota_association.user_id' > [1L] > > == > > > > A check in mysql confirms that only the following 3 tables were added > during the migration from schema 79 to 85: > > * default_quota_association > > * quota > > * tool_shed_repository > > > > The following 9 tables are missing: > > cloud_*, user_quota_association, group_quota_association > > Hi Leon, > > Were there any errors output when you ran mange_db.sh upgrade? > > --nate > > > > > Has anyone experienced a similar problem? Any hint on how to solve this > is highly appreciate! > > > > Thanks, > > Leon > > > > > > -- > > Hailiang (Leon) Mei > > Netherlands Bioinformatics Center > > BioAssist NGS Taskforce > > - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing > > Skype: leon_meiMobile: +31 6 41709231 > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > > -- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing Skype: leon_meiMobile: +31 6 41709231 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Urgent help needed
Thank you so much Dannon.
I appreciate your help but I need some more clarification.
in the tool I am integrating (which is java classes) if the input parameters are
./main.bash eps0.3_40reads.fa population10_ref.fa 15 6 120
Then teh output is in eps0.3_40reads_I_6_15_CNTGS_DIST0_EM20.txt
As u notice the output file name is not specified in teh command line, and
actually I dont have the source files to go over this naming restriction.
how I can use from_work_dir attribute to tell galaxy this the file to display?
note: there are 11 output files and I am interested in this one only
..._CNTGS_DIST0_EM20.txt
any other method to overcome such problem
I have presentation tomorrwo. hope things will work fine.
Thank you,
From: Dannon Baker [dannonba...@me.com]
Sent: Monday, December 05, 2011 2:41 PM
To: Toqa Manasrah
Subject: Re: [galaxy-dev] Urgent help needed
Sure. See the link in my previous email
(http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax) and look for the
section for an explanation of the attributes available to you. For an
example of a tool that uses the from_work_dir element, see the cufflinks, cuff
compare, top hat, or trinity wrappers in your galaxy install.
-Dannon
On Dec 5, 2011, at 2:25 PM, Toqa Manasrah wrote:
> Thank you Dannon,
> I mean the first choice. no way to specify the output name in tool and it has
> many putputs but I am intereseted in one only.
>
> Actually I dont know to use this attribute from_work_dir and I cant find
> help. can u help plz?
>
>
> From: Dannon Baker [dannonba...@me.com]
> Sent: Monday, December 05, 2011 1:54 PM
> To: Toqa Manasrah
> Cc: galaxy-...@bx.psu.edu dev; Jennifer Jackson; galaxy-user
> Subject: Re: [galaxy-dev] Urgent help needed
>
> Toqa,
>
> Just to make sure I've understood your question: the problem is that a tool
> that you're trying to wrap doesn't provide a way to specify a particular
> output filename? Take a look at the from_work_dir attribute of a
> element.
>
> Or are you asking how to define outputs in general? For that, see the wiki
> at
> http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Coutputs.3E_tag_set
>
> Lastly, for new questions please start new email threads, instead of replying
> to old ones. This makes it much easier for us to track our responses and
> make sure people get appropriate help.
>
> -Dannon
>
>
> On Dec 5, 2011, at 1:26 PM, Toqa Manasrah wrote:
>
>> Hi Everybody,
>> I am running a tool which is not defining the output file name as parameter.
>> the tool is running and I can see the output file in the history directory
>> but not in the history frame.
>>
>> how could I solve the problem? I should see and download the output file
>> from the history but I see an error message instead. the error message not a
>> real error but rather galaxy cannot figure out which file to display?
>>
>> your help is appreciated,
>>
>>
>>
>> From: Jennifer Jackson [j...@bx.psu.edu]
>> Sent: Tuesday, November 08, 2011 8:22 AM
>> To: Toqa Manasrah
>> Cc: Galaxy Dev
>> Subject: Re: [galaxy-dev] Installing Galaxy on local server
>>
>> Hello Tuqa,
>>
>> Galaxy will not stop by itself. That said, you will want to monitor the
>> instance and any associated cluster, just as you would any other
>> important processes.
>>
>> Please send all follow-up with a cc to the mailing list ("reply-all").
>> The development community is a great resource for administrative advice
>> and they may have more to add to this reply, i.e. how stable they have
>> found Galaxy to be under various conditions.
>>
>> Best,
>>
>> Jen
>> Galaxy team
>>
>> On 11/8/11 4:01 AM, Toqa Manasrah wrote:
>>> Thank you so much,
>>>
>>> I did the installation and I ran it under my server.
>>> I have one thing concerning about now. if u can help plz.
>>>
>>> how to run Galaxy in such way that the html interface will continuously
>>> working and available for all users visiting the page?
>>>
>>> i mean how to guarantee that the "sh run.sh" will keep running without any
>>> interrupt?
>>>
>>> Thank you,
>>> Tuqa
>>>
>>>
>>> From: Jennifer Jackson [j...@bx.psu.edu]
>>> Sent: Thursday, November 03, 2011 6:47 PM
>>> To: Toqa Manasrah
>>> Cc: galaxy-...@bx.psu.edu
>>> Subject: Re: [galaxy-dev] Installing Galaxy on local server
>>>
>>> Hello Toqa,
>>>
>>> From your description, this sounds like a local installation for the
>>> Galaxy instance itself. Then you can add on a cluster (local or cloud)
>>> as needed.
>>>
>>> Help is under "Deploy Galaxy" at http://wiki.g2.bx.psu.edu/
>>> Also see the "Tool Shed" at http://wiki.g2.bx.psu.edu/Tool%20Shed
>>>
>>> Hopefully the documentation is helpful,
>>>
>>> Best,
>>>
>>> Jen
>>> Galaxy team
>>>
>>> On 11/3/11 9:40 AM, Toqa Manasrah wrote:
Hello,
I am a graduate student at GSU. I am looking for install
[galaxy-dev] Display VCF in IGV problem
I am unable to get VCF files to display in IGV in my local galaxy installation. BAM files display fine, however, whenever I click on the link to display a VCF file I get the following error message: "You must import this dataset into your current history before you can view it at the desired display application." When I return to Galaxy, my history is indeed changed to a new empty history. I think this may have to do with the fact that I had to setup Apache to allow unauthenticated access to the bam/vcf files (see my notes on how I did this at https://sites.google.com/site/princetonhtseq/tutorials/visualization-with-galaxy-and-igv/igv-as-a-display-application-in-galaxy). However, I can't figure out why BAM files work and VCF files don't. Any advice or insight would be appreciated. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-bugs] Problem with indexing a genome through tophat on a local copy of Galaxy...
Hi David, There should be additional information in the galaxy database about why the job failed; take a look at stderr column of the failed job using some SQL like this: -- select * from job where state='error' and tool_id='tophat' and stderr like '%indexing reference%' order by id desc; -- If that doesn't work, try something simpler: -- select * from job where state='error' and tool_id='tophat' order by id desc; -- What are the errors that you see? Finally, please direct future questions about local Galaxy installations to galaxy-dev (cc'd) so that the community can learn from and participate in discussions. Thanks, J. On Dec 5, 2011, at 10:09 AM, David Matthews wrote: > Hi, > > We have a local version of Galaxy here at Bristol and it is very nice. > However, it does not like to run TopHat on large genomes (i.e. Human) but is > happy with small ones (e.g. viruses). The genome we have is hg19 but missing > chr Y and this genome works fine on the PSU version. However, we seem to be > hitting a wall when we run it here and it seems to relate to the indexing > part right at the begining. Here is the error output we see in the red box: > > An error occurred running this job:Settings: > Output files: "/tmp/tmppA7Hrk/dataset_942.*.ebwt" > Line rate: 6 (line is 64 bytes) > Lines per side: 1 (side is 64 bytes) > Offset rate: 5 (one in 32) > FTable chars: 10 > Strings: unpacked > Max bucket size: default > Max bucket size, sqrt m > > And here is the stuff we see when we click on the green bug (the second set > is just the last bit of a long list of stuff): > > > Error indexing reference sequence > Returning block of 520230381 > Getting block 6 of 7 > Reserving size (531691900) for bucket > Calculating Z arrays > Calculating Z arrays time: 00:00:00 > Entering block accumulator loop: > 10% > 20% > 30% > 40% > 50% > 60% > 70% > 80% > 90% > 100% > Block accumulator loop time: 00:00:59 > Sorting block of length 346541282 > (Using difference cover) > Sorting block time: 00:06:04 > Returning block of 346541283 > Getting block 7 of 7 > Reserving size (531691900) for bucket > Calculating Z arrays > Calculating Z arrays time: 00:00:00 > Entering block accumulator loop: > 10% > 20% > 30% > 40% > 50% > 60% > 70% > 80% > 90% > 100% > Block accumulator loop time: 00:00:40 > Sorting block of length 408062248 > (Using difference cover) > Sorting block time: 00:07:08 > Returning block of 408062249 > Exited Ebwt loop > fchr[A]: 0 > fchr[C]: 837200431 > fchr[G]: 1417119193 > fchr[T]: 1997326330 > fchr[$]: 2835690133 > Exiting Ebwt::buildToDisk() > Total time for backward call to driver() for mirror index: 01:16:32 > TopHat v1.2.0 > We are running on our HPC and have directed the job to land on a whole node > with 8GB of ram on board. Any ideas why the index is failing? > > > Best Wishes, > David. > > __ > Dr David A. Matthews > > Senior Lecturer in Virology > Room E49 > Department of Cellular and Molecular Medicine, > School of Medical Sciences > University Walk, > University of Bristol > Bristol. > BS8 1TD > U.K. > > Tel. +44 117 3312058 > Fax. +44 117 3312091 > > d.a.matth...@bristol.ac.uk > > > > > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] possibly weird config requests...
Thanx guys, that should give me heaps to go on with :-)
--Russell
> -Original Message-
> From: Nate Coraor [mailto:n...@bx.psu.edu]
> Sent: Tuesday, 6 December 2011 8:21 a.m.
> To: Ross
> Cc: Smithies, Russell; galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] possibly weird config requests...
>
> On Dec 4, 2011, at 5:39 PM, Ross wrote:
>
> > Hi Russell,
> >
> > Just addressing AD/LDAP authentication - authentication is trivially and
> > best
> (IMHO) left to an external (eg apache) proxy - save yourself a lot of effort -
> it's known to work well.
> > Lock down the paste process so it only talks to your apache and provide
> pass through authentication.
> > Doing that deals with your first and second questions since galaxy will
> create new users as they appear if appropriately configured - although you
> are stuck with user@yourconfigured.domain as the user ids inside Galaxy
> even though the user authenticates with the sAMAccountName in the
> example below so it's the same login for all our AD logins.
> >
> > The recipe has been posted a few times and should be easy to find using a
> search.
> >
> > Something like:
> >
> >
> > Options FollowSymLinks
> > AuthType Basic
> > AuthBasicProvider ldap
> > AuthzLDAPAuthoritative Off
> > AuthName "BHRI LDAP"
> > Order Allow,Deny
> > Allow From All
> > Require valid-user
> > AuthLDAPBindDN "user@domain"
> > AuthLDAPBindPassword "passwordforuser@domain"
> > AuthLDAPURL
> "ldap://172.16.1.245:389/OU=Baker,DC=bhri,DC=internal?sAMAccountName
> ?sub?(objectClass=*)"
> > RequestHeader set REMOTE_USER
> %{AUTHENTICATE_sAMAccountName}e
> >
> >
> > works for us on a local AD server
>
> The documentation for the feature explained by Ross is here:
>
> http://usegalaxy.org/production
>
> Responses to other questions follow:
>
> >
> >
> > On Mon, Dec 5, 2011 at 9:21 AM, Smithies, Russell
> wrote:
> > This may be a list of weird requests but our Galaxy server runs internally
> and is only accessible to our users so I'd like to make it a bit easier to
> manage.
> >
> > I could customize a copy but hopefully some of the "features" I'm after are
> already available. If not, I'm happy to work on them and integrate if others
> think they'll be useful.
> >
> >
> >
> > * Rather than email address as usernames, we'd rather just use our
> corporate logins (lastname, first initial, eg.mine is smithiesr) Or can I
> have an
> alias list (like samba)?
>
> I know this has been hacked in by another person in their instance, but you
> should also not really need to do this if you use the authentication via proxy
> as explained above.
>
> > * Is there a bulk user creation script? Rather than have everyone
> > create
> their own account, I'd rather do them all at once. It's not that I don't trust
> users but...
>
> Accounts are automatically created when authentication is delegated to the
> upstream proxy server.
>
> > * Can I have every user's home dir automagiclly added as a data
> > dir? eg.
> I'd like to have /home/smithiesr available. Would be nice if this was part of
> a
> bulk useradd script
>
> Sort of. You could create a directory of symlinks named as the user's Galaxy
> email address that link to their home directories and then set this directory
> as
> the value of ftp_upload_dir. The user would then see the contents of his or
> her home directory on the upload form (but I imagine this would be quite
> slow for large directories). Instead of their entire directory, I'd suggest
> using
> a subdirectory or another designated space on the filesystem.
>
> Importing to a history via the Upload tool may fail if the Galaxy user is
> unable
> to write to the file and/or directory since this is not what the FTP upload
> functionality was originally designed for. Documentation is here (not all of
> it
> is relevant to this non-FTP usage):
>
> http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP
>
> > * Any chance of AD integration? I have no idea if Python plays well
> > with
> Active Directory and/or LDAP.
> >
> > * Is it possible to add new tools without restarting Galaxy? I know
> > tool
> configs can be reloaded from the admin console but I'd like our users to be
> able to incorporate their existing scripts as tools. Would a local tool-shed
> make this easier?
>
> The public tool shed or a local tool shed would allow for this. Note that
> users
> still need administrative privileges to install tools from a tool shed.
>
> --nate
>
> > * We use NTLM for authentication on most of our internal sites - am
> > I
> going to have problems getting Galaxy/Python to authenticate? A few years
> ago we ran into the same requirement with a Java app so ended up hacking
> together a script that passed SSO creds to Internet Exploiter for
> authentication.
> >
> >
> >
> > Sorry if some of these sound dumb or are already implemented, I'm still
> reading thru the docs.
> >
> >
> >
[galaxy-dev] Workflow API (runtime modification of tool parameters)
Hello, I was wondering what would be the best way to extend Galaxy's API functionality to allow for runtime modification of tool parameters? I have successfully been able to run workflows programmatically using the API, following the basic steps in: scripts/api/execute_workflow.py. scripts/api/example_watch_folder.py However, it is unclear to me, what would be the best way to run workflows through API with specific parameters at various steps. Should I generate new workflows for every workflow that requires different parameters and upload this to galaxy? Or would it be better to extend the API to allow for runtime parameter configuration? If this is the case, any pointers on how to extend this would be greatly appreciated. Thank you, Richard Park ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Workflow API (runtime modification of tool parameters)
Richard, You're correct in that currently the workflow API affords no method for runtime modification of tool parameters, other than inputs. Depending on your needs, it might be feasible to have a few static workflows that you reuse often via the workflow API. If that isn't the case, and you think it'd be likely that you'll want to, say, programmatically modify parameters at runtime based on input data heuristics, then the API does need to be extended. The first place to look at would be lib/galaxy/web/controllers/workflow.py (run() method), to see the general approach to running workflows and providing parameters in the existing infrastructure. Then, see the API version at lib/galaxy/web/api/workflows.py (create() method) for a comparison. Let me know if you need any help with this at all. Once it's done, if you're willing, we'd certainly like to merge the changes back in and include the functionality in the base galaxy distribution. -Dannon On Dec 5, 2011, at 5:10 PM, Richard Park wrote: > Hello, > I was wondering what would be the best way to extend Galaxy's API > functionality to allow for runtime modification of tool parameters? > > I have successfully been able to run workflows programmatically using > the API, following the basic steps in: > scripts/api/execute_workflow.py. > scripts/api/example_watch_folder.py > > However, it is unclear to me, what would be the best way to run > workflows through API with specific parameters at various steps. > Should I generate new workflows for every workflow that requires > different parameters and upload this to galaxy? Or would it be better > to extend the API to allow for runtime parameter configuration? If > this is the case, any pointers on how to extend this would be greatly > appreciated. > > Thank you, > Richard Park > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
