[galaxy-dev] Citing Galaxy + Toolshed in an app note of a small tool

2014-03-27 Thread Assaf Gordon
Hello Galaxy People! (it's been a while since I've last been here... a pleasure to be back). I intend to publish a small command-line utility, which will also be available through Galaxy Toolshed. It'll be a small application note, so not a lot of space for many citations. The relevant

Re: [galaxy-dev] Tool run on cluster returns error state on clean install

2013-03-27 Thread Assaf Gordon
Hello Carrie, Ganote, Carrie L wrote, On 03/27/2013 01:22 PM: I am trying to upgrade our instance to the newest dist but I get the following in stderr when I attempt to run Bowtie or BWA: PREFIX/subs/galaxy_31.sh: command substitution: line 7: syntax error near unexpected token `('

Re: [galaxy-dev] Exporting histories fails: no space left on device

2013-03-26 Thread Assaf Gordon
Hello Joachim, Joachim Jacob | VIB | wrote, On 03/26/2013 10:01 AM: abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on

Re: [galaxy-dev] Exporting histories fails: no space left on device

2013-03-25 Thread Assaf Gordon
Hello Joachim, Couple of things to check: On Mar 25, 2013, at 10:01 AM, Joachim Jacob | VIB | wrote: Hi, About the exporting of history, which fails: 1. the preparation seems to work fine: meaning: choosing 'Export this history' in the History menu leads to a URL that reports initially

Re: [galaxy-dev] Production and Dev server on the same database.

2013-03-25 Thread Assaf Gordon
Hello Ted! Peter Cock wrote, On 03/25/2013 01:51 PM: On Mon, Mar 25, 2013 at 5:41 PM, Ted Goldstein tedgoldst...@gmail.com wrote: Hi Does anyone have any experience or advice about running two servers on the same database? My advice is don't do it. I'll second Peter's advice, don't ever

Re: [galaxy-dev] Production and Dev server on the same database.

2013-03-25 Thread Assaf Gordon
Couple of clarifications for the script, which aren't obvious: Assaf Gordon wrote, On 03/25/2013 02:03 PM: To facilitate that, we use the following script, that can quickly copy the data from our production server to our development server. Once it's run, the databases contain the same

Re: [galaxy-dev] A Galaxy environment that can support up to 35 users?

2012-09-17 Thread Assaf Gordon
Hello Dan, Couple of lessons we learned from setting up similar workshop-galaxies: Dan Sullivan wrote, On 09/17/2012 01:04 PM: Hi, Galaxy Developers, Is anybody out there managing a Galaxy environment that was designed and or has been tested to support 35 concurrent users? The reason why

Re: [galaxy-dev] Cannot install freebayes with migration script

2012-08-29 Thread Assaf Gordon
Hello Philip, Philip Mabon wrote, On 08/29/2012 10:08 AM: Looks like I forgot to reset that value. So making progress... I can install freebayes successfully (the front end timeout but it worked) However, samtools fails with the following message in web worker log: [localhost] local: sed

Re: [galaxy-dev] Installing Galaxy and Hooking into a SGE Cluster

2012-08-23 Thread Assaf Gordon
(Moving to Galaxy-dev, seems more appropriate). John Jones wrote, On 08/23/2012 05:29 PM: His original question was about getting Galaxy to recognise LDAP authentication and personal storage space rather than shared storage space as is usual with Galaxy. Licensing only came into it

Re: [galaxy-dev] NTLM auth not passing http_remote_user?

2012-07-17 Thread Assaf Gordon
Hello Russell, Smithies, Russell wrote, On 07/17/2012 05:41 PM: It appears that NTLM isn't passing the HTTP_REMOTE_USER through for us. Anyone else struck this and found a solution? Our apache config looks like this: Location / AuthName Galaxy NTLMAuth on

Re: [galaxy-dev] preserving newlines in input parameters

2012-06-19 Thread Assaf Gordon
Hello Jesse, Dam, Jesse van wrote, On 06/19/2012 01:27 PM: I found a solution, it seemed that I made a stupid mistake somewhere. But the following solution will work to preserve the original text in the edit field. Add the following to your parameter [...] Then I use the following code

[galaxy-dev] CloudMan/Galaxy wiki suggestion

2012-04-13 Thread Assaf Gordon
Hello, Great work with the Galaxy on the Cloud - it's the first time I've used it and it looks very impressive (and I'm not a big cloud fan :) ). Once I got it working it was very nice, but before it worked I encountered two issues (not sure if those were voodoo or not) that could be perhaps

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread Assaf Gordon
Hello Sarah, Sarah Maman wrote, On 01/19/2012 10:49 AM: I'm talking to Open LDAP. I have changed RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e to RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e But I always have the same message /(Access to Galaxy is denied Galaxy is configured

[galaxy-dev] Rpy-1.0.3 and R-2.13.1

2011-09-22 Thread Assaf Gordon
Hello, It seems this issue (rpy) pops up every now and then: Has anyone managed to get rpy-1.0.3 working properly with R-13.1 (including loading packages) ? After compiling R-2.13.1 from source, and recompiling rpy-1.0.3, I get the following error when trying to load an R module through rpy:

[galaxy-dev] Filtering Illumina CASAVA 1.8 FASTQ files

2011-08-07 Thread Assaf Gordon
Hello, With the recent upgrade to CASAVA 1.8, Illumina's FASTQ files now contain both filtered and non-filtered reads (i.e. high and low quality reads, what used to be called reads that passed filtering). Here's a program (+galaxy wrapper) that can filter those FASTQ files, preparing them for

Re: [galaxy-dev] configuring apache proxy server issue

2011-08-05 Thread Assaf Gordon
Your rewrite rules are not taking affect (as evident by the fact that apache is looking for a folder named galaxy in your document root folder (/usr/local/apache/htdocs/). First thing to try is to take the RewriteRules statements outside the location block. Then, enable mod_rewrite logging

Re: [galaxy-dev] HOW TO RETRIEVE DATA FROM HISTORY??!!

2011-08-05 Thread Assaf Gordon
Hi Colin, colin molter wrote, On 08/05/2011 01:54 PM: Is there a way to directly move/copy data from your galaxy history to a given location in the filesystem of the same galaxy server? Said differently, there is a nice way to import data from the server to galaxy, is it possible to do the

Re: [galaxy-dev] TEMP variable in cluster install

2011-08-04 Thread Assaf Gordon
As a follow-up for SGE + TMP variables, I've also encountered problems with settings TEMP,TMP and TMPDIR variables. Tried setting ~/.sge_request and changing the Galaxy environment variables before starting python - nothing worked - TMPDIR and TMP were always set to /tmp/all.q.X by SGE .

[galaxy-dev] BWA sampe runs 'forever'

2011-07-29 Thread Assaf Gordon
Hello, Question for galaxy maintainers: have you encountered situations where BWA jobs run 'forever' (for days) ? A little digging shows that it's the bwa sampe step, and SEQanswers thread mention it's somewhat common: http://seqanswers.com/forums/showthread.php?t=11652

Re: [galaxy-dev] BWA sampe runs 'forever'

2011-07-29 Thread Assaf Gordon
Chris Fields wrote, On 07/29/2011 12:35 PM: On Jul 29, 2011, at 11:00 AM, Assaf Gordon wrote: Question for galaxy maintainers: have you encountered situations where BWA jobs run 'forever' (for days) ? ... I'm wondering if this is common enough to justify adding -A to the wrapper

Re: [galaxy-dev] BWA wrapper with Illumina-1.3+ FASTQ files

2011-07-28 Thread Assaf Gordon
Hello Peter and all, Peter Cock wrote, On 07/28/2011 05:08 PM: It concerns me that you're doing this for both fastqillumina format (good) and fastqsolexa (bad). Treating the later as fastqillumina would give negative scores and probably cause trouble. Unless BWA copes but if so it is a poor

Re: [galaxy-dev] Apache Configuration for Exporting Histories and Workflows

2011-07-22 Thread Assaf Gordon
Hi John, John Chilton wrote, On 07/22/2011 11:40 AM: We have Galaxy behind an apache proxy, and apache is taking care of the authentication. If I want to enable exporting workflows and histories to other galaxy instances do I just need to disable the authentication checks for locations

[galaxy-dev] Per-user tools permissions

2011-07-21 Thread Assaf Gordon
Hi, I've noticed a ticket on the wiki for per-user tool permissions: https://bitbucket.org/galaxy/galaxy-central/issue/269/use-galaxy-security-to-restrict-tool If anyone is interested, the following patch implements such thing (but requires manual code changes whenever the access needs to be

Re: [galaxy-dev] Detailed SGE timing information about galaxy jobs

2011-07-20 Thread Assaf Gordon
? Best, Jen Galaxy team On 7/19/11 11:45 AM, Assaf Gordon wrote: Hi all, I got around to writing the report I always needed: how much time each job actually runs and how much time it spends waiting in the SGE queue. The attached shell script produces the report, combining information

[galaxy-dev] Detailed SGE timing information about galaxy jobs

2011-07-19 Thread Assaf Gordon
Hi all, I got around to writing the report I always needed: how much time each job actually runs and how much time it spends waiting in the SGE queue. The attached shell script produces the report, combining information from SGE's QACCT with the galaxy job/dataset information. The output

[galaxy-dev] more cleanup scripts messages

2011-07-13 Thread Assaf Gordon
Hi Greg (again), This patch adds messages for the purge histories step: 1. A message is printed when processing a new history (so that the user can tell to which history the following dataset/hda messages belong). 2. in info_only mode, a message is printed about each deleted history 3. in

[galaxy-dev] cleanups for cleanup scripts

2011-07-12 Thread Assaf Gordon
Hi Greg, Another small patch for cleanup_datasets.py, that makes the output messages slightly more informative. The changes are: 1. If a dataset is skipped (because it's shared/cloned and was already process), no message is printed at all. 2. If a dataset can not be deleted because it is

Re: [galaxy-dev] Sharing authentication between Galaxy and other WSGI apps on the same web server (with custom UI)?

2011-06-20 Thread Assaf Gordon
Hello John, I'm not an apache expert, but I can try to help with some info. your question involves two different issues, which are not dependent on one another. First, can one setup apache authentication that will affect both Galaxy and other things on your server ? The answer is yes.

Re: [galaxy-dev] question/bug with cuffcompare wrapper

2011-06-07 Thread Assaf Gordon
Hi all, Jeremy Goecks wrote, On 06/07/2011 01:25 PM: If you modify the Cufflinks/compare/diff wrappers to be compatible with 1.0.2/3, please either put them in the toolshed or send them to me and I'll incorporate them into the Galaxy code base. We are actually working on customized

[galaxy-dev] question/bug with cuffcompare wrapper

2011-06-06 Thread Assaf Gordon
Hi, It seems the cuffcompare XML wrapper expects a file named cc_output to contain the transcript accuracy report. I guess it worked with versions 0.9.3 and 1.0.1, but at least in version 1.0.2, this file is now named cc_output.stats, and the transcript accuracy file is empty. so the XML

[galaxy-dev] CuffDiff output mis-named ?

2011-06-06 Thread Assaf Gordon
Hi, In cuffdiff_wrapper.xml, there are three outputs associated with CDS: data format=tabular name=cds_exp_fpkm_tracking label=${tool.name} on ${on_string}: CDS Expression FPKM Tracking/ data format=tabular name=cds_fpkm_tracking label=${tool.name} on ${on_string}: CDS FPKM tracking/ data

Re: [galaxy-dev] suggestion for multithreading

2011-06-02 Thread Assaf Gordon
(moved to galaxy-dev) Nate Coraor wrote, On 06/02/2011 01:31 PM: Peter Cock wrote: On Thu, Jun 2, 2011 at 6:23 PM, Nate Coraor n...@bx.psu.edu wrote: pbs.py then knows to translate 'resource type=cores8/resource' to '-l nodes=1:ppn=8'. Your tool can access that value a bunch, like

Re: [galaxy-dev] Update of cufflinks

2011-06-02 Thread Assaf Gordon
Just FYI, With the recent version of cufflinks (at least 1.0.2), the developers added a 'feature' of automatic version check. While the idea is nice, the implications are not: cufflinks,cuffcompare and cuffdiff will connect to host cufflinks.cbcb.umd.edu every time you run them. I'm sure the

Re: [galaxy-dev] fasta width formatter

2011-05-11 Thread Assaf Gordon
I would also recommend getting the latest XML wrappers from here: http://hannonlab.cshl.edu/fastx_toolkit/download.html Or at least changing zcat to cat in your XML files, as zcat on Mac-OS behaves differently than linux's (and that is the source of your error message). Nate Coraor wrote,

Re: [galaxy-dev] Getting tool dirpath in a Python code file

2011-04-29 Thread Assaf Gordon
Hi Leandro, Leandro Hermida wrote, On 04/29/2011 06:17 AM: But I have one complication I guess. My tool is already running a Python code file to dynamically create a drop-down menu using the dynamic_options attribute a la: param type=select dynamic_options=my_options() ... code

Re: [galaxy-dev] Getting tool dirpath in a Python code file

2011-04-28 Thread Assaf Gordon
Leandro Hermida wrote, On 04/28/2011 08:55 AM: Hi galaxy developers, Just want to double-check, there is no way to import some kind of galaxy tool context info into python code you are running for a tool? Nothing is impossible... just depends on how messy you want to get :) for me, the

[galaxy-dev] problem running latest galaxy-central

2011-04-28 Thread Assaf Gordon
Question: The recent galaxy-central revision ( 2b057869b72f ) doesn't work on my computer, saying DistributionNotFound: cython=0.12.1. Example: $ hg clone http://bitbucket.org/galaxy/galaxy-central/ galaxy_hidden_dataset_test $ cd galaxy_hidden_dataset_test $ hg id 2b057869b72f tip $ sh

Re: [galaxy-dev] problem running latest galaxy-central

2011-04-28 Thread Assaf Gordon
it :) http://cython.org/#download From: galaxy-dev-boun...@lists.bx.psu.edu [galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Assaf Gordon [gor...@cshl.edu] Sent: 28 April 2011 20:52 To: galaxy-...@bx.psu.edu Subject: [galaxy-dev] problem running latest

Re: [galaxy-dev] Galaxy tools' priority level (nice)

2011-04-21 Thread Assaf Gordon
Hi Peter, Peter Cock wrote, On 04/21/2011 11:00 AM: Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place:

Re: [galaxy-dev] Galaxy tools' priority level (nice)

2011-04-21 Thread Assaf Gordon
Peter Cock wrote, On 04/21/2011 12:59 PM: On Thu, Apr 21, 2011 at 5:43 PM, Assaf Gordon gor...@cshl.edu wrote: Hi Peter, Peter Cock wrote, On 04/21/2011 11:00 AM: Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything

Re: [galaxy-dev] Question about cuffcompare tool implementation

2011-04-11 Thread Assaf Gordon
Thanks for the quick answer, Jeremy Goecks wrote, On 04/11/2011 12:50 PM: Assuming cuffcompare generates a TMAP and REFMAP file for each input file, does this mean the wrapper tool retrieves only the first and second input files' TMAP and REFMAP files ? Yes, this is the way it works

Re: [galaxy-dev] another update problem

2011-04-11 Thread Assaf Gordon
Couple of voodoo things to try: 1. Manually clean the cache of your web browser 2. Restart your apache/nginx Depending on your last update revision, it's possible that some of your javascripts were updated by the latest pull, but your webbrowser maintains a cached older version. Those are the

Re: [galaxy-dev] another update problem

2011-04-11 Thread Assaf Gordon
, apache, and cleared my cache. Still didn't help. I'd be happy to share the URL of my instance via private email if it'll help. On 4/11/11 3:05 PM, Assaf Gordon wrote: Couple of voodoo things to try: 1. Manually clean the cache of your web browser 2. Restart your apache/nginx Depending

Re: [galaxy-dev] improved view details button

2011-04-11 Thread Assaf Gordon
:13 PM, Assaf Gordon wrote: It doesn't give the modification date of the dataset (also: what exactly is modification ? all datasets in Galaxy are immutable). It seems that the modification date is affected by any modification to the history. Just note that changing the history name affects

Re: [galaxy-dev] Recommended Specs for Production System

2011-04-08 Thread Assaf Gordon
Assaf Gordon wrote, On 04/08/2011 10:07 AM: Processes: The servers processes that you should plan for are: 1 galaxy process for job-runner 2 or 3 galaxy processes for web-fronts 1 process of postgres 1 process of apache optionally 1 process of galaxy-reports you'll also want to leave

Re: [galaxy-dev] samtools sam-to-bam problem

2011-04-07 Thread Assaf Gordon
the error message was not so informative at all, to a point a traceback was needed... -gordon On Apr 7, 2011, at 1:05 AM, Assaf Gordon wrote: Just another example why python's misleadingly simple idioms are quite dangerous in production code (couldn't help myself from teasing about

Re: [galaxy-dev] Trouble viewing interval files in UCSC with Apache+XSendFile

2011-04-06 Thread Assaf Gordon
Until a better solution comes along, this tiny patch makes the temp files world-readable: http://cancan.cshl.edu/labmembers/gordon/files/apache_xsendfile_temp_files.patch Assaf Gordon wrote, On 04/05/2011 11:14 AM: Hello, I've encountered a strange combination of factors that results

Re: [galaxy-dev] samtools sam-to-bam problem

2011-04-06 Thread Assaf Gordon
/samtools/sam_to_bam.py, line 150, in __main__ if len( open( tmp_aligns_file_name ).read() ) == 0: MemoryError Error extracting alignments from (/home/galaxy/galaxy-dist/database/files/000/dataset_785.dat), (galaxy_env)[galaxy@vail pbs]$ On 4/6/11 7:29 PM, Assaf Gordon wrote: Ryan, Since we're

Re: [galaxy-dev] samtools sam-to-bam problem

2011-04-06 Thread Assaf Gordon
/labmembers/gordon/files/cshl_sam_to_bam.tar.bz2 /shameless plug On 04/07/2011 01:05 AM, Assaf Gordon wrote: Just another example why python's misleadingly simple idioms are quite dangerous in production code (couldn't help myself from teasing about python... sorry about that). Seems like line 150

[galaxy-dev] Trouble viewing interval files in UCSC with Apache+XSendFile

2011-04-05 Thread Assaf Gordon
Hello, I've encountered a strange combination of factors that results in file access problems. Perhaps I'm doing something wrong - any advice will be appreciated. I'm using Apache + XSendFile. when viewing a strict BED file, everything works, because the actual dataset filename is passed on

Re: [galaxy-dev] Password input field for tools?

2011-04-04 Thread Assaf Gordon
Daniel, Note that this (and similar) password solutions in Galaxy is not secure in any real way (especially if you're dealing with patient data or HIPPA): The password (hidden or visible in a text field) is still transmitted in clear-text over the network (unless you're using HTTPS/SSL). If

Re: [galaxy-dev] Alternative Bowtie/TopHat tools

2011-04-04 Thread Assaf Gordon
More bug fixes, and a paired-end TopHat tool. Available here: http://cancan.cshl.edu/labmembers/gordon/files/cshl_rnaseq_2011_04_04.tar.bz2 See the interface here: http://cancan.cshl.edu/publicgalaxy/root?tool_id=cshl_tophat_pe1 Comments are very welcomed, Cufflinks is next, -gordon Assaf

[galaxy-dev] suggestions for the SAM-to-BAM tool

2011-04-01 Thread Assaf Gordon
Hi, Couple of things that can be slightly improved in the SAM-to-BAM tool: 1. Reference list is not informative (it's the technical way to say: list of chromosomes and their sizes based on a FASTA file). Users do not generally know what reference list is. 2. The Locally Cached option is not

[galaxy-dev] Community server bug and request

2011-04-01 Thread Assaf Gordon
Tiny bug: When viewing a file inside a tarball, the content-type is forced to text/plain. Works for most files, but not with image files (example: the Sequence Logo tool has a jpg image in the tarball, the browser will display it as text/plain with binary characters). This is done in

Re: [galaxy-dev] Anyone has tools for NCBI Lite-SRA format ?

2011-04-01 Thread Assaf Gordon
anton wrote, On 04/01/2011 09:02 PM: We are getting multiple requests for this but did not have the bandwidth to do it. So if you are planning to implement - share through the toolshed and we will roll it to main. To start, this patch adds SRA binary type with sniffing and upload support:

[galaxy-dev] Fresh galaxy installation tips/requests

2011-03-30 Thread Assaf Gordon
Hi all, It's been a long while since I had to install a fresh production galaxy server, and I can offer some tips or requests for minor improvements (but to give credit were it's due - installation is much smoother now, and DRMAA works great with SGE out of the box). These are just minor

Re: [galaxy-dev] Alternative bowtie tools

2011-03-29 Thread Assaf Gordon
Hi Peter, Peter Cock wrote, On 03/29/2011 05:39 AM: 2. the tools accepts FASTA, FASTQ in both Sanger and Illumina format (no more need for grooming). Illumina is the default for newly uploaded FASTQ files. I think that's a bad idea - use Sanger FASTQ as the default to be consistent with

Re: [galaxy-dev] Alternative bowtie tools

2011-03-29 Thread Assaf Gordon
Hi Dan, Daniel Blankenberg wrote, On 03/29/2011 10:55 AM: When files are added to Galaxy, the datatype can be directly set to any of the fastq variants (e.g. fastqillumina), which removes the requirement of grooming (but should only be done when users know what they are doing). I'm not using

Re: [galaxy-dev] Alternative bowtie tools

2011-03-29 Thread Assaf Gordon
Dan and Peter, Peter Cock wrote, On 03/29/2011 12:08 PM: Why not do the Illumina to Sanger conversion as part of your pipeline that gets the data into Galaxy (and mark the files as fastqsanger)? As Glen said, with a C tool that isn't really so slow. That future proofs you for the pending

Re: [galaxy-dev] Database cleanup and user management

2011-03-14 Thread Assaf Gordon
Hello Sebastian, We are doing similar clean-ups as you've requested (except the automated email notification, which will be implemented soon). We would like to see how much space is used by an individual user and evenutally be able to remove them after some time of inactivity (180 days?),

Re: [galaxy-dev] [galaxy-user] operating on, and getting at, large files in galaxy...

2011-02-21 Thread Assaf Gordon
Hi Nick, If you're running your own local instance, then nothing is impossible - it's just a bit ugly... Nate Coraor wrote, On 02/21/2011 12:36 PM: [...] many tools that operate on these files produces multiple, uncompressed large files which quickly eat up the disk allocation. [...] With no

[galaxy-dev] Problem upgrading DB from 67 to 68

2011-02-01 Thread Assaf Gordon
Hi, Trying to upgrade DB schema ('sh manage_db.sh upgrade') from 67 to 68 fails with the following message: === sqlalchemy.exc.ProgrammingError: (ProgrammingError) syntax error at or near RENAME LINE 1: ALTER SEQUENCE sequencer_id_seq RENAME TO external_service_i...