[galaxy-dev] data_columns

2014-11-07 Thread Jorrit Boekel
where are those columns set on the metadata? Is it stored, generated from the input file? My files are not official formats, just tsv subclasses. I’m on changeset 14567:007f6a80629a in galaxy-dist. cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm

Re: [galaxy-dev] data_columns

2014-11-07 Thread Jorrit Boekel
No, I am not subclassing from Tabular. But when I do, it suddenly works. Thanks loads Dan! cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden On 07 Nov 2014, at 19:07, Daniel Blankenberg wrote: > Hi Jorrit, > > Are you subclassing

[galaxy-dev] cloudman/galaxy on EC2 - snapshot not found

2012-02-14 Thread Jorrit Boekel
(testing on AWS free tier), but I've tried m1.large, and that didn't help. I noted that the snapshot ID is always the same. Since the AMI is from March 2011 (at least the one I select manually), I assume that I have something wrong in my settings or that Amazon has changed something

[galaxy-dev] which galaxy version is on the AMI?

2012-02-15 Thread Jorrit Boekel
Dear list, As a first attempt to put my tools in the cloudman/galaxy environment, I have tried to just crudely copy them to their respective directories, along with tool_conf.xml. This worked, but some tools raise errors upon loading them that I cannot reproduce in my local Galaxy instance (n

[galaxy-dev] uploads stuck in history

2012-02-22 Thread Jorrit Boekel
Dear list, I have stumbled on some strange behaviour when uploading files to galaxy via the upload.py tool. At times, the upload seems to be stalled in history and is never actually performed, followed by a seemingly infinite history update (see log below). My system is Ubuntu 11.10 and runs

Re: [galaxy-dev] uploads stuck in history

2012-02-22 Thread Jorrit Boekel
ge system b) ...eventually, unless the user has deleted the history item, it does turn green and the job is listed as 'ok'. Do you experience this problem with all files, even small ones? Regards, Hans On 02/22/2012 10:14 AM, Jorrit Boekel wrote: Dear list, I have stumbled on s

[galaxy-dev] babel egg: epiphany locale setting fix

2012-02-24 Thread Jorrit Boekel
Dear list, I encountered problems when using galaxy with ubuntu's epiphany browser. It seems that when using this browser, HTTP_ENVIRON's locale setting is sometimes 'en-us', but sometimes ' en-us'. That's a whitespace before the locale. I solved the problem by adding a simple .strip() in egg

[galaxy-dev] uploading multiple files into one dataset

2012-02-29 Thread Jorrit Boekel
. If there was a "tar all sets in library and import to history" I'd be using that, but I've only encountered "tar/zip sets and download locally" which would be a bit of a workaround. I haven't found much on this subject in the mailing list, has

Re: [galaxy-dev] uploads stuck in history

2012-03-07 Thread Jorrit Boekel
Van: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] Namens Jorrit Boekel Verzonden: woensdag 22 februari 2012 10:15 Aan: galaxy-dev@lists.bx.psu.edu Onderwerp: [galaxy-dev] uploads stuck in history Dear list, I have stumbled on some strange behaviour when uploading

[galaxy-dev] cloudman on ubuntu 12.04

2012-05-11 Thread Jorrit Boekel
e fix (two lines or so in sge.py). cheers, jorrit boekel ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the

Re: [galaxy-dev] cloudman on ubuntu 12.04

2012-05-11 Thread Jorrit Boekel
I now see that there is already support for this in the current repo, but I was using the cm.tar.gz in the default cloudman bucket, which is older and doesn't have the ubuntu 12.04 support. My mistake, pretend you never saw it. cheers, jorrit On 05/11/2012 12:27 PM, Jorrit Boekel

Re: [galaxy-dev] Uploading problem

2012-05-22 Thread Jorrit Boekel
Hi, Hanging uploads have been a problem before with small and large files: http://comments.gmane.org/gmane.science.biology.galaxy.devel/4583 I don't know if it's been fixed, but it seemed to depend on browser choice and on what server was running. I was recommended to use the nginx server IIRC

Re: [galaxy-dev] uploading multiple files into one dataset

2012-06-01 Thread Jorrit Boekel
emand for this sort of function, but if anyone else would like to use/modify it, I can fork and issue a pull request. cheers, jorrit On 03/06/2012 06:56 PM, Nate Coraor wrote: On Feb 29, 2012, at 11:34 AM, Jorrit Boekel wrote: Dear list, Our lab's proteomics data is frequently outputt

Re: [galaxy-dev] Appending _task_%d suffix to multi files

2013-07-31 Thread Jorrit Boekel
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Jorrit Boekel Sent: Thursday, 25 October 2012 8:35 PM To: Peter Cock Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] the multi job splitter I keep the files matched by keeping a _task_%d suffix to their names. So each task is

Re: [galaxy-dev] Appending _task_%d suffix to multi files

2013-08-01 Thread Jorrit Boekel
such?) Cheers, -Piotr *From:*Jorrit Boekel [mailto:jorrit.boe...@scilifelab.se] *Sent:* Wednesday, July 31, 2013 8:18 PM *To:* Khassapov, Alex (CSIRO IM&T, Clayton) *Cc:* p.j.a.c...@googlemail.com; jmchil...@gmail.com; galaxy-dev@lists.bx.psu.edu; Szul, Piotr (ICT Centre, Marsfield); Burdet

[galaxy-dev] local_task_queue_workers

2013-11-07 Thread Jorrit Boekel
currently set to 2. However, 5 tasks are run simultaneously, leading to memory problems. Am I overlooking something that anyone knows of? cheers, jorrit boekel -- Scientific programmer Mass spec analysis support @ BILS Janne Lehtiö / Lukas Käll labs SciLifeLab Stockholm

[galaxy-dev] galaxy on ubuntu 14.04: hangs on metadata cleanup

2014-05-07 Thread Jorrit Boekel
). I don’t know where to go from here with debugging, but has anyone seen anything similar? Right now it feels like it may be caused by timing rather than actual code problems. cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockho

Re: [galaxy-dev] galaxy on ubuntu 14.04: hangs on metadata cleanup

2014-05-07 Thread Jorrit Boekel
I should probably mention that the data filesystem is NFS, exported by the master from /mnt/galaxy/data and mounted on the worker. No separate fileserver. Master is the one that hangs. cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden On 07

Re: [galaxy-dev] galaxy on ubuntu 14.04: hangs on metadata cleanup

2014-05-08 Thread Jorrit Boekel
It seems to be an NFS related issue. When I run a separate VM as an NFS server that hosts the galaxy data (files, job workdir, tmp, ftp), problems are gone. There’s probably an explanation for that, but I’m going to leave it at this. cheers, — Jorrit Boekel Proteomics systems developer BILS

Re: [galaxy-dev] stdout in history

2014-05-31 Thread Jorrit Boekel
://bitbucket.org/glormph/adapt/commits/86ec8b1cfeb737dfa31a1b6faf18d918bea7b4c3 I started implementing it in datatypes (first commit), but moved it to tool config instead. If this is really interesting I guess I can submit a pull request, but it felt a bit hacky at the time. cheers, — Jorrit Boekel

Re: [galaxy-dev] tool dependencies fail

2014-05-31 Thread Jorrit Boekel
(as galaxy user, or whoever owns the tools)? cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden On 31 May 2014, at 01:53, Isabelle Phan wrote: > Hello, > > I'm following "Managed Tool Dependencies" to the letter and sti

[galaxy-dev] Deleted jobs on the cloud keep running

2012-08-29 Thread Jorrit Boekel
the default behaviour when deleting jobs in the UI is that they keep running but hidden, or is there something wrong in my config? Is there a way to change this with a setting? cheers, jorrit boekel ___ Please keep all replies on the list by

[galaxy-dev] generating composite files

2012-08-31 Thread Jorrit Boekel
ole thing feels a bit like a hack, so I wanted to ask if there is a way for tools to generate composite files the same way uploads can. cheers, jorrit boekel ___ Please keep all replies on the list by using "reply all" in you

[galaxy-dev] python egg cache exists error

2012-09-14 Thread Jorrit Boekel
Dear list, I am running galaxy-dist on Amazon EC2 through Cloudman, and am using the enable_tasked_jobs to run jobs in parallel. Yes, I know it's not recommended in production. My jobs usually get split in 72 parts, and sometimes (but not always, maybe in 30-50% of cases), errors are returned

Re: [galaxy-dev] DRMAA: TypeError: check_tool_output() takes exactly 5 arguments (4 given)

2012-09-18 Thread Jorrit Boekel
Is it possible that you are looking at different classes? TaskWrapper's finish method does not use the job variable in my recently merged code either (line ~1045), while JobWrapper's does around line 315. cheers, jorrit On 09/18/2012 03:55 PM, Scott McManus wrote: I have to admit that I'm

Re: [galaxy-dev] python egg cache exists error

2012-09-18 Thread Jorrit Boekel
solve this in a non-hackish way, so until someone finds out, I may use reading from a 'eggs_extracted.txt' file to determine if the eggs have been extracted. And locking the file when writing to it of course. cheers, jorrit On 09/14/2012 10:57 AM, Jorrit Boekel wrote: Dear list,

Re: [galaxy-dev] python egg cache exists error

2012-09-19 Thread Jorrit Boekel
24 PM, James Taylor wrote: Interesting. If I'm reading this correctly the problem is happening inside pkg_resources? (galaxy.eggs unzips eggs, but I think it does so on install [fetch_eggs] time not run time which would avoid this). If so this would seem to be a locking bug in pkg_resources.

Re: [galaxy-dev] python egg cache exists error

2012-09-19 Thread Jorrit Boekel
becomes a zombie or something, but since it's a small script, this may do the job. If anyone sees more problems, I'd be happy to know. cheers, jorrit On 09/19/2012 09:16 AM, Jorrit Boekel wrote: For completeness, here's two tracebacks (there were more similar ones) from the sam

Re: [galaxy-dev] DRMAA: TypeError: check_tool_output() takes exactly 5 arguments (4 given)

2012-09-19 Thread Jorrit Boekel
Odd, it works for me on EC2/Cloudman. jorrit On 09/19/2012 03:29 PM, Peter Cock wrote: On Tue, Sep 18, 2012 at 7:11 PM, Scott McManus wrote: Sorry - that's changeset 7714:3f12146d6d81 -Scott Hi Scott, The good news is this error does seem to be fixed as of that commit: TypeError: check_to

[galaxy-dev] pass more information on a dataset merge

2012-10-01 Thread Jorrit Boekel
Dear list, I thought I was working with fairly large datasets, but they have recently started to include ~2Gb files in sets of >50. I have ran these sort of things before as merged data by using tar to roll them up in one set, but when dealing with >100Gb tarfiles, Galaxy on EC2 seems to get

Re: [galaxy-dev] Gestion d'un nombre indéfini de fichiers output / Managment of undefined outputs

2012-10-02 Thread Jorrit Boekel
Hello-, I have had similar problems, and you can possibly solve them by using composite datasets, which allow you to pass files to a directory whilst having a primary file that represents the dataset (I believe Rgenetics uses a primary HTML file containing hyperlinks to the files in the dir).

Re: [galaxy-dev] pass more information on a dataset merge

2012-10-16 Thread Jorrit Boekel
eventually get around to it. -John John Chilton Senior Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223 Bitbucket: https://bitbucket.org/jmchilton Github: https://github.com/jmchilton Web: http://jmchilton.net On Mon, Oc

[galaxy-dev] windows nodes on cloudman/ec2

2012-10-19 Thread Jorrit Boekel
Dear list, Our analysis pipeline is normally fed with proprietary raw file types from mass spectrometry instruments. These can currently only be read on windows systems where vendor DLLs are installed. Since Cloudman/Galaxy does not interface with Windows nodes out of the box, I implemented

[galaxy-dev] the multi job splitter

2012-10-25 Thread Jorrit Boekel
Dear list, In my galaxy fork, I extensively use the job splitters. Sometimes though, I have to split to different file types for the same job. That raises an exception in the lib/galaxy/jobs/splitters/multi.py module. I have turned this behaviour off for my own work, but am now wondering whe

Re: [galaxy-dev] the multi job splitter

2012-10-25 Thread Jorrit Boekel
On 10/25/2012 10:54 AM, Peter Cock wrote: On Thu, Oct 25, 2012 at 9:36 AM, Jorrit Boekel wrote: Dear list, In my galaxy fork, I extensively use the job splitters. Sometimes though, I have to split to different file types for the same job. That raises an exception in the lib/galaxy/jobs

Re: [galaxy-dev] the multi job splitter

2012-10-25 Thread Jorrit Boekel
On 10/25/2012 11:25 AM, Peter Cock wrote: On Thu, Oct 25, 2012 at 10:00 AM, Jorrit Boekel wrote: On 10/25/2012 10:54 AM, Peter Cock wrote: On Thu, Oct 25, 2012 at 9:36 AM, Jorrit Boekel wrote: Dear list, In my galaxy fork, I extensively use the job splitters. Sometimes though, I have to

Re: [galaxy-dev] the multi job splitter

2012-10-25 Thread Jorrit Boekel
On 10/25/2012 12:02 PM, Peter Cock wrote: On Thu, Oct 25, 2012 at 10:35 AM, Jorrit Boekel wrote: My question is still though if it would be bad to not raise an exception when different filetypes are split in the same job. In general splitting multiple files of different types seems dangerous

[galaxy-dev] galaxy/cloudman failure handling

2012-11-12 Thread Jorrit Boekel
etwork problems, etc? If not, would it be hard to implement? cheers, jorrit boekel ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please

Re: [galaxy-dev] Working with files outside galaxy without copying?

2012-11-14 Thread Jorrit Boekel
This may be implemented, I believe some people run Galaxy on the cluster, but if not: Galaxy normally operates on files stored in the database/files folder. If your Galaxy instance has access to the files you need on your parallel fs, I guess you could start by writing a tool that creates links to

Re: [galaxy-dev] Improved Workflows through Multiple File Datasets

2012-12-06 Thread Jorrit Boekel
If one could upvote pull request, I'd be doing that now! thanks loads John, jorrit On 12/05/2012 07:14 PM, John Chilton wrote: Here is my video-based sales pitch for multiple file datasets (specifically pull request 86 and 87 and subsequent future pull requests). http://www.youtube.com/watch?v