Re: [galaxy-dev] Default values for repeat sections

[Bjoern Gruening]

Am Mittwoch, den 16.10.2013, 01:41 -0500 schrieb Eric Rasche:
> Hi Bjoern,
> 
> On 10/16/2013 01:26 AM, Bjoern Gruening wrote:
> > Hi Eric,
> > 
> >> Hello all,
> >>
> >> I've been digging through the available galaxy tool config files and
> >> have been unable to find a solution, so I bring my problem to you.
> >>
> >> I have the following (pretty generic) repeat section in a tool config
> >> I'm working on
> >>
> >> 
> >>  >> label="String of letters that should be matched " />
> >>  >> label="Foreground colour for a group " />
> >>  >> label="Background colour for a group " />
> >>  
> >>
> >> However, for this particular tool, the default ought to be the repeat
> >> section three times, with different defaults each time.
> > 
> > What means different default each time?
> 
> Yes, a different set of default values in each parameter of the three
> repeats.
> 
> I.e., the first instance of a repeat might have a "fg" value of "red",
> the second would have an fg value of green, etc.
> 
> > 
> >> Is there any way to specify a repeat section, with a default of being
> >> repeated N times with an appropriate number of defaults set for each 
> >> repeat?
> > 
> > Try:
> >  for example
> > 
> > 
> 
> Well, for my case I would like our end users to be able to specify less
> than three, so I did wish to have a minimum of one. I just would like
> the parameter to show up on the user interface a default of 3 times with
> the different each time.

Ah ok got it, its to early here and no coffee until now ;)
Unfortunately, I don't think such things are possible for now. Maybe
someone else has an Idea.

Just as a side note, I guess you are developing a visualization tool:
Did you consider to use the new awesome galaxy visualization
infrastructure?

http://wiki.galaxyproject.org/VisualizationsRegistry

Cheers,
Bjoern

> > 
> > Cheers,
> > Bjoern
> > 
> >> Cheers,
> >> Eric Rasche
> >> ___
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>   http://lists.bx.psu.edu/
> >>
> >> To search Galaxy mailing lists use the unified search at:
> >>   http://galaxyproject.org/search/mailinglists/
> > 
> > 
> > 
> 
> Cheers,
> Eric



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Default values for repeat sections

[Eric Rasche]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Bjoern,

On 10/16/2013 01:26 AM, Bjoern Gruening wrote:
> Hi Eric,
> 
>> Hello all,
>>
>> I've been digging through the available galaxy tool config files and
>> have been unable to find a solution, so I bring my problem to you.
>>
>> I have the following (pretty generic) repeat section in a tool config
>> I'm working on
>>
>> 
>> > label="String of letters that should be matched " />
>> > label="Foreground colour for a group " />
>> > label="Background colour for a group " />
>>  
>>
>> However, for this particular tool, the default ought to be the repeat
>> section three times, with different defaults each time.
> 
> What means different default each time?

Yes, a different set of default values in each parameter of the three
repeats.

I.e., the first instance of a repeat might have a "fg" value of "red",
the second would have an fg value of green, etc.

> 
>> Is there any way to specify a repeat section, with a default of being
>> repeated N times with an appropriate number of defaults set for each repeat?
> 
> Try:
>  for example
> 
> 

Well, for my case I would like our end users to be able to specify less
than three, so I did wish to have a minimum of one. I just would like
the parameter to show up on the user interface a default of 3 times with
the different each time.

> 
> Cheers,
> Bjoern
> 
>> Cheers,
>> Eric Rasche
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
> 
> 
> 

Cheers,
Eric
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBAgAGBQJSXjT/AAoJEMqDXdrsMcpVDA0P/iuRwTfxamKgUICvD/jnaUUv
Ewn+l8N4BOaCgjZx603UZt15jo+CJ/MgQby6sAsCQKqnmqm8s957gA668WsjZcyV
cWkuIr9gob4/c4dsQBTbZeD/wOaa5xkj8sZ69AV+AicYHWahr+vfGegVPm2HOzy6
7GijiP1fT+rxtnW61OBfAQWn58glZZxb3JRzjw6FTTmyqsqW3OMcSrVRsOSB8sCY
Ju9cs/Vs299reS3Mh9uCqRMHB1nERMrgB1gd/Zb5ZDGpaxnV7ZhoW7yr//xRw7PE
FQQaY4otiy4y22Fw4KC1ANeq/U08PRb7/3KTxlrtJAZNvXFCkpOhtYO39JQPfxjk
17W1g3stpiz2hngEMdGPL7S2LxQyAUy14LoOVBMzwT7oUvmMbrJAj97B0gyVFCzY
8LIXm/tBT7RChl1nb0Ys38sk6YJvHLwaGWqwPRtlQ2nyymhFxtlq4yfHWOHPfrYB
et/yzdyQizudx9UecdaSzrlyyiifVUkynwVvoKttSnfcZnzyq+tfE+4kSk67/kmQ
2qKoHYwIEiVICv/pS73LUl6Dr4TcQSj+fMdD/SI/3qZH4bhFETpbB5NWNnICcrqd
LRz0lUu96m/NjkfHKudKyTAPyxGvfcBU36FrLhsl77zE8NYaZwsFCOt1jzy224wp
pZkEZ/Tg1ef0/04UduiT
=iPd8
-END PGP SIGNATURE-
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Default values for repeat sections

[Bjoern Gruening]

Hi Eric,

> Hello all,
> 
> I've been digging through the available galaxy tool config files and
> have been unable to find a solution, so I bring my problem to you.
> 
> I have the following (pretty generic) repeat section in a tool config
> I'm working on
> 
> 
>  label="String of letters that should be matched " />
>  label="Foreground colour for a group " />
>  label="Background colour for a group " />
>  
> 
> However, for this particular tool, the default ought to be the repeat
> section three times, with different defaults each time.

What means different default each time?

> Is there any way to specify a repeat section, with a default of being
> repeated N times with an appropriate number of defaults set for each repeat?

Try:
 for example



Cheers,
Bjoern

> Cheers,
> Eric Rasche
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] colaborativ pages

[Philippe Moncuquet]

Hi,

Is there any way to make collaboration on pages possible. I would like user
to be able to modify an existing and publicly shared page when the creator
of the page wants to offer this opportunity. It might be a bit complicated
and out of the scope of Galaxy right now but I wanted to hear what you
think about that ?

Philip
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Default values for repeat sections

[Eric Rasche]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello all,

I've been digging through the available galaxy tool config files and
have been unable to find a solution, so I bring my problem to you.

I have the following (pretty generic) repeat section in a tool config
I'm working on





 

However, for this particular tool, the default ought to be the repeat
section three times, with different defaults each time.

Is there any way to specify a repeat section, with a default of being
repeated N times with an appropriate number of defaults set for each repeat?

Cheers,
Eric Rasche
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBAgAGBQJSXgHgAAoJEMqDXdrsMcpVxUIQAM3UC3CZl//N47YPm+/Y5OVm
Im2A9Y784rf6nk8Guq0wz4UOMSvnpgJnjlDew49Ly6UGgTdpZqBlbv+o0VQlk/Zz
zs90TLVwQ8SMt5x4cSrEF8CJY7y4MlUDCxlaqSeGxmrH7FV3Ph+aRyfhsm32JJx0
duasl4WlJb3jtsBUhMBEQksuhIXTz8SgI0pfTnZ/EIp9ryWNRJiOqeLvCoHq0eNT
othEkHmQn7G/o303fPGOcjpkM7SI+uFDtJLowSYBQeds+nIq9iYgcqeEBe7SmsUJ
o9Fl4/duQ5aztx5Mg858U4y3lWRJoMwK9MxUqii9zacyepWMCrDFo7ZSyBgik5q8
RXgzr/ehifbMfPH04VaOI2rVtmQVlEQUVhok6JnG92V/Pu1tvZr9PQs5csNVl+ov
2xerZjKeUILeMkP2uOwCm4kPVlVwBW0mOMaSPEUQyFdsz/0HKV3h7RDLB7YFDcc8
DxIeHBY3mvBCd30RcniN42HZUes0bq+cg5eFej9zEwZxEinQKzEWRQFtCR+Mt+z9
4mBUE2gDO5OVL+m6iIt+k8lMevKx3cHom9say+jTNKiK1zbXfwqTda4vYYKxfQY8
1Nv/XPs1bIoFPFJJJIkQALeweENsjZ6iY0aJ5kGR4ZLRAMXVt34g5+r2PmXiphbT
eZvA/INKRhwwiPBUf3Hg
=RJCY
-END PGP SIGNATURE-
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Consistently denied server access

[Rhea Datta]

Hi Jennifer,
Thank you for your response.

The server address I have been using is main.g2.bx.psu.edu. I don't see the
new address on the website. Can you let me know what it is?

Thanks again
Rhea


On Tue, Oct 15, 2013 at 2:20 PM, Jennifer Jackson  wrote:

>  Hi Rhea,
>
> Are you using the new server address? This is noted on the upload form: "
> usegalaxy.org"
>
> We will be updating the screencasts and such soon, there are other small
> changes that will go along with this one and we want to capture it all in
> one place. Meanwhile, I'll try to make this more obvious in the current
> wiki.
>
> Good question, I am sure others will read and benefit from it!
>
> If your problem is occurring with the new address, please let us know.
>
> Take care,
>
> Jen
> Galaxy team
>
>
> On 10/14/13 11:23 AM, Rhea Datta wrote:
>
> Hello,
>
>  My account is under rrdatta@gmail.com.
>
>  I have been able to successfully transfer files and performed FTP
> uploads using Filezilla in the past. However in the last few days I have
> been consistently denied server access. The report I get is:
>
>  Status: Connection attempt failed with "ECONNREFUSED - Connection
> refused by server".
>
>  Please let me know if you have a solution to this problem.
> Thank you so much
> Rhea
>
>
>  --
> *Rhea R Datta, PhD*
>  Postdoctoral Associate
> New York University
> r...@nyu.edu
> rrdatta@gmail.com
> (917) 613 4719
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
>
> --
> Jennifer Hillman-Jacksonhttp://galaxyproject.org
>
>


-- 
*Rhea R Datta, PhD*
Postdoctoral Associate
New York University
r...@nyu.edu
rrdatta@gmail.com
(917) 613 4719
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Errors running DRMAA and PBS on remote server running Torque 4

[Ganote, Carrie L]

Hi List,

I've sprouted some grays in the last week after my Galaxy instances all 
simultaneously ceased to submit jobs to our main cluster.

Some Galaxy instances are running the PBS job runner, and others use DRMAA. For 
the DRMAA runner I was getting:
galaxy.jobs.runners ERROR 2013-10-15 08:40:14,942 (1024) Unhandled exception 
calling queue_job
Traceback (most recent call last):
  File "galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 60, in run_next
method(arg)
  File "galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 188, in queue_job
external_job_id = self.ds.runJob(jt)
  File "build/bdist.linux-x86_64/egg/drmaa/__init__.py", line 331, in runJob
_h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate)
  File "build/bdist.linux-x86_64/egg/drmaa/helpers.py", line 213, in c
return f(*(args + (error_buffer, sizeof(error_buffer
  File "build/bdist.linux-x86_64/egg/drmaa/errors.py", line 90, in error_check
raise _ERRORS[code-1]("code %s: %s" % (code, error_buffer.value))
InternalException: code 1: (qsub) cannot access script file: Unauthorized 
Request  MSG=can not authorize request  (0-Success)

And in my PBS runner:
galaxy.jobs.runners.pbs WARNING 2013-10-14 17:13:07,319 (550) pbs_submit failed 
(try 1/5), PBS error 15044: Resources temporarily unavailable

To give some background, I had recently requested a new virtual machine to put 
my test/dev Galaxy on. I copied our production Galaxy to this new VM. I secured 
a new domain name for it and set it running. Everything was going well until I 
tried to hook it up to the cluster; at first I got an error saying that I 
didn't have permission to submit jobs. Makes sense, the new VM was not a 
qualified submit host for the cluster. I asked the sysadmins to add the VM as a 
submit host to the cluster using qmgr. As soon as this was done, not only could 
I still not submit jobs from the test Galaxy, but no Galaxy was able to submit 
jobs to the cluster.

The issue isn't with Galaxy here but the underlying calls that it makes - for 
drmaa, I tracked it back to pbs-drmaa/bin/drmaa-run. For PBS, I'm sure it's 
somewhere in with libtorque. In every case, I could call qsub from the command 
line and it would correctly submit jobs, which was more perplexing.

I re-installed python, drmaa.egg, pbs-drmaa, and rebooted the VM. I of course 
restarted Galaxy with each step, to no avail. I worked with the admins to see 
what was happening in the server logs, but the same cryptic error showed up - 
cannot authorize request. I've had this issue before in the past, more or less, 
but usually just gave up on it. It seemed to come and go sporadically, but 
rebooting the clusters seemed to help.

This time, with our production server no longer functioning, I begged for help 
and the admins looked through the pbs_server config but couldn't find any 
mistypes or problems. Reloading the config by sending hangup signals to 
pbs_server didn't help. Then we tried pausing the scheduler and restarting 
pbs_server completely - and eureka, all problems went away. PBS and DRMAA 
runners are back up and working fine. This really seems to be a bug in Torque 
4.1.5.1.

I hope this saves someone a lot of headache! Newer versions of Torque may be 
the answer. I would also advise against making changes to the pbs_server 
configuration while in production - we have monthly maintenance, and I don't 
think I'll ever request changes when there won't be an immediate reboot to 
flush the server!

Cheers,

Carrie
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] launching CloudMan 2.0 on AWS

[Vipin TS]

Hi Enis,

Thank you for the information! I saw many GalaxyIndicesFS are accumulated
on my account and cleaned up.

Later I tried to launch cloudman ami, a different message I saw in
paster.log:

18:35:32 - Master starting
18:35:33 - Successfully retrieved root user's public key from file.
18:35:34 - Trouble getting volume reference for volume vol-f9c4728e:
EC2ResponseError: 400 Bad Request InvalidVolume.NotFoundThe volume
'vol-f9c4728e' does not exist.c69dcb4e-f924-4cbc-9c35-ef630bd18ff2
18:35:34 - Attempting to connect to a non-existent volume vol-f9c4728e
18:35:34 - Error processing filesystems in existing cluster
configuration: 'NoneType' object has no attribute '__getitem__'
18:35:34 - * Manager failed to start *
18:37:14 - Initializing 'Galaxy' cluster type. Please wait...

The "vol-f9c4728e" name I can see from the instance (ssh to the instance)
is encoded in /mnt/cm/pd.yaml file.
In my account I have seen the new volumes are created after launching the
cloudman they are having different id's:
vol-06992971 cm-3102c4d1aa831d8da4e99f3c9896390c GalaxyTools
vol-05992972 cm-3102c4d1aa831d8da4e99f3c9896390c GalaxyIndices

The df:
Filesystem  Size  Used Avail Use% Mounted on
/dev/xvda1  7.9G  5.2G  2.3G  70% /
udev3.7G  8.0K  3.7G   1% /dev
tmpfs   1.5G  228K  1.5G   1% /run
none5.0M 0  5.0M   0% /run/lock
none3.7G 0  3.7G   0% /run/shm
cgroup  3.7G 0  3.7G   0% /sys/fs/cgroup
/dev/xvdb   414G  203M  393G   1% /mnt

I think I have to manually mount the new volumes to the instance then edit
the configuration and restart the cludman ?

or are you seeing something wrong from my side.

Thanks in advance,
Vipin
http://galaxy.cbio.mskcc.org

Hi Vipin,
> It seems there are too many volumes (or too much volume storage) in your
> AWS account and AWS is preventing any more from being created. You can
> either delete some volumes if you don't need them or send a request to AWS
> to increase your volume limit:
> https://aws.amazon.com/contact-us/ebs_volume_limit_request/
>
> Cheers,
> Enis
>
>
>>
>> On Mon, Oct 14, 2013 at 10:06 PM, Vipin TS  wrote:
>>
>>> Hi dev-team,
>>>
>>> I am trying to launch cloudman on AWS us-east-1a and I am getting the
>>> message
>>>   *All cluster services started; the cluster is ready for use.
>>> (2013-10-14 19:45:52)*
>>>
>>> I am seeing some error message in the log.
>>> * 19:05:33 - Master starting*
>>> *19:05:35 - Could not find service class matching userData service
>>> entry: PSS*
>>> *19:05:35 - Completed the initial cluster startup process. This is
>>> a new cluster; waiting to configure the type.*
>>> *19:06:11 - Migration service prerequisites OK; starting the service
>>> *
>>> *19:06:11 - SGE service prerequisites OK; starting the service*
>>> *19:06:26 - Setting up SGE...*
>>> *19:06:40 - HTCondor service prerequisites OK; starting the service*
>>> *19:06:48 - Hadoop service prerequisites OK; starting the service*
>>> *19:07:04 - Done adding Hadoop service; service running.*
>>> *19:08:13 - Initializing 'Galaxy' cluster type. Please wait...*
>>> *19:08:14 - Error creating volume: EC2ResponseError: 400 Bad
>>> Request VolumeLimitExceededMaximum number of active volumes bytes, 20,
>>> exceeded.66659429-b0dd-4f88-b763-36fd9232d3c4*
>>> *19:08:17 - Adding volume vol-f9c4728e (FS object for galaxy)...*
>>> *19:08:42 - Successfully grew file system FS object for galaxy*
>>> *19:08:42 - Successfully mounted file system /mnt/galaxy from
>>> /dev/xvdf*
>>>
>>> It seems that the galaxyIndices filesytem is not mounted to the
>>> instance. Also I am not able to start the postgres or Galaxy service
>>> through cloudman interface, both says that Cannot find the service.
>>>
>>> Please find the attached image. I think I am missing the galaxyIndices
>>> volume from the image,. any idea about the failure to launch the instance.
>>>
>>>
>>> Thanks,
>>>  Vipin
>>> http://galaxy.cbio.mskcc.org
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   http://lists.bx.psu.edu/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Deploying LOC files for tool built-in data during a tool installation

[Daniel Blankenberg]

Hi all,

I think what we have are two similar, but somewhat separate problems:
1.) We need a way via the UI for an admin to be able to add additional 
configuration entries to data tables / .loc files.
2.) We need a way to bootstrap/initialize a Galaxy installation with data 
table/ .loc file entries ('built-in data') during installation for 
a.) a 'production' Galaxy instance - this would include local 
dev/testing/etc instances
b.) automated testing framework - tests should run fast, but 
meaningfully test a tool, e.g., the horse mitochondrial genome could be a fine 
built-in genome for running automated tool tests, but not desired to be 
automatically installed into a production Galaxy instance


For 1.), we now have Data Managers. A Data Manager will do all the heavy 
lifting of adding additional data table entries. e.g. for bwa, it can build the 
mapping indexes and add the properly delimited line to the .loc file. These are 
accessed through the admin interface, under Manage local data. Data Managers 
are installed from a ToolShed, or can be installed manually. In addition to 
direct interactive usage, Data Manager tools can be included in workflows or 
accessed via the tools API. Not only does the use of a Data Manager remove the 
technical burdens/concerns of adding new entries to a data table / .loc file, 
it also provides for the same reproducibility and provenance tracking that is 
afforded to regular Galaxy tools. The documentation for Data Managers is 
currently limited to the tutorial-style doc here: 
http://wiki.galaxyproject.org/Admin/Tools/DataManagers/HowTo/Define; a more 
formal / config syntax type of page will also be made available, although the 
tutorial is a !
 pretty inclusive description of the steps needed to define a Data Manager.


For 2.): bootstrapping data during an installation process is something that 
still needs to be more completely spec'd out and implemented. This 
bootstrapping process should be able to make use of the Data Managers or 
download/move/utilize pre-built configurations. (A Data Manager itself can have 
its underlying actions being a downloading process, e.g. the fetch genomes data 
manager)

Lets start by considering the Users' point of view. We have 2 types of users: 
GalaxyAdmin and ToolDev and use a BWA tool as an example.
GalaxyAdmin: 
Clicks buttons to install tool suite that includes the BWA tool and a 
BWA indexer Data Manager. (so far there is no change from how it works now)
The Galaxy installer methodology recognizes that it is possible to add 
built-in data:
Some preassembled mapping indexes are available (pre-built 
built-in)
Mapping indexes can be created for any entry in the all_fasta 
data table.
The User clicks checkboxes/multiple selects for preassembled data to 
download and also selects the fasta entries to be indexed with the Data Manager 
tool.

ToolDev:
In ToolShed repository, needs to provide a description of a and b; for 
simplicity we can assume b is a subset of a, but with a different 
attribute/flag (e.g. test_only, 'real', both) or perhaps a different filename; 
abstractly, they are the same thing just run at different times with the 
testing ones not requiring user interaction/selection.


So, the real question becomes, what does this description look like? It is 
probably an XML file, for now lets call it  '__data_table_bootstrap__.xml' 
(alternatively, we can roll it directly into the existing data_manager_conf.xml 
files in the toolshed, although for a list of static downloads, we don't need 
an actual data manager tool). It could look something like this (quick and 
dirty pass, elements and values are made up):

 

 
 

 
 

 

Re: [galaxy-dev] Consistently denied server access

[Jennifer Jackson]

Hi Rhea,

Are you using the new server address? This is noted on the upload form: 
"usegalaxy.org"


We will be updating the screencasts and such soon, there are other small 
changes that will go along with this one and we want to capture it all 
in one place. Meanwhile, I'll try to make this more obvious in the 
current wiki.


Good question, I am sure others will read and benefit from it!

If your problem is occurring with the new address, please let us know.

Take care,

Jen
Galaxy team

On 10/14/13 11:23 AM, Rhea Datta wrote:

Hello,

My account is under rrdatta@gmail.com .

I have been able to successfully transfer files and performed FTP 
uploads using Filezilla in the past. However in the last few days I 
have been consistently denied server access. The report I get is:


Status:Connection attempt failed with "ECONNREFUSED - Connection 
refused by server".


Please let me know if you have a solution to this problem.
Thank you so much
Rhea


--
*Rhea R Datta, PhD*
Postdoctoral Associate
New York University
r...@nyu.edu 
rrdatta@gmail.com 
(917) 613 4719


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


--
Jennifer Hillman-Jackson
http://galaxyproject.org

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Test Toolshed Biopython package dependency & NumPy

[Bjoern Gruening]

Hi Dave,

thanks for the pointer.

I updated the testtoolshed package. For me installing fine. I will wait
one more day and have a look at Peter's Testcases. I will than migrate
it to the main toolshed. Changes are now also in github.

Thanks,
Björn

> Björn, Peter,
> 
> In the course of my investigation, I've discovered a possible 
> improvement to the lapack build recipe. If you change the cmake command 
> from:
> 
> cmake .. -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR/lapack
> 
> to:
> 
> cmake .. -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR/lapack 
> -DCMAKE_Fortran_FLAGS='-O2 -fPIC'
> 
> the need for the earlier sed command is eliminated, which may reduce the 
> likelihood of environment-specific failures. Specifically, I attempted 
> to install get_orfs_or_cdss on a local (linux) system, and the numpy 
> installation's error message informed me that lapack was built without 
> the -fPIC flag.
> 
> 
> --Dave B.
> 
> On 10/11/2013 04:51 AM, Peter Cock wrote:
> > On Thu, Oct 10, 2013 at 10:00 AM, Peter Cock  
> > wrote:
> >> On Wed, Oct 9, 2013 at 10:01 PM, Björn Grüning
> >>  wrote:
> >>> Am Mittwoch, den 09.10.2013, 19:15 +0100 schrieb Peter Cock:
>  On Tue, Oct 8, 2013 at 9:57 PM, Dave Bouvier  wrote:
> > Peter, Björn,
> >
> > I have restored gfortran to the repository test host. I believe it was
> > inadvertently removed, possibly as a side effect of updating packages or
> > removing packages that were causing conflicts with the Galaxy eggs. 
> > Numpy
> > and biopython should now install successfully, as they do in my local
> > environment.
> >
> > --Dave B.
> 
>  Thanks Dave, fingers crossed for tonight's test run :)
> 
>  Bjoern - it looks like we can leave the NumPy definition as is...
> >>>
> >>> Ah sorry, totally forgot about it :(
> >>>
> >>
> >> Progress, Fortran is working and lapack seems to have installed, but:
> >> e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename
> >>
> >> Installation errors - no functional tests were run for any tools in
> >> this changeset revision
> >> Tool dependencies
> >> TypeNameVersion
> >> biopython package 1.62
> >> Error
> >> running install Numerical Python (NumPy) is not installed. This
> >> package is required for many Biopython features. Please install it
> >> before you install Biopython. You can install Biopython anyway, but
> >> anything dependent on NumPy will not work. If you do this, and later
> >> install NumPy, you should then re-install Biopython. You can find
> >> NumPy at http://numpy.scipy.org
> >> TypeNameVersion
> >> numpy package 1.7.1
> >> Error
> >> Running from numpy source directory.
> >> /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmp3kNwuR/tmp-toolshed-mtdGJ0d5H/numpy-1.7.1/numpy/distutils/system_info.py:1494:
> >> UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not
> >> found. Directories to search for the libraries can be specified in the
> >> numpy/distutils/site.cfg file (section [atlas]) or by setting the
> >> ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__)
> >> /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmp3kNwuR/tmp-toolshed-mtdGJ0d5H/numpy-1.7.1/numpy/distutils/system_info.py:1408:
> >> UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not
> >> found. Directories to search for the libraries can be specified in the
> >> numpy/distutils/site.cfg file (section [atlas]) or by setting the
> >> ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__)
> >> error: Command "/usr/bin/gfortran -Wall -Wall -shared
> >> build/temp.linux-x86_64-2.7/numpy/linalg/lapack_litemodule.o
> >> build/temp.linux-x86_64-2.7/numpy/linalg/python_xerbla.o
> >> -L/ToolDependencies/lapack/3.4.2/iuc/package_lapack_3_4/60957bd68fe2/lapack/lib
> >> -Lbuild/temp.linux-x86_64-2.7 -llapack -lblas -lgfortran -o
> >> build/lib.linux-x86_64-2.7/numpy/linalg/lapack_lite.so" failed with
> >> exit status 1
> >>
> >>
> >> Most of that is warnings about not using ATLAS (which we
> >> expect) but I'm not quite sure why it fails...
> >
> > Hi Guys,
> >
> > Bjoern - good news for you, the installation of NumPy can
> > work on the current Test Tool Shed.
> >
> > Dave - bad news for you, last night I had 3 NumPy install failures
> > (all different) and 2 successes - which to me suggests a possible
> > race condition in the nightly testing?
> >
> > (1) Last night on the Test Tool Shed this failed as in email above:
> >
> > http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_primer_clip/834cf6618a14
> >
> > (2) This failed with the simpler message:
> >
> > http://testtoolshed.g2.bx.psu.edu/view/peterjc/get_orfs_or_cdss/308ec20faefb
> >
> > Installation errors - no functional tests were run for any tools in
> > this changeset revision
> > Tool dependencies
> > TypeNam

Re: [galaxy-dev] Fastx toolkit broken pipe

[UMD Bioinformatics]

Carlos,

Thank you very much. What I ended up doing to fix this issue was to uninstall 
fastx_toolkit from the tool shed then I created a .bashrc file for galaxy. I 
added this to the environmental_setup_file path and restarted galaxy. This was 
the fix I needed. Things seem to be working now. Here is the contents of my 
.bashrc file for others having issues.

export PATH=${PATH}:/usr/bin:/usr/local/bin:/bin:/opt/local/bin:/opt/local/sbin

I placed the .bashrc file in the galaxy-dist folder and source it from there. 

Cheers
Ian

On Oct 15, 2013, at 9:53 AM, Carlos Borroto  wrote:

> On Mon, Oct 14, 2013 at 9:20 PM, UMD Bioinformatics
>  wrote:
>> On Oct 14, 2013, at 9:16 PM, Brad Chapman  wrote:
>> 
>>> 
>>> Ian;
>>> 
 I have my PATH set as well as the env.sh and my ./bash_profile. Again
 this all works fine from the command line. I don't know where else
 Galaxy is trying to find the fastx commands.
>>> 
>>> I know zero about setting up fastx_toolkit from the toolshed but from
>>> reading your error messages:
>>> 
 command not found cat: stdout: Broken pipe
 command not found gzip: stdout: Broken pipe/
>>> 
>>> It looks like it's not having trouble finding fastx_toolkit, but rather
>>> standard unix commands 'cat' and 'gzip'. Is it possible the PATH you're
>>> feeding to Galaxy somehow avoids include `/bin` and `/usr/bin` (or
>>> wherever cat and gzip are on your system)?
>> 
>> This is what I have for cat and gzip
>> 
>> $ which cat
>> /bin/cat
>> $ which gzip
>> /usr/bin/gzip
>> $ echo $PATH
>> /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:
>> 
>>> From what i can tell they should be available on the $PATH so I'm not quite 
>>> sure why Galaxy cannot "see" them.
>> 
> 
> Hi Ian,
> 
> In the past I had trouble making sure Galaxy would run jobs with the
> proper environment settings. I think there are a few moving parts
> depending on your local configuration(ex. local vs cluster jobs). All
> of my problems went away after I started using something like this in
> my 'universe_wsgi.ini':
> environment_setup_file = /local/opt/galaxy/.bashrc
> 
> Any env configuration I want Galaxy to use, I put it in that bashrc file.
> 
> Hope it helps,
> Carlos


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] /dev/xvda1 filling up when downloading large files via ftp

[Dannon Baker]

And I can answer the first part;  this is a known issue we're trying to
nail down and get a fix out for this week, along with a base volume update
to address other issues.  Trello card: https://trello.com/c/8XSIeoG0


On Tue, Oct 15, 2013 at 10:23 AM, Peter Cock wrote:

> On Tue, Oct 15, 2013 at 10:13 AM, Stanislas Werfel
>  wrote:
> >
> > I'm also having another issue: when I delete files in the history in
> galaxy,
> > they don't acually get deleted in the files/000/ folder. And galaxy is
> still
> > showing them under "Using ..." in the total space used. So if I download
> > a 3GB fastq file, then delete everything in the history, Using is still
> showing
> > 3GB and the file is still there in the folder.
>
> I can't help with the first problem, but I believe the second is easier:
> The data isn't immediately deleted and therefore your usage figure
> (and quota) do not get updated right away. This is usually done via a
> house keeping cron job, see:
>
>
> http://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%20and%20Datasets
>
> Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Test Toolshed Biopython package dependency & NumPy

[Dave Bouvier]

Björn, Peter,

In the course of my investigation, I've discovered a possible 
improvement to the lapack build recipe. If you change the cmake command 
from:


cmake .. -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR/lapack

to:

cmake .. -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR/lapack 
-DCMAKE_Fortran_FLAGS='-O2 -fPIC'


the need for the earlier sed command is eliminated, which may reduce the 
likelihood of environment-specific failures. Specifically, I attempted 
to install get_orfs_or_cdss on a local (linux) system, and the numpy 
installation's error message informed me that lapack was built without 
the -fPIC flag.



   --Dave B.

On 10/11/2013 04:51 AM, Peter Cock wrote:

On Thu, Oct 10, 2013 at 10:00 AM, Peter Cock  wrote:

On Wed, Oct 9, 2013 at 10:01 PM, Björn Grüning
 wrote:

Am Mittwoch, den 09.10.2013, 19:15 +0100 schrieb Peter Cock:

On Tue, Oct 8, 2013 at 9:57 PM, Dave Bouvier  wrote:

Peter, Björn,

I have restored gfortran to the repository test host. I believe it was
inadvertently removed, possibly as a side effect of updating packages or
removing packages that were causing conflicts with the Galaxy eggs. Numpy
and biopython should now install successfully, as they do in my local
environment.

--Dave B.


Thanks Dave, fingers crossed for tonight's test run :)

Bjoern - it looks like we can leave the NumPy definition as is...


Ah sorry, totally forgot about it :(



Progress, Fortran is working and lapack seems to have installed, but:
e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename

Installation errors - no functional tests were run for any tools in
this changeset revision
Tool dependencies
TypeNameVersion
biopython package 1.62
Error
running install Numerical Python (NumPy) is not installed. This
package is required for many Biopython features. Please install it
before you install Biopython. You can install Biopython anyway, but
anything dependent on NumPy will not work. If you do this, and later
install NumPy, you should then re-install Biopython. You can find
NumPy at http://numpy.scipy.org
TypeNameVersion
numpy package 1.7.1
Error
Running from numpy source directory.
/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmp3kNwuR/tmp-toolshed-mtdGJ0d5H/numpy-1.7.1/numpy/distutils/system_info.py:1494:
UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not
found. Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting the
ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__)
/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmp3kNwuR/tmp-toolshed-mtdGJ0d5H/numpy-1.7.1/numpy/distutils/system_info.py:1408:
UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not
found. Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting the
ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__)
error: Command "/usr/bin/gfortran -Wall -Wall -shared
build/temp.linux-x86_64-2.7/numpy/linalg/lapack_litemodule.o
build/temp.linux-x86_64-2.7/numpy/linalg/python_xerbla.o
-L/ToolDependencies/lapack/3.4.2/iuc/package_lapack_3_4/60957bd68fe2/lapack/lib
-Lbuild/temp.linux-x86_64-2.7 -llapack -lblas -lgfortran -o
build/lib.linux-x86_64-2.7/numpy/linalg/lapack_lite.so" failed with
exit status 1


Most of that is warnings about not using ATLAS (which we
expect) but I'm not quite sure why it fails...


Hi Guys,

Bjoern - good news for you, the installation of NumPy can
work on the current Test Tool Shed.

Dave - bad news for you, last night I had 3 NumPy install failures
(all different) and 2 successes - which to me suggests a possible
race condition in the nightly testing?

(1) Last night on the Test Tool Shed this failed as in email above:

http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_primer_clip/834cf6618a14

(2) This failed with the simpler message:

http://testtoolshed.g2.bx.psu.edu/view/peterjc/get_orfs_or_cdss/308ec20faefb

Installation errors - no functional tests were run for any tools in
this changeset revision
Tool dependencies
TypeNameVersion
biopython package 1.62
Error
running install Numerical Python (NumPy) is not installed. This
package is required for many Biopython features. Please install it
before you install Biopython. You can install Biopython anyway, but
anything dependent on NumPy will not work. If you do this, and later
install NumPy, you should then re-install Biopython. You can find
NumPy at http://numpy.scipy.org

(3) Here the installation seems to have failed silently, leading to a
test failure at the import stage:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_select_by_id/9e9e3b860aa0

Tests that failed
Tool id: seq_select_by_id
Tool version: seq_select_by_id
Test: test_tool_00
(functional.test_toolbox.TestForTo

Re: [galaxy-dev] /dev/xvda1 filling up when downloading large files via ftp

[Peter Cock]

On Tue, Oct 15, 2013 at 10:13 AM, Stanislas Werfel
 wrote:
>
> I'm also having another issue: when I delete files in the history in galaxy,
> they don't acually get deleted in the files/000/ folder. And galaxy is still
> showing them under "Using ..." in the total space used. So if I download
> a 3GB fastq file, then delete everything in the history, Using is still 
> showing
> 3GB and the file is still there in the folder.

I can't help with the first problem, but I believe the second is easier:
The data isn't immediately deleted and therefore your usage figure
(and quota) do not get updated right away. This is usually done via a
house keeping cron job, see:

http://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%20and%20Datasets

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] launching CloudMan 2.0 on AWS

[Enis Afgan]

Hi Vipin,
It seems there are too many volumes (or too much volume storage) in your
AWS account and AWS is preventing any more from being created. You can
either delete some volumes if you don't need them or send a request to AWS
to increase your volume limit:
https://aws.amazon.com/contact-us/ebs_volume_limit_request/

Cheers,
Enis


>
> On Mon, Oct 14, 2013 at 10:06 PM, Vipin TS  wrote:
>
>> Hi dev-team,
>>
>> I am trying to launch cloudman on AWS us-east-1a and I am getting the
>> message
>>   *All cluster services started; the cluster is ready for use.
>> (2013-10-14 19:45:52)*
>>
>> I am seeing some error message in the log.
>> * 19:05:33 - Master starting*
>> *19:05:35 - Could not find service class matching userData service
>> entry: PSS*
>> *19:05:35 - Completed the initial cluster startup process. This is a
>> new cluster; waiting to configure the type.*
>> *19:06:11 - Migration service prerequisites OK; starting the service*
>> *19:06:11 - SGE service prerequisites OK; starting the service*
>> *19:06:26 - Setting up SGE...*
>> *19:06:40 - HTCondor service prerequisites OK; starting the service*
>> *19:06:48 - Hadoop service prerequisites OK; starting the service*
>> *19:07:04 - Done adding Hadoop service; service running.*
>> *19:08:13 - Initializing 'Galaxy' cluster type. Please wait...*
>> *19:08:14 - Error creating volume: EC2ResponseError: 400 Bad Request
>> VolumeLimitExceededMaximum number of active volumes bytes, 20,
>> exceeded.66659429-b0dd-4f88-b763-36fd9232d3c4*
>> *19:08:17 - Adding volume vol-f9c4728e (FS object for galaxy)...*
>> *19:08:42 - Successfully grew file system FS object for galaxy*
>> *19:08:42 - Successfully mounted file system /mnt/galaxy from
>> /dev/xvdf*
>>
>> It seems that the galaxyIndices filesytem is not mounted to the instance.
>> Also I am not able to start the postgres or Galaxy service through cloudman
>> interface, both says that Cannot find the service.
>>
>> Please find the attached image. I think I am missing the galaxyIndices
>> volume from the image,. any idea about the failure to launch the instance.
>>
>>
>> Thanks,
>>  Vipin
>> http://galaxy.cbio.mskcc.org
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] /dev/xvda1 filling up when downloading large files via ftp

[Stanislas Werfel]

Dear all,

this problem has been mentioned before by mcdow...@vims.ed, but there was no 
solution so far posted. 
I'm using the current cloudman on Amazon AWS. When I download a fastq.gz file 
via an EBI-SRA ftp link using the "Upload File" function, although the file 
itself is loading to the /mnt/galaxy drive, at the same time /dev/xvda1 is 
filling up and at some point the download stops and says "[Errno 28] No space 
left on device". 
I've checked in the universe_wsgi.ini and the ftp upload path is set correctly 
to /mnt/galaxy/tmp/ftp.

Here's the df output:
Filesystem 1K-blocks  Used Available Use% Mounted on
/dev/xvda1  10321208   9797012 0 100% /
udev 8736124 8   8736116   1% /dev
tmpfs3497960   232   3497728   1% /run
none5120 0  5120   0% /run/lock
none 8744896 0   8744896   0% /run/shm
cgroup   8744896 0   8744896   0% /sys/fs/cgroup
/dev/xvdb  423135208207440 401433728   1% /mnt
/dev/xvdf  104847360  22055572  82791788  22% /mnt/galaxy
/dev/xvdg  733900800 685188288  48712512  94% /mnt/galaxyIndices

I'm also having another issue: when I delete files in the history in galaxy, 
they don't acually get deleted in the files/000/ folder. And galaxy is still 
showing them under "Using ..." in the total space used. So if I download a 3GB 
fastq file, then delete everything in the history, Using is still showing 3GB 
and the file is still there in the folder. 

Could the two issues be connected? Are there maybe some rights missing for the 
galaxy instance to delete files?

Thanks in advance for any help
Best
Stanislas






___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Fastx toolkit broken pipe

[Carlos Borroto]

On Mon, Oct 14, 2013 at 9:20 PM, UMD Bioinformatics
 wrote:
> On Oct 14, 2013, at 9:16 PM, Brad Chapman  wrote:
>
>>
>> Ian;
>>
>>> I have my PATH set as well as the env.sh and my ./bash_profile. Again
>>> this all works fine from the command line. I don't know where else
>>> Galaxy is trying to find the fastx commands.
>>
>> I know zero about setting up fastx_toolkit from the toolshed but from
>> reading your error messages:
>>
>>> command not found cat: stdout: Broken pipe
>>> command not found gzip: stdout: Broken pipe/
>>
>> It looks like it's not having trouble finding fastx_toolkit, but rather
>> standard unix commands 'cat' and 'gzip'. Is it possible the PATH you're
>> feeding to Galaxy somehow avoids include `/bin` and `/usr/bin` (or
>> wherever cat and gzip are on your system)?
>
> This is what I have for cat and gzip
>
> $ which cat
> /bin/cat
> $ which gzip
> /usr/bin/gzip
> $ echo $PATH
> /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:
>
> >From what i can tell they should be available on the $PATH so I'm not quite 
> >sure why Galaxy cannot "see" them.
>

Hi Ian,

In the past I had trouble making sure Galaxy would run jobs with the
proper environment settings. I think there are a few moving parts
depending on your local configuration(ex. local vs cluster jobs). All
of my problems went away after I started using something like this in
my 'universe_wsgi.ini':
environment_setup_file = /local/opt/galaxy/.bashrc

Any env configuration I want Galaxy to use, I put it in that bashrc file.

Hope it helps,
Carlos
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Occasional job failure with /scripts/set_metadata.py ElementTree.parse(fname)

[Peter Cock]

Hi all,

Running galaxy-central default branch, I've seen a couple of cases where
what otherwise appears to be a successful job has failed with:

Traceback (most recent call last):
  File "./scripts/set_metadata.py", line 122, in 
__main__()
  File "./scripts/set_metadata.py", line 72, in __main__
datatypes_registry.load_datatypes( root_dir=config_root,
config=datatypes_config )
  File "/mnt/galaxy/galaxy-central/lib/galaxy/datatypes/registry.py",
line 67, in load_datatypes
tree = galaxy.util.parse_xml( config )
  File "/mnt/galaxy/galaxy-central/lib/galaxy/util/__init__.py", line
135, in parse_xml
tree = ElementTree.parse(fname)
  File 
"/mnt/galaxy/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py",
line 859, in parse
  File 
"/mnt/galaxy/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py",
line 576, in parse
IOError: [Errno 2] No such file or directory:
'/mnt/galaxy/galaxy-central/database/tmp/tmpWJZ1Aw'

I'm not quite sure what was in the temporary XML file which
no longer exists, perhaps there is a race condition with the
removal of the temporary files?

Has anyone else observed this?

Thanks,

Peter

One minor change which might help give a clearer error message
here (and in related issues like the functional tests where not all
the tools listed in tool_conf.xml.sample always exists on disk) is:

$ hg diff /mnt/galaxy/galaxy-central/lib/galaxy/util/__init__.py
diff -r 01b94ad2c377 lib/galaxy/util/__init__.py
--- a/lib/galaxy/util/__init__.pyTue Oct 15 00:12:08 2013 -0400
+++ b/lib/galaxy/util/__init__.pyTue Oct 15 14:20:01 2013 +0100
@@ -132,6 +132,8 @@

 def parse_xml(fname):
 """Returns a parsed xml tree"""
+if not os.path.isfile(fname):
+raise IOError("Asked to parse non-existant XML file %r" % fname)
 tree = ElementTree.parse(fname)
 root = tree.getroot()
 ElementInclude.include(root)
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] /dev/xvda1 filling up when downloading large files via ftp

[Stanislas Werfel]

Dear all,

this problem has been mentioned before by mcdow...@vims.ed, but there was no 
solution so far posted. 
I'm using the current cloudman on Amazon AWS. When I download a fastq.gz file 
via an EBI-SRA ftp link using the "Upload File" function, although the file 
itself is loading to the /mnt/galaxy drive, at the same time /dev/xvda1 is 
filling up and at some point the download stops and says "[Errno 28] No space 
left on device". 
I've checked in the universe_wsgi.ini and the ftp upload path is set correctly 
to /mnt/galaxy/tmp/ftp.

Here's the df output:
Filesystem 1K-blocks  Used Available Use% Mounted on
/dev/xvda1  10321208   9797012 0 100% /
udev 8736124 8   8736116   1% /dev
tmpfs3497960   232   3497728   1% /run
none5120 0  5120   0% /run/lock
none 8744896 0   8744896   0% /run/shm
cgroup   8744896 0   8744896   0% /sys/fs/cgroup
/dev/xvdb  423135208207440 401433728   1% /mnt
/dev/xvdf  104847360  22055572  82791788  22% /mnt/galaxy
/dev/xvdg  733900800 685188288  48712512  94% /mnt/galaxyIndices

I'm also having another issue: when I delete files in the history in galaxy, 
they don't acually get deleted in the files/000/ folder. And galaxy is still 
showing them under "Using ..." in the total space used. So if I download a 3GB 
fastq file, then delete everything in the history, Using is still showing 3GB 
and the file is still there in the folder. 

Could the two issues be connected? Are there maybe some rights missing for the 
galaxy instance to delete files?

Thanks in advance for any help
Best
Stanislas




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] about the sqlalchemy-migrate

[Hans-Rudolf Hotz]

Hi "ngsf...@hygenomics.com"

You ask four different questions. I strongly recommend you ask them in 
four different mails each with an appropriate subject line, next time.

This will make it much easier for the community to help you.
Also, copy paste the logs and don't use screenshots, as they will get 
lost in the reply.


On 10/14/2013 10:08 AM, ngsf...@hygenomics.com wrote:> hi,all:
> 1. when I started the galaxy daemon. In the paster.log has a
> warning.who can help me. url= 
http://code.google.com/p/sqlalchemy-migrate/


When you start galaxy a second time, do you still get the error?
Can you manually fetch the egg?

>  2. missing file.

It is just a warning: You don't provide the index files. Do you really 
need all those tools?


This page will probably help you:

http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup

>  3.What should I do about it?

You don't provide an "job_conf.xml" file. It still works, as explained 
in the warning message. Altenatively, have a look at


~/galaxy_dist/job_conf.xml.sample_advanced
~/galaxy_dist/job_conf.xml.sample_basic


>  4.What should I do about it?

Again, just warnings. You need to investigate the individual tool xml files


Hope this helps

Regards, Hans-Rudolf



> 
> ngsf...@hygenomics.com
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/