[galaxy-dev] Executing a binary file from Galaxy job

[Roger Ngo]

Hi, I have a perl script which executes a Java JAR file located in
tool-data/shared/jars/ and this Jar file at one point executes 2 other
binary programs during its job process. These 2 programs are located in the
same directory as the JAR file.

I am getting errors in Galaxy saying "No such file or directory" when these
2 external programs are called. I am assuming this is the case because they
are not located in current working directory of the current Galaxy job
process.

Is there a way to somehow redirect Galaxy into the correct directory where
the binary files are located?

Thanks.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] New tool not recognised unless i reboot

[Daniel Blankenberg]

Hi Makis,

Ordinarily in interactive mode, ctrl+c should work. However, when just using 
'run.sh' certain python/OS combinations experience issues. You'll need to 
manually kill your instance, e.g. using 'kill '. Next time you start 
Galaxy you can use 'run.sh --reload' which should enable interactive usage with 
ctrl-c working. 

You can also stat Galaxy daemonized using 'sh run.sh --daemon'; stop with sh 
run.sh --stop-daemon'.


Thanks for using Galaxy,

Dan


On Dec 14, 2011, at 3:36 PM, Makis Ladoukakis wrote:

> I couldn't find a way to stop the instance through the terminal. Ctrl+C 
> doesn't work so i just close the terminal itself. Is there a command that 
> stops the instance?
> 
> > Date: Wed, 14 Dec 2011 17:32:41 +
> > Subject: Re: [galaxy-dev] New tool not recognised unless i reboot
> > From: p.j.a.c...@googlemail.com
> > To: makis4e...@hotmail.com
> > CC: galaxy-dev@lists.bx.psu.edu
> > 
> > 2011/12/14 Makis Ladoukakis :
> > > Dear galaxy users,
> > >
> > > I have downloaded a local instance of Galaxy on my linux distribution
> > > (ubuntu) and i am adding new tools i made myself. To add a new tool i 
> > > follow
> > > all the steps (tool definition file and the modification of tool_conf.xml)
> > > and run 'sh run.sh' for Galaxy to include it in its interface. 
> > > Nevertheless
> > > when i try to do the same later after i close Galaxy, when i run 'sh 
> > > run.sh'
> > > again i get this error:
> > >
> > > socket.error: [Errno 98] Address already in use
> > 
> > Did you stop the old Galaxy instance which is still running and
> > using that network address?
> > 
> > Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] New tool not recognised unless i reboot

[Makis Ladoukakis]

I couldn't find a way to stop the instance through the terminal. Ctrl+C doesn't 
work so i just close the terminal itself. Is there a command that stops the 
instance?

> Date: Wed, 14 Dec 2011 17:32:41 +
> Subject: Re: [galaxy-dev] New tool not recognised unless i reboot
> From: p.j.a.c...@googlemail.com
> To: makis4e...@hotmail.com
> CC: galaxy-dev@lists.bx.psu.edu
> 
> 2011/12/14 Makis Ladoukakis :
> > Dear galaxy users,
> >
> > I have downloaded a local instance of Galaxy on my linux distribution
> > (ubuntu) and i am adding new tools i made myself. To add a new tool i follow
> > all the steps (tool definition file and the modification of tool_conf.xml)
> > and run 'sh run.sh' for Galaxy to include it in its interface. Nevertheless
> > when i try to do the same later after i close Galaxy, when i run 'sh run.sh'
> > again i get this error:
> >
> > socket.error: [Errno 98] Address already in use
> 
> Did you stop the old Galaxy instance which is still running and
> using that network address?
> 
> Peter
  ___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Fwd: Galaxy issues

[David Matthews]

Hi Guys,

Sorry to be a pain but this seems to be getting worse for us. Here are the 
latest tracebacks - any suggestions would be gratefully received!!

Cheers
David



> galaxy.jobs.runners.pbs ERROR 2011-12-13 19:57:57,689 Uncaught exception 
> checking jobs
> Traceback (most recent call last):
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py",
>  line 338, in monitor
>self.check_watched_items()
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py",
>  line 351, in check_watched_items
>( failures, statuses ) = self.check_all_jobs()
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py",
>  line 462, in check_all_jobs
>statuses.update( self.convert_statjob_to_bunches( jobs ) )
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py",
>  line 476, in convert_statjob_to_bunches
>statuses[ job.name ] = Bunch( **status )
> MemoryError
> Unhandled exception in thread started by
> Traceback (most recent call last):
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 504, in __bootstrap
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 580, in __bootstrap_inner
> MemoryError
> Unhandled exception in thread started by  >
> Traceback (most recent call last):
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 504, in __bootstrap
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 545, in __bootstrap_inner
> MemoryError
> Unexpected exception in worker  at 0x883acf8>
> Traceback (most recent call last):
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py",
>  line 863, in worker_thread_callback
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py",
>  line 1037, in 
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py",
>  line 1056, in process_request_in_thread
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py",
>  line 1044, in handle_error
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/SocketServer.py", line 
> 334, in handle_error
> MemoryError
> Unhandled exception in thread started by  >
> Traceback (most recent call last):
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 504, in __bootstrap
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 545, in __bootstrap_inner
> MemoryError
> 
> Exception happened during processing of request from ('xxx.xxx.xxx.xxx', 
> 44389)
> Traceback (most recent call last):
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py",
>  line 1053, in process_request_in_thread
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/SocketServer.py", line 
> 322, in finish_request
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/SocketServer.py", line 
> 616, in __init__
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/SocketServer.py", line 
> 657, in setup
> MemoryError
> 
> 
> Exception happened during processing of request from ('xxx.xxx.xx.xx', 60069)
> Unexpected exception in worker  at 0x883a2a8>Traceback 
> (most recent call last):
> 
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py",
>  line 1053, in process_request_in_thread
> Unhandled exception in thread started by  >
> Traceback (most recent call last):
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 504, in __bootstrap
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 545, in __bootstrap_inner
> MemoryError  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/SocketServer.py", line 322, 
> in finish_request
> 
> Unexpected exception in worker  at 0x8721410>
> Traceback (most recent call last):
>  File 
> "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py",
>  line 863, in worker_thread_callback
> Unhandled exception in thread started by  >
> Traceback (most recent call last):
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 504, in __bootstrap
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/threading.py", line 
> 545, in __bootstrap_inner
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/traceback.py", line 
> 242, in format_exc
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/traceback.py", line 
> 142, in format_exception
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/traceback.py", line 76, 
> in format_tb
>  File "/gpfs/cluster/isys/galaxy/Galaxy/lib/python2.6/traceback.py", line 
> 101, in extract_tb
> 

[galaxy-dev] Looking for Galaxy related papers (or have written one)? There is a new Galaxy Group @ CiteULike

[Dave Clements]

Hello all,

A new Galaxy Group
has been
created at the
CiteULike  social bookmarking service to organize
papers that are about, use, or reference Galaxy. The Galaxy
Groupis open
to any CiteULike user (and anyone can become a CiteULike user).
Group members can add papers to the group, assign tags, and rate papers.

CiteULike makes extensive use of *tags* to organize and categorize papers.
The Galaxy group uses this set of
tagsto categorize
papers.  If you want to help other Galaxy users and sites
find papers that are relevant to them, or help the Galaxy Project keep
track of what papers (and communities) are using Gaalxy, then please help
by adding relevant papers to the Galaxy Group
Libraryand/or
rating and tagging the papers that are there.
*We particularly need help with papers that were published before this
year, as very few of them are in the initial list. *
Links:
  http:/galaxyproject.org/wiki/CiteULike - Explanation of tags
  http://www.citeulike.org/group/16008/order/group_rating - Galaxy Group
list of papers

Thanks,

Dave Clements

PS: To cite the Galaxy Project, see
http://galaxyproject.org/wiki/Citing%20Galaxy.

-- 
http://galaxyproject.org/
http://getgalaxy.org/
http://usegalaxy.org/
http://galaxyproject.org/wiki/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] concurrent job run parameter problem

[Oleksandr Moskalenko]

Scenario: Running two velvet jobs with the same parameters except for the 

"Conditional (contig_lgth) either 0 or 

1 (with minimum contig length = 500)


If the scheduler runs these jobs separately they produce expected results. 
However, if they happen to start running simultaneously then one of the jobs 
looses its parameters and both jobs produce identical results corresponding to 
the contig_lgth not set.

Galaxy config is load-balanced with jobs tracked in the database and metadata 
set externally in postgresql. the .sh files that are generated appear to be 
correct and job directories are created appropriately. Whatever fouls up the 
resuts seems to happen on the rear end of the job I guess. I wonder if anyone 
ran into this issue? It should be a common scenario where people try out 
different config parameters for the same tool running on the same dataset, so I 
wonder if it's more likely to be my galaxy configuration problem then an actual 
galaxy bug.

My job runner config: http://pastebin.com/vMWDHAQm


Thanks,

Alex




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Job runner out of memory problem

[Oleksandr Moskalenko]

It seems that even under very light i.e. just a couple of long-running jobs 
present load the job runner slowly, but steadily (1-3Mb per sec) grows its 
memory consumption until it's killed by linux OOM killer.

My job runner config: http://pastebin.com/vMWDHAQm

I'm currently restarting the runner out of crontab when it is killed by OOM, 
but it's not a sensible solution by any means.

I wonder if anyone encountered this and how it was solved.

Thanks,

Alex



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] New tool not recognised unless i reboot

[Peter Cock]

2011/12/14 Makis Ladoukakis :
> Dear galaxy users,
>
> I have downloaded a local instance of Galaxy on my linux distribution
> (ubuntu) and i am adding new tools i made myself. To add a new tool i follow
> all the steps (tool definition file and the modification of tool_conf.xml)
> and run 'sh run.sh' for Galaxy to include it in its interface. Nevertheless
> when i try to do the same later after i close Galaxy, when i run 'sh run.sh'
> again i get this error:
>
> socket.error: [Errno 98] Address already in use

Did you stop the old Galaxy instance which is still running and
using that network address?

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] abolut ftp setup

[Richard Liao]

Dear all,
I'm a student at Fudan University in China. I was trying to enable ftp
uploading in my local version of GALAXY.
I tried the setting in
http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP, but I got the
following problems:
1. The galaxyftp seems to be a pasql database, shall I build it myself? or
does it already exists in galaxy directory?
2. the galaxy_user seems to be a table, shall I build it myself? and how
shall I update it and integrate it with galaxy?

Thanks for your attention, and looking forward to your reply!

Best wishes

Liao Ruiqi
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] New tool not recognised unless i reboot

[Makis Ladoukakis]

Dear galaxy users,

I have downloaded a local instance of Galaxy on my linux distribution (ubuntu) 
and i am adding new tools i made myself. To add a new tool i follow all the 
steps (tool definition file and the modification of tool_conf.xml) and run 'sh 
run.sh' for Galaxy to include it in its interface. Nevertheless when i try to 
do the same later after i close Galaxy, when i run 'sh run.sh' again i get this 
error:

socket.error: [Errno 98] Address already in use

and the tool is not recognized. For the run.sh to work again i have to reboot 
my pc. In a virtual machine ubuntu image it's even worse. The 'run.sh' command 
keeps sending this error even when i reboot the image or even when i reboot the 
pc itself. Is there a way around this problem? 

Thank you,
Makis
  ___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] about ftp setting up

[Liao_Ruiqi]

Hello there，
I'm a student at Fudan University in China. I was trying to enable ftp 
uploading in my local version of GALAXY.
I tried the setting in 
http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP, but I got the 
following problems:
1. The galaxyftp seems to be a pasql database, shall I build it myself? or dose 
it already exists in galaxy directory?
2. the galaxy_user seems to be a table, shall I build it myself? and how shall 
I update it and integrate it with galaxy?

Thanks for your attention, and looking forward to your reply!
 
Best wishes

Liao Ruiqi


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] error when running UNIT_TEST script

[Paniagua, Eric]

Hi James,

I don't know if I can offer the whole story on why you're seeing that error, 
but at first glance it looks like the unit test scripts are assuming that all 
dependencies are installed for all tools.  In particular, the GeneTrack tools.

Here's a practical answer that's probably more helpful anyway:

You are missing dependencies for GeneTrack.  When I recently resolved this 
problem on our local instance, I had to install the following dependencies:

The one your error refers to:
PyTableshttp://www.pytables.org/moin

And its dependencies:
numexpr http://code.google.com/p/numexpr/
Cythonhttp://cython.org/
HDF5 (C libraries)http://www.hdfgroup.org/HDF5/

Be sure to read the READMEs and installation notes to be sure you get the right 
version for your system.  It'll take a little bit of time to install all of 
these (HDF5, especially).

Hope that was helpful.  Good luck!

Best,
Eric


From: galaxy-dev-boun...@lists.bx.psu.edu [galaxy-dev-boun...@lists.bx.psu.edu] 
on behalf of James Boocock [smilefr...@gmx.com]
Sent: Tuesday, December 13, 2011 1:19 PM
To: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] error when running UNIT_TEST script

Hi When running the run_unit_tests.sh script on my server i get the
following Error. Wondering what It means and whether It is something I
have done or something within galaxy


#Start Error

Doctest: galaxy.web.form_builder.TextArea ... ok
TextField (galaxy.web.form_builder)
Doctest: galaxy.web.form_builder.TextField ... ok

==
ERROR: Failure: ImportError (No module named tables)
--
Traceback (most recent call last):
   File
"/home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/loader.py",
line 379, in loadTestsFromName
 addr.filename, addr.module)
   File
"/home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/importer.py",
line 39, in importFromPath
 return self.importFromDir(dir_path, fqname)
   File
"/home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/importer.py",
line 86, in importFromDir
 mod = load_module(part_fqname, fh, filename, desc)
   File
"/home/james/galaxy-dist/lib/galaxy/datatypes/converters/bed_to_genetrack_converter.py",
line 16, in 
 from genetrack.scripts import tabs2genetrack
   File
"/home/james/galaxy-dist/eggs/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.7.egg/genetrack/__init__.py",
line 38, in 
 from genetrack.hdflib import PositionalData
   File
"/home/james/galaxy-dist/eggs/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.7.egg/genetrack/hdflib.py",
line 5, in 
 from tables import openFile
ImportError: No module named tables

--
Ran 60 tests in 11.380s

#End Error

Cheers James
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/