[galaxy-dev] Implementing iRODS with Galaxy

[Yip, Miu ki]

Hello Galaxy Team,

We're from the Bioinformatics Shared Resources (BSR) at Cold Spring Harbor Labs 
and we're looking to implement irods on a newly installed version of Galaxy.

We want to incorporate iRODS (http://irods.org/). Is there a recommended way to 
do this, or no?

Thanks in advance for the advice.

Best,
Miuki Yip
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed

[Yip, Miu ki]

Hello Galaxy Team,

We're from the Bioinformatics Shared Resources at Cold Spring Harbor Lab and 
we're trying to create a new flavour of Galaxy for local use in the lab using 
docker containers.

We want to add tools that are specific for our lab, such as importing files 
from our system, onto this flavour of Galaxy using Docker containers. Is there 
a recommended way to do this?

I've taken a look at the README from: 
https://github.com/bgruening/docker-galaxy-stable

I have been able to successfully create an instance of Galaxy where the 
homepage is different and tools are imported from the Tool Shed. However, I 
cannot figure out how to install tools onto Galaxy that are not from the Tool 
Shed. I have tried creating a container, putting the tools into the tools 
folder, changing the tool conf and the integrated tool panel xml file. Then 
committing the container and creating a new image. From the new image, I tried 
creating another container and running it. This does not create a Galaxy 
instance with the new tool. (This also does not seem like the best way to go 
about something like this.)

Thank you for your help.

Best,
Miuki Yip
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed

[Yip, Miu ki]

Hello Bjoern,

Thanks for the quick response, I appreciate that.

Concerning the instructions, I am unclear of a couple of things.

I have a folder with my tools, there is no preinstalled folder that these tools 
need to go in called my_tools_folder, right? Additionally, would I place this 
tool in the same directory level as Galaxy_Central or as the config directory?
(My current set up is galaxy_storage -> galaxy-central -> config, I have my 
tools folder in the same level as galaxy-central.)
 
Lastly, the environment variable, is there a specific file that all the 
environment variables are stored for docker Galaxy? I have tried looking for an 
env file, but could not find it.

Thanks so much!
Miuki

From: Björn Grüning [bjoern.gruen...@gmail.com]
Sent: Wednesday, July 20, 2016 2:23 AM
To: Yip, Miu ki; galaxy-dev@lists.galaxyproject.org
Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not 
in Toolshed

Hi Miuki!

Am 19.07.2016 um 22:36 schrieb Yip, Miu ki:
> Hello Galaxy Team,
>
> We're from the Bioinformatics Shared Resources at Cold Spring Harbor
> Lab and we're trying to create a new flavour of Galaxy for local use
> in the lab using docker containers.
>
> We want to add tools that are specific for our lab, such as importing
> files from our system, onto this flavour of Galaxy using Docker
> containers. Is there a recommended way to do this?

Can you try the following, assuming that all your tools are in
my_tools_tolder.

ADD ./my_tools_folder /inhouse_tools/

Now we need a tool_conf.xml file. You can create an additional one, no
need to modify the existing one. I would propose to include this config
file also in ./my_tools_folder so it will end up in
/inhouse_tools/tool_conf.xml
With the following environment variable you tell Galaxy about the
additional tool_conf file.

ENV GALAXY_CONFIG_TOOL_CONFIG_FILE =
config/tool_conf.xml,config/shed_tool_conf.xml,/inhouse_tools/tool_conf.xml

Please make sure that your new tools in /inhouse_tools/tool_conf.xml
have as path "/inhouse_tools/.xml".

Let me know if this works, we should add this to the readme.

Thanks,
Bjoern

> I've taken a look at the README from:
> https://github.com/bgruening/docker-galaxy-stable
>
> I have been able to successfully create an instance of Galaxy where
> the homepage is different and tools are imported from the Tool Shed.
> However, I cannot figure out how to install tools onto Galaxy that
> are not from the Tool Shed. I have tried creating a container,
> putting the tools into the tools folder, changing the tool conf and
> the integrated tool panel xml file. Then committing the container and
> creating a new image. From the new image, I tried creating another
> container and running it. This does not create a Galaxy instance with
> the new tool. (This also does not seem like the best way to go about
> something like this.)


> Thank you for your help.
>
> Best, Miuki Yip
> ___ Please
> keep all replies on the list by using "reply all" in your mail
> client.  To manage your subscriptions to this and other Galaxy lists,
> please use the interface at: https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed

[Yip, Miu ki]

Hi,

I apologize for all the questions, but would all of this work be done within 
the container after it is created by the Docker run command or are you 
referring to the local out of container area?

I am getting an error like:
DEBUG:galaxy.app:python path is: lib/, ., , 
/galaxy-central/eggs/PasteDeploy-1.5.0-py2.7.egg, 
/galaxy-central/.venv/lib/python2.7, 
/galaxy-central/.venv/lib/python2.7/plat-x86_64-linux-gnu, 
/galaxy-central/.venv/lib/python2.7/lib-tk, 
/galaxy-central/.venv/lib/python2.7/lib-old, 
/galaxy-central/.venv/lib/python2.7/lib-dynload, /usr/lib/python2.7, 
/usr/lib/python2.7/plat-x86_64-linux-gnu, /usr/lib/python2.7/lib-tk, 
/galaxy-central/.venv/local/lib/python2.7/site-packages
Traceback (most recent call last):
  File "lib/galaxy/webapps/galaxy/buildapp.py", line 63, in paste_app_factory
app = galaxy.app.UniverseApplication( global_conf=global_conf, **kwargs )
  File "lib/galaxy/app.py", line 44, in __init__
self.config.check()
  File "lib/galaxy/config.py", line 636, in check
raise ConfigurationError("Tool config file not found: %s" % path )
ConfigurationError: Tool config file not found: 
/root/../home/user/galaxy_storage/galaxy-central/config/tool_conf.xml.main

if I try to run when I am adding outside of the container, I've cat-ed the path 
and standard out displays the contents of the file. I am getting another error 
about files not being found also when I add to the container itself when I 
attach. 

Thanks so much,
Miuki

From: Björn Grüning [bjoern.gruen...@gmail.com]
Sent: Wednesday, July 20, 2016 2:05 PM
To: Yip, Miu ki; Björn Grüning; galaxy-dev@lists.galaxyproject.org
Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not 
in Toolshed

Hi,

> Hello Bjoern,
>
> Thanks for the quick response, I appreciate that.
>
> Concerning the instructions, I am unclear of a couple of things.
>
> I have a folder with my tools, there is no preinstalled folder that
> these tools need to go in called my_tools_folder, right?

No, you can create such a folder wherever you want and name it like you
want :)

> Additionally, would I place this tool in the same directory level as
> Galaxy_Central or as the config directory? (My current set up is
> galaxy_storage -> galaxy-central -> config, I have my tools folder in
> the same level as galaxy-central.)

This does not matter, what matters is that the folder path is correct in
the env variable.

> Lastly, the environment variable, is there a specific file that all
> the environment variables are stored for docker Galaxy? I have tried
> looking for an env file, but could not find it.

There is no file. These are real environment variables that are created
by Docker during container start. You can change them during start with
(e.g): docker run -e MY_ENV_VAR=foo bgruening/galaxy-stable

If such a variable, beginning with GALAXY_CONFIG_XXX exists this will be
picked up by Galaxy. Have a look at the container there are a lot
GALAXY_CONFIG_ vars set.

Hope this helps,
Bjoern


> Thanks so much! Miuki ____ From:
> Björn Grüning [bjoern.gruen...@gmail.com] Sent: Wednesday, July 20,
> 2016 2:23 AM To: Yip, Miu ki; galaxy-dev@lists.galaxyproject.org
> Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing
> tools not in Toolshed
>
> Hi Miuki!
>
> Am 19.07.2016 um 22:36 schrieb Yip, Miu ki:
>> Hello Galaxy Team,
>>
>> We're from the Bioinformatics Shared Resources at Cold Spring
>> Harbor Lab and we're trying to create a new flavour of Galaxy for
>> local use in the lab using docker containers.
>>
>> We want to add tools that are specific for our lab, such as
>> importing files from our system, onto this flavour of Galaxy using
>> Docker containers. Is there a recommended way to do this?
>
> Can you try the following, assuming that all your tools are in
> my_tools_tolder.
>
> ADD ./my_tools_folder /inhouse_tools/
>
> Now we need a tool_conf.xml file. You can create an additional one,
> no need to modify the existing one. I would propose to include this
> config file also in ./my_tools_folder so it will end up in
> /inhouse_tools/tool_conf.xml With the following environment variable
> you tell Galaxy about the additional tool_conf file.
>
> ENV GALAXY_CONFIG_TOOL_CONFIG_FILE =
> config/tool_conf.xml,config/shed_tool_conf.xml,/inhouse_tools/tool_conf.xml
>
>  Please make sure that your new tools in
> /inhouse_tools/tool_conf.xml have as path "/inhouse_tools/.xml".
>
> Let me know if this works, we should add this to the readme.
>
> Thanks, Bjoern
>
>> I've taken a look at the README from:
>> https://github.com/bgruening/docker-galaxy-stable
>>
>

[galaxy-dev] Galaxy in Docker and nginx

[Yip, Miu ki]

Hi Galaxy Team,

I'm currently trying to use Galaxy in Docker, and I wanted to know if there are 
special actions that I have to do in Galaxy for nginx. I'm trying to set up a 
reserve proxy and have url forwarding. Do I need to inform Galaxy in Docker 
that nginx is being use?

Thanks,
Miuki
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy in Docker and nginx

[Yip, Miu ki]

Hi,

I ended up getting nginx running. Turns out it was just a selinux issue.

Thanks and sorry for the trouble!

Best,
Miuki

From: Yip, Miu ki
Sent: Thursday, July 28, 2016 4:28 PM
To: galaxy-dev@lists.galaxyproject.org
Subject: Galaxy in Docker and nginx

Hi Galaxy Team,

I'm currently trying to use Galaxy in Docker, and I wanted to know if there are 
special actions that I have to do in Galaxy for nginx. I'm trying to set up a 
reserve proxy and have url forwarding. Do I need to inform Galaxy in Docker 
that nginx is being use?

Thanks,
Miuki
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Connecting LDAP with Galaxy

[Yip, Miu ki]

Hi,

I’m trying to configure a new instance of Galaxy and I want to use LDAP for 
user authentication. I have taken a look at this wiki page: 
https://wiki.galaxyproject.org/Admin/Config/ExternalUserAuth

I followed the instructions and when I looked into the auth_conf.xml file, I 
was unsure of what exactly Galaxy is looking for when trying to connect with 
LDAP. Are there specific options that I must give it? May I ask how Galaxy uses 
LDAP to authenticate users?

Thank you,
Miuki Yip
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Docker ToolShed Imports Issue

[Yip, Miu ki]

Hi all,

I’m trying to import shed tools into Galaxy for Docker. When I build the 
container, I get this error message:

TASK [galaxy_tools : Install Tool Shed tools] **
 [WARNING]: The loop variable 'item' is already in use. You should set the
`loop_var` value in the `loop_control` option for the task to something else to
avoid variable collisions and unexpected behavior.
failed: [localhost] (item={u'owner': u'vlefort', u'tool_shed_url': 
u'https://toolshed.g2.bx.psu.edu', 
u'tool_panel_section_label': u'Import Files', u'name': u'phyml'}) => 
{"changed": false, "cmd": ["/tmp/venv/bin/python", 
"install_tool_shed_tools.py", "-y", "name: phyml\nowner: 
vlefort\ntool_panel_section_label: Import Files\ntool_shed_url: 
https://toolshed.g2.bx.psu.edu\n";, "-a", "admin", "-g", "localhost:8080"], 
"delta": "0:00:00.781449", "end": "2016-08-17 23:48:07.389199", "failed": true, 
"failed_when_result": true, "item": {"name": "phyml", "owner": "vlefort", 
"tool_panel_section_label": "Import Files", "tool_shed_url": 
"https://toolshed.g2.bx.psu.edu"}, "rc": 0, "start": "2016-08-17 
23:48:06.607750", "stderr": "(1/1) Installing tool phyml from vlefort to 
section \"Import Files\" at revision f6dadc1de75f (TRT: 0:00:00.308611)\r\n\t* 
Error installing a tool (after phyml)! Name: 0:00:00.247213,owner: vlefort, 
revision: f6dadc1de75f, error: {\"err_msg\": \"Missing required parameter 
'shed_tool_conf'.\", \"err_code\": 47}\r\nInstalled tools (0): 
[]\r\nSkipped tools (0): []\r\nErrored tools (1): [('phyml', 
u'f6dadc1de75f')]\r\nAll tools listed in 'None' have been processed.\r\nTotal 
run time: 0:00:00.556216", "stdout": "", "stdout_lines": [], "warnings": []}
…ignoring

Is this a conf file issue?

Best,
Miuki
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy Pulsar unable to access DRMAA so

[Yip, Miu ki]

Hi all,

When trying to submit a job to the cluster via Pulsar from Galaxy, an error is 
thrown and the job won’t be submitted.

The paster.log file has the following error:
Exception: Attempt to use DRMAA, but DRMAA Python library cannot be loaded. 
OSError - problem loading shared library [/opt/uge/lib/lx-amd64/libdrmaa.so: 
cannot open shared object file: No such file or directory].

The shared object file can be found and accessed manually. Am not sure what 
this error means or how to go about looking for more information.

Thank you in advance for your help.

Best,
Miuki
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown

[Yip, Miu ki]

Hi all,

While trying to install tools from the toolshed via the browser interface, an 
error keeps appearing and nothing is abled to be installed. Currently using the 
latest version of Galaxy and having tried to install several different tools, 
the following error keeps appearing in the log file.

galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload on 
'main'
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
self.run()
  File "/usr/lib64/python2.7/threading.py", line 764, in run
self.__target(*self.__args, **self.__kwargs)
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", 
line 147, in on_any_event
self._handle(event)
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", 
line 150, in _handle
self.reload_callback()
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
line 78, in 
self._tool_conf_watcher = get_tool_conf_watcher( lambda: 
reload_toolbox(app))
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 56, 
in reload_toolbox
app.toolbox = _get_new_toolbox(app)
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 72, 
in _get_new_toolbox
new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, 
app.toolbox._tool_conf_watcher)
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", line 
110, in __init__
tool_conf_watcher=tool_conf_watcher
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
line 1037, in __init__
super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, 
app, tool_conf_watcher=tool_conf_watcher)
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
line 69, in __init__
self._init_integrated_tool_panel( app.config )
  File 
"/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py",
 line 36, in _init_integrated_tool_panel
self._load_integrated_tool_panel_keys()
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
line 368, in _load_integrated_tool_panel_keys
tree = parse_xml( self._integrated_tool_panel_config )
  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 
211, in parse_xml
root = tree.parse( fname, parser=ElementTree.XMLParser( 
target=DoctypeSafeCallbackTarget() ) )
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
parser.feed(data)
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
self._raiseerror(v)
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in 
_raiseerror
raise err
ParseError: not well-formed (invalid token): line 44, column 71

Is there an XML or configuration file which needs to be changed for this to 
work? Not sure how to proceed.

Thanks!
Miuki
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown

[Yip, Miu ki]

Hi Marius,

Thanks for the reply!

I have tried moving the file and just deleting the file all together. I am 
still running into this error.

Thanks,
Miuki

> On Nov 3, 2016, at 10:29 AM, Marius van den Beek  
> wrote:
> 
> Hello Miuki,
> 
> this looks like your integrated_tool_panel.xml file has some problems.
> Can you try to move this file to a different location and try the 
> installation again?
> It should be located in your galaxy directory.
> 
> Best,
> Marius
> 
> 
> On 11/03/2016 03:16 PM, Yip, Miu ki wrote:
>> Hi all,
>> 
>> While trying to install tools from the toolshed via the browser interface, 
>> an error keeps appearing and nothing is abled to be installed. Currently 
>> using the latest version of Galaxy and having tried to install several 
>> different tools, the following error keeps appearing in the log file.
>> 
>> galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload 
>> on 'main'
>> Exception in thread Thread-2:
>> Traceback (most recent call last):
>>   File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
>> self.run()
>>   File "/usr/lib64/python2.7/threading.py", line 764, in run
>> self.__target(*self.__args, **self.__kwargs)
>>   File 
>> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", 
>> line 147, in on_any_event
>> self._handle(event)
>>   File 
>> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", 
>> line 150, in _handle
>> self.reload_callback()
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 78, in 
>> self._tool_conf_watcher = get_tool_conf_watcher( lambda: 
>> reload_toolbox(app))
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
>> 56, in reload_toolbox
>> app.toolbox = _get_new_toolbox(app)
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
>> 72, in _get_new_toolbox
>> new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, 
>> app.toolbox._tool_conf_watcher)
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", 
>> line 110, in __init__
>> tool_conf_watcher=tool_conf_watcher
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 1037, in __init__
>> super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, 
>> app, tool_conf_watcher=tool_conf_watcher)
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 69, in __init__
>> self._init_integrated_tool_panel( app.config )
>>   File 
>> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py",
>>  line 36, in _init_integrated_tool_panel
>> self._load_integrated_tool_panel_keys()
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 368, in _load_integrated_tool_panel_keys
>> tree = parse_xml( self._integrated_tool_panel_config )
>>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 
>> 211, in parse_xml
>> root = tree.parse( fname, parser=ElementTree.XMLParser( 
>> target=DoctypeSafeCallbackTarget() ) )
>>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
>> parser.feed(data)
>>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
>> self._raiseerror(v)
>>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in 
>> _raiseerror
>> raise err
>> ParseError: not well-formed (invalid token): line 44, column 71
>> 
>> Is there an XML or configuration file which needs to be changed for this to 
>> work? Not sure how to proceed.
>> 
>> Thanks!
>> Miuki
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
> 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown

[Yip, Miu ki]

Awesome! After changing “&” to “and”, it worked!

Thank you!

> On Nov 3, 2016, at 12:59 PM, Peter Cock  wrote:
> 
> Good news - this is the bug I though it was, the ampersand
> should be escaped as & in the XML file:
> 
> https://github.com/galaxyproject/galaxy/issues/3084
> 
> More good news: This will likely be fixed in the next release
> of Galaxy:
> 
> https://github.com/galaxyproject/galaxy/pull/3122
> 
> Right now I suggest you manually edit these XML files to
> use the word "and" instead of "&".
> 
> Peter
> 
> 
> On Thu, Nov 3, 2016 at 2:36 PM, Yip, Miu ki  wrote:
>> Hi Peter,
>> 
>> Line 44 is oddly specific. Nevertheless, we have this as the 44th line in 
>> the xml file.
>> 
>> > version="" />
>> 
>> This tool is not the one trying to be installed.
>> 
>> Thanks,
>> Miuki
>> On Nov 3, 2016, at 10:28 AM, Peter Cock 
>> mailto:p.j.a.c...@googlemail.com>> wrote:
>> 
>> I think you may have hit the same bug I found recently:
>> 
>> https://github.com/galaxyproject/galaxy/issues/3084
>> 
>> What is line 44 of your integrated_tool_panel.xml file?
>> 
>> Peter
>> 
>> On Thu, Nov 3, 2016 at 2:16 PM, Yip, Miu ki  wrote:
>> Hi all,
>> 
>> While trying to install tools from the toolshed via the browser interface, 
>> an error keeps appearing and nothing is abled to be installed. Currently 
>> using the latest version of Galaxy and having tried to install several 
>> different tools, the following error keeps appearing in the log file.
>> 
>> galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload 
>> on 'main'
>> Exception in thread Thread-2:
>> Traceback (most recent call last):
>> File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
>>   self.run()
>> File "/usr/lib64/python2.7/threading.py", line 764, in run
>>   self.__target(*self.__args, **self.__kwargs)
>> File 
>> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", 
>> line 147, in on_any_event
>>   self._handle(event)
>> File 
>> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", 
>> line 150, in _handle
>>   self.reload_callback()
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 78, in 
>>   self._tool_conf_watcher = get_tool_conf_watcher( lambda: 
>> reload_toolbox(app))
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
>> 56, in reload_toolbox
>>   app.toolbox = _get_new_toolbox(app)
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
>> 72, in _get_new_toolbox
>>   new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, 
>> app.toolbox._tool_conf_watcher)
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", line 
>> 110, in __init__
>>   tool_conf_watcher=tool_conf_watcher
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 1037, in __init__
>>   super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, 
>> app, tool_conf_watcher=tool_conf_watcher)
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 69, in __init__
>>   self._init_integrated_tool_panel( app.config )
>> File 
>> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py",
>>  line 36, in _init_integrated_tool_panel
>>   self._load_integrated_tool_panel_keys()
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
>> line 368, in _load_integrated_tool_panel_keys
>>   tree = parse_xml( self._integrated_tool_panel_config )
>> File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 
>> 211, in parse_xml
>>   root = tree.parse( fname, parser=ElementTree.XMLParser( 
>> target=DoctypeSafeCallbackTarget() ) )
>> File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
>>   parser.feed(data)
>> File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
>>   self._raiseerror(v)
>> File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in 
>> _raiseerror
>>   raise err
>> ParseError: not well-formed (invalid token): line 44, column 71
>> 
>> Is there an XML or configuration file which needs t

[galaxy-dev] Galaxy ToolShed Installation Question

[Yip, Miu ki]

Hi all,

When a tool is installed onto Galaxy via the GUI from the ToolShed, where is 
the tool itself installed on the system?

For instance, having installed Bed Tools from the ToolShed, the tool errors out 
on a run with the following error:

Fatal error: Exit code 127 ()
/galaxy/database/job_working_directory/000/84/tool_script.sh: line 9: bedtools: 
command not found

This means that a path needs to be set somewhere in Galaxy to the tool? Where 
is the tool installed?

Having checked the location below, only the xml files are listed.
/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/bedtools/7b3aaff0d78c/bedtools

Is there a configuration that is missing?

Thanks!
Miuki Yip
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] ToolShed devteam tools error

[Yip, Miu ki]

Hi all,

After installing the following devteam tools from the ToolShed: Base 
Coverage/Cluster/Complement/Concatenate/Flanking_features/get_flanks/Intersect/Join/Merge/Subtract

The tools give an error after being run.

Traceback (most recent call last):
  File 
"/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/basecoverage/0a3e3133b09d/basecoverage/gops_basecoverage.py",
 line 15, in 
from galaxy.tools.util.galaxyops import fail, parse_cols_arg, skipped
  File "/galaxy/galaxy/lib/galaxy/tools/__init__.py", line 18, in 
from mako.template import Template
ImportError: No module named mako.template

Both the python virtual environment and the machine locally has the module mako 
installed. This module is also accessible via the interactive python shell. Is 
the error coming from the galaxy.tools.util.galaxyops or something else?

Thanks!
Miuki
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] AD/LDAP Integration

[Yip, Miu ki]

Hi all,

After setting up AD/LDAP in Galaxy, we noticed that for some reason, some users 
are able to log in but others were not. We don’t think that it has anything to 
do with the set up of LDAP or AD at our institute and we’ve tried to find 
common threads between the users who can log in but we have not found anything 
interesting.

Could someone explain what exactly Galaxy looks for when signing users in with 
AD/LDAP? We found the source code here: ./lib/galaxy/auth/providers/ldap_ad.py 
but all the code seems to be functioning properly.

Here is one of the error outputs in the log when somebody tries to log in for 
the first time, but cannot:

galaxy.auth.providers.ldap_ad DEBUG 2017-02-01 17:16:17,954 LDAP authenticate: 
whoami is u:SCIEDU\chess
galaxy.auth.providers.ldap_ad DEBUG 2017-02-01 17:16:17,954 LDAP authentication 
successful
server ip - - [01/Feb/2017:17:16:12 -0400] "POST 
/galaxy/user/login?use_panels=False HTTP/1.0" 200 - 
“http:/galaxyurl.edu/galaxy/user/login?use_panels=False"
 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, 
like Gecko) Chrome/55.0.2883.95 Safari/537.36"

Exception happened during processing of request from ('127.0.0.1', 55724)
Traceback (most recent call last):
  File 
"/localdata1/galaxy/.venv/lib/python2.7/site-packages/paste/httpserver.py", 
line 1085, in process_request_in_thread
self.finish_request(request, client_address)
  File "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python2.7/SocketServer.py", line 651, in __init__
self.finish()
  File "/usr/lib64/python2.7/SocketServer.py", line 710, in finish
self.wfile.close()
  File "/usr/lib64/python2.7/socket.py", line 279, in close
self.flush()
  File "/usr/lib64/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe


Any insight would be appreciated. Thanks!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy Restart & Jobs finishing

[Yip, Miu ki]

Hi all,

Is there a way to restart Galaxy without having to wait till all the 
outstanding jobs are finished running? I’ve noticed that whenever Galaxy daemon 
is stopped and then started, the UI does not return back online until all the 
jobs that were running are finished or killed because they are all running on 
the same port as the original program.

Is there a configuration that needs to be changed so that the application can 
restart right away without having to wait or kill jobs?

Thanks!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy and Torque

[Yip, Miu ki]

Hi all,

Is Torque still supported as a resource manager?

This page ( https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster ) 
mentioned the usage of Torque and how to set it up. However, the setup 
instructions refer to a /script/scramble.py file which I assume interfaces with 
eggs. If I recall correctly, eggs were upgraded to wheels. Does this mean there 
are upgraded scripts in place of scramble.py and the eggs.ini?

I saw this page ( https://biostar.usegalaxy.org/p/20470/ ) and it looks like 
nobody has a solution to this.

Thanks!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] All Jobs Error Out

[Yip, Miu ki]

Hi all,

This afternoon, our local instance of Galaxy suddenly started producing errors 
for all jobs. This is the trace that gets printed out. Not sure what is 
happening because the error doesn’t give much information other than python not 
being able to find a module.

Traceback (most recent call last):
  File 
"/localdata1/galaxy/database/jobs_directory/002/2931/set_metadata_LMpe2m.py", 
line 1, in 
from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
  File "/localdata1/galaxy/lib/galaxy_ext/metadata/set_metadata.py", line 25, 
in 
import galaxy.model.mapping  # need to load this before we unpickle, in 
order to setup properties assigned by the mappers
  File "/localdata1/galaxy/lib/galaxy/model/mapping.py", line 19, in 
from galaxy.model.custom_types import JSONType, MetadataType, 
TrimmedString, UUIDType
  File "/localdata1/galaxy/lib/galaxy/model/custom_types.py", line 13, in 
from galaxy import app
  File "/localdata1/galaxy/lib/galaxy/app.py", line 17, in 
from galaxy.visualization.plugins.registry import VisualizationsRegistry
  File "/localdata1/galaxy/lib/galaxy/visualization/plugins/registry.py", line 
15, in 
from galaxy.visualization.plugins import plugin as vis_plugins
  File "/localdata1/galaxy/lib/galaxy/visualization/plugins/plugin.py", line 
13, in 
from galaxy.web.base import interactive_environments
  File "/localdata1/galaxy/lib/galaxy/web/base/interactive_environments.py", 
line 14, in 
from galaxy.tools.deps.docker_util import DockerVolume
  File "/localdata1/galaxy/lib/galaxy/tools/__init__.py", line 60, in 
import tool_shed.util.repository_util as repository_util
  File "/localdata1/galaxy/lib/tool_shed/util/repository_util.py", line 16, in 
from tool_shed.util import hg_util
  File "/localdata1/galaxy/lib/tool_shed/util/hg_util.py", line 11, in 
from mercurial.exchange import readbundle
  File "/lib64/python2.7/site-packages/mercurial/exchange.py", line 18, in 
from . import (
  File "/lib64/python2.7/site-packages/mercurial/bundle2.py", line 168, in 
urlerr = util.urlerr
AttributeError: 'module' object has no attribute 'urlerr'

Could anyone provide any insight on the issue? Thanks!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] All Jobs Error Out

[Yip, Miu ki]

Hi John,

Thanks for the reply. I saw in the stack trace that there was a issue with 
mercurial, which was recently installed in the system. After uninstalling it, 
everything ran again. Not entirely sure what happened.

Thanks,
Miuki
On Mar 14, 2017, at 8:27 AM, John Chilton 
mailto:jmchil...@gmail.com>> wrote:

https://docs.galaxyproject.org/en/master/admin/framework_dependencies.html#galaxy-job-handlers

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy Start Up Error - Double Requirements, Tempita

[Yip, Miu ki]

Hi all,

When attempting to start up Galaxy, via the run.sh script, we’re getting the 
following error:

galaxyd@genomics:/galaxy$ sh run.sh
Activating virtualenv at .venv
Requirement already satisfied: pip>=8.1 in ./.venv/lib/python2.7/site-packages
Double requirement given: Tempita==0.5.3.dev0 (from -r requirements.txt (line 
155)) (already in Tempita==0.5.3dev (from -r requirements.txt (line 57)), 
name='Tempita')

This occurs when the start up script is run with the —no-create-venv and 
without the flag. It looks like there are two “Tempita” items in the 
requirements.txt, if one of them is removed, it leads to another error about 
“ethtools”

galaxyd@genomics:/galaxy$ sh run.sh --daemon --no-create-venv
Activating virtualenv at .venv
Requirement already satisfied: pip>=8.1 in ./.venv/lib/python2.7/site-packages
...
Collecting crypto==1.4.1 (from -r requirements.txt (line 81))
  Using cached crypto-1.4.1-py2.py3-none-any.whl
Requirement already satisfied: cycler==0.10.0 in 
./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 82))
Collecting Cython==0.25.2 (from -r requirements.txt (line 83))
  Using cached Cython-0.25.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: docopt==0.6.2 in 
./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 86))
Requirement already satisfied: drmaa==0.7.6 in 
./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 88))
Collecting ethtool==0.8 (from -r requirements.txt (line 90))
  Could not find a version that satisfies the requirement ethtool==0.8 (from -r 
requirements.txt (line 90)) (from versions: )
No matching distribution found for ethtool==0.8 (from -r requirements.txt (line 
90))

We’re not sure if both versions of “Tempita” are required and if they are, how 
can the application be started up? Running both the run.sh and the 
run_reports.sh scripts lead to this error about “Tempita”. Has anyone 
encountered this error before?

Thanks!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Tool Shed tools - Uncaught Error

[Yip, Miu ki]

Hi all,

We’re having a bit of trouble with a local version of Galaxy where all the 
tools that were installed via tool shed is showing an uncaught error on the 
user interface.

The tools have clearly been installed and can be seen in the admin panel on the 
interface as well as the psql database via the command line. It seems like the 
in house tools are all working fine and the tool shed tools were working fine 
until recently. 

Can anyone suggest what steps to take next to fix this error or to find the 
cause? We are using docker galaxy and it doesn’t seem like anything is 
appearing on the error logs.

Thanks!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Job output not returned from Cluster Error

[Yip, Miu ki]

Hi,

We’re having a bit of an error where the GUI shows that a job had an error. 
When the user submits the job, the job is seen to have run via terminal and 
uge. The job working directory exists and the out and error file is in there. 
Once the job completes, it seems like the GUI will say that it works. But then 
several minutes later, the GUI shows the job turning from green to red and 
giving the error that the “Job output not returned from the Cluster” error. 

I tried setting the retry_job_output_collection but it did not work (we’re 
using a local file system): 
http://dev.list.galaxyproject.org/Problem-related-to-a-job-that-quot-failed-quot-td4627406.html#a4632168

This seems to happen randomly for different jobs. Some jobs receive errors but 
others do not receive this error. 

Does anybody have any suggestions on how to fix this? There is no error that 
appears in the paster.log file and we are using SGE. 

Thanks in advance!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/