[galaxy-dev] galaxy config from 17.05 to 18.05

[Fernandez Edgar]

Hello gents,

I would greatly appreciate some help configuring my new installation of galaxy 
18.05.

I found most of the configuration I needed but I'm missing the following 
configuration in green.
Here is my galaxy 17.05:
[galaxy@esilbac3a ~]$ diff galaxy-17.05/config/galaxy.ini  
galaxy-17.05/config/galaxy.ini.sample
32c32
< port = 7112
---
> #port = 8080
37c37
< host = 0.0.0.0
---
> #host = 127.0.0.1
50,59d49
< #  HTTP PBS DRMAA HANDLER ---
<
< # Configuration of the pbs drmaa handler
<
< [server:handler0]
< use = egg:Paste#http
< port = 8088
< use_threadpool = True
< threadpool_workers = 10
<
74c64
< prefix = /galaxy-prod
---
> prefix = /galaxy
95c85
< filter-with = proxy-prefix
---
> #filter-with = proxy-prefix
109c99
< database_connection = 
mysql://galaxy:qaz1wsx2@localhost/galaxy_db_prod2?unix_socket=/var/lib/mysql/mysql.sock
---
> #database_connection = 
> sqlite:///./database/universe.sqlite?isolation_level=IMMEDIATE
119c109
< database_engine_option_pool_recycle = 7200
---
> #database_engine_option_pool_recycle = -1
152c142
< file_path = /home/galaxy/galaxy-prod/database/files
---
> #file_path = database/files
155c145
< new_file_path = /home/galaxy/galaxy-prod/database/tmp
---
> #new_file_path = database/tmp
161c151
< tool_config_file = 
/home/galaxy/galaxy-prod/config/tool_conf.xml,/home/galaxy/galaxy-prod/config/shed_tool_conf.xml
---
> #tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml
197c187
< tool_dependency_dir = 
/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir
---
> #tool_dependency_dir = database/dependencies
401c391
< job_working_directory = /home/galaxy/galaxy-prod/database/jobs_directory
---
> #job_working_directory = database/jobs_directory
466c456
< smtp_server = smtp.umontreal.ca:25
---
> #smtp_server = None
475c465
< smtp_ssl = False
---
> #smtp_ssl = False
486c476
< error_email_to = edgar.fernan...@umontreal.ca
---
> #error_email_to = None
493c483
< email_from = Galaxy ESI Project 
---
> #email_from = None
679c669
< apache_xsendfile = True
---
> #apache_xsendfile = False
693c683
< upstream_gzip = False
---
> #upstream_gzip = False
848c838
< debug = False
---
> #debug = False
861c851
< use_interactive = False
---
> use_interactive = True
989c979
< id_secret = 3s1G@l@xyPr0j3ct
---
> #id_secret = USING THE DEFAULT IS NOT SECURE!
1038c1028
< admin_users = root...@esi.umontreal.ca
---
> #admin_users = None
1041c1031
< require_login = True
---
> #require_login = False
1049c1039
< allow_user_creation = False
---
> #allow_user_creation = True
1052c1042
< allow_user_deletion = True
---
> #allow_user_deletion = False
1055c1045
< allow_user_impersonation = True
---
> #allow_user_impersonation = False
1060c1050
< allow_user_dataset_purge = True
---
> #allow_user_dataset_purge = True
1066c1056
< new_user_dataset_access_role_default_private = True
---
> #new_user_dataset_access_role_default_private = False
1200c1190
< enable_quotas = True
---
> #enable_quotas = False
1234c1224
< job_config_file = /home/galaxy/galaxy-prod/config/job_conf.xml
---
> #job_config_file = config/job_conf.xml
1292c1282
< retry_job_output_collection = 20
---
> #retry_job_output_collection = 0
1310c1300
< cleanup_job = onsuccess
---
> #cleanup_job = always


Please help me find what I am missing.

Best regards,

Edgar Fernandez
Administrateur Système (Linux)
Technologies de l'Information
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
*  Bur. : 1-514-343-6111 poste 16568

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] new installation galaxy-17.05 - Uncaught error.

[Fernandez Edgar]

Hello Frederic,

I actually just fixed the problem not 5 minutes ago...
In your file: /etc/httpd/vhosts.d/galaxy_prod.conf
Add the following line: AllowEncodedSlashes NoDecode

Please let me know what you think...

Regards,

Edgar Fernandez
System Administrator (Linux)
Technologies de l'Information
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
*  Bur. : 1-514-343-6111 poste 16568

From: SAPET, Frederic [mailto:frederic.sa...@biogemma.com]
Sent: June 15, 2017 10:36 AM
To: Fernandez Edgar <edgar.fernan...@umontreal.ca>; galaxy-...@bx.psu.edu
Subject: RE: [galaxy-dev] new installation galaxy-17.05 - Uncaught error.

Hi

Same for me.
But It looks like to be related with tools installed via the toolshed (Ie : 
bedtools, emboss, blast+ suite...).
My own tools seem to be ok.

Just like you Edgar, nothing in the log...

Fred

De : galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] De la part 
de Fernandez Edgar
Envoyé : mercredi 14 juin 2017 17:11
À : galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>
Objet : [galaxy-dev] new installation galaxy-17.05 - Uncaught error.

Hey Guys,

Hope everyone is doing ok.

I have a small issue with a brand new installation of galaxy-1705.
It starts properly and I installed some tools.
But when I try to test any tools I get and "Uncaught error" (see picture).

I can't find anywhere in the logs what is happening.

Any ideas?

Regards,

Edgar Fernandez
System Administrator (Linux)
Technologies de l'Information
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
*  Bur. : 1-514-343-6111 poste 16568

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] galaxy - velvet devteam revision 08256557922f

[Fernandez Edgar]

Hey everyone,

I haven’t got a feedback from you.
If you have any suggestions, I’m a little bit confused with this problem.

Thank you very much!

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Fernandez Edgar
Sent: October-31-16 11:40 AM
To: galaxy-...@bx.psu.edu
Subject: [galaxy-dev] galaxy - velvet devteam revision 08256557922f

Hey Guys,

I need your help understanding what’s happening with velvet  devteam revision 
08256557922f.

After installing it, velvet was giving me the following error:
galaxy.jobs.handler WARNING 2016-10-20 12:15:11,986 (1599) Tool 
'toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.0' removed from 
tool config, unable to run job galaxy.jobs.handler ERROR 2016-10-20 
12:15:12,165 (1599) Error checking job readiness
galaxy.jobs.handler ERROR 2016-10-20 12:15:12,165 (1599) Error checking job 
readiness

I realized velvet needed that galaxy was restarted.

Now, velvet finishes his jobs (becomes green) however, the following message is 
on galaxy’s
Job output not returned from cluster

And you can see in the logs of the handler:
galaxy.jobs DEBUG 2016-10-31 10:17:08,548 (1669) Working directory for job is: 
/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669
galaxy.jobs.handler DEBUG 2016-10-31 10:17:08,567 (1669) Dispatching to drmaa 
runner
galaxy.jobs DEBUG 2016-10-31 10:17:08,621 (1669) Persisting job destination 
(destination id: torque)
galaxy.jobs.runners DEBUG 2016-10-31 10:17:08,630 Job [1669] queued (63.397 ms)
galaxy.jobs.handler INFO 2016-10-31 10:17:08,652 (1669) Job dispatched
galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Building dependency shell 
command for dependency 'velvet'
galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Find dependency velvet version 
1.2.10
galaxy.tools.deps DEBUG 2016-10-31 10:17:08,896 Resolver tool_shed_packages 
returned  (isnull? False)
galaxy.jobs.command_factory INFO 2016-10-31 10:17:08,945 Built script 
[/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh] for 
tool 
command[PACKAGE_BASE=/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f;
 export PACKAGE_BASE; . 
/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f/env.sh;
 velveth 2>&1 | grep "Version" | sed -e 's/Version //' > 
/home/galaxy/galaxy-prod/database/tmp/GALAXY_VERSION_STRING_1669 2>&1; python 
/home/galaxy/galaxy-prod/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/velvet/08256557922f/velvet/velveth_wrapper.py
 '/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat' 
'/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/dataset_4306_files' 
21  -fasta -short /home/galaxy/galaxy-prod/database/files/003/dataset_3537.dat]
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Building dependency shell 
command for dependency 'samtools'
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Find dependency samtools 
version None
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver tool_shed_packages 
returned  
(isnull? True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages 
returned  
(isnull? True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages 
returned  
(isnull? True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned 
 (isnull? 
True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned 
 (isnull? 
True)
galaxy.tools.deps WARNING 2016-10-31 10:17:09,145 Failed to resolve dependency 
on 'samtools', ignoring
galaxy.jobs.runners DEBUG 2016-10-31 10:17:09,153 (1669) command is: mkdir -p 
working; cd working; 
/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh; 
return_code=$?; cd 
'/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669'; python 
"/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/set_metadata_KQTExg.py"
 "/home/galaxy/galaxy-prod/database/tmp/tmpLU0HQy" 
"/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/working/galaxy.json"
 
"/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_in_HistoryDatasetAssociation_4437_kIYXWa,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_kwds_HistoryDatasetAssociation_4437_ArCXyb,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_out_HistoryDatasetAssociation_4437_l28qWr,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_results_HistoryDatasetAssociation_4437_oTDjOd,/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_override_HistoryDatasetAssociation_4437_8VzdJw"
 5242880; sh -c "exit $return_code"
galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:09,202 (1669

[galaxy-dev] galaxy - velvet devteam revision 08256557922f

[Fernandez Edgar]

Hey Guys,

I need your help understanding what’s happening with velvet  devteam revision 
08256557922f.

After installing it, velvet was giving me the following error:
galaxy.jobs.handler WARNING 2016-10-20 12:15:11,986 (1599) Tool 
'toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.0' removed from 
tool config, unable to run job galaxy.jobs.handler ERROR 2016-10-20 
12:15:12,165 (1599) Error checking job readiness

I realized velvet needed that galaxy was restarted.

Now, velvet finishes his jobs (becomes green) however, the following message is 
on galaxy’s
Job output not returned from cluster

And you can see in the logs of the handler:
galaxy.jobs DEBUG 2016-10-31 10:17:08,548 (1669) Working directory for job is: 
/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669
galaxy.jobs.handler DEBUG 2016-10-31 10:17:08,567 (1669) Dispatching to drmaa 
runner
galaxy.jobs DEBUG 2016-10-31 10:17:08,621 (1669) Persisting job destination 
(destination id: torque)
galaxy.jobs.runners DEBUG 2016-10-31 10:17:08,630 Job [1669] queued (63.397 ms)
galaxy.jobs.handler INFO 2016-10-31 10:17:08,652 (1669) Job dispatched
galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Building dependency shell 
command for dependency 'velvet'
galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Find dependency velvet version 
1.2.10
galaxy.tools.deps DEBUG 2016-10-31 10:17:08,896 Resolver tool_shed_packages 
returned  (isnull? False)
galaxy.jobs.command_factory INFO 2016-10-31 10:17:08,945 Built script 
[/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh] for 
tool 
command[PACKAGE_BASE=/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f;
 export PACKAGE_BASE; . 
/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f/env.sh;
 velveth 2>&1 | grep "Version" | sed -e 's/Version //' > 
/home/galaxy/galaxy-prod/database/tmp/GALAXY_VERSION_STRING_1669 2>&1; python 
/home/galaxy/galaxy-prod/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/velvet/08256557922f/velvet/velveth_wrapper.py
 '/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat' 
'/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/dataset_4306_files' 
21  -fasta -short /home/galaxy/galaxy-prod/database/files/003/dataset_3537.dat]
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Building dependency shell 
command for dependency 'samtools'
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Find dependency samtools 
version None
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver tool_shed_packages 
returned  
(isnull? True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages 
returned  
(isnull? True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages 
returned  
(isnull? True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned 
 (isnull? 
True)
galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned 
 (isnull? 
True)
galaxy.tools.deps WARNING 2016-10-31 10:17:09,145 Failed to resolve dependency 
on 'samtools', ignoring
galaxy.jobs.runners DEBUG 2016-10-31 10:17:09,153 (1669) command is: mkdir -p 
working; cd working; 
/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh; 
return_code=$?; cd 
'/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669'; python 
"/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/set_metadata_KQTExg.py"
 "/home/galaxy/galaxy-prod/database/tmp/tmpLU0HQy" 
"/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/working/galaxy.json"
 
"/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_in_HistoryDatasetAssociation_4437_kIYXWa,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_kwds_HistoryDatasetAssociation_4437_ArCXyb,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_out_HistoryDatasetAssociation_4437_l28qWr,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_results_HistoryDatasetAssociation_4437_oTDjOd,/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_override_HistoryDatasetAssociation_4437_8VzdJw"
 5242880; sh -c "exit $return_code"
galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:09,202 (1669) submitting file 
/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/galaxy_1669.sh
galaxy.jobs.runners.drmaa INFO 2016-10-31 10:17:09,210 (1669) queued as 
1573.esilbac3a.esi.umontreal.ca
galaxy.jobs DEBUG 2016-10-31 10:17:09,214 (1669) Persisting job destination 
(destination id: torque)
galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:09,642 
(1669/1573.esilbac3a.esi.umontreal.ca) state change: job is running
galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:14,848 
(1669/1573.esilbac3a.esi.umontreal.ca) state change: job finished normally
galaxy.jobs.runners ERROR 2016-10-31 10:17:24,906 
(1669/1573.esilbac3a.esi.umontreal.ca) Job output not returned from 

Re: [galaxy-dev] galaxy - prokka - tbl2asn

[Fernandez Edgar]

Hey Guys,

I’ve fixed this problem by bypassing the system perl and compiling my own with 
all the right modules.
So I don’t get any more perl errors.

Also, I’ve realized that the latest prokka package (Revision: f5e44aad6498) has 
a dependency with the package_tbl2_asn_24_3 (Revision 41764d6a6a3c).

However, when you execute prokka, tbl2asn gives you the following error:
[tbl2asn] This copy of tbl2asn is more than a year old.  Please download the 
current version.

What would you suggest I do?

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Nicola Soranzo [mailto:nicola.sora...@gmail.com] On Behalf Of Nicola 
Soranzo
Sent: October-07-16 3:14 PM
To: Fernandez Edgar <edgar.fernan...@umontreal.ca>; galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] galaxy - bioperl - prokka

Hi Edgar,
presently the Prokka wrapper requires some dependencies (in particular some 
Perl modules) to be installed separetely, see the README file at 
https://toolshed.g2.bx.psu.edu/view/crs4/prokka/

We are planning to release an updated Prokka wrapper which will use Conda 
dependencies, you can track the progress at:

https://github.com/galaxyproject/tools-iuc/pull/985

Cheers,
Nicola
On 07/10/16 15:06, Fernandez Edgar wrote:
Hello,

I have installed prokka, perl and bioperl in my Galaxy’s ToolShed.
However, my users are getting this error during execution of prokka:
Can't locate Bio/Root/Version.pm in @INC (@INC contains: /usr/local/lib64/perl5 
/usr/local/share/perl5 /usr/lib64/perl5/vendor_perl 
/usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at 
/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
 line 29. BEGIN failed--compilation aborted at 
/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
 line 29.


I was under the impression that the galaxy’s tools will use the internal perl.
Do I just need to install Bio::Root:Version on my system installation of perl?
Please give me whatever information you can on this issue.

Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568






___

Please keep all replies on the list by using "reply all"

in your mail client.  To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

  https://lists.galaxyproject.org/



To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] galaxy - bioperl - prokka

[Fernandez Edgar]

Hello,

I have installed prokka, perl and bioperl in my Galaxy's ToolShed.
However, my users are getting this error during execution of prokka:
Can't locate Bio/Root/Version.pm in @INC (@INC contains: /usr/local/lib64/perl5 
/usr/local/share/perl5 /usr/lib64/perl5/vendor_perl 
/usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at 
/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
 line 29. BEGIN failed--compilation aborted at 
/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
 line 29.


I was under the impression that the galaxy's tools will use the internal perl.
Do I just need to install Bio::Root:Version on my system installation of perl?
Please give me whatever information you can on this issue.

Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
*  Bur. : 1-514-343-6111 poste 16568


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] delete user and all trace of user

[Fernandez Edgar]

Hello Martin,

Thanks for the answer.
I was hoping to find a way to remove the user or at the very least hide them 
from the ADMIN user page.

After a small research in the database, I found a column called "ACTIVE" in the 
galaxy_user table.
Can that column help me at all?

OR

I could write a script that search all the deleted/purged users, picks up their 
ID in the galaxy_user table and destroys all the lines in all the tables that 
has that FOREIGN KEY???

Let me know what you think…

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Martin Čech [mailto:mar...@bx.psu.edu]
Sent: September-08-16 12:30 PM
To: Fernandez Edgar <edgar.fernan...@umontreal.ca>; galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] delete user and all trace of user

Hi Edgar,

there is no built-in way in Galaxy to remove a user completely. 'Deleting' just 
flips the flag in the DB and 'purging' is related to their data/histories. You 
need to access the DB directly if you want to completely remove a user. Beware 
that there might be constraints/relations attached to the user row.

Thanks for using Galaxy,

Martin

On Thu, Sep 8, 2016 at 12:22 PM Fernandez Edgar 
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello gents!

I’ve successfully got my version 16.04 up and running and everything is still 
there (user, groups roles, data and tools).
Unfortunately , I’m still having a problem with deleting a user. I’ve purge him 
and then I deleted him.
but, he still shows up in my list of users as "purged".

Is there a way to completely remove him from galaxy?
Do I need to run a script in a cron job to clean my galaxy periodically?

Thanks in advance!

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] delete user and all trace of user

[Fernandez Edgar]

Hello gents!

I’ve successfully got my version 16.04 up and running and everything is still 
there (user, groups roles, data and tools).
Unfortunately , I’m still having a problem with deleting a user. I’ve purge him 
and then I deleted him.
but, he still shows up in my list of users as "purged".

Is there a way to completely remove him from galaxy?
Do I need to run a script in a cron job to clean my galaxy periodically?

Thanks in advance!

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hey guys,

I successfully fixed my problem.
Here's the solution if you would like to know:

1. download zlib-1.2.8.tar.gz
2. ./configure --prefix=/usr/local/zlib-1.2.8
3. make
4. make install
5. add the following in the file config/local_env.sh
export 
LD_LIBRARY_PATH=/usr/local/zlib-1.2.8${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}



Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
I  Bur. : 1-514-343-6111 poste 16568


-Original Message-
From: Peter Cock [mailto:p.j.a.c...@googlemail.com] 
Sent: May-26-16 3:32 AM
To: Fernandez Edgar <edgar.fernan...@umontreal.ca>; Nate Coraor 
<n...@bx.psu.edu>
Cc: galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] not sure what I can do about this...

Sorry Edgar,

It looks like something else is causing this error (and my guess about 
different versions of Linux on Galaxy and the cluster was
wrong):

ImportError: 
/home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pysam/libchtslib.so:
undefined symbol: gzopen64

It may be important how the pysam wheel was compiled, perhaps Nate may be able 
to comment (CC'd).

Peter

On Wed, May 25, 2016 at 12:40 PM, Fernandez Edgar 
<edgar.fernan...@umontreal.ca> wrote:
> Hello Peter,
>
> For now, my new installation of galaxy v16.01 is on one test desktop running 
> Oracle Linux 6.
> Before changing my production installation, I'm doing my due diligent.
>
> Let me know what you think...
>
> Edgar Fernandez
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hello Peter,

For now, my new installation of galaxy v16.01 is on one test desktop running 
Oracle Linux 6.
Before changing my production installation, I'm doing my due diligent.

Let me know what you think...

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
I  Bur. : 1-514-343-6111 poste 16568



-Original Message-
From: Peter Cock [mailto:p.j.a.c...@googlemail.com] 
Sent: May-24-16 5:40 PM
To: Fernandez Edgar <edgar.fernan...@umontreal.ca>
Cc: galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] not sure what I can do about this...

Hi Edgar,

Are your Galaxy server AND your cluster are all running Oracle Linux 6?

If yes, my guess about the cluster using an older version of Linux compared to 
the Galaxy server is unrelated.

Peter

On Tue, May 24, 2016 at 7:47 PM, Fernandez Edgar <edgar.fernan...@umontreal.ca> 
wrote:
> Hello Peter,
>
> What would your suggestion be?
> Wait until a new version of galaxy is compatible with my Oracle Linux 6 ?
>
> Best regards,
>
> Edgar Fernandez
> System Administrator (Linux)
> Direction Générale des Technologies de l'Information et de la 
> Communication Université de Montréal
>
> PAVILLON ROGER-GAUDRY, bureau X-210
> I  Bur. : 1-514-343-6111 poste 16568
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hello Peter,

What would your suggestion be?
Wait until a new version of galaxy is compatible with my Oracle Linux 6 ?

Best regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
I  Bur. : 1-514-343-6111 poste 16568


-Original Message-
From: Peter Cock [mailto:p.j.a.c...@googlemail.com] 
Sent: May-24-16 2:29 PM
To: Fernandez Edgar <edgar.fernan...@umontreal.ca>
Cc: galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] not sure what I can do about this...

Hi Fernandez,

We saw something very like that with mixed versions of Linux, specifically our 
Galaxy head node where pysam was installed had CentOS 7, but the node where 
Galaxy was trying to run the metadata detection was CentOS 6. Unfortunately I 
do not have a record of the exact traceback - but from memory it was the same 
problem in pysam with gzopen64.

(We'd already fought problems with Python mis-matches, GLIBC versions, etc and 
had carefully built Python on CentOS 6 in order to run on both CentOS 6 and 7.)

At this point we gave up, and will run the Galaxy node on the same version of 
Linux as the Cluster - and plan to upgrade the cluster before the Galaxy node. 
That way all the stuff Galaxy installs (e.g. via the ToolShed) ought to work on 
the cluster.

Peter

On Tue, May 24, 2016 at 7:08 PM, Fernandez Edgar <edgar.fernan...@umontreal.ca> 
wrote:
>
> Hello guys,
>
>
>
> I’m still stuck on this error.
>
> I’ve installed a brand new galaxy v16.01
>
> And when I start it, I get the big error mentioned below.
>
> Here’s the last message in the error:
>
> ImportError: 
> /home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pysam/lib
> chtslib.so: undefined symbol: gzopen64
>
>
>
> Thanks for your assistance!
>
>
>
> Cheers,
>
>
>
> Edgar Fernandez
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hello guys,

I’m still stuck on this error.
I’ve installed a brand new galaxy v16.01
And when I start it, I get the big error mentioned below.
Here’s the last message in the error:
ImportError: 
/home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pysam/libchtslib.so:
 undefined symbol: gzopen64

Thanks for your assistance!

Cheers,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Fernandez Edgar
Sent: May-19-16 8:48 AM
To: 'galaxy-...@bx.psu.edu' <galaxy-...@bx.psu.edu>
Subject: RE: [galaxy-dev] not sure what I can do about this...

Hey Guys,

I’ve fixed my first problem with the environment.
But I’m still getting the second one.

Also, did any of you guys try to deploy galaxy with Microsoft Azure?
I’m actually looking into it and I was wondering if you had any input…

Let me know if you have any info on my libchtslib.so: input error or Microsoft 
Azure.

Cheers!

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Fernandez Edgar
Sent: May-18-16 2:00 PM
To: galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>
Subject: RE: [galaxy-dev] not sure what I can do about this...

Hey Guys,

Hope everyone is doing well.

So I’m back at galaxy v16.01

I’m having a small problem.
My python system version is 2.6.6

So I installed python version 2.7.8 from  Software Collection Library release 
1.2 packages for Oracle Linux 6 
(x86_64)<https://ulns.dit.umontreal.ca/rhn/channels/ChannelDetail.do?cid=149>
This means I need to source an environment before being about to use it.
So what I’ve created a file config/local_env.sh with:
# Source python27
export PATH=/opt/rh/python27/root/usr/bin${PATH:+:${PATH}}
export 
LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export MANPATH=/opt/rh/python27/root/usr/share/man:${MANPATH}
# For systemtap
export 
XDG_DATA_DIRS=/opt/rh/python27/root/usr/share${XDG_DATA_DIRS:+:${XDG_DATA_DIRS}}
# For pkg-config
export 
PKG_CONFIG_PATH=/opt/rh/python27/root/usr/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}

These export comes directly with the installation of python 2.7.8 in the file 
/opt/rh/python27/enable.
But when I started my galaxy it didn’t stick.

I had to insert those exports into the file run.sh right before the following 
line:
./scripts/common_startup.sh $common_startup_args || exit 1

Any suggestions to make it work?

Also, I get this error :
Traceback (most recent call last):
  File "./scripts/paster.py", line 27, in 
serve.run()
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
1061, in run
invoke(command, command_name, options, args[1:])
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
1067, in invoke
exit_code = runner.run(args)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
223, in run
result = self.command()
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
639, in command
app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 295, in loadapp
return loadobj(APP, uri, name=name, **kw)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 319, in loadobj
global_conf=global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 344, in loadcontext
global_conf=global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 368, in _loadconfig
return loader.get_context(object_type, name, global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 506, in get_context
section)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 565, in _context_from_explicit
value = import_string(found_expr)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 125, in import_string
return pkg_resources.EntryPoint.parse("x=" + s).load(False)
  File 
"/home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pkg_resources.py",
 line 2260, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/web/buildapp.py", line 5, in 

from galaxy.webapps.galaxy.buildapp import app_factory
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/webapps/galaxy/buildapp.py", line 
15, in 
import galaxy.a

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hey Guys,

I’ve fixed my first problem with the environment.
But I’m still getting the second one.

Also, did any of you guys try to deploy galaxy with Microsoft Azure?
I’m actually looking into it and I was wondering if you had any input…

Let me know if you have any info on my libchtslib.so: input error or Microsoft 
Azure.

Cheers!

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Fernandez Edgar
Sent: May-18-16 2:00 PM
To: galaxy-...@bx.psu.edu
Subject: RE: [galaxy-dev] not sure what I can do about this...

Hey Guys,

Hope everyone is doing well.

So I’m back at galaxy v16.01

I’m having a small problem.
My python system version is 2.6.6

So I installed python version 2.7.8 from  Software Collection Library release 
1.2 packages for Oracle Linux 6 
(x86_64)<https://ulns.dit.umontreal.ca/rhn/channels/ChannelDetail.do?cid=149>
This means I need to source an environment before being about to use it.
So what I’ve created a file config/local_env.sh with:
# Source python27
export PATH=/opt/rh/python27/root/usr/bin${PATH:+:${PATH}}
export 
LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export MANPATH=/opt/rh/python27/root/usr/share/man:${MANPATH}
# For systemtap
export 
XDG_DATA_DIRS=/opt/rh/python27/root/usr/share${XDG_DATA_DIRS:+:${XDG_DATA_DIRS}}
# For pkg-config
export 
PKG_CONFIG_PATH=/opt/rh/python27/root/usr/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}

These export comes directly with the installation of python 2.7.8 in the file 
/opt/rh/python27/enable.
But when I started my galaxy it didn’t stick.

I had to insert those exports into the file run.sh right before the following 
line:
./scripts/common_startup.sh $common_startup_args || exit 1

Any suggestions to make it work?

Also, I get this error :
Traceback (most recent call last):
  File "./scripts/paster.py", line 27, in 
serve.run()
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
1061, in run
invoke(command, command_name, options, args[1:])
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
1067, in invoke
exit_code = runner.run(args)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
223, in run
result = self.command()
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
639, in command
app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 295, in loadapp
return loadobj(APP, uri, name=name, **kw)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 319, in loadobj
global_conf=global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 344, in loadcontext
global_conf=global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 368, in _loadconfig
return loader.get_context(object_type, name, global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 506, in get_context
section)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 565, in _context_from_explicit
value = import_string(found_expr)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 125, in import_string
return pkg_resources.EntryPoint.parse("x=" + s).load(False)
  File 
"/home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pkg_resources.py",
 line 2260, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/web/buildapp.py", line 5, in 

from galaxy.webapps.galaxy.buildapp import app_factory
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/webapps/galaxy/buildapp.py", line 
15, in 
import galaxy.app
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/app.py", line 6, in 
from galaxy import config, jobs
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/jobs/__init__.py", line 19, in 

from galaxy import model, util
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/model/__init__.py", line 26, in 

import galaxy.datatypes.registry
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/datatypes/registry.py", line 9, 
in 
import tabular
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/datatypes/tabular.py", line 18, 
in 
from galaxy.datatypes.sniff import get_headers
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/datatypes/sniff.py", line 19, in 

from galaxy.datatypes.binary import Binary
  File &

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hey Guys,

Hope everyone is doing well.

So I’m back at galaxy v16.01

I’m having a small problem.
My python system version is 2.6.6

So I installed python version 2.7.8 from  Software Collection Library release 
1.2 packages for Oracle Linux 6 
(x86_64)
This means I need to source an environment before being about to use it.
So what I’ve created a file config/local_env.sh with:
# Source python27
export PATH=/opt/rh/python27/root/usr/bin${PATH:+:${PATH}}
export 
LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export MANPATH=/opt/rh/python27/root/usr/share/man:${MANPATH}
# For systemtap
export 
XDG_DATA_DIRS=/opt/rh/python27/root/usr/share${XDG_DATA_DIRS:+:${XDG_DATA_DIRS}}
# For pkg-config
export 
PKG_CONFIG_PATH=/opt/rh/python27/root/usr/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}

These export comes directly with the installation of python 2.7.8 in the file 
/opt/rh/python27/enable.
But when I started my galaxy it didn’t stick.

I had to insert those exports into the file run.sh right before the following 
line:
./scripts/common_startup.sh $common_startup_args || exit 1

Any suggestions to make it work?

Also, I get this error :
Traceback (most recent call last):
  File "./scripts/paster.py", line 27, in 
serve.run()
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
1061, in run
invoke(command, command_name, options, args[1:])
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
1067, in invoke
exit_code = runner.run(args)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
223, in run
result = self.command()
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/serve.py", line 
639, in command
app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 295, in loadapp
return loadobj(APP, uri, name=name, **kw)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 319, in loadobj
global_conf=global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 344, in loadcontext
global_conf=global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 368, in _loadconfig
return loader.get_context(object_type, name, global_conf)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 506, in get_context
section)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 565, in _context_from_explicit
value = import_string(found_expr)
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/util/pastescript/loadwsgi.py", 
line 125, in import_string
return pkg_resources.EntryPoint.parse("x=" + s).load(False)
  File 
"/home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pkg_resources.py",
 line 2260, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/web/buildapp.py", line 5, in 

from galaxy.webapps.galaxy.buildapp import app_factory
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/webapps/galaxy/buildapp.py", line 
15, in 
import galaxy.app
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/app.py", line 6, in 
from galaxy import config, jobs
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/jobs/__init__.py", line 19, in 

from galaxy import model, util
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/model/__init__.py", line 26, in 

import galaxy.datatypes.registry
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/datatypes/registry.py", line 9, 
in 
import tabular
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/datatypes/tabular.py", line 18, 
in 
from galaxy.datatypes.sniff import get_headers
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/datatypes/sniff.py", line 19, in 

from galaxy.datatypes.binary import Binary
  File "/home/galaxy/galaxy-v16.01/lib/galaxy/datatypes/binary.py", line 13, in 

import pysam
  File 
"/home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pysam/__init__.py",
 line 1, in 
from pysam.libchtslib import *
ImportError: 
/home/galaxy/galaxy-v16.01/.venv/lib/python2.7/site-packages/pysam/libchtslib.so:
 undefined symbol: gzopen64
Removing PID file paster.pid

PLEASE HELP ME !

Best regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy 

Re: [galaxy-dev] errors installing tool dependencies...

[Fernandez Edgar]

Hello Martin.

I changed the setting “new_file_path” like you said and the same error occurs 
(with the full path obviously).
I check that the full path exists. And is rwx for the galaxy user.
Any other ideas I could try?

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Martin Čech [mailto:mar...@bx.psu.edu]
Sent: January-25-16 9:33 AM
To: Fernandez Edgar <edgar.fernan...@umontreal.ca>; galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] errors installing tool dependencies...

Hi Edgar,

I am not sure about this one. Please try setting 'new_file_path' in your 
galaxy.ini to an absolute path, try again and check whether something changes.

Martin

On Mon, Jan 25, 2016 at 9:23 AM Fernandez Edgar 
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello Martin,

Thank you!
Your instructions for my problem with Perl package worked.
For the URL of hmmer, it is wrong and I will report it like you suggested it.

Finally, a new error occurred for bowtie.
Would you please take a look?

My Sincere Salutations,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568


From: Martin Čech [mailto:mar...@bx.psu.edu<mailto:mar...@bx.psu.edu>]
Sent: January-22-16 2:34 PM
To: Fernandez Edgar 
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>>; 
galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>
Subject: Re: [galaxy-dev] errors installing tool dependencies...

Hi Edgar,

for the package_perl issue please try installing updates to the dependencies 
(package_gnu_coreutils and package_perl in their respective versions) and then 
return to this screen and click the 'repository action' button and then 'manage 
dependencies' from the menu. Uninstall both dependencies (gnu_coreutil and 
perl) and then install them again.
That should pick up the newest versions and you should be set.

Regarding the hmmer that is a more prosaic problem, at the time of the 
installation the software was not available at the given URL. Check the 
machine's internet connection and if it is ok try downloading the file 
manually. If it succeeds the problem might have been on their side and is ok 
now. If it does not the problem can still be on their side and in that case 
either wait a bit and try again or report the issue at 
https://github.com/galaxyproject/tools-iuc/issues

thank you for using Galaxy.

Martin
On Fri, Jan 22, 2016 at 2:24 PM Fernandez Edgar 
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello guys,

Happy new year everyone!
Wish you all the best!

I have a couple of small problems I’ve attached as pictures…
Can you please help me?

INFO: My galaxy installation and database is updated today!

My sincere salutations,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
•  Bur. : 1-514-343-6111 poste 16568
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] FASTQ Groomer tool

[Fernandez Edgar]

Hello gents,

I just installed "FASTQ Groomer tool" for the teachers to try it out and I've 
been getting the following error than I do not understand:

galaxy.jobs DEBUG 2015-11-04 08:16:19,957 (756) Working directory for job is: 
/home/galaxy/galaxy-prod/database/job_working_directory/000/756
galaxy.jobs.handler WARNING 2015-11-04 08:16:19,958 (756) Tool 
'toolshed.g2.bx.psu.edu/repos/devteam/fastq_groomer/fastq_groomer/1.0.4' 
removed from tool config, unable to run job
galaxy.jobs.handler ERROR 2015-11-04 08:16:20,159 failure running job 756
Traceback (most recent call last):
  File "/home/galaxy/galaxy-prod/lib/galaxy/jobs/handler.py", line 273, in 
__monitor_step
job_state = self.__check_job_state( job )
  File "/home/galaxy/galaxy-prod/lib/galaxy/jobs/handler.py", line 342, in 
__check_job_state
state, job_destination = self.__verify_job_ready( job, job_wrapper )
  File "/home/galaxy/galaxy-prod/lib/galaxy/jobs/handler.py", line 362, in 
__verify_job_ready
job_wrapper.fail( e )
  File "/home/galaxy/galaxy-prod/lib/galaxy/jobs/__init__.py", line 984, in fail
self.sa_session.flush()
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/scoping.py", line 150, in do
return getattr(self.registry(), name)(*args, **kwargs)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2004, in 
flush
self._flush(objects)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2122, in 
_flush
transaction.rollback(_capture_exception=True)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/util/langhelpers.py", line 60, 
in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2086, in 
_flush
flush_context.execute()
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 373, 
in execute
rec.execute(self)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 532, 
in execute
uow
 File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 170, 
in save_obj
mapper, table, update)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 673, 
in _emit_update_statements
execute(statement, multiparams)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 914, in 
execute
return meth(self, multiparams, params)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line 323, in 
_execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1010, in 
_execute_clauseelement
compiled_sql, distilled_params
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1146, in 
_execute_context
context)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1332, in 
_handle_dbapi_exception
exc_info
  File "build/bdist.linux-x86_64/egg/sqlalchemy/util/compat.py", line 199, in 
raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1139, in 
_execute_context
context)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 442, 
in do_execute
cursor.execute(statement, parameters)
  File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/cursors.py", line 173, in 
execute
self.errorhandler(self, exc, value)
  File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 36, in 
defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: (_mysql_exceptions.ProgrammingError) (1064, "You have an 
error in your SQL syntax; check the manual that corresponds to your MySQL 
server version for the right syntax to use near '), blurb='tool error' WHERE 
history_dataset_association.id = 1498' at line 1") [SQL: u'UPDATE 
history_dataset_association SET update_time=%s, info=%s, blurb=%s WHERE 
history_dataset_association.id = %s'] [parameters: (datetime.datetime(2015, 11, 
4, 13, 16, 20, 153739), ('This tool was disabled before the job completed.  
Please contact your Galaxy administrator.',), 'tool error', 1498L)]


According to the tool shed repo, fastq_groomer is installed properly.
Mysql is at its latest for oracle linux 6:
[0]# mysql --version
mysql  Ver 14.14 Distrib 5.1.73, for redhat-linux-gnu (x86_64) using readline 
5.1

[0]# rpm -qa | grep -i mysql
mysql-5.1.73-5.el6_6.x86_64


The only thing I have not done is update galaxy for the last 6 months because 
they're students using it and I do not want to change anything while it's in 
production.
I'm a little bit lost: Would you please indicate what I should check next?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
*  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218

___
Please keep all 

[galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hello Gents,

So someone asked me to install the package Prokka on Galaxy.
I did and everything went well!

However, the following error comes up, when they try to work with Prokka:

Fatal error: Exit code 2 ()

[tbl2asn] This copy of tbl2asn is more than a year old.  Please download the 
current version.

[20:20:53] Could not run command: tbl2asn -V b -a r10k -l paired-ends -M n -N 1 
-y 'Annotated using prokka 1.10 from http://www.vicbioinformatics.com' -Z 
outdir\/prokka\.err -i outdir\/prokka\.fsa 2> /dev/null


According to a google search, there is very little on this error.
And it has been around for a while but I didn't find a posted solution.

Any suggestions?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
*  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Hello Björn,

Thank you for answering!
When I installed Prokka from the Galaxy ToolShed, all the dependencies were 
installed too.
tbl2asn is one of those dependencies...
How would I replace it?

Cordialement / Regards, 
Edgar Fernandez


-Message d'origine-
De : Björn Grüning [mailto:bjoern.gruen...@gmail.com] 
Envoyé : September-29-15 8:10 AM
À : Fernandez Edgar; galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] not sure what I can do about this...

Hi Edgar,

please download the latest version of tbl2asn from here:

ftp://ftp.ncbi.nih.gov/toolbox/ncbi_tools/converters/by_program/tbl2asn/

and replace it with your version, this will probably fix your error.

Cheers,
Bjoern

Am 29.09.2015 um 14:05 schrieb Fernandez Edgar:
> Hello Gents,
> 
> So someone asked me to install the package Prokka on Galaxy.
> I did and everything went well!
> 
> However, the following error comes up, when they try to work with Prokka:
> 
> Fatal error: Exit code 2 ()
> 
> [tbl2asn] This copy of tbl2asn is more than a year old.  Please download the 
> current version.
> 
> [20:20:53] Could not run command: tbl2asn -V b -a r10k -l paired-ends 
> -M n -N 1 -y 'Annotated using prokka 1.10 from 
> http://www.vicbioinformatics.com' -Z outdir\/prokka\.err -i 
> outdir\/prokka\.fsa 2> /dev/null
> 
> 
> According to a google search, there is very little on this error.
> And it has been around for a while but I didn't find a posted solution.
> 
> Any suggestions?
> 
> Cordialement / Regards,
> 
> Edgar Fernandez
> System Administrator (Linux)
> Direction Générale des Technologies de l'Information et de la 
> Communication
> *  Bur. : 1-514-343-6111 poste 16568
> 
> Université de Montréal
> PAVILLON ROGER-GAUDRY, bureau X-218
> 
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this and other 
> Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] not sure what I can do about this...

[Fernandez Edgar]

Guys, I can express how thankful I am for your help!!!

Personally, I would comment out the following line in the tbl2asn code:

8869<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/source/demo/tbl2asn.c#L8869>
   if 
(MoreThanYearOld<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/ident?i=MoreThanYearOld>
 ()) {
8870<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/source/demo/tbl2asn.c#L8870>
 too_old = 
TRUE<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/ident?i=TRUE>;
8871<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/source/demo/tbl2asn.c#L8871>
 Message<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/ident?i=Message> 
(MSG_POST<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/ident?i=MSG_POST>, 
"This copy of tbl2asn is more than a year old.  Please download the current 
version.");
8872<http://www.ncbi.nlm.nih.gov/IEB/ToolBox/C_DOC/lxr/source/demo/tbl2asn.c#L8872>
   }



In the meantime, would you suggest I change the files located inside my galaxy 
installation?
Or should I wait for Philip’s release?

I ask because I found the following binary files:
-rwxr-x--- 1 galaxy users 7349340 Sep 15 08:52
./tool-data/toolshed.dependency.dir/tbl2asn/23.7/takadonet/package_tbl2asn_23_7/2271d52a8531/bin/tbl2asn

-rwxrwxr-x 1 galaxy users 12675348 Jul 28  2014
./tool-data/toolshed.dependency.dir/prokka/1.10/crs4/prokka/3ad7ef0ba385/binaries/darwin/tbl2asn

-rwxrwxr-x 1 galaxy users 6872896 Jul 28  2014
./tool-data/toolshed.dependency.dir/prokka/1.10/crs4/prokka/3ad7ef0ba385/binaries/linux/tbl2asn

Oddly, they’re all different.
What exactly can I do with this and how can I compile the tblasn.c file?

Cordialement / Regards,
Edgar Fernandez

De : Philip Mabon [mailto:philipma...@gmail.com]
Envoyé : September-29-15 10:27 AM
À : Peter Cock; Fernandez Edgar; Nicola Soranzo (TGAC)
Cc : Björn Grüning; galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] not sure what I can do about this...

I will make a new package for tbl2asn. Our lab is not too happy that tbl2asn 
expires and causes lots of issues for us.



On Tue, Sep 29, 2015 at 9:24 AM Peter Cock 
<p.j.a.c...@googlemail.com<mailto:p.j.a.c...@googlemail.com>> wrote:
On Tue, Sep 29, 2015 at 1:12 PM, Fernandez Edgar
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>> wrote:
> Hello Björn,
>
> Thank you for answering!
> When I installed Prokka from the Galaxy ToolShed, all the dependencies were 
> installed too.
> tbl2asn is one of those dependencies...
> How would I replace it?
>
> Cordialement / Regards,
> Edgar Fernandez

The way the NCBI deliberately make tbl2asn expire is very
annoying for packaging and breaks strict reproducibility :(

I think Bjoern was suggesting manually updating the copy of tbl2asn
which the Tool Shed had downloaded (as a quick short term solution).
You'd need to find this file within the Tool Shed dependencies folder...

Philip - can you update package_tbl2asn_23_7 or add a new
package if the version number has changed? Or would you like
the IUC group to look after this in future?

https://toolshed.g2.bx.psu.edu/view/takadonet/package_tbl2asn_23_7/

Nicola - once that's done, can you update the prokka dependency
list - or would that fall to your old colleagues at CRS4?

https://toolshed.g2.bx.psu.edu/view/crs4/prokka/

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] need help badly

[Fernandez Edgar]

Hello Josh,

Thank you so much for the answer.
I thought of that but it doesn’t solve my problem.
I need to create a routine that looks inside the new job’s directory :
So for example, a process that runs every second and looks inside
/home/galaxy/galaxy-prod/database/job_working_directory/000
And if there is a new job (new folder), it looks/enters inside with cd.

However, is it possible to program galaxy server to create the job’s directory 
and enter the directory so when a torque slave writes inside the directory, the 
master is already inside?

Cordialement / Regards,
Edgar Fernandez

De : Josh Woodring [mailto:woodring.jos...@gmail.com]
Envoyé : September-17-15 11:14 AM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] need help badly

Hi Edgar,
I am a pretty big newbie to Galaxy but I have had similar problems on my Dev 
build with grabbing outputs from the working directories. Its a temporary fix 
but I have put routines in a bash wrapper that uses a working directory that I 
made and know. If your cluster has access to a shared drive that would work 
that way it may work.
Josh

On Thu, Sep 17, 2015 at 11:05 PM, Fernandez Edgar 
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello gents,

I figured out something: it has nothing to do with galaxy but maybe one of you 
could explain me why I’m having this behavior :
I connect ssh on the torque master (where the galaxy web service is running) 
and:

1.   su - galaxy

2.   cd /home/galaxy/galaxy-prod/database/job_working_directory/000

I execute a job #1234, it fails with the error mentioned below.
I execute the exact same job #1235 and while it’s running I enter the directory 
1235 --> job finishes successfully.

My directory /home/galaxy is an nfs mount in /etc/fstab.
Any ideas?

Cordialement / Regards,
Edgar Fernandez

De : Fernandez Edgar
Envoyé : September-17-15 9:44 AM
À : galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>
Objet : [galaxy-dev] need help badly

Hello gents,

I hope everyone is doing ok…
I’m in need of your help so badly it’s a little bit urgent:

So my galaxy wasn’t used in a while but the last time everything was working 
well:

1.   The web-site was functional

2.   The local handler upload the files locally on the master

3.   Torque was shooting all the jobs correctly and getting an answer back.

But today, one of my students tried some typical trimmomatic and all his jobs 
came back with the following error:
Job output not returned from cluster.

However, when I checked his datasets with that error, the view data (icon that 
is an eye) had some results.
Also, when I looking inside the logs, I got:
Job output not returned from cluster: [Errno 2] No such file or directory: 
'/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o'
But that file galaxy_406.o existed and had
TrimmomaticPE: Completed successfully

I tried changing the value from 0 to 5 of "retry_job_output_collection" in the 
config/galaxy.ini file without any success.

I’ve look through the internet and I saw a lot of people had that error but no 
one found a solution.

Can you guys please help?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
•  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
Nil mihi rescribas, tu tamen ipse veni!
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] need help badly

[Fernandez Edgar]

Hello gents,

I hope everyone is doing ok...
I'm in need of your help so badly it's a little bit urgent:

So my galaxy wasn't used in a while but the last time everything was working 
well:

1.   The web-site was functional

2.   The local handler upload the files locally on the master

3.   Torque was shooting all the jobs correctly and getting an answer back.

But today, one of my students tried some typical trimmomatic and all his jobs 
came back with the following error:
Job output not returned from cluster.

However, when I checked his datasets with that error, the view data (icon that 
is an eye) had some results.
Also, when I looking inside the logs, I got:
Job output not returned from cluster: [Errno 2] No such file or directory: 
'/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o'
But that file galaxy_406.o existed and had
TrimmomaticPE: Completed successfully

I tried changing the value from 0 to 5 of "retry_job_output_collection" in the 
config/galaxy.ini file without any success.

I've look through the internet and I saw a lot of people had that error but no 
one found a solution.

Can you guys please help?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
*  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] need help badly

[Fernandez Edgar]

Thank you again for the precisions.

On another note, I have a user that when he goes into his History Options --> 
Saved Histories -->Advanced Search -->status: all, and he views one of his 
deleted histories, he sees a job with those spinning dots. However, everything 
is deleted: history, dataset, …

And he also sees two different memory usage: on top of the web page he sees 
Using 2.7Gb and in his only active history he sees 1.1Gb.

I’m guessing these two issues are related.

Anything I can do to help him?

Cordialement / Regards,
Edgar Fernandez

De : Josh Woodring [mailto:woodring.jos...@gmail.com]
Envoyé : September-17-15 12:34 PM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] need help badly

Hi Edgar,
Hm, I haven't gotten it to work but galaxy is supposed to support looking in 
the working directory for the file. Its likely for reasons similar to your 
error. If you have access to the program's script you could try to force it to 
write the results somewhere else. I would be surprised if ".." didn't work. the 
job_working_directory/000 folder always exists.  Of course these are similar to 
my prior suggestion.
I haven't had to interact with more cloud stuff than the google APIs for 
classes or shared drives. If you want to do a script do something along the 
following and at it to the programs run at system start up.
Point A.
For folder in job_working_directory/000 as $var
cd $var
cd ..
end loop
GOTO A.
Lastly, and I probably should have started here, any chance that its a folder 
permissions issue?
Good Luck
Josh


On Thu, Sep 17, 2015 at 11:35 PM, Fernandez Edgar 
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello Josh,

Thank you so much for the answer.
I thought of that but it doesn’t solve my problem.
I need to create a routine that looks inside the new job’s directory :
So for example, a process that runs every second and looks inside
/home/galaxy/galaxy-prod/database/job_working_directory/000
And if there is a new job (new folder), it looks/enters inside with cd.

However, is it possible to program galaxy server to create the job’s directory 
and enter the directory so when a torque slave writes inside the directory, the 
master is already inside?

Cordialement / Regards,
Edgar Fernandez

De : Josh Woodring 
[mailto:woodring.jos...@gmail.com<mailto:woodring.jos...@gmail.com>]
Envoyé : September-17-15 11:14 AM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>
Objet : Re: [galaxy-dev] need help badly

Hi Edgar,
I am a pretty big newbie to Galaxy but I have had similar problems on my Dev 
build with grabbing outputs from the working directories. Its a temporary fix 
but I have put routines in a bash wrapper that uses a working directory that I 
made and know. If your cluster has access to a shared drive that would work 
that way it may work.
Josh

On Thu, Sep 17, 2015 at 11:05 PM, Fernandez Edgar 
<edgar.fernan...@umontreal.ca<mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello gents,

I figured out something: it has nothing to do with galaxy but maybe one of you 
could explain me why I’m having this behavior :
I connect ssh on the torque master (where the galaxy web service is running) 
and:

1.   su - galaxy

2.   cd /home/galaxy/galaxy-prod/database/job_working_directory/000

I execute a job #1234, it fails with the error mentioned below.
I execute the exact same job #1235 and while it’s running I enter the directory 
1235 --> job finishes successfully.

My directory /home/galaxy is an nfs mount in /etc/fstab.
Any ideas?

Cordialement / Regards,
Edgar Fernandez

De : Fernandez Edgar
Envoyé : September-17-15 9:44 AM
À : galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>
Objet : [galaxy-dev] need help badly

Hello gents,

I hope everyone is doing ok…
I’m in need of your help so badly it’s a little bit urgent:

So my galaxy wasn’t used in a while but the last time everything was working 
well:

1.   The web-site was functional

2.   The local handler upload the files locally on the master

3.   Torque was shooting all the jobs correctly and getting an answer back.

But today, one of my students tried some typical trimmomatic and all his jobs 
came back with the following error:
Job output not returned from cluster.

However, when I checked his datasets with that error, the view data (icon that 
is an eye) had some results.
Also, when I looking inside the logs, I got:
Job output not returned from cluster: [Errno 2] No such file or directory: 
'/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o'
But that file galaxy_406.o existed and had
TrimmomaticPE: Completed successfully

I tried changing the value from 0 to 5 of "retry_job_output_collection" in the 
config/galaxy.ini file without any success.

I’ve look through the internet and I saw a lot of people had that error but no 
one found a solu

[galaxy-dev] need help badly

[Fernandez Edgar]

Hello gents,

I hope everyone is doing ok...
I'm in need of your help so badly it's a little bit urgent:

So my galaxy wasn't used in a while but the last time everything was working 
well:

1.   The web-site was functional

2.   The local handler upload the files locally on the master

3.   Torque was shooting all the jobs correctly and getting an answer back.

But today, one of my students tried some typical trimmomatic and all his jobs 
came back with the following error:
Job output not returned from cluster.

However, when I checked his datasets with that error, the view data (icon that 
is an eye) had some results.
Also, when I looking inside the logs, I got:
Job output not returned from cluster: [Errno 2] No such file or directory: 
'/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o'
But that file galaxy_406.o existed and had
TrimmomaticPE: Completed successfully

I tried changing the value from 0 to 5 of "retry_job_output_collection" in the 
config/galaxy.ini file without any success.

I've look through the internet and I saw a lot of people had that error but no 
one found a solution.

Can you guys please help?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
*  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] galaxy report page - server error solution?

[Fernandez Edgar]

Good catch on that one!
There wasn't originally the date_trunc function there but that groups the jobs 
by months together.
Thanks!!!

Cordialement / Regards, 
Edgar Fernandez

-Message d'origine-
De : Hans-Rudolf Hotz [mailto:h...@fmi.ch] 
Envoyé : February-05-15 10:49 AM
À : Fernandez Edgar; galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy report page - server error solution?

Hi again

for jobs.py I have modified the same lines as you,
   plus line 285 (group_by)

for users.py I have modified the same lines as you


Hans-Rudolf



On 02/05/2015 04:20 PM, Fernandez Edgar wrote:
 Actually, here’s what I came up with :

 diff users.py ARCHIVES/users.py_20150205_9.28.18

 23c23

  q = sa.select( ( sa.func.date(
 galaxy.model.User.table.c.create_time ).label( 'date' ),

 ---

 q = sa.select( ( sa.func.date_trunc( 'month', sa.func.date( 
 galaxy.model.User.table.c.create_time ) ).label( 'date' ),

 26c26

 group_by = [ sa.func.year (
 galaxy.model.User.table.c.create_time ), sa.func.month ( 
 galaxy.model.User.table.c.create_time ) ],

 ---

group_by = [ sa.func.date_trunc( 'month', 
 sa.func.date( galaxy.model.User.table.c.create_time ) ) ],

 48c48

  q = sa.select( ( sa.func.date(
 galaxy.model.User.table.c.create_time ).label( 'date' ),

 ---

 q = sa.select( ( sa.func.date_trunc( 'day', sa.func.date( 
 galaxy.model.User.table.c.create_time ) ).label( 'date' ),

 53c53

 group_by = [ sa.func.day( sa.func.date(
 galaxy.model.User.table.c.create_time ) ) ],

 ---

group_by = [ sa.func.date_trunc( 'day', 
 sa.func.date( galaxy.model.User.table.c.create_time ) ) ],

 79c79

  q = sa.select( ( sa.func.date(
 galaxy.model.User.table.c.create_time ).label( 'date' ),

 ---

 q = sa.select( ( sa.func.date_trunc( 'day', sa.func.date( 
 galaxy.model.User.table.c.create_time ) ).label( 'date' ),

 diff jobs.py ARCHIVES/jobs.py_20150205_9.28.22

 282c282

  jobs_by_month = sa.select( ( sa.func.date(
 model.Job.table.c.create_time ).label( 'date' ),

 ---

 jobs_by_month = sa.select( ( sa.func.date_trunc( 'month', 
 model.Job.table.c.create_time ).label( 'date' ),

 315c315

  jobs_in_error_by_month = sa.select( ( sa.func.date(
 model.Job.table.c.create_time ).label( 'date' ),

 ---

 jobs_in_error_by_month = sa.select( ( sa.func.date_trunc( 
 'month', sa.func.date( model.Job.table.c.create_time ) ).label( 
 'date' ),

 320c320

 group_by=[ sa.func.year (
 model.Job.table.c.create_time ), sa.func.month ( sa.func.date( 
 model.Job.table.c.create_time ) ) ],

 ---

group_by=[ 
 sa.func.date_trunc( 'month', sa.func.date( 
 model.Job.table.c.create_time ) ) ],

 364c364

  q = sa.select( ( sa.func.date( model.Job.table.c.create_time
 ).label( 'date' ),

 ---

 q = sa.select( ( sa.func.date_trunc( 'month', sa.func.date( 
 model.Job.table.c.create_time ) ).label( 'date' ),

 371c371

 group_by=[ sa.func.year (
 model.Job.table.c.create_time ), sa.func.month ( sa.func.date( 
 model.Job.table.c.create_time ) ) ],

 ---

group_by=[ sa.func.date_trunc( 'month', 
 sa.func.date( model.Job.table.c.create_time ) ) ],

 420c420

  q = sa.select( ( sa.func.date( model.Job.table.c.create_time
 ).label( 'date' ),

 ---

 q = sa.select( ( sa.func.date_trunc( 'month', sa.func.date( 
 model.Job.table.c.create_time ) ).label( 'date' ),

 425c425

 group_by=[ sa.func.year (
 model.Job.table.c.create_time ), sa.func.month ( sa.func.date( 
 model.Job.table.c.create_time ) ) ],

 ---

group_by=[ sa.func.date_trunc( 'month', 
 sa.func.date( model.Job.table.c.create_time ) ) ],

 Let me know what you think!!!

 Cordialement / Regards,

 Edgar Fernandez

 *De :*Fernandez Edgar
 *Envoyé :* February-05-15 9:49 AM
 *À :* 'galaxy-...@bx.psu.edu'
 *Objet :* galaxy report page - server error solution?

 Hello guys,

 So I’ve been working on the Galaxy report page.

 It works fine however I’m using MySQL and I’m getting some server 
 errors on some webpages.

 I found the solution:
 https://lists.galaxyproject.org/pipermail/galaxy-dev/2009-August/00064
 1.html

 But can some please confirm the following:

 1.The location of the files users.py and jobs.py is 
 /home/galaxy/galaxy-prod/lib/galaxy/webapps/reports/controllers

 2.There is three (3) select closes using five (5) date_trunc
 functions in users.py.

 3.There is eight (8) select closes using fourteen (4) date_trunc
 functions in jobs.py.

 4.Here is an example of what I can come up with. Is it correct?

 ·WRONG:

 jobs_in_error_by_month = sa.select( ( sa.func.date_trunc( 'month', 
 sa.func.date( model.Job.table.c.create_time ) ).label( 'date' ), 
 sa.func.count( model.Job.table.c.id

[galaxy-dev] data sharing without duplication?

[Fernandez Edgar]

Good morning Gents,

I hope everyone is well.

I was wondering if it's possible to share data on Galaxy without all the users 
downloading the data to their history.
Let me explain, teachers will be sharing data with their students.
And the students will need to work with that specific data.
However, I would like them to be able to read that data without replicating it 
on their history.
Is that possible?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
*  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] show stopper for my galaxy installation

[Fernandez Edgar]

Hello Eric,

It works great! Thank you!
Is it possible to use the same logic to overwrite an input parameters to a job.
For example, trimmomatic executes by default with the input parameters 
-threads 6”.
Can I change that parameter to threads 4?

Cordialement / Regards,
Edgar Fernandez

De : Eric Rasche [mailto:e...@tamu.edu]
Envoyé : January-22-15 3:05 PM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] show stopper for my galaxy installation

The tool to fetch data isn't being run on your submit node but instead on the 
cluster, which is the problem.

It's a fairly easy to solve problem:

  *   In your job_conf you'll need to define a way to run jobs locally (either 
via local runner, or via a special queue dedicated to your submit node)
  *   Then you'll need to specify that the upload tool is forced to that job 
runner.
Something like this (untested, probably invalid xml... )

plugins workers=4
  plugin id=local type=runner 
load=galaxy.jobs.runners.local:LocalJobRunner /
  !-- your torque runner here --
/plugins
destinations default=cluster
  destination id=local runner=local
  !-- your cluster dest here --
/destinations
tools

  !-- we force the upload tool to the local job runner, thereby bypassing the 
issue of internet access on the cluster nodes --
  tool id=upload1 destination=local /
/tools

Cheers,
Eric

2015-01-22 13:51 GMT-06:00 Fernandez Edgar 
edgar.fernan...@umontreal.camailto:edgar.fernan...@umontreal.ca:
Hello gents,

I started this new thread because I have a showstopper for my installation of 
galaxy.
Here’s my situation:

1.   I have a galaxy server that has internet access.

2.   My galaxy server is also my torque server and my only torque submit 
node.

3.   I have three torque compute nodes that DOESN’T have internet access.

Now, I’ve tried uploading a file (larger than 2Gb) via and URL and it fails.
However, I’ve added my galaxy server as a compute node and re-tried to upload 
the same file and I made sure the job will run on that machine and it works.

Now, my compute nodes are in a network that is completely protected from the 
outside world.
So no internet access.

What are my option in this case?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
•  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
Eric Rasche
Programmer II

Center for Phage Technology
Rm 312A, BioBio
Texas AM University
College Station, TX 77843
404-692-2048
e...@tamu.edumailto:e...@tamu.edu
rasche.e...@yandex.rumailto:rasche.e...@yandex.ru

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] show stopper for my galaxy installation

[Fernandez Edgar]

Hello guys!

I’m impressed!
You guys really thought of everything!
This is amazing!
I’ll try it and give you some news asap.
I’m so impressed that I used too many exclamation points in this email!!!

Cordialement / Regards,
Edgar Fernandez

De : Eric Rasche [mailto:e...@tamu.edu]
Envoyé : January-22-15 3:05 PM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] show stopper for my galaxy installation

The tool to fetch data isn't being run on your submit node but instead on the 
cluster, which is the problem.

It's a fairly easy to solve problem:

  *   In your job_conf you'll need to define a way to run jobs locally (either 
via local runner, or via a special queue dedicated to your submit node)
  *   Then you'll need to specify that the upload tool is forced to that job 
runner.
Something like this (untested, probably invalid xml... )

plugins workers=4
  plugin id=local type=runner 
load=galaxy.jobs.runners.local:LocalJobRunner /
  !-- your torque runner here --
/plugins
destinations default=cluster
  destination id=local runner=local
  !-- your cluster dest here --
/destinations
tools

  !-- we force the upload tool to the local job runner, thereby bypassing the 
issue of internet access on the cluster nodes --
  tool id=upload1 destination=local /
/tools

Cheers,
Eric

2015-01-22 13:51 GMT-06:00 Fernandez Edgar 
edgar.fernan...@umontreal.camailto:edgar.fernan...@umontreal.ca:
Hello gents,

I started this new thread because I have a showstopper for my installation of 
galaxy.
Here’s my situation:

1.   I have a galaxy server that has internet access.

2.   My galaxy server is also my torque server and my only torque submit 
node.

3.   I have three torque compute nodes that DOESN’T have internet access.

Now, I’ve tried uploading a file (larger than 2Gb) via and URL and it fails.
However, I’ve added my galaxy server as a compute node and re-tried to upload 
the same file and I made sure the job will run on that machine and it works.

Now, my compute nodes are in a network that is completely protected from the 
outside world.
So no internet access.

What are my option in this case?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
•  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
Eric Rasche
Programmer II

Center for Phage Technology
Rm 312A, BioBio
Texas AM University
College Station, TX 77843
404-692-2048
e...@tamu.edumailto:e...@tamu.edu
rasche.e...@yandex.rumailto:rasche.e...@yandex.ru

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] galaxy and torque - resource allocation

[Fernandez Edgar]

Hello John,

This is totally excellent!
This gave me a great idea for my little torque installation.

I hope you don't mind but I have an unrelated issue with torque that I do not 
seem to grasp.
My issue is that it take a rediculious amount of time to upload a 2Gb file.
I tried many different configurations of apache to upload that 2Gb file without 
success.
Galaxy keeps showing that arrow going up constantly (over 12h).
And this is an upload from a file located on the server running galaxy.

Here is some information from my installation.
1. Server redhat6 with galaxy behind an Apache HTTP Server proxy like described 
on your website.
2. I have Apache HTTP Server running as the galaxy user with
a. Serving Galaxy at a sub directory
b. Compression and caching
c. Sending files using Apache

You will see in attachment my apache's galaxy, deflate, expire and xsendfile 
configuration file.
Furthermore, I've changed the following in galaxy.ini:
apache_xsendfile = True
upstream_gzip = False

Any suggestions?
Any information I missed?
PLEASE HELP!!!

Cordialement / Regards, 
Edgar Fernandez


-Message d'origine-
De : John Chilton [mailto:jmchil...@gmail.com] 
Envoyé : January-21-15 9:34 PM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy and torque - resource allocation

Was hoping someone with an actual torque cluster would respond. I think the way 
you would configure this might be for instance might be setting 
native_specification like -l nodes=1:ppn=n. Depending on how things are 
configured or simply because I am ignorant - this may be wrong - but I think 
what you really want should look a lot like your arguments to qsub. So let say 
you have defined a job runner plugin called pbs_drmaa at the top of your 
job_conf.xml file. Then you could default everything to a single core by 
default and send big jobs to a destination with 8 cores with destinations and 
tool sections that look something like this...

...

  destinations default=singlecore
destination id=singlecore runner=pbs_drmaa
  param id=native_specification-l nodes=1:ppn=1/param
/destination
destination id=multicore runner=pbs_drmaa
  param id=native_specification-l nodes=1:ppn=8/param
/destination
  /destinations

and ...

  tools
tool id=bowtie2 destination=multicore /
tool id=bowtie destination=multicore /
tool id=deeptools destination=multicore /
tool id=cufflinksdestination=multicore /
tool id=cuffdiff destination=multicore /
tool id=cuffmerge destination=multicore /
tool id=tophat2 destination=multicore /
  /tools

Again - that native specification could be wrong - probably best to test it out 
locally (or maybe Nate or JJ will step in and correct me).

Hope this helps,
-John


On Mon, Jan 19, 2015 at 8:05 AM, Fernandez Edgar edgar.fernan...@umontreal.ca 
wrote:
 Hello gents,

 Once again I would like to convey my most sincere appreciation for your 
 help!!!
 And yes I would like to hear your elaboration on my DRMAA runner which is 
 what I'm using.
 So my installation looks like: galaxy -- pbs_drmaa -- torque

 Cordialement / Regards,
 Edgar Fernandez

 -Message d'origine-
 De : John Chilton [mailto:jmchil...@gmail.com] Envoyé : January-18-15 
 8:59 PM À : Fernandez Edgar Cc : galaxy-...@bx.psu.edu Objet : Re: 
 [galaxy-dev] galaxy and torque - resource allocation

 Galaxy generally defers to the DRM (torque in your case) for dealing with 
 these things. In your job_conf.xml you can specify limits for memory or CPUs 
 and Galaxy will pass these along to the DRM at which point it is up to the 
 DRM to enforce these - details depend on if you are using the PBS runner or 
 the DRMAA runner (let me know which and I can try to elobrate if that would 
 be useful).

 In your particular case - I don't believe torque schedules RAM so 
 things will generally only be... throttled... by CPUs counts. This is 
 what I was told by the admins at MSI when I worked there anyway. If 
 you want to place hard limits on RAM I think you need to upgrade to 
 the MOAB scheduler or switch over to a different DRM entirely like 
 SLURM. Even for DRMs that deal more directly with memory - Galaxy 
 doesn't provide a general mechanism for passing this along to the tool
 (https://trello.com/c/3RkTDnIn) - so it would be up to the tool to interact 
 with the environment variables your DRM sets .

 This sounds really terrible in the abstract - but it reality it usually isn't 
 an issue - most of Galaxy's multi-core mappers say have relatively low memory 
 per CPU usage - and for things like assemblers where this is more important - 
 one can usually just assign them to their own CPU or something like that to 
 ensure they get all the memory available.

 Unlike memory - Galaxy will attempt to pass the number of slots allocated to 
 a job to the tools by setting the GALAXY_SLOTS environment variable. All

Re: [galaxy-dev] galaxy and torque - resource allocation

[Fernandez Edgar]

: paste.recursive.RecursiveMiddleware object at 0x7fd1500e9b10
  is_api_request: False
  paste.cookies: (SimpleCookie: 
galaxysession='d122f4f53e5da72f439b77b098b6a77c8fe4c3cee6d2a1f587a8445290a468f305b3911dd27efa9e',
 
'galaxysession=d122f4f53e5da72f16add67ded507460da12cda8c8b0781476c9ddbea84bd40a1e299b3d9b7a9e3a;
 
galaxysession=d122f4f53e5da72f439b77b098b6a77c8fe4c3cee6d2a1f587a8445290a468f305b3911dd27efa9e')
  paste.expected_exceptions: [class 'paste.httpexceptions.HTTPException']
  paste.httpexceptions: paste.httpexceptions.HTTPExceptionHandler object at 
0x7fd1500e9550
  paste.httpserver.thread_pool: paste.httpserver.ThreadPool object at 
0x7ef2850
  paste.recursive.forward: paste.recursive.Forwarder from /galaxy-prod
  paste.recursive.include: paste.recursive.Includer from /galaxy-prod
  paste.recursive.include_app_iter: paste.recursive.IncluderAppIter from 
/galaxy-prod
  paste.recursive.script_name: '/galaxy-prod'
  paste.throw_errors: True
  request_id: 'c93de906a24911e4bd5d002264f83610'
  webob._parsed_query_vars: (MultiDict([]), '')
  wsgi process: 'Multithreaded'

ip_addr - - [22/Jan/2015:10:17:36 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:40 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:44 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:48 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:52 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:59 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:18:04 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:18:10 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:18:29 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:18:36 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36

However, I still see on my galaxy webpage the document in blue with the arrow 
going up constantly.

Do you think it’s an apache configuration error or galaxy or both?

Cordialement / Regards,
Edgar Fernandez

De : Nate Coraor [mailto:n...@bx.psu.edu]
Envoyé : January-22-15 10:10 AM
À : Fernandez Edgar
Cc : John Chilton; galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy and torque - resource allocation

Hi Edgar,

I'd suggest checking your Apache logs to see if the upload was denied by the 
server configuration (possibly for being too large).

Also, please try using the modal upload method to see whether the upload is 
actually completing. This can be found by clicking the icon of a box with an up 
arrow at the top of the tool panel.

There are multiple steps to an upload - the first is sending the data to the 
server. After that, Galaxy creates a job and detects metadata about the 
uploaded dataset. Prior to the modal upload, a (purple uploading) history 
item would be created as soon as the upload started, but if the transfer to the 
server

[galaxy-dev] show stopper for my galaxy installation

[Fernandez Edgar]

Hello gents,

I started this new thread because I have a showstopper for my installation of 
galaxy.
Here's my situation:

1.   I have a galaxy server that has internet access.

2.   My galaxy server is also my torque server and my only torque submit 
node.

3.   I have three torque compute nodes that DOESN'T have internet access.

Now, I've tried uploading a file (larger than 2Gb) via and URL and it fails.
However, I've added my galaxy server as a compute node and re-tried to upload 
the same file and I made sure the job will run on that machine and it works.

Now, my compute nodes are in a network that is completely protected from the 
outside world.
So no internet access.

What are my option in this case?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
*  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] galaxy and torque - resource allocation

[Fernandez Edgar]

Hello Nate,

I just tried the modal upload dialog and it says File exceeds 2GB. Please use 
FTP client.
Is there a way to increase that file size limitation?

Cordialement / Regards,
Edgar Fernandez

De : Nate Coraor [mailto:n...@bx.psu.edu]
Envoyé : January-22-15 12:02 PM
À : Fernandez Edgar
Cc : John Chilton; galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy and torque - resource allocation

Hi Edgar,

Please make sure you have disabled the developer options `use_interactive` and 
`debug` in galaxy.ini, then restart your Galaxy server and try the upload again.

Also, please try out the modal upload dialog as I suggested, it should prevent 
a stuck upload in the future.

--nate



On Thu, Jan 22, 2015 at 10:25 AM, Fernandez Edgar 
edgar.fernan...@umontreal.camailto:edgar.fernan...@umontreal.ca wrote:
Hello Nate,


Thank you for answer so quickly.
My apache logs (/var/log/httpd/galaxy-prod-access_log) looks like this
ip_addr - - [22/Jan/2015:10:17:31 -0500] POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1 200 37
ip_addr - - [22/Jan/2015:10:17:32 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:32 -0500] GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1 200 1401
ip_addr - - [22/Jan/2015:10:17:36 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:40 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:44 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:48 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235

And my galaxy logs (/home/galaxy/galaxy-prod/config/main.log):
ip_addr - - [22/Jan/2015:10:17:31 -0400] POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:32 -0400] POST /galaxy-prod/tool_runner/index 
HTTP/1.1 500 - http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; 
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/40.0.2214.91 Safari/537.36
Error - type 'exceptions.OverflowError': signed integer is greater than 
maximum
ip_addr - - [22/Jan/2015:10:17:32 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:32 -0400] GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
URL: http://esiservera.esi.umontreal.ca:8081/galaxy-prod/tool_runner/index
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/error.py', 
line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
  return self.application(environ, start_response)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
 line 633 in __call__
  return self.application(environ, start_response)
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 132 in 
__call__
  return self.handle_request( environ, start_response )
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 185 in 
handle_request
  kwargs = trans.request.params.mixed()
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 900 in params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 892 in 
str_params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 818 in str_POST
File '/usr/lib64/python2.6/cgi.py', line 508 in __init__
  self.read_multi(environ, keep_blank_values, strict_parsing)
File '/usr/lib64/python2.6/cgi.py', line 635 in read_multi
  headers = rfc822.Message(self.fp)
File '/usr/lib64/python2.6/rfc822.py', line 108 in __init__
  self.readheaders()
File '/usr/lib64/python2.6/rfc822.py', line 155 in readheaders
  line = self.fp.readline()
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py', 
line 482 in readline
  data = self.file.readline(max_read)
File '/usr/lib64/python2.6/socket.py', line 415 in readline
  bline = buf.readline(size)
OverflowError: signed integer is greater than maximum


CGI Variables
-
  CONTENT_LENGTH: '2625522525tel:2625522525'
  CONTENT_TYPE: 'multipart/form-data; 
boundary=WebKitFormBoundary9c9hOVrAGAXKJtdd'
  HTTP_ACCEPT: '*/*'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.8'
  HTTP_CONNECTION: 'Keep-Alive

Re: [galaxy-dev] galaxy and torque - resource allocation

[Fernandez Edgar]

Hello guys,

I want to apologies.
I didn’t see the note:
Due to browser limitations, uploading files larger than 2GB is guaranteed to 
fail. To upload large files, use the URL method (below) or FTP (if enabled by 
the site administrator).
and I forgot my file was that big.

Please accept my sincere apologies.

Cordialement / Regards,
Edgar Fernandez

De : Fernandez Edgar
Envoyé : January-22-15 12:35 PM
À : 'Nate Coraor'
Cc : John Chilton; galaxy-...@bx.psu.edu
Objet : RE: [galaxy-dev] galaxy and torque - resource allocation

Hello Nate,

I just tried the modal upload dialog and it says File exceeds 2GB. Please use 
FTP client.
Is there a way to increase that file size limitation?

Cordialement / Regards,
Edgar Fernandez

De : Nate Coraor [mailto:n...@bx.psu.edu]
Envoyé : January-22-15 12:02 PM
À : Fernandez Edgar
Cc : John Chilton; galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy and torque - resource allocation

Hi Edgar,

Please make sure you have disabled the developer options `use_interactive` and 
`debug` in galaxy.ini, then restart your Galaxy server and try the upload again.

Also, please try out the modal upload dialog as I suggested, it should prevent 
a stuck upload in the future.

--nate



On Thu, Jan 22, 2015 at 10:25 AM, Fernandez Edgar 
edgar.fernan...@umontreal.camailto:edgar.fernan...@umontreal.ca wrote:
Hello Nate,


Thank you for answer so quickly.
My apache logs (/var/log/httpd/galaxy-prod-access_log) looks like this
ip_addr - - [22/Jan/2015:10:17:31 -0500] POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1 200 37
ip_addr - - [22/Jan/2015:10:17:32 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:32 -0500] GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1 200 1401
ip_addr - - [22/Jan/2015:10:17:36 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:40 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:44 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235
ip_addr - - [22/Jan/2015:10:17:48 -0500] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 235

And my galaxy logs (/home/galaxy/galaxy-prod/config/main.log):
ip_addr - - [22/Jan/2015:10:17:31 -0400] POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:32 -0400] POST /galaxy-prod/tool_runner/index 
HTTP/1.1 500 - http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; 
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/40.0.2214.91 Safari/537.36
Error - type 'exceptions.OverflowError': signed integer is greater than 
maximum
ip_addr - - [22/Jan/2015:10:17:32 -0400] GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
ip_addr - - [22/Jan/2015:10:17:32 -0400] GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1 200 - 
http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root; Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36
URL: http://esiservera.esi.umontreal.ca:8081/galaxy-prod/tool_runner/index
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/error.py', 
line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
  return self.application(environ, start_response)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
 line 633 in __call__
  return self.application(environ, start_response)
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 132 in 
__call__
  return self.handle_request( environ, start_response )
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 185 in 
handle_request
  kwargs = trans.request.params.mixed()
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 900 in params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 892 in 
str_params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 818 in str_POST
File '/usr/lib64/python2.6/cgi.py', line 508 in __init__
  self.read_multi(environ, keep_blank_values, strict_parsing)
File '/usr/lib64/python2.6/cgi.py', line 635 in read_multi
  headers = rfc822.Message(self.fp)
File '/usr/lib64/python2.6/rfc822.py', line 108 in __init__
  self.readheaders()
File '/usr/lib64/python2.6/rfc822.py', line 155 in readheaders
  line = self.fp.readline()
File 
'/home/galaxy/galaxy

Re: [galaxy-dev] galaxy with torque

[Fernandez Edgar]

Thank you for that correction.

Just a small FYI (maybe it will be useful to update the wiki)…

I had to export three variables to make the scramble possible:

export PBS_PYTHON_INCLUDEDIR=/usr/local/torque/include/
export PBSCONFIG=/usr/local/torque/bin/pbs-config
export LIBTORQUE_DIR=/usr/local/torque/lib/libtorque.so
python scripts/scramble.py -e pbs_python

Cordialement / Regards,
Edgar Fernandez

De : Rémy Dernat [mailto:remy...@gmail.com]
Envoyé : December-02-14 11:49 AM
À : Fernandez Edgar
Cc : John Chilton; galaxy-...@bx.psu.edu
Objet : Re: Re : [galaxy-dev] galaxy with torque

Sorry for answer 7. There is no benefit to do that. Once the egg is done, there 
is nothing to do from here, except if you change your python version... If that 
variable is empty, that is normal, because it is not an environment variable, 
it is just used by the following python command:
LIBTORQUE_DIR=/path/to/libtorque python scripts/scramble.py -e pbs_python

2014-12-02 16:27 GMT+01:00 Rémy Dernat 
remy...@gmail.commailto:remy...@gmail.com:
Hi Edgar,

You are right. It is very annoying...

So, to answer your questions:
1/ First answer of google / wikipedia with DRMAA : 
http://en.wikipedia.org/wiki/DRMAA
It is a pattern to talk with any DRM system (SGE, torque, whatever...)

2/ This is a library (python version) for your Torque installation.

3/ MySQL access is only needed by your galaxy frontend.

4/ Internet access is not required for your compute node (except the galaxy 
one), but it is better if you want to use a package manager on your compute 
nodes, for example...

5/ On my part, I use permissions like 760 on galaxy directory. It depends on 
your needs... Some applications might need an access to your galaxy 
installation, but you should split binaries, the galaxy installation and your 
data (datasets, libraries...). But do not forget to share this folders by NFS 
(if needed).

6/ Sorry, no idea; but I see no reason for that to become unavailable, if your 
proxy is well configured.

7/ You have to put this command line into a file which will be sourced like 
$HOME_GALAXY/.bashrc or your environment file (environment_setup_file in 
universe_wsgi.ini or config/galaxy.ini)

Regards,
Remy

2014-12-02 13:59 GMT+01:00 Fernandez Edgar 
edgar.fernan...@umontreal.camailto:edgar.fernan...@umontreal.ca:
Hi everyone,

I’m guessing Remy, you click on the send button by mistake on your HTC device.
It happens to me ALL the time…

I wanted to take this opportunity to add some questions to the three questions 
in my previous email.
So here goes:


4.   Is it necessary that my torque compute nodes have internet access?

Because right now, only my torque server and my galaxy server has internet 
access.

However, communication between the submit node (a.k.a. galaxy server) and the 
torque server is enabled.

Likewise, between the torque server and the compute nodes.

5.   Furthermore, would I disable any galaxy functionalities if I change 
the permissions on the whole galaxy installation directory like so: chmod –R 
700 galaxy_install_directory

I have created a user galaxy for running everything that is galaxy.

6.   I’ve actually installed my galaxy server using a proxy server (as 
described on your web site). Can I still use the Report Tool functionalities on 
galaxy?

7.   Any instructions on making PBS (running jobs via TORQUE resource 
manager), because I have successfully scrambled the egg pbs_python with the 
command:

LIBTORQUE_DIR=/path/to/libtorque python scripts/scramble.py -e pbs_python.

However, LIBTORQUE_DIR is empty.

Once again, thank you in advance for all your help!

Cordialement / Regards,
Edgar Fernandez

De : remy...@gmail.commailto:remy...@gmail.com 
[mailto:remy...@gmail.commailto:remy...@gmail.com]
Envoyé : December-02-14 6:34 AM
À : Fernandez Edgar; John Chilton

Cc : galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu
Objet : Re : [galaxy-dev] galaxy with torque

1 a drmaa d

Envoyé depuis mon HTC

- Reply message -
De : Fernandez Edgar 
edgar.fernan...@umontreal.camailto:edgar.fernan...@umontreal.ca
Pour : John Chilton jmchil...@gmail.commailto:jmchil...@gmail.com
Cc : galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu 
galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu
Objet : [galaxy-dev] galaxy with torque
Date : mer., nov. 26, 2014 19:09


Hi John,

First, thank you very much for your prompt answer.
It's extremely appreciated.

Secondly, I have some other questions: whatever answers you can provide me 
with, will be greatly helpful.
Please, forgive my beginner level understanding of an DRM system.

1. Once I compile, make and install the Torque Submit Node code against the 
server running Galaxy, what exactly is the purpose for an DRMAA ?

2. What is exactly the PBS step describe here: 
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
galaxy_user@galaxy_server% LIBTORQUE_DIR=/path/to/libtorque python 
scripts/scramble.py -e pbs_python
What does it do exactly ?

3

Re: [galaxy-dev] galaxy with torque

[Fernandez Edgar]

Hi John,

First, thank you very much for your prompt answer.
It's extremely appreciated.

Secondly, I have some other questions: whatever answers you can provide me 
with, will be greatly helpful.
Please, forgive my beginner level understanding of an DRM system.

1. Once I compile, make and install the Torque Submit Node code against the 
server running Galaxy, what exactly is the purpose for an DRMAA ?

2. What is exactly the PBS step describe here: 
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster 
galaxy_user@galaxy_server% LIBTORQUE_DIR=/path/to/libtorque python 
scripts/scramble.py -e pbs_python
What does it do exactly ?

3. All servers which includes my torque server, torque compute nodes and torque 
submit node (a.k.a galaxy server) have the galaxy user defines and its home 
shared on all of them. This means all of them have access (via NFS) of the 
installation directory of galaxy. But what about the MySQL server access. Does 
the torque server or the compute nodes need access to that service? 

I hope I'm not sending too many emails/questions...
Thank you very much!

Cordialement / Regards, 
Edgar Fernandez

-Message d'origine-
De : John Chilton [mailto:jmchil...@gmail.com] 
Envoyé : November-25-14 12:07 PM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy with torque

I am not sure we have a walkthrough for Torque specifically - but if you have 
Galaxy up and running and you can qsub commands to torque - hopefully you have 
done all of the hard parts.

You will need a DRMAA library for your torque setup - 
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
suggests compiling pbs_drmaa and outlines how to set it up. After that you just 
need to add a plugin and default destination to your job_conf.xml file - also 
outlined on that wiki page.

Other good resources to consult if you are scaling up your Galaxy this way are:
https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer
https://wiki.galaxyproject.org/Events/GCC2014/TrainingDay/AdminWalkthrough

Good luck and let us know if you encounter any problems.

-John

On Fri, Nov 21, 2014 at 2:30 PM, Fernandez Edgar edgar.fernan...@umontreal.ca 
wrote:
 Hello all,



 My name is Edgar Fernandez. I’m a sys. admin. at University of Montreal.

 I’ve contacted you a while back about installing galaxy and I’ve 
 successfully done it on a redhat 6 server.



 I see myself in a situation where I need to utilise all my redhat 
 servers (who are identical to the server running the galaxy website).



 I’ve also successfully installed a server torque with compute notes 
 and clients nodes.



 What are the last step to make the link between galaxy and torque?

 Also, once that connection is made, how will galaxy keep track of the 
 jobs sent?

 I mean who will it know this job that just finished is for this user 
 and not another ?



 Also, my torque installation is so that my server running galaxy is a 
 submit node and a client node.

 I hope this is not a problem.



 Please help!



 Cordialement / Regards,



 Edgar Fernandez

 System Administrator (Linux)

 Direction Générale des Technologies de l'Information et de la 
 Communication

 (  Bur. : 1-514-343-6111 poste 16568



 Université de Montréal

 PAVILLON ROGER-GAUDRY, bureau X-218




 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this and other 
 Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/