[galaxy-dev] Setting multiple parameters for a workflow (API)

2014-06-11 Thread Carlos del Ojo
Hi all,

I am working in a project that requires running workflows with multiple
parameter by using the Galaxy API.

However I realised that /api/workflows webservice does not allow to set
several parameters for one sigle step.

You can check that ingalaxy-dist/lib/galaxy/workflow/run.py
  _update_step_parameters __doc__ string:

  PARAM_DICT = {'param': NAME, 'value': VALUE}

Note that this format allows only one parameter to be set per step.

I've done a change in the script so it allows setting multiple parameters
per step.

So the format of the PARAM_DICT would be:
PARAM_DICT={'param': ['threshold','path'], 'value':
[2.3,'/tmp/kYzoax02.tmp']

a list of parameters and a list of values, that will be combined and
assigned to the step.

I'm attaching a patch to the run.py file.
Is there any thought to fix that?
Could that be integrated in the main repository?
We have been patching the same part of the code for the last 2 months and I
think it would be interesting to add this feature permanently.

Thanks,
C
--- old/run.py	2014-06-11 11:30:43.0 +
+++ run.py	2014-06-11 11:34:15.0 +
@@ -307,16 +307,16 @@
 Finally (again, for backwards compatibility), PARAM_DICT can also
 be specified as::
 
-  PARAM_DICT = {'param': NAME, 'value': VALUE}
-
-Note that this format allows only one parameter to be set per step.
+  PARAM_DICT = {'param': [list of param names], 'value': [list of param values]}
+  eg: PARAM_DICT={'param': ['threshold','path'], 'value': [2.3,'/tmp/kYzoax02.tmp']
 
 param_dict = param_map.get(step.tool_id, {}).copy()
 param_dict.update(param_map.get(str(step.id), {}))
+
 if param_dict:
 if 'param' in param_dict and 'value' in param_dict:
-param_dict[param_dict['param']] = param_dict['value']
+for k,v in zip(param_dict['param'],param_dict['value']):
+param_dict[k] =v
 step.state.inputs.update(param_dict)
 
-
 __all__ = [ invoke, WorkflowRunConfig ]
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Setting multiple parameters for a workflow (API)

2014-06-11 Thread John Chilton
Thanks for the contribution Carlos. The limitation that workflows only
can set one parameter be step is old at this point - the workflow API
definitely allows setting any number of parameters on any step - with
steps specified by individual step id or by tool id. This seems to
repeatedly trip people up. We need to be better about updating stale
documentation, sorry about that.

More information on setting parameters can be found on this biostar
post - https://biostar.usegalaxy.org/p/7598/

And here is the documentation in the code:


Update ``step`` parameters based on the user-provided ``param_map`` dict.

``param_map`` should be structured as follows::

  PARAM_MAP = {STEP_ID: PARAM_DICT, ...}
  PARAM_DICT = {NAME: VALUE, ...}

For backwards compatibility, the following (deprecated) format is
also supported for ``param_map``::

  PARAM_MAP = {TOOL_ID: PARAM_DICT, ...}

in which case PARAM_DICT affects all steps with the given tool id.
If both by-tool-id and by-step-id specifications are used, the
latter takes precedence.

Finally (again, for backwards compatibility), PARAM_DICT can also
be specified as::

  PARAM_DICT = {'param': NAME, 'value': VALUE}

Note that this format allows only one parameter to be set per step.


The variant of parameter setting you modified is only there for
backward compatibility (I would consider it deprecated at this point)
- and since your change breaks that backward compatibility I don't
think we should really merge it into Galaxy - sorry.

Thanks again,
-John

On Wed, Jun 11, 2014 at 6:54 AM, Carlos del Ojo deep...@gmail.com wrote:
 Hi all,

 I am working in a project that requires running workflows with multiple
 parameter by using the Galaxy API.

 However I realised that /api/workflows webservice does not allow to set
 several parameters for one sigle step.

 You can check that ingalaxy-dist/lib/galaxy/workflow/run.py
   _update_step_parameters __doc__ string:

   PARAM_DICT = {'param': NAME, 'value': VALUE}

 Note that this format allows only one parameter to be set per step.

 I've done a change in the script so it allows setting multiple parameters
 per step.

 So the format of the PARAM_DICT would be:
 PARAM_DICT={'param': ['threshold','path'], 'value':
 [2.3,'/tmp/kYzoax02.tmp']

 a list of parameters and a list of values, that will be combined and
 assigned to the step.

 I'm attaching a patch to the run.py file.
 Is there any thought to fix that?
 Could that be integrated in the main repository?
 We have been patching the same part of the code for the last 2 months and I
 think it would be interesting to add this feature permanently.

 Thanks,
 C

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Toolshed upload error message

2014-06-11 Thread Greg Von Kuster
Hi Vipin,

Can you elaborate on the error you are seeing?

Thanks,

Greg Von Kuster

On Jun 10, 2014, at 8:07 PM, Vipin TS vipin...@gmail.com wrote:

 Hi Greg, 
 
 When I trying to upload a next release version o my 
 https://toolshed.g2.bx.psu.edu/repos/vipints/fml_gff3togtf converter program 
 to Community Toolshed, I am getting an internal error message. 
 
 I am uploading a tar.gz file to the page https://toolshed.g2.bx.psu.edu/ but 
 this fails. 
 
 Could you please pass the error message or let me how I can add new files to 
 the repository and delete the old version. 
 
 Thanks, 
 
 Vipin | Rätsch Lab
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] upload problems

2014-06-11 Thread John Chilton
So the permissions on these files seem fine right. That is odd.

Next things I would check are that each directory leading up to
/panfs/storage.local/scratch/galaxy-data/ is world executable and that
the whole thing (/panfs/storage.local/scratch/galaxy-data/) is mounted
and with the same permissions on the cluster worker nodes.

If users can indeed read that file - the error message must be
misleading some how (that would not be entirely surprising).

-John

On Tue, Jun 10, 2014 at 4:48 PM, Shrum, Donald C dcsh...@admin.fsu.edu wrote:
 Hi Jon and thanks for the reply.I am attempting to run jobs as the real 
 user as jobs will go to our HPC cluster.  This will be an enterprise server.

 /panfs/storage.local/scratch/galaxy-data/ is world writable-
 drwxrwxrwx  4 galaxy  galaxy   4096 May  7 09:08 galaxy-data

 as is tmp
 -bash-4.1$ ls -l /panfs/storage.local/scratch/galaxy-data/
 total 160
 drwxrwxrwx 2 galaxy  galaxy  20480 Jun 10 17:00 tmp

 I got a little lost on the integrated datatypes configuration file... is that 
 an XML file?  I'm not sure which file I'm looking for and I'm new to galaxy.

 --Donny

 -Original Message-
 From: John Chilton [mailto:jmchil...@gmail.com]
 Sent: Tuesday, June 10, 2014 5:33 PM
 To: Shrum, Donald C
 Cc: galaxy-dev@lists.bx.psu.edu
 Subject: Re: [galaxy-dev] upload problems

 You didn't include this context, but I am guessing you are attempting to run 
 jobs as the real user? If not, ignore the rest of the e-mail.

 I would generally not recommend running the uploads as real user - it is a 
 complex process but should go relatively quick.

 Understand that may not be possible though. So that file is the integrated 
 datatypes configuration file I believe. There is just one global copy that 
 gets created with Galaxy boots up - so it cannot be chown-ed on a per job 
 basis. The thing is that Galaxy should be modifying it to be world readable
 (https://bitbucket.org/galaxy/galaxy-central/annotate/e2b761a9b1d6d41db71b28df8b62862c7c300eba/lib/galaxy/datatypes/registry.py?at=default#cl-811)
 - something is going wrong if it is not. Can you verify the file is 644?

 That leads me to believe that users don't have read access to the global temp 
 directory. Can you check if users can read 
 /panfs/storage.local/scratch/galaxy-data/tmp/? I think they will need to to 
 use some tools including uploads?

 If you cannot make this directory accessible to users - can you change 
 Galaxy's new_file_path so that it is some directory globally readable?

 -John

 On Tue, Jun 10, 2014 at 4:10 PM, Shrum, Donald C dcsh...@admin.fsu.edu 
 wrote:
 Hi all,

 I'm working with a problem with user uploaded files.  After digging
 through the logs a bit and running the commands on at a time manually
 I think I've narrowed it to a permissions problem.  This was confirmed
 by just running galaxy as root and the problem went away ;)

 -bash-4.1$ PYTHONPATH=/panfs/storage.local/software/galaxy-dist/lib/
 -bash-4.1$ python 
 /panfs/storage.local/software/galaxy-dist/tools/data_source/upload.py 
 /panfs/storage.local/software/galaxy-dist//panfs/storage.local/scratch/galaxy-data/tmp/tmpSuHquR
  /panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf 
 6:/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/6/dataset_6_files:/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/6/galaxy_dataset_6.dat
 Traceback (most recent call last):
   File 
 /panfs/storage.local/software/galaxy-dist/tools/data_source/upload.py, 
 line 394, in module
 __main__()
   File 
 /panfs/storage.local/software/galaxy-dist/tools/data_source/upload.py, 
 line 369, in __main__
 registry.load_datatypes( root_dir=sys.argv[1], config=sys.argv[2] )
   File 
 /panfs/storage.local/software/galaxy-dist/lib/galaxy/datatypes/registry.py,
  line 97, in load_datatypes
 tree = galaxy.util.parse_xml( config )
   File 
 /panfs/storage.local/software/galaxy-dist/lib/galaxy/util/__init__.py, 
 line 154, in parse_xml
 tree = ElementTree.parse(fname)
   File build/bdist.linux-x86_64-ucs4/egg/elementtree/ElementTree.py, line 
 859, in parse
   File build/bdist.linux-x86_64-ucs4/egg/elementtree/ElementTree.py,
 line 576, in parse
 IOError: [Errno 13] Permission denied: 
 '/panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf'


 -bash-4.1$ ls -l
 /panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf
 -rw--- 1 dcshrum dcshrum 317 Jun 10 16:30
 /panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf


 It does not look like galaxy is using sudo to run the script.  Suggestions 
 to work this out?



 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this and other
 Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/



Re: [galaxy-dev] upload problems

2014-06-11 Thread Shrum, Donald C
Hi John,

Here is what I've tracked down this morning.  On the off chance this seems 
wildly off the mark I don't know much about python or galaxy.  I am drinking 
fancy coffee though ;)

I did confirm all the folders are world executable.  The upload is running as a 
local job, not a cluster job.  

I put some debugging code into galaxy-dist/lib/galaxy/datatypes/registry.py and 
the chmod in that code block is not called for the uploaded file.

I made an edit to scripts/external_chown_script.py
I added the line:
os.system('chmod 0644 %s' %(path))

that made the uploaded file world readable and pushed me to the next error that 
reads:
galaxy.jobs ERROR 2014-06-11 09:33:31,562 fail(): Missing output file in 
working directory: 
[Errno 13] Permission denied: 
'/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/8/galaxy_dataset_8.dat'

It looks like the problem is with the permissions on the newly created folder 
(8 in this case)
-bash-4.1$ ls -l database/job_working_directory/000/
total 64
drw-r--r-- 2 galaxy galaxy 4096 Jun 11 09:33 8

The folder is does not have execute permissions turned on.  Could someone point 
me to the block of code where the mkdir is run?  Or of course if there is some 
setting I've missed some place please let me know!

Thanks for the help.
Donny
FSU RCC



-Original Message-
From: John Chilton [mailto:jmchil...@gmail.com] 
Sent: Wednesday, June 11, 2014 9:30 AM
To: Shrum, Donald C
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] upload problems

So the permissions on these files seem fine right. That is odd.

Next things I would check are that each directory leading up to 
/panfs/storage.local/scratch/galaxy-data/ is world executable and that the 
whole thing (/panfs/storage.local/scratch/galaxy-data/) is mounted and with the 
same permissions on the cluster worker nodes.

If users can indeed read that file - the error message must be misleading some 
how (that would not be entirely surprising).

-John

On Tue, Jun 10, 2014 at 4:48 PM, Shrum, Donald C dcsh...@admin.fsu.edu wrote:
 Hi Jon and thanks for the reply.I am attempting to run jobs as the real 
 user as jobs will go to our HPC cluster.  This will be an enterprise server.

 /panfs/storage.local/scratch/galaxy-data/ is world writable-
 drwxrwxrwx  4 galaxy  galaxy   4096 May  7 09:08 galaxy-data

 as is tmp
 -bash-4.1$ ls -l /panfs/storage.local/scratch/galaxy-data/
 total 160
 drwxrwxrwx 2 galaxy  galaxy  20480 Jun 10 17:00 tmp

 I got a little lost on the integrated datatypes configuration file... is that 
 an XML file?  I'm not sure which file I'm looking for and I'm new to galaxy.

 --Donny

 -Original Message-
 From: John Chilton [mailto:jmchil...@gmail.com]
 Sent: Tuesday, June 10, 2014 5:33 PM
 To: Shrum, Donald C
 Cc: galaxy-dev@lists.bx.psu.edu
 Subject: Re: [galaxy-dev] upload problems

 You didn't include this context, but I am guessing you are attempting to run 
 jobs as the real user? If not, ignore the rest of the e-mail.

 I would generally not recommend running the uploads as real user - it is a 
 complex process but should go relatively quick.

 Understand that may not be possible though. So that file is the 
 integrated datatypes configuration file I believe. There is just one 
 global copy that gets created with Galaxy boots up - so it cannot be 
 chown-ed on a per job basis. The thing is that Galaxy should be 
 modifying it to be world readable
 (https://bitbucket.org/galaxy/galaxy-central/annotate/e2b761a9b1d6d41d
 b71b28df8b62862c7c300eba/lib/galaxy/datatypes/registry.py?at=default#c
 l-811)
 - something is going wrong if it is not. Can you verify the file is 644?

 That leads me to believe that users don't have read access to the global temp 
 directory. Can you check if users can read 
 /panfs/storage.local/scratch/galaxy-data/tmp/? I think they will need to to 
 use some tools including uploads?

 If you cannot make this directory accessible to users - can you change 
 Galaxy's new_file_path so that it is some directory globally readable?

 -John

 On Tue, Jun 10, 2014 at 4:10 PM, Shrum, Donald C dcsh...@admin.fsu.edu 
 wrote:
 Hi all,

 I'm working with a problem with user uploaded files.  After digging 
 through the logs a bit and running the commands on at a time manually 
 I think I've narrowed it to a permissions problem.  This was 
 confirmed by just running galaxy as root and the problem went away ;)

 -bash-4.1$ PYTHONPATH=/panfs/storage.local/software/galaxy-dist/lib/
 -bash-4.1$ python 
 /panfs/storage.local/software/galaxy-dist/tools/data_source/upload.py 
 /panfs/storage.local/software/galaxy-dist//panfs/storage.local/scratch/galaxy-data/tmp/tmpSuHquR
  /panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf 
 6:/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/6/dataset_6_files:/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/6/galaxy_dataset_6.dat
 Traceback (most 

Re: [galaxy-dev] Galaxy updated botched?

2014-06-11 Thread Kandalaft, Iyad
This is a follow up for those that are interested with regards to my failed 
schema upgrade.

I believe I have determined why all the tables are set to use the MyISAM 
engine.  When I initialized galaxy on our enterprise servers, they were running 
a dated version of CentOS. Hence, the OS defaults to a dated MySQL version.  
MySQL only switched to using InnoDB (over MyISAM) as the default engine in 
version 5.5.  If I'm not mistaken, I initialized galaxy in MySQL 5.1 without 
changing the default engine to InnoDB (big mistake).  Due to my ignorance, I 
will now have to try to compare v118 of our database with a new install of 
galaxy running schema v118.  Then, I will try migrating to the true schema 
state without destroying the data.

I do feel that I should still modify the galaxy schema to set the MySQL engine 
to InnoDB to thwart problems like this for other unsuspecting users.  If anyone 
can point me at some documentation about how Galaxy schema changes should occur 
in this case, that would be great.
As per my previous comment, I'm not certain whether I would edit all schema 
versions to ensure table definitions include the mysql_engine=InnoDB attribute 
or whether using a DDL event in SQLAlchemy.  I suspect that setting this option 
globally would mean that future developers don't need to remember to define 
mysql_engine on every new table.


Iyad Kandalaft
Microbial Biodiversity Bioinformatics
Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
960 Carling Ave.| 960 Ave. Carling
Ottawa, ON| Ottawa (ON) K1A 0C6
E-mail Address / Adresse courriel  iyad.kandal...@agr.gc.ca
Telephone | Téléphone 613-759-1228
Facsimile | Télécopieur 613-759-1701
Teletypewriter | Téléimprimeur 613-773-2600
Government of Canada | Gouvernement du Canada



From: Kandalaft, Iyad
Sent: Tuesday, June 10, 2014 1:39 PM
To: 'galaxy-...@bx.psu.edu'
Subject: Re: Galaxy updated botched?

Hi Everyone,

This is follow-up information/questions to the issue I ran into with the galaxy 
June 2nd, 2014 update.  I hope to receive feedback on how to proceed.

Background:

-  Running Galaxy (DB Schema 118) with a MySQL 5.5 back-end

-  When updating galaxy to the june 2nd release, the v120 DB schema has 
referential integrity constraints, which produced errors during the upgrade.

-  Completed two galaxy updates in the past 4 months without 
encountering this before (schema changes included)

Discussion:
In the past, referential integrity in the DB schema was never an issue.  I 
checked backups and the current database to find that the database tables are 
using the MyISAM engine.  MyISAM =  no referential integrity support, no 
transactions.
I reviewed galaxy's SQLAlchemy templates and determined that 
mysql_engine='InnoDB' isn't set on tables.  This explains why all tables were 
created with the MyISAM engine.  If the mysql_engine is not innodb, SQL Alchemy 
is supposed to drop any referential integrity constraints defined in the 
schema.  What I don't understand is why SQL Alchemy is no longer ignoring the 
referential integrity constraints.

Going forward, can anyone propose how I can salvage the database or continue 
ignoring referential integrity for now?
Assuming that my limited understanding of SQLAlchemy holds water, I was looking 
at fixing the galaxy code base but I need some clarification on the DB schema 
versioning.  Do I edit schema v1 and add the appropriate table args to make 
every table an innodb engine table or do I add a new schema and modify all 
tables to use the innodb engine?  Alternatively, I can use DDL events
def after_create(target, connection, **kw):
connection.execute(ALTER TABLE %s ENGINE=InnoDB;
   (target.name, target.name))

Thank you for your help.

Regards,
Iyad Kandalaft

Bioinformatics Application Developer
Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
KW Neatby Bldg | éd. KW Neatby
960 Carling Ave| 960, avenue Carling
Ottawa, ON | Ottawa (ON) K1A 0C6
E-mail Address / Adresse courriel: 
iyad.kandal...@agr.gc.camailto:iyad.kandal...@agr.gc.ca
Telephone | Téléphone 613- 759-1228
Facsimile | Télécopieur 613-759-1701
Government of Canada | Gouvernement du Canada

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] upload problems

2014-06-11 Thread Shrum, Donald C
Derp... that is a result of the change I just made.  I'll see if I can add a 
line that checks to see if the uploaded file is a folder and sets it to 0755 if 
that is the case.  Seems like a security risk to just do a blanket 0755 and let 
people upload executable files.

If (path is a file)
os.system('chmod 0644 %s' %(path))
else
os.system('chmod 0755 %s' %(path))

-Original Message-
From: Shrum, Donald C 
Sent: Wednesday, June 11, 2014 9:48 AM
To: 'John Chilton'
Cc: galaxy-dev@lists.bx.psu.edu
Subject: RE: [galaxy-dev] upload problems

Hi John,

Here is what I've tracked down this morning.  On the off chance this seems 
wildly off the mark I don't know much about python or galaxy.  I am drinking 
fancy coffee though ;)

I did confirm all the folders are world executable.  The upload is running as a 
local job, not a cluster job.  

I put some debugging code into galaxy-dist/lib/galaxy/datatypes/registry.py and 
the chmod in that code block is not called for the uploaded file.

I made an edit to scripts/external_chown_script.py I added the line:
os.system('chmod 0644 %s' %(path))

that made the uploaded file world readable and pushed me to the next error that 
reads:
galaxy.jobs ERROR 2014-06-11 09:33:31,562 fail(): Missing output file in 
working directory: 
[Errno 13] Permission denied: 
'/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/8/galaxy_dataset_8.dat'

It looks like the problem is with the permissions on the newly created folder 
(8 in this case) -bash-4.1$ ls -l database/job_working_directory/000/
total 64
drw-r--r-- 2 galaxy galaxy 4096 Jun 11 09:33 8

The folder is does not have execute permissions turned on.  Could someone point 
me to the block of code where the mkdir is run?  Or of course if there is some 
setting I've missed some place please let me know!

Thanks for the help.
Donny
FSU RCC



-Original Message-
From: John Chilton [mailto:jmchil...@gmail.com]
Sent: Wednesday, June 11, 2014 9:30 AM
To: Shrum, Donald C
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] upload problems

So the permissions on these files seem fine right. That is odd.

Next things I would check are that each directory leading up to 
/panfs/storage.local/scratch/galaxy-data/ is world executable and that the 
whole thing (/panfs/storage.local/scratch/galaxy-data/) is mounted and with the 
same permissions on the cluster worker nodes.

If users can indeed read that file - the error message must be misleading some 
how (that would not be entirely surprising).

-John

On Tue, Jun 10, 2014 at 4:48 PM, Shrum, Donald C dcsh...@admin.fsu.edu wrote:
 Hi Jon and thanks for the reply.I am attempting to run jobs as the real 
 user as jobs will go to our HPC cluster.  This will be an enterprise server.

 /panfs/storage.local/scratch/galaxy-data/ is world writable-
 drwxrwxrwx  4 galaxy  galaxy   4096 May  7 09:08 galaxy-data

 as is tmp
 -bash-4.1$ ls -l /panfs/storage.local/scratch/galaxy-data/
 total 160
 drwxrwxrwx 2 galaxy  galaxy  20480 Jun 10 17:00 tmp

 I got a little lost on the integrated datatypes configuration file... is that 
 an XML file?  I'm not sure which file I'm looking for and I'm new to galaxy.

 --Donny

 -Original Message-
 From: John Chilton [mailto:jmchil...@gmail.com]
 Sent: Tuesday, June 10, 2014 5:33 PM
 To: Shrum, Donald C
 Cc: galaxy-dev@lists.bx.psu.edu
 Subject: Re: [galaxy-dev] upload problems

 You didn't include this context, but I am guessing you are attempting to run 
 jobs as the real user? If not, ignore the rest of the e-mail.

 I would generally not recommend running the uploads as real user - it is a 
 complex process but should go relatively quick.

 Understand that may not be possible though. So that file is the 
 integrated datatypes configuration file I believe. There is just one 
 global copy that gets created with Galaxy boots up - so it cannot be 
 chown-ed on a per job basis. The thing is that Galaxy should be 
 modifying it to be world readable 
 (https://bitbucket.org/galaxy/galaxy-central/annotate/e2b761a9b1d6d41d
 b71b28df8b62862c7c300eba/lib/galaxy/datatypes/registry.py?at=default#c
 l-811)
 - something is going wrong if it is not. Can you verify the file is 644?

 That leads me to believe that users don't have read access to the global temp 
 directory. Can you check if users can read 
 /panfs/storage.local/scratch/galaxy-data/tmp/? I think they will need to to 
 use some tools including uploads?

 If you cannot make this directory accessible to users - can you change 
 Galaxy's new_file_path so that it is some directory globally readable?

 -John

 On Tue, Jun 10, 2014 at 4:10 PM, Shrum, Donald C dcsh...@admin.fsu.edu 
 wrote:
 Hi all,

 I'm working with a problem with user uploaded files.  After digging 
 through the logs a bit and running the commands on at a time manually 
 I think I've narrowed it to a permissions problem.  This was 
 confirmed by just running galaxy as root 

Re: [galaxy-dev] Galaxy updated botched?

2014-06-11 Thread John Chilton
Spent a couple hours yesterday trying to track down this - I was not
getting anywhere though and I see why now.

Well this definitely an unfortunate situation - but I think
documentation improvements are the right fix not enforcing the engine
type in sqlalchemy migrations. For one, I don't think we can assume a
particular engine type because different deployers may already going
to have both kinds out there right?

I think it would be better to update the wiki to encourage InnoDB for
all new installations and warn this can happen for MySQL upgrades.

Is this okay?

Sorry about this.

-John



On Wed, Jun 11, 2014 at 8:48 AM, Kandalaft, Iyad
iyad.kandal...@agr.gc.ca wrote:
 This is a follow up for those that are interested with regards to my failed
 schema upgrade.



 I believe I have determined why all the tables are set to use the MyISAM
 engine.  When I initialized galaxy on our enterprise servers, they were
 running a dated version of CentOS. Hence, the OS defaults to a dated MySQL
 version.  MySQL only switched to using InnoDB (over MyISAM) as the default
 engine in version 5.5.  If I’m not mistaken, I initialized galaxy in MySQL
 5.1 without changing the default engine to InnoDB (big mistake).  Due to my
 ignorance, I will now have to try to compare v118 of our database with a new
 install of galaxy running schema v118.  Then, I will try “migrating” to the
 true schema state without destroying the data.



 I do feel that I should still modify the galaxy schema to set the MySQL
 engine to InnoDB to thwart problems like this for other unsuspecting users.
 If anyone can point me at some documentation about how Galaxy schema changes
 should occur in this case, that would be great.

 As per my previous comment, I’m not certain whether I would edit all schema
 versions to ensure table definitions include the mysql_engine=InnoDB
 attribute or whether using a DDL event in SQLAlchemy.  I suspect that
 setting this option globally would mean that future developers don’t need to
 remember to define mysql_engine on every new table.





 Iyad Kandalaft

 Microbial Biodiversity Bioinformatics

 Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
 960 Carling Ave.| 960 Ave. Carling

 Ottawa, ON| Ottawa (ON) K1A 0C6

 E-mail Address / Adresse courriel  iyad.kandal...@agr.gc.ca
 Telephone | Téléphone 613-759-1228
 Facsimile | Télécopieur 613-759-1701
 Teletypewriter | Téléimprimeur 613-773-2600

 Government of Canada | Gouvernement du Canada







 From: Kandalaft, Iyad
 Sent: Tuesday, June 10, 2014 1:39 PM
 To: 'galaxy-...@bx.psu.edu'
 Subject: Re: Galaxy updated botched?



 Hi Everyone,



 This is follow-up information/questions to the issue I ran into with the
 galaxy June 2nd, 2014 update.  I hope to receive feedback on how to proceed.



 Background:

 -  Running Galaxy (DB Schema 118) with a MySQL 5.5 back-end

 -  When updating galaxy to the june 2nd release, the v120 DB schema
 has referential integrity constraints, which produced errors during the
 upgrade.

 -  Completed two galaxy updates in the past 4 months without
 encountering this before (schema changes included)



 Discussion:

 In the past, referential integrity in the DB schema was never an issue.  I
 checked backups and the current database to find that the database tables
 are using the MyISAM engine.  MyISAM =  no referential integrity support, no
 transactions.

 I reviewed galaxy’s SQLAlchemy templates and determined that
 mysql_engine='InnoDB' isn’t set on tables.  This explains why all tables
 were created with the MyISAM engine.  If the mysql_engine is not innodb, SQL
 Alchemy is supposed to drop any referential integrity constraints defined in
 the schema.  What I don’t understand is why SQL Alchemy is no longer
 ignoring the referential integrity constraints.



 Going forward, can anyone propose how I can salvage the database or continue
 ignoring referential integrity for now?

 Assuming that my limited understanding of SQLAlchemy holds water, I was
 looking at fixing the galaxy code base but I need some clarification on the
 DB schema versioning.  Do I edit schema v1 and add the appropriate table
 args to make every table an innodb engine table or do I add a new schema and
 modify all tables to use the innodb engine?  Alternatively, I can use DDL
 events

 def after_create(target, connection, **kw):

 connection.execute(ALTER TABLE %s ENGINE=InnoDB;

(target.name, target.name))



 Thank you for your help.



 Regards,

 Iyad Kandalaft



 Bioinformatics Application Developer

 Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada

 KW Neatby Bldg | éd. KW Neatby

 960 Carling Ave| 960, avenue Carling

 Ottawa, ON | Ottawa (ON) K1A 0C6

 E-mail Address / Adresse courriel: iyad.kandal...@agr.gc.ca

 Telephone | Téléphone 613- 759-1228

 Facsimile | Télécopieur 613-759-1701

 Government of Canada | Gouvernement du Canada




 

Re: [galaxy-dev] Galaxy updated botched?

2014-06-11 Thread Dannon Baker
Hey Iyad,

I just want to second (and add slightly) to what John said here.  I had
also failed to reproduce this locally, but that makes perfect sense now --
I'm glad you figured it out, and thanks for looking into this so closely
and reporting back.

Like John said, I wouldn't modify the migrations to force or assume a
particular engine, but if you wanted to make a contribution to the code
base it would probably be worth improving the detection and reporting of
this particular error condition to help anyone else who might run into the
issue -- if you wanted to take a stab at that.


On Wed, Jun 11, 2014 at 10:08 AM, John Chilton jmchil...@gmail.com wrote:

 Spent a couple hours yesterday trying to track down this - I was not
 getting anywhere though and I see why now.

 Well this definitely an unfortunate situation - but I think
 documentation improvements are the right fix not enforcing the engine
 type in sqlalchemy migrations. For one, I don't think we can assume a
 particular engine type because different deployers may already going
 to have both kinds out there right?

 I think it would be better to update the wiki to encourage InnoDB for
 all new installations and warn this can happen for MySQL upgrades.

 Is this okay?

 Sorry about this.

 -John



 On Wed, Jun 11, 2014 at 8:48 AM, Kandalaft, Iyad
 iyad.kandal...@agr.gc.ca wrote:
  This is a follow up for those that are interested with regards to my
 failed
  schema upgrade.
 
 
 
  I believe I have determined why all the tables are set to use the MyISAM
  engine.  When I initialized galaxy on our enterprise servers, they were
  running a dated version of CentOS. Hence, the OS defaults to a dated
 MySQL
  version.  MySQL only switched to using InnoDB (over MyISAM) as the
 default
  engine in version 5.5.  If I’m not mistaken, I initialized galaxy in
 MySQL
  5.1 without changing the default engine to InnoDB (big mistake).  Due to
 my
  ignorance, I will now have to try to compare v118 of our database with a
 new
  install of galaxy running schema v118.  Then, I will try “migrating” to
 the
  true schema state without destroying the data.
 
 
 
  I do feel that I should still modify the galaxy schema to set the MySQL
  engine to InnoDB to thwart problems like this for other unsuspecting
 users.
  If anyone can point me at some documentation about how Galaxy schema
 changes
  should occur in this case, that would be great.
 
  As per my previous comment, I’m not certain whether I would edit all
 schema
  versions to ensure table definitions include the mysql_engine=InnoDB
  attribute or whether using a DDL event in SQLAlchemy.  I suspect that
  setting this option globally would mean that future developers don’t
 need to
  remember to define mysql_engine on every new table.
 
 
 
 
 
  Iyad Kandalaft
 
  Microbial Biodiversity Bioinformatics
 
  Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
  960 Carling Ave.| 960 Ave. Carling
 
  Ottawa, ON| Ottawa (ON) K1A 0C6
 
  E-mail Address / Adresse courriel  iyad.kandal...@agr.gc.ca
  Telephone | Téléphone 613-759-1228
  Facsimile | Télécopieur 613-759-1701
  Teletypewriter | Téléimprimeur 613-773-2600
 
  Government of Canada | Gouvernement du Canada
 
 
 
 
 
 
 
  From: Kandalaft, Iyad
  Sent: Tuesday, June 10, 2014 1:39 PM
  To: 'galaxy-...@bx.psu.edu'
  Subject: Re: Galaxy updated botched?
 
 
 
  Hi Everyone,
 
 
 
  This is follow-up information/questions to the issue I ran into with the
  galaxy June 2nd, 2014 update.  I hope to receive feedback on how to
 proceed.
 
 
 
  Background:
 
  -  Running Galaxy (DB Schema 118) with a MySQL 5.5 back-end
 
  -  When updating galaxy to the june 2nd release, the v120 DB
 schema
  has referential integrity constraints, which produced errors during the
  upgrade.
 
  -  Completed two galaxy updates in the past 4 months without
  encountering this before (schema changes included)
 
 
 
  Discussion:
 
  In the past, referential integrity in the DB schema was never an issue.
  I
  checked backups and the current database to find that the database tables
  are using the MyISAM engine.  MyISAM =  no referential integrity
 support, no
  transactions.
 
  I reviewed galaxy’s SQLAlchemy templates and determined that
  mysql_engine='InnoDB' isn’t set on tables.  This explains why all tables
  were created with the MyISAM engine.  If the mysql_engine is not innodb,
 SQL
  Alchemy is supposed to drop any referential integrity constraints
 defined in
  the schema.  What I don’t understand is why SQL Alchemy is no longer
  ignoring the referential integrity constraints.
 
 
 
  Going forward, can anyone propose how I can salvage the database or
 continue
  ignoring referential integrity for now?
 
  Assuming that my limited understanding of SQLAlchemy holds water, I was
  looking at fixing the galaxy code base but I need some clarification on
 the
  DB schema versioning.  Do I edit schema v1 and add the appropriate table

Re: [galaxy-dev] Join Paired-End reads (version 1.0.3)

2014-06-11 Thread Jennifer Jackson

Hello Julien,

I don't know if you have had help already or not, but contacting the 
owners of this instance is the probably the quickest path to an answer 
(and possibly shared code). At this instance, under Help - Support, 
they have modified the link to point to the email address for their 
team. I'd start there.


Take care,

Jen
Galaxy team

On 5/28/14 7:38 AM, Julien Daligault wrote:

Hello,

I am interested by a wrapper in the galaxy-dev.jgi-psf.org : Join 
Paired-End reads (version 1.0.3) 
(https://galaxy-dev.jgi-psf.org/tool_runner?tool_id=jgi_rnd_merge_pe)
I didn't find it in the tool shed, where is it possible to access to 
this wrapper ?


Regards,

Julien



--
Jennifer Hillman-Jackson
http://galaxyproject.org

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] sorting of files in data libraries

2014-06-11 Thread Jennifer Jackson

Hi Sarah,

Data libraries are in an active enhancement phase. There are many 
changes recent, in progress, and planned that you can review in Trello.


This is one of them, and the specific card is here: 
https://trello.com/c/0RnzM6lP


For now, uploading with the name you wish to use in the UI is how to 
achieve the desired sort.


Thanks!

Jen
Galaxy team

On 5/23/14 2:37 AM, Sarah Diehl wrote:

Hi everyone,

I uploaded a bunch of files to the Galaxy data libraries with the option Upload files from 
filesystem paths and Link to files without copying to Galaxy. Afterwards I edited 
the names that are displayed in Galaxy. However, now the files are not alphabetically sorted 
anymore. I tried moving them around, but the order stayed the same. Is this on purpose and is there 
a way to change it?

Thanks,
Sarah
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


--
Jennifer Hillman-Jackson
http://galaxyproject.org

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Cannot find 'Export this revision' option on main or test toolsheds

2014-06-11 Thread Dave Bouvier

Will,

Thank you for reporting this issue, it has been fixed as of 
13827:5b67e047172f and 13826:0af48b3c8f33, which are now running on the 
test and main tool sheds. The change to the stable branch will be 
pushed to the galaxy-dist repository within a few days.


  --Dave B.

On Wed 11 Jun 2014 01:54:28 PM EDT, Will Holtz wrote:

I am running local toolshed with a recent version (latest_2014.06.02
for changeset fb68af9a775a). When viewing one of my tools, I can click
on the 'Repository Actions' button and see the option 'Export this
revision.' When I am using the public main or test toolsheds, I don't
see the 'Export this revision' option. In all cases I am a logged in
user. On the local toolshed I own the tool I am looking at, while on
the main and test toolsheds I am looking at tools owned by others (a
specific example would be package_biopython_1_62). Is this expected
behavior? How do I generate a capsule from the main or test toolsheds
to import into my local toolsheld?

thanks,
-Will


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


[galaxy-dev] Import of Capsules failing on local toolshed instance

2014-06-11 Thread Will Holtz
I am now able to export capsules from the main/test toolsheds -- thanks
Dave! When attempting to import these capsules into my local toolshed
(latest_2014.06.02
for changeset fb68af9a775a) I receive the following error:

URL: https://galaxy.lygos.com:99/repository/import_capsule
File
'/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py',
line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File
'/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py',
line 106 in __call__
  environ, self.app)
File
'/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py',
line 543 in intercept_output
  app_iter = application(environ, replacement_start_response)
File
'/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py',
line 84 in __call__
  return self.application(environ, start_response)
File
'/home/galaxy/galaxy-dist/lib/galaxy/webapps/tool_shed/framework/middleware/remoteuser.py',
line 74 in __call__
  return self.app( environ, start_response )
File
'/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py',
line 633 in __call__
  return self.application(environ, start_response)
File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132
in __call__
  return self.handle_request( environ, start_response )
File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190
in handle_request
  body = method( trans, **kwargs )
File
'/home/galaxy/galaxy-dist/lib/galaxy/webapps/tool_shed/controllers/repository.py',
line 1992 in import_capsule
  import_util.check_status_and_reset_downloadable( trans,
import_results_tups )
File '/home/galaxy/galaxy-dist/lib/tool_shed/util/import_util.py', line 34
in check_status_and_reset_downloadable
  tip_changeset_revision = repository.tip( trans.app )
AttributeError: 'NoneType' object has no attribute 'tip'

I have seen the same behavior for capsules based on the following
repositories from the test toolshed: package_biopython_1_62,
package_vienna_rna_2_1, and package_bowtie_0_12_7. I am logged in as an
admin user for the import process.

thanks,
-Will
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy updated botched?

2014-06-11 Thread Kandalaft, Iyad
Dannon  John,

Thank you for your feedback.  I agree that documentation is a good start.

I’m still in the process of figuring out what actually happens to the database 
schema.  It may be completely fine with no referential integrity constraints, 
which I’m okay with to some degree (we’re still in the infancy stages with 
Galaxy).  I find it odd that these referential integrity errors popped up “all 
of a sudden” because I don’t recall noticing these errors when I first 
installed galaxy/initialized the database.  I did move the database to the new 
version of MySQL, where InnoDB is the default and referencing primary keys on a 
MyISAM table caused the problem.

As a side note, I would be interested to know your reasons for avoiding a 
hardcoded mysql engine.  If galaxy depends on referential integrity (not that I 
am assuming it does), then setting the MySQL engine to Memory or MyISAM would 
be disastrous.  Also, do you see any distinct advantages to using MyISAM for 
galaxy?

Regards,

Iyad Kandalaft
Microbial Biodiversity Bioinformatics
Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
960 Carling Ave.| 960 Ave. Carling
Ottawa, ON| Ottawa (ON) K1A 0C6
E-mail Address / Adresse courriel  iyad.kandal...@agr.gc.ca
Telephone | Téléphone 613-759-1228
Facsimile | Télécopieur 613-759-1701
Teletypewriter | Téléimprimeur 613-773-2600
Government of Canada | Gouvernement du Canada


From: Dannon Baker [mailto:dannon.ba...@gmail.com]
Sent: Wednesday, June 11, 2014 10:23 AM
To: John Chilton
Cc: Kandalaft, Iyad; galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy updated botched?

Hey Iyad,
I just want to second (and add slightly) to what John said here.  I had also 
failed to reproduce this locally, but that makes perfect sense now -- I'm glad 
you figured it out, and thanks for looking into this so closely and reporting 
back.

Like John said, I wouldn't modify the migrations to force or assume a 
particular engine, but if you wanted to make a contribution to the code base it 
would probably be worth improving the detection and reporting of this 
particular error condition to help anyone else who might run into the issue -- 
if you wanted to take a stab at that.

On Wed, Jun 11, 2014 at 10:08 AM, John Chilton 
jmchil...@gmail.commailto:jmchil...@gmail.com wrote:
Spent a couple hours yesterday trying to track down this - I was not
getting anywhere though and I see why now.

Well this definitely an unfortunate situation - but I think
documentation improvements are the right fix not enforcing the engine
type in sqlalchemy migrations. For one, I don't think we can assume a
particular engine type because different deployers may already going
to have both kinds out there right?

I think it would be better to update the wiki to encourage InnoDB for
all new installations and warn this can happen for MySQL upgrades.

Is this okay?

Sorry about this.

-John



On Wed, Jun 11, 2014 at 8:48 AM, Kandalaft, Iyad
iyad.kandal...@agr.gc.camailto:iyad.kandal...@agr.gc.ca wrote:
 This is a follow up for those that are interested with regards to my failed
 schema upgrade.



 I believe I have determined why all the tables are set to use the MyISAM
 engine.  When I initialized galaxy on our enterprise servers, they were
 running a dated version of CentOS. Hence, the OS defaults to a dated MySQL
 version.  MySQL only switched to using InnoDB (over MyISAM) as the default
 engine in version 5.5.  If I’m not mistaken, I initialized galaxy in MySQL
 5.1 without changing the default engine to InnoDB (big mistake).  Due to my
 ignorance, I will now have to try to compare v118 of our database with a new
 install of galaxy running schema v118.  Then, I will try “migrating” to the
 true schema state without destroying the data.



 I do feel that I should still modify the galaxy schema to set the MySQL
 engine to InnoDB to thwart problems like this for other unsuspecting users.
 If anyone can point me at some documentation about how Galaxy schema changes
 should occur in this case, that would be great.

 As per my previous comment, I’m not certain whether I would edit all schema
 versions to ensure table definitions include the mysql_engine=InnoDB
 attribute or whether using a DDL event in SQLAlchemy.  I suspect that
 setting this option globally would mean that future developers don’t need to
 remember to define mysql_engine on every new table.





 Iyad Kandalaft

 Microbial Biodiversity Bioinformatics

 Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
 960 Carling Ave.| 960 Ave. Carling

 Ottawa, ON| Ottawa (ON) K1A 0C6

 E-mail Address / Adresse courriel  
 iyad.kandal...@agr.gc.camailto:iyad.kandal...@agr.gc.ca
 Telephone | Téléphone 613-759-1228tel:613-759-1228
 Facsimile | Télécopieur 613-759-1701tel:613-759-1701
 Teletypewriter | Téléimprimeur 613-773-2600tel:613-773-2600

 Government of Canada | Gouvernement du Canada







 From: Kandalaft, Iyad
 Sent: Tuesday, June 10, 2014 1:39 PM
 

Re: [galaxy-dev] Galaxy updated botched?

2014-06-11 Thread John Chilton
On Wed, Jun 11, 2014 at 3:26 PM, Kandalaft, Iyad
iyad.kandal...@agr.gc.ca wrote:
 Dannon  John,



 Thank you for your feedback.  I agree that documentation is a good start.



 I’m still in the process of figuring out what actually happens to the
 database schema.  It may be completely fine with no referential integrity
 constraints, which I’m okay with to some degree (we’re still in the infancy
 stages with Galaxy).  I find it odd that these referential integrity errors
 popped up “all of a sudden” because I don’t recall noticing these errors
 when I first installed galaxy/initialized the database.  I did move the
 database to the new version of MySQL, where InnoDB is the default and
 referencing primary keys on a MyISAM table caused the problem.



 As a side note, I would be interested to know your reasons for avoiding a
 hardcoded mysql engine.  If galaxy depends on referential integrity (not
 that I am assuming it does), then setting the MySQL engine to Memory or
 MyISAM would be disastrous.  Also, do you see any distinct advantages to
 using MyISAM for galaxy?


Well there have never been any reports of anyone having problems with
MyISAM in particular - its the switch here that seems to be the
problem as far as I can tell. If I had to venture a guess as to why no
one has complained about non-InnoDB engines - I would say the
referential integrity constraints are never violated by the statements
that the ORM generates - so they are a nice to have backup primary
protection - but are not strictly required.

Certainly, if one has to use MySQL and has a choice I would recommend
InnoDB for most interesting applications and Galaxy in particular, but
*IF* we forced it we would break existing Galaxy installations using
MyISAM right? If one did not have to worry about breaking Galaxy
installations - I think a majority of the devteam would like drop
support for MySQL entirely (and I think our recommendation of Postgres
is unanimous).

If anyone out there has tired using Galaxy with strictly MyISAM tables
and found it untenable - by all means please chime in.

-John



 Regards,



 Iyad Kandalaft

 Microbial Biodiversity Bioinformatics

 Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
 960 Carling Ave.| 960 Ave. Carling

 Ottawa, ON| Ottawa (ON) K1A 0C6

 E-mail Address / Adresse courriel  iyad.kandal...@agr.gc.ca
 Telephone | Téléphone 613-759-1228
 Facsimile | Télécopieur 613-759-1701
 Teletypewriter | Téléimprimeur 613-773-2600
 Government of Canada | Gouvernement du Canada





 From: Dannon Baker [mailto:dannon.ba...@gmail.com]
 Sent: Wednesday, June 11, 2014 10:23 AM
 To: John Chilton
 Cc: Kandalaft, Iyad; galaxy-...@bx.psu.edu
 Subject: Re: [galaxy-dev] Galaxy updated botched?



 Hey Iyad,

 I just want to second (and add slightly) to what John said here.  I had also
 failed to reproduce this locally, but that makes perfect sense now -- I'm
 glad you figured it out, and thanks for looking into this so closely and
 reporting back.

 Like John said, I wouldn't modify the migrations to force or assume a
 particular engine, but if you wanted to make a contribution to the code base
 it would probably be worth improving the detection and reporting of this
 particular error condition to help anyone else who might run into the issue
 -- if you wanted to take a stab at that.



 On Wed, Jun 11, 2014 at 10:08 AM, John Chilton jmchil...@gmail.com wrote:

 Spent a couple hours yesterday trying to track down this - I was not
 getting anywhere though and I see why now.

 Well this definitely an unfortunate situation - but I think
 documentation improvements are the right fix not enforcing the engine
 type in sqlalchemy migrations. For one, I don't think we can assume a
 particular engine type because different deployers may already going
 to have both kinds out there right?

 I think it would be better to update the wiki to encourage InnoDB for
 all new installations and warn this can happen for MySQL upgrades.

 Is this okay?

 Sorry about this.

 -John




 On Wed, Jun 11, 2014 at 8:48 AM, Kandalaft, Iyad
 iyad.kandal...@agr.gc.ca wrote:
 This is a follow up for those that are interested with regards to my
 failed
 schema upgrade.



 I believe I have determined why all the tables are set to use the MyISAM
 engine.  When I initialized galaxy on our enterprise servers, they were
 running a dated version of CentOS. Hence, the OS defaults to a dated MySQL
 version.  MySQL only switched to using InnoDB (over MyISAM) as the default
 engine in version 5.5.  If I’m not mistaken, I initialized galaxy in MySQL
 5.1 without changing the default engine to InnoDB (big mistake).  Due to
 my
 ignorance, I will now have to try to compare v118 of our database with a
 new
 install of galaxy running schema v118.  Then, I will try “migrating” to
 the
 true schema state without destroying the data.



 I do feel that I should still modify the galaxy schema to set the MySQL
 engine to InnoDB to thwart problems 

[galaxy-dev] bug: API broken under remote_user

2014-06-11 Thread Eric Rasche
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

https://trello.com/c/AGKePuHZ/1630-expose-use-remote-user-via-configuration-api

I don't know if this is the correct card (it's been a long day and I may
be misreading it) but the API is completely broken under REMOTE_USER
authentication.

running ./scripts/api/display.py {key} http://localhost:8300 returns 403
forbidden.

running ./scripts/api/display.py {key} https://fqdn/galaxy/ returns 401
Authorization Required

-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)

iQIcBAEBAgAGBQJTmPeFAAoJEMqDXdrsMcpVtmkP/1r36cFmedYJXnMFl2CG0e7P
VtSoMHYbtesc/IKnGwKEH1xtf2RNF7p8n0/0mC3QNAoI0n3A8XDoZRXaWzgmsZ4z
heGipPgotCyhFt/ud2W8eb0IlZ3AhtyWwvhG1hXcwQV4eXXHfuQCepPEbfyUUR2N
F+VCLMGMv3dnsv0ForC7iYesV4qqOsX48Wry6InVD41UpXOE0Bor5uB/HcohYjKS
sVhOmNSjLpME35Rka4vUW4oNIzeeBsQIQyrpCuZ2KTKImEkSoSEbTho9fhNUi0ll
EHlTu+qt4iaXZpqNOKU05p3TqdjxAmc2w/McmMzifwPZdmrxTZjaitFn+g1qXxYs
0ZVJtL/OZviFHuK+8qbbsCPIYYm/dVr5EPy821/F01dAGNw9k+lbk5kZdgC8zcBz
xFc5yk+QZGuc3KeDYLOidtn4c471Ez4LhgTd9wapYkSwva7aM5T1betpPZKufZbw
+SSC/KFi6J2rjVDOkHLCry185Bwha2hrn1tc2hd1+A/5zOuyqQ/u0isvRu+O7dxO
uGF33WjXCYqwVGNzXN+pZMVPStIXKlLmoA2CoWbUM9rHSQWHVRQKIiZ05v2tZYn7
/cpYLOwudWbQtDX+AscPd18bJ4OqSmy+aXo9nSkmJMbCWtN9TxKljx3MNzfg8DxV
r+SaijGDeWJ3xgiv2L6U
=svZc
-END PGP SIGNATURE-
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/