[galaxy-dev] proxy / ssl

[Shrum, Donald C]

Thanks for the help today Eric...

I'm not sure if this is worth adding to the docs but I thought I'd post it in 
case someone would like to.

I have an apache proxy setup for galaxy with ldap authentication.  I wanted the 
user sign on to be over https as users are using their HPC accounts to sign in.

I modified /etc/httpd/conf/httpd.conf to redirect requests from http to https:
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]

I then added the proxy configuration to /etc/httpd/conf.d/ssl.conf to the 
virtual host section:

RewriteEngine On
RewriteRule ^/static/style/(.*) 
/panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1 [L]
RewriteRule ^/static/scripts/(.*) 
/panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L]
RewriteRule ^/static/(.*) /panfs/storage.local/opt/galaxy-dist/static/$1 [L]
RewriteRule ^/favicon.ico 
/panfs/storage.local/opt/galaxy-dist/static/favicon.ico [L]
RewriteRule ^/robots.txt /panfs/storage.local/opt/galaxy-dist/static/robots.txt 
[L]
RewriteRule ^(.*) http://144.174.51.187:8080$1 [P]

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Developer Events in Europe this Fall: Swiss-German Galaxy Tour

[Dave Clements]

Hello all,

The *2014 Swiss-German Galaxy Tour
* is happening from 30
September through 2 October.  The tour features 3 events and they will all
be great ways to meet fellow developers, and learn more about defining
tools and deploying and administering Galaxy.  Space is limited at all
these events, so if you are interested, *you are strongly encouraged to
register now*.

The tour is organized (and hosted) by Hans-Rudolf Hotz and Björn Grüning.
The events are:

*Galaxy Training and Demo Day
*

30 September
Bern, Switzerland

*(second Swiss) Galaxy Workshop
 *

1 October
Bern, Switzerland


*German Galaxy Developers Day
 *

2 October
Freiburg, Germany


There are also several events for Galaxy Users planned this fall.  See
the Galaxy
Events page  for more.


Thanks,

Dave C

-- 
http://galaxyproject.org/
http://getgalaxy.org/
http://usegalaxy.org/
https://wiki.galaxyproject.org/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Problem Running TopHat2

[Alper Kucukural]

Hey Kevin, 
You need to install samtools and bowtie(2) into your system. I assume, the 
package you installed to galaxy is just a wrapper to call samtools. It doesn’t 
come with samtools binary.
Best,
Alper Kucukural, PhD
Bioinformatics Core,
University of Massachusetts Medical School
368 Plantation St.Room AS4.2067
Worcester, MA 01605-2324
Phone: 774-312-4493
E-mail: alper.kucuku...@umassmed.edu



On Aug 26, 2014, at 11:36 AM, Kevin  wrote:

> Hello All,
> 
> 
> I am running a Galaxy instance on Linux and have installed tools from the 
> Galaxy Main Tool Shed. However, whenever I run TopHat2, the program is unable 
> to complete the job before I get the following error message:
> 
> Traceback (most recent call last):
> 
> 
>   File "/home/ubuntu/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, 
> in queue_job
> 
> 
> job_wrapper.finish( stdout, stderr, exit_code )
> 
> 
>   File "/home/ubuntu/galaxy-dist/lib/galaxy/jobs/__init__.py", line 998, in 
> finish
> 
> 
> dataset.datatype.set_meta( dataset, overwrite=False )  # call 
> datatype.set_meta directly for the initial set_meta call during dataset 
> creation
> 
> 
>   File "/home/ubuntu/galaxy-dist/lib/galaxy/datatypes/binary.py", line 251, 
> in set_meta
> 
> 
> raise Exception, "Error Setting BAM Metadata: %s" % stderr
> 
> 
> Exception: Error Setting BAM Metadata: /bin/sh: 1: samtools: not found
> 
> So far, I have made sure that the samtools do appear to be installed since 
> they can be found listed under "Managed Installed Tool Shed Repositories". Is 
> it possible that they were installed incorrectly?
> Any help is much appreciated.
> 
> 
> Thanks for taking the time to read this,
> Kevin Chang
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] problems with database migration 119 -> 120

[Nate Coraor]

Hi Hans,

I'll add to John's statement about the ordering - a different ordering than the 
model is entirely possible because alters to add columns always add them at the 
end. The tables for Galaxy Main are so old that they are in a very different 
order than most newer installations. As John says, SQLAlchemy always references 
columns by name, so the order does not matter.

The reason we are using an old version of SQLAlchemy is due to it being the 
last version that can be used with sqlalchemy_migrate. Upgrading to newer 
versions will also require a significant amount of work to rewrite to use a new 
versioning system. This will be done eventually but hasn't reached the top of 
the pile yet.

Finally, you should switch to PostgreSQL. ;D

--nate

On Aug 26, 2014, at 11:37 AM, Hans-Rudolf Hotz  wrote:

> Hi John
> 
> Thanks for the insights
> 
> 
> Hi Iyad
> 
> Yes, the user account has the ability to drop and alter tables.
> 
> 
> Hans-Rudolf
> 
> 
> On 08/26/2014 03:08 PM, John Chilton wrote:
>> Well it looks like the migration file has these columns listed in a
>> different order than the mapping Galaxy uses - and the order yours
>> appeared in were the ones from Galaxy's mapping file. So somehow
>> Galaxy is automatically creating those tables prior to running the
>> migration based on the code in Galaxy proper. That is odd.
>> 
>> Ultimately though - I don't think it is harmful that the order is
>> wrong since Galaxy always references the columns by name instead of
>> order. I would be more concerned that you aren't getting the right
>> indices / foreign keys - but it looks like you are still on MyISAM so
>> you aren't going to get a ton of forced integrity anyway (and maybe
>> this is why these errors have been okay in the past?).
>> 
>> -John
>> 
>> On Tue, Aug 26, 2014 at 8:52 AM, Kandalaft, Iyad
>>  wrote:
>>> I've ran into other mysql problems but never anything like that.
>>> 
>>> This is just a hunch and not based on anything concrete, but using an 
>>> outdated version of sqlalchemy is probably not helping things.  We're 
>>> talking 2 migrations.  Are they even implementing fixes for 0.7 anymore?  
>>> It is odd you would ever get "Table already exists" since the ORM's job is 
>>> to ensure the model consistency in the database.  Why it would try to 
>>> create the table before it checks for its existence is beyond me.
>>> 
>>> You might want to make sure that the database user account has the ability 
>>> to drop and alter tables.  It may be that it tried to revert a failed 
>>> upgrade and it wasn't able to.
>>> 
>>> Iyad Kandalaft
>>> 
>>> 
>>> 
>>> -Original Message-
>>> From: galaxy-dev-boun...@lists.bx.psu.edu 
>>> [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Hans-Rudolf Hotz
>>> Sent: Tuesday, August 26, 2014 5:22 AM
>>> To: 
>>> Subject: [galaxy-dev] problems with database migration 119 -> 120
>>> 
>>> Hi all
>>> 
>>> I am in the process of updating our galaxy servers (from 
>>> "release_2014.04.14" to "latest_2014.08.11").
>>> 
>>> 
>>> when I execute
>>> 
>>> ~/lib/galaxy/model/migrate/versions/0120_dataset_collections.py
>>> 
>>> as part of the 'manage_db.sh upgrade' I run into a bizarre error:
>>> 
>>> First, it produces 10 "Mysql 1050 Error 'Table already exists' ", I have
>>> encountered this before, and usually everything is fine. The table gets
>>> created and for whatever reason, the command get's executed a second
>>> time - no big deal.
>>> 
>>> However, this time for two of those ten table the situation has been
>>> different. As usual, I have checked all the tables (where I got the
>>> errors) with the MySQL command describe .
>>> 
>>> For two tables:
>>> 
>>> history_dataset_collection_association
>>> 
>>> library_dataset_collection_association
>>> 
>>> the order of the columns was wrong (ie did not correspond to the order
>>> in the create statement) - see below for example.
>>> 
>>> I have dropped the tables and executed the create statements manually,
>>> everything seems fine, eg
>>> 
>>> 
>>> 
>>> mysql> describe library_dataset_collection_association;
>>> +---+--+--+-+-++
>>> | Field | Type | Null | Key | Default | Extra  |
>>> +---+--+--+-+-++
>>> | id| int(11)  | NO   | PRI | NULL| auto_increment |
>>> | collection_id | int(11)  | YES  | MUL | NULL||
>>> | folder_id | int(11)  | YES  | MUL | NULL||
>>> | name  | varchar(255) | YES  | | NULL||
>>> | deleted   | tinyint(1)   | YES  | | NULL||
>>> +---+--+--+-+-++
>>> 5 rows in set (0.00 sec)
>>> 
>>> mysql> drop table library_dataset_collection_association;
>>> Query OK, 0 rows affected (0.02 sec)
>>> 
>>> mysql> CREATE TABLE library_dataset_collection_association (
>>> 

[galaxy-dev] Problem Running TopHat2

[Kevin]

Hello All,
I am running a Galaxy instance on Linux and have installed tools from the 
Galaxy Main Tool Shed. However, whenever I run TopHat2, the program is unable 
to complete the job before I get the following error message:
Traceback (most recent call last):
File "/home/ubuntu/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in 
queue_job
job_wrapper.finish( stdout, stderr, exit_code )
File "/home/ubuntu/galaxy-dist/lib/galaxy/jobs/__init__.py", line 998, in finish
dataset.datatype.set_meta( dataset, overwrite=False ) # call datatype.set_meta 
directly for the initial set_meta call during dataset creation
File "/home/ubuntu/galaxy-dist/lib/galaxy/datatypes/binary.py", line 251, in 
set_meta
raise Exception, "Error Setting BAM Metadata: %s" % stderr
Exception: Error Setting BAM Metadata: /bin/sh: 1: samtools: not found
So far, I have made sure that the samtools do appear to be installed since they 
can be found listed under "Managed Installed Tool Shed Repositories". Is it 
possible that they were installed incorrectly?
Any help is much appreciated.
Thanks for taking the time to read this,
Kevin Chang

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] problems with database migration 119 -> 120

[Hans-Rudolf Hotz]

Hi John

Thanks for the insights


Hi Iyad

Yes, the user account has the ability to drop and alter tables.


Hans-Rudolf


On 08/26/2014 03:08 PM, John Chilton wrote:

Well it looks like the migration file has these columns listed in a
different order than the mapping Galaxy uses - and the order yours
appeared in were the ones from Galaxy's mapping file. So somehow
Galaxy is automatically creating those tables prior to running the
migration based on the code in Galaxy proper. That is odd.

Ultimately though - I don't think it is harmful that the order is
wrong since Galaxy always references the columns by name instead of
order. I would be more concerned that you aren't getting the right
indices / foreign keys - but it looks like you are still on MyISAM so
you aren't going to get a ton of forced integrity anyway (and maybe
this is why these errors have been okay in the past?).

-John

On Tue, Aug 26, 2014 at 8:52 AM, Kandalaft, Iyad
 wrote:

I've ran into other mysql problems but never anything like that.

This is just a hunch and not based on anything concrete, but using an outdated version of 
sqlalchemy is probably not helping things.  We're talking 2 migrations.  Are they even 
implementing fixes for 0.7 anymore?  It is odd you would ever get "Table already 
exists" since the ORM's job is to ensure the model consistency in the database.  Why 
it would try to create the table before it checks for its existence is beyond me.

You might want to make sure that the database user account has the ability to 
drop and alter tables.  It may be that it tried to revert a failed upgrade and 
it wasn't able to.

Iyad Kandalaft



-Original Message-
From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Hans-Rudolf Hotz
Sent: Tuesday, August 26, 2014 5:22 AM
To: 
Subject: [galaxy-dev] problems with database migration 119 -> 120

Hi all

I am in the process of updating our galaxy servers (from "release_2014.04.14" to 
"latest_2014.08.11").


when I execute

~/lib/galaxy/model/migrate/versions/0120_dataset_collections.py

as part of the 'manage_db.sh upgrade' I run into a bizarre error:

First, it produces 10 "Mysql 1050 Error 'Table already exists' ", I have
encountered this before, and usually everything is fine. The table gets
created and for whatever reason, the command get's executed a second
time - no big deal.

However, this time for two of those ten table the situation has been
different. As usual, I have checked all the tables (where I got the
errors) with the MySQL command describe .

For two tables:

history_dataset_collection_association

library_dataset_collection_association

the order of the columns was wrong (ie did not correspond to the order
in the create statement) - see below for example.

I have dropped the tables and executed the create statements manually,
everything seems fine, eg



mysql> describe library_dataset_collection_association;
+---+--+--+-+-++
| Field | Type | Null | Key | Default | Extra  |
+---+--+--+-+-++
| id| int(11)  | NO   | PRI | NULL| auto_increment |
| collection_id | int(11)  | YES  | MUL | NULL||
| folder_id | int(11)  | YES  | MUL | NULL||
| name  | varchar(255) | YES  | | NULL||
| deleted   | tinyint(1)   | YES  | | NULL||
+---+--+--+-+-++
5 rows in set (0.00 sec)

mysql> drop table library_dataset_collection_association;
Query OK, 0 rows affected (0.02 sec)

mysql> CREATE TABLE library_dataset_collection_association (
  -> id INTEGER NOT NULL AUTO_INCREMENT,
  -> collection_id INTEGER,
  -> name VARCHAR(255),
  -> deleted BOOL,
  -> folder_id INTEGER,
  -> PRIMARY KEY (id),
  -> FOREIGN KEY(collection_id) REFERENCES dataset_collection (id),
  -> CHECK (deleted IN (0, 1)),
  -> FOREIGN KEY(folder_id) REFERENCES library_folder (id)
  -> );
Query OK, 0 rows affected (0.01 sec)

mysql> describe library_dataset_collection_association;
+---+--+--+-+-++
| Field | Type | Null | Key | Default | Extra  |
+---+--+--+-+-++
| id| int(11)  | NO   | PRI | NULL| auto_increment |
| collection_id | int(11)  | YES  | MUL | NULL||
| name  | varchar(255) | YES  | | NULL||
| deleted   | tinyint(1)   | YES  | | NULL||
| folder_id | int(11)  | YES  | MUL | NULL||
+---+--+--+-+-++
5 rows in set (0.00 sec)

mysql>



Has anyo

Re: [galaxy-dev] Galaxy and object stores

[John Chilton]

Thanks Enis, just to elaborate on Pulsar - I suspect it would work
with something like configuring Galaxy with S3 object store right now
- but it would do so by having Galaxy cache the data locally and then
Pulsar would negotiate the transfer with Galaxy (many different ways
this could occur depending on who things are mounted). Ideally - it
wouldn't happen this way though - I would love it if Galaxy could
determine the job is going to be run remotely and not attempt the
cache and then configure the remote Pulsar to cache the file directly
from the object store abstraction. In addition to eliminating the
extra cache and transfer, it could allow Pulsar and Galaxy to have
different views of the underlying data sources (e.g. here the data is
mounted as X and there the data is mounted as Y - or here the data is
directly available and there get it via IRODS, etc...).

There are some ... initial grasps... at this sort of thing in Pulsar
and Galaxy but it is not fully (or even substantially) implemented
currently.

-John

On Tue, Aug 26, 2014 at 11:18 AM, Enis Afgan  wrote:
> Hi Inge,
> There is an implementation for using the AWS S3 object store as the data
> store for a given Galaxy instance. The implementation is located here
> https://bitbucket.org/galaxy/galaxy-central/src/3a51eaf209f2502bf32dbb421ecabb7fe46243ea/lib/galaxy/objectstore/s3.py?at=default
> and it offers several config options in universe_wsgi.ini.
>
> The data stored in S3 is locally cached while it's being operated on but
> always synced with the back end object store.
>
> Pulsar seems to have some support for S3 but, as the docs say in the
> implementation, it's explicitly beta:
> https://github.com/galaxyproject/pulsar/blob/b32b7caafc6582a3a28e694e2dbb75e7a8f2bffc/galaxy/objectstore/pulsar.py
>
> As a side note, there are some planned enhancements to how the object store
> implementation is handled and there will hopefully be quite a bit of
> activity on this topic in the near future (eg,
> https://trello.com/c/YynQKq8m).
>
> Hope this at least clarifies the state of object store support,
> Enis
>
>
> On Mon, Aug 25, 2014 at 10:24 AM, Raknes Inge Alexander
>  wrote:
>>
>> I have a few questions about object stores in Galaxy:
>>
>> 1: Can all Galaxy data sets be stored in an object store?
>> 2: If so,  does Galaxy still need to maintain a local copy of the data?
>> 3: Is LWR or Pulsar able to get the data directly from the object store,
>> or does it still have to go through Galaxy?
>>
>> We are planning to let users of our Galaxy installation handle large
>> input/output files (~30G) and we expect that the VM containing our Galaxy
>> installation will become a bottleneck if all data needs to travel through
>> that node.
>>
>> - Inge Alexander Raknes
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy and object stores

[Enis Afgan]

Hi Inge,
There is an implementation for using the AWS S3 object store as the data
store for a given Galaxy instance. The implementation is located here
https://bitbucket.org/galaxy/galaxy-central/src/3a51eaf209f2502bf32dbb421ecabb7fe46243ea/lib/galaxy/objectstore/s3.py?at=default
and it offers several config options in universe_wsgi.ini.

The data stored in S3 is locally cached while it's being operated on but
always synced with the back end object store.

Pulsar seems to have some support for S3 but, as the docs say in the
implementation, it's explicitly beta:
https://github.com/galaxyproject/pulsar/blob/b32b7caafc6582a3a28e694e2dbb75e7a8f2bffc/galaxy/objectstore/pulsar.py

As a side note, there are some planned enhancements to how the object store
implementation is handled and there will hopefully be quite a bit of
activity on this topic in the near future (eg, https://trello.com/c/YynQKq8m
).

Hope this at least clarifies the state of object store support,
Enis


On Mon, Aug 25, 2014 at 10:24 AM, Raknes Inge Alexander <
inge.a.rak...@uit.no> wrote:

>   ​I have a few questions about object stores in Galaxy:
>
>  1: Can all Galaxy data sets be stored in an object store?
>  2: If so,  does Galaxy still need to maintain a local copy of the data?
>  3: Is LWR or Pulsar able to get the data directly from the object store,
> or does it still have to go through Galaxy?
>
>  We are planning to let users of our Galaxy installation handle large
> input/output files (~30G) and we expect that the VM containing our Galaxy
> installation will become a bottleneck if all data needs to travel
> through that node.
>
>  - Inge Alexander Raknes
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ssl proxy

[Eric Rasche]

> is just an apache page

Is it an error page/the default index.html/other?

Next: Are you doing the proxying with mod_proxy or with mod_rewrite?

mod_rewrite looks like
RewriteRule ^(.*) https://144.174.51.187:8080$1 [P]

while mod_proxy looks like
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/


Cheers,
Eric

On 08/26/2014 09:39 AM, Shrum, Donald C wrote:
> If I go directly to https://galaxy.rcc.fsu.edu what I get is just an apache 
> page.  It's not doing the proxy to galaxy.
> 
> The debug shows...
> [Tue Aug 26 10:36:45 2014] [debug] proxy_util.c(1837): proxy: worker 
> proxy:reverse already initialized
> [Tue Aug 26 10:36:45 2014] [debug] proxy_util.c(1934): proxy: initialized 
> single connection worker 0 in child 21513 for (*)
> [Tue Aug 26 10:36:46 2014] [debug] proxy_util.c(1818): proxy: grabbed 
> scoreboard slot 0 in child 21514 for worker proxy:reverse
> [Tue Aug 26 10:36:46 2014] [debug] proxy_util.c(1837): proxy: worker 
> proxy:reverse already initialized 
> 
> -Original Message-
> From: Eric Rasche [mailto:rasche.e...@yandex.ru] 
> Sent: Tuesday, August 26, 2014 10:34 AM
> To: Shrum, Donald C
> Subject: Re: [galaxy-dev] ssl proxy
> 
>> not forwarded to the galaxy server
> 
> hmm. (technically it shouldn't be forwarding, once you hit https, it should 
> display galaxy directly). Is it just a blank white screen or a proper error 
> code?
> 
> Why don't you increase the log level and see if anything shows up in the 
> logs? Add
> 
>> LogLevel debug
> 
> and then tail all of your apache logs. What do those look like when you 
> request https://your-domain/ ?
> 
> Cheers,
> Eric
> 
> On 08/26/2014 09:25 AM, Shrum, Donald C wrote:
>> I get a redirect from http to https but then I'm not forwarded to the galaxy 
>> server.  
>>
>> If you have a suggestion I'll take it and thanks for the help with this.
>>
>> -Original Message-
>> From: Eric Rasche [mailto:rasche.e...@yandex.ru]
>> Sent: Tuesday, August 26, 2014 10:19 AM
>> To: Shrum, Donald C; Kandalaft, Iyad
>> Cc: galaxy-dev@lists.bx.psu.edu
>> Subject: Re: [galaxy-dev] ssl proxy
>>
>> What issues are you still seeing? How is it non-functional?
>>
>>
>> On 08/26/2014 08:32 AM, Shrum, Donald C wrote:
>>> I think I'd also need to change the proxy config to include the https
>>>
>>> 
>>> AddDefaultCharset off
>>> Order Deny,Allow
>>> Allow from all
>>>
>>>
>>>   https://144.174.51.187:8080>
>>> Order deny,allow
>>> Allow from all
>>>   
>>>
>>>
>>> Even with that and your change... no luck.  I'll keep tinkering. 
>>>
>>> -Original Message-
>>> From: Kandalaft, Iyad [mailto:iyad.kandal...@agr.gc.ca]
>>> Sent: Tuesday, August 26, 2014 8:44 AM
>>> To: Shrum, Donald C; Eric Rasche
>>> Cc: galaxy-dev@lists.bx.psu.edu
>>> Subject: RE: [galaxy-dev] ssl proxy
>>>
>>> Sorry Don, I forgot the http"s" in the proxy line:
>>> RewriteRule ^(.*) https://144.174.51.187:8080$1 [P]
>>>
>>> That should do it :)
>>>
>>>
>>>
>>> Iyad Kandalaft
>>>
>>>
>>> -Original Message-
>>> From: Shrum, Donald C [mailto:dcsh...@admin.fsu.edu]
>>> Sent: Monday, August 25, 2014 9:27 PM
>>> To: Kandalaft, Iyad; Eric Rasche
>>> Cc: galaxy-dev@lists.bx.psu.edu
>>> Subject: RE: [galaxy-dev] ssl proxy
>>>
>>> I'm not using a virtual host.  Apache is just there as a proxy.  I tried 
>>> the rewrite rules below...
>>>
>>> I get a redirect from 80 to 443 as expected for the ldap authentication.  
>>> After authenticating I do not get redirected to 8080 for galaxy.  I've 
>>> tried a few different ways and always get that.  I'll dig in tomorrow 
>>> morning and after some coffee to see if I can figure it out.  
>>>
>>> It's probably worth posting the solution in the docs.
>>>
>>> -Original Message-
>>> From: Kandalaft, Iyad [mailto:iyad.kandal...@agr.gc.ca]
>>> Sent: Monday, August 25, 2014 8:42 PM
>>> To: Eric Rasche; Shrum, Donald C
>>> Cc: galaxy-dev@lists.bx.psu.edu
>>> Subject: RE: [galaxy-dev] ssl proxy
>>>
>>> Hi Eric
>>>
>>> I'm not sure that he is using a virtual host on 443.  I suppose that a  
>>> "redirect permanent" would work in either case, but I was thinking along 
>>> the lines of:
>>>
>>>
>>> RewriteEngine On
>>>
>>> RewriteCond %{HTTPS} !=on
>>>
>>> RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
>>>
>>> RewriteRule ^/static/style/(.*)
>>> /panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1
>>> [L] RewriteRule ^/static/scripts/(.*)
>>> /panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L] 
>>> RewriteRule ^/static/(.*)
>>> /panfs/storage.local/opt/galaxy-dist/static/$1 [L] RewriteRule 
>>> ^/favicon.ico /panfs/storage.local/opt/galaxy-dist/static/favicon.ico
>>> [L]
>>>
>>> RewriteRule ^/robots.txt
>>> /panfs/storage.local/opt/galaxy-dist/static/robots.txt [L] 
>>> RewriteRule
>>> ^(.*) http://144.174.51.187:8080$1 [P]
>>>
>>>
>>> What do you think?
>>>
>>>
>>>
>>> Iyad Kandalaft
>>> Bioinformatics Programmer
>>> M

Re: [galaxy-dev] ssl proxy

[Eric Rasche]

What issues are you still seeing? How is it non-functional?


On 08/26/2014 08:32 AM, Shrum, Donald C wrote:
> I think I'd also need to change the proxy config to include the https
> 
> 
> AddDefaultCharset off
> Order Deny,Allow
> Allow from all
>
> 
>   https://144.174.51.187:8080>
> Order deny,allow
> Allow from all
>   
> 
> 
> Even with that and your change... no luck.  I'll keep tinkering. 
> 
> -Original Message-
> From: Kandalaft, Iyad [mailto:iyad.kandal...@agr.gc.ca] 
> Sent: Tuesday, August 26, 2014 8:44 AM
> To: Shrum, Donald C; Eric Rasche
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: RE: [galaxy-dev] ssl proxy
> 
> Sorry Don, I forgot the http"s" in the proxy line:
> RewriteRule ^(.*) https://144.174.51.187:8080$1 [P]
> 
> That should do it :)
> 
> 
> 
> Iyad Kandalaft
> 
> 
> -Original Message-
> From: Shrum, Donald C [mailto:dcsh...@admin.fsu.edu]
> Sent: Monday, August 25, 2014 9:27 PM
> To: Kandalaft, Iyad; Eric Rasche
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: RE: [galaxy-dev] ssl proxy
> 
> I'm not using a virtual host.  Apache is just there as a proxy.  I tried the 
> rewrite rules below...
> 
> I get a redirect from 80 to 443 as expected for the ldap authentication.  
> After authenticating I do not get redirected to 8080 for galaxy.  I've tried 
> a few different ways and always get that.  I'll dig in tomorrow morning and 
> after some coffee to see if I can figure it out.  
> 
> It's probably worth posting the solution in the docs.
> 
> -Original Message-
> From: Kandalaft, Iyad [mailto:iyad.kandal...@agr.gc.ca]
> Sent: Monday, August 25, 2014 8:42 PM
> To: Eric Rasche; Shrum, Donald C
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: RE: [galaxy-dev] ssl proxy
> 
> Hi Eric
> 
> I'm not sure that he is using a virtual host on 443.  I suppose that a  
> "redirect permanent" would work in either case, but I was thinking along the 
> lines of:
> 
> 
> RewriteEngine On
> 
> RewriteCond %{HTTPS} !=on
> 
> RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
> 
> RewriteRule ^/static/style/(.*) 
> /panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1 [L] 
> RewriteRule ^/static/scripts/(.*) 
> /panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule 
> ^/static/(.*) /panfs/storage.local/opt/galaxy-dist/static/$1 [L] RewriteRule 
> ^/favicon.ico /panfs/storage.local/opt/galaxy-dist/static/favicon.ico [L]
> 
> RewriteRule ^/robots.txt 
> /panfs/storage.local/opt/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) 
> http://144.174.51.187:8080$1 [P]
> 
> 
> What do you think?
> 
> 
> 
> Iyad Kandalaft
> Bioinformatics Programmer
> Microbial Biodiversity Bioinformatics
> Science & Technology Branch
> Agriculture & Agri-Food Canada
> iyad.kandal...@agr.gc.ca | (613) 759-1228 
> From: galaxy-dev-boun...@lists.bx.psu.edu 
> [galaxy-dev-boun...@lists.bx.psu.edu] on behalf of Eric Rasche 
> [rasche.e...@yandex.ru]
> Sent: August 25, 2014 4:21 PM
> To: dcsh...@admin.fsu.edu
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] ssl proxy
> 
> We use:
> 
> 
>   ServerName 
>   DocumentRoot "/var/www/"
>   
> Options Indexes FollowSymLinks MultiViews
> AllowOverride None
> Require all granted
>   
>   Redirect permanent / https:///
> 
> 
> Ours is managed with puppet so I've tried to redact the non-relevant blocks, 
> but that should work for you.
> 
> Cheers,
> Eric
> 
> On 08/25/2014 03:03 PM, Shrum, Donald C wrote:
>> Hi all,
>>
>> I have a galaxy install with an apache proxy and ldap authentication.
>>
>> 
>>RequestHeader set X-URL-SCHEME https
>>AuthBasicProvider ldap
>>   Other stuff
>> 
>>
>> RewriteEngine on
>> RewriteRule ^/static/style/(.*)
>> /panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1
>> [L] RewriteRule ^/static/scripts/(.*)
>> /panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L] 
>> RewriteRule ^/static/(.*)
>> /panfs/storage.local/opt/galaxy-dist/static/$1 [L] RewriteRule 
>> ^/favicon.ico /panfs/storage.local/opt/galaxy-dist/static/favicon.ico
>> [L] RewriteRule ^/robots.txt
>> /panfs/storage.local/opt/galaxy-dist/static/robots.txt [L] RewriteRule
>> ^(.*) http://144.174.51.187:8080$1 [P]
>>
>> What is the best way to manage the redirect from http to https for the 
>> authentication?
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this and other 
>> Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
> 
> 
> 
> Eric Rasche
> Programmer II
> Center for Phage Technology
> Texas A&M University
> College Station, TX 77843
> 404-692-2048 
> e...@tamu.edu 
> 

Re: [galaxy-dev] ssl proxy

[Shrum, Donald C]

I think I'd also need to change the proxy config to include the https


AddDefaultCharset off
Order Deny,Allow
Allow from all
   

  https://144.174.51.187:8080>
Order deny,allow
Allow from all
  


Even with that and your change... no luck.  I'll keep tinkering. 

-Original Message-
From: Kandalaft, Iyad [mailto:iyad.kandal...@agr.gc.ca] 
Sent: Tuesday, August 26, 2014 8:44 AM
To: Shrum, Donald C; Eric Rasche
Cc: galaxy-dev@lists.bx.psu.edu
Subject: RE: [galaxy-dev] ssl proxy

Sorry Don, I forgot the http"s" in the proxy line:
RewriteRule ^(.*) https://144.174.51.187:8080$1 [P]

That should do it :)



Iyad Kandalaft


-Original Message-
From: Shrum, Donald C [mailto:dcsh...@admin.fsu.edu]
Sent: Monday, August 25, 2014 9:27 PM
To: Kandalaft, Iyad; Eric Rasche
Cc: galaxy-dev@lists.bx.psu.edu
Subject: RE: [galaxy-dev] ssl proxy

I'm not using a virtual host.  Apache is just there as a proxy.  I tried the 
rewrite rules below...

I get a redirect from 80 to 443 as expected for the ldap authentication.  After 
authenticating I do not get redirected to 8080 for galaxy.  I've tried a few 
different ways and always get that.  I'll dig in tomorrow morning and after 
some coffee to see if I can figure it out.  

It's probably worth posting the solution in the docs.

-Original Message-
From: Kandalaft, Iyad [mailto:iyad.kandal...@agr.gc.ca]
Sent: Monday, August 25, 2014 8:42 PM
To: Eric Rasche; Shrum, Donald C
Cc: galaxy-dev@lists.bx.psu.edu
Subject: RE: [galaxy-dev] ssl proxy

Hi Eric

I'm not sure that he is using a virtual host on 443.  I suppose that a  
"redirect permanent" would work in either case, but I was thinking along the 
lines of:


RewriteEngine On

RewriteCond %{HTTPS} !=on

RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]

RewriteRule ^/static/style/(.*) 
/panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1 [L] 
RewriteRule ^/static/scripts/(.*) 
/panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule 
^/static/(.*) /panfs/storage.local/opt/galaxy-dist/static/$1 [L] RewriteRule 
^/favicon.ico /panfs/storage.local/opt/galaxy-dist/static/favicon.ico [L]

RewriteRule ^/robots.txt /panfs/storage.local/opt/galaxy-dist/static/robots.txt 
[L] RewriteRule ^(.*) http://144.174.51.187:8080$1 [P]


What do you think?



Iyad Kandalaft
Bioinformatics Programmer
Microbial Biodiversity Bioinformatics
Science & Technology Branch
Agriculture & Agri-Food Canada
iyad.kandal...@agr.gc.ca | (613) 759-1228 
From: galaxy-dev-boun...@lists.bx.psu.edu [galaxy-dev-boun...@lists.bx.psu.edu] 
on behalf of Eric Rasche [rasche.e...@yandex.ru]
Sent: August 25, 2014 4:21 PM
To: dcsh...@admin.fsu.edu
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] ssl proxy

We use:


  ServerName 
  DocumentRoot "/var/www/"
  
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Require all granted
  
  Redirect permanent / https:///


Ours is managed with puppet so I've tried to redact the non-relevant blocks, 
but that should work for you.

Cheers,
Eric

On 08/25/2014 03:03 PM, Shrum, Donald C wrote:
> Hi all,
>
> I have a galaxy install with an apache proxy and ldap authentication.
>
> 
>RequestHeader set X-URL-SCHEME https
>AuthBasicProvider ldap
>   Other stuff
> 
>
> RewriteEngine on
> RewriteRule ^/static/style/(.*)
> /panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1
> [L] RewriteRule ^/static/scripts/(.*)
> /panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L] 
> RewriteRule ^/static/(.*)
> /panfs/storage.local/opt/galaxy-dist/static/$1 [L] RewriteRule 
> ^/favicon.ico /panfs/storage.local/opt/galaxy-dist/static/favicon.ico
> [L] RewriteRule ^/robots.txt
> /panfs/storage.local/opt/galaxy-dist/static/robots.txt [L] RewriteRule
> ^(.*) http://144.174.51.187:8080$1 [P]
>
> What is the best way to manage the redirect from http to https for the 
> authentication?
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this and other 
> Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/



Eric Rasche
Programmer II
Center for Phage Technology
Texas A&M University
College Station, TX 77843
404-692-2048 
e...@tamu.edu 
--
Eric Rasche Programmer II Center for Phage Technology Texas A&M University 
College Station, TX 77843 404-692-2048 e...@tamu.edu 
rasche.e...@yandex.ru

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Gala

Re: [galaxy-dev] problems with database migration 119 -> 120

[John Chilton]

Well it looks like the migration file has these columns listed in a
different order than the mapping Galaxy uses - and the order yours
appeared in were the ones from Galaxy's mapping file. So somehow
Galaxy is automatically creating those tables prior to running the
migration based on the code in Galaxy proper. That is odd.

Ultimately though - I don't think it is harmful that the order is
wrong since Galaxy always references the columns by name instead of
order. I would be more concerned that you aren't getting the right
indices / foreign keys - but it looks like you are still on MyISAM so
you aren't going to get a ton of forced integrity anyway (and maybe
this is why these errors have been okay in the past?).

-John

On Tue, Aug 26, 2014 at 8:52 AM, Kandalaft, Iyad
 wrote:
> I've ran into other mysql problems but never anything like that.
>
> This is just a hunch and not based on anything concrete, but using an 
> outdated version of sqlalchemy is probably not helping things.  We're talking 
> 2 migrations.  Are they even implementing fixes for 0.7 anymore?  It is odd 
> you would ever get "Table already exists" since the ORM's job is to ensure 
> the model consistency in the database.  Why it would try to create the table 
> before it checks for its existence is beyond me.
>
> You might want to make sure that the database user account has the ability to 
> drop and alter tables.  It may be that it tried to revert a failed upgrade 
> and it wasn't able to.
>
> Iyad Kandalaft
>
>
>
> -Original Message-
> From: galaxy-dev-boun...@lists.bx.psu.edu 
> [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Hans-Rudolf Hotz
> Sent: Tuesday, August 26, 2014 5:22 AM
> To: 
> Subject: [galaxy-dev] problems with database migration 119 -> 120
>
> Hi all
>
> I am in the process of updating our galaxy servers (from "release_2014.04.14" 
> to "latest_2014.08.11").
>
>
> when I execute
>
> ~/lib/galaxy/model/migrate/versions/0120_dataset_collections.py
>
> as part of the 'manage_db.sh upgrade' I run into a bizarre error:
>
> First, it produces 10 "Mysql 1050 Error 'Table already exists' ", I have
> encountered this before, and usually everything is fine. The table gets
> created and for whatever reason, the command get's executed a second
> time - no big deal.
>
> However, this time for two of those ten table the situation has been
> different. As usual, I have checked all the tables (where I got the
> errors) with the MySQL command describe .
>
> For two tables:
>
> history_dataset_collection_association
>
> library_dataset_collection_association
>
> the order of the columns was wrong (ie did not correspond to the order
> in the create statement) - see below for example.
>
> I have dropped the tables and executed the create statements manually,
> everything seems fine, eg
>
>
>
> mysql> describe library_dataset_collection_association;
> +---+--+--+-+-++
> | Field | Type | Null | Key | Default | Extra  |
> +---+--+--+-+-++
> | id| int(11)  | NO   | PRI | NULL| auto_increment |
> | collection_id | int(11)  | YES  | MUL | NULL||
> | folder_id | int(11)  | YES  | MUL | NULL||
> | name  | varchar(255) | YES  | | NULL||
> | deleted   | tinyint(1)   | YES  | | NULL||
> +---+--+--+-+-++
> 5 rows in set (0.00 sec)
>
> mysql> drop table library_dataset_collection_association;
> Query OK, 0 rows affected (0.02 sec)
>
> mysql> CREATE TABLE library_dataset_collection_association (
>  -> id INTEGER NOT NULL AUTO_INCREMENT,
>  -> collection_id INTEGER,
>  -> name VARCHAR(255),
>  -> deleted BOOL,
>  -> folder_id INTEGER,
>  -> PRIMARY KEY (id),
>  -> FOREIGN KEY(collection_id) REFERENCES dataset_collection (id),
>  -> CHECK (deleted IN (0, 1)),
>  -> FOREIGN KEY(folder_id) REFERENCES library_folder (id)
>  -> );
> Query OK, 0 rows affected (0.01 sec)
>
> mysql> describe library_dataset_collection_association;
> +---+--+--+-+-++
> | Field | Type | Null | Key | Default | Extra  |
> +---+--+--+-+-++
> | id| int(11)  | NO   | PRI | NULL| auto_increment |
> | collection_id | int(11)  | YES  | MUL | NULL||
> | name  | varchar(255) | YES  | | NULL||
> | deleted   | tinyint(1)   | YES  | | NULL||
> | folder_id | int(11)  | YES  | MUL | NULL||
> +---+--+--+-+-++
> 5 rows in set (0.00 sec)
>
> mysql>
>
>
>
> Has anyon

Re: [galaxy-dev] problems with database migration 119 -> 120

[Kandalaft, Iyad]

I've ran into other mysql problems but never anything like that.

This is just a hunch and not based on anything concrete, but using an outdated 
version of sqlalchemy is probably not helping things.  We're talking 2 
migrations.  Are they even implementing fixes for 0.7 anymore?  It is odd you 
would ever get "Table already exists" since the ORM's job is to ensure the 
model consistency in the database.  Why it would try to create the table before 
it checks for its existence is beyond me.

You might want to make sure that the database user account has the ability to 
drop and alter tables.  It may be that it tried to revert a failed upgrade and 
it wasn't able to.

Iyad Kandalaft



-Original Message-
From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Hans-Rudolf Hotz
Sent: Tuesday, August 26, 2014 5:22 AM
To: 
Subject: [galaxy-dev] problems with database migration 119 -> 120

Hi all

I am in the process of updating our galaxy servers (from "release_2014.04.14" 
to "latest_2014.08.11").


when I execute

~/lib/galaxy/model/migrate/versions/0120_dataset_collections.py

as part of the 'manage_db.sh upgrade' I run into a bizarre error:

First, it produces 10 "Mysql 1050 Error 'Table already exists' ", I have 
encountered this before, and usually everything is fine. The table gets 
created and for whatever reason, the command get's executed a second 
time - no big deal.

However, this time for two of those ten table the situation has been 
different. As usual, I have checked all the tables (where I got the 
errors) with the MySQL command describe .

For two tables:

history_dataset_collection_association

library_dataset_collection_association

the order of the columns was wrong (ie did not correspond to the order 
in the create statement) - see below for example.

I have dropped the tables and executed the create statements manually, 
everything seems fine, eg



mysql> describe library_dataset_collection_association;
+---+--+--+-+-++
| Field | Type | Null | Key | Default | Extra  |
+---+--+--+-+-++
| id| int(11)  | NO   | PRI | NULL| auto_increment |
| collection_id | int(11)  | YES  | MUL | NULL||
| folder_id | int(11)  | YES  | MUL | NULL||
| name  | varchar(255) | YES  | | NULL||
| deleted   | tinyint(1)   | YES  | | NULL||
+---+--+--+-+-++
5 rows in set (0.00 sec)

mysql> drop table library_dataset_collection_association;
Query OK, 0 rows affected (0.02 sec)

mysql> CREATE TABLE library_dataset_collection_association (
 -> id INTEGER NOT NULL AUTO_INCREMENT,
 -> collection_id INTEGER,
 -> name VARCHAR(255),
 -> deleted BOOL,
 -> folder_id INTEGER,
 -> PRIMARY KEY (id),
 -> FOREIGN KEY(collection_id) REFERENCES dataset_collection (id),
 -> CHECK (deleted IN (0, 1)),
 -> FOREIGN KEY(folder_id) REFERENCES library_folder (id)
 -> );
Query OK, 0 rows affected (0.01 sec)

mysql> describe library_dataset_collection_association;
+---+--+--+-+-++
| Field | Type | Null | Key | Default | Extra  |
+---+--+--+-+-++
| id| int(11)  | NO   | PRI | NULL| auto_increment |
| collection_id | int(11)  | YES  | MUL | NULL||
| name  | varchar(255) | YES  | | NULL||
| deleted   | tinyint(1)   | YES  | | NULL||
| folder_id | int(11)  | YES  | MUL | NULL||
+---+--+--+-+-++
5 rows in set (0.00 sec)

mysql>



Has anyone else (among those few who are still using MySQL) seen 
something similar?


Regards, Hans-Rudolf


PS: Please don't make any comments about the fact, that I should change
 to PostgreSQL. I am more than aware of that, and eventually, I will
 do it.   ;)




-- 



Hans-Rudolf Hotz, PhD
Bioinformatics Support

Friedrich Miescher Institute for Biomedical Research
Maulbeerstrasse 66
4058 Basel/Switzerland
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and

Re: [galaxy-dev] ssl proxy

[Kandalaft, Iyad]

Sorry Don, I forgot the http"s" in the proxy line:
RewriteRule ^(.*) https://144.174.51.187:8080$1 [P]

That should do it :)



Iyad Kandalaft


-Original Message-
From: Shrum, Donald C [mailto:dcsh...@admin.fsu.edu] 
Sent: Monday, August 25, 2014 9:27 PM
To: Kandalaft, Iyad; Eric Rasche
Cc: galaxy-dev@lists.bx.psu.edu
Subject: RE: [galaxy-dev] ssl proxy

I'm not using a virtual host.  Apache is just there as a proxy.  I tried the 
rewrite rules below...

I get a redirect from 80 to 443 as expected for the ldap authentication.  After 
authenticating I do not get redirected to 8080 for galaxy.  I've tried a few 
different ways and always get that.  I'll dig in tomorrow morning and after 
some coffee to see if I can figure it out.  

It's probably worth posting the solution in the docs.

-Original Message-
From: Kandalaft, Iyad [mailto:iyad.kandal...@agr.gc.ca]
Sent: Monday, August 25, 2014 8:42 PM
To: Eric Rasche; Shrum, Donald C
Cc: galaxy-dev@lists.bx.psu.edu
Subject: RE: [galaxy-dev] ssl proxy

Hi Eric

I'm not sure that he is using a virtual host on 443.  I suppose that a  
"redirect permanent" would work in either case, but I was thinking along the 
lines of:


RewriteEngine On

RewriteCond %{HTTPS} !=on

RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]

RewriteRule ^/static/style/(.*) 
/panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1 [L] 
RewriteRule ^/static/scripts/(.*) 
/panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule 
^/static/(.*) /panfs/storage.local/opt/galaxy-dist/static/$1 [L] RewriteRule 
^/favicon.ico /panfs/storage.local/opt/galaxy-dist/static/favicon.ico [L]

RewriteRule ^/robots.txt /panfs/storage.local/opt/galaxy-dist/static/robots.txt 
[L] RewriteRule ^(.*) http://144.174.51.187:8080$1 [P]


What do you think?



Iyad Kandalaft
Bioinformatics Programmer
Microbial Biodiversity Bioinformatics
Science & Technology Branch
Agriculture & Agri-Food Canada
iyad.kandal...@agr.gc.ca | (613) 759-1228 
From: galaxy-dev-boun...@lists.bx.psu.edu [galaxy-dev-boun...@lists.bx.psu.edu] 
on behalf of Eric Rasche [rasche.e...@yandex.ru]
Sent: August 25, 2014 4:21 PM
To: dcsh...@admin.fsu.edu
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] ssl proxy

We use:


  ServerName 
  DocumentRoot "/var/www/"
  
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Require all granted
  
  Redirect permanent / https:///


Ours is managed with puppet so I've tried to redact the non-relevant blocks, 
but that should work for you.

Cheers,
Eric

On 08/25/2014 03:03 PM, Shrum, Donald C wrote:
> Hi all,
>
> I have a galaxy install with an apache proxy and ldap authentication.
>
> 
>RequestHeader set X-URL-SCHEME https
>AuthBasicProvider ldap
>   Other stuff
> 
>
> RewriteEngine on
> RewriteRule ^/static/style/(.*)
> /panfs/storage.local/opt/galaxy-dist/static/june_2007_style/blue/$1
> [L] RewriteRule ^/static/scripts/(.*)
> /panfs/storage.local/opt/galaxy-dist/static/scripts/packed/$1 [L] 
> RewriteRule ^/static/(.*)
> /panfs/storage.local/opt/galaxy-dist/static/$1 [L] RewriteRule 
> ^/favicon.ico /panfs/storage.local/opt/galaxy-dist/static/favicon.ico
> [L] RewriteRule ^/robots.txt
> /panfs/storage.local/opt/galaxy-dist/static/robots.txt [L] RewriteRule
> ^(.*) http://144.174.51.187:8080$1 [P]
>
> What is the best way to manage the redirect from http to https for the 
> authentication?
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this and other 
> Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/



Eric Rasche
Programmer II
Center for Phage Technology
Texas A&M University
College Station, TX 77843
404-692-2048 
e...@tamu.edu 
--
Eric Rasche Programmer II Center for Phage Technology Texas A&M University 
College Station, TX 77843 404-692-2048 e...@tamu.edu 
rasche.e...@yandex.ru

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Concept for a Galaxy Versioned Fasta Data Retrieval Tool

[Sebastian Schaaf]

Damion,

Thanks a lot - consequently treating the toppic of 'reproducable 
science' is a competition, but absolutely required. Björn really touched 
my mind when he gave the linked talk in Chicago (GCC 2012). Although for 
a longer time things got stuck, I think that Galaxy is still a (the?) 
key to it, because the principal structures allow it - it's somehow 
native to it. Since some important and powerful elements came up (think 
of e.g. the API), the gap for reaching reproducability also from the 
reference side using 'on-board tools' of the framework (without bending 
or disassembling the code too strong) has hardly narrowed.


Due to the medical context our instance is in, we really need features 
like this. In some particular detail I would maybe implement the 
respective functionalities a bit different (due to performance), but in 
vast majority I agree: this sounds attractive!


Marius' remark on data managers (which are brand new as far as I 
understood the GCC talks) sounds reasonable, although I did not get in 
touch with it yet.


So, count me in, I'm already a bit excited :).

Cheers,
Sebastian



Dooley, Damion schrieb:

Ok, I'll be very happy to see what you've accomplished there.  I will read 
through what you've done when I return from vacation in a week!

A key need is to have whatever data comes in show up as linked data in one's 
history to avoid server overhead; a second objective was to not need to modify 
existing workflows - as long as they could work of data in history that is 
typed appropriately.  So your 'select type' solution sounds intreguing!

And certainly interested in your use of git - I tried using git, using a 1-line 
fasta data format, but git seemed to choke on protein fasta files?  And did it 
run into performance problems with larger files?  That was my experience.  I 
think I read its authors say that its upper limit was 15gb.  That was the 
motivation for writing a simple key-value master file diff system that seems to 
have the same I/O as git on smaller files, but more reliable for the fasta data 
case, and no problems with larger files - it outputs a new version in the same 
time it takes to read a master file.  It has drawbacks though - incoming data 
to compare master with must be sorted in 1 line fasta format first.

Thanks for your input; looking forward to your project writeup...

Damion

Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for 
Disease Control
655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

From: Björn Grüning [bjoern.gruen...@gmail.com]
Sent: Saturday, August 23, 2014 12:17 AM
To: Dooley, Damion; galaxy-dev@lists.bx.psu.edu
Cc: Hsiao, William
Subject: Re: [galaxy-dev] Concept for a Galaxy Versioned Fasta Data Retrieval 
Tool

Hi Damion,

the idea sounds fantastic!
Can we go a step further and use a specific datatype that keeps entire
fasta files versioned and the user can choose which version he wants to
use, in any tool? Please have a look at my talk at GCC2012. Maybe you
are interested in the (old) patches. I would be very interested to
restart this old project.

https://wiki.galaxyproject.org/Events/GCC2012/Abstracts#Keeping_Track_of_Life_Science_Data


Am 23.08.2014 um 03:24 schrieb Dooley, Damion:

We are about to implement a fasta database (file) versioning system as a Galaxy 
tool.  I wanted to get interested people's feedback first before we roll ahead 
with the prototype implementation.  The versioning system aims to:

- Simple makeblastdb or bowtie-build commands, or more specific workflows 
that include dustmasker etc can be implemented.

Does this sound attractive?

I think all of the use cases are covered by the old project mentioned
above. But I did not create a new tool I have created a new 'select
type' everyone can use in all tools. It was using git underneath (yeah,
I have the entire PDB in git and it is working fine :)) but we can
probably change git with a database if you like.

To answer your question: Yes, very attractive!


We're hoping such a vision could handle Fasta databases from 12mb to e.g. 200Gb 
(probably requires makeblastdb in parallel at that scale).

Preliminary work suggests this project is doable via the Galaxy API without 
galaxy customization - does that sound right?!

Yes, as long as the User has an API key.

Cheers,
Bjoern
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/



--
Sebastian Schaaf, M.Sc. Bioinformatics
Faculty Coordinator NGS Infrastructure
Chair of Biometry and Bioinformatics
Department of Medical Informatics,
 Biometry and Epidemiology (IBE)
University of Munich
Marchioninistr. 15, K U

[galaxy-dev] problems with database migration 119 -> 120

[Hans-Rudolf Hotz]

Hi all

I am in the process of updating our galaxy servers (from 
"release_2014.04.14" to "latest_2014.08.11").



when I execute

~/lib/galaxy/model/migrate/versions/0120_dataset_collections.py

as part of the 'manage_db.sh upgrade' I run into a bizarre error:

First, it produces 10 "Mysql 1050 Error 'Table already exists' ", I have 
encountered this before, and usually everything is fine. The table gets 
created and for whatever reason, the command get's executed a second 
time - no big deal.


However, this time for two of those ten table the situation has been 
different. As usual, I have checked all the tables (where I got the 
errors) with the MySQL command describe .


For two tables:

history_dataset_collection_association

library_dataset_collection_association

the order of the columns was wrong (ie did not correspond to the order 
in the create statement) - see below for example.


I have dropped the tables and executed the create statements manually, 
everything seems fine, eg




mysql> describe library_dataset_collection_association;
+---+--+--+-+-++
| Field | Type | Null | Key | Default | Extra  |
+---+--+--+-+-++
| id| int(11)  | NO   | PRI | NULL| auto_increment |
| collection_id | int(11)  | YES  | MUL | NULL||
| folder_id | int(11)  | YES  | MUL | NULL||
| name  | varchar(255) | YES  | | NULL||
| deleted   | tinyint(1)   | YES  | | NULL||
+---+--+--+-+-++
5 rows in set (0.00 sec)

mysql> drop table library_dataset_collection_association;
Query OK, 0 rows affected (0.02 sec)

mysql> CREATE TABLE library_dataset_collection_association (
-> id INTEGER NOT NULL AUTO_INCREMENT,
-> collection_id INTEGER,
-> name VARCHAR(255),
-> deleted BOOL,
-> folder_id INTEGER,
-> PRIMARY KEY (id),
-> FOREIGN KEY(collection_id) REFERENCES dataset_collection (id),
-> CHECK (deleted IN (0, 1)),
-> FOREIGN KEY(folder_id) REFERENCES library_folder (id)
-> );
Query OK, 0 rows affected (0.01 sec)

mysql> describe library_dataset_collection_association;
+---+--+--+-+-++
| Field | Type | Null | Key | Default | Extra  |
+---+--+--+-+-++
| id| int(11)  | NO   | PRI | NULL| auto_increment |
| collection_id | int(11)  | YES  | MUL | NULL||
| name  | varchar(255) | YES  | | NULL||
| deleted   | tinyint(1)   | YES  | | NULL||
| folder_id | int(11)  | YES  | MUL | NULL||
+---+--+--+-+-++
5 rows in set (0.00 sec)

mysql>



Has anyone else (among those few who are still using MySQL) seen 
something similar?



Regards, Hans-Rudolf


PS: Please don't make any comments about the fact, that I should change
to PostgreSQL. I am more than aware of that, and eventually, I will
do it.   ;)




--



Hans-Rudolf Hotz, PhD
Bioinformatics Support

Friedrich Miescher Institute for Biomedical Research
Maulbeerstrasse 66
4058 Basel/Switzerland
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/