Re: [galaxy-dev] External user auth and API

[Eric Rasche]

Hi Sarah,

On 08. aug. 2016 13:58, Sarah DIEHL wrote:
> Hi Eric,
>
> thanks for the hint regarding uWSGI. What doesn't work is importing
> files from the local disk (of the galaxy server) to the data library
> (see attached screenshot). Everything else seems to be fine, I haven't
> encountered any other issues. Maybe it's a bug just in that function?
> Here is the error:
>
> 10.184.132.10 - - [30/Jul/2016:18:09:27 +0200] "POST
> /api/libraries/datasets?encoded_folder_id=F7b46bd6d01de922f=userdir_file=160308_WTCHG_254732_201.bam_type=auto=?
> HTTP/1.1" 500 - "https://galaxy-server.uni.lu/library/list;
> "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101
> Firefox/45.0"
> Error - : use_remote_user is set but
> HTTP_REMOTE_USER header was not provided
> URL:
> https://galaxy-server.uni.lu/api/libraries/datasets?encoded_folder_id=F7b46bd6d01de922f=userdir_file=160308_WTCHG_254732_201.bam_type=auto=?
> File
> '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py',
> line 151 in __call__
>   app_iter = self.application(environ, sr_checker)
> File
> '/mnt/gaiagpfs/projects/galaxy/internal/.venv/local/lib/python2.7/site-packages/paste/recursive.py',
> line 85 in __call__
>   return self.application(environ, start_response)
> File
> '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py',
> line 76 in __call__
>   return self.app( environ, start_response )
> File
> '/mnt/gaiagpfs/projects/galaxy/internal/.venv/local/lib/python2.7/site-packages/paste/httpexceptions.py',
> line 640 in __call__
>   return self.application(environ, start_response)
> File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line
> 131 in __call__
>   return self.handle_request( environ, start_response )
> File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line
> 158 in handle_request
>   trans = self.transaction_factory( environ )
> File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py',
> line 68 in 
>   self.set_transaction_factory( lambda e: self.transaction_chooser( e,
> galaxy_app, session_cookie ) )
> File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py',
> line 99 in transaction_chooser
>   return GalaxyWebTransaction( environ, galaxy_app, self, session_cookie )
> File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py',
> line 198 in __init__
>   self.error_message = self._authenticate_api( session_cookie )
> File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py',
> line 380 in _authenticate_api
>   self._ensure_valid_session( session_cookie )
> File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py',
> line 432 in _ensure_valid_session
>   "use_remote_user is set but %s header was not provided" %
> self.app.config.remote_user_header
> AssertionError: use_remote_user is set but HTTP_REMOTE_USER header was
> not provided
I'm afraid I do not have any help for this. Hopefully another galaxy
person can chime in here and help debug this.

>
>
> Best regards,
> Sarah
>
> 
> Sarah Diehl
> HPC System Administrator
>  
> UNIVERSITÉ DU LUXEMBOURG
>  
> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE
> Campus Belval | Biotech II
> 6, avenue du Swing
> L-4371 Belvaux
> T +352 46 66 44 5360
> sarah.di...@uni.lu <mailto:sarah.di...@uni.lu> http://lcsb.uni.lu
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lcsb.uni.lu_=CwMF-g=ODFT-G5SujMiGrKuoJJjVg=p9uZby14OqW9zcjBSjiDKw=pwYDgdauhpWnUSnnqVAB6Xyn2OzpGsIMkWdC8fyNRu0=Ll-0NRWHk-0Kb159GLi8cgxakoFwK4BQyp39IRY0S48=>
> -
> This message is confidential and may contain privileged information.
> It is intended for the named recipient only. If you receive it in
> error please notify me and permanently delete the original message and
> any copies.
> -
>
>
> From: Eric Rasche <e...@tamu.edu <mailto:e...@tamu.edu>>
> Organization: TAMU
> Date: Monday 8 August 2016 15:43
> To: Sarah DIEHL <sarah.di...@uni.lu <mailto:sarah.di...@uni.lu>>,
> "galaxy-dev@lists.galaxyproject.org
> <mailto:galaxy-dev@lists.galaxyproject.org>"
> <galaxy-dev@lists.galaxyproject.org
> <mailto:galaxy-dev@lists.galaxyproject.org>>
> Subject: Re: [galaxy-dev] External user auth and API
>
>
>
> On 08. aug. 2016 13:25, Sarah DIEHL wrote:
>> Hi Eric,
>>
>> thanks a lot for the info and help! I'm running version 16.04 and my
>> apache conf is a bit different because I balance over multiple galaxy
>> web servers:
> Just as an aside, if you are still using this configuration method,
> you might consider switching to uWSGI
> <https://urldefense.proofpoint.com/v2/url

Re: [galaxy-dev] External user auth and API

[Eric Rasche]

Hi Sarah,


On 08. aug. 2016 07:44, Sarah DIEHL wrote:
> Dear all,
>
> since no one replied so far to the main problem I had and it might
> have gotten lost in the conversation, I ask again: Does somebody know
> how to configure external user auth with apache such that API (from
> external, e.g. bioblend) and dataset import in the data libraries
> work? When I configure apache to require auth for everything, the API
> does not work. If I except the API from the apache auth, the dataset
> import does not work.

Our configuration looks like the following (just switching CAS for LDAP.)


Satisfy Any
Allow from all



AuthName "CAS"
AuthType CAS
Require valid-user
RequestHeader set X-URL-SCHEME https
XSendFile on
XSendFilePath /
RequestHeader set CAS-User "%{REMOTE_USER}s...@tamu.edu"

ProxyPass /galaxy uwsgi://127.0.0.1:4001/

I.e. we disable authentication on the /api route. On 16.01+ (I think it
was patched then, but 16.04 is a safer bet) this will work correctly and
your users will be able to use the API. On previous versions the /api
route would fail for web users if exposed in this manner.
>
> If I switch to the new galaxy-internal LDAP auth features, will that
> solve this problem?
Yes, this is an alternate solution.
>
> Any hints are appreciated!
>
> Best regards,
> Sarah
>
>
> 
> Sarah Diehl
> HPC System Administrator
>  
> UNIVERSITÉ DU LUXEMBOURG
>  
> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE
> Campus Belval | Biotech II
> 6, avenue du Swing
> L-4371 Belvaux
> T +352 46 66 44 5360
> sarah.di...@uni.lu  http://lcsb.uni.lu
> 
> -
> This message is confidential and may contain privileged information.
> It is intended for the named recipient only. If you receive it in
> error please notify me and permanently delete the original message and
> any copies.
> -
>
>
> From: galaxy-dev  > on behalf of
> Sarah DIEHL >
> Date: Monday 1 August 2016 13:06
> To: Nicola Soranzo >,
> "galaxy-dev@lists.galaxyproject.org
> "
>  >
> Subject: Re: [galaxy-dev] Remote user auth and API
>
> Hi Nicola,
>
> thanks a lot for the help! Yes, it's a self-signed certificate, I
> didn't bother with letsencrypt yet ;-).
>
> So now the error turned to
> ConnectionError: GET: error 401: b' HTML 2.0//EN">\n\n401 Authorization
> Required\n\nAuthorization
> Required\nThis server could not verify that you\nare
> authorized to access the document\nrequested. Either you supplied the
> wrong\ncredentials (e.g., bad password), or your\nbrowser doesn\'t
> understand how to supply\nthe credentials
> required.\n\n', 0 attempts left: None
> which is what I expected, since apache now wants the authentication
> through LDAP.
>
> So anybody know what the right settings are to get both the dataset
> import and the API working with external user auth over apache and LDAP?
>
> Thanks,
> Sarah
>
> 
> Sarah Diehl
> HPC System Administrator
>  
> UNIVERSITÉ DU LUXEMBOURG
>  
> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE
> Campus Belval | Biotech II
> 6, avenue du Swing
> L-4371 Belvaux
> T +352 46 66 44 5360
> sarah.di...@uni.lu  http://lcsb.uni.lu
> 
> -
> This message is confidential and may contain privileged information.
> It is intended for the named recipient only. If you receive it in
> error please notify me and permanently delete the original message and
> any copies.
> -
>
>
> From: Nicola Soranzo  > on behalf of Nicola Soranzo
> >
> Date: Monday 1 August 2016 12:58
> To: Sarah DIEHL >,
> "galaxy-dev@lists.galaxyproject.org
> "
>  >
> Subject: Re: [galaxy-dev] Remote user auth and API
>
> Hi Sarah!
> I guess that your problem is with an untrusted certificate, you can
> get one for free at https://letsencrypt.org/
>
> You can disable certificate verification in bioblend as in the example
> below:
>
> import bioblend.galaxy
> gi = bioblend.galaxy.GalaxyInstance(url=my_server, 

[galaxy-dev] External user auth and API

[Sarah DIEHL]

Dear all,

since no one replied so far to the main problem I had and it might have gotten 
lost in the conversation, I ask again: Does somebody know how to configure 
external user auth with apache such that API (from external, e.g. bioblend) and 
dataset import in the data libraries work? When I configure apache to require 
auth for everything, the API does not work. If I except the API from the apache 
auth, the dataset import does not work.

If I switch to the new galaxy-internal LDAP auth features, will that solve this 
problem?

Any hints are appreciated!

Best regards,
Sarah



Sarah Diehl
HPC System Administrator

UNIVERSITÉ DU LUXEMBOURG

LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE
Campus Belval | Biotech II
6, avenue du Swing
L-4371 Belvaux
T +352 46 66 44 5360
sarah.di...@uni.lu 
http://lcsb.uni.lu
-
This message is confidential and may contain privileged information. It is 
intended for the named recipient only. If you receive it in error please notify 
me and permanently delete the original message and any copies.
-


From: galaxy-dev 
>
 on behalf of Sarah DIEHL >
Date: Monday 1 August 2016 13:06
To: Nicola Soranzo >, 
"galaxy-dev@lists.galaxyproject.org" 
>
Subject: Re: [galaxy-dev] Remote user auth and API

Hi Nicola,

thanks a lot for the help! Yes, it's a self-signed certificate, I didn't bother 
with letsencrypt yet ;-).

So now the error turned to

ConnectionError: GET: error 401: b'\n\n401 Authorization 
Required\n\nAuthorization Required\nThis 
server could not verify that you\nare authorized to access the 
document\nrequested.  Either you supplied the wrong\ncredentials (e.g., bad 
password), or your\nbrowser doesn\'t understand how to supply\nthe credentials 
required.\n\n', 0 attempts left: None

which is what I expected, since apache now wants the authentication through 
LDAP.

So anybody know what the right settings are to get both the dataset import and 
the API working with external user auth over apache and LDAP?

Thanks,
Sarah


Sarah Diehl
HPC System Administrator

UNIVERSITÉ DU LUXEMBOURG

LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE
Campus Belval | Biotech II
6, avenue du Swing
L-4371 Belvaux
T +352 46 66 44 5360
sarah.di...@uni.lu 
http://lcsb.uni.lu
-
This message is confidential and may contain privileged information. It is 
intended for the named recipient only. If you receive it in error please notify 
me and permanently delete the original message and any copies.
-


From: Nicola Soranzo 
> on behalf of Nicola 
Soranzo >
Date: Monday 1 August 2016 12:58
To: Sarah DIEHL >, 
"galaxy-dev@lists.galaxyproject.org" 
>
Subject: Re: [galaxy-dev] Remote user auth and API

Hi Sarah!
I guess that your problem is with an untrusted certificate, you can get one for 
free at https://letsencrypt.org/

You can disable certificate verification in bioblend as in the example below:

import bioblend.galaxy
gi = bioblend.galaxy.GalaxyInstance(url=my_server, key=my_key)
gi.verify = False

Cheers,
Nicola

On 01/08/16 09:08, Sarah DIEHL wrote:
Dear all,

since the recent update to 16.04 I get the following error when trying to 
import a file from a user directory to a data library:

AssertionError: use_remote_user is set but HTTP_REMOTE_USER header was not 
provided

I use apache as a proxy and use an LDAP server for authentication. In order to 
get the API to work previously the apache had to be set to not check 
authentication for the requests to /api. In the logs I can see that the dataset 
import is an request to the API, so since the auth is not checked then, there 
is also no REMOTE_USER header set.

What is the recommended way to solve this issue with the current Galaxy 
version? I disabled the special settings for /api and the dataset import works 
now.

I tried to check the API with an old test script based on bioblend, but I now 
get the following error:

ConnectionError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed 
(_ssl.c:645), 0 attempts left: None

Previously I could disable it with

import requests
requests.packages.urllib3.disable_warnings()

but that doesn't seem to work anymore (switched to Python 3 now). Since 
bioblend wraps all the requests methods, I cannot apply any of the common 
solutions I found online (e.g. set verify=False).

Any help to solve these issues