Hi Sarah, On 08. aug. 2016 13:58, Sarah DIEHL wrote: > Hi Eric, > > thanks for the hint regarding uWSGI. What doesn't work is importing > files from the local disk (of the galaxy server) to the data library > (see attached screenshot). Everything else seems to be fine, I haven't > encountered any other issues. Maybe it's a bug just in that function? > Here is the error: > > 10.184.132.10 - - [30/Jul/2016:18:09:27 +0200] "POST > /api/libraries/datasets?encoded_folder_id=F7b46bd6d01de922f&source=userdir_file&path=160308_WTCHG_254732_201.bam&file_type=auto&dbkey=? > HTTP/1.1" 500 - "https://galaxy-server.uni.lu/library/list"; > "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 > Firefox/45.0" > Error - <type 'exceptions.AssertionError'>: use_remote_user is set but > HTTP_REMOTE_USER header was not provided > URL: > https://galaxy-server.uni.lu/api/libraries/datasets?encoded_folder_id=F7b46bd6d01de922f&source=userdir_file&path=160308_WTCHG_254732_201.bam&file_type=auto&dbkey=? > File > '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', > line 151 in __call__ > app_iter = self.application(environ, sr_checker) > File > '/mnt/gaiagpfs/projects/galaxy/internal/.venv/local/lib/python2.7/site-packages/paste/recursive.py', > line 85 in __call__ > return self.application(environ, start_response) > File > '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py', > line 76 in __call__ > return self.app( environ, start_response ) > File > '/mnt/gaiagpfs/projects/galaxy/internal/.venv/local/lib/python2.7/site-packages/paste/httpexceptions.py', > line 640 in __call__ > return self.application(environ, start_response) > File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line > 131 in __call__ > return self.handle_request( environ, start_response ) > File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line > 158 in handle_request > trans = self.transaction_factory( environ ) > File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py', > line 68 in <lambda> > self.set_transaction_factory( lambda e: self.transaction_chooser( e, > galaxy_app, session_cookie ) ) > File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py', > line 99 in transaction_chooser > return GalaxyWebTransaction( environ, galaxy_app, self, session_cookie ) > File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py', > line 198 in __init__ > self.error_message = self._authenticate_api( session_cookie ) > File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py', > line 380 in _authenticate_api > self._ensure_valid_session( session_cookie ) > File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/webapp.py', > line 432 in _ensure_valid_session > "use_remote_user is set but %s header was not provided" % > self.app.config.remote_user_header > AssertionError: use_remote_user is set but HTTP_REMOTE_USER header was > not provided I'm afraid I do not have any help for this. Hopefully another galaxy person can chime in here and help debug this.
> > > Best regards, > Sarah > > ---- > Sarah Diehl > HPC System Administrator > > UNIVERSITÉ DU LUXEMBOURG > > LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE > Campus Belval | Biotech II > 6, avenue du Swing > L-4371 Belvaux > T +352 46 66 44 5360 > sarah.di...@uni.lu <mailto:sarah.di...@uni.lu> http://lcsb.uni.lu > <https://urldefense.proofpoint.com/v2/url?u=http-3A__lcsb.uni.lu_&d=CwMF-g&c=ODFT-G5SujMiGrKuoJJjVg&r=p9uZby14OqW9zcjBSjiDKw&m=pwYDgdauhpWnUSnnqVAB6Xyn2OzpGsIMkWdC8fyNRu0&s=Ll-0NRWHk-0Kb159GLi8cgxakoFwK4BQyp39IRY0S48&e=> > ----- > This message is confidential and may contain privileged information. > It is intended for the named recipient only. If you receive it in > error please notify me and permanently delete the original message and > any copies. > ----- > > > From: Eric Rasche <e...@tamu.edu <mailto:e...@tamu.edu>> > Organization: TAMU > Date: Monday 8 August 2016 15:43 > To: Sarah DIEHL <sarah.di...@uni.lu <mailto:sarah.di...@uni.lu>>, > "galaxy-dev@lists.galaxyproject.org > <mailto:galaxy-dev@lists.galaxyproject.org>" > <galaxy-dev@lists.galaxyproject.org > <mailto:galaxy-dev@lists.galaxyproject.org>> > Subject: Re: [galaxy-dev] External user auth and API > > > > On 08. aug. 2016 13:25, Sarah DIEHL wrote: >> Hi Eric, >> >> thanks a lot for the info and help! I'm running version 16.04 and my >> apache conf is a bit different because I balance over multiple galaxy >> web servers: > Just as an aside, if you are still using this configuration method, > you might consider switching to uWSGI > <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.galaxyproject.org_Admin_Config_Performance_Scaling&d=CwMF-g&c=ODFT-G5SujMiGrKuoJJjVg&r=p9uZby14OqW9zcjBSjiDKw&m=pwYDgdauhpWnUSnnqVAB6Xyn2OzpGsIMkWdC8fyNRu0&s=FcalyPc9I-0TF6rCq1c3yxRT6-Rrzmf7mNE3GVfCNBI&e=>, > it will drastically simplify your configuration and should be just as > performant (if not more). > >> >> >> # API requests get routed through balancer with no authentication >> RewriteCond %{QUERY_STRING} key= >> RewriteRule ^/api/(.*) balancer://galaxy-noauth/api/$1 [P] >> >> # Regular requests get routed through balancer with LDAP >> authentication >> RewriteRule ^(.*) balancer://galaxy$1 [P] >> >> <Proxy balancer://galaxy/*> >> BalancerMember http://localhost:8080 >> BalancerMember http://localhost:8081 >> BalancerMember http://localhost:8082 >> BalancerMember http://localhost:8083 >> >> RequestHeader set X-URL-SCHEME https >> >> AuthType Basic >> AuthBasicProvider ldap >> AuthName "UL HPC Platform Authorized Users Only" >> AuthLDAPBindAuthoritative off >> AuthLDAPURL "ldap://…"; >> Require valid-user >> RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e >> >> XSendFile on >> XSendFilePath / >> >> SetOutputFilter DEFLATE >> SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary >> SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary >> SetEnvIfNoCase Request_URI /history/export_archive no-gzip >> dont-vary >> </Proxy> >> >> <Proxy balancer://galaxy-noauth/*> >> BalancerMember http://localhost:8080 >> BalancerMember http://localhost:8081 >> BalancerMember http://localhost:8082 >> BalancerMember http://localhost:8083 >> >> RequestHeader set X-URL-SCHEME https >> >> Satisfy any >> >> XSendFile on >> XSendFilePath / >> >> SetOutputFilter DEFLATE >> SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary >> SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary >> SetEnvIfNoCase Request_URI /history/export_archive no-gzip >> dont-vary >> </Proxy> >> >> What doesn't work when configured this way is data libraries' import >> from user directory function. Can you tell if my apache configuration >> is equivalent to yours? > Your configuration /looks/ equivalent, yes, but I may have overlooked > something, I have since forgotten some of the semantics of proxy > balancers. >> Does the import functionality in the data libraries work for you? > Import datasets from data libraries to history? Yes, this does for me. > In what way does it break for you? Unauthenticated /api requests? It > seems odd that data libraries would be the only affected thing here, > since other things like the tool panel also make /api requests. >> >> This configuration worked fine with release 15.10, but doesn't >> anymore since I upgraded. Between those two version the default data >> libraries interface changed. What used to be "beta" before is now the >> default and only available option. > Old data libraries were deprecated > <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.galaxyproject.org_en_master_releases_15.10-5Fannounce.html-23deprecation-2Dnotices&d=CwMF-g&c=ODFT-G5SujMiGrKuoJJjVg&r=p9uZby14OqW9zcjBSjiDKw&m=pwYDgdauhpWnUSnnqVAB6Xyn2OzpGsIMkWdC8fyNRu0&s=p6Y66XyId9RfOGHSilb-eCggUDdkXaloeEyrXZBLQbM&e=> > in favour of the new ones and removed in 16.01. >> >> Best regards, >> Sarah >> >> >> ---- >> Sarah Diehl >> HPC System Administrator >> >> UNIVERSITÉ DU LUXEMBOURG >> >> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE >> Campus Belval | Biotech II >> 6, avenue du Swing >> L-4371 Belvaux >> T +352 46 66 44 5360 >> sarah.di...@uni.lu <mailto:sarah.di...@uni.lu> http://lcsb.uni.lu >> ----- >> This message is confidential and may contain privileged information. >> It is intended for the named recipient only. If you receive it in >> error please notify me and permanently delete the original message >> and any copies. >> ----- >> >> >> From: Eric Rasche <e...@tamu.edu> >> Organization: TAMU >> Date: Monday 8 August 2016 15:09 >> To: Sarah DIEHL <sarah.di...@uni.lu>, >> "galaxy-dev@lists.galaxyproject.org >> <mailto:galaxy-dev@lists.galaxyproject.org>" >> <galaxy-dev@lists.galaxyproject.org >> <mailto:galaxy-dev@lists.galaxyproject.org>> >> Subject: Re: [galaxy-dev] External user auth and API >> >> Hi Sarah, >> >> >> On 08. aug. 2016 07:44, Sarah DIEHL wrote: >>> Dear all, >>> >>> since no one replied so far to the main problem I had and it might >>> have gotten lost in the conversation, I ask again: Does somebody >>> know how to configure external user auth with apache such that API >>> (from external, e.g. bioblend) and dataset import in the data >>> libraries work? When I configure apache to require auth for >>> everything, the API does not work. If I except the API from the >>> apache auth, the dataset import does not work. >> >> Our configuration looks like the following (just switching CAS for LDAP.) >> >> <Location "/galaxy/api/"> >> Satisfy Any >> Allow from all >> </Location> >> >> <Location "/galaxy"> >> AuthName "CAS" >> AuthType CAS >> Require valid-user >> RequestHeader set X-URL-SCHEME https >> XSendFile on >> XSendFilePath / >> RequestHeader set CAS-User "%{REMOTE_USER}s...@tamu.edu" >> </Location> >> ProxyPass /galaxy uwsgi://127.0.0.1:4001/ >> >> I.e. we disable authentication on the /api route. On 16.01+ (I think >> it was patched then, but 16.04 is a safer bet) this will work >> correctly and your users will be able to use the API. On previous >> versions the /api route would fail for web users if exposed in this >> manner. >>> >>> If I switch to the new galaxy-internal LDAP auth features, will that >>> solve this problem? >> Yes, this is an alternate solution. >>> >>> Any hints are appreciated! >>> >>> Best regards, >>> Sarah >>> >>> >>> ---- >>> Sarah Diehl >>> HPC System Administrator >>> >>> UNIVERSITÉ DU LUXEMBOURG >>> >>> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE >>> Campus Belval | Biotech II >>> 6, avenue du Swing >>> L-4371 Belvaux >>> T +352 46 66 44 5360 >>> <mailto:sarah.di...@uni.lu>sarah.di...@uni.lu http://lcsb.uni.lu >>> ----- >>> This message is confidential and may contain privileged information. >>> It is intended for the named recipient only. If you receive it in >>> error please notify me and permanently delete the original message >>> and any copies. >>> ----- >>> >>> >>> From: galaxy-dev <galaxy-dev-boun...@lists.galaxyproject.org> on >>> behalf of Sarah DIEHL <sarah.di...@uni.lu <mailto:sarah.di...@uni.lu>> >>> Date: Monday 1 August 2016 13:06 >>> To: Nicola Soranzo <nsora...@tiscali.it>, >>> "galaxy-dev@lists.galaxyproject.org >>> <mailto:galaxy-dev@lists.galaxyproject.org>" >>> <galaxy-dev@lists.galaxyproject.org >>> <mailto:galaxy-dev@lists.galaxyproject.org>> >>> Subject: Re: [galaxy-dev] Remote user auth and API >>> >>> Hi Nicola, >>> >>> thanks a lot for the help! Yes, it's a self-signed certificate, I >>> didn't bother with letsencrypt yet ;-). >>> >>> So now the error turned to >>> ConnectionError: GET: error 401: b'<!DOCTYPE HTML PUBLIC >>> "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>401 Authorization >>> Required</title>\n</head><body>\n<h1>Authorization >>> Required</h1>\n<p>This server could not verify that you\nare >>> authorized to access the document\nrequested. Either you supplied >>> the wrong\ncredentials (e.g., bad password), or your\nbrowser >>> doesn\'t understand how to supply\nthe credentials >>> required.</p>\n</body></html>\n', 0 attempts left: None >>> which is what I expected, since apache now wants the authentication >>> through LDAP. >>> >>> So anybody know what the right settings are to get both the dataset >>> import and the API working with external user auth over apache and LDAP? >>> >>> Thanks, >>> Sarah >>> >>> ---- >>> Sarah Diehl >>> HPC System Administrator >>> >>> UNIVERSITÉ DU LUXEMBOURG >>> >>> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE >>> Campus Belval | Biotech II >>> 6, avenue du Swing >>> L-4371 Belvaux >>> T +352 46 66 44 5360 >>> <mailto:sarah.di...@uni.lu>sarah.di...@uni.lu http://lcsb.uni.lu >>> ----- >>> This message is confidential and may contain privileged information. >>> It is intended for the named recipient only. If you receive it in >>> error please notify me and permanently delete the original message >>> and any copies. >>> ----- >>> >>> >>> From: Nicola Soranzo <nicola.sora...@gmail.com >>> <mailto:nicola.sora...@gmail.com>> on behalf of Nicola Soranzo >>> <nsora...@tiscali.it> >>> Date: Monday 1 August 2016 12:58 >>> To: Sarah DIEHL <sarah.di...@uni.lu <mailto:sarah.di...@uni.lu>>, >>> "galaxy-dev@lists.galaxyproject.org >>> <mailto:galaxy-dev@lists.galaxyproject.org>" >>> <galaxy-dev@lists.galaxyproject.org >>> <mailto:galaxy-dev@lists.galaxyproject.org>> >>> Subject: Re: [galaxy-dev] Remote user auth and API >>> >>> Hi Sarah! >>> I guess that your problem is with an untrusted certificate, you can >>> get one for free at https://letsencrypt.org/ >>> >>> You can disable certificate verification in bioblend as in the >>> example below: >>> >>> import bioblend.galaxy >>> gi = bioblend.galaxy.GalaxyInstance(url=my_server, key=my_key) >>> gi.verify = False >>> >>> Cheers, >>> Nicola >>> >>> On 01/08/16 09:08, Sarah DIEHL wrote: >>>> Dear all, >>>> >>>> since the recent update to 16.04 I get the following error when >>>> trying to import a file from a user directory to a data library: >>>> >>>> AssertionError: use_remote_user is set but HTTP_REMOTE_USER header >>>> was not provided >>>> >>>> I use apache as a proxy and use an LDAP server for authentication. >>>> In order to get the API to work previously the apache had to be set >>>> to not check authentication for the requests to /api. In the logs I >>>> can see that the dataset import is an request to the API, so since >>>> the auth is not checked then, there is also no REMOTE_USER header set. >>>> >>>> What is the recommended way to solve this issue with the current >>>> Galaxy version? I disabled the special settings for /api and the >>>> dataset import works now. >>>> >>>> I tried to check the API with an old test script based on bioblend, >>>> but I now get the following error: >>>> >>>> ConnectionError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate >>>> verify failed (_ssl.c:645), 0 attempts left: None >>>> >>>> Previously I could disable it with >>>> >>>> import requests >>>> requests.packages.urllib3.disable_warnings() >>>> >>>> but that doesn't seem to work anymore (switched to Python 3 now). >>>> Since bioblend wraps all the requests methods, I cannot apply any >>>> of the common solutions I found online (e.g. set verify=False). >>>> >>>> Any help to solve these issues is highly appreciated :-). >>>> >>>> Best regards, >>>> Sarah >>>> >>>> >>>> >>>> ---- >>>> Sarah Diehl >>>> HPC System Administrator >>>> >>>> UNIVERSITÉ DU LUXEMBOURG >>>> >>>> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE >>>> Campus Belval | Biotech II >>>> 6, avenue du Swing >>>> L-4371 Belvaux >>>> T +352 46 66 44 5360 >>>> <mailto:sarah.di...@uni.lu>sarah.di...@uni.lu http://lcsb.uni.lu >>>> ----- >>>> This message is confidential and may contain privileged >>>> information. It is intended for the named recipient only. If you >>>> receive it in error please notify me and permanently delete the >>>> original message and any copies. >>>> ----- >>>> >>>> >>>> >>>> ___________________________________________________________ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> https://lists.galaxyproject.org/ >>>> >>>> To search Galaxy mailing lists use the unified search at: >>>> http://galaxyproject.org/search/mailinglists/ >>> >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >> >> -- >> Eric Rasche >> Programmer II >> >> Center for Phage Technology >> Rm 312A, Biochemistry & Biophysics >> Texas A&M University >> College Station, TX 77843 >> e...@tamu.edu <mailto:e...@tamu.edu> > > -- > Eric Rasche > Programmer II > > Center for Phage Technology > Rm 312A, Biochemistry & Biophysics > Texas A&M University > College Station, TX 77843 > e...@tamu.edu <mailto:e...@tamu.edu> -- Eric Rasche Programmer II Center for Phage Technology Rm 312A, Biochemistry & Biophysics Texas A&M University College Station, TX 77843 e...@tamu.edu <mailto:e...@tamu.edu>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
