Re: FBR: Fix openshift websockets
On 09/11/2018 04:58 PM, Patrick Uiterwijk wrote: > Hi all, > > After a bunch of debugging, I found that the problems with Openshifts > websockets (used for logs and shell etc) were due to: > - HTTP/2 (Upgrade: and Connection: headers get silently dropped for HTTP/2, > websockets aren't defined yet for it...) > - Balancer protocols > > I'd like +1s to apply the underneath patch to fix this, and enable logs from > the prod openshift web console. > It is working in staging. +1. Pity it does not also fix firefox, but oh well. kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
FBR: Fix openshift websockets
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Hi all,
After a bunch of debugging, I found that the problems with Openshifts
websockets (used for logs and shell etc) were due to:
- - HTTP/2 (Upgrade: and Connection: headers get silently dropped for HTTP/2,
websockets aren't defined yet for it...)
- - Balancer protocols
I'd like +1s to apply the underneath patch to fix this, and enable logs from
the prod openshift web console.
It is working in staging.
commit 095fe0257320998e0f316787042cb4a0245ad345 (HEAD -> master)
Author: Patrick Uiterwijk
Date: Wed Sep 12 01:55:40 2018 +0200
Fix websockets for prod openshift
Signed-off-by: Patrick Uiterwijk
diff --git a/playbooks/include/proxies-websites.yml
b/playbooks/include/proxies-websites.yml
index 403e1e6f1..5e9bf89ae 100644
- --- a/playbooks/include/proxies-websites.yml
+++ b/playbooks/include/proxies-websites.yml
@@ -576,6 +576,9 @@
site_name: os.fedoraproject.org
sslonly: true
cert_name: "{{wildcard_cert_name}}"
+# The Connection and Upgrade headers don't work for h2
+# So non-h2 is needed to fix websockets.
+use_h2: false
tags:
- os.fedoraproject.org
@@ -585,6 +588,9 @@
sslonly: true
cert_name: "{{os_wildcard_cert_name}}"
SSLCertificateChainFile: "{{os_wildcard_int_file}}"
+# The Connection and Upgrade headers don't work for h2
+# So non-h2 is needed to fix websockets.
+use_h2: false
tags:
- app.os.fedoraproject.org
diff --git a/roles/httpd/reverseproxy/templates/reversepassproxy.conf
b/roles/httpd/reverseproxy/templates/reversepassproxy.conf
index 77a8dd35b..06f913720 100644
- --- a/roles/httpd/reverseproxy/templates/reversepassproxy.conf
+++ b/roles/httpd/reverseproxy/templates/reversepassproxy.conf
@@ -19,7 +19,6 @@ ProxyPreserveHost On
{% if balancer_name is defined %}
SSLProxyEngine On
- -{% if env == "staging" %}
{% for member in balancer_members %}
@@ -32,7 +31,6 @@ RewriteCond %{HTTP:Upgrade} ^WebSocket$ [NC]
RewriteCond %{HTTP:Connection} ^Upgrade$ [NC]
RewriteRule .* "balancer://{{ balancer_name }}-websocket%{REQUEST_URI}" [P]
- -{% endif %}
{% for member in balancer_members %}
BalancerMember "https://{{ member }}"
-BEGIN PGP SIGNATURE-
iQIcBAEBCgAGBQJbmFaOAAoJEIZXmA2atR5QgWEP/RsP1mtvud2LOSFLcg6CLbl1
dfB4hH39rmY27/Gkr0keLxgtBczejrhJ1HQKMDZiInegESvgE7+msLN/SAHnjG8H
5KZ5SNRWfyEdlYvXxk5CMZjpOIeZMWHff2+LJvfBX2Z5GNq6OPzgtiM+7jPwjlma
NDGqgHgDRDkhQKoAy34l069GieHBF3P7W0rP8A3Jf56WdBrahCAspdhHYU79TTyg
8tM0wsFYCBFxOJfH1YfUKp6vmfs2Fi255PBALETUZe7LsLD+E5RQI5ILZwWhHUlr
JD9EpV74wk128mygwfDZCfWpnEPuHVaUWAB5yHpIuoXzHENysIZTJ0oF3JNmxh5K
F2c0tHutUD9zSigKbUKLql2ZHQCRz9iM8Hhl/EL7YqQAZosHzf68HqhTjjwazD9U
vMXBrwmIZrmjgqtqOyhWJ3SH8MuosmD9Exbx14ekaq0p8uj7g8kzug9le1oQjk3Q
4HfxVphfWHdZ2LMAn4lOboiZACqU3rkvYEk+7uwCXpRLioO0UdVTaqLoyx9Z2NeP
StHuClZZOMgFAQ38b6VsLGBOa7nUj/JzDeEVrdaIX0tzaNWGeaEVY9kpOIFDRaOg
h02O1aJJCzX+n5Xz3DtJH84GgWqL2Ir4sX8ZgIpkRJW1mpES0JXOpAbhh/I9M3zG
IfmiXpXmSa1CK0FXufNB
=9UjA
-END PGP SIGNATURE-
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Fedora Infrastructure 2018-09-13 meeting moved to 1500 UTC
Due to another meeting, we need to move this meeting back an hour on Thursday to 1500 UTC. This will be a 1 time move back. We apologize for any problems this causes people in very early time zones. -- Stephen J Smoogen. ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: Moving forward with Fedora's PDC
On 09/11/2018 11:48 AM, Clement Verna wrote: > Hi all, > > I would like to start working on the replacement application for PDC in > Fedora. So far I have been mostly looking at the technical side of the > application and how it will be setup in our infrastructure. > > Below is a list of the design decisions I am looking to take. I would > really value some feedback on this in order to see if this is realistic or > not. > > # Technical stack > * Python 3 (just making it obvious) > * Django 1.11 LTS (supported until at least April 2020) [0] > * DRF 3.8.2 (latest version) > Using Django and DRF (Django REST Framework) should allow us to reuse some > of PDC code. > > # Application architecture > * 1 backend service used to update the PDC database (currently > pdc-updater [1]) > * 1 frontend service serving the API HTTP endpoint and the HTML > documentation of these endpoint. > * 1 database > > # Deployment architecture > * Backend service runs in Openshift (need access to the > fedora-messaging bus) > * Frontend service runs in Openshift > * Database runs in on Fedora Infrastructure database hosts. > > # Development Workflow > * I would like to have the possibility for these services to be > automatically deployed on our Openshift instances (stg and prod) using the > Github build trigger [2] available in Openshift. > An idea to achieve this is to use Openshift S2I (source-to-image) [3] > feature and use PyPi to install the application dependencies. In order to > control which dependencies we are using and pulling from PyPi we could use > devpi [4] as a caching proxy. This all sounds perfectly great to me... everything should be there for this with one exception: We really want to have some check in place for s2i so that it checks license, so we don't accidentally push out something thats not under a open source license. This doesn't need to be a blocker, but it would be great to get in place soon. kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: staging openshift reinstall
On 09/11/2018 09:26 AM, Aurelien Bompard wrote: >> I've been playing around with openshift staging for the last few weeks >> and enabling some cool features. :) > > Cool! I seem to remember that having persistent storage in our > Openshift instance was a difficult thing. I'm considering Openshift to > setup a PyPI caching proxy for us, and that will require some disk > space. Is this still an issue? Well, sorta yeah. We do have it to where we can use nfs volumes for the registery (because they required storage for that), but it's not really a good solution for everything as it's very manual. Could this caching proxy just use EmpyDir (ie, only for the life of that pod) and just refresh when it restarts? If it really needs disk, might be better to do on a vm at this point. Hopefully we can get to where our persistent storage story is better down the road... kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Moving forward with Fedora's PDC
Hi all, I would like to start working on the replacement application for PDC in Fedora. So far I have been mostly looking at the technical side of the application and how it will be setup in our infrastructure. Below is a list of the design decisions I am looking to take. I would really value some feedback on this in order to see if this is realistic or not. # Technical stack * Python 3 (just making it obvious) * Django 1.11 LTS (supported until at least April 2020) [0] * DRF 3.8.2 (latest version) Using Django and DRF (Django REST Framework) should allow us to reuse some of PDC code. # Application architecture * 1 backend service used to update the PDC database (currently pdc-updater [1]) * 1 frontend service serving the API HTTP endpoint and the HTML documentation of these endpoint. * 1 database # Deployment architecture * Backend service runs in Openshift (need access to the fedora-messaging bus) * Frontend service runs in Openshift * Database runs in on Fedora Infrastructure database hosts. # Development Workflow * I would like to have the possibility for these services to be automatically deployed on our Openshift instances (stg and prod) using the Github build trigger [2] available in Openshift. An idea to achieve this is to use Openshift S2I (source-to-image) [3] feature and use PyPi to install the application dependencies. In order to control which dependencies we are using and pulling from PyPi we could use devpi [4] as a caching proxy. Thanks, Clément [0] - https://www.djangoproject.com/download/ [1] - https://github.com/fedora-infra/pdc-updater [2] - https://docs.openshift.com/online/dev_guide/builds/triggering_builds.html#github-webhooks [3] - https://github.com/openshift/source-to-image [4] - https://devpi.net/docs/devpi/devpi/latest/%2Bd/index.html ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: staging openshift reinstall
> I've been playing around with openshift staging for the last few weeks > and enabling some cool features. :) Cool! I seem to remember that having persistent storage in our Openshift instance was a difficult thing. I'm considering Openshift to setup a PyPI caching proxy for us, and that will require some disk space. Is this still an issue? A. ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: suggested patch for review - issue 7158
Sorry this has taken so long - I've been fooling around trying to test
locally. I _believe_ I've followed Kevin's guidance. Without further ado;
- adding planet vars to people role;
diff --git a/playbooks/groups/people.yml b/playbooks/groups/people.yml
index e7661b4b4..1d00f77ca 100644
--- a/playbooks/groups/people.yml
+++ b/playbooks/groups/people.yml
@@ -68,7 +68,6 @@
- cgit/clean_lock_cron
- cgit/make_pkgs_list
- clamav
- - planet
- fedmsg/base
- git/server
@@ -79,6 +78,18 @@
SSLCertificateChainFile:
wildcard-2017.fedorapeople.org.intermediate.cert
- people
+
+ - role: planet
+ certbot: true
+ certbot_addhost: fedoraplanet.org
+ site_name: fedoraplanet.org
+ cert_name: wildcard-2018.fedoraplanet.org
+ server_aliases: planet.fedoraproject.org
+ server_admin: webmas...@fedoraproject.org
+ ssl: true
+ sslonly: false
+ SSLCertificateChainFile:
wildcard-2018.fedoraplanet.org.intermediate.cert
+ gzip: false
tasks:
- import_tasks: "{{ tasks_path }}/yumrepos.yml"
---
And planet redirect https -> http patch;
diff --git a/roles/planet/templates/planet.conf
b/roles/planet/templates/planet.conf
index 319923d2a..7e12b8f35 100644
--- a/roles/planet/templates/planet.conf
+++ b/roles/planet/templates/planet.conf
@@ -14,6 +14,11 @@
ErrorLog logs/planet-error.log
CustomLog logs/fedoraplanet.org-access.log common
+
+ # let certbot get an answer from certgetter01
+ RewriteEngine on
+ RewriteRule
^/\.well-known/(.*)/srv/web/acme-challenge/.well-known/$1 [L]
+ RewriteRule "^/?(.*)" "https://certgetter01/$1; [L,R=301,NE]
UserDir disable
AddCharset UTF-8 .xml
@@ -79,3 +84,32 @@
RedirectMatch permanent /(.*) http://fedoraplanet.org/$1
+
+ ##
+ # Domain: fedoraplanet.org
+ # Owner: ad...@fedoraplanet.org
+ #
+ ServerName fedoraplanet.org
+
+ SSLEngine on
+ SSLCertificateFile /etc/letsencrypt/live/fedoraplanet.org/cert.pem
+ SSLCertificateKeyFile
/etc/letsencrypt/live/fedoraplanet.org/privkey.pem
+ SSLCertificateChainFile
/etc/letsencrypt/live/fedoraplanet.org/fullchain.pem
+ SSLHonorCipherOrder On
+ SSLCipherSuite RC4-SHA:AES128-SHA:ALL:!ADH:!EXP:!LOW:!MD5:!SSLV2:!NULL
+ SSLProtocol ALL -SSLv2
+
+ ServerAdmin ad...@fedoraplanet.org
+ ServerName fedoraplanet.org
+
+ DocumentRoot "/srv/planet/site/"
+
+ ErrorLog logs/planet-error.log
+ CustomLog logs/planet.fedoraproject.org-access.log common
+
+ UserDir disable
+ AddCharset UTF-8 .xml
+
+ RedirectMatch permanent /(.*) http://fedoraplanet.org/$1
+
+
Regards!
On 9/3/18 7:48 PM, Kevin Fenzi wrote:
> On 08/29/2018 11:51 AM, Zach Villers wrote:
>> As discussed in infra meeting 16 aug around the 14:30 mark
>>
>> regarding Issue #7158: Planet Fedora doesn't have a valid certificate
>> .
>>
>> I created two patches (attached) based on my reading/understanding of
>> the certbot role README. Text below. I think we are in Freeze right now
>> and I probably have _many_ things to fix.
>>
>> Thanks to everyone that guided me (hopefully I'm on the right track :)
> Sorry for taking so long to look this over. ;(
>
> And thanks a bunch for working on it...
>
>>
>>
>> diff --git a/playbooks/include/proxies-websites.yml
>> b/playbooks/include/proxies-websites.yml
>> index 8013c539e..5cd82375c 100644
>> --- a/playbooks/include/proxies-websites.yml
>> +++ b/playbooks/include/proxies-websites.yml
>> @@ -932,3 +932,15 @@
>> tags:
>> - pkgs.fedoraproject.org
>> when: env == "staging" and "phx2" in inventory_hostname
>> +# cert for https://fedoraplanet.org which redirects to
>> http://fedoraplanet.org
>> +
>> + - role: httpd/website
>> + site_name: fedoraplanet.org
>> + server_aliases:
>> + - www.fedoraplanet.org
>> + ssl: true
>> + sslonly: true
>> + certbot: true
>> + certbot_addhost: fedoraplanet.org
>> + tags:
>> + - fedoraplanet.org
> So, this will work if we add this to our proxies, so we would need to
> change DNS to point there (currently fedoraplanet.org is pointing only
> to the people02 server, not the proxies), but that won't work as the
> content is still on fedoapeople.org. ;( So I think we should drop this
> part unless we just proxy everything from our proxies to people02, which
> could be slow.
>
> The problem is that our certbot/letsencrypt role is setup mostly for the
> proxies and not for people02, but we can still do it with a but more
> poking. :) So, look at roles/httpd/website/tasks/main.yml, and you will
> see:
>
> - name: Letsencrypt certificate stuff
> include_role: name=letsencrypt
> when: certbot == True
>
> we can call this in our 'people' role at the end...
> but we will also need to pass it all those variables that the
> httpd/website role already uses, ie,
