Is pkgs stage available?

[Chenxiong Qi]

Hi,

src.stg.fedoraproject.org is redirected to src.fedoraproject.org. Is 
this expected? Is there a pkgs stage running?


--
Regards,
Chenxiong Qi
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: src.fedoraproject.org vs pkgs.fedoraproject.org and TLS

[Kevin Fenzi]

On Tue, 13 Dec 2016 17:24:03 -0500
Colin Walters  wrote:

> Did we lose TLS-authenticated access to the pkg git?

Nope. It just changed. 
pkgs.fedoraproject.org now redirects http/https to
src.fedoraproject.org which is behind our proxies and uses a well known
cert. 

> I see on the cgit webpage:
> https://src.fedoraproject.org/cgit/rpms/golang-googlecode-go-crypto.git/
> It only offers anonymous transports without integrity (http://,
> git://).

We missed fixing this when we made changes sunday night. 
Oops. Thanks for pointing it out. 

I have now done so, and it should only offer https://
 
> Specifically for the CentOS Atomic Host SIG builds we
> go out of our way to use ca-pinning[1]:
> 
> https://github.com/CentOS/sig-atomic-buildscripts/blob/master/overlay.yml#L13
> 
> However, this broke, and I am not immediately working out
> the apparent cyclical redirects between src.fp.org and pkgs.fp.org.
> 
> Trying e.g.:
> 
> $ curl -L -v -k
> https://pkgs.fedoraproject.org/git/rpms/golang-googlecode-go-crypto/
> < HTTP/1.1 302 Found < Location:
> https://src.fedoraproject.org/git/rpms/golang-googlecode-go-crypto/ <
> HTTP/1.1 404 Not Found
> 
> [1] Because I think CA pinning + GPG signatures on upstream source
>   is stronger and better than having humans manually upload
> tarballs 

pkgs redirects http/https to src.fedoraproject.org. 

You should use https://src.fedoraproject.org/ and it's well known cert
now. (It's our digicert wildcard cert)

If you see anything else broken, please do let us know... 

kevin


pgpVvui5nYY9q.pgp
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Cert penning, Certs and related

[Kevin Fenzi]

FYI, I marked this thread to reply to, but I simply have not had time
lately with last week on site at the datacenter and this weekend
prepping for the flag day and this week helping people with fallout
from the flag day. 

I'll try and get back to this this week, but please have some patience. 

kevin


pgpB5LPuA8jLa.pgp
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

src.fedoraproject.org vs pkgs.fedoraproject.org and TLS

[Colin Walters]

Did we lose TLS-authenticated access to the pkg git?

I see on the cgit webpage:
https://src.fedoraproject.org/cgit/rpms/golang-googlecode-go-crypto.git/
It only offers anonymous transports without integrity (http://, git://).

Specifically for the CentOS Atomic Host SIG builds we
go out of our way to use ca-pinning[1]:

https://github.com/CentOS/sig-atomic-buildscripts/blob/master/overlay.yml#L13

However, this broke, and I am not immediately working out
the apparent cyclical redirects between src.fp.org and pkgs.fp.org.

Trying e.g.:

$ curl -L -v -k  
https://pkgs.fedoraproject.org/git/rpms/golang-googlecode-go-crypto/
< HTTP/1.1 302 Found
< Location: https://src.fedoraproject.org/git/rpms/golang-googlecode-go-crypto/
< HTTP/1.1 404 Not Found

[1] Because I think CA pinning + GPG signatures on upstream source
  is stronger and better than having humans manually upload tarballs
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Cert penning, Certs and related

[Colin Walters]

On Tue, Dec 13, 2016, at 01:49 PM, Stephen John Smoogen wrote:

> So the parts I think I am seeing different answers are:
> 1. What are we trying to accomplish and where?
> 2. What infrastructure is needed to accomplish this?

I think this stuff is pretty well covered in the thread and should
be hashed out by the people who are going to do the work.

> 3. When does this need to be done?

I don't quite understand how this process stretched to 6
months already...but soon, please - I'd like Fedora to 
be a reference architecture.  If it takes another
few months something is clearly going wrong.

> 4. What is the budget cost for setting up these certificates,
> reporting systems, pinned ips etc
> 5. Who is paying for this (hardware, certificates, domain names, etc).

Fedora infra.

> 6. Who is doing the implementation work and where?

I can take point on making and testing the client side changes.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Cert penning, Certs and related

[Stephen John Smoogen]

On 13 December 2016 at 12:37, Colin Walters  wrote:
>
>
> On Fri, Dec 9, 2016, at 05:38 PM, Stephen John Smoogen wrote:
>
>> I don't think anyone is understanding each other.. because that isn't
>> what I was getting from this thread until now.
>
> The thread has been 95% just me and Kevin on and off over the last 6
> months.  I asked him for clarification.  Now, aAre you going to help?
> If so, can you elaborate on what you are/aren't understanding?

So the parts I think I am seeing different answers are:
1. What are we trying to accomplish and where?
2. What infrastructure is needed to accomplish this?
3. When does this need to be done?
4. What is the budget cost for setting up these certificates,
reporting systems, pinned ips etc
5. Who is paying for this (hardware, certificates, domain names, etc).
6. Who is doing the implementation work and where?

To me Kevin, Patrick and you have said similar things but that you
each have different ideas about what they mean and so aren't actually
communicating.

> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Future of Koschei staging

[Kevin Fenzi]

On Tue, 13 Dec 2016 13:45:17 +0100
Mikolaj Izdebski  wrote:

> Hello,
> 
> So far we've been using staging Koschei for two different things:
> pre-production deployment testing and to aid development by testing
> new upstream features and bugfixes (by deploying snapshots).
> 
> After recent introduction of replication to PostgreSQL databases, we
> can no longer run some of database migrations without sysadmin-main
> assistance. Moreover, staning-sync playbook is broken as it worked by
> dropping and re-creating koschei database. (Note that Koschei *must*
> be synced after Koji sync, or it will be broken.)

I don't think this is true. I think we can get it working with
replication. ;) Or at least it's worth trying some more... 

> I can see two alternative solutions to this problem:
> 
> Option 1: Switch to "dev-stg-prod" model that some other apps are
> using. By that I mean creating a separate development environment in
> cloud, with separate database (and possibly, even separate Koji).
> Staging would be used only for pre-production deployment testing. Dev
> instance could be created on-demand, only when actually needed, and
> terminated afterwards.
> 
> Option 2: Use separate db for Koschei staging (possibly on one of
> existing Koschei hosts) without replication enabled.
> 
> What should be the preferred course of action?

Option 3: Try and get things working with replication. ;) 

I just replied to your ticket about this: 
https://pagure.io/fedora-infrastructure/issue/5584#comment-46177

can you try that and let us know if it works?

Ideally I would like to get some standard manual playbooks to do these
things on the replicated db's. 

kevin


pgp0GDzM_3wjW.pgp
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Cert penning, Certs and related

[Colin Walters]

On Fri, Dec 9, 2016, at 05:38 PM, Stephen John Smoogen wrote:

> I don't think anyone is understanding each other.. because that isn't
> what I was getting from this thread until now.

The thread has been 95% just me and Kevin on and off over the last 6
months.  I asked him for clarification.  Now, aAre you going to help? 
If so, can you elaborate on what you are/aren't understanding?
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Future of Koschei staging

[Pierre-Yves Chibon]

On Tue, Dec 13, 2016 at 01:45:17PM +0100, Mikolaj Izdebski wrote:
> Hello,
> 
> So far we've been using staging Koschei for two different things:
> pre-production deployment testing and to aid development by testing new
> upstream features and bugfixes (by deploying snapshots).
> 
> After recent introduction of replication to PostgreSQL databases, we can
> no longer run some of database migrations without sysadmin-main
> assistance. Moreover, staning-sync playbook is broken as it worked by
> dropping and re-creating koschei database. (Note that Koschei *must* be
> synced after Koji sync, or it will be broken.)
> 
> I can see two alternative solutions to this problem:
> 
> Option 1: Switch to "dev-stg-prod" model that some other apps are using.
> By that I mean creating a separate development environment in cloud,
> with separate database (and possibly, even separate Koji). Staging would
> be used only for pre-production deployment testing. Dev instance could
> be created on-demand, only when actually needed, and terminated afterwards.
> 
> Option 2: Use separate db for Koschei staging (possibly on one of
> existing Koschei hosts) without replication enabled.
> 
> What should be the preferred course of action?

You are not saying which one you would prefer :)


Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Future of Koschei staging

[Mikolaj Izdebski]

Hello,

So far we've been using staging Koschei for two different things:
pre-production deployment testing and to aid development by testing new
upstream features and bugfixes (by deploying snapshots).

After recent introduction of replication to PostgreSQL databases, we can
no longer run some of database migrations without sysadmin-main
assistance. Moreover, staning-sync playbook is broken as it worked by
dropping and re-creating koschei database. (Note that Koschei *must* be
synced after Koji sync, or it will be broken.)

I can see two alternative solutions to this problem:

Option 1: Switch to "dev-stg-prod" model that some other apps are using.
By that I mean creating a separate development environment in cloud,
with separate database (and possibly, even separate Koji). Staging would
be used only for pre-production deployment testing. Dev instance could
be created on-demand, only when actually needed, and terminated afterwards.

Option 2: Use separate db for Koschei staging (possibly on one of
existing Koschei hosts) without replication enabled.

What should be the preferred course of action?

-- 
Mikolaj Izdebski
IRC: mizdebsk
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: EPEL-only packages

[Pierre-Yves Chibon]

On Tue, Dec 13, 2016 at 11:56:28AM +0100, Aurelien Bompard wrote:
>Hey people,
> 
>I'm in a situation that I don't think is covered by the guidelines. Here's
>the thing: RHEL/CentOS ships some python packages in their Python 2
>version, but not for Python 3 since RHEL/CentOS does not ship Python 3.
> 
>Let's take the example of python-zope-interface. I need the python3
>version, so I thought I'd adapt the existing python-zope-interface package
>we have to only build the Python 3 version in EPEL. Then Kevin told me
>that this was a bad idea because koji (I think it was koji?) works on
>source packages, and won't like having two python-zope-interface packages,
>one from RHEL/CentOS and one from EPEL.
>So I built a python3-zope-interface package that only builds the Python3
>version. It got accepted, it builds fine, but now I realize that I don't
>want this package to be built for Rawhide, since Rawhide already has the
>Python 3 version of Zope Interface (built from the main package, as
>usual).
>I removed the spec file from the master branch in git and put a README
>explaining all that, but I wonder if our release engineering process will
>like that. Do I need a special exception, as we have for orphan packages,
>so it does not error out?

No that should be no problem, you can basically just retire the master branch
and keep only the epel one. We have this for other packages, such as: 
https://admin.fedoraproject.org/pkgdb/package/rpms/python-sqlalchemy0.8/
or 
https://admin.fedoraproject.org/pkgdb/package/rpms/python-sqlalchemy0.7/
back when we needed them in infra on rhel6.


Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

EPEL-only packages

[Aurelien Bompard]

Hey people,

I'm in a situation that I don't think is covered by the guidelines. Here's
the thing: RHEL/CentOS ships some python packages in their Python 2
version, but not for Python 3 since RHEL/CentOS does not ship Python 3.

Let's take the example of python-zope-interface. I need the python3
version, so I thought I'd adapt the existing python-zope-interface package
we have to only build the Python 3 version in EPEL. Then Kevin told me that
this was a bad idea because koji (I think it was koji?) works on source
packages, and won't like having two python-zope-interface packages, one
from RHEL/CentOS and one from EPEL.

So I built a python3-zope-interface package that only builds the Python3
version. It got accepted, it builds fine, but now I realize that I don't
want this package to be built for Rawhide, since Rawhide already has the
Python 3 version of Zope Interface (built from the main package, as usual).
I removed the spec file from the master branch in git and put a README
explaining all that, but I wonder if our release engineering process will
like that. Do I need a special exception, as we have for orphan packages,
so it does not error out?

I don't know if I have been clear, if not please ask me to rephrase. Thanks
for your insight!

Aurélien
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org