FBR: update the mailman & postorius codebase

2018-03-06 Thread Aurelien Bompard
Hey! :-) Since the beta freeze break may be longer, I'd like to make a bugfix update to the Mailman & Postorius (admin UI) RPMs. There's no ansible change, just a code update, it should all go smoothly. The outage ticket is: https://pagure.io/fedora-infrastructure/issue/6762 I plan on doing it

Re: Deploying Hubs to our Openshift instance

2018-02-18 Thread Aurelien Bompard
> Well, the idea was that we have a admin user that can change schema and > drop things and the like and the 'normal' user that the app runs with > that cannot do those things. That way if the application is compromised, > they can only do things the normal user could do, not dropping entire >

Re: Deploying Hubs to our Openshift instance

2018-02-16 Thread Aurelien Bompard
> >- What's the Ipsilon instance I should register with? I used to > register > >on iddev.fedorainfracloud.org but I guess that's no good for staging > > I'll let Patrick answer, but I believe this is going to be id.stg.fp.o for > which > the registration is different (ie: not

Re: Deploying Hubs to our Openshift instance

2018-02-16 Thread Aurelien Bompard
In the normal vm case, I have a couple questions: - What will my URL be? https://hubs.stg.fedoraproject.org I guess? - What's the Ipsilon instance I should register with? I used to register on iddev.fedorainfracloud.org but I guess that's no good for staging - I need the following passwords set

Re: Deploying Hubs to our Openshift instance

2018-02-15 Thread Aurelien Bompard
> Since deploying hubs is time sensitive, perhaps we should just initially > do a staging on a normal vm and look at openshift down the road? I would prefer the normal vm route for now. I'll look into deploying on openshift when we decide to open Hubs to a wider range of teams. Thanks! Aurélien

Deploying Hubs to our Openshift instance

2018-02-15 Thread Aurelien Bompard
Hey folks! I would like to try and do the Fedora Hubs deployment in our Openshift instance. The thing is, I have never deployed anything in Openshift, much less using our ansible playbooks. Do you know of a documentation I could read up on to understand what our `openshift/project`,

Re: Deploying JS-based application (Hubs)

2018-02-10 Thread Aurelien Bompard
> I think doing it with a macro would be sweet. Well, it wasn't so hard : https://abompard.fedorapeople.org/misc/nodejs-npm-rpm-macros/ $ rpm -qp --provides build/fedora-hubs-0.0.1-7.fc27.noarch.rpm | head Provides: bundled(nodejs-abab) = 1.0.4 bundled(nodejs-accepts) = 1.3.4

Re: Deploying JS-based application (Hubs)

2018-02-08 Thread Aurelien Bompard
> Basically you do this: > Provides: bundled(bootstrap) = 3.0.1 > This way it is possible to search the collection of RPMs for all that > contain versions of dependencies with known security issues. > That's neat, I could write a script that extracts the versions from the npm-shrinkwrap file.

Re: Deploying JS-based application (Hubs)

2018-02-08 Thread Aurelien Bompard
> I think this could work ok, but you might run into problems with updates > if the versions shift around... but I guess as long as you re > 'shrinkwrap' on updates it should work. That's the plan, re-run shrinkwrap after each change. > We don't have any way to track > security issues for all

Re: Deploying JS-based application (Hubs)

2018-02-05 Thread Aurelien Bompard
Hey, sorry for the silence on this subject, I was experimenting with deployment options. So far, here's what I've come up with. Npm has a command called "shrinkwrap" which will list all the installed dependencies and their versions recursively and dump that in a file. It's pretty much like "pip

Re: Python 3.6 in EPEL7

2017-12-06 Thread Aurelien Bompard
> Who is going to support it for CVEs and security issues? There's > python 3.6 in software collections, although I'm not sure they get > used in infra or not, at least it would have support for CVEs though. > That's a good question, I'd rather not do that support if possible (although I guess I

Python 3.6 in EPEL7

2017-12-05 Thread Aurelien Bompard
Hey folks! I have a need for Python 3.5+ in EPEL7 (newest versions of Mailman 3 require it), so I tried rebuilding the Python 3.6 RPM as a parallel-installable package by taking advantage of our %python3_other_* macros. There's a lot of packages that need to be built in the right order to get to

Re: Outage: HyperKitty database schema change - 2017-08-04 21:00 UTC

2017-07-28 Thread Aurelien Bompard
> > Seems ok to me. Unfortunate that it takes so long to run, but such is life. > Yeah it's basically rewriting the entire email and thread tables... I'll try to find a way to make it faster until next week, that would be a bonus. A. ___

Re: Outage: HyperKitty database schema change - 2017-08-04 21:00 UTC

2017-07-28 Thread Aurelien Bompard
If nobody has an issue with the timeslot I chose, I'm going to post this to devel-announce too. A. ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Outage: HyperKitty database schema change - 2017-08-04 21:00 UTC

2017-07-28 Thread Aurelien Bompard
There will be an outage starting at 2017-08-04 21:00 UTC, which will last approximately 15 hours. To convert UTC to your local time, take a look at https://fedoraproject.org/wiki/UTCHowto or run: date -d '2017-08-04 21:00 UTC' Reason for outage: important HyperKitty database schema change

Re: Subscription lists

2017-07-19 Thread Aurelien Bompard
FYI, I've reported this upstream: https://gitlab.com/mailman/postorius/issues/205 A. ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Big HyperKitty schema change

2017-06-09 Thread Aurelien Bompard
> > Are you thinking of something like the following (is it even possible?)? > [...] > I hadn't thought of this "2-step" process, I'm not sure it's possible since there are primary key type changes, but I'll think about it some more. Thanks. A. ___

Re: Big HyperKitty schema change

2017-06-09 Thread Aurelien Bompard
> > Yes, I agree. If this can't be put off until a couple of weeks after > GA, let's do it soon. > Thanks. Actually, it could be put off a couple weeks after GA. How many do you think would be best? A. ___ infrastructure mailing list --

Re: Retroactive FBR: Fix mailman3 "template" scheme

2017-06-02 Thread Aurelien Bompard
Hi! >> While I was debugging settings not saving for Mailman, I found out > >> that this was caused > >> by the schema of the "template" table being incorrect. > >> > >> It had the following gem: > >> password | timestamp without time zone > >> > >> I've just ran the following query to make it be

State of the Review Server (Fresque)

2017-05-02 Thread Aurelien Bompard
Hey people, Around 2 years ago, we (Pingou, Rahul, Vivek and me) started designing and writing some code for Fresque, a package review server for Fedora. The aim was to replace our current Bugzilla-based process. However, stuff happened in the meantime that were higher priority and Fresque

Re: Subscription lists

2017-04-18 Thread Aurelien Bompard
Hey Misha, > I can not re-enable or change sending of mailing list mail to myself. > > https://lists.fedoraproject.org/admin/accounts/subscriptions > > Internal Server Error I've managed to reproduce the error, it apparently happens when someone has a lot of subscriptions (say, more than 15)

Re: Update to the Mailman stack in prod

2017-02-01 Thread Aurelien Bompard
> Mailed the announcement out. Thanks for writing that up. ;) Thanks! > Oh, one other list thing: > https://pagure.io/fedora-infrastructure/issue/5478 > is waiting on a list rename before we do other changes. > Can we schedule that sometime after the outage? Yeah, that should be possible. I

Update to the Mailman stack in prod

2017-01-31 Thread Aurelien Bompard
Hey folks! I've been working on updating the Mailman production code. There are big schema changes that I tried to optimize and finally came up with a solution that will minimize unavailability. I think I need a 1 hour window of planned outage. I'm at UTC+1 so I could do it in my morning, while

Re: Importing prod data to staging

2017-01-17 Thread Aurelien Bompard
Done that, but now rbac-playbook says I'm not allowed to run : manual/staging-sync/mailman.yml Could someone add the ACL for me please? Thanks! Aurélien ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an

Re: Importing prod data to staging

2017-01-16 Thread Aurelien Bompard
> Make a staging sync playbook. ;) > Look at the ansible/playbooks/manual/staging-sync/koji.yml > Great, thanks! :-) A. ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to

Importing prod data to staging

2017-01-16 Thread Aurelien Bompard
Hey folks, I have a big mailman update that I'd like to test on staging, and I think that my testing would be more realisting if I had some production data on the staging DB. I mostly need public data, I don't need the user accounts or the private lists data for example. I realize that I could

EPEL-only packages

2016-12-13 Thread Aurelien Bompard
Hey people, I'm in a situation that I don't think is covered by the guidelines. Here's the thing: RHEL/CentOS ships some python packages in their Python 2 version, but not for Python 3 since RHEL/CentOS does not ship Python 3. Let's take the example of python-zope-interface. I need the python3

Triage of fedmsg issues

2016-11-02 Thread Aurelien Bompard
Hey folks, I'd like to do some triage of the fedmsg issues currently open on Github, then discuss with you what we should prioritize and decide on a roadmap. Triage, first. I haven't had a chance to get to know fedmsg inimately enough so I hope you'll correct me. Here is the list of open issues

Re: Mailman or Hyperkitty bug ?

2016-10-06 Thread Aurelien Bompard
> Can you file an issue on this and @abompard in it so he knows to take a > look? > FYI the issue is being handled here: https://pagure.io/fedora-infrastructure/issue/5513 Aurélien ___ infrastructure mailing list --

Re: Update on the mailing-lists

2016-09-19 Thread Aurelien Bompard
> Thanks for the heads-up, I was wondering, do you have some order of > magnitude of > how many accounts we're talking about? That's a little less than 2000 accounts (1840). > Could it be possible to check how many of these people have a FAS account > with > the same email? Hmm, I'm not sure

Update on the mailing-lists

2016-09-19 Thread Aurelien Bompard
Hey! Last Friday I deployed a new version of the Mailman / Postorius / HyperKitty stack on prod. There are a lot of improvements, but one of the most visible (and maybe the main reason for writing those changes) is the login system. Previously, we mainly relied on Mozilla Persona and FAS. As you

Re: Post-freeze code update

2016-08-22 Thread Aurelien Bompard
> Is that too far out? > This is fine. ;-) A. ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org

Post-freeze code update

2016-08-22 Thread Aurelien Bompard
Hey people! As you all know, Mozilla Persona is going down in the next few months. I've spent the last weeks since Flock updating the Mailman UI code to use a library that will allow us to support local accounts properly, on top of external services accounts. I'm currently testing it in staging,

Re: Mailman3 known issues

2016-06-27 Thread Aurelien Bompard
> ok. Have safe travels. ;) > Thanks! > Oh, while I am looking at them... how about I go over the outstanding > mailman3 bugs in trac we have? > > https://fedorahosted.org/fedora-infrastructure/ticket/1002 > "Enable syndication of fedora mailing lists" > > * This is an old ticket and we had

Mailman3 known issues

2016-06-23 Thread Aurelien Bompard
Hey team, Next week I'll be going offline for 3 weeks (two of them being really off the grid, I'm going to the European equivalent of the Burning Man festival...), so I'd like to check with you what errors I've seen our production Mailman3 instance do these last weeks. I think that the main

Re: Mailman 3 migration status

2016-05-18 Thread Aurelien Bompard
> > > My next step is to download the old HTML archives and make them > > > > available on mailman01 through Apache to preserve existing URLs in > > > > the wild. OK, I've migrated those files, setup the proper Apache directives, and tested it. As far as I know, we can remove the old servers

Re: Mailman 3 migration status

2016-05-14 Thread Aurelien Bompard
> I think thats fine. For some reason this change is pending on all > machines tho, need to sort out why it's not mailman/smtp-mm only. > Hmm, that's strange, I only changed 'roles/base/files/postfix/ master.cf/master.cf.mailman', so it should only affect hosts in the 'mailman' group. I also

Mailman 3 migration status

2016-05-13 Thread Aurelien Bompard
Hey there, Quick update on the Mailman3 migration. I have now migrated all lists to the new mailman01 server (including fedorahosted lists). I've also migrated the SpamAssassin configuration and database and it seems to be working fine. I had to make a small tweak to the postfix configuration

Re: Freeze break: increase memory on mailman01

2016-05-11 Thread Aurelien Bompard
> +1 for me as well, although figuring out which page is causing the issue > would > be nice (maybe something for minimot?) I believe it's the fulltext indexing process that causes these memory feasts. In my understanding, it eats memory until the swap is full and it's OOM-killed, and afterwards

Re: Last of mailing lists migration

2016-05-06 Thread Aurelien Bompard
Sorry for my late reply, yesterday was a national holiday in my country and I was offline. > I can post that blog post a few days before the migration? > Or just publish it now? I'd say publish it now, some people like to be warned way beforehand, especially if they must change their custom

Re: Mailing list settings

2016-04-06 Thread Aurelien Bompard
Hello. This is a known bug, it's been fixed already but the fix wasn't deployed. I just patched the production code, is it working now? Aurélien ___ infrastructure mailing list infrastructure@lists.fedoraproject.org

Re: Freeze break request: update Mailman3+Postorius+HyperKitty

2016-03-21 Thread Aurelien Bompard
> +1 if you send a quick note to the list how we could perform the downgrade :) Sure, that would be: # sudo -u mailman python34-alembic -c /usr/lib/python3.4/site-packages/mailman/config/alembic.cfg downgrade -1 While the new packages are still installed, and then downgrade the RPMs. A.

Re: Freeze break request: update Mailman3+Postorius+HyperKitty

2016-03-19 Thread Aurelien Bompard
> If we update them and need to back out would a downgrade work ok, or > are their database changes, etc? > There are database changes, but I have upgrade and downgrade migration scripts in the packages. Aurélien ___ infrastructure mailing list

Freeze break request: update Mailman3+Postorius+HyperKitty

2016-03-19 Thread Aurelien Bompard
Hi! There's been a lot of recent changes in the Mailman stack, including a new feature that will let us migrate more lists. I'd like to update the software stack on prod and keep it under observation for a few days (there are apparently performance issues for some people). This freeze break does

Re: Additional account with read DB access

2016-02-05 Thread Aurelien Bompard
> Could we generate UUIDs for the imported mail? That would be incredibly > valuable in my analysis _anyway_. > Hmm yes I could create the users in Mailman3 even if they never sent an email in the new system. But I may have a better idea, here's what I could do: - get the old emails from

Re: Additional account with read DB access

2016-02-04 Thread Aurelien Bompard
> > How sensible is the data stored in the DB? Too much to be made public? > > Yeah, I wondered the same. Could we sanatize any user auth/account data > and just make the raw posts of all the public lists available? > Hmm, There's the issue of people's email addresses, it could become a gold mine

Additional account with read DB access

2016-02-03 Thread Aurelien Bompard
Hey folks, Matt would like read access on HyperKitty's production database to collect some statistics[1], and I wondered if we have a procedure for this kind of request. [1] https://fedorahosted.org/fedora-infrastructure/ticket/5070 Should it be done via a script that would be stored in our

Re: hyperkitty thinks t...@fedorahosted.org is "fedora-badges" ... or is it "FESCo Trac"?

2015-12-07 Thread Aurelien Bompard
> Right, all trac's on fedorahosted send email as: > > "Trac name" t...@fedorahosted.org > > and I think the name in "s has confused hyperkitty. > Very likely, it's probably using the last one it's seen as the display name. Since the email address is the key, I'm not sure if we can do much about

Re: Broken links?

2015-12-02 Thread Aurelien Bompard
> On Tue, Dec 01, 2015 at 09:41:59AM -0500, Patrick Uiterwijk wrote: > > We made these 404's when we found out that they were redirecting to > > the wrong posts. Once all lists are moved to mailman, we will be > > putting the old archives back, restoring all original links to their > > original

Re: Mailman 3 archiver eating whitespace (was: Re: Migration to HyperKitty / Mailman 3)

2015-11-18 Thread Aurelien Bompard
Hey Martin, > Our patch review list (anaconda-patches) has been migrated, but it > looks like the new Mailman 3 archiver is eating leading whitespace, as > can be seen for example here: > https://lists.fedorahosted.org/archives/list/anaconda-patches%40lists.f >

Re: Migration to HyperKitty / Mailman 3

2015-11-10 Thread Aurelien Bompard
> Let's instead give really simple instructions for adding it. And make > sure it happens on at least the big lists. > > Can we add it in cases where the footer isn't customized? I'll check that. OK I've sent the emails to the lists, they are obviously held for moderation, could someone with the

Re: Migration to HyperKitty / Mailman 3

2015-11-09 Thread Aurelien Bompard
> I know it's hard to estimate times, but perhaps we could look at > migrating all of lists.fedorahosted.org (aside the ones that use > filters and topics) first on the 16th, then start on the > lists.fedoraproject.org ones after that? Or keeping a status on the > wiki might be nice so people know

Migration to HyperKitty / Mailman 3

2015-11-06 Thread Aurelien Bompard
Hi people! Since it's Friday evening, I just unilateraly decided to migrate all the lists to Mailman3 without telling you anything in advance... Nah just kidding :-) However, I think the best time to migrate the rest of the lists would be the week before Thanksgiving. That would give me some

Re: Migration of lists.fedorahosted.org failed

2015-08-25 Thread Aurelien Bompard
Once you can implement upstream, do you plan to try again with a patched version while we await an upstream point-release? Yes, that's my plan. My RPM already carries a few patches that are not released, and even still under review. I update them (and eventually remove them :-) ) as they get

Migration of lists.fedorahosted.org failed

2015-08-21 Thread Aurelien Bompard
Hey folks, I'm sorry to announce that I had to rollback the migration of lists.fedorahosted.org to Mailman 3. There's a missing feature again, that is heavily used by two lists on this server, it's the header_filter_rules. It lets the admin decide on header regexes that will set a different

Re: Move to mailman3

2015-08-12 Thread Aurelien Bompard
Hey guys, I'm currently demoingo Hyperkitty with this list ! Say Hi! A.

Re: Move to mailman3

2015-08-07 Thread Aurelien Bompard
Did you use an old subscription list? I have not been subscribed to @infra for about a month and today I start receiving these from you guys. Ah, yes, sorry, I did get an updated subscription list but it only added new entries. Performing the unsubscribe process again does not send me a

Re: Move to mailman3

2015-08-07 Thread Aurelien Bompard
Might be an idea to send one when you start the migration as well :] Just that we are not surprised or that we watch if one of our email doesn't land on the list. Yeah, that would have been a good idea, but it's too late: your message was processed by Mailman3 already ;-)

Re: Move to mailman3

2015-08-07 Thread Aurelien Bompard
Performing the unsubscribe process again does not send me a confirmation e-mail. I looks like you're still subscribed, which unsubscribe process did you follow? Thanks and sorry for the inconvenience. Aurélien

Re: Move to mailman3

2015-08-07 Thread Aurelien Bompard
Ooh, can we switch from gravatar to libravatar? Will do, thanks :-) A.

Test list available on Mailman3/Hyperkitty

2014-03-17 Thread Aurelien Bompard
Okay, there's a list available for testing on the new Mailman3/HyperKitty server : test-mailm...@lists.fedoraproject.org You can subscribe to it via the admin interface (called Postorius): https://lists.fedoraproject.org/admin/lists/test-mailm...@lists.fedoraproject.org/ Please report any bug you

<    1   2