Re: Duplicates in package owner alias cronjob?

2017-08-09 Thread Ralph Bean 
Fixed in 
https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=24141e596b989de79c04111291aab09effe49580
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Duplicates in package owner alias cronjob?

2017-08-09 Thread Patrick Uiterwijk 
On Wed, Aug 9, 2017 at 7:32 PM, Patrick Uiterwijk  wrote:
> On Wed, Aug 9, 2017 at 7:29 PM, Pierre-Yves Chibon  
> wrote:
>> On Wed, Aug 09, 2017 at 05:24:27PM -, Patrick  マルタインアンドレアス  Uiterwijk 
>> wrote:
>>> > I'm confused about something.  Can any postfix experts help debug?
>>> >
>>> > On Friday I put a new package-owner-alias cronjob in place on bastion
>>> > (that Matt Prahl wrote).  It generates the package owner list from
>>> > pagure over dist-git instead of generating it from pkgdb.
>>> >
>>> >
>>> > https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24...
>>> >
>>> > Kevin saw that the hourly cronjob is spitting out some errors from the
>>> > postfix command.
>>> >
>>> > However, when I inspect the file, I don't find any duplicates:
>>> >
>>> > $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d
>>> >
>>> > Anybody know what's wrong with that file/output?
>>>
>>> The problem is not in the file/output. It's in the running of the script.
>>> If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns 
>>> an HTML page with "this app is offline".
>>>
>>> That is what's causing the failure:
>>> simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 
>>> 1)
>>> error creating owner-alias file
>>>
>>> Given that the script asks Pagure to page through a lot of projects 
>>> everytime, the chance of any one of those requests failing gets quite high.
>>> Thus, basically every run crashes some part through the run because a 
>>> Pagure result took too long or was aborted by haproxy.
>>
>> We're speaking about two different issues here:
>> 1/ pagure returning 503 a certain number of time in a row making the entire
>> script crash
>> 2/ these errors:
>> ```
>> /etc/cron.hourly/package-owner-aliases.sh:
>>
>> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
>> "perl-mogilefs-server-owner"
>> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
>> "simplyhtml-owner"
>> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
>> "packagekit-qt-owner"
>> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
>> "php-pimple-owner"
>> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
>> "vlgothic-fonts-owner"
>> ```
>>
>> I believe these are the ones Ralph was referring to, not the former.
>
> Oh. That's because they're in /etc/aliases, generated by fasClient.

Wrong again on my part. It was case sensitivity: for postfix,
MySQL-owner and mysql-owner are the same alias, since email addresses
are case insensitive.

>
>>
>>
>> Pierre
>> ___
>> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
>> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Duplicates in package owner alias cronjob?

2017-08-09 Thread Patrick Uiterwijk 
On Wed, Aug 9, 2017 at 7:29 PM, Pierre-Yves Chibon  wrote:
> On Wed, Aug 09, 2017 at 05:24:27PM -, Patrick  マルタインアンドレアス  Uiterwijk 
> wrote:
>> > I'm confused about something.  Can any postfix experts help debug?
>> >
>> > On Friday I put a new package-owner-alias cronjob in place on bastion
>> > (that Matt Prahl wrote).  It generates the package owner list from
>> > pagure over dist-git instead of generating it from pkgdb.
>> >
>> >
>> > https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24...
>> >
>> > Kevin saw that the hourly cronjob is spitting out some errors from the
>> > postfix command.
>> >
>> > However, when I inspect the file, I don't find any duplicates:
>> >
>> > $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d
>> >
>> > Anybody know what's wrong with that file/output?
>>
>> The problem is not in the file/output. It's in the running of the script.
>> If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns 
>> an HTML page with "this app is offline".
>>
>> That is what's causing the failure:
>> simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
>> error creating owner-alias file
>>
>> Given that the script asks Pagure to page through a lot of projects 
>> everytime, the chance of any one of those requests failing gets quite high.
>> Thus, basically every run crashes some part through the run because a Pagure 
>> result took too long or was aborted by haproxy.
>
> We're speaking about two different issues here:
> 1/ pagure returning 503 a certain number of time in a row making the entire
> script crash
> 2/ these errors:
> ```
> /etc/cron.hourly/package-owner-aliases.sh:
>
> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
> "perl-mogilefs-server-owner"
> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
> "simplyhtml-owner"
> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
> "packagekit-qt-owner"
> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
> "php-pimple-owner"
> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
> "vlgothic-fonts-owner"
> ```
>
> I believe these are the ones Ralph was referring to, not the former.

Oh. That's because they're in /etc/aliases, generated by fasClient.

>
>
> Pierre
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Duplicates in package owner alias cronjob?

2017-08-09 Thread Pierre-Yves Chibon 
On Wed, Aug 09, 2017 at 05:24:27PM -, Patrick  マルタインアンドレアス  Uiterwijk wrote:
> > I'm confused about something.  Can any postfix experts help debug?
> > 
> > On Friday I put a new package-owner-alias cronjob in place on bastion
> > (that Matt Prahl wrote).  It generates the package owner list from
> > pagure over dist-git instead of generating it from pkgdb.
> > 
> >
> > https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24...
> > 
> > Kevin saw that the hourly cronjob is spitting out some errors from the
> > postfix command.
> > 
> > However, when I inspect the file, I don't find any duplicates:
> > 
> > $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d
> > 
> > Anybody know what's wrong with that file/output?
> 
> The problem is not in the file/output. It's in the running of the script.
> If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns an 
> HTML page with "this app is offline".
> 
> That is what's causing the failure:
> simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
> error creating owner-alias file
> 
> Given that the script asks Pagure to page through a lot of projects 
> everytime, the chance of any one of those requests failing gets quite high.
> Thus, basically every run crashes some part through the run because a Pagure 
> result took too long or was aborted by haproxy.

We're speaking about two different issues here:
1/ pagure returning 503 a certain number of time in a row making the entire
script crash
2/ these errors:
```
/etc/cron.hourly/package-owner-aliases.sh:

postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
"perl-mogilefs-server-owner"
postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
"simplyhtml-owner"
postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
"packagekit-qt-owner"
postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
"php-pimple-owner"
postalias: warning: /etc/postfix/package-owner.db: duplicate entry: 
"vlgothic-fonts-owner"
```

I believe these are the ones Ralph was referring to, not the former.


Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Duplicates in package owner alias cronjob?

2017-08-09 Thread Patrick マルタインアンドレアス Uiterwijk 
> I'm confused about something.  Can any postfix experts help debug?
> 
> On Friday I put a new package-owner-alias cronjob in place on bastion
> (that Matt Prahl wrote).  It generates the package owner list from
> pagure over dist-git instead of generating it from pkgdb.
> 
>
> https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24...
> 
> Kevin saw that the hourly cronjob is spitting out some errors from the
> postfix command.
> 
> However, when I inspect the file, I don't find any duplicates:
> 
> $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d
> 
> Anybody know what's wrong with that file/output?

The problem is not in the file/output. It's in the running of the script.
If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns an 
HTML page with "this app is offline".

That is what's causing the failure:
simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
error creating owner-alias file

Given that the script asks Pagure to page through a lot of projects everytime, 
the chance of any one of those requests failing gets quite high.
Thus, basically every run crashes some part through the run because a Pagure 
result took too long or was aborted by haproxy.

> 
> -Ralph
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org