Re: Duplicates in package owner alias cronjob?
Introducing some retry stuff to be more resilient when pagure blips: https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=b084a7762f00f110f2b73158a66b577ec01a9cfd ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Duplicates in package owner alias cronjob?
Fixed in https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=24141e596b989de79c04111291aab09effe49580 ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Duplicates in package owner alias cronjob?
On Wed, Aug 9, 2017 at 7:32 PM, Patrick Uiterwijk wrote: > On Wed, Aug 9, 2017 at 7:29 PM, Pierre-Yves Chibon > wrote: >> On Wed, Aug 09, 2017 at 05:24:27PM -, Patrick マルタインアンドレアス Uiterwijk >> wrote: >>> > I'm confused about something. Can any postfix experts help debug? >>> > >>> > On Friday I put a new package-owner-alias cronjob in place on bastion >>> > (that Matt Prahl wrote). It generates the package owner list from >>> > pagure over dist-git instead of generating it from pkgdb. >>> > >>> > >>> > https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24... >>> > >>> > Kevin saw that the hourly cronjob is spitting out some errors from the >>> > postfix command. >>> > >>> > However, when I inspect the file, I don't find any duplicates: >>> > >>> > $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d >>> > >>> > Anybody know what's wrong with that file/output? >>> >>> The problem is not in the file/output. It's in the running of the script. >>> If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns >>> an HTML page with "this app is offline". >>> >>> That is what's causing the failure: >>> simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char >>> 1) >>> error creating owner-alias file >>> >>> Given that the script asks Pagure to page through a lot of projects >>> everytime, the chance of any one of those requests failing gets quite high. >>> Thus, basically every run crashes some part through the run because a >>> Pagure result took too long or was aborted by haproxy. >> >> We're speaking about two different issues here: >> 1/ pagure returning 503 a certain number of time in a row making the entire >> script crash >> 2/ these errors: >> ``` >> /etc/cron.hourly/package-owner-aliases.sh: >> >> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: >> "perl-mogilefs-server-owner" >> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: >> "simplyhtml-owner" >> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: >> "packagekit-qt-owner" >> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: >> "php-pimple-owner" >> postalias: warning: /etc/postfix/package-owner.db: duplicate entry: >> "vlgothic-fonts-owner" >> ``` >> >> I believe these are the ones Ralph was referring to, not the former. > > Oh. That's because they're in /etc/aliases, generated by fasClient. Wrong again on my part. It was case sensitivity: for postfix, MySQL-owner and mysql-owner are the same alias, since email addresses are case insensitive. > >> >> >> Pierre >> ___ >> infrastructure mailing list -- infrastructure@lists.fedoraproject.org >> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Duplicates in package owner alias cronjob?
On Wed, Aug 9, 2017 at 7:29 PM, Pierre-Yves Chibon wrote: > On Wed, Aug 09, 2017 at 05:24:27PM -, Patrick マルタインアンドレアス Uiterwijk > wrote: >> > I'm confused about something. Can any postfix experts help debug? >> > >> > On Friday I put a new package-owner-alias cronjob in place on bastion >> > (that Matt Prahl wrote). It generates the package owner list from >> > pagure over dist-git instead of generating it from pkgdb. >> > >> > >> > https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24... >> > >> > Kevin saw that the hourly cronjob is spitting out some errors from the >> > postfix command. >> > >> > However, when I inspect the file, I don't find any duplicates: >> > >> > $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d >> > >> > Anybody know what's wrong with that file/output? >> >> The problem is not in the file/output. It's in the running of the script. >> If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns >> an HTML page with "this app is offline". >> >> That is what's causing the failure: >> simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 1) >> error creating owner-alias file >> >> Given that the script asks Pagure to page through a lot of projects >> everytime, the chance of any one of those requests failing gets quite high. >> Thus, basically every run crashes some part through the run because a Pagure >> result took too long or was aborted by haproxy. > > We're speaking about two different issues here: > 1/ pagure returning 503 a certain number of time in a row making the entire > script crash > 2/ these errors: > ``` > /etc/cron.hourly/package-owner-aliases.sh: > > postalias: warning: /etc/postfix/package-owner.db: duplicate entry: > "perl-mogilefs-server-owner" > postalias: warning: /etc/postfix/package-owner.db: duplicate entry: > "simplyhtml-owner" > postalias: warning: /etc/postfix/package-owner.db: duplicate entry: > "packagekit-qt-owner" > postalias: warning: /etc/postfix/package-owner.db: duplicate entry: > "php-pimple-owner" > postalias: warning: /etc/postfix/package-owner.db: duplicate entry: > "vlgothic-fonts-owner" > ``` > > I believe these are the ones Ralph was referring to, not the former. Oh. That's because they're in /etc/aliases, generated by fasClient. > > > Pierre > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Duplicates in package owner alias cronjob?
On Wed, Aug 09, 2017 at 05:24:27PM -, Patrick マルタインアンドレアス Uiterwijk wrote: > > I'm confused about something. Can any postfix experts help debug? > > > > On Friday I put a new package-owner-alias cronjob in place on bastion > > (that Matt Prahl wrote). It generates the package owner list from > > pagure over dist-git instead of generating it from pkgdb. > > > > > > https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24... > > > > Kevin saw that the hourly cronjob is spitting out some errors from the > > postfix command. > > > > However, when I inspect the file, I don't find any duplicates: > > > > $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d > > > > Anybody know what's wrong with that file/output? > > The problem is not in the file/output. It's in the running of the script. > If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns an > HTML page with "this app is offline". > > That is what's causing the failure: > simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 1) > error creating owner-alias file > > Given that the script asks Pagure to page through a lot of projects > everytime, the chance of any one of those requests failing gets quite high. > Thus, basically every run crashes some part through the run because a Pagure > result took too long or was aborted by haproxy. We're speaking about two different issues here: 1/ pagure returning 503 a certain number of time in a row making the entire script crash 2/ these errors: ``` /etc/cron.hourly/package-owner-aliases.sh: postalias: warning: /etc/postfix/package-owner.db: duplicate entry: "perl-mogilefs-server-owner" postalias: warning: /etc/postfix/package-owner.db: duplicate entry: "simplyhtml-owner" postalias: warning: /etc/postfix/package-owner.db: duplicate entry: "packagekit-qt-owner" postalias: warning: /etc/postfix/package-owner.db: duplicate entry: "php-pimple-owner" postalias: warning: /etc/postfix/package-owner.db: duplicate entry: "vlgothic-fonts-owner" ``` I believe these are the ones Ralph was referring to, not the former. Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Duplicates in package owner alias cronjob?
> I'm confused about something. Can any postfix experts help debug? > > On Friday I put a new package-owner-alias cronjob in place on bastion > (that Matt Prahl wrote). It generates the package owner list from > pagure over dist-git instead of generating it from pkgdb. > > > https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb24... > > Kevin saw that the hourly cronjob is spitting out some errors from the > postfix command. > > However, when I inspect the file, I don't find any duplicates: > > $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d > > Anybody know what's wrong with that file/output? The problem is not in the file/output. It's in the running of the script. If Pagure on Dist-Git takes too long to reply, or is down, haproxy returns an HTML page with "this app is offline". That is what's causing the failure: simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 1) error creating owner-alias file Given that the script asks Pagure to page through a lot of projects everytime, the chance of any one of those requests failing gets quite high. Thus, basically every run crashes some part through the run because a Pagure result took too long or was aborted by haproxy. > > -Ralph ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Duplicates in package owner alias cronjob?
I'm confused about something. Can any postfix experts help debug? On Friday I put a new package-owner-alias cronjob in place on bastion (that Matt Prahl wrote). It generates the package owner list from pagure over dist-git instead of generating it from pkgdb. https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=eb241f903dcd6d7d3529539739508f6dc1b00982 Kevin saw that the hourly cronjob is spitting out some errors from the postfix command. However, when I inspect the file, I don't find any duplicates: $ cat /etc/postfix/package-owner | awk ' { print $1 } ' | uniq -d Anybody know what's wrong with that file/output? -Ralph signature.asc Description: PGP signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org