[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
There's a question from Robie on the bug which is why this hasn't been accepted by any of the SRU Team. Marking incomplete. ** Changed in: percona-toolkit (Ubuntu Xenial) Status: In Progress => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
> To that end I submit that it will be difficult to test this patch in a reliable fashion, and would seek feedback on what we should do about that. Thank you for your honest appraisal. What's the impact to users if we don't fix this, or only fix the tools that you were able to test (is that possible?)? I think it's important to consider how this weighs up against regression risk. Xenial was released over three years ago now, and new deployments are likely to be on Bionic. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
Uploaded to Xenial. ** Changed in: percona-toolkit (Ubuntu Xenial) Status: Confirmed => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
** Changed in: percona-toolkit (Ubuntu) Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
The attachment "lp1824762-percona-toolkit-sprintf-warnings.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team. [This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.] ** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
Made some progress on the test suite, but many tests already fail in the relevant upstream branch: sudo apt-get install percona-server-server-5.6 lathiat@xenial:~/percona-toolkit$ PERCONA_TOOLKIT_BRANCH=$HOME/percona- toolkit PERCONA_TOOLKIT_SANDBOX=/usr ./sandbox/test-env start (edit Makefile, change '*.t' to '**/*.t') lathiat@xenial:~/percona-toolkit$ PERCONA_TOOLKIT_BRANCH=$HOME/percona- toolkit PERCONA_TOOLKIT_SANDBOX=/usr make test ** Description changed: [Impact] - On Xenial, various tools from percona-toolkit fail because of new - warnings from sprintf that are generated by the new Perl version in - Xenial. + On Xenial (Bionic and later are not affected due to a newer version + already incorporating this patch), various tools from percona-toolkit + fail because of new warnings from sprintf that are generated by the new + Perl version in Xenial. These warnings are fatal because all of the tools set "use strict" and "use warnings FATAL => 'all';" This can be tested simply by parsing any MySQL log with pt-query-digest, however, it affects many other tools. [Test Case] apt-get install mysql-server percona-toolkit # set any root password you like, we'll just use the debian.cnf credentials mysql --defaults-file=/etc/mysql/debian.cnf -e "set GLOBAL slow_query_log = on, long_query_time=0.0; select * from mysql.user;" pt-query-digest /var/lib/mysql/$(hostname)-slow.log Result: The pipeline caused an error: Pipeline process 5 (iteration) caused an error: Redundant argument in sprintf at /usr/bin/pt-query-digest line 2556. Terminating pipeline because process 4 (iteration) caused too many errors. This issue was fixed upstream: https://github.com/percona/percona-toolkit/pull/73/ - - [Regression Potential] + [Regression Potential] The following tools are updated by this patch: pt-diskstats pt-fk-error-logger pt-heartbeat pt-index-usage pt-kill pt-online-schema-change pt-query-digest pt-slave-delay pt-slave-find pt-table-checksum pt-table-sync pt-table-usage pt-upgrade Generally speaking most of these tools fail to work at all due to this bug, which will limit the regression potential to a reasonable extent. This patch is taken from upstream and largely updates common code used by all tools in a generic fashion. However it is difficult to fully test all of the tools, as the test suite is not included in the released binary. I attempted to run the upstream test suite, however there are no instructions on how to do so and I failed in figuring out how to get it to successfully execute. To that end I submit that it will be difficult to test this patch in a reliable fashion, and would seek feedback on what we should do about that. [Other Info] - - The way the upstream percona-toolkit repository works is they have these perl modules that are statically "compiled in" to the various tool perl scripts, with both the source libraries and that resulting static tool committed to git. + + The way the upstream percona-toolkit repository works is they have these + perl modules that are statically "compiled in" to the various tool perl + scripts, with both the source libraries and that resulting static tool + committed to git. They do not always update every tool when changes are made to those modules. So if you rebuild every tool, even from the same source as the release, you will get unrelated changes to the actual fix. The source of the package (as released) does not contain the libraries (lib/**) or the tests (t/**) so these are not included in the package patch, in contrast to the upstream git diff. ** Tags added: sts -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
** Description changed: - On Xenial, various tools fail because of warnings from sprintf that are - generated on the new Perl version in Xenial. + [Impact] + + On Xenial, various tools from percona-toolkit fail because of new + warnings from sprintf that are generated by the new Perl version in + Xenial. These warnings are fatal because all of the tools set "use strict" and "use warnings FATAL => 'all';" - This can be tested simply by parsing any MySQL log with pt-query-digest + This can be tested simply by parsing any MySQL log with pt-query-digest, + however, it affects many other tools. - * Test Case * + [Test Case] apt-get install mysql-server percona-toolkit # set any root password you like, we'll just use the debian.cnf credentials - mysql --defaults-file=/etc/mysql/debian.cnf -e "set GLOBAL slow_query_log = on, + mysql --defaults-file=/etc/mysql/debian.cnf -e "set GLOBAL slow_query_log = on, long_query_time=0.0; select * from mysql.user;" pt-query-digest /var/lib/mysql/$(hostname)-slow.log Result: The pipeline caused an error: Pipeline process 5 (iteration) caused an error: Redundant argument in sprintf at /usr/bin/pt-query-digest line 2556. Terminating pipeline because process 4 (iteration) caused too many errors. - This issue was fixed upstream: https://github.com/percona/percona-toolkit/pull/73/ + + + [Regression Potential] + + The following tools are updated by this patch: + pt-diskstats + pt-fk-error-logger + pt-heartbeat + pt-index-usage + pt-kill + pt-online-schema-change + pt-query-digest + pt-slave-delay + pt-slave-find + pt-table-checksum + pt-table-sync + pt-table-usage + pt-upgrade + + Generally speaking most of these tools fail to work at all due to this + bug, which will limit the regression potential to a reasonable extent. + + This patch is taken from upstream and largely updates common code used + by all tools in a generic fashion. + + However it is difficult to fully test all of the tools, as the test + suite is not included in the released binary. I attempted to run the + upstream test suite, however there are no instructions on how to do so + and I failed in figuring out how to get it to successfully execute. + + To that end I submit that it will be difficult to test this patch in a + reliable fashion, and would seek feedback on what we should do about + that. + + [Other Info] + + The way the upstream percona-toolkit repository works is they have these perl modules that are statically "compiled in" to the various tool perl scripts, with both the source libraries and that resulting static tool committed to git. + + They do not always update every tool when changes are made to those + modules. So if you rebuild every tool, even from the same source as the + release, you will get unrelated changes to the actual fix. + + The source of the package (as released) does not contain the libraries + (lib/**) or the tests (t/**) so these are not included in the package + patch, in contrast to the upstream git diff. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824762] Re: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings
Attaching patch for the issue. The way the upstream percona-toolkit repository works is they have these perl modules that are statically "compiled in" to the various tool perl scripts, with both the source libraries and that resulting static tool committed to git. They do not always update every tool when changes are made to those modules. So if you rebuild every tool, even from the same source as the release, you will get unrelated changes to the actual fix. The source of the package (as released) does not contain the libraries (lib/**) or the tests (t/**) so these are not included in the package patch, in contrast to the upstream git diff. I fixed all occurrences of the problem in all tools, but only tested pt- heartbeat and pt-query-digest. It is difficult to accurately test all of the tools, however, pretty much all of the tools were completely broken in basic functionality by the lack of this fix, and the change is an exact copy of the upstream fix. The changed tools are: pt-diskstats pt-fk-error-logger pt-heartbeat pt-index-usage pt-kill pt-online-schema-change pt-query-digest pt-slave-delay pt-slave-find pt-table-checksum pt-table-sync pt-table-usage pt-upgrade ** Patch added: "lp1824762-percona-toolkit-sprintf-warnings.debdiff" https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+attachment/5255849/+files/lp1824762-percona-toolkit-sprintf-warnings.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824762 Title: various tools (including pt-query-digest, pt-heartbeat) failing due to fatal sprintf warnings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/percona-toolkit/+bug/1824762/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
