Guozhang and all others who involved. Thanks for your help!
> 7 окт. 2020 г., в 19:42, Guozhang Wang <wangg...@gmail.com> написал(а): > > Hello Nikolay, > > I've merged the PR to trunk. Thanks for your huge effort and patience going > through the review! > > Guozhang > > On Wed, Oct 7, 2020 at 6:52 AM Nikolay Izhikov <nizhi...@apache.org> wrote: > >> Great news! >> Thanks Magnus! >> >> I’ve updated the PR. >> >> Looks like we ready to merge it. >> >>> 7 окт. 2020 г., в 15:29, Magnus Edenhill <mag...@edenhill.se> >> написал(а): >>> >>> Hi, >>> >>> ducktape v0.8.0 is now released. >>> >>> Regards, >>> Magnus >>> >>> >>> Den ons 7 okt. 2020 kl 10:50 skrev Nikolay Izhikov <nizhi...@apache.org >>> : >>> >>>> Hello. >>>> >>>> Got 4 approvals for PR [1] >>>> The only thing we need to be able to merge it is a ducktape 0.8 release. >>>> If ducktape team need any help with the release, please, let me know. >>>> >>>> [1] https://github.com/apache/kafka/pull/9196 >>>> >>>> >>>>> 21 сент. 2020 г., в 12:58, Nikolay Izhikov <nizhikov....@gmail.com> >>>> написал(а): >>>>> >>>>> Hello. >>>>> >>>>> I fixed two system tests that fails in trunk, also. >>>>> >>>>> >> streams_upgrade_test.py::StreamsUpgradeTest.test_version_probing_upgrade >>>>> streams_static_membership_test.py >>>>> >>>>> Please, take a look at my PR [1] >>>>> >>>>> [1] https://github.com/apache/kafka/pull/9312 >>>>> >>>>>> 20 сент. 2020 г., в 06:11, Guozhang Wang <wangg...@gmail.com> >>>> написал(а): >>>>>> >>>>>> I've triggered a system test on top of your branch. >>>>>> >>>>>> Maybe you could also re-run the jenkins unit tests since currently all >>>> of >>>>>> them fails but you've only touched on system tests, so I'd like to >>>> confirm >>>>>> at least one successful run. >>>>>> >>>>>> On Wed, Sep 16, 2020 at 3:37 AM Nikolay Izhikov <nizhi...@apache.org> >>>> wrote: >>>>>> >>>>>>> Hello, Guozhang. >>>>>>> >>>>>>>> I can help run the test suite once your PR is cleanly rebased to >>>> verify >>>>>>> the whole suite works >>>>>>> >>>>>>> Thank you for joining to the review. >>>>>>> >>>>>>> 1. PR rebased on the current trunk. >>>>>>> >>>>>>> 2. I triggered all tests in my private environment to verify them >> after >>>>>>> rebase. >>>>>>> Will inform you once tests passed on my environment. >>>>>>> >>>>>>> 3. We need a new ducktape release [1] to be able to merge PR [2]. >>>>>>> For now, PR based on the ducktape trunk branch [3], not some >>>>>>> specific release. >>>>>>> If ducktape team need any help with the release, please, let me >>>>>>> know. >>>>>>> >>>>>>> [1] https://github.com/confluentinc/ducktape/issues/245 >>>>>>> [2] https://github.com/apache/kafka/pull/9196 >>>>>>> [3] >>>>>>> >>>> >> https://github.com/apache/kafka/pull/9196/files#diff-9235a7bdb1ca9268681c0e56f3f3609bR39 >>>>>>> >>>>>>>> 16 сент. 2020 г., в 07:32, Guozhang Wang <wangg...@gmail.com> >>>>>>> написал(а): >>>>>>>> >>>>>>>> Hello Nikolay, >>>>>>>> >>>>>>>> I can help run the test suite once your PR is cleanly rebased to >>>> verify >>>>>>> the >>>>>>>> whole suite works and then I can merge (I'm trusting Ivan and Magnus >>>> here >>>>>>>> for their reviews :) >>>>>>>> >>>>>>>> Guozhang >>>>>>>> >>>>>>>> On Mon, Sep 14, 2020 at 3:56 AM Nikolay Izhikov < >> nizhi...@apache.org> >>>>>>> wrote: >>>>>>>> >>>>>>>>> Hello! >>>>>>>>> >>>>>>>>> I got 2 approvals from Ivan Daschinskiy and Magnus Edenhill. >>>>>>>>> Committers, please, join the review. >>>>>>>>> >>>>>>>>>> 3 сент. 2020 г., в 11:06, Nikolay Izhikov <nizhikov....@gmail.com >>> >>>>>>>>> написал(а): >>>>>>>>>> >>>>>>>>>> Hello! >>>>>>>>>> >>>>>>>>>> Just a friendly reminder. >>>>>>>>>> >>>>>>>>>> Patch to resolve some kind of technical debt - python2 in system >>>> tests >>>>>>>>> is ready! >>>>>>>>>> Can someone, please, take a look? >>>>>>>>>> >>>>>>>>>> https://github.com/apache/kafka/pull/9196 >>>>>>>>>> >>>>>>>>>>> 28 авг. 2020 г., в 11:19, Nikolay Izhikov < >> nizhikov....@gmail.com> >>>>>>>>> написал(а): >>>>>>>>>>> >>>>>>>>>>> Hello! >>>>>>>>>>> >>>>>>>>>>> Any feedback on this? >>>>>>>>>>> What I should additionally do to prepare system tests migration? >>>>>>>>>>> >>>>>>>>>>>> 24 авг. 2020 г., в 11:17, Nikolay Izhikov < >> nizhikov....@gmail.com >>>>> >>>>>>>>> написал(а): >>>>>>>>>>>> >>>>>>>>>>>> Hello. >>>>>>>>>>>> >>>>>>>>>>>> PR [1] is ready. >>>>>>>>>>>> Please, review. >>>>>>>>>>>> >>>>>>>>>>>> But, I need help with the two following questions: >>>>>>>>>>>> >>>>>>>>>>>> 1. We need a new release of ducktape which includes fixes [2], >> [3] >>>>>>> for >>>>>>>>> python3. >>>>>>>>>>>> I created the issue in ducktape repo [4]. >>>>>>>>>>>> Can someone help me with the release? >>>>>>>>>>>> >>>>>>>>>>>> 2. I know that some companies run system tests for the trunk on >> a >>>>>>>>> regular bases. >>>>>>>>>>>> Can someone show me some results of these runs? >>>>>>>>>>>> So, I can compare failures in my PR and in the trunk. >>>>>>>>>>>> >>>>>>>>>>>> Results [5] of run all for my PR available in the ticket [6] >>>>>>>>>>>> >>>>>>>>>>>> ``` >>>>>>>>>>>> SESSION REPORT (ALL TESTS) >>>>>>>>>>>> ducktape version: 0.8.0 >>>>>>>>>>>> session_id: 2020-08-23--002 >>>>>>>>>>>> run time: 1010 minutes 46.483 seconds >>>>>>>>>>>> tests run: 684 >>>>>>>>>>>> passed: 505 >>>>>>>>>>>> failed: 9 >>>>>>>>>>>> ignored: 170 >>>>>>>>>>>> ``` >>>>>>>>>>>> >>>>>>>>>>>> [1] https://github.com/apache/kafka/pull/9196 >>>>>>>>>>>> [2] >>>>>>>>> >>>>>>> >>>> >> https://github.com/confluentinc/ducktape/commit/23bd5ab53802e3a1e1da1ddf3630934f33b02305 >>>>>>>>>>>> [3] >>>>>>>>> >>>>>>> >>>> >> https://github.com/confluentinc/ducktape/commit/bfe53712f83b025832d29a43cde3de3d7803106f >>>>>>>>>>>> [4] https://github.com/confluentinc/ducktape/issues/245 >>>>>>>>>>>> [5] >>>>>>>>> >> https://issues.apache.org/jira/secure/attachment/13010366/report.txt >>>>>>>>>>>> [6] https://issues.apache.org/jira/browse/KAFKA-10402 >>>>>>>>>>>> >>>>>>>>>>>>> 14 авг. 2020 г., в 21:26, Ismael Juma <ism...@juma.me.uk> >>>>>>> написал(а): >>>>>>>>>>>>> >>>>>>>>>>>>> +1 >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Aug 14, 2020 at 7:42 AM John Roesler < >>>> vvcep...@apache.org> >>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks Nikolay, >>>>>>>>>>>>>> >>>>>>>>>>>>>> No objection. This would be very nice to have. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> John >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Aug 14, 2020, at 09:18, Nikolay Izhikov wrote: >>>>>>>>>>>>>>> Hello. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If anyone's interested in porting it to Python 3 it would >> be a >>>>>>> good >>>>>>>>>>>>>> change. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I’ve created a ticket [1] to upgrade system tests to python3. >>>>>>>>>>>>>>> Does someone have any additional inputs or objections for >> this >>>>>>>>> change? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-10402 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1 июля 2020 г., в 00:26, Gokul Ramanan Subramanian < >>>>>>>>>>>>>> gokul24...@gmail.com> написал(а): >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks Colin. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> While at the subject of system tests, there are a few times >> I >>>> see >>>>>>>>> tests >>>>>>>>>>>>>>>> timed out (even on a large machine such as m5.4xlarge EC2 >> with >>>>>>>>> Linux). >>>>>>>>>>>>>> Are >>>>>>>>>>>>>>>> there any knobs that system tests provide to control >> timeouts >>>> / >>>>>>>>>>>>>> throughputs >>>>>>>>>>>>>>>> across all tests? >>>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Jun 30, 2020 at 6:32 PM Colin McCabe < >>>> cmcc...@apache.org >>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Ducktape runs on Python 2. You can't use it with Python 3, >>>> as >>>>>>>>> you are >>>>>>>>>>>>>>>>> trying to do here. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If anyone's interested in porting it to Python 3 it would >> be >>>> a >>>>>>>>> good >>>>>>>>>>>>>> change. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Otherwise, using docker as suggested here seems to be the >>>> best >>>>>>>>> way to >>>>>>>>>>>>>> go. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> best, >>>>>>>>>>>>>>>>> Colin >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Mon, Jun 29, 2020, at 02:14, Gokul Ramanan Subramanian >>>> wrote: >>>>>>>>>>>>>>>>>> Hi. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Has anyone had luck running Kafka system tests on a Mac. I >>>>>>> have a >>>>>>>>>>>>>> MacOS >>>>>>>>>>>>>>>>>> Mojave 10.14.6. I got Python 3.6.9 using pyenv. However, >> the >>>>>>>>> command >>>>>>>>>>>>>>>>>> *ducktape tests/kafkatest/tests* yields the following >> error, >>>>>>>>> making >>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>> look >>>>>>>>>>>>>>>>>> like some Python incompatibility issue. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> $ ducktape tests/kafkatest/tests >>>>>>>>>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>>>>>>>>> File "/Users/gokusubr/.pyenv/versions/3.6.9/bin/ducktape", >>>> line >>>>>>>>> 11, >>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>> <module> >>>>>>>>>>>>>>>>>> load_entry_point('ducktape', 'console_scripts', >>>> 'ducktape')() >>>>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>>>>>>> line 487, in load_entry_point >>>>>>>>>>>>>>>>>> return get_distribution(dist).load_entry_point(group, >> name) >>>>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>>>>>>> line 2728, in load_entry_point >>>>>>>>>>>>>>>>>> return ep.load() >>>>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>>>>>>> line 2346, in load >>>>>>>>>>>>>>>>>> return self.resolve() >>>>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>>>>>>> line 2352, in resolve >>>>>>>>>>>>>>>>>> module = __import__(self.module_name, >> fromlist=['__name__'], >>>>>>>>>>>>>>>>>> level=0) >>>>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/ducktape-0.7.6-py3.6.egg/ducktape/command_line/main.py", >>>>>>>>>>>>>>>>>> line 127 >>>>>>>>>>>>>>>>>> print "parameters are not valid json: " + str(e.message) >>>>>>>>>>>>>>>>>> ^ >>>>>>>>>>>>>>>>>> SyntaxError: invalid syntax >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I followed the instructions in tests/README.md to setup a >>>>>>>>> cluster of >>>>>>>>>>>>>> 9 >>>>>>>>>>>>>>>>>> worker machines. That worked well. When I ran *python >>>> setup.py >>>>>>>>>>>>>> develop* >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>> install the necessary dependencies (including ducktape), I >>>> got >>>>>>>>>>>>>> similar >>>>>>>>>>>>>>>>>> errors to above, but the overall command completed >>>>>>> successfully. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Any help appreciated. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> -- Guozhang >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> -- Guozhang >>>>> >>>> >>>> >> >> > > -- > -- Guozhang
