Hi Benjamin, I suspect that the problem is that the maxRecords variable in KafkaSourceConsumerFn is a static one. Can you try to change it so that the variable belongs to the instance and not the class?
I think this will fix the issue. I was about to run this test, but I have to go out for today. If not I'll try tomorrow:) Best -P. On Mon, May 17, 2021, 3:44 PM Pablo Estrada <[email protected]> wrote: > Hi Benjamin! > I just ran it today. I have not found out anything yet - for now I'm > trying to find the logs coming from the Java container. I'll let you know > how far I get today. > Best > -P. > > On Fri, May 14, 2021 at 10:57 AM Benjamin Gonzalez Delgado < > [email protected]> wrote: > >> Hi Pablo! >> Did you have a chance to execute the test to find something rare? >> Do you know if there is a way to use breakpoints when running this kind >> of test I mean breakpoints in java code when running a pipeline from python? >> >> >> On Fri, May 7, 2021 at 4:54 PM Benjamin Gonzalez Delgado < >> [email protected]> wrote: >> >>> Hi Pablo!, >>> >>> >>> I run the gradle task >>> >>> ./gradlew :sdks:python:test-suites:portable:py37:postCommitPy37IT >>> >>> >>> You can comment the other test in >>> sdks/python/test-suites/portable/common.gradle >>> <https://github.com/benWize/beam/pull/12/files#diff-3b0c84ea390ab209fe959cda93b113ce47f7485f1b6046bbc5597da7d17f784d> >>> for >>> a faster run. >>> >>> Thanks. >>> >>> On Fri, May 7, 2021 at 4:42 PM Pablo Estrada <[email protected]> wrote: >>> >>>> Hi Benjamin! >>>> How do I run the test on your PR? I can try to run it to see what's >>>> going on.[ >>>> Best >>>> -P. >>>> >>>> On Thu, May 6, 2021 at 1:46 PM Pablo Estrada <[email protected]> >>>> wrote: >>>> >>>>> Hi Benjamin! >>>>> That's strange if the Java test is working well that the Python/xlang >>>>> test would run into issues. Let me take a look at your PR and see if I can >>>>> see anything suspicious. >>>>> Best >>>>> -P. >>>>> >>>>> On Thu, May 6, 2021 at 10:49 AM Benjamin Gonzalez Delgado < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi team, >>>>>> I am working on BEAM-11838[1], but I am a little stuck right now[2], >>>>>> my issue is that when I set param maxNumberOfRecords through >>>>>> ExternalTransformBuilder(L117) its sets the value correctly to the >>>>>> constructor on KafkaSourceConsumerFn (L111), but when it arrives at >>>>>> tryClaim method(L307) the value for maxNumberOfRecords is null again and >>>>>> it >>>>>> keeps running forever so my test never ends. >>>>>> I wonder if some of you have experience with KafkaSourceConsumerFn >>>>>> class or creating other cross-language transforms that could help. >>>>>> Any guidance on this would be appreciated. >>>>>> >>>>>> PD. The existing java test DebeziumIOMySqlConnectorIT does not have >>>>>> the same problem. >>>>>> >>>>>> [1] https://issues.apache.org/jira/browse/BEAM-11838 >>>>>> [2] https://github.com/benWize/beam/pull/12/files# >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *This email and its contents (including any attachments) are being >>>>>> sent toyou on the condition of confidentiality and may be protected by >>>>>> legalprivilege. Access to this email by anyone other than the intended >>>>>> recipientis unauthorized. If you are not the intended recipient, please >>>>>> immediatelynotify the sender by replying to this message and delete the >>>>>> materialimmediately from your system. Any further use, dissemination, >>>>>> distributionor reproduction of this email is strictly prohibited. >>>>>> Further, >>>>>> norepresentation is made with respect to any content contained in this >>>>>> email.* >>>>> >>>>> >> >> >> >> >> >> >> >> *This email and its contents (including any attachments) are being sent >> toyou on the condition of confidentiality and may be protected by >> legalprivilege. Access to this email by anyone other than the intended >> recipientis unauthorized. If you are not the intended recipient, please >> immediatelynotify the sender by replying to this message and delete the >> materialimmediately from your system. Any further use, dissemination, >> distributionor reproduction of this email is strictly prohibited. Further, >> norepresentation is made with respect to any content contained in this >> email.* > >
