Re: Last sprint to finish line: Replace EasyMock/Powermock with Mockito

[Christo]

 Hello!
Apologies for the delay in my reply. I have started working on migrating the 
TaskManagerTest to Mockito, but have not raised a pull request yet. It is on 
the list in https://issues.apache.org/jira/browse/KAFKA-14133 and I will add 
you as a reviewer the moment I publish it.
Best,Christo

[jira] [Resolved] (KAFKA-9397) Deprecate Direct Zookeeper access in Kafka Administrative Tools

[Ismael Juma (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma resolved KAFKA-9397.

Resolution: Fixed

> Deprecate Direct Zookeeper access in Kafka Administrative Tools
> ---
>
> Key: KAFKA-9397
> URL: https://issues.apache.org/jira/browse/KAFKA-9397
> Project: Kafka
>  Issue Type: Improvement
>  Components: tools
>Affects Versions: 2.5.0
>Reporter: Colin McCabe
>Assignee: Colin McCabe
>Priority: Major
> Fix For: 2.5.0
>
>
> KIP-555: Deprecate Direct Zookeeper access in Kafka Administrative Tools



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

PR Review

[Nandini Anagondi]

Hi,

Can you help me in reviewing this pr please.?
https://github.com/apache/kafka/pull/11516/files.

Thanks,
Nandini A.

Re: [DISCUSS] KIP-852 Optimize calculation of size for log in remote tier

[Satish Duggana]

Thanks for the KIP Divij!

This is a nice improvement to avoid recalculation of size. Customized RLMMs
can implement the best possible approach by caching or maintaining the size
in an efficient way. But this is not a big concern for the default topic
based RLMM as mentioned in the KIP.

~Satish.

On Wed, 13 Jul 2022 at 18:48, Divij Vaidya  wrote:

> Thank you for your review Luke.
>
> > Reg: is that would the new `RemoteLogSizeBytes` metric be a performance
> overhead? Although we move the calculation to a seperate API, we still
> can't assume users will implement a light-weight method, right?
>
> This metric would be logged using the information that is already being
> calculated for handling remote retention logic, hence, no additional work
> is required to calculate this metric. More specifically, whenever
> RemoteLogManager calls getRemoteLogSize API, this metric would be captured.
> This API call is made every time RemoteLogManager wants to handle expired
> remote log segments (which should be periodic). Does that address your
> concern?
>
> Divij Vaidya
>
>
>
> On Tue, Jul 12, 2022 at 11:01 AM Luke Chen  wrote:
>
> > Hi Divij,
> >
> > Thanks for the KIP!
> >
> > I think it makes sense to delegate the responsibility of calculation to
> the
> > specific RemoteLogMetadataManager implementation.
> > But one thing I'm not quite sure, is that would the new
> > `RemoteLogSizeBytes` metric be a performance overhead?
> > Although we move the calculation to a seperate API, we still can't assume
> > users will implement a light-weight method, right?
> >
> > Thank you.
> > Luke
> >
> > On Fri, Jul 1, 2022 at 5:47 PM Divij Vaidya 
> > wrote:
> >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-852%3A+Optimize+calculation+of+size+for+log+in+remote+tier
> > >
> > >
> > > Hey folks
> > >
> > > Please take a look at this KIP which proposes an extension to KIP-405.
> > This
> > > is my first KIP with Apache Kafka community so any feedback would be
> > highly
> > > appreciated.
> > >
> > > Cheers!
> > >
> > > --
> > > Divij Vaidya
> > > Sr. Software Engineer
> > > Amazon
> > >
> >
>

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1198

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 503596 lines...]
[2022-09-05T10:41:57.760Z] KStreamAggregationIntegrationTest > 
shouldReduce(TestInfo) PASSED
[2022-09-05T10:41:57.760Z] 
[2022-09-05T10:41:57.760Z] KStreamAggregationIntegrationTest > 
shouldAggregate(TestInfo) STARTED
[2022-09-05T10:41:58.263Z] 
[2022-09-05T10:41:58.263Z] FineGrainedAutoResetIntegrationTest > 
shouldOnlyReadRecordsWhereEarliestSpecifiedWithInvalidCommittedOffsets() PASSED
[2022-09-05T10:41:58.263Z] 
[2022-09-05T10:41:58.263Z] FineGrainedAutoResetIntegrationTest > 
shouldOnlyReadRecordsWhereEarliestSpecifiedWithNoCommittedOffsetsWithDefaultGlobalAutoOffsetResetEarliest()
 STARTED
[2022-09-05T10:42:00.813Z] 
[2022-09-05T10:42:00.813Z] KStreamAggregationIntegrationTest > 
shouldAggregate(TestInfo) PASSED
[2022-09-05T10:42:00.813Z] 
[2022-09-05T10:42:00.813Z] KStreamAggregationIntegrationTest > 
shouldCount(TestInfo) STARTED
[2022-09-05T10:42:04.982Z] 
[2022-09-05T10:42:04.982Z] KStreamAggregationIntegrationTest > 
shouldCount(TestInfo) PASSED
[2022-09-05T10:42:04.982Z] 
[2022-09-05T10:42:04.982Z] KStreamAggregationIntegrationTest > 
shouldGroupByKey(TestInfo) STARTED
[2022-09-05T10:42:09.328Z] 
[2022-09-05T10:42:09.328Z] KStreamAggregationIntegrationTest > 
shouldGroupByKey(TestInfo) PASSED
[2022-09-05T10:42:09.328Z] 
[2022-09-05T10:42:09.328Z] KStreamAggregationIntegrationTest > 
shouldCountWithInternalStore(TestInfo) STARTED
[2022-09-05T10:42:12.069Z] 
[2022-09-05T10:42:12.069Z] KStreamAggregationIntegrationTest > 
shouldCountWithInternalStore(TestInfo) PASSED
[2022-09-05T10:42:12.069Z] 
[2022-09-05T10:42:12.069Z] KStreamAggregationIntegrationTest > 
shouldCountUnlimitedWindows() STARTED
[2022-09-05T10:42:15.936Z] 
[2022-09-05T10:42:15.936Z] KStreamAggregationIntegrationTest > 
shouldCountUnlimitedWindows() PASSED
[2022-09-05T10:42:15.936Z] 
[2022-09-05T10:42:15.936Z] KStreamAggregationIntegrationTest > 
shouldReduceWindowed(TestInfo) STARTED
[2022-09-05T10:42:18.681Z] 
[2022-09-05T10:42:18.681Z] KStreamAggregationIntegrationTest > 
shouldReduceWindowed(TestInfo) PASSED
[2022-09-05T10:42:18.681Z] 
[2022-09-05T10:42:18.681Z] KStreamAggregationIntegrationTest > 
shouldCountSessionWindows() STARTED
[2022-09-05T10:42:20.544Z] 
[2022-09-05T10:42:20.544Z] KStreamAggregationIntegrationTest > 
shouldCountSessionWindows() PASSED
[2022-09-05T10:42:20.544Z] 
[2022-09-05T10:42:20.544Z] KStreamAggregationIntegrationTest > 
shouldAggregateWindowed(TestInfo) STARTED
[2022-09-05T10:42:23.538Z] 
[2022-09-05T10:42:23.538Z] KStreamAggregationIntegrationTest > 
shouldAggregateWindowed(TestInfo) PASSED
[2022-09-05T10:42:29.583Z] streams-5: SMOKE-TEST-CLIENT-CLOSED
[2022-09-05T10:42:29.583Z] streams-1: SMOKE-TEST-CLIENT-CLOSED
[2022-09-05T10:42:29.583Z] streams-3: SMOKE-TEST-CLIENT-CLOSED
[2022-09-05T10:42:29.583Z] streams-0: SMOKE-TEST-CLIENT-CLOSED
[2022-09-05T10:42:29.583Z] streams-6: SMOKE-TEST-CLIENT-CLOSED
[2022-09-05T10:42:29.583Z] streams-4: SMOKE-TEST-CLIENT-CLOSED
[2022-09-05T10:42:29.583Z] streams-2: SMOKE-TEST-CLIENT-CLOSED
[2022-09-05T10:42:35.607Z] 
[2022-09-05T10:42:35.607Z] FAILURE: Build failed with an exception.
[2022-09-05T10:42:35.607Z] 
[2022-09-05T10:42:35.607Z] * What went wrong:
[2022-09-05T10:42:35.607Z] Execution failed for task ':core:unitTest'.
[2022-09-05T10:42:35.607Z] > Process 'Gradle Test Executor 136' finished with 
non-zero exit value 1
[2022-09-05T10:42:35.607Z]   This problem might be caused by incorrect test 
process configuration.
[2022-09-05T10:42:35.607Z]   Please refer to the test execution section in the 
User Manual at 
https://docs.gradle.org/7.5.1/userguide/java_testing.html#sec:test_execution
[2022-09-05T10:42:35.607Z] 
[2022-09-05T10:42:35.607Z] * Try:
[2022-09-05T10:42:35.607Z] > Run with --stacktrace option to get the stack 
trace.
[2022-09-05T10:42:35.607Z] > Run with --info or --debug option to get more log 
output.
[2022-09-05T10:42:35.607Z] > Run with --scan to get full insights.
[2022-09-05T10:42:35.607Z] 
[2022-09-05T10:42:35.607Z] * Get more help at https://help.gradle.org
[2022-09-05T10:42:35.607Z] 
[2022-09-05T10:42:35.607Z] BUILD FAILED in 2h 33m 44s
[2022-09-05T10:42:35.607Z] 212 actionable tasks: 115 executed, 97 up-to-date
[2022-09-05T10:42:35.607Z] 
[2022-09-05T10:42:35.607Z] See the profiling report at: 
file:///home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk/build/reports/profile/profile-2022-09-05-08-08-55.html
[2022-09-05T10:42:35.607Z] A fine-grained performance profile is available: use 
the --scan option.
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Failed in branch JDK 17 and Scala 2.13
[2022-09-05T10:42:42.729Z] 
[2022-09-05T10:42:42.729Z]