kirktrue commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1364315852
########## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetry.java: ########## @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.kafka.common.telemetry; + +import org.apache.kafka.common.metrics.MetricsContext; +import org.apache.kafka.common.metrics.MetricsReporter; +import org.apache.kafka.common.annotation.InterfaceStability; + +import java.util.Optional; + +/** + * A {@link MetricsReporter} may implement this interface to indicate support for collecting client Review Comment: If this interface is only useful if it's implementing class is a `MetricsReporter`, why not have this interface extend from `MetricsReporter`? I'm sure there's a good reason that I'm missing. ########## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ########## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.kafka.common.telemetry; + +import org.apache.kafka.common.KafkaException; +import org.apache.kafka.common.annotation.InterfaceStability; +import org.apache.kafka.common.metrics.MetricsReporter; +import org.apache.kafka.common.protocol.ApiKeys; +import org.apache.kafka.common.requests.AbstractResponse; +import org.apache.kafka.common.requests.AbstractRequest.Builder; + +import java.util.Optional; + +/** + * A {@link MetricsReporter} may implement this interface to indicate support for sending client + * telemetry to the broker. + */ [email protected] +public interface ClientTelemetrySender extends AutoCloseable { + + /** + * Return the next time when the telemetry API should be attempted (i.e., interval time has elapsed). + * <p> + * If another telemetry API is in-flight, then {@code timeoutMs} should be returned as the + * maximum wait time. + * + * @param timeoutMs The timeout for the inflight telemetry API call. + * @return remaining time in ms till the telemetry API be attempted again. + */ + long timeToNextUpdate(long timeoutMs); + + /** + * Return the telemetry request based on client state i.e. determine if + * {@link org.apache.kafka.common.requests.GetTelemetrySubscriptionsRequest} or + * {@link org.apache.kafka.common.requests.PushTelemetryRequest} be constructed. + * + * @return request for telemetry API call. + */ + Optional<Builder<?>> createRequest(); Review Comment: What is the benefit to having the implementation drive the determination of which request type is sent? Am I being a bit paranoid in thinking that allowing _any_ request type to be generated here will be confusing for implementors? ########## clients/src/main/java/org/apache/kafka/common/telemetry/collector/MetricsCollector.java: ########## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.kafka.common.telemetry.collector; + +import org.apache.kafka.common.annotation.InterfaceStability; +import org.apache.kafka.common.telemetry.emitter.Emitter; + +/** + * A {@code MetricsCollector} is responsible for scraping a source of metrics and forwarding + * them to the given {@link Emitter}. For example, a given collector might be used to collect + * system metrics, Kafka metrics, JVM metrics, or other metrics that are to be captured, exposed, + * and/or forwarded. + * + * <p/> + * + * In general, a {@code MetricsCollector} implementation is closely managed by another entity + * (that entity is colloquially referred to as the "telemetry reporter") that will be in + * charge of its lifecycle via the {@link #start()} and {@link #stop()} methods. The telemetry + * reporter should ensure that the {@link #start()} method is invoked <i>once and only once</i> + * before calls to {@link #collect(Emitter)} are made. Implementations of {@code MetricsCollector} + * should allow for the corner-case that {@link #stop()} is called before {@link #start()}, + * which might happen in the case of error on startup of the telemetry reporter. + * + * <p/> + * + * Regarding threading, the {@link #start()} and {@link #stop()} methods may be called from + * different threads and so proper care should be taken by implementations of the + * {@code MetricsCollector} interface to be thread-safe. However, the telemetry reporter must + * ensure that the {@link #collect(Emitter)} method should only be invoked in a synchronous + * manner. + * + * @see Emitter + */ [email protected] +public interface MetricsCollector { + + /** + * The {@code collect} method is called by the telemetry reporter to retrieve the value + * of its desired set of metrics, and then forward those on to the provided + * {@link Emitter}. The implementation may choose to collect all the metrics before forwarding + * them to the {@code emitter}, or they may be forwarded as they are collected. + * + * <p/> + * + * In general, the implementation should try not to presume the characteristics of the + * {@link Emitter} so as to keep a loose coupling. + * + * @param emitter {@link Emitter} to which the metric values will be passed once collected + */ + void collect(Emitter emitter); + + /** + * Allows the {@code MetricsCollector} implementation to initialize itself. This method should + * be invoked by the telemetry reporter before calls to {@link #collect(Emitter)} are made. The + * telemetry reporter should not invoke this method more than once. + */ + default void start() { + // Do nothing... + } + + /** + * Allows the {@code MetricsCollector} implementation to stop itself and dispose of any resources. + * This method should ideally be invoked only once by the telemetry reporter. + * + * <p/> + * + * Calls to {@link #collect(Emitter)} once this method has been invoked should be expected to + * fail by the telemetry reporter; it should take caution to handle that case. Review Comment: If I wrote this comment, I apologize, but when it says the method should "fail", does that mean throw an exception, log an error, or something else? ########## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetry.java: ########## @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.kafka.common.telemetry; + +import org.apache.kafka.common.metrics.MetricsContext; +import org.apache.kafka.common.metrics.MetricsReporter; +import org.apache.kafka.common.annotation.InterfaceStability; + +import java.util.Optional; + +/** + * A {@link MetricsReporter} may implement this interface to indicate support for collecting client + * telemetry on the client or server side. Review Comment: Also, I'm confused if this is implemented on the client, broker, or both? It seems like its two methods are somewhat mutually exclusive, depending on the context (client or broker) in which it is being run. ########## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetryReceiver.java: ########## @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.kafka.common.telemetry; + +import org.apache.kafka.common.annotation.InterfaceStability; +import org.apache.kafka.server.authorizer.AuthorizableRequestContext; + [email protected] +public interface ClientTelemetryReceiver { Review Comment: Originally this interface was implemented and executed on the broker upon receipt of the payload. Is it now also optionally executed on the client? ########## clients/src/main/java/org/apache/kafka/common/metrics/MetricsReporter.java: ########## @@ -24,13 +24,14 @@ import org.apache.kafka.common.Reconfigurable; import org.apache.kafka.common.annotation.InterfaceStability; import org.apache.kafka.common.config.ConfigException; +import org.apache.kafka.common.telemetry.ClientTelemetry; /** * A plugin interface to allow things to listen as new metrics are created so they can be reported. * <p> * Implement {@link org.apache.kafka.common.ClusterResourceListener} to receive cluster metadata once it's available. Please see the class documentation for ClusterResourceListener for more information. */ -public interface MetricsReporter extends Reconfigurable, AutoCloseable { +public interface MetricsReporter extends Reconfigurable, AutoCloseable, ClientTelemetry { Review Comment: It does seem like it should be the reverse. But given that the `ClientTelemetry` interface's methods both have defaults, this wouldn't technically break anything, would it? Still, I don't think this tight coupling is desired. ########## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ########## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.kafka.common.telemetry; + +import org.apache.kafka.common.KafkaException; +import org.apache.kafka.common.annotation.InterfaceStability; +import org.apache.kafka.common.metrics.MetricsReporter; +import org.apache.kafka.common.protocol.ApiKeys; +import org.apache.kafka.common.requests.AbstractResponse; +import org.apache.kafka.common.requests.AbstractRequest.Builder; + +import java.util.Optional; + +/** + * A {@link MetricsReporter} may implement this interface to indicate support for sending client + * telemetry to the broker. + */ [email protected] +public interface ClientTelemetrySender extends AutoCloseable { + + /** + * Return the next time when the telemetry API should be attempted (i.e., interval time has elapsed). + * <p> + * If another telemetry API is in-flight, then {@code timeoutMs} should be returned as the + * maximum wait time. + * + * @param timeoutMs The timeout for the inflight telemetry API call. + * @return remaining time in ms till the telemetry API be attempted again. + */ + long timeToNextUpdate(long timeoutMs); + + /** + * Return the telemetry request based on client state i.e. determine if + * {@link org.apache.kafka.common.requests.GetTelemetrySubscriptionsRequest} or + * {@link org.apache.kafka.common.requests.PushTelemetryRequest} be constructed. + * + * @return request for telemetry API call. + */ + Optional<Builder<?>> createRequest(); + + /** + * Handle response for telemetry APIs + * + * @param response either {@link org.apache.kafka.common.requests.GetTelemetrySubscriptionsResponse} or + * {@link org.apache.kafka.common.requests.PushTelemetryResponse} telemetry API response. + */ + void handleResponse(AbstractResponse response); Review Comment: In more precise terms, I'm thinking this should be two methods: ```java void handleResponse(GetTelemetrySubscriptionsResponse response); void handleResponse(PushTelemetryResponse response); ``` ########## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ########## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.kafka.common.telemetry; + +import org.apache.kafka.common.KafkaException; +import org.apache.kafka.common.annotation.InterfaceStability; +import org.apache.kafka.common.metrics.MetricsReporter; +import org.apache.kafka.common.protocol.ApiKeys; +import org.apache.kafka.common.requests.AbstractResponse; +import org.apache.kafka.common.requests.AbstractRequest.Builder; + +import java.util.Optional; + +/** + * A {@link MetricsReporter} may implement this interface to indicate support for sending client + * telemetry to the broker. + */ [email protected] +public interface ClientTelemetrySender extends AutoCloseable { + + /** + * Return the next time when the telemetry API should be attempted (i.e., interval time has elapsed). + * <p> + * If another telemetry API is in-flight, then {@code timeoutMs} should be returned as the + * maximum wait time. + * + * @param timeoutMs The timeout for the inflight telemetry API call. + * @return remaining time in ms till the telemetry API be attempted again. + */ + long timeToNextUpdate(long timeoutMs); + + /** + * Return the telemetry request based on client state i.e. determine if + * {@link org.apache.kafka.common.requests.GetTelemetrySubscriptionsRequest} or + * {@link org.apache.kafka.common.requests.PushTelemetryRequest} be constructed. + * + * @return request for telemetry API call. + */ + Optional<Builder<?>> createRequest(); + + /** + * Handle response for telemetry APIs + * + * @param response either {@link org.apache.kafka.common.requests.GetTelemetrySubscriptionsResponse} or + * {@link org.apache.kafka.common.requests.PushTelemetryResponse} telemetry API response. + */ + void handleResponse(AbstractResponse response); Review Comment: Is there any reason not to have overloaded "handle" methods for the two response types we expect the implementation to handle? ########## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ########## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.kafka.common.telemetry; + +import org.apache.kafka.common.KafkaException; +import org.apache.kafka.common.annotation.InterfaceStability; +import org.apache.kafka.common.metrics.MetricsReporter; +import org.apache.kafka.common.protocol.ApiKeys; +import org.apache.kafka.common.requests.AbstractResponse; +import org.apache.kafka.common.requests.AbstractRequest.Builder; + +import java.util.Optional; + +/** + * A {@link MetricsReporter} may implement this interface to indicate support for sending client + * telemetry to the broker. + */ [email protected] +public interface ClientTelemetrySender extends AutoCloseable { + + /** + * Return the next time when the telemetry API should be attempted (i.e., interval time has elapsed). + * <p> + * If another telemetry API is in-flight, then {@code timeoutMs} should be returned as the + * maximum wait time. + * + * @param timeoutMs The timeout for the inflight telemetry API call. + * @return remaining time in ms till the telemetry API be attempted again. + */ + long timeToNextUpdate(long timeoutMs); + + /** + * Return the telemetry request based on client state i.e. determine if + * {@link org.apache.kafka.common.requests.GetTelemetrySubscriptionsRequest} or + * {@link org.apache.kafka.common.requests.PushTelemetryRequest} be constructed. + * + * @return request for telemetry API call. + */ + Optional<Builder<?>> createRequest(); + + /** + * Handle response for telemetry APIs + * + * @param response either {@link org.apache.kafka.common.requests.GetTelemetrySubscriptionsResponse} or + * {@link org.apache.kafka.common.requests.PushTelemetryResponse} telemetry API response. + */ + void handleResponse(AbstractResponse response); + + /** + * Handle response for failed telemetry request. + * + * @param apiKey determining the telemetry API request type. + * @param kafkaException the fatal exception. + */ + void handleFailedRequest(ApiKeys apiKey, KafkaException kafkaException); Review Comment: As with the above comment, why not... In more precise terms, I'm thinking this should be two methods: ```java void handleFailedGetTelemetrySubscriptionsRequest(. . .); void handlePushTelemetryRequest(. . .); ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
