zJiaJun opened a new pull request, #224: URL: https://github.com/apache/skywalking-go/pull/224
This pull request introduces comprehensive support for a Kafka reporter within the SkyWalking Go agent. The integration of Kafka as a reporting mechanism enhances the agent's data handling capabilities, providing users with an alternative to the existing gRPC reporter for sending trace, metrics, and log data. Key changes and features include: - **Kafka Reporter Implementation**: A new Kafka reporter has been implemented (plugins/core/reporter/kafka/kafka.go), including connection management to Kafka brokers, dedicated channels for sending trace segments, metrics, and logs, and robust error handling. - **Configuration Options**: Users can now configure Kafka reporter settings, such as broker addresses, topics for segments, meters, logging, and management, as well as parameters for send queue size, batch size, batch bytes, batch timeout, and acknowledgment requirements. These configurations are available in agent.default.yaml and through kafka_opts_enhance.go. - **Data Transformation**: A transform.go component has been added to convert reported spans and metrics into the format required by the Apache SkyWalking agent before sending to Kafka. - **Connection and Configuration Management**: New components like conn_manager.go and cds_manager.go have been introduced to manage gRPC connections and Configuration Discovery Services (CDS) effectively, particularly in the context of the Kafka reporter. - **Instrumentation Updates**: The agent's instrumentation has been updated to support the initialization and use of the Kafka reporter, allowing for a seamless switch or parallel operation with the gRPC reporter. - **Testing and Scenarios**: New test scenarios, startup scripts and configuration files have been added to ensure the Kafka reporter functions correctly. - **Dependency Management**: Necessary Kafka client libraries (e.g., github.com/segmentio/kafka-go) have been added to go.mod and go.sum. This enhancement provides greater flexibility in deploying and managing SkyWalking Go agent, allowing users to leverage Kafka's robust and scalable messaging system for their observability data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
