ivan-penchev opened a new pull request, #1234:
URL: https://github.com/apache/pulsar-client-go/pull/1234
<!--
### Contribution Checklist
- Name the pull request in the form "[Issue XYZ][component] Title of the
pull request", where *XYZ* should be replaced by the actual issue number.
Skip *Issue XYZ* if there is no associated github issue for this pull
request.
Skip *component* if you are unsure about which is the best component.
E.g. `[docs] Fix typo in produce method`.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from
multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and
this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
Hi 👋, first time contributor here
### Motivation
My team uses `log/slog` package from the standard library to control the
level and output type of our logs in our services.
In order for us to not have to import logrus as a direct dependency for part
of our testing suit, it would be nice if we can use `slog` package instead, and
wrap that in the provided by `pulsar/log` interfaces.
*Explain here the context, and why you're making that change. What is the
problem you're trying to solve.*
### Modifications
One additional file `/pulsar/log/wrapper_slog.go` is added.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change is a trivial rework / code cleanup without any test coverage.
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (no)
- The schema: (no)
- The default values of configurations: (no)
- The wire protocol: (no)
### Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable / docs / GoDocs /
not documented)
- If a feature is not applicable for documentation, explain why?
- If a feature is not documented yet in this PR, please create a followup
issue for adding the documentation
### Notes
Please direct me if I'm missing something 😄.
I attempted to include a meaningful test to enhance the quality of this pull
request. Regrettably, I was unsuccessful, as none of the scenarios I considered
seemed appropriate.
Throughout the development process, I utilized testcontainers, which I
recognize cannot be added because it would introduce an unnecessary dependency.
As well as it creates a circular dependacy between `pulsar` and `pulsar/log`.
However in order to verify the changes I am submitting I created a file
named `log_test.go` in the root directory of the repository. Within this file,
I initiated an Apache Pulsar testcontainer, then set up both a logrus logger
and an slog logger. These were utilized to establish a client and a producer.
Subsequently, I compared the length of the output from both loggers to ensure
the keys corresponded.
Below is the test for those interested in confirming that this change
functions correctly:
<details>
<summary><b>Integration test code, must run go mod tidy before you
run</b></summary>
```go
package logtest
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"sort"
"strings"
"testing"
"time"
"github.com/apache/pulsar-client-go/pulsar"
plog "github.com/apache/pulsar-client-go/pulsar/log"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/require"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)
func TestSlogWrapper_Integration(t *testing.T) {
pulsarC, pulsarUrl, _ := createPulsarContainer(t, context.Background())
topic := createPulsarTopic(t, pulsarC, "test", "test-topic")
var logBufferLogrus bytes.Buffer
loggerLogrus := logrus.New()
loggerLogrus.Out = &logBufferLogrus
loggerLogrus.Level = logrus.DebugLevel
loggerLogrus.SetFormatter(&logrus.JSONFormatter{})
clientLogrus, err := pulsar.NewClient(pulsar.ClientOptions{
URL: pulsarUrl,
ConnectionTimeout: 15 * time.Second,
Logger: plog.NewLoggerWithLogrus(loggerLogrus),
})
require.NoError(t, err, "Failed to create Logrus Pulsar client")
producerLogrus, err :=
clientLogrus.CreateProducer(pulsar.ProducerOptions{
Topic: topic,
})
require.NoError(t, err, "Failed to create Logrus producer")
_, err = producerLogrus.Send(context.Background(),
&pulsar.ProducerMessage{
Payload: []byte("test"),
})
require.NoError(t, err, "Failed to send message with Logrus producer")
producerLogrus.Close()
clientLogrus.Close()
var logBufferSlog bytes.Buffer
loggerSlog := slog.New(slog.NewJSONHandler(&logBufferSlog,
&slog.HandlerOptions{Level: slog.LevelDebug}))
clientSlog, err := pulsar.NewClient(pulsar.ClientOptions{
URL: pulsarUrl,
ConnectionTimeout: 15 * time.Second,
Logger: plog.NewLoggerWithSlog(loggerSlog),
})
require.NoError(t, err, "Failed to create Slog Pulsar client")
producerSlog, err := clientSlog.CreateProducer(pulsar.ProducerOptions{
Topic: topic,
})
require.NoError(t, err, "Failed to create Slog producer")
_, err = producerSlog.Send(context.Background(),
&pulsar.ProducerMessage{
Payload: []byte("test"),
})
require.NoError(t, err, "Failed to send message with Slog producer")
producerSlog.Close()
clientSlog.Close()
logOutputLogrus := logBufferLogrus.String()
logOutputSlog := logBufferSlog.String()
logrusLines := strings.Split(strings.TrimSpace(logOutputLogrus), "\n")
slogLines := strings.Split(strings.TrimSpace(logOutputSlog), "\n")
require.Equal(t, len(logrusLines), len(slogLines), "Log outputs have
different number of lines")
for i, logrusLine := range logrusLines {
var logrusJSON, slogJSON map[string]interface{}
err := json.Unmarshal([]byte(logrusLine), &logrusJSON)
require.NoError(t, err, "Logrus line %d is not valid JSON: %s",
i+1, logrusLine)
err = json.Unmarshal([]byte(slogLines[i]), &slogJSON)
require.NoError(t, err, "Slog line %d is not valid JSON: %s",
i+1, slogLines[i])
logrusKeys := getKeys(logrusJSON)
slogKeys := getKeys(slogJSON)
require.Equal(t, logrusKeys, slogKeys, "JSON keys do not match
on line %d\nLogrus: %v\nSlog: %v", i+1, logrusKeys, slogKeys)
}
}
func getKeys(m map[string]interface{}) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
}
func createPulsarContainer(t *testing.T, ctx context.Context) (pulsarC
testcontainers.Container, pulsarUrl, pulsarAdminUrl string) {
pulsarC, err := testcontainers.GenericContainer(ctx,
testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "apachepulsar/pulsar:3.3.0",
Cmd: []string{"bin/pulsar", "standalone"},
Name: "shared-pulsar-testcontainer",
ExposedPorts: []string{"6650/tcp", "8080/tcp"},
SkipReaper: true,
WaitingFor: wait.ForAll(
wait.ForExposedPort(),
wait.ForHTTP("/admin/v2/brokers/health").WithPort("8080/tcp").WithPollInterval(10*time.Second).WithResponseMatcher(func(r
io.Reader) bool {
respBytes, _ := io.ReadAll(r)
resp := string(respBytes)
isReady := strings.Contains(resp, `ok`)
return isReady
}),
),
},
Reuse: true,
Started: true,
})
require.NoError(t, err)
timeout := 60 * time.Second
pollInterval := 10 * time.Second
require.Eventually(t, func() bool {
pulsarUrl, err = pulsarC.PortEndpoint(ctx, "6650/tcp", "pulsar")
if err != nil {
return false
}
pulsarAdminUrl, err = pulsarC.PortEndpoint(ctx, "8080/tcp",
"http")
//nolint:all
if err != nil {
return false
}
return true
}, timeout, pollInterval, "The Pulsar container did not start within
the expected time")
return pulsarC, pulsarUrl, pulsarAdminUrl
}
func createPulsarTopic(t *testing.T, pulsarContainer
testcontainers.Container, namespace, topicName string) (topic string) {
tenant := "pulsar"
fullTopicName := fmt.Sprintf("persistent://%s/%s/%s", tenant,
namespace, topicName)
namespaceCommand := fmt.Sprintf("bin/pulsar-admin namespaces create
%s/%s || true", tenant, namespace)
topicCommand := fmt.Sprintf("bin/pulsar-admin topics create %s",
fullTopicName)
exitCodeNamespace, _, errNamespace :=
pulsarContainer.Exec(context.Background(), []string{
"/bin/sh", "-c", namespaceCommand,
})
if errNamespace != nil {
t.Fatalf("Error creating namespace: %s", errNamespace)
}
if exitCodeNamespace != 0 {
t.Fatalf("failed to create namespace, exit code: %d",
exitCodeNamespace)
}
exitCodeTopic, _, errTopic :=
pulsarContainer.Exec(context.Background(), []string{
"/bin/sh", "-c", topicCommand,
})
if errTopic != nil {
t.Fatalf("Error creating topic: %s", errTopic)
}
if exitCodeTopic != 0 {
t.Fatalf("failed to create topic, exit code: %d", exitCodeTopic)
}
return fullTopicName
}
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
