[jira] [Commented] (ZOOKEEPER-4757) Support JSON format logging
[
https://issues.apache.org/jira/browse/ZOOKEEPER-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774903#comment-17774903
]
Jan Høydahl commented on ZOOKEEPER-4757:
> What do you mean exactly by 'Maven can help keep versions' and why is it an
> advantage over a docker project?
Dockerfile is not particaulary suited for pulling jar files. You end up with
wget from mvnrepo or similar and no dependabot support or similar for staying
up to date. Let's see what downstream says.
Is there anything at all from my PR you want to accept? Docs with pointers?
Commented-out appender?
> Support JSON format logging
> ---
>
> Key: ZOOKEEPER-4757
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4757
> Project: ZooKeeper
> Issue Type: Improvement
>Reporter: Jan Høydahl
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> More and more enterprise users request structured JSON format logging for
> their applications. This removes the need for configuring custom log line
> parsers for every application when collecting logs centrally.
> Zookeeper has flexible logging through Slf4j and Logback, for which there are
> several ways to achieve JSON logging. But for end users (such as helm chart
> user) it is very difficult to achieve. It should ideally be as simple as a
> configuration option.
> OpenTelemetry is a CNCF project that has become the defacto standard for
> metrics and traces collection. They also cover logging, and they recently
> [standardized on ECS JSON
> format|https://opentelemetry.io/blog/2023/ecs-otel-semconv-convergence/] as
> their log schema for OTEL-logging. Although there are other JSON formats in
> use, a pragmatic option for zookeeper is to only support ECS.
> Proposed way to enable JSON logging:
> {code:java}
> export ZOO_LOG_FORMAT=json
> bin/zkServer.sh start-foreground{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ZOOKEEPER-4757) Support JSON format logging
[
https://issues.apache.org/jira/browse/ZOOKEEPER-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774883#comment-17774883
]
Andor Molnar commented on ZOOKEEPER-4757:
-
{quote} I have contacted both those downstream projects with similar requests,
...
{quote}
That makes perfect sense to me. I think such addition would be better fit for a
ZooKeeper docker project.
{quote}...I still feel the best place to add this convenience feature is
upstream where Maven can help keep versions in sync etc.
{quote}
What do you mean exactly by 'Maven can help keep versions' and why is it an
advantage over a docker project?
> Support JSON format logging
> ---
>
> Key: ZOOKEEPER-4757
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4757
> Project: ZooKeeper
> Issue Type: Improvement
>Reporter: Jan Høydahl
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> More and more enterprise users request structured JSON format logging for
> their applications. This removes the need for configuring custom log line
> parsers for every application when collecting logs centrally.
> Zookeeper has flexible logging through Slf4j and Logback, for which there are
> several ways to achieve JSON logging. But for end users (such as helm chart
> user) it is very difficult to achieve. It should ideally be as simple as a
> configuration option.
> OpenTelemetry is a CNCF project that has become the defacto standard for
> metrics and traces collection. They also cover logging, and they recently
> [standardized on ECS JSON
> format|https://opentelemetry.io/blog/2023/ecs-otel-semconv-convergence/] as
> their log schema for OTEL-logging. Although there are other JSON formats in
> use, a pragmatic option for zookeeper is to only support ECS.
> Proposed way to enable JSON logging:
> {code:java}
> export ZOO_LOG_FORMAT=json
> bin/zkServer.sh start-foreground{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ZOOKEEPER-4753) Explicit handling of DIGEST-MD5 vs GSSAPI in quorum auth
[
https://issues.apache.org/jira/browse/ZOOKEEPER-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774848#comment-17774848
]
Luke Chen commented on ZOOKEEPER-4753:
--
I see. Thanks.
> Explicit handling of DIGEST-MD5 vs GSSAPI in quorum auth
>
>
> Key: ZOOKEEPER-4753
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4753
> Project: ZooKeeper
> Issue Type: Improvement
> Components: server
>Affects Versions: 3.9.0
>Reporter: Damien Diederen
>Assignee: Damien Diederen
>Priority: Major
> Fix For: 3.7.2, 3.8.3, 3.9.1
>
>
> The SASL-based quorum authorizer does not explicitly distinguish between the
> DIGEST-MD5 and GSSAPI mechanisms: it is simply relying on {{NameCallback}}
> and {{PasswordCallback}} for authentication with the former and examining
> Kerberos principals in {{AuthorizeCallback}} for the latter.
> It turns out that some SASL/DIGEST-MD5 configurations cause authentication
> and authorization IDs not to match the expected format, and the
> DIGEST-MD5-based portions of the quorum test suite to fail with obscure
> errors. (They can be traced to failures to join the quorum, but only by
> looking into detailed logs.)
> We can use the login module name to determine whether DIGEST-MD5 or GSSAPI is
> used, and relax the authentication ID check for the former. As a cleanup, we
> can keep the password-based credential map empty when Kerberos principals are
> expected. Finally, we can adapt tests to ensure "weirdly-shaped" credentials
> only cause authentication failures in the GSSAPI case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ZOOKEEPER-4753) Explicit handling of DIGEST-MD5 vs GSSAPI in quorum auth
[
https://issues.apache.org/jira/browse/ZOOKEEPER-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774847#comment-17774847
]
Andor Molnar commented on ZOOKEEPER-4753:
-
[~showuon] Yes it is.
> Explicit handling of DIGEST-MD5 vs GSSAPI in quorum auth
>
>
> Key: ZOOKEEPER-4753
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4753
> Project: ZooKeeper
> Issue Type: Improvement
> Components: server
>Affects Versions: 3.9.0
>Reporter: Damien Diederen
>Assignee: Damien Diederen
>Priority: Major
> Fix For: 3.7.2, 3.8.3, 3.9.1
>
>
> The SASL-based quorum authorizer does not explicitly distinguish between the
> DIGEST-MD5 and GSSAPI mechanisms: it is simply relying on {{NameCallback}}
> and {{PasswordCallback}} for authentication with the former and examining
> Kerberos principals in {{AuthorizeCallback}} for the latter.
> It turns out that some SASL/DIGEST-MD5 configurations cause authentication
> and authorization IDs not to match the expected format, and the
> DIGEST-MD5-based portions of the quorum test suite to fail with obscure
> errors. (They can be traced to failures to join the quorum, but only by
> looking into detailed logs.)
> We can use the login module name to determine whether DIGEST-MD5 or GSSAPI is
> used, and relax the authentication ID check for the former. As a cleanup, we
> can keep the password-based credential map empty when Kerberos principals are
> expected. Finally, we can adapt tests to ensure "weirdly-shaped" credentials
> only cause authentication failures in the GSSAPI case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ZOOKEEPER-4757) Support JSON format logging
[
https://issues.apache.org/jira/browse/ZOOKEEPER-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774842#comment-17774842
]
Jan Høydahl commented on ZOOKEEPER-4757:
As a committer I'm well aware that projects are not obliged to anything else
than publishing the source code tarball. You will find among ASF projects
different levels of commitment to the end users. Some projects rely on
downstream companies taking care of user fridendly packaging, documentation,
docker etc. Others (like e.g. Solr where I'm a committer) have chosen to
include an official, hardened Dockerfile as part of the project (which end
users can tweak if they like), and accepted the incubation of the solr-operator
as well. There have been proposals for maintaining RPM/DEB packages too but we
don't do that as no committers can commit to :) maintaining such.
The zookeeper project seems to lack a clear downstream re-packager. I applaud
the volunteer initiative of publishing the Zookeeper docker image, hardening
the upstream experience with additional env.vars and documentation; it helps a
lot. Earlier, [Pravega|https://github.com/pravega/zookeeper-operator] did a
great job on production ready Zookeeper Operator with their own repackaged ZK
docker image, but sadly that project is dying. Bitnami still does a great job
with the ZK helmchart, so not all is dark, though fragmented. I have contacted
both those downstream projects with similar requests, however I still feel the
best place to add this convenience feature is upstream where Maven can help
keep versions in sync etc.
But it is of course up to you to accept or reject the contribution.
> Support JSON format logging
> ---
>
> Key: ZOOKEEPER-4757
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4757
> Project: ZooKeeper
> Issue Type: Improvement
>Reporter: Jan Høydahl
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> More and more enterprise users request structured JSON format logging for
> their applications. This removes the need for configuring custom log line
> parsers for every application when collecting logs centrally.
> Zookeeper has flexible logging through Slf4j and Logback, for which there are
> several ways to achieve JSON logging. But for end users (such as helm chart
> user) it is very difficult to achieve. It should ideally be as simple as a
> configuration option.
> OpenTelemetry is a CNCF project that has become the defacto standard for
> metrics and traces collection. They also cover logging, and they recently
> [standardized on ECS JSON
> format|https://opentelemetry.io/blog/2023/ecs-otel-semconv-convergence/] as
> their log schema for OTEL-logging. Although there are other JSON formats in
> use, a pragmatic option for zookeeper is to only support ECS.
> Proposed way to enable JSON logging:
> {code:java}
> export ZOO_LOG_FORMAT=json
> bin/zkServer.sh start-foreground{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
