[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19374: [fix][broker] Fix delete namespace fail by a In-flight topic
mattisonchao commented on code in PR #19374:
URL: https://github.com/apache/pulsar/pull/19374#discussion_r1106766784
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java:
##
@@ -204,23 +208,48 @@ protected CompletableFuture>
internalGetNonPersistentTopics(Policie
});
}
-@SuppressWarnings("unchecked")
-protected CompletableFuture internalDeleteNamespaceAsync(boolean
force) {
+/**
+ * Delete the namespace and retry to resolve some topics that were not
created successfully(in metadata)
+ * during the deletion.
+ */
+protected @Nonnull CompletableFuture
internalDeleteNamespaceAsync(boolean force) {
+final CompletableFuture future = new CompletableFuture<>();
+RetryUtil.retryAsynchronously(() ->
internalDeleteNamespaceAsync0(force),
+new BackoffBuilder()
+.setInitialTime(200, TimeUnit.MILLISECONDS)
+.setMandatoryStop(15, TimeUnit.SECONDS)
Review Comment:
Hi, @eolivelli
I am trying to use the number of retrying to instead back off.
>what happens if the user gives up waiting and then it issues again the same
command while the backoff is still running ? Ideally everything should work
well and the second execution should wait for the previous execution to
complete. The expectation for the user is that when the command completes the
namespace is deleted.
For this problem, we can give some kind of the same operation a distributed
lock to avoid calling the same operation concurrently. I can send the
discussion to the mailing list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] StevenLuMT commented on pull request #19513: [cleanup][broker] Cleanup ManagedLedgerImpl's nouse method: isLedgersReadonly
StevenLuMT commented on PR #19513: URL: https://github.com/apache/pulsar/pull/19513#issuecomment-1430866204 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
nodece commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106729856
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -737,15 +737,15 @@ public void authChallengeSuccessCallback(AuthData
authChallenge,
// 2. an authentication refresh, in which case we need to
refresh authenticationData
AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String newAuthRole = authState.getAuthRole();
+AuthenticationDataSource newAuthDataSource =
authState.getAuthDataSource();
-// Refresh the auth data.
-this.authenticationData = authState.getAuthDataSource();
-if (log.isDebugEnabled()) {
-log.debug("[{}] Auth data refreshed for role={}",
remoteAddress, this.authRole);
-}
-
+// Refresh the auth data and role.
if (!useOriginalAuthState) {
this.authRole = newAuthRole;
+this.authenticationData = newAuthDataSource;
+} else {
+this.originalAuthData = newAuthDataSource;
+this.originalPrincipal = newAuthRole;
Review Comment:
> the `authRole` and the `originalPrincipal` are not allowed to change.
I suggest updating the role to keep correct with authentication data.
In the next logic, when the new role doesn't equal the old role, the broker
disconnects the client.
I don't know why it is so designed, but I still follow this rule. Maybe we
can remove this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
nodece commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106723829
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -737,15 +737,15 @@ public void authChallengeSuccessCallback(AuthData
authChallenge,
// 2. an authentication refresh, in which case we need to
refresh authenticationData
AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String newAuthRole = authState.getAuthRole();
+AuthenticationDataSource newAuthDataSource =
authState.getAuthDataSource();
-// Refresh the auth data.
-this.authenticationData = authState.getAuthDataSource();
-if (log.isDebugEnabled()) {
-log.debug("[{}] Auth data refreshed for role={}",
remoteAddress, this.authRole);
-}
-
+// Refresh the auth data and role.
if (!useOriginalAuthState) {
this.authRole = newAuthRole;
+this.authenticationData = newAuthDataSource;
+} else {
+this.originalAuthData = newAuthDataSource;
+this.originalPrincipal = newAuthRole;
Review Comment:
Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
nodece commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106723472
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -737,15 +737,15 @@ public void authChallengeSuccessCallback(AuthData
authChallenge,
// 2. an authentication refresh, in which case we need to
refresh authenticationData
AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String newAuthRole = authState.getAuthRole();
+AuthenticationDataSource newAuthDataSource =
authState.getAuthDataSource();
-// Refresh the auth data.
-this.authenticationData = authState.getAuthDataSource();
-if (log.isDebugEnabled()) {
-log.debug("[{}] Auth data refreshed for role={}",
remoteAddress, this.authRole);
-}
-
+// Refresh the auth data and role.
if (!useOriginalAuthState) {
this.authRole = newAuthRole;
+this.authenticationData = newAuthDataSource;
+} else {
+this.originalAuthData = newAuthDataSource;
+this.originalPrincipal = newAuthRole;
Review Comment:
Simplifying the authentication process is an important thing, I also think
is a good goal, but I've thought about it and haven't found a way to do it yet,
so made this PR.
This PR does not affect the built-in plugins.
After `AuthChallenge`, once the AuthorizationProvider checks the role and
authentication data, which will find a mismatch, this happened in our project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #406: [fix][doc] fix broken links
Anonymitaet commented on code in PR #406: URL: https://github.com/apache/pulsar-site/pull/406#discussion_r1106719488 ## docs/security-tls-authentication.md: ## @@ -136,7 +136,7 @@ var client = PulsarClient.Builder() ## Configure TLS authentication in CLI tools -[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md), and [`pulsar-client`](reference-cli-tools.md) use the `conf/client.conf` config file in a Pulsar installation. +[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](https://pulsar.apache.org/reference/#/next/pulsar-admin/), [`pulsar-perf`](https://pulsar.apache.org/reference/#/next/pulsar-perf), and [`pulsar-client`](https://pulsar.apache.org/reference/#/next/pulsar-client/) use the `conf/client.conf` config file in a Pulsar installation. Review Comment: Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] massakam commented on pull request #19298: [feat][cli] Add read command to pulsar-client-tools
massakam commented on PR #19298: URL: https://github.com/apache/pulsar/pull/19298#issuecomment-1430844064 @momo-jun The reference for the "read" subcommand added in this PR seems to have been automatically generated and committed to the pulsar-site repository. So I don't think I should do more. https://github.com/apache/pulsar-site/commit/75f16503bc3ce96e710f5072234b1178b3a8aff8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-python] keenborder786 commented on a diff in pull request #90: [fix] exception bug as per #89
keenborder786 commented on code in PR #90:
URL:
https://github.com/apache/pulsar-client-python/pull/90#discussion_r1106708871
##
pulsar/schema/schema_avro.py:
##
@@ -82,15 +82,6 @@ def decode(self, data):
return self._record_cls(**d)
else:
return d
-
else:
-class AvroSchema(Schema):
-def __init__(self, _record_cls, _schema_definition=None):
-raise Exception("Avro library support was not found. Make sure to
install Pulsar client " +
+raise Exception("Avro library support was not found. Make sure to install
Pulsar client " +
Review Comment:
okay let me have a look at it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-site] massakam opened a new pull request, #410: [improve][doc] Add description for using Copper Argos
massakam opened a new pull request, #410: URL: https://github.com/apache/pulsar-site/pull/410 ### Documentation In https://github.com/apache/pulsar/pull/19445, I added some new parameters to the authentication plugin for Athenz, so I added a description about them. - [ ] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on pull request #19498: [fix][client] fix receive dumplicated messages after seek to a timestamp in MultiTo…
aloyszhang commented on PR #19498: URL: https://github.com/apache/pulsar/pull/19498#issuecomment-1430816848 PTAL @poorbarcode @AnonHxy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] StevenLuMT commented on pull request #19513: [cleanup][broker] Cleanup ManagedLedgerImpl's nouse method: isLedgersReadonly
StevenLuMT commented on PR #19513: URL: https://github.com/apache/pulsar/pull/19513#issuecomment-1430811489 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] shy-share commented on issue #18963: Add metrics for failed topic load operation
shy-share commented on issue #18963: URL: https://github.com/apache/pulsar/issues/18963#issuecomment-1430808739 @tjiuming you can directly submit a pr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
michaeljmarshall commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106693259
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockAuthenticationProvider.java:
##
@@ -65,4 +71,41 @@ public String authenticate(AuthenticationDataSource
authData) throws Authenticat
"Not a valid principle. Should be
[pass|fail|error].[pass|fail|error], found " + principal);
}
+@Override
+public AuthenticationState newAuthState(AuthData authData, SocketAddress
remoteAddress, SSLSession sslSession)
+throws AuthenticationException {
+return new MockAuthState(this);
+}
+
+private static class MockAuthState implements AuthenticationState {
Review Comment:
I used `MockAlwaysExpiredAuthenticationProvider` to test authentication
refresh because it guaranteed the `ServerCnx` would send the `AuthChallenge`
command.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on issue #19332: Simplify authentication state management in Broker's ServerCnx class when networking with proxy
michaeljmarshall commented on issue #19332: URL: https://github.com/apache/pulsar/issues/19332#issuecomment-1430801299 > A. Proxy sends connect command with only `originalPrincipal` I learned while working on #19506 that we do not actually support refreshing authentication for this case. That greatly simplifies the cases I outlined at the start of this issue. The remaining issues: * `authenticationData` field is updated incorrectly. That will be solved by #19519. * we are using the proxy's stale authentication data when performing authorization -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
michaeljmarshall commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106683183
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -737,15 +737,15 @@ public void authChallengeSuccessCallback(AuthData
authChallenge,
// 2. an authentication refresh, in which case we need to
refresh authenticationData
AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String newAuthRole = authState.getAuthRole();
+AuthenticationDataSource newAuthDataSource =
authState.getAuthDataSource();
-// Refresh the auth data.
-this.authenticationData = authState.getAuthDataSource();
-if (log.isDebugEnabled()) {
-log.debug("[{}] Auth data refreshed for role={}",
remoteAddress, this.authRole);
-}
-
+// Refresh the auth data and role.
if (!useOriginalAuthState) {
this.authRole = newAuthRole;
+this.authenticationData = newAuthDataSource;
+} else {
+this.originalAuthData = newAuthDataSource;
+this.originalPrincipal = newAuthRole;
Review Comment:
Nit: the `authRole` and the `originalPrincipal` are not allowed to change. I
wonder if it would make more sense to set them when `state != State.Connected`.
It might help make the code a bit more readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-manager] tuteng commented on issue #505: Starting Docker container pulsar-manager-0.3.0 with internal PostreSQL does not work
tuteng commented on issue #505: URL: https://github.com/apache/pulsar-manager/issues/505#issuecomment-1430788012 @urfreespace Can you help take a look at it? thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
michaeljmarshall commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106671416
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -737,15 +737,15 @@ public void authChallengeSuccessCallback(AuthData
authChallenge,
// 2. an authentication refresh, in which case we need to
refresh authenticationData
AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String newAuthRole = authState.getAuthRole();
+AuthenticationDataSource newAuthDataSource =
authState.getAuthDataSource();
-// Refresh the auth data.
-this.authenticationData = authState.getAuthDataSource();
-if (log.isDebugEnabled()) {
-log.debug("[{}] Auth data refreshed for role={}",
remoteAddress, this.authRole);
-}
-
+// Refresh the auth data and role.
if (!useOriginalAuthState) {
this.authRole = newAuthRole;
+this.authenticationData = newAuthDataSource;
+} else {
+this.originalAuthData = newAuthDataSource;
+this.originalPrincipal = newAuthRole;
Review Comment:
> Sure, but I'm doing this to fix the store of the authentication data
correctly, and make sense.
I think I was too focused on the end goal. Your solution is a good first
step because it corrects how we store the authentication data. I think we
should merge this before worrying about changing the whole paradigm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[pulsar-client-go] branch xiangying/txn/tc_client updated (f49f946 -> fb0d50d)
This is an automated email from the ASF dual-hosted git repository. xiangying pushed a change to branch xiangying/txn/tc_client in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git from f49f946 delete some redundant resource creation logic add fb0d50d grant anonymous role to the `public` tenant No new revisions were added by this update. Summary of changes: scripts/pulsar-test-service-start.sh | 1 + 1 file changed, 1 insertion(+)
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
michaeljmarshall commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106670604
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ServerCnxTest.java:
##
@@ -1030,6 +1030,55 @@ public void
testVerifyAuthRoleAndAuthDataFromDirectConnectionBroker() throws Exc
}));
}
+@Test
+public void testRefreshOriginalPrincipalWithAuthDataForwardedFromProxy()
throws Exception {
Review Comment:
I must not have written some of the assertions I thought I did. You're right
that those tests all pass. It might be worth removing the comments that
reference #19332 because your PR will make them incorrect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] massakam opened a new pull request, #19527: [fix][security] Upgrade kafka client to 3.4.0 to fix CVE-2023-25194
massakam opened a new pull request, #19527: URL: https://github.com/apache/pulsar/pull/19527 ### Motivation OWASP dependency checks are currently failing on some other PRs (e.g. https://github.com/apache/pulsar/actions/runs/4180151982/jobs/7241507184). ``` Error: Failed to execute goal org.owasp:dependency-check-maven:8.0.1:aggregate (default) on project pulsar: Error: Error: One or more dependencies were identified with vulnerabilities that have a CVSS score greater than or equal to '7.0': Error: Error: kafka-clients-2.8.2.jar: CVE-2023-25194(8.8) Error: Error: See the dependency-check report for more details. Error: -> [Help 1] Error: Error: To see the full stack trace of the errors, re-run Maven with the -e switch. Error: Re-run Maven using the -X switch to enable full debug logging. Error: Error: For more information about the errors and possible solutions, please read the following articles: Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Error: Error: After correcting the problems, you can resume the build with the command Error:mvn -rf :pulsar Error: Process completed with exit code 1. ``` Apparently `kafka-clients-2.8.2` seems to have a high level security vulnerability. ### Modifications Upgraded `kafka-clients` to 3.4.0, the latest version. This is a major version upgrade, but I don't know if there are other things to fix. ### Verifying this change - [ ] Make sure that the change passes the CI checks. ### Documentation - [ ] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] nodece commented on pull request #18336: [feat][authentication] Add JWKS support for AuthenticationProviderToken
nodece commented on PR #18336: URL: https://github.com/apache/pulsar/pull/18336#issuecomment-1430776249 > #8152 asks for support to rotate public keys and to load new public keys "on the fly". This PR appears to add support for loading multiple public keys on start up, but it does not provide support for rotation or dynamic loading. Is that correct? If so, I don't think we should say this PR fixes #8152. You are right, I think we can listener to the file changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #18336: [feat][authentication] Add JWKS support for AuthenticationProviderToken
nodece commented on code in PR #18336:
URL: https://github.com/apache/pulsar/pull/18336#discussion_r1106660384
##
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java:
##
@@ -409,4 +445,134 @@ public String getHeader(String name) {
return super.getHeader(name);
}
}
+
+private static final class TokenSigningKeyResolver implements
SigningKeyResolver {
+private final JWK jwk;
+
+public TokenSigningKeyResolver(String data) {
+jwk = new JWK(data);
+}
+
+@Override
+public Key resolveSigningKey(JwsHeader header, Claims claims) {
+return jwk.get(header.getKeyId());
+}
+
+@Override
+public Key resolveSigningKey(JwsHeader header, String plaintext) {
+return jwk.get(header.getKeyId());
+}
+}
+
+// https://datatracker.ietf.org/doc/html/rfc7517
+@Slf4j
+private static final class JWK {
+private static final String ALGORITHM_RSA = "RSA";
+private static final String ALGORITHM_EC = "EC";
+
+private static final Map CURVE_MAP = new HashMap<>();
+
+static {
+//
https://openid.net/specs/draft-jones-json-web-key-03.html#anchor7
+//
https://docs.oracle.com/en/java/javase/17/docs/specs/security/standard-names.html#parameterspec-names
+CURVE_MAP.put("P-256", "secp256r1");
+CURVE_MAP.put("P-384", "secp384r1");
+CURVE_MAP.put("P-521", "secp521r1");
+}
+
+private final Map keyMap = new HashMap<>();
+
+public JWK(String data) {
+String json;
+try {
+byte[] bytes = AuthTokenUtils.readKeyFromUrl(data);
+if (bytes == null || bytes.length == 0) {
+throw new IOException("invalid JWKS");
+}
+json = new String(bytes, StandardCharsets.UTF_8);
+} catch (IOException e) {
+throw new IllegalArgumentException(e);
+}
+
+if (log.isDebugEnabled()) {
+log.debug("JWKS: {}", json);
+}
+
+JsonNode rootNode;
+try {
+rootNode = new ObjectMapper().readTree(json);
+} catch (IOException e) {
+throw new IllegalArgumentException(e);
+}
+
+if (rootNode == null) {
+return;
+}
+
+JsonNode keysNode = rootNode.get("keys");
+if (keysNode == null) {
+return;
+}
+
+Iterator elements = keysNode.elements();
+while (elements.hasNext()) {
+JsonNode node = elements.next();
+String type = node.get("kty").textValue();
+String kid = node.get("kid").textValue();
+KeyFactory kf;
+// Reference from:
+//
https://github.com/auth0/jwks-rsa-java/blob/0.21.2/src/main/java/com/auth0/jwk/Jwk.java#L176
Review Comment:
We just use the `Jwk` class, and don't want to introduce the other
dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
nodece commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106658206
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ServerCnxTest.java:
##
@@ -1030,6 +1030,55 @@ public void
testVerifyAuthRoleAndAuthDataFromDirectConnectionBroker() throws Exc
}));
}
+@Test
+public void testRefreshOriginalPrincipalWithAuthDataForwardedFromProxy()
throws Exception {
Review Comment:
For current design, it works fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
nodece commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106657809
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockAuthenticationProvider.java:
##
@@ -65,4 +71,41 @@ public String authenticate(AuthenticationDataSource
authData) throws Authenticat
"Not a valid principle. Should be
[pass|fail|error].[pass|fail|error], found " + principal);
}
+@Override
+public AuthenticationState newAuthState(AuthData authData, SocketAddress
remoteAddress, SSLSession sslSession)
+throws AuthenticationException {
+return new MockAuthState(this);
+}
+
+private static class MockAuthState implements AuthenticationState {
Review Comment:
Make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
nodece commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106656201
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -737,15 +737,15 @@ public void authChallengeSuccessCallback(AuthData
authChallenge,
// 2. an authentication refresh, in which case we need to
refresh authenticationData
AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String newAuthRole = authState.getAuthRole();
+AuthenticationDataSource newAuthDataSource =
authState.getAuthDataSource();
-// Refresh the auth data.
-this.authenticationData = authState.getAuthDataSource();
-if (log.isDebugEnabled()) {
-log.debug("[{}] Auth data refreshed for role={}",
remoteAddress, this.authRole);
-}
-
+// Refresh the auth data and role.
if (!useOriginalAuthState) {
this.authRole = newAuthRole;
+this.authenticationData = newAuthDataSource;
+} else {
+this.originalAuthData = newAuthDataSource;
+this.originalPrincipal = newAuthRole;
Review Comment:
> My primary concern about this approach is that it doesn't address the fact
that the `authenticationData` will never be updated after the initial handshake
when a proxy is involved.
Sure, but I'm doing this to fix the store of the authentication data
correctly, and make sense.
> I have been thinking it might make sense to get rid of the
`originalAuthData` field entirely because when it is set, we won't ever get the
proxy's auth data. The drawback of my approach is that it also means we would
skip authorization of the proxy's role. I personally think this is reasonable
because we can verify that the proxy is in the `proxyRoles` initially and then
not worry about the actual permission later.
It makes sense to remove 'originalAuthData' to make the authentication
process simpler.
Thank you for your review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #18336: [feat][authentication] Add JWKS support for AuthenticationProviderToken
michaeljmarshall commented on code in PR #18336:
URL: https://github.com/apache/pulsar/pull/18336#discussion_r1056636702
##
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java:
##
@@ -409,4 +445,134 @@ public String getHeader(String name) {
return super.getHeader(name);
}
}
+
+private static final class TokenSigningKeyResolver implements
SigningKeyResolver {
+private final JWK jwk;
+
+public TokenSigningKeyResolver(String data) {
+jwk = new JWK(data);
+}
+
+@Override
+public Key resolveSigningKey(JwsHeader header, Claims claims) {
+return jwk.get(header.getKeyId());
+}
+
+@Override
+public Key resolveSigningKey(JwsHeader header, String plaintext) {
+return jwk.get(header.getKeyId());
+}
+}
+
+// https://datatracker.ietf.org/doc/html/rfc7517
+@Slf4j
+private static final class JWK {
+private static final String ALGORITHM_RSA = "RSA";
+private static final String ALGORITHM_EC = "EC";
+
+private static final Map CURVE_MAP = new HashMap<>();
+
+static {
+//
https://openid.net/specs/draft-jones-json-web-key-03.html#anchor7
+//
https://docs.oracle.com/en/java/javase/17/docs/specs/security/standard-names.html#parameterspec-names
+CURVE_MAP.put("P-256", "secp256r1");
+CURVE_MAP.put("P-384", "secp384r1");
+CURVE_MAP.put("P-521", "secp521r1");
+}
+
+private final Map keyMap = new HashMap<>();
+
+public JWK(String data) {
+String json;
+try {
+byte[] bytes = AuthTokenUtils.readKeyFromUrl(data);
+if (bytes == null || bytes.length == 0) {
+throw new IOException("invalid JWKS");
+}
+json = new String(bytes, StandardCharsets.UTF_8);
+} catch (IOException e) {
+throw new IllegalArgumentException(e);
+}
+
+if (log.isDebugEnabled()) {
+log.debug("JWKS: {}", json);
+}
+
+JsonNode rootNode;
+try {
+rootNode = new ObjectMapper().readTree(json);
+} catch (IOException e) {
+throw new IllegalArgumentException(e);
+}
+
+if (rootNode == null) {
+return;
+}
+
+JsonNode keysNode = rootNode.get("keys");
+if (keysNode == null) {
+return;
+}
+
+Iterator elements = keysNode.elements();
+while (elements.hasNext()) {
+JsonNode node = elements.next();
+String type = node.get("kty").textValue();
+String kid = node.get("kid").textValue();
+KeyFactory kf;
+// Reference from:
+//
https://github.com/auth0/jwks-rsa-java/blob/0.21.2/src/main/java/com/auth0/jwk/Jwk.java#L176
Review Comment:
Is there a reason we cannot use this library directly instead of copying the
code here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19519: [fix][authentication] Store the original authentication data
michaeljmarshall commented on code in PR #19519:
URL: https://github.com/apache/pulsar/pull/19519#discussion_r1106643857
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockAuthenticationProvider.java:
##
@@ -65,4 +71,41 @@ public String authenticate(AuthenticationDataSource
authData) throws Authenticat
"Not a valid principle. Should be
[pass|fail|error].[pass|fail|error], found " + principal);
}
+@Override
+public AuthenticationState newAuthState(AuthData authData, SocketAddress
remoteAddress, SSLSession sslSession)
+throws AuthenticationException {
+return new MockAuthState(this);
+}
+
+private static class MockAuthState implements AuthenticationState {
Review Comment:
I think it might make sense to extend this `MockAuthenticationProvider` to
get the semantics for the `newAuthState`. I did that with
`MockAlwaysExpiredAuthenticationProvider` and with
`MockMultiStageAuthenticationProvider`.
My primary concern is that existing tests are likely verifying the behavior
of the `OneStageAuthenticationState` class, and changing this method might
affect those tests.
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -737,15 +737,15 @@ public void authChallengeSuccessCallback(AuthData
authChallenge,
// 2. an authentication refresh, in which case we need to
refresh authenticationData
AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String newAuthRole = authState.getAuthRole();
+AuthenticationDataSource newAuthDataSource =
authState.getAuthDataSource();
-// Refresh the auth data.
-this.authenticationData = authState.getAuthDataSource();
-if (log.isDebugEnabled()) {
-log.debug("[{}] Auth data refreshed for role={}",
remoteAddress, this.authRole);
-}
-
+// Refresh the auth data and role.
if (!useOriginalAuthState) {
this.authRole = newAuthRole;
+this.authenticationData = newAuthDataSource;
+} else {
+this.originalAuthData = newAuthDataSource;
+this.originalPrincipal = newAuthRole;
Review Comment:
My primary concern about this approach is that it doesn't address the fact
that the `authenticationData` will never be updated after the initial handshake
when a proxy is involved.
I documented some of the challenges here
https://github.com/apache/pulsar/issues/19332.
I have been thinking it might make sense to get rid of the
`originalAuthData` field entirely because when it is set, we won't ever get the
proxy's auth data. The drawback of my approach is that it also means we would
skip authorization of the proxy's role. I personally think this is reasonable
because we can verify that the proxy is in the `proxyRoles` initially and then
not worry about the actual permission later.
However, it could make sense to merge this PR and then implement my proposed
change later. I will spend some time thinking about this and provide more
feedback tomorrow.
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ServerCnxTest.java:
##
@@ -1030,6 +1030,55 @@ public void
testVerifyAuthRoleAndAuthDataFromDirectConnectionBroker() throws Exc
}));
}
+@Test
+public void testRefreshOriginalPrincipalWithAuthDataForwardedFromProxy()
throws Exception {
Review Comment:
Note that the tests that comment about
`https://github.com/apache/pulsar/issues/19332` should fail because they make
assertions on the current behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on pull request #19498: [fix][client] fix receive dumplicated messages after seek to a timestamp in MultiTo…
aloyszhang commented on PR #19498: URL: https://github.com/apache/pulsar/pull/19498#issuecomment-1430747609 cc @lhotari -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] momo-jun commented on a diff in pull request #406: [fix][doc] fix broken links
momo-jun commented on code in PR #406: URL: https://github.com/apache/pulsar-site/pull/406#discussion_r1106640713 ## docs/security-tls-authentication.md: ## @@ -136,7 +136,7 @@ var client = PulsarClient.Builder() ## Configure TLS authentication in CLI tools -[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md), and [`pulsar-client`](reference-cli-tools.md) use the `conf/client.conf` config file in a Pulsar installation. +[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](https://pulsar.apache.org/reference/#/next/pulsar-admin/), [`pulsar-perf`](https://pulsar.apache.org/reference/#/next/pulsar-perf), and [`pulsar-client`](https://pulsar.apache.org/reference/#/next/pulsar-client/) use the `conf/client.conf` config file in a Pulsar installation. Review Comment: Agreed. I will also double-check and fix all the broken links that have been warned during the local build for 2.8.x and later versions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] momo-jun commented on pull request #368: ci: warn on broken links
momo-jun commented on PR #368: URL: https://github.com/apache/pulsar-site/pull/368#issuecomment-1430740651 @tisonkun Thanks for enabling this feature! Since I've been fixing the reported broken links through https://github.com/apache/pulsar-site/pull/406, I will double-check and fix all the broken links that have been warned during the local build for 2.8.x and later versions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #19492: [Doc] Document PULSAR_CLIENT_CONF env variable. Missing TLS-related CLI arguments
Anonymitaet commented on issue #19492: URL: https://github.com/apache/pulsar/issues/19492#issuecomment-1430735017 @visortelle Thanks! This issue contains two types of improvements: ### Doc improvements ☘️ **TO-DOs** Add the `environment variables` section to the following pages. - Reference site - [`pulsar-perf` home page](https://pulsar.apache.org/reference/#/next/pulsar-perf/) - Reference site - [`pulsar-admin` home page](https://pulsar.apache.org/reference/#/next/pulsar-admin/) - Reference site - [`pulsar-client` home page](https://pulsar.apache.org/reference/#/next/pulsar-client/) ☘️ **Reasons** This section is shown on 2.10.x and earlier markdown docs (e.g., https://pulsar.apache.org/docs/2.10.x/reference-cli-tools/#pulsar-perf) but not available on 2.11.x and later Reference site. Since the reference pages of `pulsar-perf`, `pulsar-admin`, and `pulsar-client` are [generated automatically](https://pulsar.apache.org/contribute/document-contribution/#update-command-line-tool-docs), we can add the `environment variables` section to the their homepage as a temporary workaround. We can optimize it once we find better solutions. @momo-jun @DaveDuggins @D-2-Ed WDYT? ### Code improvements I've double-checked and tested the current code behavior and then recorded it in https://github.com/apache/pulsar/issues/19526. Feel free to correct me if I'm inaccurate, thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Anonymitaet opened a new issue, #19526: [improve][CLI] show environment variables in CLI and optimize them to `--config-file` argument
Anonymitaet opened a new issue, #19526: URL: https://github.com/apache/pulsar/issues/19526 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Motivation I've separated the code improvements from https://github.com/apache/pulsar/issues/19492 (created by @visortelle) and discussed them with @codelipenghui as below. ### 1. Show `enviroment variables` for `pulsar-admin` and `pulsar-client` on CLI Contexts: I've double-checked the current behavior: - `pulsar-perf -help` shows `environment variables` https://user-images.githubusercontent.com/50226895/218926299-9b56c9cb-b972-4208-8e90-29cdf3c1ec9e.png;> - while `pulsar-admin -help` and `pulsar-client` do not show `environment variables` https://user-images.githubusercontent.com/50226895/218927276-b72f5bae-58ef-4b59-a182-da9028e5fd80.png;> https://user-images.githubusercontent.com/50226895/218926938-bd543168-f46f-4c41-95a2-c183c6aeff0c.png;> To give users complete info,`environment variables` should be shown on `pulsar-admin` and `pulsar-client` CLI as well. ### 2. Implement `--config-file CLI` argument instead of `environment variables`. @visortelle has explained it in https://github.com/apache/pulsar/issues/19492 ### Solution _No response_ ### Alternatives _No response_ ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on pull request #19467: [improve][txn] Allow superusers to abort transactions
michaeljmarshall commented on PR #19467: URL: https://github.com/apache/pulsar/pull/19467#issuecomment-1430733126 > The last one is very interesting but I think we should do it for all the ServerCnx methods in one shot in another pull request. I completely agree, thanks for addressing my feedback. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19517: [fix][broker] Copy command fields and fix potential thread-safety in ServerCnx
michaeljmarshall commented on code in PR #19517:
URL: https://github.com/apache/pulsar/pull/19517#discussion_r1106630819
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -2555,7 +2556,7 @@ private CompletableFuture
verifyTxnOwnership(TxnID txnID) {
} else {
return CompletableFuture.completedFuture(false);
}
-});
+}, ctx.executor());
Review Comment:
I requested this change last week because `getAuthenticationData()` was
getting a non-volatile variable. However, before this PR was opened, I
contributed #19507. @nicoloboschi - we can probably remove this part of the
diff since reading the `authenticationData` is now thread safe.
In the long term, it might be worth exploring if we should only read this
metadata from the event loop, but for now, these variables are volatile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on issue #19134: Remove default 30s ackTimeout when using deadLetterPolicy
michaeljmarshall commented on issue #19134: URL: https://github.com/apache/pulsar/issues/19134#issuecomment-1430726479 @klevy-toast - that explanation makes sense and that behavior would definitely be problematic. Thanks for submitting the PR to fix it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch main updated: Document connecting through a proxy (#408)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new 348701a5566 Document connecting through a proxy (#408)
348701a5566 is described below
commit 348701a55662318f7c7ad59a818605e2ec8b4495
Author: Michael Marshall
AuthorDate: Tue Feb 14 22:04:31 2023 -0600
Document connecting through a proxy (#408)
* Add binary protocol doc for CommandConnect
* Document proxyRoles
Adding docs for apache/pulsar#19455
---
docs/developing-binary-protocol.md | 3 +++
docs/security-authorization.md | 2 ++
2 files changed, 5 insertions(+)
diff --git a/docs/developing-binary-protocol.md
b/docs/developing-binary-protocol.md
index bbbc70f9f27..798015f734f 100644
--- a/docs/developing-binary-protocol.md
+++ b/docs/developing-binary-protocol.md
@@ -142,6 +142,9 @@ Fields:
* `auth_method_name`: *(optional)* Name of the authentication plugin if auth
is enabled.
* `auth_data`: *(optional)* Plugin specific authentication data.
* `protocol_version`: Indicates the protocol version supported by the client.
Broker will not send commands introduced in newer revisions of the protocol.
Broker might be enforcing a minimum version.
+ * `original_principal`: Added by the proxy. Regular clients are not expected
to supply this value. When set and when authorization is enabled, the
`auth_data` must map to one of the `proxyRoles` in the broker.conf.
+ * `original_auth_method`: Added by the proxy. Regular clients are not
expected to supply this value.
+ * `original_auth_data`: Added by the proxy when configured to do so. Regular
clients are not expected to supply this value.
```protobuf
message CommandConnected {
diff --git a/docs/security-authorization.md b/docs/security-authorization.md
index a9c286b6a63..f11c85b165b 100644
--- a/docs/security-authorization.md
+++ b/docs/security-authorization.md
@@ -35,6 +35,8 @@ By default, the broker treats the connection between a proxy
and the broker as a
Pulsar uses *Proxy roles* to enable the authentication. Proxy roles are
specified in the broker configuration file,
[`conf/broker.conf`](reference-configuration.md#broker). If a client that is
authenticated with a broker is one of its `proxyRoles`, all requests from that
client must also carry information about the role of the client that is
authenticated with the proxy. This information is called the *original
principal*. If the *original principal* is absent, the client is not able to
[...]
+Note that if a Proxy is not correctly configured to use a role that is in the
`proxyRoles`, the connection will get rejected.
+
You must authorize both the *proxy role* and the *original principal* to
access a resource to ensure that the resource is accessible via the proxy.
Administrators can take two approaches to authorize the *proxy role* and the
*original principal*.
The more secure approach is to grant access to the proxy roles each time you
grant access to a resource. For example, if you have a proxy role named
`proxy1`, when the superuser creates a tenant, you should specify `proxy1` as
one of the admin roles. When a role is granted permission to produce or consume
from a namespace, if that client wants to produce or consume through a proxy,
you should also grant `proxy1` the same permissions.
[GitHub] [pulsar-site] michaeljmarshall merged pull request #408: Document connecting through a proxy
michaeljmarshall merged PR #408: URL: https://github.com/apache/pulsar-site/pull/408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.9 updated: [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.9
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.9 by this push:
new 612dc99b621 [fix][broker] ServerCnx broken after recent cherry-picks
(#19521)
612dc99b621 is described below
commit 612dc99b62122e78e31aa42980d8971de465b75b
Author: Michael Marshall
AuthorDate: Tue Feb 14 19:35:24 2023 -0600
[fix][broker] ServerCnx broken after recent cherry-picks (#19521)
I broke all release branches when I cherry picked
2847dd19f6e8a546f4d45bf51eb2b72aae0869ce to them. This change takes some of the
underlying logic from #19409, without taking the async logic.
* Make changes to `ServerCnx` to make tests pass
Tests are currently failing, so passing tests will show that this solution
is correct.
- [x] `doc-not-needed`
(cherry picked from commit 8246da282ca38e891bdf8a4e9abc47f640b22384)
(cherry picked from commit 15e4198e19ebb2045777c696ac39f969b2a57f66)
(cherry picked from commit 6132b46efae60d87979966aca075b80ab7e2a87d)
---
.../apache/pulsar/broker/service/ServerCnx.java| 49 +++---
1 file changed, 24 insertions(+), 25 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index 48e00fe4fd0..8b76e32362f 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -624,15 +624,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// complete the connect and sent newConnected command
private void completeConnect(int clientProtoVersion, String clientVersion)
{
-if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
-if (!isValidRoleAndOriginalPrincipal()) {
-state = State.Failed;
-service.getPulsarStats().recordConnectionCreateFail();
-final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
-
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
-return;
-}
-}
ctx.writeAndFlush(Commands.newConnected(clientProtoVersion,
maxMessageSize));
state = State.Connected;
service.getPulsarStats().recordConnectionCreateSuccess();
@@ -649,7 +640,8 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
}
// According to auth result, send newConnected or newAuthChallenge command.
-private State doAuthentication(AuthData clientData,
+private void doAuthentication(AuthData clientData,
+ boolean useOriginalAuthState,
int clientProtocolVersion,
String clientVersion) throws Exception {
@@ -657,8 +649,7 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// in presence of a proxy and if the proxy is forwarding the
credentials).
// In this case, the re-validation needs to be done against the
original client
// credentials.
-boolean useOriginalAuthState = (originalAuthState != null);
-AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
+AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String authRole = useOriginalAuthState ? originalPrincipal :
this.authRole;
AuthData brokerData = authState.authenticate(clientData);
@@ -691,6 +682,15 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
if (state != State.Connected) {
// First time authentication is done
+if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
+if (!isValidRoleAndOriginalPrincipal()) {
+state = State.Failed;
+service.getPulsarStats().recordConnectionCreateFail();
+final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
+
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
+return;
+}
+}
completeConnect(clientProtocolVersion, clientVersion);
} else {
// If the connection was already ready, it means we're doing a
refresh
@@ -704,18 +704,16 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
}
}
}
+} else {
-return State.Connected;
-}
-
[pulsar] branch branch-2.8 updated: [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.8 by this push:
new 1d7b8f1b320 [fix][broker] ServerCnx broken after recent cherry-picks
(#19521)
1d7b8f1b320 is described below
commit 1d7b8f1b3203f37d9c4c10e68f70f5176726cc4e
Author: Michael Marshall
AuthorDate: Tue Feb 14 19:35:24 2023 -0600
[fix][broker] ServerCnx broken after recent cherry-picks (#19521)
I broke all release branches when I cherry picked
2847dd19f6e8a546f4d45bf51eb2b72aae0869ce to them. This change takes some of the
underlying logic from #19409, without taking the async logic.
* Make changes to `ServerCnx` to make tests pass
Tests are currently failing, so passing tests will show that this solution
is correct.
- [x] `doc-not-needed`
(cherry picked from commit 8246da282ca38e891bdf8a4e9abc47f640b22384)
(cherry picked from commit 15e4198e19ebb2045777c696ac39f969b2a57f66)
(cherry picked from commit 6132b46efae60d87979966aca075b80ab7e2a87d)
---
.../apache/pulsar/broker/service/ServerCnx.java| 50 +++---
1 file changed, 25 insertions(+), 25 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index 9a55a728717..d44febb002e 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -604,15 +604,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// complete the connect and sent newConnected command
private void completeConnect(int clientProtoVersion, String clientVersion)
{
-if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
-if (!isValidRoleAndOriginalPrincipal()) {
-state = State.Failed;
-service.getPulsarStats().recordConnectionCreateFail();
-final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
-
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
-return;
-}
-}
ctx.writeAndFlush(Commands.newConnected(clientProtoVersion,
maxMessageSize));
state = State.Connected;
service.getPulsarStats().recordConnectionCreateSuccess();
@@ -626,7 +617,8 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
}
// According to auth result, send newConnected or newAuthChallenge command.
-private State doAuthentication(AuthData clientData,
+private void doAuthentication(AuthData clientData,
+ boolean useOriginalAuthState,
int clientProtocolVersion,
String clientVersion) throws Exception {
@@ -634,8 +626,7 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// in presence of a proxy and if the proxy is forwarding the
credentials).
// In this case, the re-validation needs to be done against the
original client
// credentials.
-boolean useOriginalAuthState = (originalAuthState != null);
-AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
+AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String authRole = useOriginalAuthState ? originalPrincipal :
this.authRole;
AuthData brokerData = authState.authenticate(clientData);
@@ -668,6 +659,15 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
if (state != State.Connected) {
// First time authentication is done
+if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
+if (!isValidRoleAndOriginalPrincipal()) {
+state = State.Failed;
+service.getPulsarStats().recordConnectionCreateFail();
+final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
+
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
+return;
+}
+}
completeConnect(clientProtocolVersion, clientVersion);
} else {
// If the connection was already ready, it means we're doing a
refresh
@@ -681,18 +681,16 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
}
}
}
+} else {
-return State.Connected;
-}
-
[pulsar] branch branch-2.10 updated: [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.10
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.10 by this push:
new 6132b46efae [fix][broker] ServerCnx broken after recent cherry-picks
(#19521)
6132b46efae is described below
commit 6132b46efae60d87979966aca075b80ab7e2a87d
Author: Michael Marshall
AuthorDate: Tue Feb 14 19:35:24 2023 -0600
[fix][broker] ServerCnx broken after recent cherry-picks (#19521)
I broke all release branches when I cherry picked
2847dd19f6e8a546f4d45bf51eb2b72aae0869ce to them. This change takes some of the
underlying logic from #19409, without taking the async logic.
* Make changes to `ServerCnx` to make tests pass
Tests are currently failing, so passing tests will show that this solution
is correct.
- [x] `doc-not-needed`
(cherry picked from commit 8246da282ca38e891bdf8a4e9abc47f640b22384)
(cherry picked from commit 15e4198e19ebb2045777c696ac39f969b2a57f66)
---
.../apache/pulsar/broker/service/ServerCnx.java| 49 +++---
1 file changed, 24 insertions(+), 25 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index be2386bd369..899bdd49626 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -626,15 +626,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// complete the connect and sent newConnected command
private void completeConnect(int clientProtoVersion, String clientVersion)
{
-if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
-if (!isValidRoleAndOriginalPrincipal()) {
-state = State.Failed;
-service.getPulsarStats().recordConnectionCreateFail();
-final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
-
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
-return;
-}
-}
ctx.writeAndFlush(Commands.newConnected(clientProtoVersion,
maxMessageSize));
state = State.Connected;
service.getPulsarStats().recordConnectionCreateSuccess();
@@ -651,7 +642,8 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
}
// According to auth result, send newConnected or newAuthChallenge command.
-private State doAuthentication(AuthData clientData,
+private void doAuthentication(AuthData clientData,
+ boolean useOriginalAuthState,
int clientProtocolVersion,
String clientVersion) throws Exception {
@@ -659,8 +651,7 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// in presence of a proxy and if the proxy is forwarding the
credentials).
// In this case, the re-validation needs to be done against the
original client
// credentials.
-boolean useOriginalAuthState = (originalAuthState != null);
-AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
+AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String authRole = useOriginalAuthState ? originalPrincipal :
this.authRole;
AuthData brokerData = authState.authenticate(clientData);
@@ -693,6 +684,15 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
if (state != State.Connected) {
// First time authentication is done
+if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
+if (!isValidRoleAndOriginalPrincipal()) {
+state = State.Failed;
+service.getPulsarStats().recordConnectionCreateFail();
+final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
+
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
+return;
+}
+}
completeConnect(clientProtocolVersion, clientVersion);
} else {
// If the connection was already ready, it means we're doing a
refresh
@@ -706,18 +706,16 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
}
}
}
+} else {
-return State.Connected;
-}
-
-// auth not complete, continue auth with client side.
-
[GitHub] [pulsar] Demogorgon314 opened a new pull request, #19525: [refactor][admin] Refactor namespace bundle transfer admin api
Demogorgon314 opened a new pull request, #19525: URL: https://github.com/apache/pulsar/pull/19525 ### Motivation PIP192 admin API also used `destinationBroker`, we need to refactor the `setNamespaceBundleAffinity` method to `setNamespaceBundleAffinityAsync`, so we can set use `destinationBroker` in the same method. ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` GitHub Actions provides separate quota for pull requests that are executed in a forked repository. The tests will be run in the forked repository until all PR review comments have been handled, the tests pass and the PR is approved by a reviewer. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Hongten commented on a diff in pull request #11184: [PIP-82] [pulsar-broker] Add publish ratelimiter to resource group
Hongten commented on code in PR #11184:
URL: https://github.com/apache/pulsar/pull/11184#discussion_r1106611240
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##
@@ -2199,6 +2199,14 @@ public void startSendOperation(Producer producer, int
msgSize, int numMessages)
}
isPublishRateExceeded =
producer.getTopic().isBrokerPublishRateExceeded();
} else {
+if (producer.getTopic().isResourceGroupRateLimitingEnabled()) {
Review Comment:
It seems if we set `preciseTopicPublishRateLimitingEnable=true`, then it
will disable the RG rate limiter. But this parameter doesn't provide a
description of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-client-python] tisonkun commented on issue #31: Invalid use of `type`s in error messages in `definition.py`
tisonkun commented on issue #31: URL: https://github.com/apache/pulsar-client-python/issues/31#issuecomment-1430691307 @erichare Thanks for your updates! Closing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-python] tisonkun closed issue #31: Invalid use of `type`s in error messages in `definition.py`
tisonkun closed issue #31: Invalid use of `type`s in error messages in `definition.py` URL: https://github.com/apache/pulsar-client-python/issues/31 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-python] erichare commented on issue #66: [feature request] Give a human readable description for MessageId
erichare commented on issue #66: URL: https://github.com/apache/pulsar-client-python/issues/66#issuecomment-1430688825 Merged in as of today! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-python] erichare commented on issue #31: Invalid use of `type`s in error messages in `definition.py`
erichare commented on issue #31: URL: https://github.com/apache/pulsar-client-python/issues/31#issuecomment-1430688499 @BewareMyPower I think we can close this now that it's been merged in. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #399: [doc][feat] Added doc to deploy a pulsar cluster on IBM cloud services
Anonymitaet commented on code in PR #399: URL: https://github.com/apache/pulsar-site/pull/399#discussion_r1106590631 ## versioned_docs/version-2.10.x/deploy-ibm.md: ## @@ -0,0 +1,237 @@ +--- +id: deploy-ibm +title: Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart +sidebar_label: "IBM Cloud Services" +original_id: deploy-ibm +--- +# Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart + +:::note + + This doc referes to the Apache pulsar 2.93 Version. if you want to upgrade the Apache Pulsar version you need to follow the [helm-upgrade](https://pulsar.apache.org/docs/2.10.x/helm-upgrade/) Document for upgrade on the perticular version. + +::: + + + +- [Setup a virtual machine (VM) on IBM Cloud.](#create-vm-on-ibm-cloud) +- [Create Kubernete cluster on IBM Cloud.](#create-kubernetes-cluster-on-ibm) +- [Prepare VM for connecting to Kubernetes cluster and deploy Pulsar Helm chart on Kubernetes cluster.](#prepare-vm-for-connecting-to-kubernetes-cluster-and-deploy-pulsar-helm-chart-on-kubernetes-cluster) +- [Verify the deployment.](#run-kubectl-commands-to-verify-the-deployment) + + + Create VM on IBM Cloud + +- Go to [IBM Cloud]( https://cloud.ibm.com/login) and login with your credentials. +- Search for Virtual Server. +- Select Virtual Server for Classic. + + + +- Select the type of virtual server as selected "Public" in the image. +- Type the host name, quantity of the machine and billing method. + + + +- Select location value according to your region. For example: In below image we have selected Chennai in Asia region. + + + +- Select the profile of virtual machine. + + + +- Select the operating system and version. + + + +- Select network interface according to use. + + + +- Select the security group. + + + +- Leave rest of the things could be default. Click on the "Create" button. + + + +- Check created VM in "Navigation Menu" -> "Resource list" Devices + + + + + +- Check the detail of VM in overview + + + +- Check the devices list, click on the menu option on the same page. + + + + Create Kubernetes Cluster on IBM + +- Search for the Kubernetes services + + + +- Select the plan details to "Standard". Please note "Free" plan has the limited resources, which can not fulfill the Pulsar cluster requirement. + + + +- Select the infrastructure type we are going with the classic and also select the Kubernetes version . + + + +- Select location and resource group. Select single single zone or multi zone as per your need. + + + + + +- Select worker pool size and flavor (vCPU, Memory) of the worker. + + + + + +- Set cluster name as you want. + + +- Leave rest of the things as we are selecting as default. You can disable below options. Click on create and wait for provisioning of the cluster. + + + +- After the cluster is successfully provisioned, connect to the cluster. In order to connect click on the "Action" button then click on "Connect via CLI", it will give you commands, copy that and run in your VM so that we can communicate to cluster through VM. We have to configure VM for communicating to cluster in the next steps will describe how to configure VM for that. + + + +- Check the created cluster list through clicking on the clusters options + + + +- It will show you the list of all the created clusters. + + + + Prepare VM for connecting to Kubernetes cluster and deploy Pulsar Helm chart on Kubernetes cluster. + +**Requirements** : +- Installation of [IBM Cloud CLI](https://cloud.ibm.com/docs/cli?topic=cli-install-ibmcloud-cli): For login in IBM cloud and for connection to Kubernetes master node. + +- Installation of [IBM Cloud CLI Plugins](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install) Required for connect to IKS (IBM
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #399: [doc][feat] Added doc to deploy a pulsar cluster on IBM cloud services
Anonymitaet commented on code in PR #399: URL: https://github.com/apache/pulsar-site/pull/399#discussion_r1106576582 ## versioned_docs/version-2.10.x/deploy-ibm.md: ## @@ -0,0 +1,237 @@ +--- +id: deploy-ibm +title: Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart +sidebar_label: "IBM Cloud Services" +original_id: deploy-ibm +--- +# Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart + +:::note + + This doc referes to the Apache pulsar 2.93 Version. if you want to upgrade the Apache Pulsar version you need to follow the [helm-upgrade](https://pulsar.apache.org/docs/2.10.x/helm-upgrade/) Document for upgrade on the perticular version. + +::: + + + +- [Setup a virtual machine (VM) on IBM Cloud.](#create-vm-on-ibm-cloud) +- [Create Kubernete cluster on IBM Cloud.](#create-kubernetes-cluster-on-ibm) +- [Prepare VM for connecting to Kubernetes cluster and deploy Pulsar Helm chart on Kubernetes cluster.](#prepare-vm-for-connecting-to-kubernetes-cluster-and-deploy-pulsar-helm-chart-on-kubernetes-cluster) +- [Verify the deployment.](#run-kubectl-commands-to-verify-the-deployment) + + + Create VM on IBM Cloud + +- Go to [IBM Cloud]( https://cloud.ibm.com/login) and login with your credentials. Review Comment: 1. From line 25 - 157, please use ordered lists instead of unordered lists to indicate the sequence. 2. Line 25 - 117 has not been reviewed because https://github.com/apache/pulsar-site/pull/399#discussion_r1106580020. We can review it later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #399: [doc][feat] Added doc to deploy a pulsar cluster on IBM cloud services
Anonymitaet commented on code in PR #399: URL: https://github.com/apache/pulsar-site/pull/399#discussion_r1106571571 ## versioned_docs/version-2.10.x/deploy-ibm.md: ## @@ -0,0 +1,237 @@ +--- +id: deploy-ibm +title: Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart +sidebar_label: "IBM Cloud Services" +original_id: deploy-ibm +--- +# Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart + +:::note + + This doc referes to the Apache pulsar 2.93 Version. if you want to upgrade the Apache Pulsar version you need to follow the [helm-upgrade](https://pulsar.apache.org/docs/2.10.x/helm-upgrade/) Document for upgrade on the perticular version. Review Comment: ```suggestion This tutorial uses Apache Pulsar 2.9.3 as an example. If you want to upgrade Pulsar version, follow the instructions in [Helm Upgrade Guide](https://pulsar.apache.org/docs/2.10.x/helm-upgrade/). ``` Do you intend to mean this? ## versioned_docs/version-2.10.x/deploy-ibm.md: ## @@ -0,0 +1,237 @@ +--- +id: deploy-ibm +title: Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart +sidebar_label: "IBM Cloud Services" +original_id: deploy-ibm +--- +# Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart + +:::note + + This doc referes to the Apache pulsar 2.93 Version. if you want to upgrade the Apache Pulsar version you need to follow the [helm-upgrade](https://pulsar.apache.org/docs/2.10.x/helm-upgrade/) Document for upgrade on the perticular version. + +::: + + Review Comment: ```suggestion Deploying a Pulsar cluster on IBM cloud consists of the following steps: ``` ## versioned_docs/version-2.10.x/deploy-ibm.md: ## @@ -0,0 +1,237 @@ +--- +id: deploy-ibm +title: Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart +sidebar_label: "IBM Cloud Services" +original_id: deploy-ibm +--- +# Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart + +:::note Review Comment: ```suggestion :::tip ``` It's a tip rather than note. ## versioned_docs/version-2.10.x/deploy-ibm.md: ## @@ -0,0 +1,237 @@ +--- +id: deploy-ibm +title: Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart +sidebar_label: "IBM Cloud Services" +original_id: deploy-ibm +--- +# Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart Review Comment: ```suggestion ``` Line 3 already shows the title ## versioned_docs/version-2.10.x/deploy-ibm.md: ## @@ -0,0 +1,237 @@ +--- +id: deploy-ibm +title: Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart +sidebar_label: "IBM Cloud Services" +original_id: deploy-ibm +--- +# Apache Pulsar Installation on IBM Kubernetes Cluster through Helm chart + +:::note + + This doc referes to the Apache pulsar 2.93 Version. if you want to upgrade the Apache Pulsar version you need to follow the [helm-upgrade](https://pulsar.apache.org/docs/2.10.x/helm-upgrade/) Document for upgrade on the perticular version. + +::: + + + +- [Setup a virtual machine (VM) on IBM Cloud.](#create-vm-on-ibm-cloud) +- [Create Kubernete cluster on IBM Cloud.](#create-kubernetes-cluster-on-ibm) +- [Prepare VM for connecting to Kubernetes cluster and deploy Pulsar Helm chart on Kubernetes cluster.](#prepare-vm-for-connecting-to-kubernetes-cluster-and-deploy-pulsar-helm-chart-on-kubernetes-cluster) +- [Verify the deployment.](#run-kubectl-commands-to-verify-the-deployment) + + + Create VM on IBM Cloud + +- Go to [IBM Cloud]( https://cloud.ibm.com/login) and login with your credentials. +- Search for Virtual Server. +- Select Virtual Server for Classic. + + + +- Select the type of virtual server as selected "Public" in the image. +- Type the host name, quantity of the machine and billing method. + + + +- Select location value according to your region. For example: In below image we have selected Chennai in Asia region. + + + +- Select the profile of virtual machine. + + + +- Select the operating system and version. + + + +- Select network interface according to use. + + + +- Select the security group. + + + +- Leave rest of the things could be default. Click on the "Create" button. + + + +- Check created VM in "Navigation Menu" -> "Resource list" Devices + + + + + +- Check the detail of VM in overview + + + +- Check the devices list, click on
[GitHub] [pulsar-site] momo-jun commented on a diff in pull request #386: [feat][doc] Add docs for message dispatch throttling
momo-jun commented on code in PR #386: URL: https://github.com/apache/pulsar-site/pull/386#discussion_r1106598026 ## docs/concepts-throttling.md: ## @@ -0,0 +1,167 @@ +--- +id: concepts-throttling +title: Message dispatch throttling +sidebar_label: "Message throttling" +--- + +## Overview + +### What is message dispatch throttling? + +Large message payloads can cause memory usage spikes that lead to performance decreases. Pulsar adopts a rate-limit throttling mechanism for message dispatch, avoiding a traffic surge and improving message deliverability. You can set a threshold to limit the number of messages and the byte size of entries that can be delivered to clients, blocking subsequent deliveries when the traffic per unit of time exceeds the threshold. + +For example, when you configure the dispatch rate limit to 10 messages per second, then the number of messages that can be delivered to the client per second is up to 10. + + + +### Why use it? + +Message dispatch throttling brings the following benefits in detail: + +- **Limit broker’s read request loads to BookKeeper** + + Messages are persistently stored in the BookKeeper cluster. If a large number of read requests cannot be fulfilled using the cached data, the BookKeeper cluster may become too busy to respond, and the broker's I/O or CPU resources can be fully occupied. Using the message dispatch throttling feature can regulate the data flow by limiting the broker’s I/O and CPU load, as well as BookKeeper’s read request load. + +- **Balance the allocation of broker’s hardware resources at topic/subscription levels** + + A broker instance serves multiple topics at one time. If a topic is frequently looked up, it will occupy almost all of the I/O, CPU, and memory resources, causing other topics cannot be looked up. Using the message dispatch throttling feature can limit the allocation of broker’s hardware resources across topics. + +- **Limit the allocation of client’s hardware resources at topic/subscription levels** + + When there is a large backlog of messages to consume, clients may receive a large amount of data in a short period of time, which monopolizes their computing resources. Since the client has no mechanisms to proactively limit the consumption rate, using the message dispatch throttling feature can also regulate the allocation of the client’s hardware resources. Review Comment: @poorbarcode can you pls take a look at this question? ## docs/concepts-throttling.md: ## @@ -0,0 +1,167 @@ +--- +id: concepts-throttling +title: Message dispatch throttling +sidebar_label: "Message throttling" +--- + +## Overview + +### What is message dispatch throttling? + +Large message payloads can cause memory usage spikes that lead to performance decreases. Pulsar adopts a rate-limit throttling mechanism for message dispatch, avoiding a traffic surge and improving message deliverability. You can set a threshold to limit the number of messages and the byte size of entries that can be delivered to clients, blocking subsequent deliveries when the traffic per unit of time exceeds the threshold. + +For example, when you configure the dispatch rate limit to 10 messages per second, then the number of messages that can be delivered to the client per second is up to 10. + + + +### Why use it? + +Message dispatch throttling brings the following benefits in detail: + +- **Limit broker's read request loads to BookKeeper** + + Messages are persistently stored in the BookKeeper cluster. If a large number of read requests cannot be fulfilled using the cached data, the BookKeeper cluster may become too busy to respond, and the broker's I/O or CPU resources can be fully occupied. Using the message dispatch throttling feature can regulate the data flow to limit the broker’s read request loads to BookKeeper. + +- **Balance the allocation of broker's hardware resources at topic/subscription levels** + + A broker instance serves multiple topics at one time. If a topic is overloaded with requests, it will occupy almost all of the I/O, CPU, and memory resources of the broker, causing other topics cannot be read. Using the message dispatch throttling feature can limit the allocation of broker’s hardware resources across topics. + +- **Limit the allocation of client's hardware resources at topic/subscription levels** + + When there is a large backlog of messages to consume, clients may receive a large amount of data in a short period of time, which monopolizes their computing resources. Since the client has no mechanisms to proactively limit the consumption rate, using the message dispatch throttling feature can also regulate the allocation of the client's hardware resources. + +### How it works? + +The process of message dispatch throttling can be divided
[GitHub] [pulsar] massakam opened a new pull request, #19524: [fix][client] Add explicit dependency on athenz-cert-refresher
massakam opened a new pull request, #19524: URL: https://github.com/apache/pulsar/pull/19524 ### Motivation In https://github.com/apache/pulsar/pull/19445, I added import statements for the `com.oath.auth.*` classes to the `AuthenticationAthenz` class. These classes are in a package named `athenz-cert-refresher`, so we should add a dependency on this package. However, due to the indirect dependency, `athenz-cert-refresher` will be installed without doing this and the build will not fail. ### Verifying this change - [ ] Make sure that the change passes the CI checks. ### Documentation - [ ] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] momo-jun commented on a diff in pull request #386: [feat][doc] Add docs for message dispatch throttling
momo-jun commented on code in PR #386: URL: https://github.com/apache/pulsar-site/pull/386#discussion_r1106595382 ## docs/concepts-throttling.md: ## @@ -0,0 +1,167 @@ +--- +id: concepts-throttling +title: Message dispatch throttling +sidebar_label: "Message throttling" +--- + +## Overview + +### What is message dispatch throttling? + +Large message payloads can cause memory usage spikes that lead to performance decreases. Pulsar adopts a rate-limit throttling mechanism for message dispatch, avoiding a traffic surge and improving message deliverability. You can set a threshold to limit the number of messages and the byte size of entries that can be delivered to clients, blocking subsequent deliveries when the traffic per unit of time exceeds the threshold. + +For example, when you configure the dispatch rate limit to 10 messages per second, then the number of messages that can be delivered to the client per second is up to 10. + + + +### Why use it? + +Message dispatch throttling brings the following benefits in detail: + +- **Limit broker's read request loads to BookKeeper** + + Messages are persistently stored in the BookKeeper cluster. If a large number of read requests cannot be fulfilled using the cached data, the BookKeeper cluster may become too busy to respond, and the broker's I/O or CPU resources can be fully occupied. Using the message dispatch throttling feature can regulate the data flow to limit the broker’s read request loads to BookKeeper. + +- **Balance the allocation of broker's hardware resources at topic/subscription levels** + + A broker instance serves multiple topics at one time. If a topic is overloaded with requests, it will occupy almost all of the I/O, CPU, and memory resources of the broker, causing other topics cannot be read. Using the message dispatch throttling feature can limit the allocation of broker’s hardware resources across topics. + +- **Limit the allocation of client's hardware resources at topic/subscription levels** + + When there is a large backlog of messages to consume, clients may receive a large amount of data in a short period of time, which monopolizes their computing resources. Since the client has no mechanisms to proactively limit the consumption rate, using the message dispatch throttling feature can also regulate the allocation of the client's hardware resources. + +### How it works? + +The process of message dispatch throttling can be divided into the following steps: +1. The broker approximates the number of entries to read from the bookies by calculating the remaining quota. +2. The broker reads the messages from the bookies. +3. The broker dispatches the messages to the client and updates the counter to decrease the quota. A scheduled task refreshes the quota when a throttling period ends. + +:::note + +- The quota cannot be decreased before step 3, because the broker doesn't know the actual number of messages per entry or the actual entry size until it reads the data. +- Operations like `seek` or `redeliver` may deliver messages to a client multiple times. The broker counts them as different messages and updates the counter. + +::: + +## Concepts + +### Throttling levels + +The following table outlines the three levels that you can throttle message dispatch. + +Level | Description +:-|: +Per broker | All subscriptions in a single broker share the quota. +Per topic | All subscriptions in the same topic share the quota.If it's a non-partitioned topic, the quota equals the maximum number of messages the topic can deliver per unit of time.If a topic has multiple partitions, the quota refers to the maximum number of messages each partition can deliver per unit of time. In other words, the actual dispatch rate limit of a [partitioned topic](concepts-messaging.md#partitioned-topics) is N times the configured one (N is the number of partitions inside the topic). For example, the topic `t0` has two partitions. If you set the quota to `10/s`, then the rate limit of both `t0-p0` and `t0-p1` is `10/s`, while the total rate limit of `t0` is `20/s`. Note that the quota cannot be shared among partitions, but can be shared among subscriptions inside a partition. +Per subscription | If it's a non-partitioned topic, the rate limit refers to the maximum number of messages a subscription can deliver to clients per unit of time.If the subscribed topic has multiple partitions, the rate limit refers to the maximum number of messages the subscription can deliver per partition per unit of time. In other words, a subscription's actual dispatch rate limit for a [partitioned topic](concepts-messaging.md#partitioned-topics) is N times the configured one (N is the number of partitions inside the topic). For example, the topic `t1` has two partitions with the subscription `s1`. If you set the rate limit to `10/s`, then the
[GitHub] [pulsar-client-python] BewareMyPower merged pull request #93: #66: Add Human-readable description of MessageId object
BewareMyPower merged PR #93: URL: https://github.com/apache/pulsar-client-python/pull/93 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-client-python] branch main updated: Add Human-readable description of MessageId object (#93)
This is an automated email from the ASF dual-hosted git repository.
xyz pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-client-python.git
The following commit(s) were added to refs/heads/main by this push:
new ed6983b Add Human-readable description of MessageId object (#93)
ed6983b is described below
commit ed6983b6f0575e9807c5386891edf20de914a662
Author: Eric Hare
AuthorDate: Tue Feb 14 19:45:29 2023 -0700
Add Human-readable description of MessageId object (#93)
---
src/message.cc | 6 ++
1 file changed, 6 insertions(+)
diff --git a/src/message.cc b/src/message.cc
index 1924bc2..6e8dd3f 100644
--- a/src/message.cc
+++ b/src/message.cc
@@ -55,6 +55,12 @@ void export_message(py::module_& m) {
oss << msgId;
return oss.str();
})
+.def("__repr__",
+ [](const MessageId& msgId) {
+ std::ostringstream oss;
+ oss << msgId;
+ return oss.str();
+ })
.def("__eq__", ::operator==)
.def("__ne__", ::operator!=)
.def("__le__", ::operator<=)
[GitHub] [pulsar] horizonzy commented on issue #19466: [Doc] management API docs misleading and unclear re: namespace properties
horizonzy commented on issue #19466: URL: https://github.com/apache/pulsar/issues/19466#issuecomment-1430672934 We should modify both the code and the doc. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- closed pull request #19514: [fix] [admin] Admin API can not work if uri too large
Technoboy- closed pull request #19514: [fix] [admin] Admin API can not work if uri too large URL: https://github.com/apache/pulsar/pull/19514 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- closed issue #16029: Help!We have some questions about flink-connector-pulsar.
Technoboy- closed issue #16029: Help!We have some questions about flink-connector-pulsar. URL: https://github.com/apache/pulsar/issues/16029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on issue #16029: Help!We have some questions about flink-connector-pulsar.
Technoboy- commented on issue #16029: URL: https://github.com/apache/pulsar/issues/16029#issuecomment-1430662013 Fixed by https://github.com/apache/pulsar/pull/16072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Hongten commented on a diff in pull request #11184: [PIP-82] [pulsar-broker] Add publish ratelimiter to resource group
Hongten commented on code in PR #11184: URL: https://github.com/apache/pulsar/pull/11184#discussion_r1106577833 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/resourcegroup/ResourceGroup.java: ## @@ -78,6 +80,9 @@ protected ResourceGroup(ResourceGroupService rgs, String name, this.setResourceGroupMonitoringClassFields(); this.setResourceGroupConfigParameters(rgConfig); this.setDefaultResourceUsageTransportHandlers(); +this.resourceGroupPublishLimiter = new ResourceGroupPublishLimiter(rgConfig, rgs.getPulsar().getExecutor()); Review Comment: Another question: There is only `PublishLimiter`. It seems we need another limiter called `DispatchLimiter`. right? I wonder, do we use `PIP 3: Message dispatch throttling` as `DispatchLimiter`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Hongten commented on a diff in pull request #11184: [PIP-82] [pulsar-broker] Add publish ratelimiter to resource group
Hongten commented on code in PR #11184: URL: https://github.com/apache/pulsar/pull/11184#discussion_r1106577833 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/resourcegroup/ResourceGroup.java: ## @@ -78,6 +80,9 @@ protected ResourceGroup(ResourceGroupService rgs, String name, this.setResourceGroupMonitoringClassFields(); this.setResourceGroupConfigParameters(rgConfig); this.setDefaultResourceUsageTransportHandlers(); +this.resourceGroupPublishLimiter = new ResourceGroupPublishLimiter(rgConfig, rgs.getPulsar().getExecutor()); Review Comment: Another question: There is only `PublishLimiter`. It seems we need another limiter called `DispatchLimiter`. right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Hongten commented on a diff in pull request #11184: [PIP-82] [pulsar-broker] Add publish ratelimiter to resource group
Hongten commented on code in PR #11184: URL: https://github.com/apache/pulsar/pull/11184#discussion_r1106575873 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/resourcegroup/ResourceGroup.java: ## @@ -78,6 +80,9 @@ protected ResourceGroup(ResourceGroupService rgs, String name, this.setResourceGroupMonitoringClassFields(); this.setResourceGroupConfigParameters(rgConfig); this.setDefaultResourceUsageTransportHandlers(); +this.resourceGroupPublishLimiter = new ResourceGroupPublishLimiter(rgConfig, rgs.getPulsar().getExecutor()); Review Comment: The default thread pool(pulsar executor) size is 20. However, as the ResourceGroup number increases it gives pressure on the thread pool to process. Is it better to provide a new thread pool here to process? It means one RG has one thread pool. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-manager] doug-ba commented on issue #505: Starting Docker container pulsar-manager-0.3.0 with internal PostreSQL does not work
doug-ba commented on issue #505: URL: https://github.com/apache/pulsar-manager/issues/505#issuecomment-1430648414 @tuteng , sorry to ping you directly, but it looks like you push the container images. Is there any chance of getting a new version of the pulsar-manager with the fix for this pushed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #19122: no org_apache_pulsar_shade_netty_tcnative_x86_64 in java.library.path
github-actions[bot] commented on issue #19122: URL: https://github.com/apache/pulsar/issues/19122#issuecomment-1430648026 The issue had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #19169: [Bug][txn] exclusive subscription would lead to high cpu usage when do tailing-read
github-actions[bot] commented on issue #19169: URL: https://github.com/apache/pulsar/issues/19169#issuecomment-1430647965 The issue had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #19466: [Doc] management API docs misleading and unclear re: namespace properties
Anonymitaet commented on issue #19466: URL: https://github.com/apache/pulsar/issues/19466#issuecomment-1430644595 Hi @horizonzy can you please double-check this change from the technical perspective? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-manager] doug-ba commented on issue #505: Starting Docker container pulsar-manager-0.3.0 with internal PostreSQL does not work
doug-ba commented on issue #505: URL: https://github.com/apache/pulsar-manager/issues/505#issuecomment-1430639363 @splio-rbechon , the fix was committed in August of 2022. Is there any chance of getting a new container image release? The last release is effectively broken. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] shy-share commented on issue #19272: Facilitate Pulsar Shell download and installation
shy-share commented on issue #19272: URL: https://github.com/apache/pulsar/issues/19272#issuecomment-1430638073 @nicoloboschi now only the msi for windows is missing? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #19512: [improve][doc] flags (parameters) of `pulsar-daemon start` are missing in docs
Anonymitaet commented on issue #19512: URL: https://github.com/apache/pulsar/issues/19512#issuecomment-1430634240 @horizonzy thanks for your confirmation! > The` link is not match the content. When providing links to the Pulsar Reference site, we suggest using the general link (https://pulsar.apache.org/reference) instead of specific links (e.g., https://pulsar.apache.org/reference/#/next/pulsar/standalone) for all versioned docs to reduce maintenance costs. Readers can choose their desired pages after they are navigated to the home page. We apply this strategy in our daily work to keep consistent, e.g., https://github.com/apache/pulsar/issues/19491#issuecomment-1428991305. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] tisonkun closed issue #19511: [Doc] ( or ] ？
tisonkun closed issue #19511: [Doc] ( or ] ？ URL: https://github.com/apache/pulsar/issues/19511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch main updated: Fix the problem of the left and right boundary of the value (#409)
This is an automated email from the ASF dual-hosted git repository. tison pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new aa9506f97b6 Fix the problem of the left and right boundary of the value (#409) aa9506f97b6 is described below commit aa9506f97b6eb8a143dd48e56279b2277decd4fa Author: sunheyi <50973219+shy-sh...@users.noreply.github.com> AuthorDate: Wed Feb 15 09:36:52 2023 +0800 Fix the problem of the left and right boundary of the value (#409) --- docs/concepts-messaging.md | 4 ++-- versioned_docs/version-2.11.x/concepts-messaging.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/concepts-messaging.md b/docs/concepts-messaging.md index 4dbed234670..11fa5f009f2 100644 --- a/docs/concepts-messaging.md +++ b/docs/concepts-messaging.md @@ -667,8 +667,8 @@ Example: Suppose we have 2 consumers (C1 and C2) each specified their ranges, then: ``` -C1 = [0, 16384), (32768, 49152] -C2 = [16384, 32768), (49,152, 65536] +C1 = [0, 16384), [32768, 49152) +C2 = [16384, 32768), [49152, 65536) 0 16,38432,768 49,152 65,536 |--- C1 --|--- C2 --|--- C1 --|--- C2 --| diff --git a/versioned_docs/version-2.11.x/concepts-messaging.md b/versioned_docs/version-2.11.x/concepts-messaging.md index 764cf3aa757..a1dc0f002cb 100644 --- a/versioned_docs/version-2.11.x/concepts-messaging.md +++ b/versioned_docs/version-2.11.x/concepts-messaging.md @@ -722,8 +722,8 @@ Example: Suppose we have 2 consumers (C1 and C2) each specified their ranges, then: ``` -C1 = [0, 16384), (32768, 49152] -C2 = [16384, 32768), (49,152, 65536] +C1 = [0, 16384), [32768, 49152) +C2 = [16384, 32768), [49152, 65536) 0 16,38432,768 49,152 65,536 |--- C1 --|--- C2 --|--- C1 --|--- C2 --|
[GitHub] [pulsar-site] tisonkun merged pull request #409: Fix the problem of the left and right boundary of the value
tisonkun merged PR #409: URL: https://github.com/apache/pulsar-site/pull/409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.11 updated: [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.11
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.11 by this push:
new 8246da282ca [fix][broker] ServerCnx broken after recent cherry-picks
(#19521)
8246da282ca is described below
commit 8246da282ca38e891bdf8a4e9abc47f640b22384
Author: Michael Marshall
AuthorDate: Tue Feb 14 19:35:24 2023 -0600
[fix][broker] ServerCnx broken after recent cherry-picks (#19521)
### Motivation
I broke all release branches when I cherry picked
2847dd19f6e8a546f4d45bf51eb2b72aae0869ce to them. This change takes some of the
underlying logic from #19409, without taking the async logic.
### Modifications
* Make changes to `ServerCnx` to make tests pass
### Verifying this change
Tests are currently failing, so passing tests will show that this solution
is correct.
### Documentation
- [x] `doc-not-needed`
---
.../apache/pulsar/broker/service/ServerCnx.java| 51 +++---
1 file changed, 25 insertions(+), 26 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index 6d0473ed99c..a297b437b1f 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -634,16 +634,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// complete the connect and sent newConnected command
private void completeConnect(int clientProtoVersion, String clientVersion,
boolean supportsTopicWatchers) {
-if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
-if (!service.getAuthorizationService()
-.isValidOriginalPrincipal(authRole, originalPrincipal,
remoteAddress)) {
-state = State.Failed;
-service.getPulsarStats().recordConnectionCreateFail();
-final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
-
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
-return;
-}
-}
ctx.writeAndFlush(Commands.newConnected(clientProtoVersion,
maxMessageSize, supportsTopicWatchers));
state = State.Connected;
service.getPulsarStats().recordConnectionCreateSuccess();
@@ -660,7 +650,8 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
}
// According to auth result, send newConnected or newAuthChallenge command.
-private State doAuthentication(AuthData clientData,
+private void doAuthentication(AuthData clientData,
+ boolean useOriginalAuthState,
int clientProtocolVersion,
String clientVersion) throws Exception {
@@ -668,8 +659,7 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
// in presence of a proxy and if the proxy is forwarding the
credentials).
// In this case, the re-validation needs to be done against the
original client
// credentials.
-boolean useOriginalAuthState = (originalAuthState != null);
-AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
+AuthenticationState authState = useOriginalAuthState ?
originalAuthState : this.authState;
String authRole = useOriginalAuthState ? originalPrincipal :
this.authRole;
AuthData brokerData = authState.authenticate(clientData);
@@ -702,6 +692,16 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
if (state != State.Connected) {
// First time authentication is done
+if (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()) {
+if (!service.getAuthorizationService()
+.isValidOriginalPrincipal(this.authRole,
originalPrincipal, remoteAddress)) {
+state = State.Failed;
+service.getPulsarStats().recordConnectionCreateFail();
+final ByteBuf msg = Commands.newError(-1,
ServerError.AuthorizationError, "Invalid roles.");
+
ctx.writeAndFlush(msg).addListener(ChannelFutureListener.CLOSE);
+return;
+}
+}
completeConnect(clientProtocolVersion, clientVersion,
enableSubscriptionPatternEvaluation);
} else {
// If the connection was already ready, it means we're doing a
refresh
@@ -715,18 +715,16 @@ public class
[GitHub] [pulsar] michaeljmarshall merged pull request #19521: [fix][broker] ServerCnx broken after recent cherry-picks
michaeljmarshall merged PR #19521: URL: https://github.com/apache/pulsar/pull/19521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on pull request #19521: [fix][broker] ServerCnx broken after recent cherry-picks
michaeljmarshall commented on PR #19521: URL: https://github.com/apache/pulsar/pull/19521#issuecomment-1430625487 The errors are unrelated to the recent cherry picks, I am going to merge this now to unblock the branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on issue #19523: Flaky-test: BrokerServiceTest.testGetTopic (branch-2.11)
michaeljmarshall commented on issue #19523: URL: https://github.com/apache/pulsar/issues/19523#issuecomment-1430625115 Seen on https://github.com/apache/pulsar/pull/19521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall opened a new issue, #19523: Flaky-test: BrokerServiceTest.testGetTopic (branch-2.11)
michaeljmarshall opened a new issue, #19523: URL: https://github.com/apache/pulsar/issues/19523 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Example failure https://github.com/apache/pulsar/actions/runs/4178889437/jobs/7238386249 ### Exception stacktrace ``` Error: Tests run: 50, Failures: 1, Errors: 0, Skipped: 37, Time elapsed: 110.274 s <<< FAILURE! - in org.apache.pulsar.broker.service.BrokerServiceTest Error: testGetTopic(org.apache.pulsar.broker.service.BrokerServiceTest) Time elapsed: 0.243 s <<< FAILURE! java.lang.AssertionError: expected [0] but found [1] at org.testng.Assert.fail(Assert.java:99) at org.testng.Assert.failNotEquals(Assert.java:1037) at org.testng.Assert.assertEqualsImpl(Assert.java:140) at org.testng.Assert.assertEquals(Assert.java:122) at org.testng.Assert.assertEquals(Assert.java:907) at org.testng.Assert.assertEquals(Assert.java:917) at org.apache.pulsar.broker.service.BrokerServiceTest.testGetTopic(BrokerServiceTest.java:1492) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132) at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45) at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73) at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) ``` ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] startjava added a comment to the discussion: what mean Key_Shared about "STICKY"?
GitHub user startjava added a comment to the discussion: what mean Key_Shared about "STICKY"? i see only can custom consumer self hash range，but i no see where embody about STICKY GitHub link: https://github.com/apache/pulsar/discussions/19522#discussioncomment-4977125 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
[pulsar] branch branch-2.10 updated: [fix][security] Fix secure problem CVE-2017-1000487 (#19479)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.10
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.10 by this push:
new fb5477b7d8b [fix][security] Fix secure problem CVE-2017-1000487
(#19479)
fb5477b7d8b is described below
commit fb5477b7d8b00bba6715ac9ad8cdd059baa6c92e
Author: ran
AuthorDate: Wed Feb 15 09:08:47 2023 +0800
[fix][security] Fix secure problem CVE-2017-1000487 (#19479)
---
pulsar-sql/pom.xml | 1 +
pulsar-sql/presto-distribution/LICENSE | 2 +-
pulsar-sql/presto-distribution/pom.xml | 9 +
pulsar-sql/presto-pulsar/pom.xml | 11 +++
4 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/pulsar-sql/pom.xml b/pulsar-sql/pom.xml
index b1c33ae5edd..a3826f3eceb 100644
--- a/pulsar-sql/pom.xml
+++ b/pulsar-sql/pom.xml
@@ -43,6 +43,7 @@
3.14.9
1.17.2
+3.0.16
diff --git a/pulsar-sql/presto-distribution/LICENSE
b/pulsar-sql/presto-distribution/LICENSE
index 53260fba017..320a98d52ae 100644
--- a/pulsar-sql/presto-distribution/LICENSE
+++ b/pulsar-sql/presto-distribution/LICENSE
@@ -377,7 +377,7 @@ The Apache Software License, Version 2.0
- plexus-container-default-1.5.5.jar
- plexus-interpolation-1.14.jar
- plexus-sec-dispatcher-1.3.jar
-- plexus-utils-2.0.6.jar
+- plexus-utils-3.0.16.jar
* Apache XBean :: Reflect
- xbean-reflect-3.4.jar
* Avro
diff --git a/pulsar-sql/presto-distribution/pom.xml
b/pulsar-sql/presto-distribution/pom.xml
index 6ec1bffd417..94ebf0ab366 100644
--- a/pulsar-sql/presto-distribution/pom.xml
+++ b/pulsar-sql/presto-distribution/pom.xml
@@ -77,6 +77,11 @@
${jersey.version}
+
+ org.codehaus.plexus
+ plexus-utils
+ ${plexus.version}
+
io.prestosql
presto-main
@@ -99,6 +104,10 @@
com.google.inject.extensions
guice-multibindings
+
+ org.codehaus.plexus
+ plexus-utils
+
diff --git a/pulsar-sql/presto-pulsar/pom.xml b/pulsar-sql/presto-pulsar/pom.xml
index d9e567deb60..cbf23c4aed6 100644
--- a/pulsar-sql/presto-pulsar/pom.xml
+++ b/pulsar-sql/presto-pulsar/pom.xml
@@ -113,11 +113,22 @@
${javax.annotation-api.version}
+
+org.codehaus.plexus
+plexus-utils
+${plexus.version}
+
io.prestosql
presto-main
${presto.version}
test
+
+
+org.codehaus.plexus
+plexus-utils
+
+
[GitHub] [pulsar] codelipenghui merged pull request #19479: [fix][security] Fix secure problem CVE-2017-1000487
codelipenghui merged PR #19479: URL: https://github.com/apache/pulsar/pull/19479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] tisonkun closed issue #5177: function querystate command causes the number of broker threads to increase
tisonkun closed issue #5177: function querystate command causes the number of broker threads to increase URL: https://github.com/apache/pulsar/issues/5177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] tisonkun commented on issue #5177: function querystate command causes the number of broker threads to increase
tisonkun commented on issue #5177: URL: https://github.com/apache/pulsar/issues/5177#issuecomment-1430606252 Closed as stale. Please open a new issue if it's still relevant to the maintained versions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] startjava commented on issue #19511: [Doc] ( or ] ？
startjava commented on issue #19511: URL: https://github.com/apache/pulsar/issues/19511#issuecomment-1430605937 49152） [49152 but now two ") " @tisonkun edit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] shy-share commented on issue #19511: [Doc] ( or ] ？
shy-share commented on issue #19511: URL: https://github.com/apache/pulsar/issues/19511#issuecomment-1430604129 > number 49152 is right ?? @startjava yes, what do you think is wrong? what should be -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] tisonkun commented on a diff in pull request #409: Fix the problem of the left and right boundary of the value
tisonkun commented on code in PR #409: URL: https://github.com/apache/pulsar-site/pull/409#discussion_r1106542455 ## docs/concepts-messaging.md: ## @@ -667,8 +667,8 @@ Example: Suppose we have 2 consumers (C1 and C2) each specified their ranges, then: ``` -C1 = [0, 16384), (32768, 49152] -C2 = [16384, 32768), (49,152, 65536] +C1 = [0, 16384), [32768, 49152) +C2 = [16384, 32768), (49152, 65536] Review Comment: ```suggestion C2 = [16384, 32768), [49152, 65536) ``` ditto below. cc @Technoboy- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] startjava commented on issue #19511: [Doc] ( or ] ？
startjava commented on issue #19511: URL: https://github.com/apache/pulsar/issues/19511#issuecomment-1430597599 @shy-share number 49152 is right ?? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall opened a new pull request, #19521: [fix][broker] ServerCnx broken after recent cherry-picks
michaeljmarshall opened a new pull request, #19521: URL: https://github.com/apache/pulsar/pull/19521 ### Motivation I broke all release branches when I cherry picked 2847dd19f6e8a546f4d45bf51eb2b72aae0869ce to them. This change takes some of the underlying logic from #19409, without taking the async logic. ### Modifications * Make changes to `ServerCnx` to make tests pass ### Verifying this change Tests are currently failing, so passing tests will show that this solution is correct. ### Documentation - [x] `doc-not-needed` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] heesung-sn closed pull request #19490: [improve][broker] PIP-192 Use msg property to mark compacted msgs from strategic compaction
heesung-sn closed pull request #19490: [improve][broker] PIP-192 Use msg property to mark compacted msgs from strategic compaction URL: https://github.com/apache/pulsar/pull/19490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on pull request #19490: [improve][broker] PIP-192 Use msg property to mark compacted msgs from strategic compaction
heesung-sn commented on PR #19490: URL: https://github.com/apache/pulsar/pull/19490#issuecomment-1430391756 >@heesung-sn I am concerned about "However, because this is invalid, the tableview will skip this msg, causing an inconsistent view." > If we skip the message when consuming the data, the consumer will only receive it again if the consumer disconnects. The read position of the cursor will move to the end of the topic, after the topic compaction task done, the compacted data will not dispatch to the consumer again. Is it will be a problem in this case? >> Good point. >> >> we should change the condition and accept all compacted messages are valid transitions (even if the prev msg is not null). I misunderstood what you said. Please ignore my above comment. I will close this PR and propose another solution without msg properties because, as you commented, topic cursors can reset to compaction horizon, which won't have message properties. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19520: [improve][broker] Require authRole is proxyRole to set originalPrincipal (#19455)
github-actions[bot] commented on PR #19520: URL: https://github.com/apache/pulsar/pull/19520#issuecomment-1430378929 @michaeljmarshall Please add the following content to your PR description and select a checkbox: ``` - [ ] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall opened a new pull request, #19520: [improve][broker] Require authRole is proxyRole to set originalPrincipal (#19455)
michaeljmarshall opened a new pull request, #19520: URL: https://github.com/apache/pulsar/pull/19520 PR to trigger tests. I'll probably just push the cherry pick once tests pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #16691: PIP-192: New Pulsar Broker Load Balancer
frankjkelly commented on issue #16691: URL: https://github.com/apache/pulsar/issues/16691#issuecomment-1430361727 @merlimat that would be awesome - so much great work has been done on load balancing on the server side, if we can make it seamless (or nearly so!) to the clients that would just be wonderful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] merlimat commented on issue #16691: PIP-192: New Pulsar Broker Load Balancer
merlimat commented on issue #16691: URL: https://github.com/apache/pulsar/issues/16691#issuecomment-1430349333 @frankjkelly One of the next goals for this effort is to optimize the hand-over of topics between brokers in a much more graceful and coordinated way, without resorting to retries and backoffs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-python] BoillMan133 opened a new issue, #94: Producer.sendAsync hangs if rate is high enough
BoillMan133 opened a new issue, #94: URL: https://github.com/apache/pulsar-client-python/issues/94 Used: python: 3.9.6 pulsar-client 3.0.0 Pulsar: docker on localhost of apachepulsar/pulsar:2.11.0 Just made a simple script that sendsAsync thousand messages (Strings) in a for loop. And script at some point just hangs. No any errors in log or in a callbacks or anything. Doesn't matter if flag `block_if_queue_full` is set or no, or what `max_pending_messages` or `batching_max_messages` are. Slowing down a client helps (Introducing artifical pauses, or often flush()) Never seen that in a java client. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.11 updated: [improve][broker] Require authRole is proxyRole to set originalPrincipal (#19455)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.11
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.11 by this push:
new c7eabc93913 [improve][broker] Require authRole is proxyRole to set
originalPrincipal (#19455)
c7eabc93913 is described below
commit c7eabc9391361508fc222469479bfea83d5d0c64
Author: Michael Marshall
AuthorDate: Tue Feb 14 00:20:27 2023 -0600
[improve][broker] Require authRole is proxyRole to set originalPrincipal
(#19455)
Co-authored-by: Lari Hotari
(cherry picked from commit aa63a5567a9e5d466b311a54d5dcc2cb05c2b5cd)
---
.../broker/authorization/AuthorizationService.java | 67 ++---
.../broker/admin/impl/PersistentTopicsBase.java| 2 +-
.../apache/pulsar/broker/service/ServerCnx.java| 31 +-
.../pulsar/broker/web/PulsarWebResource.java | 27 +++--
.../pulsar/broker/auth/AuthorizationTest.java | 41 -
.../broker/auth/MockedPulsarServiceBaseTest.java | 7 +++
.../pulsar/broker/service/ServerCnxTest.java | 4 ++
.../client/impl/AdminApiKeyStoreTlsAuthTest.java | 18 +++---
.../ProxyAuthenticatedProducerConsumerTest.java| 44 +-
.../server/ProxyWithAuthorizationNegTest.java | 2 +
.../server/ProxyWithJwtAuthorizationTest.java | 4 +-
tests/certificate-authority/generate_keystore.sh | 11
.../certificate-authority/jks/broker.keystore.jks | Bin 2254 -> 2254 bytes
.../jks/broker.truststore.jks | Bin 978 -> 969 bytes
.../jks/broker.truststore.nopassword.jks | Bin 978 -> 969 bytes
.../certificate-authority/jks/client.keystore.jks | Bin 2258 -> 2257 bytes
.../jks/client.truststore.jks | Bin 980 -> 971 bytes
.../jks/client.truststore.nopassword.jks | Bin 980 -> 971 bytes
.../jks/proxy-and-client.truststore.jks| Bin 0 -> 1891 bytes
.../jks/proxy-and-client.truststore.nopassword.jks | Bin 0 -> 1891 bytes
tests/certificate-authority/jks/proxy.keystore.jks | Bin 0 -> 2245 bytes
.../certificate-authority/jks/proxy.truststore.jks | Bin 0 -> 971 bytes
.../jks/proxy.truststore.nopassword.jks| Bin 0 -> 971 bytes
23 files changed, 163 insertions(+), 95 deletions(-)
diff --git
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authorization/AuthorizationService.java
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authorization/AuthorizationService.java
index 3baaf57990a..05f146e8953 100644
---
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authorization/AuthorizationService.java
+++
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authorization/AuthorizationService.java
@@ -19,10 +19,10 @@
package org.apache.pulsar.broker.authorization;
import static java.util.concurrent.TimeUnit.SECONDS;
+import java.net.SocketAddress;
import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
-import javax.ws.rs.core.Response;
import org.apache.commons.lang3.StringUtils;
import org.apache.pulsar.broker.PulsarServerException;
import org.apache.pulsar.broker.ServiceConfiguration;
@@ -37,7 +37,6 @@ import org.apache.pulsar.common.policies.data.PolicyOperation;
import org.apache.pulsar.common.policies.data.TenantInfo;
import org.apache.pulsar.common.policies.data.TenantOperation;
import org.apache.pulsar.common.policies.data.TopicOperation;
-import org.apache.pulsar.common.util.FutureUtil;
import org.apache.pulsar.common.util.RestException;
import org.apache.pulsar.metadata.api.MetadataStoreException;
import org.slf4j.Logger;
@@ -293,19 +292,39 @@ public class AuthorizationService {
return provider.allowSinkOpsAsync(namespaceName, role,
authenticationData);
}
-private static void validateOriginalPrincipal(Set proxyRoles,
String authenticatedPrincipal,
- String originalPrincipal) {
-if (proxyRoles.contains(authenticatedPrincipal)) {
-// Request has come from a proxy
+public boolean isValidOriginalPrincipal(String authenticatedPrincipal,
+String originalPrincipal,
+AuthenticationDataSource
authDataSource) {
+SocketAddress remoteAddress = authDataSource != null ?
authDataSource.getPeerAddress() : null;
+return isValidOriginalPrincipal(authenticatedPrincipal,
originalPrincipal, remoteAddress);
+}
+
+/**
+ * Validates that the authenticatedPrincipal and the originalPrincipal are
a valid combination.
+ * Valid combinations fulfill the following rule: the
authenticatedPrincipal is in
+ * {@link ServiceConfiguration#getProxyRoles()}, if, and only if, the
originalPrincipal is set to a role
+ * that is
[GitHub] [pulsar-client-python] erichare opened a new pull request, #93: #66: Add Human-readable description of MessageId object
erichare opened a new pull request, #93: URL: https://github.com/apache/pulsar-client-python/pull/93 @BewareMyPower sorry this took so long, got distracted by some other things. I didn't realize this was already implemented in the C++ client at least if i'm not crazy :) So basically I just pulled a binding (same as the __str__ binding) for __repr__, and now the MessageId object looks like the following. Is this sufficient for purposes of this task? https://user-images.githubusercontent.com/700235/218841335-db52d0ed-98a0-4d60-a412-aca338d0c0dd.png;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.11 updated: [fix][client] Fix the Windows absolute path not recognized in auth param string (#18403)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.11
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.11 by this push:
new 7083755a73a [fix][client] Fix the Windows absolute path not recognized
in auth param string (#18403)
7083755a73a is described below
commit 7083755a73a69246d0e6b66c11b9afa7b2c26d8f
Author: Michael Marshall
AuthorDate: Tue Feb 14 13:38:25 2023 -0600
[fix][client] Fix the Windows absolute path not recognized in auth param
string (#18403)
(cherry picked from commit 177b96a78acada4888cb92f96c10ebba3eca8db7)
---
.../broker/auth/MockedPulsarServiceBaseTest.java | 14 -
.../org/apache/pulsar/utils/ResourceUtils.java | 31 +++
.../pulsar/admin/cli/PulsarAdminToolTest.java | 5 ++-
.../pulsar/client/impl/AuthenticationUtil.java | 11 +--
.../pulsar/client/impl/AuthenticationUtilTest.java | 36 ++
5 files changed, 84 insertions(+), 13 deletions(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
index 52d13a97816..3b483efc33d 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
@@ -26,7 +26,6 @@ import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.when;
import com.google.common.collect.Sets;
-import com.google.common.io.Resources;
import com.google.common.util.concurrent.MoreExecutors;
import io.netty.channel.EventLoopGroup;
import java.lang.reflect.Field;
@@ -74,6 +73,7 @@ import org.apache.pulsar.metadata.api.MetadataStoreException;
import org.apache.pulsar.metadata.api.extended.MetadataStoreExtended;
import org.apache.pulsar.metadata.impl.ZKMetadataStore;
import org.apache.pulsar.tests.TestRetrySupport;
+import org.apache.pulsar.utils.ResourceUtils;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.MockZooKeeper;
import org.apache.zookeeper.data.ACL;
@@ -88,20 +88,20 @@ import javax.ws.rs.container.TimeoutHandler;
*/
public abstract class MockedPulsarServiceBaseTest extends TestRetrySupport {
public final static String BROKER_KEYSTORE_FILE_PATH =
-
Resources.getResource("certificate-authority/jks/broker.keystore.jks").getPath();
+
ResourceUtils.getAbsolutePath("certificate-authority/jks/broker.keystore.jks");
public final static String BROKER_TRUSTSTORE_FILE_PATH =
-
Resources.getResource("certificate-authority/jks/broker.truststore.jks").getPath();
+
ResourceUtils.getAbsolutePath("certificate-authority/jks/broker.truststore.jks");
public final static String BROKER_TRUSTSTORE_NO_PASSWORD_FILE_PATH =
-
Resources.getResource("certificate-authority/jks/broker.truststore.nopassword.jks").getPath();
+
ResourceUtils.getAbsolutePath("certificate-authority/jks/broker.truststore.nopassword.jks");
public final static String BROKER_KEYSTORE_PW = "11";
public final static String BROKER_TRUSTSTORE_PW = "11";
public final static String CLIENT_KEYSTORE_FILE_PATH =
-
Resources.getResource("certificate-authority/jks/client.keystore.jks").getPath();
+
ResourceUtils.getAbsolutePath("certificate-authority/jks/client.keystore.jks");
public final static String CLIENT_TRUSTSTORE_FILE_PATH =
-
Resources.getResource("certificate-authority/jks/client.truststore.jks").getPath();
+
ResourceUtils.getAbsolutePath("certificate-authority/jks/client.truststore.jks");
public final static String CLIENT_TRUSTSTORE_NO_PASSWORD_FILE_PATH =
-
Resources.getResource("certificate-authority/jks/client.truststore.nopassword.jks").getPath();
+
ResourceUtils.getAbsolutePath("certificate-authority/jks/client.truststore.nopassword.jks");
public final static String CLIENT_KEYSTORE_PW = "11";
public final static String CLIENT_TRUSTSTORE_PW = "11";
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/utils/ResourceUtils.java
b/pulsar-broker/src/test/java/org/apache/pulsar/utils/ResourceUtils.java
new file mode 100644
index 000..d0511e272f4
--- /dev/null
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/utils/ResourceUtils.java
@@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
[GitHub] [pulsar-client-python] erichare commented on a diff in pull request #90: [fix] exception bug as per #89
erichare commented on code in PR #90:
URL:
https://github.com/apache/pulsar-client-python/pull/90#discussion_r1106272540
##
pulsar/schema/schema_avro.py:
##
@@ -82,15 +82,6 @@ def decode(self, data):
return self._record_cls(**d)
else:
return d
-
else:
-class AvroSchema(Schema):
-def __init__(self, _record_cls, _schema_definition=None):
-raise Exception("Avro library support was not found. Make sure to
install Pulsar client " +
+raise Exception("Avro library support was not found. Make sure to install
Pulsar client " +
Review Comment:
@keenborder786 I haven't worked too much with AvroSchema specifically, but
when i run your example (and the example from the pulsar docs, I get the
following error:
```
UnknownError Traceback (most recent call last)
Cell In[11], line 1
> 1 producer = client.create_producer(
2 topic=topic,
3 schema=avro_schema)
File /opt/homebrew/lib/python3.11/site-packages/pulsar/__init__.py:645, in
Client.create_producer(self, topic, producer_name, schema, initial_sequence_id,
send_timeout_millis, compression_type, max_pending_messages,
max_pending_messages_across_partitions, block_if_queue_full, batching_enabled,
batching_max_messages, batching_max_allowed_size_in_bytes,
batching_max_publish_delay_ms, chunking_enabled, message_routing_mode,
lazy_start_partitioned_producers, properties, batching_type, encryption_key,
crypto_key_reader)
642 raise ValueError("Batching and chunking of messages can't be
enabled together.")
644 p = Producer()
--> 645 p._producer = self._client.create_producer(topic, conf)
646 p._schema = schema
647 p._client = self._client
```
Note i don't think line 642 is relevant - that's just showing the lines
around the line that failed. I don't have this problem with other schema types,
just AvroSchema...
I don't get far enough to test what you're experiencing. Have you seen this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[pulsar] branch branch-2.8 updated: [cleanup][broker] Validate originalPrincipal earlier in ServerCnx (#19270)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.8 by this push:
new 9d36b0e7604 [cleanup][broker] Validate originalPrincipal earlier in
ServerCnx (#19270)
9d36b0e7604 is described below
commit 9d36b0e7604543d4c17f8237bec7c5ba407b4857
Author: Michael Marshall
AuthorDate: Tue Feb 14 13:06:25 2023 -0600
[cleanup][broker] Validate originalPrincipal earlier in ServerCnx (#19270)
(cherry picked from commit fd3ce8b5786baf0b76f301bd9597cd0b99a412f1)
(cherry picked from commit 2847dd19f6e8a546f4d45bf51eb2b72aae0869ce)
(cherry picked from commit 01bd9860dbb272509bbc13bd71a3983715f84a2a)
---
.../apache/pulsar/broker/service/ServerCnx.java| 78 +-
.../pulsar/broker/service/ServerCnxTest.java | 38 +++
2 files changed, 68 insertions(+), 48 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index a09b8b823e4..9a55a728717 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -349,17 +349,30 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
ctx.close();
}
-/*
- * If authentication and authorization is enabled(and not sasl) and
- * if the authRole is one of proxyRoles we want to enforce
+/**
+ * When transitioning from Connecting to Connected, this method validates
the roles.
+ * If the authRole is one of proxyRoles, the following must be true:
* - the originalPrincipal is given while connecting
* - originalPrincipal is not blank
- * - originalPrincipal is not a proxy principal
+ * - originalPrincipal is not a proxy principal.
+ * @return true when roles are valid and false when roles are invalid
*/
-private boolean invalidOriginalPrincipal(String originalPrincipal) {
-return (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()
-&& proxyRoles.contains(authRole) &&
(StringUtils.isBlank(originalPrincipal)
-|| proxyRoles.contains(originalPrincipal)));
+private boolean isValidRoleAndOriginalPrincipal() {
+String errorMsg = null;
+if (proxyRoles.contains(authRole)) {
+if (StringUtils.isBlank(originalPrincipal)) {
+errorMsg = "originalPrincipal must be provided when connecting
with a proxy role.";
+} else if (proxyRoles.contains(originalPrincipal)) {
+errorMsg = "originalPrincipal cannot be a proxy role.";
+}
+}
+if (errorMsg != null) {
+log.warn("[{}] Illegal combination of role [{}] and
originalPrincipal [{}]: {}", remoteAddress, authRole,
+originalPrincipal, errorMsg);
+return false;
+} else {
+return true;
+}
}
//
@@ -426,14 +439,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
final Semaphore lookupSemaphore = service.getLookupRequestSemaphore();
if (lookupSemaphore.tryAcquire()) {
-if (invalidOriginalPrincipal(originalPrincipal)) {
-final String msg = "Valid Proxy Client role should be provided
for lookup ";
-log.warn("[{}] {} with role {} and proxyClientAuthRole {} on
topic {}", remoteAddress, msg, authRole,
-originalPrincipal, topicName);
-
ctx.writeAndFlush(newLookupErrorResponse(ServerError.AuthorizationError, msg,
requestId));
-lookupSemaphore.release();
-return;
-}
isTopicOperationAllowed(topicName, TopicOperation.LOOKUP,
authenticationData, originalAuthData).thenApply(
isAuthorized -> {
if (isAuthorized) {
@@ -490,14 +495,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
final Semaphore lookupSemaphore = service.getLookupRequestSemaphore();
if (lookupSemaphore.tryAcquire()) {
-if (invalidOriginalPrincipal(originalPrincipal)) {
-final String msg = "Valid Proxy Client role should be provided
for getPartitionMetadataRequest ";
-log.warn("[{}] {} with role {} and proxyClientAuthRole {} on
topic {}", remoteAddress, msg, authRole,
-originalPrincipal, topicName);
-
commandSender.sendPartitionMetadataResponse(ServerError.AuthorizationError,
msg, requestId);
-lookupSemaphore.release();
-return;
-}
isTopicOperationAllowed(topicName, TopicOperation.LOOKUP,
[GitHub] [pulsar] michaeljmarshall commented on pull request #19270: [cleanup][broker] Validate originalPrincipal earlier in ServerCnx
michaeljmarshall commented on PR #19270: URL: https://github.com/apache/pulsar/pull/19270#issuecomment-1430236553 I cherry-picked this change to the release branches because https://github.com/apache/pulsar/pull/19455 depends on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.9 updated: [cleanup][broker] Validate originalPrincipal earlier in ServerCnx (#19270)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch branch-2.9
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.9 by this push:
new 34a2c7947bd [cleanup][broker] Validate originalPrincipal earlier in
ServerCnx (#19270)
34a2c7947bd is described below
commit 34a2c7947bd0386b676f5ac218dc92d6e17d28a3
Author: Michael Marshall
AuthorDate: Tue Feb 14 12:44:18 2023 -0600
[cleanup][broker] Validate originalPrincipal earlier in ServerCnx (#19270)
(cherry picked from commit fd3ce8b5786baf0b76f301bd9597cd0b99a412f1)
(cherry picked from commit 2847dd19f6e8a546f4d45bf51eb2b72aae0869ce)
(cherry picked from commit 01bd9860dbb272509bbc13bd71a3983715f84a2a)
---
.../apache/pulsar/broker/service/ServerCnx.java| 78 +-
.../pulsar/broker/service/ServerCnxTest.java | 37 ++
2 files changed, 67 insertions(+), 48 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index e94950680c9..48e00fe4fd0 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -362,17 +362,30 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
ctx.close();
}
-/*
- * If authentication and authorization is enabled(and not sasl) and
- * if the authRole is one of proxyRoles we want to enforce
+/**
+ * When transitioning from Connecting to Connected, this method validates
the roles.
+ * If the authRole is one of proxyRoles, the following must be true:
* - the originalPrincipal is given while connecting
* - originalPrincipal is not blank
- * - originalPrincipal is not a proxy principal
+ * - originalPrincipal is not a proxy principal.
+ * @return true when roles are valid and false when roles are invalid
*/
-private boolean invalidOriginalPrincipal(String originalPrincipal) {
-return (service.isAuthenticationEnabled() &&
service.isAuthorizationEnabled()
-&& proxyRoles.contains(authRole) &&
(StringUtils.isBlank(originalPrincipal)
-|| proxyRoles.contains(originalPrincipal)));
+private boolean isValidRoleAndOriginalPrincipal() {
+String errorMsg = null;
+if (proxyRoles.contains(authRole)) {
+if (StringUtils.isBlank(originalPrincipal)) {
+errorMsg = "originalPrincipal must be provided when connecting
with a proxy role.";
+} else if (proxyRoles.contains(originalPrincipal)) {
+errorMsg = "originalPrincipal cannot be a proxy role.";
+}
+}
+if (errorMsg != null) {
+log.warn("[{}] Illegal combination of role [{}] and
originalPrincipal [{}]: {}", remoteAddress, authRole,
+originalPrincipal, errorMsg);
+return false;
+} else {
+return true;
+}
}
//
@@ -446,14 +459,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
final Semaphore lookupSemaphore = service.getLookupRequestSemaphore();
if (lookupSemaphore.tryAcquire()) {
-if (invalidOriginalPrincipal(originalPrincipal)) {
-final String msg = "Valid Proxy Client role should be provided
for lookup ";
-log.warn("[{}] {} with role {} and proxyClientAuthRole {} on
topic {}", remoteAddress, msg, authRole,
-originalPrincipal, topicName);
-
ctx.writeAndFlush(newLookupErrorResponse(ServerError.AuthorizationError, msg,
requestId));
-lookupSemaphore.release();
-return;
-}
isTopicOperationAllowed(topicName, TopicOperation.LOOKUP,
authenticationData, originalAuthData).thenApply(
isAuthorized -> {
if (isAuthorized) {
@@ -510,14 +515,6 @@ public class ServerCnx extends PulsarHandler implements
TransportCnx {
final Semaphore lookupSemaphore = service.getLookupRequestSemaphore();
if (lookupSemaphore.tryAcquire()) {
-if (invalidOriginalPrincipal(originalPrincipal)) {
-final String msg = "Valid Proxy Client role should be provided
for getPartitionMetadataRequest ";
-log.warn("[{}] {} with role {} and proxyClientAuthRole {} on
topic {}", remoteAddress, msg, authRole,
-originalPrincipal, topicName);
-
commandSender.sendPartitionMetadataResponse(ServerError.AuthorizationError,
msg, requestId);
-lookupSemaphore.release();
-return;
-}
isTopicOperationAllowed(topicName, TopicOperation.LOOKUP,
