[kafka] branch trunk updated: KAFKA-14533: re-enable 'false' and disable the 'true' parameter of SmokeTestDriverIntegrationTest (#13156)
This is an automated email from the ASF dual-hosted git repository.
ableegoldman pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/trunk by this push:
new 3799708ff09 KAFKA-14533: re-enable 'false' and disable the 'true'
parameter of SmokeTestDriverIntegrationTest (#13156)
3799708ff09 is described below
commit 3799708ff09b174883eddc1f6a857ddc518fa35f
Author: A. Sophie Blee-Goldman
AuthorDate: Tue Jan 24 00:14:20 2023 -0800
KAFKA-14533: re-enable 'false' and disable the 'true' parameter of
SmokeTestDriverIntegrationTest (#13156)
I immediately saw a failure with stateUpdaterEnabled = true after disabling
the false parameter, which suggests the problem actually does lie in the state
updater itself and not the act of parametrization of the test. To verify this
theory, and help stabilize the 3.4 release branch, let's try one more test by
swapping out the true build in favor of the false one. If the listOffsets
requests stop failing and causing this integration test to hit the global
timeout as is currently happen [...]
After getting in a few builds to see whether the flakiness subsides, we
should merge this PR to re-enable both parameters going forward:
https://github.com/apache/kafka/pull/13155
Reviewers: Anna Sophie Blee-Goldman
---
.../kafka/streams/integration/SmokeTestDriverIntegrationTest.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
b/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
index 34336c05046..cbe3afc7bae 100644
---
a/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
+++
b/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
@@ -97,8 +97,8 @@ public class SmokeTestDriverIntegrationTest {
private static Stream parameters() {
return Stream.of(
// TODO KAFKA-14533: debug and re-enable both parameters
-Boolean.TRUE
-//Boolean.FALSE
+//Boolean.TRUE
+Boolean.FALSE
);
}
[kafka] branch 3.4 updated: KAFKA-14533: re-enable 'false' and disable the 'true' parameter of SmokeTestDriverIntegrationTest (#13156)
This is an automated email from the ASF dual-hosted git repository.
ableegoldman pushed a commit to branch 3.4
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/3.4 by this push:
new 68d70cdc399 KAFKA-14533: re-enable 'false' and disable the 'true'
parameter of SmokeTestDriverIntegrationTest (#13156)
68d70cdc399 is described below
commit 68d70cdc399ac427e1e44cc9cfcc4497ecf832c3
Author: A. Sophie Blee-Goldman
AuthorDate: Tue Jan 24 00:14:20 2023 -0800
KAFKA-14533: re-enable 'false' and disable the 'true' parameter of
SmokeTestDriverIntegrationTest (#13156)
I immediately saw a failure with stateUpdaterEnabled = true after disabling
the false parameter, which suggests the problem actually does lie in the state
updater itself and not the act of parametrization of the test. To verify this
theory, and help stabilize the 3.4 release branch, let's try one more test by
swapping out the true build in favor of the false one. If the listOffsets
requests stop failing and causing this integration test to hit the global
timeout as is currently happen [...]
After getting in a few builds to see whether the flakiness subsides, we
should merge this PR to re-enable both parameters going forward:
https://github.com/apache/kafka/pull/13155
Reviewers: Anna Sophie Blee-Goldman
---
.../kafka/streams/integration/SmokeTestDriverIntegrationTest.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
b/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
index 6f345346949..06dd3ff0b60 100644
---
a/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
+++
b/streams/src/test/java/org/apache/kafka/streams/integration/SmokeTestDriverIntegrationTest.java
@@ -98,8 +98,8 @@ public class SmokeTestDriverIntegrationTest {
private static Stream parameters() {
return Stream.of(
// TODO KAFKA-14533: debug and re-enable both parameters
-Boolean.TRUE
-//Boolean.FALSE
+//Boolean.TRUE
+Boolean.FALSE
);
}
[kafka-site] branch asf-site updated: MINOR: Fix docs in security.html (#461)
This is an automated email from the ASF dual-hosted git repository. mimaison pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/kafka-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 64ea310b MINOR: Fix docs in security.html (#461) 64ea310b is described below commit 64ea310bd4c507054c663a93e204e19209ca0430 Author: JK-Wang <32212764+jk-w...@users.noreply.github.com> AuthorDate: Tue Jan 24 22:48:19 2023 +0800 MINOR: Fix docs in security.html (#461) Reviewers: Mickael Maison --- 33/security.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/33/security.html b/33/security.html index f401c1c1..e640b537 100644 --- a/33/security.html +++ b/33/security.html @@ -1516,7 +1516,7 @@ RULE:[n:string](regexp)s/pattern/replacement/g/U Suppose you want to add an acl "Principals User:Bob and User:Alice are allowed to perform Operation Read and Write on Topic Test-Topic from IP 198.51.100.0 and IP 198.51.100.1". You can do that by executing the CLI with following options: bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:Bob --allow-principal User:Alice --allow-host 198.51.100.0 --allow-host 198.51.100.1 --operation Read --operation Write --topic Test-topic By default, all principals that don't have an explicit acl that allows access for an operation to a resource are denied. In rare cases where an allow acl is defined that allows access to all but some principal we will have to use the --deny-principal and --deny-host option. For example, if we want to allow all users to Read from Test-topic but only deny User:BadBob from IP 198.51.100.3 we can do so using following commands: - bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:* --allow-host * --deny-principal User:BadBob --deny-host 198.51.100.3 --operation Read --topic Test-topic + bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:'*' --allow-host '*' --deny-principal User:BadBob --deny-host 198.51.100.3 --operation Read --topic Test-topic Note that --allow-host and --deny-host only support IP addresses (hostnames are not supported). Above examples add acls to a topic by specifying --topic [topic-name] as the resource pattern option. Similarly user can add acls to cluster by specifying --cluster and to a consumer group by specifying --group [group-name]. You can add acls on any resource of a certain type, e.g. suppose you wanted to add an acl "Principal User:Peter is allowed to produce to any Topic from IP 198.51.200.0"
[kafka-site] branch asf-site updated: Powerd by Dream11 section, grammar fixes (#464)
This is an automated email from the ASF dual-hosted git repository.
bbejeck pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/kafka-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2ef73a71 Powerd by Dream11 section, grammar fixes (#464)
2ef73a71 is described below
commit 2ef73a71ab4b7c2684af6bcef0ad8bc802049411
Author: VikasGite <43334864+vikasg...@users.noreply.github.com>
AuthorDate: Tue Jan 24 20:33:10 2023 +0530
Powerd by Dream11 section, grammar fixes (#464)
* Dream11 powered by Apache Kafka section added
* Dream11 powered by Apache Kafka section, fixed grammar
* Update powered-by.html
Co-authored-by: vikas.g...@dream11.com
Co-authored-by: Bill Bejeck
---
powered-by.html | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/powered-by.html b/powered-by.html
index 70e5a59d..4037c9c2 100644
--- a/powered-by.html
+++ b/powered-by.html
@@ -12,7 +12,7 @@
"link":
"https://tech.dream11.in/blog/2020-01-07_Data-Highway---Dream11-s-Inhouse-Analytics-Platform---The-Burden-and-Benefits-90b8777d282;,
"logo": "dream11.jpg",
"logoBgColor": "#e1",
-"description": "We use apache kafka heavily for data ingestion to Data
platform, streaming and batch analytics, and our micro services to communicate
one another. Kafka has been core component of overall Dream11 Tech stack"
+"description": "We use Apache Kafka heavily for data ingestion to the
Data platform, streaming as well as batch analytics, and for our microservices
to communicate with one another. Kafka is a core component of the overall
Dream11 Tech stack."
},
{
"link": "https://brainstation-23.com;,
[kafka] branch trunk updated (3799708ff09 -> bc1ce9f0f1b)
This is an automated email from the ASF dual-hosted git repository. rsivaram pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/kafka.git from 3799708ff09 KAFKA-14533: re-enable 'false' and disable the 'true' parameter of SmokeTestDriverIntegrationTest (#13156) add bc1ce9f0f1b KAFKA-14623: OAuth's HttpAccessTokenRetriever potentially leaks secrets in logging (#13119) No new revisions were added by this update. Summary of changes: .../internals/secured/HttpAccessTokenRetriever.java | 17 - 1 file changed, 12 insertions(+), 5 deletions(-)
[kafka] branch trunk updated (1d0585563b4 -> 0711375b1ef)
This is an automated email from the ASF dual-hosted git repository. cmccabe pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/kafka.git from 1d0585563b4 MINOR: fix flaky DefaultStateUpdaterTest (#13160) add 0711375b1ef KRaft brokers and controllesr should fail on Authorizer failure (#13151) No new revisions were added by this update. Summary of changes: .../src/main/scala/kafka/server/BrokerServer.scala | 8 +++ .../main/scala/kafka/server/ControllerServer.scala | 7 +++ core/src/main/scala/kafka/server/KafkaServer.scala | 7 +++ .../kafka/server/KRaftClusterTest.scala| 60 +- 4 files changed, 70 insertions(+), 12 deletions(-)
[kafka] branch trunk updated (bc1ce9f0f1b -> 1d0585563b4)
This is an automated email from the ASF dual-hosted git repository. guozhang pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/kafka.git from bc1ce9f0f1b KAFKA-14623: OAuth's HttpAccessTokenRetriever potentially leaks secrets in logging (#13119) add 1d0585563b4 MINOR: fix flaky DefaultStateUpdaterTest (#13160) No new revisions were added by this update. Summary of changes: streams/src/test/java/org/apache/kafka/test/StreamsTestUtils.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
[kafka] branch 3.4 updated: MINOR: Few cleanups to JaasContext/Utils classes
This is an automated email from the ASF dual-hosted git repository.
manikumar pushed a commit to branch 3.4
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/3.4 by this push:
new ae22ec1a0ea MINOR: Few cleanups to JaasContext/Utils classes
ae22ec1a0ea is described below
commit ae22ec1a0ea005664439c3f45111aa34390ecaa1
Author: Manikumar Reddy
AuthorDate: Fri Jan 13 23:18:02 2023 +0530
MINOR: Few cleanups to JaasContext/Utils classes
Reviewers: Rajini Sivaram
---
.../apache/kafka/common/security/JaasContext.java | 22 ++
.../apache/kafka/common/security/JaasUtils.java| 3 +-
.../kafka/common/security/JaasContextTest.java | 48 ++
3 files changed, 72 insertions(+), 1 deletion(-)
diff --git
a/clients/src/main/java/org/apache/kafka/common/security/JaasContext.java
b/clients/src/main/java/org/apache/kafka/common/security/JaasContext.java
index 48216a8a90c..34f2952bff6 100644
--- a/clients/src/main/java/org/apache/kafka/common/security/JaasContext.java
+++ b/clients/src/main/java/org/apache/kafka/common/security/JaasContext.java
@@ -30,6 +30,11 @@ import java.util.Collections;
import java.util.List;
import java.util.Locale;
import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static
org.apache.kafka.common.security.JaasUtils.DISALLOWED_LOGIN_MODULES_CONFIG;
+import static
org.apache.kafka.common.security.JaasUtils.DISALLOWED_LOGIN_MODULES_DEFAULT;
public class JaasContext {
@@ -91,11 +96,25 @@ public class JaasContext {
throw new IllegalArgumentException("JAAS config property does
not contain any login modules");
else if (contextModules.length != 1)
throw new IllegalArgumentException("JAAS config property
contains " + contextModules.length + " login modules, should be 1 module");
+
+throwIfLoginModuleIsNotAllowed(contextModules[0]);
return new JaasContext(globalContextName, contextType, jaasConfig,
dynamicJaasConfig);
} else
return defaultContext(contextType, listenerContextName,
globalContextName);
}
+private static void throwIfLoginModuleIsNotAllowed(AppConfigurationEntry
appConfigurationEntry) {
+Set disallowedLoginModuleList = Arrays.stream(
+System.getProperty(DISALLOWED_LOGIN_MODULES_CONFIG,
DISALLOWED_LOGIN_MODULES_DEFAULT).split(","))
+.map(String::trim)
+.collect(Collectors.toSet());
+String loginModuleName =
appConfigurationEntry.getLoginModuleName().trim();
+if (disallowedLoginModuleList.contains(loginModuleName)) {
+throw new IllegalArgumentException(loginModuleName + " is not
allowed. Update System property '"
++ DISALLOWED_LOGIN_MODULES_CONFIG + "' to allow " +
loginModuleName);
+}
+}
+
private static JaasContext defaultContext(JaasContext.Type contextType,
String listenerContextName,
String globalContextName) {
String jaasConfigFile =
System.getProperty(JaasUtils.JAVA_LOGIN_CONFIG_PARAM);
@@ -131,6 +150,9 @@ public class JaasContext {
throw new IllegalArgumentException(errorMessage);
}
+for (AppConfigurationEntry appConfigurationEntry : configEntries) {
+throwIfLoginModuleIsNotAllowed(appConfigurationEntry);
+}
return new JaasContext(contextName, contextType, jaasConfig, null);
}
diff --git
a/clients/src/main/java/org/apache/kafka/common/security/JaasUtils.java
b/clients/src/main/java/org/apache/kafka/common/security/JaasUtils.java
index baff5633a3f..38a8b06de45 100644
--- a/clients/src/main/java/org/apache/kafka/common/security/JaasUtils.java
+++ b/clients/src/main/java/org/apache/kafka/common/security/JaasUtils.java
@@ -25,7 +25,8 @@ import javax.security.auth.login.Configuration;
public final class JaasUtils {
private static final Logger LOG = LoggerFactory.getLogger(JaasUtils.class);
public static final String JAVA_LOGIN_CONFIG_PARAM =
"java.security.auth.login.config";
-
+public static final String DISALLOWED_LOGIN_MODULES_CONFIG =
"org.apache.kafka.disallowed.login.modules";
+public static final String DISALLOWED_LOGIN_MODULES_DEFAULT =
"com.sun.security.auth.module.JndiLoginModule";
public static final String SERVICE_NAME = "serviceName";
public static final String ZK_SASL_CLIENT = "zookeeper.sasl.client";
diff --git
a/clients/src/test/java/org/apache/kafka/common/security/JaasContextTest.java
b/clients/src/test/java/org/apache/kafka/common/security/JaasContextTest.java
index 05c1bb84a20..c7d3bf3f282 100644
---
a/clients/src/test/java/org/apache/kafka/common/security/JaasContextTest.java
+++
b/clients/src/test/java/org/apache/kafka/common/security/JaasContextTest.java
@@ -30,6 +30,7 @@ import
