[GitHub] merlimat closed pull request #1099: Support ztsUrl parameter in athenz client plugin

[GitBox]

merlimat closed pull request #1099: Support ztsUrl parameter in athenz client 
plugin
URL: https://github.com/apache/incubator-pulsar/pull/1099
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
 
b/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
index 85e9f8e58..d833e950d 100644
--- 
a/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
+++ 
b/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
@@ -56,6 +56,7 @@
 private static final String APPLICATION_X_PEM_FILE_BASE64 = 
"application/x-pem-file;base64";
 
 private transient ZTSClient ztsClient = null;
+private String ztsUrl;
 private String tenantDomain;
 private String tenantService;
 private String providerDomain;
@@ -150,6 +151,9 @@ private void setAuthParams(Map authParams) {
 if (authParams.containsKey("roleHeader")) {
 System.setProperty("athenz.auth.role.header", 
authParams.get("roleHeader"));
 }
+if (authParams.containsKey("ztsUrl")) {
+this.ztsUrl = authParams.get("ztsUrl");
+}
 }
 
 @Override
@@ -164,7 +168,7 @@ private ZTSClient getZtsClient() {
 if (ztsClient == null) {
 ServiceIdentityProvider siaProvider = new 
SimpleServiceIdentityProvider(tenantDomain, tenantService,
 privateKey, keyId);
-ztsClient = new ZTSClient(null, tenantDomain, tenantService, 
siaProvider);
+ztsClient = new ZTSClient(ztsUrl, tenantDomain, tenantService, 
siaProvider);
 }
 return ztsClient;
 }
diff --git 
a/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
 
b/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
index c927262ed..36df7f135 100644
--- 
a/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
+++ 
b/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
@@ -110,6 +110,14 @@ public void testGetAuthData() throws Exception {
 assertEquals(count, 1);
 }
 
+@Test
+public void testZtsUrl() throws Exception {
+Field field = auth.getClass().getDeclaredField("ztsUrl");
+field.setAccessible(true);
+String ztsUrl = (String) field.get(auth);
+assertEquals(ztsUrl, "https://localhost:4443/;);
+}
+
 @Test
 public void testLoadPrivateKeyBase64() throws Exception {
 try {
diff --git a/pulsar-client-auth-athenz/src/test/resources/authParams.json 
b/pulsar-client-auth-athenz/src/test/resources/authParams.json
index d7eb6902a..5d695b267 100644
--- a/pulsar-client-auth-athenz/src/test/resources/authParams.json
+++ b/pulsar-client-auth-athenz/src/test/resources/authParams.json
@@ -2,5 +2,6 @@
"tenantService": "test_service",
"privateKey": "./src/test/resources/tenant_private.pem",
"providerDomain": "test_provider",
-   "tenantDomain": "test_tenant"
+   "tenantDomain": "test_tenant",
+   "ztsUrl": "https://localhost:4443/;
 }


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Support ztsUrl parameter in athenz client plugin (#1099)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new da4bd4a  Support ztsUrl parameter in athenz client plugin (#1099)
da4bd4a is described below

commit da4bd4ae3032159a3b8fe26f05e8067b3172b7c5
Author: hrsakai 
AuthorDate: Thu Jan 25 05:05:22 2018 +0900

Support ztsUrl parameter in athenz client plugin (#1099)
---
 .../org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java  | 6 +-
 .../apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java  | 8 
 pulsar-client-auth-athenz/src/test/resources/authParams.json  | 3 ++-
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git 
a/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
 
b/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
index 85e9f8e..d833e95 100644
--- 
a/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
+++ 
b/pulsar-client-auth-athenz/src/main/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenz.java
@@ -56,6 +56,7 @@ public class AuthenticationAthenz implements Authentication, 
EncodedAuthenticati
 private static final String APPLICATION_X_PEM_FILE_BASE64 = 
"application/x-pem-file;base64";
 
 private transient ZTSClient ztsClient = null;
+private String ztsUrl;
 private String tenantDomain;
 private String tenantService;
 private String providerDomain;
@@ -150,6 +151,9 @@ public class AuthenticationAthenz implements 
Authentication, EncodedAuthenticati
 if (authParams.containsKey("roleHeader")) {
 System.setProperty("athenz.auth.role.header", 
authParams.get("roleHeader"));
 }
+if (authParams.containsKey("ztsUrl")) {
+this.ztsUrl = authParams.get("ztsUrl");
+}
 }
 
 @Override
@@ -164,7 +168,7 @@ public class AuthenticationAthenz implements 
Authentication, EncodedAuthenticati
 if (ztsClient == null) {
 ServiceIdentityProvider siaProvider = new 
SimpleServiceIdentityProvider(tenantDomain, tenantService,
 privateKey, keyId);
-ztsClient = new ZTSClient(null, tenantDomain, tenantService, 
siaProvider);
+ztsClient = new ZTSClient(ztsUrl, tenantDomain, tenantService, 
siaProvider);
 }
 return ztsClient;
 }
diff --git 
a/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
 
b/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
index c927262..36df7f1 100644
--- 
a/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
+++ 
b/pulsar-client-auth-athenz/src/test/java/org/apache/pulsar/client/impl/auth/AuthenticationAthenzTest.java
@@ -111,6 +111,14 @@ public class AuthenticationAthenzTest {
 }
 
 @Test
+public void testZtsUrl() throws Exception {
+Field field = auth.getClass().getDeclaredField("ztsUrl");
+field.setAccessible(true);
+String ztsUrl = (String) field.get(auth);
+assertEquals(ztsUrl, "https://localhost:4443/;);
+}
+
+@Test
 public void testLoadPrivateKeyBase64() throws Exception {
 try {
 String paramsStr = new 
String(Files.readAllBytes(Paths.get("./src/test/resources/authParams.json")));
diff --git a/pulsar-client-auth-athenz/src/test/resources/authParams.json 
b/pulsar-client-auth-athenz/src/test/resources/authParams.json
index d7eb690..5d695b2 100644
--- a/pulsar-client-auth-athenz/src/test/resources/authParams.json
+++ b/pulsar-client-auth-athenz/src/test/resources/authParams.json
@@ -2,5 +2,6 @@
"tenantService": "test_service",
"privateKey": "./src/test/resources/tenant_private.pem",
"providerDomain": "test_provider",
-   "tenantDomain": "test_tenant"
+   "tenantDomain": "test_tenant",
+   "ztsUrl": "https://localhost:4443/;
 }

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] rdhabalia opened a new pull request #1105: Advertise localhost address for unit test

[GitBox]

rdhabalia opened a new pull request #1105: Advertise localhost address for unit 
test
URL: https://github.com/apache/incubator-pulsar/pull/1105
 
 
   ### Motivation
   
   After #929 : I face below unit-test failures on my local mac when machine 
connects to different n/w/
   
   ```
   10484 [pulsar-47-3] INFO org.apache.pulsar.broker.PulsarService - Loading 
all topics on bundle: pulsar/global/peer1-http/0x_0x4000
   10492 [main-EventThread] INFO org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [24/Jan/2018:11:38:06 -0800] "GET 
//localhost:15005/lookup/v2/destination/persistent/pulsar/global/peer1-http/topic1
 HTTP/1.1" 200 243 "-" "Pulsar-Java-v1.22.0-incubating-SNAPSHOT" 79
   10589 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.ConnectionPool - Failed to open connection to 
C03AT1XMcName-lm:15007 : CompletionException
   10589 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Error connecting to 
broker: org.apache.pulsar.client.api.PulsarClientException: 
java.util.concurrent.CompletionException: java.net.UnknownHostException: failed 
to resolve 'C03AT1XMcName-lm' after 5 queries 
   10589 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Could not get connection 
to broker: org.apache.pulsar.client.api.PulsarClientException: 
java.util.concurrent.CompletionException: java.net.UnknownHostException: failed 
to resolve 'C03AT1XMcName-lm' after 5 queries  -- Will try again in 0.1 s
   10691 [pulsar-timer-322-1] INFO org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Reconnecting after 
connection was closed
   10703 [pulsar-web-318-9] ERROR 
org.apache.pulsar.broker.lookup.DestinationLookup - Validation check failed: 
HTTP 307 Temporary Redirect
   10705 [pulsar-web-318-9] INFO org.eclipse.jetty.server.RequestLog - 
XX.XX.XX.XXX - - [24/Jan/2018:11:38:07 -0800] "GET 
//C03AT1XMcName-lm:15021/lookup/v2/destination/persistent/pulsar/global/peer1-http/topic1
 HTTP/1.1" 307 0 "-" "Pulsar-Java-v1.22.0-incubating-SNAPSHOT" 10
   10720 [pulsar-web-106-23] INFO org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [24/Jan/2018:11:38:07 -0800] "GET 
//localhost:15005/lookup/v2/destination/persistent/pulsar/global/peer1-http/topic1
 HTTP/1.1" 200 243 "-" "Pulsar-Java-v1.22.0-incubating-SNAPSHOT" 14
   10738 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.ConnectionPool - Failed to open connection to 
C03AT1XMcName-lm:15007 : CompletionException
   10738 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Error connecting to 
broker: org.apache.pulsar.client.api.PulsarClientException: 
java.util.concurrent.CompletionException: java.net.UnknownHostException: failed 
to resolve 'C03AT1XMcName-lm' after 5 queries 
   10738 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Could not get connection 
to broker: org.apache.pulsar.client.api.PulsarClientException: 
java.util.concurrent.CompletionException: java.net.UnknownHostException: failed 
to resolve 'C03AT1XMcName-lm' after 5 queries  -- Will try again in 0.196 s
   10935 [pulsar-timer-322-1] INFO org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Reconnecting after 
connection was closed
   10940 [pulsar-web-318-10] ERROR 
org.apache.pulsar.broker.lookup.DestinationLookup - Validation check failed: 
HTTP 307 Temporary Redirect
   10941 [pulsar-web-318-10] INFO org.eclipse.jetty.server.RequestLog - 
XX.XX.XX.XXX - - [24/Jan/2018:11:38:07 -0800] "GET 
//C03AT1XMcName-lm:15021/lookup/v2/destination/persistent/pulsar/global/peer1-http/topic1
 HTTP/1.1" 307 0 "-" "Pulsar-Java-v1.22.0-incubating-SNAPSHOT" 3
   10946 [pulsar-web-106-24] INFO org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [24/Jan/2018:11:38:07 -0800] "GET 
//localhost:15005/lookup/v2/destination/persistent/pulsar/global/peer1-http/topic1
 HTTP/1.1" 200 243 "-" "Pulsar-Java-v1.22.0-incubating-SNAPSHOT" 3
   10959 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.ConnectionPool - Failed to open connection to 
C03AT1XMcName-lm:15007 : CompletionException
   10959 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Error connecting to 
broker: org.apache.pulsar.client.api.PulsarClientException: 
java.util.concurrent.CompletionException: java.net.UnknownHostException: failed 
to resolve 'C03AT1XMcName-lm' after 5 queries 
   10959 [pulsar-client-io-320-1] WARN 
org.apache.pulsar.client.impl.HandlerBase - 
[persistent://pulsar/global/peer1-http/topic1] [null] Could not get connection 
to broker: 

[GitHub] rdhabalia commented on a change in pull request #1107: Log only first exception in the connection exception handler

[GitBox]

rdhabalia commented on a change in pull request #1107: Log only first exception 
in the connection exception handler
URL: https://github.com/apache/incubator-pulsar/pull/1107#discussion_r163666414
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
 ##
 @@ -152,7 +153,24 @@ public void channelInactive(ChannelHandlerContext ctx) 
throws Exception {
 
 @Override
 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) 
throws Exception {
-log.warn("{} Exception caught: {}", ctx.channel(), cause.getMessage(), 
cause);
+if (state != State.Failed) {
+if (cause instanceof NativeIoException) {
+// No need to report stack trace for known exceptions that 
happen in disconnections
+log.warn("[{}] Got exception {} : {}", remoteAddress, 
cause.getClass().getSimpleName(),
+cause.getMessage());
+} else {
+log.warn("[{}] Got exception: {}", remoteAddress, 
cause.getMessage(), cause);
 
 Review comment:
   Instead duplicate logging logic, should we do:
   ```
   log.warn("[{}] Got exception: {}", remoteAddress, cause.getMessage(), (cause 
instanceof NativeIoException) ? cause ? null);
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1098: Consolidate deployment assets into a single directory

[GitBox]

merlimat commented on issue #1098: Consolidate deployment assets into a single 
directory
URL: https://github.com/apache/incubator-pulsar/pull/1098#issuecomment-360262323
 
 
   @lucperkins One thing I forgot about this change is that we need to fix the 
configuration for the source distribution in `all/src/assemble/src.xml`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1087: Add basic authentication plugin

[GitBox]

merlimat commented on a change in pull request #1087: Add basic authentication 
plugin
URL: https://github.com/apache/incubator-pulsar/pull/1087#discussion_r163461362
 
 

 ##
 File path: 
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationService.java
 ##
 @@ -81,12 +81,15 @@ public String authenticate(AuthenticationDataSource 
authData, String authMethodN
 }
 }
 
-public String authenticateHttpRequest(HttpServletRequest request) throws 
AuthenticationException {
+public String authenticateHttpRequest(HttpServletRequest request) throws 
PulsarHttpAuthenticationException {
+PulsarHttpAuthenticationException exception = new 
PulsarHttpAuthenticationException("Authentication required");
 
 Review comment:
   Creating the exception is a very costly operation because it fills up the 
stack trace, we should only do it in the failure path


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1104: Set loadBalancerEnabled=true in ServiceConfiguration

[GitBox]

rdhabalia commented on issue #1104: Set loadBalancerEnabled=true in 
ServiceConfiguration
URL: https://github.com/apache/incubator-pulsar/pull/1104#issuecomment-360238841
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia opened a new pull request #1106: Fix: ignore auth flag for websocket-proxy to broker authentication

[GitBox]

rdhabalia opened a new pull request #1106: Fix: ignore auth flag for 
websocket-proxy to broker authentication
URL: https://github.com/apache/incubator-pulsar/pull/1106
 
 
   ### Motivation
   
   Right now, when user wants to deploy websocket proxy which communicates to 
broker it requires
   - Disable client to websocket authentication by disabling 
`authenticationEnabled` flag.
   - Enable websocket proxy to broker authentication if auth-param present.
   So, websocket to broker authentication should not depend on 
`authenticationEnabled` flag.
   
   ### Modifications
   
   Remove websocket to broker auth dependency form `authenticationEnabled` flag.
   
   ### Result
   
   User can disable webscoket authentication and same user can enable websocket 
to broker authentication also.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat opened a new pull request #1107: Log only first exception in the connection exception handler

[GitBox]

merlimat opened a new pull request #1107: Log only first exception in the 
connection exception handler
URL: https://github.com/apache/incubator-pulsar/pull/1107
 
 
   ### Motivation
   
   After the switch to Netty-4.1, we have seen that when a connection is reset 
all the pending write operations are triggering a `NativeIoException` that is 
caught and logged in the `exceptionCaught()` handler. 
   
   If there are many topics sharing a given connection, this can print several 
thousand exception stacktraces for all the pending writes. 
   
   ### Modification 
* Only log the first exception occurrence, and silence all subsequent 
errors since the connection is being closed anyway.
* Do not print stack traces for well known exceptions that are triggered on 
connections resets.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat opened a new pull request #1108: Fixed race condition between write operation and send timeout

[GitBox]

merlimat opened a new pull request #1108: Fixed race condition between write 
operation and send timeout
URL: https://github.com/apache/incubator-pulsar/pull/1108
 
 
   ### Motivation
   
   There is a possible race condition between a send operation getting written 
on a socket channel and getting timed-out at the same time. The problem is 
reproducible by setting very low timeout (eg: 10ms) at sustained publish rate. 
   
   The reason of the race condition is that we pass the instance of the 
`OpSendMsg` to the io-thread to get the `ByteBuf` to write into the socket but 
the timeout is touching it as well. We should instead take a reference on the 
`ByteBuf` itself for which we have already a reference increased for the write 
operation.
   
   This is the seen exception:
   
   ```
   2018-01-24 14:24:08,738 - WARN  - [pulsar-client-io-2-2:Slf4JLogger@146] - A 
task raised an exception. Task: 
org.apache.pulsar.client.impl.ProducerImpl$WriteInEventLoopCallback@4b2480e1
   java.lang.NullPointerException: msg
at 
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:785)
at 
org.apache.pulsar.client.impl.ProducerImpl$WriteInEventLoopCallback.run(ProducerImpl.java:462)
at 
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at 
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
at 
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:745)
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1108: Fixed race condition between write operation and send timeout

[GitBox]

merlimat commented on a change in pull request #1108: Fixed race condition 
between write operation and send timeout
URL: https://github.com/apache/incubator-pulsar/pull/1108#discussion_r163710145
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
 ##
 @@ -440,26 +441,28 @@ private boolean canEnqueueRequest(SendCallback callback) 
{
 
 private static final class WriteInEventLoopCallback implements Runnable {
 private ProducerImpl producer;
+private ByteBufPair cmd;
+private long sequenceId;
 private ClientCnx cnx;
-private OpSendMsg op;
 
 static WriteInEventLoopCallback create(ProducerImpl producer, 
ClientCnx cnx, OpSendMsg op) {
 WriteInEventLoopCallback c = RECYCLER.get();
 c.producer = producer;
 c.cnx = cnx;
-c.op = op;
+c.sequenceId = op.sequenceId;
+c.cmd = op.cmd;
 return c;
 }
 
 @Override
 public void run() {
 if (log.isDebugEnabled()) {
 log.debug("[{}] [{}] Sending message cnx {}, sequenceId {}", 
producer.topic, producer.producerName, cnx,
-op.sequenceId);
+sequenceId);
 }
 
 try {
-cnx.ctx().writeAndFlush(op.cmd, cnx.ctx().voidPromise());
+cnx.ctx().writeAndFlush(cmd, cnx.ctx().voidPromise());
 } finally {
 
 Review comment:
   The buffer gets release when the write operation completes


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1105: Advertise localhost address for unit test

[GitBox]

rdhabalia commented on issue #1105: Advertise localhost address for unit test
URL: https://github.com/apache/incubator-pulsar/pull/1105#issuecomment-360318596
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1105: Advertise localhost address for unit test

[GitBox]

rdhabalia commented on issue #1105: Advertise localhost address for unit test
URL: https://github.com/apache/incubator-pulsar/pull/1105#issuecomment-360318596
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jai1 commented on a change in pull request #1108: Fixed race condition between write operation and send timeout

[GitBox]

jai1 commented on a change in pull request #1108: Fixed race condition between 
write operation and send timeout
URL: https://github.com/apache/incubator-pulsar/pull/1108#discussion_r163709882
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
 ##
 @@ -440,26 +441,28 @@ private boolean canEnqueueRequest(SendCallback callback) 
{
 
 private static final class WriteInEventLoopCallback implements Runnable {
 private ProducerImpl producer;
+private ByteBufPair cmd;
+private long sequenceId;
 private ClientCnx cnx;
-private OpSendMsg op;
 
 static WriteInEventLoopCallback create(ProducerImpl producer, 
ClientCnx cnx, OpSendMsg op) {
 WriteInEventLoopCallback c = RECYCLER.get();
 c.producer = producer;
 c.cnx = cnx;
-c.op = op;
+c.sequenceId = op.sequenceId;
+c.cmd = op.cmd;
 return c;
 }
 
 @Override
 public void run() {
 if (log.isDebugEnabled()) {
 log.debug("[{}] [{}] Sending message cnx {}, sequenceId {}", 
producer.topic, producer.producerName, cnx,
-op.sequenceId);
+sequenceId);
 }
 
 try {
-cnx.ctx().writeAndFlush(op.cmd, cnx.ctx().voidPromise());
+cnx.ctx().writeAndFlush(cmd, cnx.ctx().voidPromise());
 } finally {
 
 Review comment:
   Don't we need 
   ReferenceCountUtil.safeRelease(cmd);
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lucperkins opened a new pull request #1109: Fix source distribution XML config to reflect new directory structure (WIP)

[GitBox]

lucperkins opened a new pull request #1109: Fix source distribution XML config 
to reflect new directory structure (WIP)
URL: https://github.com/apache/incubator-pulsar/pull/1109
 
 
   @merlimat Should the entire `deployment` directory be included?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Introduce config to skip non-recoverable data-ledger (#1046)

[rdhabalia]

This is an automated email from the ASF dual-hosted git repository.

rdhabalia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new f5a2ec7  Introduce config to skip non-recoverable data-ledger (#1046)
f5a2ec7 is described below

commit f5a2ec74e834e50f1970927f8fa6c107f7a3453c
Author: Rajan Dhabalia 
AuthorDate: Wed Jan 24 13:38:39 2018 -0800

Introduce config to skip non-recoverable data-ledger (#1046)
---
 conf/broker.conf   |   3 +
 conf/standalone.conf   |   3 +
 .../bookkeeper/mledger/ManagedLedgerConfig.java|  15 +++
 .../bookkeeper/mledger/ManagedLedgerException.java |  13 +++
 .../bookkeeper/mledger/impl/EntryCacheImpl.java|  11 +-
 .../bookkeeper/mledger/impl/EntryCacheManager.java |   3 +-
 .../bookkeeper/mledger/impl/ManagedCursorImpl.java |  25 +++--
 .../mledger/impl/ManagedLedgerFactoryImpl.java |   3 +-
 .../bookkeeper/mledger/impl/ManagedLedgerImpl.java |  50 +++--
 .../bookkeeper/mledger/impl/OpReadEntry.java   |  25 -
 .../apache/pulsar/broker/ServiceConfiguration.java |  12 ++
 .../pulsar/broker/service/BrokerService.java   |  26 +
 .../broker/service/BrokerBkEnsemblesTests.java | 125 -
 site/_data/config/broker.yaml  |   3 +
 site/_data/config/standalone.yaml  |   2 +
 15 files changed, 287 insertions(+), 32 deletions(-)

diff --git a/conf/broker.conf b/conf/broker.conf
index 20da0a9..d983b3f 100644
--- a/conf/broker.conf
+++ b/conf/broker.conf
@@ -300,6 +300,9 @@ managedLedgerMaxUnackedRangesToPersist=1
 # zookeeper.
 managedLedgerMaxUnackedRangesToPersistInZooKeeper=1000
 
+# Skip reading non-recoverable/unreadable data-ledger under managed-ledger's 
list. It helps when data-ledgers gets
+# corrupted at bookkeeper and managed-cursor is stuck at that ledger.
+autoSkipNonRecoverableData=false
 
 ### --- Load balancer --- ###
 
diff --git a/conf/standalone.conf b/conf/standalone.conf
index ce35f2e..ec400fb 100644
--- a/conf/standalone.conf
+++ b/conf/standalone.conf
@@ -266,6 +266,9 @@ managedLedgerMaxUnackedRangesToPersist=1
 # zookeeper.
 managedLedgerMaxUnackedRangesToPersistInZooKeeper=1000
 
+# Skip reading non-recoverable/unreadable data-ledger under managed-ledger's 
list. It helps when data-ledgers gets
+# corrupted at bookkeeper and managed-cursor is stuck at that ledger.
+autoSkipNonRecoverableData=false
 
 ### --- Load balancer --- ###
 
diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
index 391a484..6f9847b 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
@@ -51,6 +51,7 @@ public class ManagedLedgerConfig {
 private double throttleMarkDelete = 0;
 private long retentionTimeMs = 0;
 private long retentionSizeInMB = 0;
+private boolean autoSkipNonRecoverableData;
 
 private DigestType digestType = DigestType.MAC;
 private byte[] password = "".getBytes(Charsets.UTF_8);
@@ -354,6 +355,20 @@ public class ManagedLedgerConfig {
 }
 
 /**
+ * Skip reading non-recoverable/unreadable data-ledger under 
managed-ledger's list. It helps when data-ledgers gets
+ * corrupted at bookkeeper and managed-cursor is stuck at that ledger.
+ * 
+ * @param autoSkipNonRecoverableData
+ */
+public boolean isAutoSkipNonRecoverableData() {
+return autoSkipNonRecoverableData;
+}
+
+public void setAutoSkipNonRecoverableData(boolean skipNonRecoverableData) {
+this.autoSkipNonRecoverableData = skipNonRecoverableData;
+}
+
+/**
  * @return max unacked message ranges that will be persisted and recovered.
  *
  */
diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
index 1817aaf..f5c4243 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
@@ -31,6 +31,13 @@ public class ManagedLedgerException extends Exception {
 super(e);
 }
 
+public static ManagedLedgerException getManagedLedgerException(Throwable 
e) {
+if (e instanceof ManagedLedgerException) {
+return (ManagedLedgerException) e;
+}
+return new ManagedLedgerException(e);
+}
+
 public static class MetaStoreException extends ManagedLedgerException {
 public MetaStoreException(Exception e) {
 super(e);
@@ -89,6 

[GitHub] rdhabalia closed pull request #1046: Introduce config to skip non-recoverable data-ledger

[GitBox]

rdhabalia closed pull request #1046: Introduce config to skip non-recoverable 
data-ledger
URL: https://github.com/apache/incubator-pulsar/pull/1046
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/conf/broker.conf b/conf/broker.conf
index 20da0a910..d983b3f2d 100644
--- a/conf/broker.conf
+++ b/conf/broker.conf
@@ -300,6 +300,9 @@ managedLedgerMaxUnackedRangesToPersist=1
 # zookeeper.
 managedLedgerMaxUnackedRangesToPersistInZooKeeper=1000
 
+# Skip reading non-recoverable/unreadable data-ledger under managed-ledger's 
list. It helps when data-ledgers gets
+# corrupted at bookkeeper and managed-cursor is stuck at that ledger.
+autoSkipNonRecoverableData=false
 
 ### --- Load balancer --- ###
 
diff --git a/conf/standalone.conf b/conf/standalone.conf
index ce35f2ebd..ec400fbc6 100644
--- a/conf/standalone.conf
+++ b/conf/standalone.conf
@@ -266,6 +266,9 @@ managedLedgerMaxUnackedRangesToPersist=1
 # zookeeper.
 managedLedgerMaxUnackedRangesToPersistInZooKeeper=1000
 
+# Skip reading non-recoverable/unreadable data-ledger under managed-ledger's 
list. It helps when data-ledgers gets
+# corrupted at bookkeeper and managed-cursor is stuck at that ledger.
+autoSkipNonRecoverableData=false
 
 ### --- Load balancer --- ###
 
diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
index 391a484f9..6f9847bb1 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java
@@ -51,6 +51,7 @@
 private double throttleMarkDelete = 0;
 private long retentionTimeMs = 0;
 private long retentionSizeInMB = 0;
+private boolean autoSkipNonRecoverableData;
 
 private DigestType digestType = DigestType.MAC;
 private byte[] password = "".getBytes(Charsets.UTF_8);
@@ -353,6 +354,20 @@ public long getRetentionSizeInMB() {
 return retentionSizeInMB;
 }
 
+/**
+ * Skip reading non-recoverable/unreadable data-ledger under 
managed-ledger's list. It helps when data-ledgers gets
+ * corrupted at bookkeeper and managed-cursor is stuck at that ledger.
+ * 
+ * @param autoSkipNonRecoverableData
+ */
+public boolean isAutoSkipNonRecoverableData() {
+return autoSkipNonRecoverableData;
+}
+
+public void setAutoSkipNonRecoverableData(boolean skipNonRecoverableData) {
+this.autoSkipNonRecoverableData = skipNonRecoverableData;
+}
+
 /**
  * @return max unacked message ranges that will be persisted and recovered.
  *
diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
index 1817aaf59..f5c4243e5 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerException.java
@@ -31,6 +31,13 @@ public ManagedLedgerException(Throwable e) {
 super(e);
 }
 
+public static ManagedLedgerException getManagedLedgerException(Throwable 
e) {
+if (e instanceof ManagedLedgerException) {
+return (ManagedLedgerException) e;
+}
+return new ManagedLedgerException(e);
+}
+
 public static class MetaStoreException extends ManagedLedgerException {
 public MetaStoreException(Exception e) {
 super(e);
@@ -89,6 +96,12 @@ public TooManyRequestsException(String msg) {
 }
 }
 
+public static class NonRecoverableLedgerException extends 
ManagedLedgerException {
+public NonRecoverableLedgerException(String msg) {
+super(msg);
+}
+}
+
 public static class InvalidReplayPositionException extends 
ManagedLedgerException {
 public InvalidReplayPositionException(String msg) {
 super(msg);
diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/EntryCacheImpl.java
 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/EntryCacheImpl.java
index b89dfb569..469a0e949 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/EntryCacheImpl.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/EntryCacheImpl.java
@@ -25,14 +25,13 @@
 import java.util.Collection;
 import java.util.List;
 
-import org.apache.bookkeeper.client.AsyncCallback.ReadCallback;
 import org.apache.bookkeeper.client.BKException;
 import org.apache.bookkeeper.client.LedgerEntry;
 import org.apache.bookkeeper.client.LedgerHandle;
 import 

[GitHub] rdhabalia commented on issue #1107: Log only first exception in the connection exception handler

[GitBox]

rdhabalia commented on issue #1107: Log only first exception in the connection 
exception handler
URL: https://github.com/apache/incubator-pulsar/pull/1107#issuecomment-360282380
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat opened a new pull request #1110: Fix ByteBufPair encoder to release buffers

[GitBox]

merlimat opened a new pull request #1110: Fix ByteBufPair encoder to release 
buffers
URL: https://github.com/apache/incubator-pulsar/pull/1110
 
 
   ### Motivation
   
   In the recent commit that changed the `DoubleByteBuf` for the `ByteBufPair`, 
the provided encoder was not releasing the ref-count on the `ByteBufPair` after 
writing on the socket channel.
   
   Note: this doesn't affect 1.21 release, just master code.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and Anti-affinity-namespace group

[GitBox]

rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and 
Anti-affinity-namespace group
URL: https://github.com/apache/incubator-pulsar/pull/896#issuecomment-360318811
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yush1ga commented on a change in pull request #1087: Add basic authentication plugin

[GitBox]

yush1ga commented on a change in pull request #1087: Add basic authentication 
plugin
URL: https://github.com/apache/incubator-pulsar/pull/1087#discussion_r163719058
 
 

 ##
 File path: 
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationService.java
 ##
 @@ -81,12 +81,15 @@ public String authenticate(AuthenticationDataSource 
authData, String authMethodN
 }
 }
 
-public String authenticateHttpRequest(HttpServletRequest request) throws 
AuthenticationException {
+public String authenticateHttpRequest(HttpServletRequest request) throws 
PulsarHttpAuthenticationException {
+PulsarHttpAuthenticationException exception = new 
PulsarHttpAuthenticationException("Authentication required");
 
 Review comment:
   I made it create `realmInformation` string to avoid creating an exception.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1108: Fixed race condition between write operation and send timeout

[GitBox]

merlimat commented on issue #1108: Fixed race condition between write operation 
and send timeout
URL: https://github.com/apache/incubator-pulsar/pull/1108#issuecomment-360316720
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1108: Fixed race condition between write operation and send timeout

[GitBox]

merlimat commented on issue #1108: Fixed race condition between write operation 
and send timeout
URL: https://github.com/apache/incubator-pulsar/pull/1108#issuecomment-360316720
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1107: Log only first exception in the connection exception handler

[GitBox]

rdhabalia commented on issue #1107: Log only first exception in the connection 
exception handler
URL: https://github.com/apache/incubator-pulsar/pull/1107#issuecomment-360282380
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1105: Advertise localhost address for unit test

[GitBox]

rdhabalia commented on issue #1105: Advertise localhost address for unit test
URL: https://github.com/apache/incubator-pulsar/pull/1105#issuecomment-360282550
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1105: Advertise localhost address for unit test

[GitBox]

rdhabalia commented on issue #1105: Advertise localhost address for unit test
URL: https://github.com/apache/incubator-pulsar/pull/1105#issuecomment-360282550
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and Anti-affinity-namespace group

[GitBox]

rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and 
Anti-affinity-namespace group
URL: https://github.com/apache/incubator-pulsar/pull/896#issuecomment-360289850
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and Anti-affinity-namespace group

[GitBox]

rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and 
Anti-affinity-namespace group
URL: https://github.com/apache/incubator-pulsar/pull/896#issuecomment-360290095
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and Anti-affinity-namespace group

[GitBox]

rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and 
Anti-affinity-namespace group
URL: https://github.com/apache/incubator-pulsar/pull/896#issuecomment-360289850
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and Anti-affinity-namespace group

[GitBox]

rdhabalia commented on issue #896: PIP-7 Introduce Failure-domain and 
Anti-affinity-namespace group
URL: https://github.com/apache/incubator-pulsar/pull/896#issuecomment-360290095
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lucperkins commented on issue #1098: Consolidate deployment assets into a single directory

[GitBox]

lucperkins commented on issue #1098: Consolidate deployment assets into a 
single directory
URL: https://github.com/apache/incubator-pulsar/pull/1098#issuecomment-360310712
 
 
   @merlimat Okay, I'll take a look.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1110: Fix ByteBufPair encoder to release buffers

[GitBox]

merlimat commented on issue #1110: Fix ByteBufPair encoder to release buffers
URL: https://github.com/apache/incubator-pulsar/pull/1110#issuecomment-360368602
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1104: Set loadBalancerEnabled=true in ServiceConfiguration

[GitBox]

merlimat commented on issue #1104: Set loadBalancerEnabled=true in 
ServiceConfiguration
URL: https://github.com/apache/incubator-pulsar/pull/1104#issuecomment-360326981
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1066: Issue 937: add CommandGetLastMessageId to make reader know the end of topic

[GitBox]

zhaijack commented on issue #1066: Issue 937: add CommandGetLastMessageId to 
make reader know the end of topic
URL: https://github.com/apache/incubator-pulsar/pull/1066#issuecomment-360331086
 
 
   @ivankelly @sijie @merlimat Would you please help take a look of the new 
change? Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Fixed race condition between write operation and send timeout (#1108)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new e1ad269  Fixed race condition between write operation and send timeout 
(#1108)
e1ad269 is described below

commit e1ad269f32334a274565c95f0745468d58a2720b
Author: Matteo Merli 
AuthorDate: Wed Jan 24 18:19:24 2018 -0800

Fixed race condition between write operation and send timeout (#1108)
---
 .../java/org/apache/pulsar/client/impl/ProducerImpl.java   | 14 +-
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
index 1124658..32d61a5 100644
--- 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
+++ 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
@@ -20,6 +20,7 @@ package org.apache.pulsar.client.impl;
 
 import static com.google.common.base.Preconditions.checkArgument;
 import static java.lang.String.format;
+import static com.google.common.base.Preconditions.checkNotNull;
 import static org.apache.pulsar.checksum.utils.Crc32cChecksum.computeChecksum;
 import static org.apache.pulsar.checksum.utils.Crc32cChecksum.resumeChecksum;
 import static org.apache.pulsar.common.api.Commands.hasChecksum;
@@ -440,14 +441,16 @@ public class ProducerImpl extends ProducerBase implements 
TimerTask {
 
 private static final class WriteInEventLoopCallback implements Runnable {
 private ProducerImpl producer;
+private ByteBufPair cmd;
+private long sequenceId;
 private ClientCnx cnx;
-private OpSendMsg op;
 
 static WriteInEventLoopCallback create(ProducerImpl producer, 
ClientCnx cnx, OpSendMsg op) {
 WriteInEventLoopCallback c = RECYCLER.get();
 c.producer = producer;
 c.cnx = cnx;
-c.op = op;
+c.sequenceId = op.sequenceId;
+c.cmd = op.cmd;
 return c;
 }
 
@@ -455,11 +458,11 @@ public class ProducerImpl extends ProducerBase implements 
TimerTask {
 public void run() {
 if (log.isDebugEnabled()) {
 log.debug("[{}] [{}] Sending message cnx {}, sequenceId {}", 
producer.topic, producer.producerName, cnx,
-op.sequenceId);
+sequenceId);
 }
 
 try {
-cnx.ctx().writeAndFlush(op.cmd, cnx.ctx().voidPromise());
+cnx.ctx().writeAndFlush(cmd, cnx.ctx().voidPromise());
 } finally {
 recycle();
 }
@@ -468,7 +471,8 @@ public class ProducerImpl extends ProducerBase implements 
TimerTask {
 private void recycle() {
 producer = null;
 cnx = null;
-op = null;
+cmd = null;
+sequenceId = -1;
 recyclerHandle.recycle(this);
 }
 

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[incubator-pulsar] branch master updated: Log only first exception in the connection exception handler (#1107)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new f57cd88  Log only first exception in the connection exception handler 
(#1107)
f57cd88 is described below

commit f57cd88d82e4412d6031301bc1b1818b3a0cbcfe
Author: Matteo Merli 
AuthorDate: Wed Jan 24 18:19:57 2018 -0800

Log only first exception in the connection exception handler (#1107)
---
 .../apache/pulsar/broker/service/ServerCnx.java| 16 ++--
 .../org/apache/pulsar/client/impl/ClientCnx.java   | 22 --
 2 files changed, 34 insertions(+), 4 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index df8e577..bd49fc7 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -44,6 +44,7 @@ import 
org.apache.pulsar.broker.service.BrokerServiceException.ServiceUnitNotRea
 import org.apache.pulsar.broker.web.RestException;
 import org.apache.pulsar.client.api.PulsarClientException;
 import org.apache.pulsar.client.impl.BatchMessageIdImpl;
+import org.apache.pulsar.client.impl.ClientCnx;
 import org.apache.pulsar.client.impl.MessageIdImpl;
 import org.apache.pulsar.common.api.CommandUtils;
 import org.apache.pulsar.common.api.Commands;
@@ -102,7 +103,7 @@ public class ServerCnx extends PulsarHandler {
 private String originalPrincipal;
 
 enum State {
-Start, Connected
+Start, Connected, Failed
 }
 
 public ServerCnx(BrokerService service) {
@@ -164,7 +165,18 @@ public class ServerCnx extends PulsarHandler {
 
 @Override
 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) 
throws Exception {
-log.warn("[{}] Got exception: {}", remoteAddress, cause.getMessage(), 
cause);
+if (state != State.Failed) {
+// No need to report stack trace for known exceptions that happen 
in disconnections
+log.warn("[{}] Got exception {} : {}", remoteAddress, 
cause.getClass().getSimpleName(), cause.getMessage(),
+ClientCnx.isKnownException(cause) ? null : cause);
+state = State.Failed;
+} else {
+// At default info level, suppress all subsequent exceptions that 
are thrown when the connection has already
+// failed
+if (log.isDebugEnabled()) {
+log.debug("[{}] Got exception: {}", remoteAddress, 
cause.getMessage(), cause);
+}
+}
 ctx.close();
 }
 
diff --git 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
index 2eaa1d4..85e2867 100644
--- a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
+++ b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
@@ -23,6 +23,7 @@ import static 
org.apache.pulsar.client.impl.HttpClient.getPulsarClientVersion;
 
 import java.net.InetSocketAddress;
 import java.net.SocketAddress;
+import java.nio.channels.ClosedChannelException;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.Semaphore;
 import java.util.concurrent.TimeUnit;
@@ -57,6 +58,7 @@ import io.netty.buffer.ByteBuf;
 import io.netty.channel.Channel;
 import io.netty.channel.ChannelHandlerContext;
 import io.netty.channel.EventLoopGroup;
+import io.netty.channel.unix.Errors.NativeIoException;
 import io.netty.util.concurrent.Promise;
 
 public class ClientCnx extends PulsarHandler {
@@ -85,7 +87,7 @@ public class ClientCnx extends PulsarHandler {
 private String proxyToTargetBrokerAddress = null;
 
 enum State {
-None, SentConnectFrame, Ready
+None, SentConnectFrame, Ready, Failed
 }
 
 public ClientCnx(ClientConfiguration conf, EventLoopGroup eventLoopGroup) {
@@ -152,10 +154,26 @@ public class ClientCnx extends PulsarHandler {
 
 @Override
 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) 
throws Exception {
-log.warn("{} Exception caught: {}", ctx.channel(), cause.getMessage(), 
cause);
+if (state != State.Failed) {
+// No need to report stack trace for known exceptions that happen 
in disconnections
+log.warn("[{}] Got exception {} : {}", remoteAddress, 
cause.getClass().getSimpleName(), cause.getMessage(),
+isKnownException(cause) ? null : cause);
+state = State.Failed;
+} else {
+// At default info level, suppress all subsequent exceptions that 
are thrown when the connection has already
+// failed
+if 

[GitHub] merlimat closed pull request #1108: Fixed race condition between write operation and send timeout

[GitBox]

merlimat closed pull request #1108: Fixed race condition between write 
operation and send timeout
URL: https://github.com/apache/incubator-pulsar/pull/1108
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
index 112465802..32d61a532 100644
--- 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
+++ 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
@@ -20,6 +20,7 @@
 
 import static com.google.common.base.Preconditions.checkArgument;
 import static java.lang.String.format;
+import static com.google.common.base.Preconditions.checkNotNull;
 import static org.apache.pulsar.checksum.utils.Crc32cChecksum.computeChecksum;
 import static org.apache.pulsar.checksum.utils.Crc32cChecksum.resumeChecksum;
 import static org.apache.pulsar.common.api.Commands.hasChecksum;
@@ -440,14 +441,16 @@ private boolean canEnqueueRequest(SendCallback callback) {
 
 private static final class WriteInEventLoopCallback implements Runnable {
 private ProducerImpl producer;
+private ByteBufPair cmd;
+private long sequenceId;
 private ClientCnx cnx;
-private OpSendMsg op;
 
 static WriteInEventLoopCallback create(ProducerImpl producer, 
ClientCnx cnx, OpSendMsg op) {
 WriteInEventLoopCallback c = RECYCLER.get();
 c.producer = producer;
 c.cnx = cnx;
-c.op = op;
+c.sequenceId = op.sequenceId;
+c.cmd = op.cmd;
 return c;
 }
 
@@ -455,11 +458,11 @@ static WriteInEventLoopCallback create(ProducerImpl 
producer, ClientCnx cnx, OpS
 public void run() {
 if (log.isDebugEnabled()) {
 log.debug("[{}] [{}] Sending message cnx {}, sequenceId {}", 
producer.topic, producer.producerName, cnx,
-op.sequenceId);
+sequenceId);
 }
 
 try {
-cnx.ctx().writeAndFlush(op.cmd, cnx.ctx().voidPromise());
+cnx.ctx().writeAndFlush(cmd, cnx.ctx().voidPromise());
 } finally {
 recycle();
 }
@@ -468,7 +471,8 @@ public void run() {
 private void recycle() {
 producer = null;
 cnx = null;
-op = null;
+cmd = null;
+sequenceId = -1;
 recyclerHandle.recycle(this);
 }
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1107: Log only first exception in the connection exception handler

[GitBox]

merlimat closed pull request #1107: Log only first exception in the connection 
exception handler
URL: https://github.com/apache/incubator-pulsar/pull/1107
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index df8e577c7..bd49fc76f 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -44,6 +44,7 @@
 import org.apache.pulsar.broker.web.RestException;
 import org.apache.pulsar.client.api.PulsarClientException;
 import org.apache.pulsar.client.impl.BatchMessageIdImpl;
+import org.apache.pulsar.client.impl.ClientCnx;
 import org.apache.pulsar.client.impl.MessageIdImpl;
 import org.apache.pulsar.common.api.CommandUtils;
 import org.apache.pulsar.common.api.Commands;
@@ -102,7 +103,7 @@
 private String originalPrincipal;
 
 enum State {
-Start, Connected
+Start, Connected, Failed
 }
 
 public ServerCnx(BrokerService service) {
@@ -164,7 +165,18 @@ public void 
channelWritabilityChanged(ChannelHandlerContext ctx) throws Exceptio
 
 @Override
 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) 
throws Exception {
-log.warn("[{}] Got exception: {}", remoteAddress, cause.getMessage(), 
cause);
+if (state != State.Failed) {
+// No need to report stack trace for known exceptions that happen 
in disconnections
+log.warn("[{}] Got exception {} : {}", remoteAddress, 
cause.getClass().getSimpleName(), cause.getMessage(),
+ClientCnx.isKnownException(cause) ? null : cause);
+state = State.Failed;
+} else {
+// At default info level, suppress all subsequent exceptions that 
are thrown when the connection has already
+// failed
+if (log.isDebugEnabled()) {
+log.debug("[{}] Got exception: {}", remoteAddress, 
cause.getMessage(), cause);
+}
+}
 ctx.close();
 }
 
diff --git 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
index 2eaa1d459..85e286745 100644
--- a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
+++ b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
@@ -23,6 +23,7 @@
 
 import java.net.InetSocketAddress;
 import java.net.SocketAddress;
+import java.nio.channels.ClosedChannelException;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.Semaphore;
 import java.util.concurrent.TimeUnit;
@@ -57,6 +58,7 @@
 import io.netty.channel.Channel;
 import io.netty.channel.ChannelHandlerContext;
 import io.netty.channel.EventLoopGroup;
+import io.netty.channel.unix.Errors.NativeIoException;
 import io.netty.util.concurrent.Promise;
 
 public class ClientCnx extends PulsarHandler {
@@ -85,7 +87,7 @@
 private String proxyToTargetBrokerAddress = null;
 
 enum State {
-None, SentConnectFrame, Ready
+None, SentConnectFrame, Ready, Failed
 }
 
 public ClientCnx(ClientConfiguration conf, EventLoopGroup eventLoopGroup) {
@@ -152,10 +154,26 @@ public void channelInactive(ChannelHandlerContext ctx) 
throws Exception {
 
 @Override
 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) 
throws Exception {
-log.warn("{} Exception caught: {}", ctx.channel(), cause.getMessage(), 
cause);
+if (state != State.Failed) {
+// No need to report stack trace for known exceptions that happen 
in disconnections
+log.warn("[{}] Got exception {} : {}", remoteAddress, 
cause.getClass().getSimpleName(), cause.getMessage(),
+isKnownException(cause) ? null : cause);
+state = State.Failed;
+} else {
+// At default info level, suppress all subsequent exceptions that 
are thrown when the connection has already
+// failed
+if (log.isDebugEnabled()) {
+log.debug("[{}] Got exception: {}", remoteAddress, 
cause.getMessage(), cause);
+}
+}
+
 ctx.close();
 }
 
+public static boolean isKnownException(Throwable t) {
+return t instanceof NativeIoException || t instanceof 
ClosedChannelException;
+}
+
 @Override
 protected void handleConnected(CommandConnected connected) {
 checkArgument(state == State.SentConnectFrame);


 


This is an automated 

[incubator-pulsar] branch master updated: Set loadBalancerEnabled=true in ServiceConfiguration (#1104)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new ed488b2  Set loadBalancerEnabled=true in ServiceConfiguration (#1104)
ed488b2 is described below

commit ed488b266c9dc63ca4728ee171dd58421727d9d7
Author: Matteo Merli 
AuthorDate: Wed Jan 24 21:23:14 2018 -0800

Set loadBalancerEnabled=true in ServiceConfiguration (#1104)

* Set loadBalancerEnabled=true in ServiceConfiguration

* Fixed tests to run with same settings as before

* One more test config fix
---
 .../src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java   | 2 +-
 .../src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java  | 1 +
 .../java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java| 3 ++-
 3 files changed, 4 insertions(+), 2 deletions(-)

diff --git 
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
 
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
index 7f01a87..88117e0 100644
--- 
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
+++ 
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
@@ -295,7 +295,7 @@ public class ServiceConfiguration implements 
PulsarConfiguration {
 
 /*** --- Load balancer --- /
 // Enable load balancer
-private boolean loadBalancerEnabled = false;
+private boolean loadBalancerEnabled = true;
 // load placement strategy[weightedRandomSelection/leastLoadedServer] 
(only used by SimpleLoadManagerImpl)
 @Deprecated
 private String loadBalancerPlacementStrategy = "leastLoadedServer"; // 
weighted random selection
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
index 8e7db9d..1deb88b 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
@@ -93,6 +93,7 @@ public class SLAMonitoringTest {
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 config.setDefaultNumberOfNamespaceBundles(1);
+config.setLoadBalancerEnabled(false);
 configurations[i] = config;
 
 pulsarServices[i] = new PulsarService(config);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
index 80ca562..a877013 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
@@ -142,7 +142,8 @@ public class LoadBalancerTest {
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 
config.setLoadManagerClassName(SimpleLoadManagerImpl.class.getName());
-config.setAdvertisedAddress(localhost+i);;
+config.setAdvertisedAddress(localhost+i);
+config.setLoadBalancerEnabled(false);
 
 pulsarServices[i] = new PulsarService(config);
 pulsarServices[i].start();

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[incubator-pulsar] branch master updated: Fix: ignore auth flag for websocket-proxy to broker authentication (#1106)

[rdhabalia]

This is an automated email from the ASF dual-hosted git repository.

rdhabalia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 574ebde  Fix: ignore auth flag for websocket-proxy to broker 
authentication (#1106)
574ebde is described below

commit 574ebdebaf0df57da7ee3f3fd19496124ae8cae7
Author: Rajan Dhabalia 
AuthorDate: Wed Jan 24 21:00:43 2018 -0800

Fix: ignore auth flag for websocket-proxy to broker authentication (#1106)
---
 .../src/main/java/org/apache/pulsar/websocket/WebSocketService.java| 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git 
a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
 
b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
index 28a0ed5..02a589f 100644
--- 
a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
+++ 
b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
@@ -184,7 +184,8 @@ public class WebSocketService implements Closeable {
 clientConf.setIoThreads(config.getWebSocketNumIoThreads());
 
clientConf.setConnectionsPerBroker(config.getWebSocketConnectionsPerBroker());
 
-if (config.isAuthenticationEnabled()) {
+if (isNotBlank(config.getBrokerClientAuthenticationPlugin())
+&& 
isNotBlank(config.getBrokerClientAuthenticationParameters())) {
 
clientConf.setAuthentication(config.getBrokerClientAuthenticationPlugin(),
 config.getBrokerClientAuthenticationParameters());
 }

-- 
To stop receiving notification emails like this one, please contact
rdhaba...@apache.org.

[GitHub] merlimat closed pull request #1104: Set loadBalancerEnabled=true in ServiceConfiguration

[GitBox]

merlimat closed pull request #1104: Set loadBalancerEnabled=true in 
ServiceConfiguration
URL: https://github.com/apache/incubator-pulsar/pull/1104
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
 
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
index 069933cf1..ec50d76de 100644
--- 
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
+++ 
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
@@ -288,7 +288,7 @@
 
 /*** --- Load balancer --- /
 // Enable load balancer
-private boolean loadBalancerEnabled = false;
+private boolean loadBalancerEnabled = true;
 // load placement strategy[weightedRandomSelection/leastLoadedServer] 
(only used by SimpleLoadManagerImpl)
 @Deprecated
 private String loadBalancerPlacementStrategy = "leastLoadedServer"; // 
weighted random selection
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
index e0d55a661..c80cec2ea 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
@@ -92,6 +92,7 @@ void setup() throws Exception {
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 config.setDefaultNumberOfNamespaceBundles(1);
+config.setLoadBalancerEnabled(false);
 configurations[i] = config;
 
 pulsarServices[i] = new PulsarService(config);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
index 8511e4058..80ed0d539 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
@@ -141,7 +141,8 @@ void setup() throws Exception {
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 
config.setLoadManagerClassName(SimpleLoadManagerImpl.class.getName());
-config.setAdvertisedAddress(localhost+i);;
+config.setAdvertisedAddress(localhost+i);
+config.setLoadBalancerEnabled(false);
 
 pulsarServices[i] = new PulsarService(config);
 pulsarServices[i].start();


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1105: Advertise localhost address for unit test

[GitBox]

rdhabalia commented on issue #1105: Advertise localhost address for unit test
URL: https://github.com/apache/incubator-pulsar/pull/1105#issuecomment-360328223
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia closed pull request #1106: Fix: ignore auth flag for websocket-proxy to broker authentication

[GitBox]

rdhabalia closed pull request #1106: Fix: ignore auth flag for websocket-proxy 
to broker authentication
URL: https://github.com/apache/incubator-pulsar/pull/1106
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
 
b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
index 28a0ed511..02a589ff5 100644
--- 
a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
+++ 
b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/WebSocketService.java
@@ -184,7 +184,8 @@ private PulsarClient createClientInstance(ClusterData 
clusterData) throws IOExce
 clientConf.setIoThreads(config.getWebSocketNumIoThreads());
 
clientConf.setConnectionsPerBroker(config.getWebSocketConnectionsPerBroker());
 
-if (config.isAuthenticationEnabled()) {
+if (isNotBlank(config.getBrokerClientAuthenticationPlugin())
+&& 
isNotBlank(config.getBrokerClientAuthenticationParameters())) {
 
clientConf.setAuthentication(config.getBrokerClientAuthenticationPlugin(),
 config.getBrokerClientAuthenticationParameters());
 }


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Advertise localhost address for unit test (#1105)

[rdhabalia]

This is an automated email from the ASF dual-hosted git repository.

rdhabalia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 72432ce  Advertise localhost address for unit test (#1105)
72432ce is described below

commit 72432ce7bb6ebbccb104147f3110a14f51c0a04c
Author: Rajan Dhabalia 
AuthorDate: Wed Jan 24 21:00:54 2018 -0800

Advertise localhost address for unit test (#1105)

* Advertise localhost address for unit test

* fix test
---
 .../src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java   | 1 +
 .../org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java  | 1 +
 .../java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java | 1 +
 .../org/apache/pulsar/broker/service/AdvertisedAddressTest.java | 1 +
 .../org/apache/pulsar/broker/service/BacklogQuotaManagerTest.java   | 1 +
 .../org/apache/pulsar/broker/service/BrokerBkEnsemblesTests.java| 1 +
 .../java/org/apache/pulsar/broker/service/ReplicatorTestBase.java   | 3 +++
 .../src/test/java/org/apache/pulsar/broker/web/WebServiceTest.java  | 1 +
 .../java/org/apache/pulsar/client/api/BrokerServiceLookupTest.java  | 6 ++
 .../java/org/apache/pulsar/client/api/NonPersistentTopicTest.java   | 3 +++
 10 files changed, 19 insertions(+)

diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
index 73071ff..8e7db9d 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
@@ -88,6 +88,7 @@ public class SLAMonitoringTest {
 ServiceConfiguration config = new ServiceConfiguration();
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 config.setClusterName("my-cluster");
+config.setAdvertisedAddress("localhost");
 config.setWebServicePort(brokerWebServicePorts[i]);
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
index 57ed5e5..c23d726 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
@@ -89,6 +89,7 @@ public abstract class MockedPulsarServiceBaseTest {
 this.conf = new ServiceConfiguration();
 this.conf.setBrokerServicePort(BROKER_PORT);
 this.conf.setBrokerServicePortTls(BROKER_PORT_TLS);
+this.conf.setAdvertisedAddress("localhost");
 this.conf.setWebServicePort(BROKER_WEBSERVICE_PORT);
 this.conf.setWebServicePortTls(BROKER_WEBSERVICE_PORT_TLS);
 this.conf.setClusterName(configClusterName);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
index 8511e40..80ca562 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
@@ -137,6 +137,7 @@ public class LoadBalancerTest {
 ServiceConfiguration config = new ServiceConfiguration();
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 config.setClusterName("use");
+config.setAdvertisedAddress("localhost");
 config.setWebServicePort(brokerWebServicePorts[i]);
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
index 266354f..dda76b1 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
@@ -50,6 +50,7 @@ public class AdvertisedAddressTest {
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setWebServicePort(BROKER_WEBSERVICE_PORT);
 config.setClusterName("usc");
+config.setAdvertisedAddress("localhost");
 config.setBrokerServicePort(BROKER_SERVICE_PORT);
 config.setAdvertisedAddress(advertisedAddress);
 config.setManagedLedgerMaxEntriesPerLedger(5);
diff --git 

[GitHub] rdhabalia closed pull request #1105: Advertise localhost address for unit test

[GitBox]

rdhabalia closed pull request #1105: Advertise localhost address for unit test
URL: https://github.com/apache/incubator-pulsar/pull/1105
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
index 73071ff17..8e7db9dfc 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/SLAMonitoringTest.java
@@ -88,6 +88,7 @@ void setup() throws Exception {
 ServiceConfiguration config = new ServiceConfiguration();
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 config.setClusterName("my-cluster");
+config.setAdvertisedAddress("localhost");
 config.setWebServicePort(brokerWebServicePorts[i]);
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
index 57ed5e5e4..c23d726db 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
@@ -89,6 +89,7 @@ protected void resetConfig() {
 this.conf = new ServiceConfiguration();
 this.conf.setBrokerServicePort(BROKER_PORT);
 this.conf.setBrokerServicePortTls(BROKER_PORT_TLS);
+this.conf.setAdvertisedAddress("localhost");
 this.conf.setWebServicePort(BROKER_WEBSERVICE_PORT);
 this.conf.setWebServicePortTls(BROKER_WEBSERVICE_PORT_TLS);
 this.conf.setClusterName(configClusterName);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
index 8511e4058..80ca562a1 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/LoadBalancerTest.java
@@ -137,6 +137,7 @@ void setup() throws Exception {
 ServiceConfiguration config = new ServiceConfiguration();
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
 config.setClusterName("use");
+config.setAdvertisedAddress("localhost");
 config.setWebServicePort(brokerWebServicePorts[i]);
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setBrokerServicePort(brokerNativeBrokerPorts[i]);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
index 266354f81..dda76b17e 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AdvertisedAddressTest.java
@@ -50,6 +50,7 @@ public void setup() throws Exception {
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
 config.setWebServicePort(BROKER_WEBSERVICE_PORT);
 config.setClusterName("usc");
+config.setAdvertisedAddress("localhost");
 config.setBrokerServicePort(BROKER_SERVICE_PORT);
 config.setAdvertisedAddress(advertisedAddress);
 config.setManagedLedgerMaxEntriesPerLedger(5);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BacklogQuotaManagerTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BacklogQuotaManagerTest.java
index 286a700b5..f1761d831 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BacklogQuotaManagerTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BacklogQuotaManagerTest.java
@@ -80,6 +80,7 @@ void setup() throws Exception {
 // start pulsar service
 config = new ServiceConfiguration();
 config.setZookeeperServers("127.0.0.1" + ":" + ZOOKEEPER_PORT);
+config.setAdvertisedAddress("localhost");
 config.setWebServicePort(BROKER_WEBSERVICE_PORT);
 config.setClusterName("usc");
 config.setBrokerServicePort(BROKER_SERVICE_PORT);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBkEnsemblesTests.java
 

[GitHub] zhaijack commented on issue #1014: Rename global zookeeper cluster to configuration/replication-configuration cluster

[GitBox]

zhaijack commented on issue #1014: Rename global zookeeper cluster to 
configuration/replication-configuration cluster
URL: 
https://github.com/apache/incubator-pulsar/issues/1014#issuecomment-360332644
 
 
   https://github.com/apache/incubator-pulsar/pull/1059 is for the fix. 
   @merlimat Since the PR is marked as 2.0, does  this mean the PR should wait 
after 1.22 released? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on issue #1106: Fix: ignore auth flag for websocket-proxy to broker authentication

[GitBox]

rdhabalia commented on issue #1106: Fix: ignore auth flag for websocket-proxy 
to broker authentication
URL: https://github.com/apache/incubator-pulsar/pull/1106#issuecomment-360338171
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] XiaoZYang commented on issue #1069: Provide a setting in consumer configuration to specify where to start consuming messages

[GitBox]

XiaoZYang commented on issue #1069: Provide a setting in consumer configuration 
to specify where to start consuming messages
URL: 
https://github.com/apache/incubator-pulsar/issues/1069#issuecomment-360351677
 
 
   I'd like to have a try on fixing this issue. :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack opened a new issue #1102: PIP-13-3: sub-regex: auto-discover topic addition/deletion

[GitBox]

zhaijack opened a new issue #1102: PIP-13-3: sub-regex: auto-discover topic 
addition/deletion
URL: https://github.com/apache/incubator-pulsar/issues/1102
 
 
   This is tracking the sub task 3 for [PIP 
13](https://github.com/apache/incubator-pulsar/wiki/PIP-13:-Subscribe-to-topics-represented-by-regular-expressions):
   PIP-13 has 3 sub-task:
   -  `TopicsConsumer` which consume from several topics under same namespace.
   - support regex based subscription.
   - auto-discover topic addition/deletion.
   
   This issue track the third task:
   auto-discover topic addition/deletion


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1095: Publish pulsar-client-admin as a shaded artifact

[GitBox]

merlimat commented on issue #1095: Publish pulsar-client-admin as a shaded 
artifact
URL: https://github.com/apache/incubator-pulsar/pull/1095#issuecomment-360164248
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack opened a new issue #1100: PIP-13-1: Provide `TopicsConsumer` to consume from several topics under same namespace.

[GitBox]

zhaijack opened a new issue #1100: PIP-13-1: Provide `TopicsConsumer` to 
consume from several topics under same namespace.
URL: https://github.com/apache/incubator-pulsar/issues/1100
 
 
   This is tracking the sub task 1 for [PIP 
13](https://github.com/apache/incubator-pulsar/wiki/PIP-13:-Subscribe-to-topics-represented-by-regular-expressions):
   
   Provide `TopicsConsumer` which consume from several topics under same 
namespace.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat opened a new pull request #1104: Set loadBalancerEnabled=true in ServiceConfiguration

[GitBox]

merlimat opened a new pull request #1104: Set loadBalancerEnabled=true in 
ServiceConfiguration
URL: https://github.com/apache/incubator-pulsar/pull/1104
 
 
   ### Motivation
   
   The intended default is to have the load manager enabled and the same 
setting is set to be the default in `conf/broker.conf`.  
   
   All the config values in the `broker.conf` file must match the 
`ServiceConfiguration` class defaults.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1094: Include BoringSSL native implementation for faster TLS

[GitBox]

merlimat closed pull request #1094: Include BoringSSL native implementation for 
faster TLS
URL: https://github.com/apache/incubator-pulsar/pull/1094
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pom.xml b/pom.xml
index 301c70b75..11797a49c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -108,7 +108,7 @@ flexible messaging model and an intuitive client 
API.
 
 4.3.1.85-yahoo
 3.4.10
-4.1.12.Final
+4.1.20.Final
 1.0.5
 9.3.11.v20160721
 1.7.17
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
index e885f..bf7bdbd3a 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
@@ -123,6 +123,7 @@
 import io.netty.channel.AdaptiveRecvByteBufAllocator;
 import io.netty.channel.ChannelOption;
 import io.netty.channel.EventLoopGroup;
+import io.netty.handler.ssl.SslContext;
 import io.netty.util.concurrent.DefaultThreadFactory;
 
 public class BrokerService implements Closeable, 
ZooKeeperCacheListener {
@@ -289,7 +290,8 @@ public void start() throws Exception {
 ServerBootstrap tlsBootstrap = bootstrap.clone();
 tlsBootstrap.childHandler(new PulsarChannelInitializer(this, 
serviceConfig, true));
 tlsBootstrap.bind(new InetSocketAddress(pulsar.getBindAddress(), 
tlsPort)).sync();
-log.info("Started Pulsar Broker TLS service on port {}", tlsPort);
+log.info("Started Pulsar Broker TLS service on port {} - TLS 
provider: {}", tlsPort,
+SslContext.defaultServerProvider());
 }
 
 // start other housekeeping functions
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
index cddf478b6..1c2b9b682 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
@@ -510,7 +510,7 @@ public void testTlsEnabled() throws Exception {
 consumer.close();
 fail("should fail");
 } catch (Exception e) {
-assertTrue(e.getMessage().contains("General SSLEngine problem"));
+assertTrue(e.getMessage().contains("General OpenSslEngine 
problem"));
 } finally {
 pulsarClient.close();
 }
diff --git a/pulsar-client-shaded/pom.xml b/pulsar-client-shaded/pom.xml
index ec8d96d4e..c3239cca8 100644
--- a/pulsar-client-shaded/pom.xml
+++ b/pulsar-client-shaded/pom.xml
@@ -75,6 +75,8 @@
   com.fasterxml.jackson.core
   io.netty:netty
   io.netty:netty-all
+  io.netty:netty-tcnative-boringssl-static
+  
   org.apache.pulsar:pulsar-common
   org.apache.pulsar:pulsar-checksum
   net.jpountz.lz4:lz4
diff --git a/pulsar-common/pom.xml b/pulsar-common/pom.xml
index 3e7bd0a3b..ccfabc1b0 100644
--- a/pulsar-common/pom.xml
+++ b/pulsar-common/pom.xml
@@ -70,5 +70,11 @@
   pulsar-checksum
   ${project.version}
 
+
+
+  io.netty
+  netty-tcnative-boringssl-static
+  2.0.7.Final
+
   
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Include BoringSSL native implementation for faster TLS (#1094)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new a6e02bf  Include BoringSSL native implementation for faster TLS (#1094)
a6e02bf is described below

commit a6e02bf7444992981a827d2eab550674ba014ed9
Author: Matteo Merli 
AuthorDate: Wed Jan 24 08:53:59 2018 -0800

Include BoringSSL native implementation for faster TLS (#1094)
---
 pom.xml | 2 +-
 .../main/java/org/apache/pulsar/broker/service/BrokerService.java   | 4 +++-
 .../java/org/apache/pulsar/broker/service/BrokerServiceTest.java| 2 +-
 pulsar-client-shaded/pom.xml| 2 ++
 pulsar-common/pom.xml   | 6 ++
 5 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/pom.xml b/pom.xml
index 301c70b..11797a4 100644
--- a/pom.xml
+++ b/pom.xml
@@ -108,7 +108,7 @@ flexible messaging model and an intuitive client 
API.
 
 4.3.1.85-yahoo
 3.4.10
-4.1.12.Final
+4.1.20.Final
 1.0.5
 9.3.11.v20160721
 1.7.17
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
index e885333..bf7bdbd 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
@@ -123,6 +123,7 @@ import io.netty.buffer.PooledByteBufAllocator;
 import io.netty.channel.AdaptiveRecvByteBufAllocator;
 import io.netty.channel.ChannelOption;
 import io.netty.channel.EventLoopGroup;
+import io.netty.handler.ssl.SslContext;
 import io.netty.util.concurrent.DefaultThreadFactory;
 
 public class BrokerService implements Closeable, 
ZooKeeperCacheListener {
@@ -289,7 +290,8 @@ public class BrokerService implements Closeable, 
ZooKeeperCacheListenercom.fasterxml.jackson.core
   io.netty:netty
   io.netty:netty-all
+  io.netty:netty-tcnative-boringssl-static
+  
   org.apache.pulsar:pulsar-common
   org.apache.pulsar:pulsar-checksum
   net.jpountz.lz4:lz4
diff --git a/pulsar-common/pom.xml b/pulsar-common/pom.xml
index 3e7bd0a..ccfabc1 100644
--- a/pulsar-common/pom.xml
+++ b/pulsar-common/pom.xml
@@ -70,5 +70,11 @@
   pulsar-checksum
   ${project.version}
 
+
+
+  io.netty
+  netty-tcnative-boringssl-static
+  2.0.7.Final
+
   
 

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] ivankelly commented on a change in pull request #1044: Compact algo

[GitBox]

ivankelly commented on a change in pull request #1044: Compact algo
URL: https://github.com/apache/incubator-pulsar/pull/1044#discussion_r163491091
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/compaction/TwoPhaseCompactor.java
 ##
 @@ -0,0 +1,275 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.compaction;
+
+import com.google.common.collect.ImmutableMap;
+import io.netty.buffer.ByteBuf;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.Future;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.Semaphore;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.bookkeeper.client.BKException;
+import org.apache.bookkeeper.client.BookKeeper;
+import org.apache.bookkeeper.client.LedgerHandle;
+
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.common.api.Commands;
+import org.apache.pulsar.common.api.proto.PulsarApi.MessageMetadata;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.PulsarClient;
+import org.apache.pulsar.client.api.RawReader;
+import org.apache.pulsar.client.api.RawMessage;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compaction will go through the topic in two passes. The first pass
+ * selects latest offset for each key in the topic. Then the second pass
+ * writes these values to a ledger.
+ *
+ * The two passes are required to avoid holding the payloads of each of
+ * the latest values in memory, as the payload can be many orders of
+ * magnitude larger than a message id.
+*/
+public class TwoPhaseCompactor extends Compactor {
+private static final Logger log = LoggerFactory.getLogger(Compactor.class);
+private static final int MAX_OUTSTANDING = 500;
+private static final String COMPACTED_TOPIC_LEDGER_PROPERTY = 
"CompactedTopicLedger";
+
+public TwoPhaseCompactor(ServiceConfiguration conf,
+ PulsarClient pulsar,
+ BookKeeper bk,
+ ScheduledExecutorService scheduler) {
+super(conf, pulsar, bk, scheduler);
+}
+
+@Override
+protected CompletableFuture doCompaction(RawReader reader, 
BookKeeper bk) {
+return phaseOne(reader).thenCompose(
+(r) -> phaseTwo(reader, r.from, r.to, r.latestForKey, bk));
+}
+
+private CompletableFuture phaseOne(RawReader reader) {
+Map latestForKey = new HashMap<>();
+
+CompletableFuture loopPromise = new 
CompletableFuture<>();
+phaseOneLoop(reader, Optional.empty(), Optional.empty(), latestForKey, 
loopPromise);
+return loopPromise;
+}
+
+private void phaseOneLoop(RawReader reader,
+  Optional firstMessageId,
+  Optional lastMessageId,
+  Map latestForKey,
+  CompletableFuture loopPromise) {
+CompletableFuture future = reader.readNextAsync();
+scheduleTimeout(future);
+future.whenComplete(
+(m, exception) -> {
+try {
+if (exception != null) {
+if (exception instanceof TimeoutException
+&& firstMessageId.isPresent()) {
+loopPromise.complete(new 
PhaseOneResult(firstMessageId.get(),
+
lastMessageId.get(),
+
latestForKey));
+} else {
+loopPromise.completeExceptionally(exception);
+}
+return;
+}
+
+MessageId id = m.getMessageId();
+String key = extractKey(m);
+latestForKey.put(key, id);
+
+  

[GitHub] hrsakai opened a new pull request #1099: Support ztsUrl parameter in athenz client plugin

[GitBox]

hrsakai opened a new pull request #1099: Support ztsUrl parameter in athenz 
client plugin
URL: https://github.com/apache/incubator-pulsar/pull/1099
 
 
   ### Motivation
   Currently, we have to place athenz configuration file for zts url.
   
   ### Modifications
   Enable zts url to be specified by authParams.
   
   ### Result
   clients don't need conf file to specify zts url.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack opened a new pull request #1103: Issue 1100: Provide `TopicsConsumer` to consume from several topics under same namespace

[GitBox]

zhaijack opened a new pull request #1103: Issue 1100: Provide `TopicsConsumer` 
to consume from several topics under same namespace
URL: https://github.com/apache/incubator-pulsar/pull/1103
 
 
   ### Motivation
   This is a first sub-task for pip-13, which would like handle subscription to 
topics under same namespace.
   
   ### Modifications
   - add subscribe methods in `PulsarClient` and `PulsarClientImpl`.
   - add `TopicsConsumerImpl`, `TopicMessageImpl`, `TopicMessageIdImpl`, add 
some test.
   - change parameter in `UnAckedMessageTracker` from `MessageIdImpl` to 
`MessageId`.
   
   ### Result
   old methods behaviour not changed, 
   user could use new method to subscribe to topics


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sschepens commented on issue #1095: Publish pulsar-client-admin as a shaded artifact

[GitBox]

sschepens commented on issue #1095: Publish pulsar-client-admin as a shaded 
artifact
URL: https://github.com/apache/incubator-pulsar/pull/1095#issuecomment-360147143
 
 
   :+1:


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack opened a new issue #1101: PIP-13-2: support regex based subscription

[GitBox]

zhaijack opened a new issue #1101: PIP-13-2: support regex based subscription
URL: https://github.com/apache/incubator-pulsar/issues/1101
 
 
   This is tracking the sub task 2 for [PIP 
13](https://github.com/apache/incubator-pulsar/wiki/PIP-13:-Subscribe-to-topics-represented-by-regular-expressions):
   PIP-13 has 3 sub-task:
   -  `TopicsConsumer` which consume from several topics under same namespace.
   - support regex based subscription.
   - auto-discover topic addition/deletion
   
   This issue track the second task:
   support regex based subscription.
   
   
   support regex based subscription.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services