[GitHub] nkurihar commented on issue #1352: Delete inactive subscriptions automatically

[GitBox]

nkurihar commented on issue #1352: Delete inactive subscriptions automatically
URL: https://github.com/apache/incubator-pulsar/pull/1352#issuecomment-382602294
 
 
   @merlimat 
   Any thoughts?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nkurihar commented on issue #1380: Refactor REST API for namespace policy

[GitBox]

nkurihar commented on issue #1380: Refactor REST API for namespace policy
URL: https://github.com/apache/incubator-pulsar/pull/1380#issuecomment-382601997
 
 
   @merlimat 
   Thanks for comments.
   Let us close this PR for now and re-consider how to refactor.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] nkurihar closed pull request #1380: Refactor REST API for namespace policy

[GitBox]

nkurihar closed pull request #1380: Refactor REST API for namespace policy
URL: https://github.com/apache/incubator-pulsar/pull/1380
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
index c71127996..e982696f2 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
@@ -39,6 +39,8 @@
 import javax.ws.rs.core.Response.Status;
 import javax.ws.rs.core.UriBuilder;
 
+import com.google.gson.Gson;
+import com.google.gson.JsonSyntaxException;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.pulsar.broker.PulsarServerException;
 import org.apache.pulsar.broker.ServiceConfiguration;
@@ -1341,28 +1343,85 @@ protected void 
internalSetMaxConsumersPerSubscription(int maxConsumersPerSubscri
 byte[] content = globalZk().getData(path, null, nodeStat);
 Policies policies = jsonMapper().readValue(content, 
Policies.class);
 if (maxConsumersPerSubscription < 0) {
-throw new RestException(Status.PRECONDITION_FAILED,
-"maxConsumersPerSubscription must be 0 or more");
+throw new RestException(Status.PRECONDITION_FAILED, 
"maxConsumersPerSubscription must be 0 or more");
 }
 policies.max_consumers_per_subscription = 
maxConsumersPerSubscription;
 globalZk().setData(path, jsonMapper().writeValueAsBytes(policies), 
nodeStat.getVersion());
 policiesCache().invalidate(path(POLICIES, 
namespaceName.toString()));
-log.info("[{}] Successfully updated maxConsumersPerSubscription 
configuration: namespace={}, value={}", clientAppId(),
-namespaceName, policies.max_consumers_per_subscription);
+log.info("[{}] Successfully updated maxConsumersPerSubscription 
configuration: namespace={}, value={}",
+clientAppId(), namespaceName, 
policies.max_consumers_per_subscription);
 
 } catch (KeeperException.NoNodeException e) {
-log.warn("[{}] Failed to update maxConsumersPerSubscription 
configuration for namespace {}: does not exist", clientAppId(),
-namespaceName);
+log.warn("[{}] Failed to update maxConsumersPerSubscription 
configuration for namespace {}: does not exist",
+clientAppId(), namespaceName);
 throw new RestException(Status.NOT_FOUND, "Namespace does not 
exist");
 } catch (KeeperException.BadVersionException e) {
-log.warn("[{}] Failed to update maxConsumersPerSubscription 
configuration for namespace {}: concurrent modification",
+log.warn(
+"[{}] Failed to update maxConsumersPerSubscription 
configuration for namespace {}: concurrent modification",
 clientAppId(), namespaceName);
 throw new RestException(Status.CONFLICT, "Concurrent 
modification");
 } catch (RestException pfe) {
 throw pfe;
 } catch (Exception e) {
-log.error("[{}] Failed to update maxConsumersPerSubscription 
configuration for namespace {}", clientAppId(), namespaceName,
-e);
+log.error("[{}] Failed to update maxConsumersPerSubscription 
configuration for namespace {}", clientAppId(),
+namespaceName, e);
+throw new RestException(e);
+}
+}
+
+protected void internalSetPolicy(Policies.PolicyProperty policyProperty, 
String jsonPolicyValue) {
+if (policyProperty.isOnlySuperUser()) {
+validateSuperUserAccess();
+}
+validatePoliciesReadOnlyAccess();
+
+try {
+Stat nodeStat = new Stat();
+final String path = path(POLICIES, namespaceName.toString());
+byte[] content = globalZk().getData(path, null, nodeStat);
+Policies policies = jsonMapper().readValue(content, 
Policies.class);
+
+Gson gson = new Gson();
+Object policyValue = gson.fromJson(jsonPolicyValue, 
policyProperty.getType());
+
+// When policyValue is Map instance, update by Map.put
+if (policyValue instanceof Map) {
+Map resources = (Map) 
Policies.class.getDeclaredField(policyProperty
+.getPropertyName()).get(policies);
+for (Map.Entry entry: ((Map) 
policyValue).entrySet()) {
+

[GitHub] zubchenok commented on issue #1616: PulsarAdmin has a significantly slower performance for creating subscriptions

[GitBox]

zubchenok commented on issue #1616: PulsarAdmin has a significantly slower 
performance for creating subscriptions
URL: 
https://github.com/apache/incubator-pulsar/issues/1616#issuecomment-382555310
 
 
   Anyway creating 1000 subscriptions during even in ~7 seconds is still a slow 
performance. It should be much faster.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zubchenok commented on issue #1616: PulsarAdmin has a significantly slower performance for creating subscriptions

[GitBox]

zubchenok commented on issue #1616: PulsarAdmin has a significantly slower 
performance for creating subscriptions
URL: 
https://github.com/apache/incubator-pulsar/issues/1616#issuecomment-382554969
 
 
   Anyway creating 1000 subscriptions during ~7 seconds is still a slow 
performance.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zubchenok commented on issue #1616: PulsarAdmin has a significantly slower performance for creating subscriptions

[GitBox]

zubchenok commented on issue #1616: PulsarAdmin has a significantly slower 
performance for creating subscriptions
URL: 
https://github.com/apache/incubator-pulsar/issues/1616#issuecomment-382554969
 
 
   Anyway creating 1000 subscriptions during ~7 seconds is still a slow 
performance.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zubchenok commented on issue #1616: PulsarAdmin has a significantly slower performance for creating subscriptions

[GitBox]

zubchenok commented on issue #1616: PulsarAdmin has a significantly slower 
performance for creating subscriptions
URL: 
https://github.com/apache/incubator-pulsar/issues/1616#issuecomment-382554740
 
 
   @sijie commented:
   > I think the different is admin.createSubscription is using http protocol, 
which I think each createSubscription is creating a new connection and close 
it. (If I remember correctly, http/1.0 doesn’t support multiplexing). while 
create consumer/close is binary protocol over tcp connection, so connection is 
multiplexed for different commands.
   
   I would think some features like creating a subscription should be moved to 
PulsarClient for faster performance with binary protobuf protocol.
   
   At the same time @merlimat noted:
   > The multiple requests should be using HTTP/1.1 with Keep-Alive


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zubchenok opened a new issue #1616: PulsarAdmin has a significantly slower performance for creating subscriptions

[GitBox]

zubchenok opened a new issue #1616: PulsarAdmin has a significantly slower 
performance for creating subscriptions
URL: https://github.com/apache/incubator-pulsar/issues/1616
 
 
    Expected behavior
   
   PulsarAdmin should have faster performance for subscription creation than 
creating and closing a Consumer.
   
    Actual behavior
   
   When I create subscription with PulsarAdmin, it is significantly slower than 
if I create a Consumer.
   
    Steps to reproduce
   
   Note: At least until 1.22.0 (including) you can run this performance 
comparison test only against single-node cluster due to bug #1615 
   
   Run the following unit test to check the performance:
   ```
   @Test
   public void subscriptionCreatePerformanceComparisonTest() throws 
Exception
   {
   String host = "vv";
   String cluster = "vv";
   
   String webServiceUrl = "http://; + host + ":8080";
   String brokerServiceUrl = "pulsar://" + host + ":6650";
   String property = "test-prop-performance";
   String adminRole = "test-admins";
   String namespace = "test-namespace-performance";
   int numberOfSubscriptions = 1000;
   String subscription = "test-subscription";
   ClientConfiguration pulsarConfig = new ClientConfiguration();
   
   PulsarAdmin pulsarAdmin = new PulsarAdmin(new URL(webServiceUrl), 
pulsarConfig);
   // create property if not exists
   try
   {
   PropertyAdmin propertyAdmin = new PropertyAdmin();
   
propertyAdmin.setAdminRoles(Collections.singletonList(adminRole));
   propertyAdmin.setAllowedClusters(ImmutableSet.of(cluster));
   pulsarAdmin.properties().createProperty(property, propertyAdmin);
   }
   catch (PulsarAdminException.ConflictException ignored)
   {
   }
   
   String namespaceName = property + "/" + cluster + "/" + namespace;
   // create namespace if not exists
   try
   {
   pulsarAdmin.namespaces().createNamespace(namespaceName);
   }
   catch (PulsarAdminException.ConflictException ignored)
   {
   }
   
   String result1;
   {
   long start = System.currentTimeMillis();
   CompletableFuture
   .allOf(IntStream
   .range(0, numberOfSubscriptions)
   .mapToObj(i -> {
   
   String topic = UUID.randomUUID().toString();
   String topicName = "persistent://" + 
namespaceName + "/" + topic;
   return 
pulsarAdmin.persistentTopics().createSubscriptionAsync(topicName, subscription, 
MessageId.earliest);
   })
   .toArray(CompletableFuture[]::new))
   .join();
   long finish = System.currentTimeMillis();
   result1 = "createSubscriptionAsync time: " + (finish - start) / 
1000d;
   System.out.println(result1);
   }
   pulsarAdmin.close();
   
   PulsarClient pulsarClient = PulsarClient.create(brokerServiceUrl, 
pulsarConfig);
   
   String result2;
   {
   long start = System.currentTimeMillis();
   CompletableFuture
   .allOf(IntStream
   .range(0, numberOfSubscriptions)
   .mapToObj(i -> {
   
   String topic = UUID.randomUUID().toString();
   String topicName = "persistent://" + 
namespaceName + "/" + topic;
   return pulsarClient
   .subscribeAsync(topicName, 
subscription)
   
.thenComposeAsync(Consumer::closeAsync);
   })
   .toArray(CompletableFuture[]::new))
   .join();
   long finish = System.currentTimeMillis();
   result2 = "subscribeAsync+closeAsync time: " + (finish - start) 
/ 1000d;
   }
   pulsarClient.close();
   
   System.out.println(result1);
   System.out.println(result2);
   }
   ```
   Actual output (FYI my ping to the node in test is ~50ms):
   ```
   [pulsar client logs skipped]
   createSubscriptionAsync time: 20.565
   subscribeAsync+closeAsync time: 7.113
   ```
   
    System configuration
   **Pulsar version**: 1.22.0
   Single node/standalone.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please 

[GitHub] zubchenok commented on issue #1615: PulsarAdmin.createSubscription method does not work against a multi-node Pulsar cluster

[GitBox]

zubchenok commented on issue #1615: PulsarAdmin.createSubscription method does 
not work against a multi-node Pulsar cluster
URL: 
https://github.com/apache/incubator-pulsar/issues/1615#issuecomment-382552461
 
 
   Similar issue happens if you create a subscription via bin/pulsar-admin or 
HTTP REST PUT method.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zubchenok opened a new issue #1615: PulsarAdmin.createSubscription method does not work against a multi-node Pulsar cluster

[GitBox]

zubchenok opened a new issue #1615: PulsarAdmin.createSubscription method does 
not work against a multi-node Pulsar cluster
URL: https://github.com/apache/incubator-pulsar/issues/1615
 
 
    Expected behavior
   
   Subscription create for a topic with PulsarAdmin.createSubscription method 
does not work against a multi-node Pulsar cluster.
   
    Actual behavior
   
   PulsarAdmin reports a `HTTP 500 Internal Server Error` if the node to which 
PulsarAdmin is connected is not the Master for the topic.
   
   Broker has the following exception in logs:
   ```
   Apr 18 17:34:30 localhost broker[10753]: 2018-04-18 17:34:30,816 - INFO  - 
[pulsar-web-61-30:PersistentTopics@1042] - 
[null][bfbb44c8-493f-4772-8195-bffa6a43260e] Creating subscription sender at 
message id 9223372036854775807:9223372036854775807:-1
   Apr 18 17:34:30 localhost broker[10753]: 2018-04-18 17:34:30,819 - WARN  - 
[pulsar-web-61-30:PersistentTopics@1077] - [null] 
[persistent://test-prop-2/prod-pulsar-cluster-1/test-namespace-6/bfbb44c8-493f-4772-8195-bffa6a43260e]
 Failed to create subscription sender at message id 
9223372036854775807:9223372036854775807:-1
   Apr 18 17:34:30 localhost broker[10753]: 
javax.ws.rs.WebApplicationException: HTTP 307 Temporary Redirect
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.apache.pulsar.broker.web.PulsarWebResource.validateDestinationOwnership(PulsarWebResource.java:503)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.apache.pulsar.broker.admin.PersistentTopics.validateAdminOperationOnDestination(PersistentTopics.java:286)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.apache.pulsar.broker.admin.PersistentTopics.createSubscription(PersistentTopics.java:1061)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
java.lang.reflect.Method.invoke(Method.java:498)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:143)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
   Apr 18 17:34:30 localhost broker[10753]: #011at 
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
   Apr 18 17:34:30 localhost 

[GitHub] merlimat closed pull request #1612: Fix pulsar broker handling WebApplicationException

[GitBox]

merlimat closed pull request #1612: Fix pulsar broker handling 
WebApplicationException
URL: https://github.com/apache/incubator-pulsar/pull/1612
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java
index 968ab6125..06b1e97b9 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java
@@ -25,6 +25,7 @@
 import javax.ws.rs.core.MediaType;
 import javax.ws.rs.core.Response;
 
+import javax.ws.rs.core.Response.Status;
 import org.apache.pulsar.client.admin.PulsarAdminException;
 import org.apache.pulsar.common.policies.data.ErrorData;
 
@@ -62,12 +63,17 @@ public RestException(PulsarAdminException cae) {
 }
 
 private static Response getResponse(Throwable t) {
-if (t instanceof RestException) {
-RestException e = (RestException) t;
+if (t instanceof RestException
+|| t instanceof WebApplicationException) {
+WebApplicationException e = (WebApplicationException) t;
 return 
Response.status(e.getResponse().getStatus()).entity(e.getResponse().getEntity())
-.type(e.getResponse().getMediaType()).build();
+.type(e.getResponse().getMediaType()).build();
 } else {
-return 
Response.status(500).entity(getExceptionData(t)).type(MediaType.TEXT_PLAIN).build();
+return Response
+.status(Status.INTERNAL_SERVER_ERROR)
+.entity(getExceptionData(t))
+.type(MediaType.TEXT_PLAIN)
+.build();
 }
 }
 }
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/web/RestExceptionTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/web/RestExceptionTest.java
new file mode 100644
index 0..cf9e95268
--- /dev/null
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/web/RestExceptionTest.java
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.web;
+
+import static org.testng.Assert.assertEquals;
+
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.Response.Status;
+import org.testng.annotations.Test;
+
+/**
+ * Unit test for pulsar functions.
+ */
+public class RestExceptionTest {
+
+@Test
+public void testRestException() {
+RestException re = new RestException(Status.TEMPORARY_REDIRECT, "test 
rest exception");
+RestException testException = new RestException(re);
+
+assertEquals(Status.TEMPORARY_REDIRECT.getStatusCode(), 
testException.getResponse().getStatus());
+assertEquals(re.getResponse().getEntity(), 
testException.getResponse().getEntity());
+}
+
+@Test
+public void testWebApplicationException() {
+WebApplicationException wae = new WebApplicationException("test web 
application exception", Status.TEMPORARY_REDIRECT);
+RestException testException = new RestException(wae);
+
+assertEquals(Status.TEMPORARY_REDIRECT.getStatusCode(), 
testException.getResponse().getStatus());
+assertEquals(wae.getResponse().getEntity(), 
testException.getResponse().getEntity());
+}
+
+@Test
+public void testOtherException() {
+Exception otherException = new Exception("test other exception");
+RestException testException = new RestException(otherException);
+
+assertEquals(Status.INTERNAL_SERVER_ERROR.getStatusCode(), 
testException.getResponse().getStatus());
+assertEquals(RestException.getExceptionData(otherException), 
testException.getResponse().getEntity());
+}
+
+}


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and 

[incubator-pulsar] branch master updated: Fix pulsar broker handling WebApplicationException (#1612)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new f8c2d25  Fix pulsar broker handling WebApplicationException (#1612)
f8c2d25 is described below

commit f8c2d2580dab7a99fa508debc22b8fedf231f7ca
Author: Sijie Guo 
AuthorDate: Wed Apr 18 15:32:37 2018 -0700

Fix pulsar broker handling WebApplicationException (#1612)

*Problem*

Currently all WebApplicationExeptions are converted to 500 / 
InternalServerError by broker before responding to rest requests.

*Solution*

Make sure RestException handles WebApplicationExeption
---
 .../apache/pulsar/broker/web/RestException.java| 14 +++--
 .../pulsar/broker/web/RestExceptionTest.java   | 59 ++
 2 files changed, 69 insertions(+), 4 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java
index 968ab61..06b1e97 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/web/RestException.java
@@ -25,6 +25,7 @@ import javax.ws.rs.WebApplicationException;
 import javax.ws.rs.core.MediaType;
 import javax.ws.rs.core.Response;
 
+import javax.ws.rs.core.Response.Status;
 import org.apache.pulsar.client.admin.PulsarAdminException;
 import org.apache.pulsar.common.policies.data.ErrorData;
 
@@ -62,12 +63,17 @@ public class RestException extends WebApplicationException {
 }
 
 private static Response getResponse(Throwable t) {
-if (t instanceof RestException) {
-RestException e = (RestException) t;
+if (t instanceof RestException
+|| t instanceof WebApplicationException) {
+WebApplicationException e = (WebApplicationException) t;
 return 
Response.status(e.getResponse().getStatus()).entity(e.getResponse().getEntity())
-.type(e.getResponse().getMediaType()).build();
+.type(e.getResponse().getMediaType()).build();
 } else {
-return 
Response.status(500).entity(getExceptionData(t)).type(MediaType.TEXT_PLAIN).build();
+return Response
+.status(Status.INTERNAL_SERVER_ERROR)
+.entity(getExceptionData(t))
+.type(MediaType.TEXT_PLAIN)
+.build();
 }
 }
 }
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/web/RestExceptionTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/web/RestExceptionTest.java
new file mode 100644
index 000..cf9e952
--- /dev/null
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/web/RestExceptionTest.java
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.web;
+
+import static org.testng.Assert.assertEquals;
+
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.Response.Status;
+import org.testng.annotations.Test;
+
+/**
+ * Unit test for pulsar functions.
+ */
+public class RestExceptionTest {
+
+@Test
+public void testRestException() {
+RestException re = new RestException(Status.TEMPORARY_REDIRECT, "test 
rest exception");
+RestException testException = new RestException(re);
+
+assertEquals(Status.TEMPORARY_REDIRECT.getStatusCode(), 
testException.getResponse().getStatus());
+assertEquals(re.getResponse().getEntity(), 
testException.getResponse().getEntity());
+}
+
+@Test
+public void testWebApplicationException() {
+WebApplicationException wae = new WebApplicationException("test web 
application exception", Status.TEMPORARY_REDIRECT);
+RestException testException = new RestException(wae);
+
+assertEquals(Status.TEMPORARY_REDIRECT.getStatusCode(), 
testException.getResponse().getStatus());
+assertEquals(wae.getResponse().getEntity(), 
testException.getResponse().getEntity());
+}
+
+  

[GitHub] merlimat commented on issue #1607: Fixed initial default namespace setup with current cluster setup

[GitBox]

merlimat commented on issue #1607: Fixed initial default namespace setup with 
current cluster setup
URL: https://github.com/apache/incubator-pulsar/pull/1607#issuecomment-382545082
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat opened a new issue #1614: Itermittent test timeout on PersistentFailoverE2ETest.testSimpleConsumerEventsWithoutPartition

[GitBox]

merlimat opened a new issue #1614: Itermittent test timeout on 
PersistentFailoverE2ETest.testSimpleConsumerEventsWithoutPartition 
URL: https://github.com/apache/incubator-pulsar/issues/1614
 
 
    Expected behavior
   The time is timing out several times. eg: 
   
https://builds.apache.org/job/pulsar_precommit_java8/724/testReport/junit/org.apache.pulsar.broker.service/PersistentFailoverE2ETest/testSimpleConsumerEventsWithoutPartition/
 
   
   Test thread is blocked on: 
   
   
https://github.com/apache/incubator-pulsar/blob/8fbb926089effcadf59e199fe69ec65b4bbe4eae/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/PersistentFailoverE2ETest.java#L217
   
   In particular is waiting on the ` listener.activeQueue.take().intValue())`
   
   
https://github.com/apache/incubator-pulsar/blob/8fbb926089effcadf59e199fe69ec65b4bbe4eae/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/PersistentFailoverE2ETest.java#L102-L105


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1602: Proxy should close connections immediately after exception caught

[GitBox]

merlimat closed pull request #1602: Proxy should close connections immediately 
after exception caught
URL: https://github.com/apache/incubator-pulsar/pull/1602
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConnection.java
 
b/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConnection.java
index d2c7f9709..eaa006d88 100644
--- 
a/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConnection.java
+++ 
b/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConnection.java
@@ -28,6 +28,7 @@
 
 import org.apache.pulsar.broker.authentication.AuthenticationDataCommand;
 import org.apache.pulsar.broker.authentication.AuthenticationDataSource;
+import org.apache.pulsar.client.impl.ClientCnx;
 import org.apache.pulsar.common.api.Commands;
 import org.apache.pulsar.common.api.PulsarHandler;
 import org.apache.pulsar.common.api.proto.PulsarApi;
@@ -88,7 +89,7 @@
 static final Counter rejectedConnections = Counter
 .build("pulsar_proxy_rejected_connections", "Counter for 
connections rejected due to throttling").create()
 .register();
-
+
 public ProxyConnection(ProxyService proxyService) {
 super(30, TimeUnit.SECONDS);
 this.service = proxyService;
@@ -111,7 +112,7 @@ public void channelUnregistered(ChannelHandlerContext ctx) 
throws Exception {
 super.channelUnregistered(ctx);
 activeConnections.dec();
 }
-
+
 @Override
 public void channelActive(ChannelHandlerContext ctx) throws Exception {
 super.channelActive(ctx);
@@ -126,6 +127,16 @@ public void channelInactive(ChannelHandlerContext ctx) 
throws Exception {
 if (directProxyHandler != null && directProxyHandler.outboundChannel 
!= null) {
 directProxyHandler.outboundChannel.close();
 }
+
+LOG.info("[{}] Connection closed", remoteAddress);
+}
+
+@Override
+public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) 
throws Exception {
+super.exceptionCaught(ctx, cause);
+LOG.warn("[{}] Got exception {} : {}", remoteAddress, 
cause.getClass().getSimpleName(), cause.getMessage(),
+ClientCnx.isKnownException(cause) ? null : cause);
+ctx.close();
 }
 
 @Override


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie closed pull request #1606: Enhance Pulsar Connect api by attaching more information about the message

[GitBox]

sijie closed pull request #1606: Enhance Pulsar Connect api by attaching more 
information about the message
URL: https://github.com/apache/incubator-pulsar/pull/1606
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
 
b/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
index c6a34baf2..73faf053a 100644
--- 
a/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
+++ 
b/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
@@ -32,6 +32,7 @@
 import com.aerospike.client.policy.ClientPolicy;
 import com.aerospike.client.policy.WritePolicy;
 import org.apache.pulsar.common.util.KeyValue;
+import org.apache.pulsar.connect.core.Message;
 import org.apache.pulsar.connect.core.Sink;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -77,10 +78,10 @@ public void close() throws Exception {
 }
 
 @Override
-public CompletableFuture write(KeyValue tuple) {
+public CompletableFuture write(Message> tuple) {
 CompletableFuture future = new CompletableFuture<>();
-Key key = new Key(aerospikeSinkConfig.getKeyspace(), 
aerospikeSinkConfig.getKeySet(), tuple.getKey().toString());
-Bin bin = new Bin(aerospikeSinkConfig.getColumnName(), 
Value.getAsBlob(tuple.getValue()));
+Key key = new Key(aerospikeSinkConfig.getKeyspace(), 
aerospikeSinkConfig.getKeySet(), tuple.getData().getKey().toString());
+Bin bin = new Bin(aerospikeSinkConfig.getColumnName(), 
Value.getAsBlob(tuple.getData().getValue()));
 AWriteListener listener = null;
 try {
 listener = queue.take();
diff --git 
a/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
 
b/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
index 1ed43a8a0..a003d8f2f 100644
--- 
a/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
+++ 
b/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
@@ -23,6 +23,7 @@
 import com.google.common.util.concurrent.FutureCallback;
 import com.google.common.util.concurrent.Futures;
 import org.apache.pulsar.common.util.KeyValue;
+import org.apache.pulsar.connect.core.Message;
 import org.apache.pulsar.connect.core.Sink;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -66,8 +67,8 @@ public void close() throws Exception {
 }
 
 @Override
-public CompletableFuture write(KeyValue tuple) {
-BoundStatement bound = statement.bind(tuple.getKey(), 
tuple.getValue());
+public CompletableFuture write(Message> tuple) {
+BoundStatement bound = statement.bind(tuple.getData().getKey(), 
tuple.getData().getValue());
 ResultSetFuture future = session.executeAsync(bound);
 CompletableFuture completable = new CompletableFuture();
 Futures.addCallback(future,
diff --git 
a/pulsar-connect/core/src/main/java/org/apache/pulsar/connect/core/Message.java 
b/pulsar-connect/core/src/main/java/org/apache/pulsar/connect/core/Message.java
new file mode 100644
index 0..26543435a
--- /dev/null
+++ 
b/pulsar-connect/core/src/main/java/org/apache/pulsar/connect/core/Message.java
@@ -0,0 +1,43 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.connect.core;
+
+/**
+ * Pulsar Connect's Message interface. Message encapsulates the
+ * information about a message being read/written from/to a Source/Sink.
+ */
+public interface Message {
+/**
+ * Retrieves the partition information if any of the message
+ * @return The partition id where the
+ */
+default String getPartitionId() { return null; }
+
+/**
+ * Retrieves the sequence id of the message

[incubator-pulsar] branch master updated: Enhance Pulsar Connect api by attaching more information about the message (#1606)

[sijie]

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new e6d1551  Enhance Pulsar Connect api by attaching more information 
about the message (#1606)
e6d1551 is described below

commit e6d1551d08188bd634f02f4c3704967342c41b18
Author: Sanjeev Kulkarni 
AuthorDate: Wed Apr 18 14:36:32 2018 -0700

Enhance Pulsar Connect api by attaching more information about the message 
(#1606)

* Enhance Pulsar Connect api by attaching more information about the message

* Fix RabbitMQ to fit into the new interface

* Made Message an interface

* Added default implementations for the Message interface
---
 .../pulsar/connect/aerospike/AerospikeSink.java|  7 ++--
 .../pulsar/connect/cassandra/CassandraSink.java|  5 ++-
 .../org/apache/pulsar/connect/core/Message.java| 43 ++
 .../org/apache/pulsar/connect/core/PushSource.java |  2 +-
 .../java/org/apache/pulsar/connect/core/Sink.java  |  2 +-
 .../org/apache/pulsar/connect/kafka/KafkaSink.java |  5 ++-
 .../apache/pulsar/connect/kafka/KafkaSource.java   | 30 +--
 .../pulsar/connect/rabbitmq/RabbitMQSource.java| 24 +---
 .../pulsar/connect/twitter/TwitterFireHose.java| 20 --
 9 files changed, 118 insertions(+), 20 deletions(-)

diff --git 
a/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
 
b/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
index c6a34ba..73faf05 100644
--- 
a/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
+++ 
b/pulsar-connect/aerospike/src/main/java/org/apache/pulsar/connect/aerospike/AerospikeSink.java
@@ -32,6 +32,7 @@ import com.aerospike.client.listener.WriteListener;
 import com.aerospike.client.policy.ClientPolicy;
 import com.aerospike.client.policy.WritePolicy;
 import org.apache.pulsar.common.util.KeyValue;
+import org.apache.pulsar.connect.core.Message;
 import org.apache.pulsar.connect.core.Sink;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -77,10 +78,10 @@ public class AerospikeSink implements 
Sink> {
 }
 
 @Override
-public CompletableFuture write(KeyValue tuple) {
+public CompletableFuture write(Message> tuple) {
 CompletableFuture future = new CompletableFuture<>();
-Key key = new Key(aerospikeSinkConfig.getKeyspace(), 
aerospikeSinkConfig.getKeySet(), tuple.getKey().toString());
-Bin bin = new Bin(aerospikeSinkConfig.getColumnName(), 
Value.getAsBlob(tuple.getValue()));
+Key key = new Key(aerospikeSinkConfig.getKeyspace(), 
aerospikeSinkConfig.getKeySet(), tuple.getData().getKey().toString());
+Bin bin = new Bin(aerospikeSinkConfig.getColumnName(), 
Value.getAsBlob(tuple.getData().getValue()));
 AWriteListener listener = null;
 try {
 listener = queue.take();
diff --git 
a/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
 
b/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
index 1ed43a8..a003d8f 100644
--- 
a/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
+++ 
b/pulsar-connect/cassandra/src/main/java/org/apache/pulsar/connect/cassandra/CassandraSink.java
@@ -23,6 +23,7 @@ import com.datastax.driver.core.*;
 import com.google.common.util.concurrent.FutureCallback;
 import com.google.common.util.concurrent.Futures;
 import org.apache.pulsar.common.util.KeyValue;
+import org.apache.pulsar.connect.core.Message;
 import org.apache.pulsar.connect.core.Sink;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -66,8 +67,8 @@ public class CassandraSink implements Sink> {
 }
 
 @Override
-public CompletableFuture write(KeyValue tuple) {
-BoundStatement bound = statement.bind(tuple.getKey(), 
tuple.getValue());
+public CompletableFuture write(Message> tuple) {
+BoundStatement bound = statement.bind(tuple.getData().getKey(), 
tuple.getData().getValue());
 ResultSetFuture future = session.executeAsync(bound);
 CompletableFuture completable = new CompletableFuture();
 Futures.addCallback(future,
diff --git 
a/pulsar-connect/core/src/main/java/org/apache/pulsar/connect/core/Message.java 
b/pulsar-connect/core/src/main/java/org/apache/pulsar/connect/core/Message.java
new file mode 100644
index 000..2654343
--- /dev/null
+++ 
b/pulsar-connect/core/src/main/java/org/apache/pulsar/connect/core/Message.java
@@ -0,0 +1,43 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor 

[GitHub] sijie opened a new pull request #1613: Add a integration test for create subscription CLI command

[GitBox]

sijie opened a new pull request #1613: Add a integration test for create 
subscription CLI command
URL: https://github.com/apache/incubator-pulsar/pull/1613
 
 
   ### Motivation
   
   due to #1612 , create subscription will fail with `500` when connecting to a 
broker that doesn't own the topic. we need to have an integration test to 
ensure CLI commands are run correctly.
   
   ### Modifications
   
   add an integration test for `createSubscription`
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: On tenants cli tool, default tenant to all clusters if not specified (#1584)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 1334005  On tenants cli tool, default tenant to all clusters if not 
specified (#1584)
1334005 is described below

commit 1334005eda24564971343c42d0df3102fbe40039
Author: Matteo Merli 
AuthorDate: Wed Apr 18 14:32:13 2018 -0700

On tenants cli tool, default tenant to all clusters if not specified (#1584)

* On tenants cli tool, default tenant to all clusters if not specified

* Fixed handling of tenant info args in cli
---
 .../org/apache/pulsar/admin/cli/CmdTenants.java| 46 --
 .../pulsar/common/policies/data/TenantInfo.java| 18 ++---
 2 files changed, 38 insertions(+), 26 deletions(-)

diff --git 
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java 
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java
index 98da636..e1cf4bb 100644
--- 
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java
+++ 
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java
@@ -18,14 +18,17 @@
  */
 package org.apache.pulsar.admin.cli;
 
-import org.apache.pulsar.client.admin.PulsarAdmin;
-import org.apache.pulsar.client.admin.PulsarAdminException;
-import org.apache.pulsar.common.policies.data.TenantInfo;
-
 import com.beust.jcommander.Parameter;
 import com.beust.jcommander.Parameters;
 import com.beust.jcommander.converters.CommaParameterSplitter;
-import com.google.common.collect.Sets;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+
+import org.apache.pulsar.client.admin.PulsarAdmin;
+import org.apache.pulsar.client.admin.PulsarAdminException;
+import org.apache.pulsar.common.policies.data.TenantInfo;
 
 @Parameters(commandDescription = "Operations about tenants")
 public class CmdTenants extends CmdBase {
@@ -55,38 +58,57 @@ public class CmdTenants extends CmdBase {
 private java.util.List params;
 
 @Parameter(names = { "--admin-roles",
-"-r" }, description = "Comma separated Admin roles", required 
= true, splitter = CommaParameterSplitter.class)
+"-r" }, description = "Comma separated list of auth principal 
allowed to administrate the tenant", required = false, splitter = 
CommaParameterSplitter.class)
 private java.util.List adminRoles;
 
 @Parameter(names = { "--allowed-clusters",
-"-c" }, description = "Comma separated allowed clusters", 
required = true, splitter = CommaParameterSplitter.class)
+"-c" }, description = "Comma separated allowed clusters. If 
empty, the tenant will have access to all clusters", required = false, splitter 
= CommaParameterSplitter.class)
 private java.util.List allowedClusters;
 
 @Override
 void run() throws PulsarAdminException {
 String tenant = getOneArgument(params);
-TenantInfo tenantInfo = new 
TenantInfo(Sets.newHashSet(adminRoles), Sets.newHashSet(allowedClusters));
+
+if (adminRoles == null) {
+adminRoles = Collections.emptyList();
+}
+
+if (allowedClusters == null || allowedClusters.isEmpty()) {
+// Default to all available cluster
+allowedClusters = admin.clusters().getClusters();
+}
+
+TenantInfo tenantInfo = new TenantInfo(new HashSet<>(adminRoles), 
new HashSet<>(allowedClusters));
 admin.tenants().createTenant(tenant, tenantInfo);
 }
 }
 
-@Parameters(commandDescription = "Updates a tenant")
+@Parameters(commandDescription = "Updates the configuration for a tenant")
 private class Update extends CliCommand {
 @Parameter(description = "tenant-name", required = true)
 private java.util.List params;
 
 @Parameter(names = { "--admin-roles",
-"-r" }, description = "Comma separated Admin roles", required 
= true, splitter = CommaParameterSplitter.class)
+"-r" }, description = "Comma separated list of auth principal 
allowed to administrate the tenant. If empty the current set of roles won't be 
modified", required = false, splitter = CommaParameterSplitter.class)
 private java.util.List adminRoles;
 
 @Parameter(names = { "--allowed-clusters",
-"-c" }, description = "Comma separated allowed clusters", 
required = true, splitter = CommaParameterSplitter.class)
+"-c" }, description = "Comma separated allowed clusters. If 
omitted, the current set of clusters will be preserved", required = false, 
splitter = CommaParameterSplitter.class)
 private java.util.List allowedClusters;
 
 @Override
 void run() 

[GitHub] merlimat closed pull request #1579: Added option to create namespace on CLI passing list of clusters

[GitBox]

merlimat closed pull request #1579: Added option to create namespace on CLI 
passing list of clusters
URL: https://github.com/apache/incubator-pulsar/pull/1579
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java
index 7f28e0f71..15ac5322a 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java
@@ -644,14 +644,17 @@ public void 
removeNamespaceAntiAffinityGroup(@PathParam("tenant") String tenant,
 }
 
 private Policies getDefaultPolicesIfNull(Policies policies) {
-if (policies != null) {
-return policies;
+if (policies == null) {
+policies = new Policies();
 }
 
-Policies defaultPolicies = new Policies();
 int defaultNumberOfBundles = 
config().getDefaultNumberOfNamespaceBundles();
-defaultPolicies.bundles = getBundles(defaultNumberOfBundles);
-return defaultPolicies;
+
+if (policies.bundles == null) {
+policies.bundles = getBundles(defaultNumberOfBundles);
+}
+
+return policies;
 }
 
 private static final Logger log = 
LoggerFactory.getLogger(Namespaces.class);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
index 7c2469255..0afe36d3a 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
@@ -29,6 +29,13 @@
 import static org.testng.Assert.assertTrue;
 import static org.testng.Assert.fail;
 
+import com.google.common.collect.BoundType;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Range;
+import com.google.common.collect.Sets;
+import com.google.common.hash.Hashing;
+
 import java.lang.reflect.Field;
 import java.net.URL;
 import java.util.ArrayList;
@@ -47,6 +54,8 @@
 import javax.ws.rs.client.InvocationCallback;
 import javax.ws.rs.client.WebTarget;
 
+import lombok.extern.slf4j.Slf4j;
+
 import org.apache.pulsar.broker.PulsarServerException;
 import org.apache.pulsar.broker.PulsarService;
 import org.apache.pulsar.broker.ServiceConfiguration;
@@ -94,8 +103,8 @@
 import org.apache.pulsar.common.policies.data.PersistentTopicInternalStats;
 import org.apache.pulsar.common.policies.data.PersistentTopicStats;
 import org.apache.pulsar.common.policies.data.Policies;
-import org.apache.pulsar.common.policies.data.TenantInfo;
 import org.apache.pulsar.common.policies.data.RetentionPolicies;
+import org.apache.pulsar.common.policies.data.TenantInfo;
 import org.apache.pulsar.common.util.Codec;
 import org.apache.pulsar.common.util.ObjectMapperFactory;
 import org.apache.pulsar.compaction.Compactor;
@@ -107,15 +116,6 @@
 import org.testng.annotations.DataProvider;
 import org.testng.annotations.Test;
 
-import com.google.common.collect.BoundType;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
-import com.google.common.collect.Range;
-import com.google.common.collect.Sets;
-import com.google.common.hash.Hashing;
-
-import lombok.extern.slf4j.Slf4j;
-
 @Slf4j
 public class AdminApiTest extends MockedPulsarServiceBaseTest {
 
@@ -624,6 +624,7 @@ public void namespaces() throws PulsarAdminException, 
PulsarServerException, Exc
 
 Policies policies = new Policies();
 policies.replication_clusters = Sets.newHashSet("test");
+policies.bundles = Policies.defaultBundle();
 policies.auth_policies.namespace_auth.put("my-role", 
EnumSet.allOf(AuthAction.class));
 
 assertEquals(admin.namespaces().getPolicies("prop-xyz/ns1"), policies);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
index 689503629..c7a26ffa3 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
@@ -624,6 +624,7 @@ public void namespaces() throws PulsarAdminException, 
PulsarServerException, Exc
 admin.namespaces().grantPermissionOnNamespace("prop-xyz/use/ns1", 
"my-role", EnumSet.allOf(AuthAction.class));
 
 Policies policies = new Policies();
+policies.bundles = Policies.defaultBundle();
 policies.auth_policies.namespace_auth.put("my-role", 

[GitHub] merlimat closed pull request #1584: On tenants cli tool, default tenant to all clusters if not specified

[GitBox]

merlimat closed pull request #1584: On tenants cli tool, default tenant to all 
clusters if not specified
URL: https://github.com/apache/incubator-pulsar/pull/1584
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java 
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java
index 98da636b7..e1cf4bb85 100644
--- 
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java
+++ 
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTenants.java
@@ -18,14 +18,17 @@
  */
 package org.apache.pulsar.admin.cli;
 
-import org.apache.pulsar.client.admin.PulsarAdmin;
-import org.apache.pulsar.client.admin.PulsarAdminException;
-import org.apache.pulsar.common.policies.data.TenantInfo;
-
 import com.beust.jcommander.Parameter;
 import com.beust.jcommander.Parameters;
 import com.beust.jcommander.converters.CommaParameterSplitter;
-import com.google.common.collect.Sets;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+
+import org.apache.pulsar.client.admin.PulsarAdmin;
+import org.apache.pulsar.client.admin.PulsarAdminException;
+import org.apache.pulsar.common.policies.data.TenantInfo;
 
 @Parameters(commandDescription = "Operations about tenants")
 public class CmdTenants extends CmdBase {
@@ -55,38 +58,57 @@ void run() throws PulsarAdminException {
 private java.util.List params;
 
 @Parameter(names = { "--admin-roles",
-"-r" }, description = "Comma separated Admin roles", required 
= true, splitter = CommaParameterSplitter.class)
+"-r" }, description = "Comma separated list of auth principal 
allowed to administrate the tenant", required = false, splitter = 
CommaParameterSplitter.class)
 private java.util.List adminRoles;
 
 @Parameter(names = { "--allowed-clusters",
-"-c" }, description = "Comma separated allowed clusters", 
required = true, splitter = CommaParameterSplitter.class)
+"-c" }, description = "Comma separated allowed clusters. If 
empty, the tenant will have access to all clusters", required = false, splitter 
= CommaParameterSplitter.class)
 private java.util.List allowedClusters;
 
 @Override
 void run() throws PulsarAdminException {
 String tenant = getOneArgument(params);
-TenantInfo tenantInfo = new 
TenantInfo(Sets.newHashSet(adminRoles), Sets.newHashSet(allowedClusters));
+
+if (adminRoles == null) {
+adminRoles = Collections.emptyList();
+}
+
+if (allowedClusters == null || allowedClusters.isEmpty()) {
+// Default to all available cluster
+allowedClusters = admin.clusters().getClusters();
+}
+
+TenantInfo tenantInfo = new TenantInfo(new HashSet<>(adminRoles), 
new HashSet<>(allowedClusters));
 admin.tenants().createTenant(tenant, tenantInfo);
 }
 }
 
-@Parameters(commandDescription = "Updates a tenant")
+@Parameters(commandDescription = "Updates the configuration for a tenant")
 private class Update extends CliCommand {
 @Parameter(description = "tenant-name", required = true)
 private java.util.List params;
 
 @Parameter(names = { "--admin-roles",
-"-r" }, description = "Comma separated Admin roles", required 
= true, splitter = CommaParameterSplitter.class)
+"-r" }, description = "Comma separated list of auth principal 
allowed to administrate the tenant. If empty the current set of roles won't be 
modified", required = false, splitter = CommaParameterSplitter.class)
 private java.util.List adminRoles;
 
 @Parameter(names = { "--allowed-clusters",
-"-c" }, description = "Comma separated allowed clusters", 
required = true, splitter = CommaParameterSplitter.class)
+"-c" }, description = "Comma separated allowed clusters. If 
omitted, the current set of clusters will be preserved", required = false, 
splitter = CommaParameterSplitter.class)
 private java.util.List allowedClusters;
 
 @Override
 void run() throws PulsarAdminException {
 String tenant = getOneArgument(params);
-TenantInfo tenantInfo = new 
TenantInfo(Sets.newHashSet(adminRoles), Sets.newHashSet(allowedClusters));
+
+if (adminRoles == null) {
+adminRoles = new 
ArrayList<>(admin.tenants().getTenantInfo(tenant).getAdminRoles());
+}
+
+if (allowedClusters == null) {
+allowedClusters = new 

[incubator-pulsar] branch master updated: Use official bookkeeper release 4.7.0 (#1605)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 732105f  Use official bookkeeper release 4.7.0 (#1605)
732105f is described below

commit 732105f45733e48bdfe0216c8148f6c6a1586261
Author: Sijie Guo 
AuthorDate: Wed Apr 18 14:31:14 2018 -0700

Use official bookkeeper release 4.7.0 (#1605)
---
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pom.xml b/pom.xml
index aeb8391..f733098 100644
--- a/pom.xml
+++ b/pom.xml
@@ -117,7 +117,7 @@ flexible messaging model and an intuitive client 
API.
 UTF-8
 true
 
-4.7.0-SNAPSHOT
+4.7.0
 3.4.10
 4.1.21.Final
 1.0.5

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[incubator-pulsar] branch master updated: Added option to create namespace on CLI passing list of clusters (#1579)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 488942e  Added option to create namespace on CLI passing list of 
clusters (#1579)
488942e is described below

commit 488942e71f1bc4cde9f2b8c35e57d98c5b6efef4
Author: Matteo Merli 
AuthorDate: Wed Apr 18 14:31:39 2018 -0700

Added option to create namespace on CLI passing list of clusters (#1579)

* Added option to create namespace on CLI passing list of clusters

* Fixed NamespaceCreateBundlesTest.testCreateNamespaceWithDefaultBundles

* Fixed expected value in test

* Use List instead of Set otherwise jcommander would get all param in a 
single string

* Added mocked test
---
 .../apache/pulsar/broker/admin/v2/Namespaces.java  |  13 +-
 .../apache/pulsar/broker/admin/AdminApiTest.java   |  21 +--
 .../pulsar/broker/admin/v1/V1_AdminApiTest.java|   1 +
 .../org/apache/pulsar/client/admin/Namespaces.java |  23 
 .../client/admin/internal/NamespacesImpl.java  |  17 +++
 .../pulsar/admin/cli/PulsarAdminToolTest.java  |  56 +++-
 .../org/apache/pulsar/admin/cli/CmdNamespaces.java | 144 -
 .../pulsar/common/policies/data/Policies.java  |   2 +-
 8 files changed, 200 insertions(+), 77 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java
index 7f28e0f..15ac532 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Namespaces.java
@@ -644,14 +644,17 @@ public class Namespaces extends NamespacesBase {
 }
 
 private Policies getDefaultPolicesIfNull(Policies policies) {
-if (policies != null) {
-return policies;
+if (policies == null) {
+policies = new Policies();
 }
 
-Policies defaultPolicies = new Policies();
 int defaultNumberOfBundles = 
config().getDefaultNumberOfNamespaceBundles();
-defaultPolicies.bundles = getBundles(defaultNumberOfBundles);
-return defaultPolicies;
+
+if (policies.bundles == null) {
+policies.bundles = getBundles(defaultNumberOfBundles);
+}
+
+return policies;
 }
 
 private static final Logger log = 
LoggerFactory.getLogger(Namespaces.class);
diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
index 7c24692..0afe36d 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
@@ -29,6 +29,13 @@ import static org.testng.Assert.assertNotNull;
 import static org.testng.Assert.assertTrue;
 import static org.testng.Assert.fail;
 
+import com.google.common.collect.BoundType;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Range;
+import com.google.common.collect.Sets;
+import com.google.common.hash.Hashing;
+
 import java.lang.reflect.Field;
 import java.net.URL;
 import java.util.ArrayList;
@@ -47,6 +54,8 @@ import java.util.concurrent.TimeUnit;
 import javax.ws.rs.client.InvocationCallback;
 import javax.ws.rs.client.WebTarget;
 
+import lombok.extern.slf4j.Slf4j;
+
 import org.apache.pulsar.broker.PulsarServerException;
 import org.apache.pulsar.broker.PulsarService;
 import org.apache.pulsar.broker.ServiceConfiguration;
@@ -94,8 +103,8 @@ import 
org.apache.pulsar.common.policies.data.PersistencePolicies;
 import org.apache.pulsar.common.policies.data.PersistentTopicInternalStats;
 import org.apache.pulsar.common.policies.data.PersistentTopicStats;
 import org.apache.pulsar.common.policies.data.Policies;
-import org.apache.pulsar.common.policies.data.TenantInfo;
 import org.apache.pulsar.common.policies.data.RetentionPolicies;
+import org.apache.pulsar.common.policies.data.TenantInfo;
 import org.apache.pulsar.common.util.Codec;
 import org.apache.pulsar.common.util.ObjectMapperFactory;
 import org.apache.pulsar.compaction.Compactor;
@@ -107,15 +116,6 @@ import org.testng.annotations.BeforeMethod;
 import org.testng.annotations.DataProvider;
 import org.testng.annotations.Test;
 
-import com.google.common.collect.BoundType;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Maps;
-import com.google.common.collect.Range;
-import com.google.common.collect.Sets;
-import com.google.common.hash.Hashing;
-
-import lombok.extern.slf4j.Slf4j;
-
 @Slf4j
 public class AdminApiTest extends MockedPulsarServiceBaseTest {
 
@@ -624,6 +624,7 @@ public class 

[GitHub] merlimat closed pull request #1605: Use official bookkeeper release 4.7.0

[GitBox]

merlimat closed pull request #1605: Use official bookkeeper release 4.7.0
URL: https://github.com/apache/incubator-pulsar/pull/1605
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pom.xml b/pom.xml
index aeb8391ab..f73309892 100644
--- a/pom.xml
+++ b/pom.xml
@@ -117,7 +117,7 @@ flexible messaging model and an intuitive client 
API.
 UTF-8
 true
 
-4.7.0-SNAPSHOT
+4.7.0
 3.4.10
 4.1.21.Final
 1.0.5


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie opened a new pull request #1612: Fix pulsar broker handling WebApplicationException

[GitBox]

sijie opened a new pull request #1612: Fix pulsar broker handling 
WebApplicationException
URL: https://github.com/apache/incubator-pulsar/pull/1612
 
 
   *Problem*
   
   Currently all WebApplicationExeptions are converted to 500 / 
InternalServerError by broker before responding to rest requests.
   
   *Solution*
   
   Make sure RestException handles WebApplicationExeption
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1586: Added config to deploy pulsar in aws using ebs storage

[GitBox]

merlimat commented on issue #1586: Added config to deploy pulsar in aws using 
ebs storage
URL: https://github.com/apache/incubator-pulsar/pull/1586#issuecomment-382520864
 
 
   @saandrews You might need to rebase/merge with master to get the test fixes


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #1586: Added config to deploy pulsar in aws using ebs storage

[GitBox]

sijie commented on issue #1586: Added config to deploy pulsar in aws using ebs 
storage
URL: https://github.com/apache/incubator-pulsar/pull/1586#issuecomment-382520889
 
 
   @saandrews you might need to rebase to latest master since cpp tests are 
flaky (due to #1611)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] saandrews commented on issue #1586: Added config to deploy pulsar in aws using ebs storage

[GitBox]

saandrews commented on issue #1586: Added config to deploy pulsar in aws using 
ebs storage
URL: https://github.com/apache/incubator-pulsar/pull/1586#issuecomment-382520004
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: cpp tests should wait for standalone up (#1611)

[sijie]

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new e2a585a  cpp tests should wait for standalone up (#1611)
e2a585a is described below

commit e2a585ab2fe1c2f8f8eb0d0f1b8ed2100e7a3fc4
Author: Sijie Guo 
AuthorDate: Wed Apr 18 13:17:08 2018 -0700

cpp tests should wait for standalone up (#1611)
---
 pulsar-client-cpp/run-unit-tests.sh | 12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/pulsar-client-cpp/run-unit-tests.sh 
b/pulsar-client-cpp/run-unit-tests.sh
index 0513bba..466eae0 100755
--- a/pulsar-client-cpp/run-unit-tests.sh
+++ b/pulsar-client-cpp/run-unit-tests.sh
@@ -34,16 +34,26 @@ PULSAR_STANDALONE_CONF=$PWD/test-conf/standalone-ssl.conf 
pulsar-dist/bin/pulsar
   --zookeeper-dir data2/standalone/zookeeper --bookkeeper-dir \
   data2/standalone/bookkeeper > broker-tls.log &
 auth_pid=$!;
-sleep 10
+
+echo "Wait for non-tls standalone up"
+until grep "Created tenant public" broker.log; do sleep 5; done
 
 # create property for test
 PULSAR_CLIENT_CONF=$PWD/test-conf/client.conf pulsar-dist/bin/pulsar-admin 
tenants create prop -r "" -c "unit"
+echo "Created tenant 'prop' - $?"
+
 PULSAR_CLIENT_CONF=$PWD/test-conf/client.conf pulsar-dist/bin/pulsar-admin 
tenants create property -r "" -c "cluster"
+echo "Created tenant 'property' - $?"
 
 PULSAR_CLIENT_CONF=$PWD/test-conf/client-ssl.conf pulsar-dist/bin/pulsar-admin 
clusters create \
 --url http://localhost:9765/ --url-secure https://localhost:9766/ \
 --broker-url pulsar://localhost:9885/ --broker-url-secure 
pulsar+ssl://localhost:9886/ \
 cluster
+PULSAR_CLIENT_CONF=$PWD/test-conf/client-ssl.conf pulsar-dist/bin/pulsar-admin 
clusters create \
+--url http://localhost:9765/ --url-secure https://localhost:9766/ \
+--broker-url pulsar://localhost:9885/ --broker-url-secure 
pulsar+ssl://localhost:9886/ \
+unit
+
 sleep 5
 
 cd tests

-- 
To stop receiving notification emails like this one, please contact
si...@apache.org.

[GitHub] sijie closed pull request #1611: cpp tests should wait for standalone up

[GitBox]

sijie closed pull request #1611: cpp tests should wait for standalone up
URL: https://github.com/apache/incubator-pulsar/pull/1611
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pulsar-client-cpp/run-unit-tests.sh 
b/pulsar-client-cpp/run-unit-tests.sh
index 0513bba57..466eae061 100755
--- a/pulsar-client-cpp/run-unit-tests.sh
+++ b/pulsar-client-cpp/run-unit-tests.sh
@@ -34,16 +34,26 @@ PULSAR_STANDALONE_CONF=$PWD/test-conf/standalone-ssl.conf 
pulsar-dist/bin/pulsar
   --zookeeper-dir data2/standalone/zookeeper --bookkeeper-dir \
   data2/standalone/bookkeeper > broker-tls.log &
 auth_pid=$!;
-sleep 10
+
+echo "Wait for non-tls standalone up"
+until grep "Created tenant public" broker.log; do sleep 5; done
 
 # create property for test
 PULSAR_CLIENT_CONF=$PWD/test-conf/client.conf pulsar-dist/bin/pulsar-admin 
tenants create prop -r "" -c "unit"
+echo "Created tenant 'prop' - $?"
+
 PULSAR_CLIENT_CONF=$PWD/test-conf/client.conf pulsar-dist/bin/pulsar-admin 
tenants create property -r "" -c "cluster"
+echo "Created tenant 'property' - $?"
 
 PULSAR_CLIENT_CONF=$PWD/test-conf/client-ssl.conf pulsar-dist/bin/pulsar-admin 
clusters create \
 --url http://localhost:9765/ --url-secure https://localhost:9766/ \
 --broker-url pulsar://localhost:9885/ --broker-url-secure 
pulsar+ssl://localhost:9886/ \
 cluster
+PULSAR_CLIENT_CONF=$PWD/test-conf/client-ssl.conf pulsar-dist/bin/pulsar-admin 
clusters create \
+--url http://localhost:9765/ --url-secure https://localhost:9766/ \
+--broker-url pulsar://localhost:9885/ --broker-url-secure 
pulsar+ssl://localhost:9886/ \
+unit
+
 sleep 5
 
 cd tests


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Fixed intermittent test BrokerServiceThrottlingTest.testLookupThrottl… (#1598)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 974cb78  Fixed intermittent test 
BrokerServiceThrottlingTest.testLookupThrottl… (#1598)
974cb78 is described below

commit 974cb78e84c4d4536a5b03a004a541e509cd9d07
Author: Matteo Merli 
AuthorDate: Wed Apr 18 13:03:43 2018 -0700

Fixed intermittent test BrokerServiceThrottlingTest.testLookupThrottl… 
(#1598)

* Fixed intermittent test 
BrokerServiceThrottlingTest.testLookupThrottlingForClientByBrokerInternalRetry

* more topic name fixes
---
 .../apache/pulsar/broker/service/BrokerServiceThrottlingTest.java   | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
index d476abe..8a2bb1c 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
@@ -89,7 +89,7 @@ public class BrokerServiceThrottlingTest extends 
BrokerTestBase {
 @Test
 public void testLookupThrottlingForClientByBroker0Permit() throws 
Exception {
 
-final String topicName = "persistent://prop/usw/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)
@@ -130,7 +130,7 @@ public class BrokerServiceThrottlingTest extends 
BrokerTestBase {
  */
 @Test
 public void testLookupThrottlingForClientByBroker() throws Exception {
-final String topicName = "persistent://prop/usw/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)
@@ -192,7 +192,7 @@ public class BrokerServiceThrottlingTest extends 
BrokerTestBase {
 @Test
 public void testLookupThrottlingForClientByBrokerInternalRetry() throws 
Exception {
 
-final String topicName = "persistent://prop/usw/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] merlimat closed pull request #1598: Fixed intermittent test BrokerServiceThrottlingTest.testLookupThrottl…

[GitBox]

merlimat closed pull request #1598: Fixed intermittent test 
BrokerServiceThrottlingTest.testLookupThrottl…
URL: https://github.com/apache/incubator-pulsar/pull/1598
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
index d476abe73..8a2bb1c65 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceThrottlingTest.java
@@ -89,7 +89,7 @@ public void testThrottlingLookupRequestSemaphore() throws 
Exception {
 @Test
 public void testLookupThrottlingForClientByBroker0Permit() throws 
Exception {
 
-final String topicName = "persistent://prop/usw/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)
@@ -130,7 +130,7 @@ public void testLookupThrottlingForClientByBroker0Permit() 
throws Exception {
  */
 @Test
 public void testLookupThrottlingForClientByBroker() throws Exception {
-final String topicName = "persistent://prop/usw/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)
@@ -192,7 +192,7 @@ public void testLookupThrottlingForClientByBroker() throws 
Exception {
 @Test
 public void testLookupThrottlingForClientByBrokerInternalRetry() throws 
Exception {
 
-final String topicName = "persistent://prop/usw/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie opened a new pull request #1611: cpp tests should wait for standalone up

[GitBox]

sijie opened a new pull request #1611: cpp tests should wait for standalone up
URL: https://github.com/apache/incubator-pulsar/pull/1611
 
 
   ### Motivation
   
   cpp tests are flaky and failing  because tenants are not created. Tenants 
are not created because standalone is not up.
   
   ### Modifications
   
   modify `run-unit-tests.sh` to make sure standalone are up running before 
creating tenants.
   
   ### Result
   
   cpp tests passed.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1609: Fix fail in BrokerServiceTest#testLookupThrottlingForClientByClient

[GitBox]

merlimat closed pull request #1609: Fix fail in 
BrokerServiceTest#testLookupThrottlingForClientByClient
URL: https://github.com/apache/incubator-pulsar/pull/1609
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
index 26aa09a3d..ee836c306 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
@@ -694,7 +694,7 @@ public void testTlsAuthUseTrustCert() throws Exception {
  */
 @Test
 public void testLookupThrottlingForClientByClient() throws Exception {
-final String topicName = "persistent://prop/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: fix name space (#1609)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 828a8c0  fix name space (#1609)
828a8c0 is described below

commit 828a8c0752be4b94e17f8d63405b5c410aa35432
Author: Jia Zhai 
AuthorDate: Wed Apr 18 11:25:59 2018 -0700

fix name space (#1609)
---
 .../test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
index 26aa09a..ee836c3 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
@@ -694,7 +694,7 @@ public class BrokerServiceTest extends BrokerTestBase {
  */
 @Test
 public void testLookupThrottlingForClientByClient() throws Exception {
-final String topicName = "persistent://prop/my-ns/newTopic";
+final String topicName = "persistent://prop/ns-abc/newTopic";
 
 String lookupUrl = new URI("pulsar://localhost:" + 
BROKER_PORT).toString();
 PulsarClient pulsarClient = 
PulsarClient.builder().serviceUrl(lookupUrl).statsInterval(0, TimeUnit.SECONDS)

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] zhaijack commented on issue #1609: Fix fail in BrokerServiceTest#testLookupThrottlingForClientByClient

[GitBox]

zhaijack commented on issue #1609: Fix fail in 
BrokerServiceTest#testLookupThrottlingForClientByClient
URL: https://github.com/apache/incubator-pulsar/pull/1609#issuecomment-382465069
 
 
   retest this please
   
   for 
   org.apache.pulsar.broker.service.v1.V1_ReplicatorTest.testReplication
   
org.apache.pulsar.broker.service.v1.V1_ReplicatorTest.testReplicationForBatchMessages


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #1604: Compression support for compaction

[GitBox]

ivankelly commented on issue #1604: Compression support for compaction
URL: https://github.com/apache/incubator-pulsar/pull/1604#issuecomment-382457687
 
 
   retest this please // 
BrokerServiceTest.testLookupThrottlingForClientByClient & c++ BasicEndToEndTest


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] saandrews commented on issue #1586: Added config to deploy pulsar in aws using ebs storage

[GitBox]

saandrews commented on issue #1586: Added config to deploy pulsar in aws using 
ebs storage
URL: https://github.com/apache/incubator-pulsar/pull/1586#issuecomment-382442708
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1593: PIP-17: the part of index block for offload.

[GitBox]

zhaijack commented on issue #1593: PIP-17:  the part of index block for offload.
URL: https://github.com/apache/incubator-pulsar/pull/1593#issuecomment-382442101
 
 
   Thanks @ivankelly for the comments, will change it and add the logic for 
reading back.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #1604: Compression support for compaction

[GitBox]

ivankelly commented on issue #1604: Compression support for compaction
URL: https://github.com/apache/incubator-pulsar/pull/1604#issuecomment-382428123
 
 
   retest this please // BrokerServiceTest.testLookupThrottlingForClientByClient


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #1549: offloadPrefix implementation for managed ledger

[GitBox]

ivankelly commented on issue #1549: offloadPrefix implementation for managed 
ledger
URL: https://github.com/apache/incubator-pulsar/pull/1549#issuecomment-382427995
 
 
   retest this please // BrokerServiceTest.testLookupThrottlingForClientByClient


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] srkukarni commented on issue #1609: Fix fail in BrokerServiceTest#testLookupThrottlingForClientByClient

[GitBox]

srkukarni commented on issue #1609: Fix fail in 
BrokerServiceTest#testLookupThrottlingForClientByClient
URL: https://github.com/apache/incubator-pulsar/pull/1609#issuecomment-382400498
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch asf-site updated: Updated site at revision b9d4912

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new af04ba8  Updated site at revision b9d4912
af04ba8 is described below

commit af04ba82b963b4b40069544faa42ca15d489f8ae
Author: jenkins 
AuthorDate: Wed Apr 18 08:46:43 2018 +

Updated site at revision b9d4912
---
 content/api/client/index-all.html  |  6 +++-
 .../apache/pulsar/client/api/ClientBuilder.html| 37 ++
 .../docs/latest/adaptors/PulsarSpark/index.html|  2 +-
 .../docs/latest/adaptors/PulsarStorm/index.html|  2 +-
 content/docs/latest/admin-api/overview/index.html  |  8 ++---
 content/docs/latest/admin/Authz/index.html |  4 +--
 content/docs/latest/clients/Java/index.html|  2 +-
 .../latest/cookbooks/PartitionedTopics/index.html  |  2 +-
 .../latest/cookbooks/RetentionExpiry/index.html|  2 +-
 .../cookbooks/message-deduplication/index.html |  2 +-
 .../docs/latest/deployment/aws-cluster/index.html  |  2 +-
 content/docs/latest/deployment/cluster/index.html  |  2 +-
 content/docs/latest/deployment/instance/index.html |  2 +-
 content/docs/latest/reference/CliTools/index.html  |  4 +--
 14 files changed, 52 insertions(+), 25 deletions(-)

diff --git a/content/api/client/index-all.html 
b/content/api/client/index-all.html
index ec53435..e07121f 100644
--- a/content/api/client/index-all.html
+++ b/content/api/client/index-all.html
@@ -1022,7 +1022,11 @@
 
 maxConcurrentLookupRequests(int)
 - Method in interface org.apache.pulsar.client.api.ClientBuilder
 
-Number of concurrent lookup-requests allowed on each 
broker-connection to prevent overload on broker.
+Number of concurrent lookup-requests allowed to send on 
each broker-connection to prevent overload on broker.
+
+maxLookupRequests(int)
 - Method in interface org.apache.pulsar.client.api.ClientBuilder
+
+Number of max lookup-requests allowed on each 
broker-connection to prevent overload on broker.
 
 maxNumberOfRejectedRequestPerConnection(int)
 - Method in interface org.apache.pulsar.client.api.ClientBuilder
 
diff --git a/content/api/client/org/apache/pulsar/client/api/ClientBuilder.html 
b/content/api/client/org/apache/pulsar/client/api/ClientBuilder.html
index 58ef073..5efb371 100644
--- a/content/api/client/org/apache/pulsar/client/api/ClientBuilder.html
+++ b/content/api/client/org/apache/pulsar/client/api/ClientBuilder.html
@@ -17,7 +17,7 @@
 catch(err) {
 }
 //-->
-var methods = 
{"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,"i8":6,"i9":6,"i10":6,"i11":6,"i12":6,"i13":6,"i14":6,"i15":6,"i16":6,"i17":6,"i18":6};
+var methods = 
{"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,"i8":6,"i9":6,"i10":6,"i11":6,"i12":6,"i13":6,"i14":6,"i15":6,"i16":6,"i17":6,"i18":6,"i19":6};
 var tabs = {65535:["t0","All Methods"],2:["t2","Instance 
Methods"],4:["t3","Abstract Methods"]};
 var altColor = "altColor";
 var rowColor = "rowColor";
@@ -210,31 +210,37 @@ extends Serializable, Cloneable
 
 ClientBuilder
 maxConcurrentLookupRequests(intmaxConcurrentLookupRequests)
-Number of concurrent lookup-requests allowed on each 
broker-connection to prevent overload on broker.
+Number of concurrent lookup-requests allowed to send on 
each broker-connection to prevent overload on broker.
 
 
 
 ClientBuilder
+maxLookupRequests(intmaxLookupRequests)
+Number of max lookup-requests allowed on each 
broker-connection to prevent overload on broker.
+
+
+
+ClientBuilder
 maxNumberOfRejectedRequestPerConnection(intmaxNumberOfRejectedRequestPerConnection)
 Set max number of broker-rejected requests in a certain 
time-frame (30 seconds) after which current connection
  will be closed and client creates a new connection that give chance to 
connect a different broker (default:
  50)
 
 
-
+
 ClientBuilder
 operationTimeout(intoperationTimeout,
 TimeUnitunit)
 Set the operation timeout (default: 30 seconds)
 
 
-
+
 ClientBuilder
 serviceUrl(StringserviceUrl)
 Configure the service URL for the Pulsar service.
 
 
-
+
 ClientBuilder
 statsInterval(longstatsInterval,
  TimeUnitunit)
@@ -242,7 +248,7 @@ extends Serializable, Cloneable
  statsIntervalSeconds It should be set to at least 1 second
 
 
-
+
 ClientBuilder
 tlsTrustCertsFilePath(StringtlsTrustCertsFilePath)
 Set the path to the trusted TLS certificate file
@@ -592,7 +598,7 @@ extends Serializable, Cloneable
 
 maxConcurrentLookupRequests
 ClientBuildermaxConcurrentLookupRequests(intmaxConcurrentLookupRequests)
-Number of concurrent lookup-requests allowed on each 
broker-connection to prevent overload on broker.
+Number of concurrent lookup-requests allowed to send on 
each broker-connection to prevent overload on broker.
  (default: 5000) It should be configured with higher value only in case 
of it