[GitHub] [pulsar] zhaohaidao commented on a change in pull request #5767: Support batch authorization of partitioned topic
zhaohaidao commented on a change in pull request #5767: Support batch
authorization of partitioned topic
URL: https://github.com/apache/pulsar/pull/5767#discussion_r352942140
##
File path:
pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/PersistentTopicsTest.java
##
@@ -312,4 +316,82 @@ public void testGetPartitionedTopicsList() throws
KeeperException, InterruptedEx
Assert.assertEquals(nonPersistentPartitionedTopics.size(), 1);
Assert.assertEquals(TopicName.get(nonPersistentPartitionedTopics.get(0)).getDomain().value(),
TopicDomain.non_persistent.value());
}
+
+@Test
+public void testGrantNonPartitionedTopic() {
+final String topicName = "non-partitioned-topic";
+persistentTopics.createNonPartitionedTopic(testTenant, testNamespace,
topicName, true);
+String role = "role";
+Set expectActions = new HashSet<>();
+expectActions.add(AuthAction.produce);
+persistentTopics.grantPermissionsOnTopic(testTenant, testNamespace,
topicName, role, expectActions);
+Map> permissions =
persistentTopics.getPermissionsOnTopic(testTenant, testNamespace, topicName);
+Assert.assertEquals(permissions.get(role), expectActions);
+}
+
+@Test
+public void testGrantPartitionedTopic() {
+final String partitionedTopicName = "partitioned-topic";
+final int numPartitions = 5;
+LocalZooKeeperCacheService mockLocalZooKeeperCacheService =
mock(LocalZooKeeperCacheService.class);
+ZooKeeperChildrenCache mockZooKeeperChildrenCache =
mock(ZooKeeperChildrenCache.class);
+
doReturn(mockLocalZooKeeperCacheService).when(pulsar).getLocalZkCacheService();
+
doReturn(mockZooKeeperChildrenCache).when(mockLocalZooKeeperCacheService).managedLedgerListCache();
Review comment:
> > you can provide this support in the internalGetPermissionsOnTopic
function.
>
> Is it better to create a new api called getPermissionsOnPartitionedTopic?
> I understand if I support searching partitioned topic in
internalGetPermissionsOnTopic. The output structure of getPermissionsOnTopic
will be changed to
> Map which will break API compatibility?
@tuteng Can you give me some advice about this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] yjshen opened a new pull request #5788: [pulsar-java-client] Decouple partition switch frequency with maxPublishDelay in producer configuration
yjshen opened a new pull request #5788: [pulsar-java-client] Decouple partition switch frequency with maxPublishDelay in producer configuration URL: https://github.com/apache/pulsar/pull/5788 # Motivation Currently, while using round-robin message router, messages would be route to different partitions at the `maxPublishDelay` boundaries. However, when GC occurs, the output partition would change frequently and results in small batches. This PR introduces a conf field setting the round-robin partition switch frequency: `batchingPartitionSwitchFrequencyByPublishDelay`. With this PR, the partition switch interval could be set as `maxPublishDelay` * `frequency`. ### Modifications `batchingPartitionSwitchFrequencyByPublishDelay` in `ProducerConfiguratiionData.java` ### Verifying this change This change is already covered by existing tests. ### Does this pull request potentially affect one of the following parts: - The public API: (yes) ### Documentation - Does this pull request introduce a new feature? (yes ) - If yes, how is the feature documented? (docs / JavaDocs) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5570: Transaction log implemention
congbobo184 removed a comment on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560979151 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5570: Transaction log implemention
congbobo184 removed a comment on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560375812 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5570: Transaction log implemention
congbobo184 removed a comment on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560335351 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5570: Transaction log implemention
congbobo184 removed a comment on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560979183 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5570: Transaction log implemention
congbobo184 removed a comment on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560375763 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] huangdx0726 commented on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink
huangdx0726 commented on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink URL: https://github.com/apache/pulsar/pull/5635#issuecomment-561013269 @tuteng I'v fixed ,PTAL. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] fangpengcheng95 commented on issue #5789: [docs] Fix persistent topics
fangpengcheng95 commented on issue #5789: [docs] Fix persistent topics URL: https://github.com/apache/pulsar/pull/5789#issuecomment-561033938 I will be very grateful if someone can give some reviews or advice. thx~ @Jennifer88huang @codelipenghui This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] huangdx0726 commented on a change in pull request #5716: [Issues 5709]remove the namespace checking
huangdx0726 commented on a change in pull request #5716: [Issues 5709]remove
the namespace checking
URL: https://github.com/apache/pulsar/pull/5716#discussion_r353019113
##
File path:
pulsar-client/src/test/java/org/apache/pulsar/client/impl/MultiTopicsConsumerImplTest.java
##
@@ -62,4 +71,43 @@ public void testGetStats() throws Exception {
impl.getStats();
}
+@Test
+public void multiTopicsInDifferentNameSpace() throws PulsarClientException
{
+List topics = new ArrayList<>();
+topics.add("persistent://public/default/MultiTopics1");
+topics.add("persistent://public/test-multi/MultiTopics2");
+topics.add("persistent://public/test-multi/MultiTopics3");
+ClientConfigurationData conf = new ClientConfigurationData();
+conf.setServiceUrl("pulsar://localhost:6650");
+conf.setStatsIntervalSeconds(100);
+
+PulsarClientImpl clientImpl = new PulsarClientImpl(conf);
+
+Consumer consumer = clientImpl.newConsumer().topics(topics)
+.subscriptionName("multiTopicSubscription")
+.messageListener(new MessageListener() {
+@Override
+public void received(Consumer consumer,
Message msg) {
+ if("producer".equals(msg.getProducerName()) ||
"producer1".equals(msg.getProducerName())){
+ String message = new String(msg.getData());
+ assertTrue(message.contains("MultiTopics"));
Review comment:
I'v fixed ,PTAL.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] zhaohaidao commented on a change in pull request #5767: Support batch authorization of partitioned topic
zhaohaidao commented on a change in pull request #5767: Support batch
authorization of partitioned topic
URL: https://github.com/apache/pulsar/pull/5767#discussion_r352938788
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
##
@@ -301,13 +295,29 @@ protected void internalGrantPermissionsOnTopic(String
role, Set acti
log.warn("[{}] Failed to grant permissions on topic {}: concurrent
modification", clientAppId(),
topicUri);
throw new RestException(Status.CONFLICT, "Concurrent
modification");
-}
-catch (Exception e) {
+} catch (Exception e) {
log.error("[{}] Failed to grant permissions for topic {}",
clientAppId(), topicUri, e);
throw new RestException(e);
}
}
+protected void internalGrantPermissionsOnTopic(String role,
Set actions) {
+// This operation should be reading from zookeeper and it should be
allowed without having admin privileges
+validateAdminAccessForTenant(namespaceName.getTenant());
+validatePoliciesReadOnlyAccess();
+
+PartitionedTopicMetadata meta = getPartitionedTopicMetadata(topicName,
true, false);
Review comment:
@tuteng As far as I understand, rest API v1 already supports partitioned
topic. Can you help me check if my understanding is right.
org.apache.pulsar.broker.admin.v1.PersistentTopics has implement API about
partitioned topic, like createPartitionedTopic and getPartitionedMetadata as
followed
```java
@PUT
@Path("/{property}/{cluster}/{namespace}/{topic}/partitions")
@ApiOperation(hidden = true, value = "Create a partitioned topic.",
notes = "It needs to be called before creating a producer on a partitioned
topic.")
@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have
admin permission"),
@ApiResponse(code = 409, message = "Partitioned topic already
exist") })
public void createPartitionedTopic(@PathParam("property") String
property, @PathParam("cluster") String cluster,
@PathParam("namespace") String namespace, @PathParam("topic")
@Encoded String encodedTopic,
int numPartitions) {
validateTopicName(property, cluster, namespace, encodedTopic);
internalCreatePartitionedTopic(numPartitions);
}
...
@GET
@Path("/{property}/{cluster}/{namespace}/{topic}/partitions")
@ApiOperation(hidden = true, value = "Get partitioned topic metadata.")
@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have
admin permission") })
public PartitionedTopicMetadata
getPartitionedMetadata(@PathParam("property") String property,
@PathParam("cluster") String cluster, @PathParam("namespace")
String namespace,
@PathParam("topic") @Encoded String encodedTopic,
@QueryParam("authoritative") @DefaultValue("false") boolean
authoritative,
@QueryParam("checkAllowAutoCreation") @DefaultValue("false")
boolean checkAllowAutoCreation) {
validateTopicName(property, cluster, namespace, encodedTopic);
return internalGetPartitionedMetadata(authoritative,
checkAllowAutoCreation);
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #5784: Create load balance znode of broker when znode missed.
sijie commented on a change in pull request #5784: Create load balance znode of
broker when znode missed.
URL: https://github.com/apache/pulsar/pull/5784#discussion_r352963190
##
File path:
pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/ModularLoadManagerImplTest.java
##
@@ -630,4 +630,14 @@ public void testOwnBrokerZnodeByMultipleBroker() throws
Exception {
pulsar.close();
}
+
+@Test
+public void testZnodeMissed() throws Exception {
+String path = LoadManager.LOADBALANCE_BROKERS_ROOT + "/" +
pulsar1.getAdvertisedAddress() + ":" +
pulsar1.getConfiguration().getWebServicePort().get();
+ZkUtils.deleteFullPathOptimistic(pulsar1.getZkClient(), path, -1);
+// Wait broker write load balance data to zookeeper, default interval
is 5s.
+Thread.sleep(6000);
Review comment:
Can we avoid sleeping in the test? It introduces flakiness to the whole test
suite. for example manually trigger load manager to write load balance data to
zookeeper.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560979151 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560979183 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on a change in pull request #5786: Fix ArrayIndexOutOfBoundsException in PerformanceProducer.
rdhabalia commented on a change in pull request #5786: Fix ArrayIndexOutOfBoundsException in PerformanceProducer. URL: https://github.com/apache/pulsar/pull/5786#discussion_r352927083 ## File path: pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceProducer.java ## @@ -87,8 +87,8 @@ private static final LongAdder totalMessagesSent = new LongAdder(); private static final LongAdder totalBytesSent = new LongAdder(); -private static Recorder recorder = new Recorder(TimeUnit.SECONDS.toMillis(12), 5); -private static Recorder cumulativeRecorder = new Recorder(TimeUnit.SECONDS.toMillis(12), 5); +private static Recorder recorder = new Recorder(TimeUnit.SECONDS.toMicros(12), 5); Review comment: are we changing latency output in microseconds? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar.wiki] branch master updated: Created PIP 51: Tenant policy support (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 76f8033 Created PIP 51: Tenant policy support (markdown) 76f8033 is described below commit 76f80330aab02a408042354a79a61c468a2175f4 Author: Sijie Guo AuthorDate: Mon Dec 2 17:45:14 2019 -0800 Created PIP 51: Tenant policy support (markdown) --- PIP-51:-Tenant-policy-support.md | 19 +++ 1 file changed, 19 insertions(+) diff --git a/PIP-51:-Tenant-policy-support.md b/PIP-51:-Tenant-policy-support.md new file mode 100644 index 000..61e687e --- /dev/null +++ b/PIP-51:-Tenant-policy-support.md @@ -0,0 +1,19 @@ +- Status: Draft +- Author: Alexandre DUVAL +- Pull request: +- Mailing list discussion: https://lists.apache.org/thread.html/a937326861b8e49fdb9fc8982010f41fb978a88311ebeb0f24bb695f@%3Cdev.pulsar.apache.org%3E +- Release: + +## Motivation + +Pulsar quotas, retentions and other policies are defined on namespaces level and only enforced at topic level. It would be great to have global policies on tenant level which can be overriden if policies are lower in namespaces level. + +The main goal is to provide a tenant for external users and a the way to define namespaces and so on with tenant level policies defined by the tenant provider. + +## Proposed changes + +The tenant's adminRoles property would be used to define the tenant's level policies. Then we will need to add a tenant userRoles which should be able to create namespaces and everything in the tenant that does not exceed the tenants level policies. + +Then the namespaces policies should inherit from tenant policies with a global verifier for all namespaces to not exceed tenant policies. + +It wouldn't be too difficult for retention and storage part. We would have to just check periodically on the already supplied brokers load report and get the overall namespace/tenant quota. If quotas exceed, then block producers only (not sure delete exceeded data because of the async block part is a good idea). \ No newline at end of file
[pulsar] branch master updated (078ba44 -> e121108)
This is an automated email from the ASF dual-hosted git repository. rxl pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 078ba44 Add documentation for HDFS offload (#5762) add e121108 [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed (#5734) No new revisions were added by this update. Summary of changes: .../stats/client/PulsarBrokerStatsClientTest.java | 61 ++ .../org/apache/pulsar/client/impl/HttpClient.java | 6 ++- .../pulsar/client/impl/PulsarClientImpl.java | 29 +- .../pulsar/client/impl/ConnectionTimeoutTest.java | 6 +-- 4 files changed, 95 insertions(+), 7 deletions(-)
[GitHub] [pulsar] wolfstudy merged pull request #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed
wolfstudy merged pull request #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed URL: https://github.com/apache/pulsar/pull/5734 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy closed issue #5597: [pulsar-client-java] retry when getPartitionedTopicMetadata failed
wolfstudy closed issue #5597: [pulsar-client-java] retry when getPartitionedTopicMetadata failed URL: https://github.com/apache/pulsar/issues/5597 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5786: Fix ArrayIndexOutOfBoundsException in PerformanceProducer.
codelipenghui commented on a change in pull request #5786: Fix ArrayIndexOutOfBoundsException in PerformanceProducer. URL: https://github.com/apache/pulsar/pull/5786#discussion_r352929587 ## File path: pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceProducer.java ## @@ -87,8 +87,8 @@ private static final LongAdder totalMessagesSent = new LongAdder(); private static final LongAdder totalBytesSent = new LongAdder(); -private static Recorder recorder = new Recorder(TimeUnit.SECONDS.toMillis(12), 5); -private static Recorder cumulativeRecorder = new Recorder(TimeUnit.SECONDS.toMillis(12), 5); +private static Recorder recorder = new Recorder(TimeUnit.SECONDS.toMicros(12), 5); Review comment: Currently, the microseconds was recorded into recorder but output in mills. We can change the record value to mills or change the highestTrackableValue to microseconds. The latter is currently selected This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
jiazhai commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-560972727 @digi691 increase the numHttpServerThreads could solve this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on issue #5683: Cannot create topic under sample/standalone/ns1
zymap commented on issue #5683: Cannot create topic under sample/standalone/ns1 URL: https://github.com/apache/pulsar/issues/5683#issuecomment-560986261 hi @candlerb, If you are using curl to request the service. You should request the v1 version of the service. In the Pulsar admin, we are using the namespace name to distinguish the client should request which version of the service. And there are some methods that are not supported in the v1 version of the service. Like the create a non-partitioned topic you are trying. You can find the supported method from here: https://github.com/apache/pulsar/tree/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1. Or you can find them in the past documentation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5786: Fix ArrayIndexOutOfBoundsException in PerformanceProducer.
codelipenghui commented on issue #5786: Fix ArrayIndexOutOfBoundsException in PerformanceProducer. URL: https://github.com/apache/pulsar/pull/5786#issuecomment-560937434 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5784: Create load balance znode of broker when znode missed.
codelipenghui commented on issue #5784: Create load balance znode of broker when znode missed. URL: https://github.com/apache/pulsar/pull/5784#issuecomment-560937714 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr edited a comment on issue #5519: Create producer issues using function worker as separate node
KannarFr edited a comment on issue #5519: Create producer issues using function worker as separate node URL: https://github.com/apache/pulsar/issues/5519#issuecomment-551084766 I think it's not because the exception occurs instantly. The ZK timeout is confired to 3ms everywhere. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (e121108 -> 9137065)
This is an automated email from the ASF dual-hosted git repository. hjf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e121108 [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed (#5734) add 9137065 [docs] Fix link issue for "Authentication and authorization in Pulsar" (#5764) No new revisions were added by this update. Summary of changes: site2/website/versioned_docs/version-2.4.1/security-authorization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [pulsar] Jennifer88huang merged pull request #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar"
Jennifer88huang merged pull request #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5764 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy closed issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy closed issue #5695: Go: Segfault on messageId serialize after getting it from producer URL: https://github.com/apache/pulsar/issues/5695 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer
wolfstudy commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer URL: https://github.com/apache/pulsar/issues/5695#issuecomment-560982931 @lspgn will close the issue if `sendWithMsgID` example does not solve your problem, please reopen or let me know, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5652: Server wedged after deleting and recreating functions
wolfstudy commented on issue #5652: Server wedged after deleting and recreating functions URL: https://github.com/apache/pulsar/issues/5652#issuecomment-560982381 @candlerb will close the issue If you encounter the same error, please let me know, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy closed issue #5652: Server wedged after deleting and recreating functions
wolfstudy closed issue #5652: Server wedged after deleting and recreating functions URL: https://github.com/apache/pulsar/issues/5652 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5784: Create load balance znode of broker when znode missed.
codelipenghui commented on a change in pull request #5784: Create load balance
znode of broker when znode missed.
URL: https://github.com/apache/pulsar/pull/5784#discussion_r352972885
##
File path:
pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/ModularLoadManagerImplTest.java
##
@@ -630,4 +630,14 @@ public void testOwnBrokerZnodeByMultipleBroker() throws
Exception {
pulsar.close();
}
+
+@Test
+public void testZnodeMissed() throws Exception {
+String path = LoadManager.LOADBALANCE_BROKERS_ROOT + "/" +
pulsar1.getAdvertisedAddress() + ":" +
pulsar1.getConfiguration().getWebServicePort().get();
+ZkUtils.deleteFullPathOptimistic(pulsar1.getZkClient(), path, -1);
+// Wait broker write load balance data to zookeeper, default interval
is 5s.
+Thread.sleep(6000);
Review comment:
Changed to manually trigger load manager to write load balance data to
zookeeper
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5787: deploy-kubernetes in minikube tutorial doc seems not work
jiazhai commented on issue #5787: deploy-kubernetes in minikube tutorial doc seems not work URL: https://github.com/apache/pulsar/issues/5787#issuecomment-560991367  minikube verison 1.5.2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-561044609 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #5784: Create load balance znode of broker when znode missed.
codelipenghui commented on a change in pull request #5784: Create load balance
znode of broker when znode missed.
URL: https://github.com/apache/pulsar/pull/5784#discussion_r352930533
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java
##
@@ -869,7 +868,13 @@ public void writeBrokerDataOnZooKeeper() {
updateLocalBrokerData();
if (needBrokerDataUpdate()) {
localData.setLastUpdate(System.currentTimeMillis());
-zkClient.setData(brokerZnodePath, localData.getJsonBytes(),
-1);
+
+try {
+zkClient.setData(brokerZnodePath,
localData.getJsonBytes(), -1);
+} catch (KeeperException.NoNodeException e) {
+ZkUtils.createFullPathOptimistic(zkClient,
brokerZnodePath, localData.getJsonBytes(),
Review comment:
Yes, Frequently appear in our recent performance tests, #5673 describe the
problem
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[pulsar.wiki] branch master updated: Updated Home (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 6a7276d Updated Home (markdown) 6a7276d is described below commit 6a7276d3650e4829c96fecea1f25a47702c49d41 Author: Sijie Guo AuthorDate: Mon Dec 2 17:43:01 2019 -0800 Updated Home (markdown) --- Home.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Home.md b/Home.md index 860a84d..bb0065b 100644 --- a/Home.md +++ b/Home.md @@ -7,6 +7,7 @@ We encourage to document any big change or feature or any addition to public use ### Proposed +* [[PIP 51: Tenant policy support]] * [[PIP 50: Package Management]] * [[PIP 49: Permission levels and inheritance]] * [[PIP 48: hierarchical admin api]]
[GitHub] [pulsar] wolfstudy commented on issue #4055: could not determine kind of name for C.pulsar_configure_set_negative_ack_redelivery_delay_ms
wolfstudy commented on issue #4055: could not determine kind of name for C.pulsar_configure_set_negative_ack_redelivery_delay_ms URL: https://github.com/apache/pulsar/issues/4055#issuecomment-560984034 @NotBad4U can you provide more info for the issue, e.g: - OS: Mac OS or Linux - CPP Client Version: - CGO client version: - Go Version: - so on.. This information can help me reproduce the problem better, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Sunkwan-Kwon commented on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library
Sunkwan-Kwon commented on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library URL: https://github.com/apache/pulsar/pull/5592#issuecomment-561008838 @sijie @wolfstudy Yes. What I intended is what @wolfstudy said. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on a change in pull request #5784: Create load balance znode of broker when znode missed.
rdhabalia commented on a change in pull request #5784: Create load balance
znode of broker when znode missed.
URL: https://github.com/apache/pulsar/pull/5784#discussion_r352928561
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java
##
@@ -869,7 +868,13 @@ public void writeBrokerDataOnZooKeeper() {
updateLocalBrokerData();
if (needBrokerDataUpdate()) {
localData.setLastUpdate(System.currentTimeMillis());
-zkClient.setData(brokerZnodePath, localData.getJsonBytes(),
-1);
+
+try {
+zkClient.setData(brokerZnodePath,
localData.getJsonBytes(), -1);
+} catch (KeeperException.NoNodeException e) {
+ZkUtils.createFullPathOptimistic(zkClient,
brokerZnodePath, localData.getJsonBytes(),
Review comment:
have you seen this behavior?
because `ModularLoadManagerImpl::start` tries to create
`createZPathIfNotExists` in the beginning.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5623: Dashboard assets referenced from absolute path fail to load when running behind a proxy
jiazhai commented on issue #5623: Dashboard assets referenced from absolute path fail to load when running behind a proxy URL: https://github.com/apache/pulsar/issues/5623#issuecomment-560978452 @bbonnin provided a work-around in https://github.com/bbonnin/pulsar-express/issues/19#issuecomment-560562570 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5690: Elasticsearch 5.x Sink connector incompatible
tuteng commented on issue #5690: Elasticsearch 5.x Sink connector incompatible URL: https://github.com/apache/pulsar/issues/5690#issuecomment-560983031 I will find some time to test the relevant version. If there is any problem, I think we may need to provide special support. Such as developing a new connector for the corresponding version. If there are others who need this feature, please leave a message under this issue to let us know. Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lspgn edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer
lspgn edited a comment on issue #5695: Go: Segfault on messageId serialize after getting it from producer URL: https://github.com/apache/pulsar/issues/5695#issuecomment-561032305 @wolfstudy sorry, can we re-open (was on vacations ; also I don't have access of a button to re-open), I ran tests with the example file: I'm getting the following: It rarely manages to display a message Id (maybe a race condition?), most of the time, it just panic at the first one. ``` The message Id value is: [(4294967295,140734859843417,57,0)] The message Id value is: [(75552472,0,2433,36)] fatal error: unexpected signal during runtime execution [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x470fafa] runtime stack: runtime.throw(0x434b363, 0x2a) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/panic.go:774 +0x72 runtime.sigpanic() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/signal_unix.go:378 +0x47c goroutine 1 [syscall]: runtime.cgocall(0x42932a0, 0xc000109800, 0x40) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc0001097d0 sp=0xc000109798 pc=0x400519b github.com/apache/pulsar/pulsar-client-go/pulsar._Cfunc_pulsar_message_id_str(0x6e003b0, 0x0) _cgo_gotypes.go:1279 +0x4a fp=0xc000109800 sp=0xc0001097d0 pc=0x4279c3a github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String.func1(0xca8080, 0x4037e80) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x5e fp=0xc000109840 sp=0xc000109800 pc=0x428906e github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String(0xca8080, 0x0, 0x0) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x37 fp=0xc0001098a8 sp=0xc000109840 pc=0x4280ba7 fmt.(*pp).handleMethods(0xc00012, 0x76, 0x1) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:630 +0x302 fp=0xc000109ab8 sp=0xc0001098a8 pc=0x40bc262 fmt.(*pp).printArg(0xc00012, 0x42e3bc0, 0xca8080, 0xc00076) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:713 +0x206 fp=0xc000109b50 sp=0xc000109ab8 pc=0x40bc826 fmt.(*pp).doPrintf(0xc00012, 0x4346be0, 0x1f, 0xc000109e20, 0x1, 0x1) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:1030 +0x15b fp=0xc000109c38 sp=0xc000109b50 pc=0x40c01eb fmt.Fprintf(0x43a5ce0, 0xca8008, 0x4346be0, 0x1f, 0xc000109e20, 0x1, 0x1, 0x0, 0x0, 0x0) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:204 +0x72 fp=0xc000109ca0 sp=0xc000109c38 pc=0x40b9092 fmt.Printf(...) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:213 main.main() /Users/me/repos/pulsar-test/producer.go:61 +0x2ee fp=0xc000109f60 sp=0xc000109ca0 pc=0x4291d1e runtime.main() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/proc.go:203 +0x21e fp=0xc000109fe0 sp=0xc000109f60 pc=0x4030cae runtime.goexit() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc000109fe8 sp=0xc000109fe0 pc=0x405b561 exit status 2 ``` ``` The message Id value is: [(75552472,0,2433,36)] fatal error: unexpected signal during runtime execution [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x470fafa] runtime stack: runtime.throw(0x434b363, 0x2a) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/panic.go:774 +0x72 runtime.sigpanic() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/signal_unix.go:378 +0x47c goroutine 1 [syscall]: runtime.cgocall(0x42932a0, 0xc000109800, 0xa0) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc0001097d0 sp=0xc000109798 pc=0x400519b github.com/apache/pulsar/pulsar-client-go/pulsar._Cfunc_pulsar_message_id_str(0x7001240, 0x0) _cgo_gotypes.go:1279 +0x4a fp=0xc000109800 sp=0xc0001097d0 pc=0x4279c3a github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String.func1(0xca8078, 0x4037e80) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x5e fp=0xc000109840 sp=0xc000109800 pc=0x428906e github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String(0xca8078, 0x0, 0x0) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x37 fp=0xc0001098a8 sp=0xc000109840 pc=0x4280ba7 fmt.(*pp).handleMethods(0xc00011e000, 0x76, 0x1) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:630 +0x302 fp=0xc000109ab8 sp=0xc0001098a8 pc=0x40bc262 fmt.(*pp).printArg(0xc00011e000, 0x42e3bc0, 0xca8078, 0x76) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:713 +0x206 fp=0xc000109b50 sp=0xc000109ab8 pc=0x40bc826 fmt.(*pp).doPrintf(0xc00011e000, 0x4346be0, 0x1f, 0xc000109e20, 0x1, 0x1)
[GitHub] [pulsar-client-go] jiazhai commented on issue #88: in go mod 1.12 compile error
jiazhai commented on issue #88: in go mod 1.12 compile error URL: https://github.com/apache/pulsar-client-go/issues/88#issuecomment-561032560 @sunvooker have you tried the method that @reugn suggested? Does it solve your issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] lspgn commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer
lspgn commented on issue #5695: Go: Segfault on messageId serialize after getting it from producer URL: https://github.com/apache/pulsar/issues/5695#issuecomment-561032305 @wolfstudy sorry to re-open (was on vacations), I ran tests with the example file: I'm getting the following: It rarely manages to display a message Id (maybe a race condition?), most of the time, it just panic at the first one. ``` The message Id value is: [(4294967295,140734859843417,57,0)] The message Id value is: [(75552472,0,2433,36)] fatal error: unexpected signal during runtime execution [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x470fafa] runtime stack: runtime.throw(0x434b363, 0x2a) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/panic.go:774 +0x72 runtime.sigpanic() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/signal_unix.go:378 +0x47c goroutine 1 [syscall]: runtime.cgocall(0x42932a0, 0xc000109800, 0x40) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc0001097d0 sp=0xc000109798 pc=0x400519b github.com/apache/pulsar/pulsar-client-go/pulsar._Cfunc_pulsar_message_id_str(0x6e003b0, 0x0) _cgo_gotypes.go:1279 +0x4a fp=0xc000109800 sp=0xc0001097d0 pc=0x4279c3a github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String.func1(0xca8080, 0x4037e80) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x5e fp=0xc000109840 sp=0xc000109800 pc=0x428906e github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String(0xca8080, 0x0, 0x0) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x37 fp=0xc0001098a8 sp=0xc000109840 pc=0x4280ba7 fmt.(*pp).handleMethods(0xc00012, 0x76, 0x1) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:630 +0x302 fp=0xc000109ab8 sp=0xc0001098a8 pc=0x40bc262 fmt.(*pp).printArg(0xc00012, 0x42e3bc0, 0xca8080, 0xc00076) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:713 +0x206 fp=0xc000109b50 sp=0xc000109ab8 pc=0x40bc826 fmt.(*pp).doPrintf(0xc00012, 0x4346be0, 0x1f, 0xc000109e20, 0x1, 0x1) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:1030 +0x15b fp=0xc000109c38 sp=0xc000109b50 pc=0x40c01eb fmt.Fprintf(0x43a5ce0, 0xca8008, 0x4346be0, 0x1f, 0xc000109e20, 0x1, 0x1, 0x0, 0x0, 0x0) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:204 +0x72 fp=0xc000109ca0 sp=0xc000109c38 pc=0x40b9092 fmt.Printf(...) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:213 main.main() /Users/me/repos/pulsar-test/producer.go:61 +0x2ee fp=0xc000109f60 sp=0xc000109ca0 pc=0x4291d1e runtime.main() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/proc.go:203 +0x21e fp=0xc000109fe0 sp=0xc000109f60 pc=0x4030cae runtime.goexit() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc000109fe8 sp=0xc000109fe0 pc=0x405b561 exit status 2 ``` ``` The message Id value is: [(75552472,0,2433,36)] fatal error: unexpected signal during runtime execution [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x470fafa] runtime stack: runtime.throw(0x434b363, 0x2a) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/panic.go:774 +0x72 runtime.sigpanic() /usr/local/Cellar/go/1.13.3/libexec/src/runtime/signal_unix.go:378 +0x47c goroutine 1 [syscall]: runtime.cgocall(0x42932a0, 0xc000109800, 0xa0) /usr/local/Cellar/go/1.13.3/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc0001097d0 sp=0xc000109798 pc=0x400519b github.com/apache/pulsar/pulsar-client-go/pulsar._Cfunc_pulsar_message_id_str(0x7001240, 0x0) _cgo_gotypes.go:1279 +0x4a fp=0xc000109800 sp=0xc0001097d0 pc=0x4279c3a github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String.func1(0xca8078, 0x4037e80) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x5e fp=0xc000109840 sp=0xc000109800 pc=0x428906e github.com/apache/pulsar/pulsar-client-go/pulsar.(*messageID).String(0xca8078, 0x0, 0x0) /Users/me/go/pkg/mod/github.com/apache/pulsar/pulsar-client-go@v0.0.0-20191203033040-9137065ffcac/pulsar/c_message.go:199 +0x37 fp=0xc0001098a8 sp=0xc000109840 pc=0x4280ba7 fmt.(*pp).handleMethods(0xc00011e000, 0x76, 0x1) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:630 +0x302 fp=0xc000109ab8 sp=0xc0001098a8 pc=0x40bc262 fmt.(*pp).printArg(0xc00011e000, 0x42e3bc0, 0xca8078, 0x76) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:713 +0x206 fp=0xc000109b50 sp=0xc000109ab8 pc=0x40bc826 fmt.(*pp).doPrintf(0xc00011e000, 0x4346be0, 0x1f, 0xc000109e20, 0x1, 0x1) /usr/local/Cellar/go/1.13.3/libexec/src/fmt/print.go:1030
[GitHub] [pulsar] fangpengcheng95 opened a new pull request #5789: [docs] Fix persistent topics
fangpengcheng95 opened a new pull request #5789: [docs] Fix persistent topics URL: https://github.com/apache/pulsar/pull/5789 This PR in order to fix the issue #5708, the non persistent topics will be made a new PR ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed
wolfstudy commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed URL: https://github.com/apache/pulsar/pull/5734#issuecomment-560968379 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet edited a comment on issue #5779: [docs] Reorganize "Tiered storage" content
Anonymitaet edited a comment on issue #5779: [docs] Reorganize "Tiered storage" content URL: https://github.com/apache/pulsar/issues/5779#issuecomment-560971815 @Jennifer88huang FYI: Suggestions from @sijie: - Keep tiered storage in the "Concepts and Architecture" chapter to let users know we have this feature. - Tiered storage is one of the most important features in Pulsar. Same as what we did for functions, connector, and schema, we can create an independent chapter for tiered storage and add sections to explain principles of offloaders, how to use offloaders, how to manage offloaders (admin API), and so on. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5779: [docs] Reorganize "Tiered storage" content
Anonymitaet commented on issue #5779: [docs] Reorganize "Tiered storage" content URL: https://github.com/apache/pulsar/issues/5779#issuecomment-560971815 @Jennifer88huang FYI: Suggestions from @sijie: - Keep tiered storage in the "Concepts and Architecture" chapter to let users know we have this feature. - Tiered storage is one of the most important features in Pulsar. Same as we did for functions, connector, and schema, we can create an independent chapter for tiered storage and add sections to explain principles of offloaders, how to use offloaders, how to manage offloaders (admin API), and so on. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-560984383 I understand that it is possible to solve this problem by increasing the number of threads of the parameter numHttpServerThreads. If the amount of concurrency for HTTP requests is large, we should increase the value of this parameter. We may need to add documents to illustrate the HTTP request problem under the condition of high concurrency. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5222: 2.4.1 pulsar-admin should have 8 schema compatibility check strategies
congbobo184 commented on issue #5222: 2.4.1 pulsar-admin should have 8 schema compatibility check strategies URL: https://github.com/apache/pulsar/issues/5222#issuecomment-561024846 #5227 has fixed it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] kaykillyou commented on issue #5668: bookie server's log error: Failed to restore rocksdb 000000000000000000/000000000000000001/000000000000000000
kaykillyou commented on issue #5668: bookie server's log error: Failed to restore rocksdb 00/01/00 URL: https://github.com/apache/pulsar/issues/5668#issuecomment-560974189 @jiazhai i start bookie with command "./pulsar-daemon start bookie", i found to configurate function "extraServerComponents=org.apache.bookkeeper.stream.server.StreamStorageLifecycleComponent " for the bookkeeper.conf, now i close it and seems normal now, I will observe it for a while. thanks very much! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5781: [Issue:4753] Add encryption docs to Functions develop
Jennifer88huang commented on issue #5781: [Issue:4753] Add encryption docs to Functions develop URL: https://github.com/apache/pulsar/pull/5781#issuecomment-560989855 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-561024988 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] merlimat merged pull request #109: Fix subscriber bug and use Consumer interface for multi topic consumer.
merlimat merged pull request #109: Fix subscriber bug and use Consumer interface for multi topic consumer. URL: https://github.com/apache/pulsar-client-go/pull/109 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar-client-go] branch master updated (ae057bd -> 4328f1b)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git. from ae057bd Fix repository link (#106) add 4328f1b Fix subscriber bug and use Consumer interface for multi topic consumer. (#109) No new revisions were added by this update. Summary of changes: pulsar/consumer_impl.go | 2 +- pulsar/consumer_multitopic.go | 41 +++-- pulsar/consumer_regex.go | 17 ++--- 3 files changed, 14 insertions(+), 46 deletions(-)
[GitHub] [pulsar-client-go] merlimat commented on a change in pull request #108: [Issue #107] SetWriteDeadline on connection
merlimat commented on a change in pull request #108: [Issue #107]
SetWriteDeadline on connection
URL: https://github.com/apache/pulsar-client-go/pull/108#discussion_r352739028
##
File path: pulsar/internal/connection.go
##
@@ -309,7 +310,13 @@ func (c *connection) WriteData(data []byte) {
func (c *connection) internalWriteData(data []byte) {
c.log.Debug("Write data: ", len(data))
- if _, err := c.cnx.Write(data); err != nil {
+ err := c.cnx.SetWriteDeadline(time.Now().Add(writeDeadline))
Review comment:
Shouldn't this be set just once per each connection? doing it here would set
it each time we write on the connection.
Also, since this will affect the publish timeouts, it would be better to
keep it configurable (as a client option).
Finally, what is the exact meaning of the write deadline? Will it fail the
write op if the connection is not writable for the specified amount of time? I
think we are already covering this scenario with the keep-alive mechanism that
detects either a clogged or stale TCP connection.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] wmccarley commented on issue #5395: Enable/disable allowAutoTopicCreation per namespace
wmccarley commented on issue #5395: Enable/disable allowAutoTopicCreation per namespace URL: https://github.com/apache/pulsar/issues/5395#issuecomment-560462427 @sleungtoast I am in a similar situation as you I would love to control this behavior at namespace level. For most of my clients I want allowAutoTopicCreation = false so I can funnel topic creation through a workflow and have an admin review the request etc.. However I have one client that uses only non-persistent topics and needs allowAutoTopicCreation = true. The way I have accomplished this is using namespace isolation policies to force that one client's namespaces to different brokers with different configurations. Not the best solution IMO but it will allow you to accommodate both use cases in one cluster. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] PierreZ commented on issue #5735: Provide raw counters metrics
PierreZ commented on issue #5735: Provide raw counters metrics URL: https://github.com/apache/pulsar/issues/5735#issuecomment-560480210 Found a naive solution, by creating a [Counter directly on the AbstractTopic class](https://github.com/apache/pulsar/commit/8625db717c021318d65f70b9b19b0c181ebbc64b). I am not sure if this is recommend though. What do you think @sijie? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai opened a new issue #5787: deploy-kubernetes in minikube tutorial doc seems not work
jiazhai opened a new issue #5787: deploy-kubernetes in minikube tutorial doc seems not work URL: https://github.com/apache/pulsar/issues/5787 **Describe the bug** In Pulsar website: http://pulsar.apache.org/docs/en/deploy-kubernetes/#minikube It is suggested use this command: > minikube start --memory=8192 --cpus=4 \ >--kubernetes-version=v1.10.5 but this will report error in macos 10.14.2  **To Reproduce** Steps to reproduce the behavior: 1. follow instructions in above like to install the latest minikube 2. run 'minikube start --memory=8192 --cpus=4 \ --kubernetes-version=v1.10.5' 3. See error **Expected behavior** No error happens This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5787: deploy-kubernetes in minikube tutorial doc seems not work
jiazhai commented on issue #5787: deploy-kubernetes in minikube tutorial doc seems not work URL: https://github.com/apache/pulsar/issues/5787#issuecomment-560459970 add @wolfstudy to take a look This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 opened a new pull request #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema.
congbobo184 opened a new pull request #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema. URL: https://github.com/apache/pulsar/pull/5782 ### Motivation Now, the logical is if isAllowAutoUpdateSchema = false, producer can't connect broker. So modify the logical, when isAllowAutoUpdateSchema = false, the first to check the schema whether is registered. ### Verifying this change Add the tests for it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library
wolfstudy edited a comment on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library URL: https://github.com/apache/pulsar/pull/5592#issuecomment-560308131 ping @sijie PTAL In here, it seems that it has nothing to do with the implementation mechanism of pulsar. The logic is that after I send a message to `logtopic`, I need to clear the data in `StrEntry`(all log messages will be appended to here). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library
wolfstudy edited a comment on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library URL: https://github.com/apache/pulsar/pull/5592#issuecomment-560308131 ping @sijie PTAL In here, it seems that it has nothing to do with the implementation mechanism of pulsar. The logic is that after I send a message to `logtopic`, I need to clear the data in `[]StrEntry`(all log messages will be appended to here). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy edited a comment on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library
wolfstudy edited a comment on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library URL: https://github.com/apache/pulsar/pull/5592#issuecomment-560308131 ping @sijie PTAL In here, it seems that it has nothing to do with the implementation mechanism of pulsar. The logic is that after I send a message to `logtopic`, I need to clear the data in `StrEntry`(All log messages will be appended to here). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560326550 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema.
congbobo184 commented on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema. URL: https://github.com/apache/pulsar/pull/5782#issuecomment-560327483 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5767: Support batch authorization of partitioned topic
tuteng commented on issue #5767: Support batch authorization of partitioned topic URL: https://github.com/apache/pulsar/pull/5767#issuecomment-560327265 Please check unit test use command: ``` mvn test -Dtest=org.apache.pulsar.broker.admin.AdminTest#persistentTopics -pl pulsar-broker ``` https://builds.apache.org/job/pulsar_precommit_java8/13483/testReport/junit/org.apache.pulsar.broker.admin/AdminTest/persistentTopics/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library
wolfstudy commented on issue #5592: [Issue 5589][pulsar-function-go] Fix a memory leak of pulsar-function-go library URL: https://github.com/apache/pulsar/pull/5592#issuecomment-560308131 ping @sijie PTAL In here, it seems that it has nothing to do with the implementation mechanism of pulsar. The logic is that after I send a message to `logtopic`, I need to clear the data in `StrEntry`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed
wolfstudy commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed URL: https://github.com/apache/pulsar/pull/5734#issuecomment-560312271 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui opened a new pull request #5784: Create load balance znode of broker when znode missed.
codelipenghui opened a new pull request #5784: Create load balance znode of broker when znode missed. URL: https://github.com/apache/pulsar/pull/5784 Fixes #5763 ### Motivation Create load balance znode of broker when znode missed. ### Modifications Catch the NoNodeException for write load balance data to zookeeper ### Verifying this change Added unit test. ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: ( no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation - Does this pull request introduce a new feature? (no) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on a change in pull request #5767: Support batch authorization of partitioned topic
tuteng commented on a change in pull request #5767: Support batch authorization
of partitioned topic
URL: https://github.com/apache/pulsar/pull/5767#discussion_r352513441
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
##
@@ -301,13 +295,29 @@ protected void internalGrantPermissionsOnTopic(String
role, Set acti
log.warn("[{}] Failed to grant permissions on topic {}: concurrent
modification", clientAppId(),
topicUri);
throw new RestException(Status.CONFLICT, "Concurrent
modification");
-}
-catch (Exception e) {
+} catch (Exception e) {
log.error("[{}] Failed to grant permissions for topic {}",
clientAppId(), topicUri, e);
throw new RestException(e);
}
}
+protected void internalGrantPermissionsOnTopic(String role,
Set actions) {
+// This operation should be reading from zookeeper and it should be
allowed without having admin privileges
+validateAdminAccessForTenant(namespaceName.getTenant());
+validatePoliciesReadOnlyAccess();
+
+PartitionedTopicMetadata meta = getPartitionedTopicMetadata(topicName,
true, false);
Review comment:
@zhaohaidao We need support rest API of v1 version for the partitioned topic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560335351 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink
tuteng commented on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink URL: https://github.com/apache/pulsar/pull/5635#issuecomment-560311429 Please check unit test: ``` 2019-12-02\T\05:14:57.921 [ERROR] testWriteGoodMessage(org.apache.pulsar.io.mongodb.MongoSinkTest) Time elapsed: 1.027 s <<< FAILURE! Wanted but not invoked: record.ack(); -> at org.apache.pulsar.io.mongodb.MongoSinkTest.testWriteGoodMessage(MongoSinkTest.java:137) However, there were exactly 2 interactions with this mock: record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) at org.apache.pulsar.io.mongodb.MongoSinkTest.testWriteGoodMessage(MongoSinkTest.java:137) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54) at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2019-12-02\T\05:14:57.923 [ERROR] testWriteMultipleMessages(org.apache.pulsar.io.mongodb.MongoSinkTest) Time elapsed: 1.014 s <<< FAILURE! Wanted but not invoked: record.ack(); -> at org.apache.pulsar.io.mongodb.MongoSinkTest.testWriteMultipleMessages(MongoSinkTest.java:151) However, there were exactly 6 interactions with this mock: record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) ``` You can try command: ``` mvn test -Dtest=unit-test-name (e.g: ConsumerBuilderImplTest) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc
congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-560327800 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema.
congbobo184 commented on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema. URL: https://github.com/apache/pulsar/pull/5782#issuecomment-560327644 run java8 test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc
congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-560327861 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 removed a comment on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema.
congbobo184 removed a comment on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema. URL: https://github.com/apache/pulsar/pull/5782#issuecomment-560327644 run java8 test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema.
congbobo184 commented on issue #5782: Modify the logical in producer carry a schema to connect broker with isAllowAutoUpdateSchema. URL: https://github.com/apache/pulsar/pull/5782#issuecomment-560328391 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] bry00 commented on issue #4162: kubectl appply -f bookie.yaml andthen Error
bry00 commented on issue #4162: kubectl appply -f bookie.yaml andthen Error URL: https://github.com/apache/pulsar/issues/4162#issuecomment-560298462 I have the same issue. Any update on this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-560302726 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ChangWinde commented on issue #5580: error when using pulsar-client with Kerberos
ChangWinde commented on issue #5580: error when using pulsar-client with Kerberos URL: https://github.com/apache/pulsar/issues/5580#issuecomment-560308786 > @ChangWinde Have you configure the broker? You can follow this document https://pulsar.apache.org/docs/en/security-kerberos/ to config the broker. If you using proxy service, you need to configure the proxy too. thanks for your reply.it seems something wrong with my kerberos, and it can work correctly~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5781: [Issue:4753] Add encryption docs to Functions develop
wolfstudy commented on issue #5781: [Issue:4753] Add encryption docs to Functions develop URL: https://github.com/apache/pulsar/pull/5781#issuecomment-560314520 @sijie @Jennifer88huang PTAL, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5570: Transaction log implemention
congbobo184 commented on issue #5570: Transaction log implemention URL: https://github.com/apache/pulsar/pull/5570#issuecomment-560326496 run Integration Tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 opened a new pull request #5785: Modify the logical for getting schema compatibility strategy
congbobo184 opened a new pull request #5785: Modify the logical for getting schema compatibility strategy URL: https://github.com/apache/pulsar/pull/5785 ### Motivation Modify the logical for getting schema compatibility strategy, if schemaCompatibilityStrategy = SchemaCompatibilityStrategy.UNDEFINED return the SchemaAutoUpdateCompatibilityStrategy. ### Verifying this change Add the tests for it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy opened a new pull request #5781: [Issue:4753] Add encryption docs to Functions develop
wolfstudy opened a new pull request #5781: [Issue:4753] Add encryption docs to Functions develop URL: https://github.com/apache/pulsar/pull/5781 Signed-off-by: xiaolong.ran Fixes #4753 ### Motivation The encryption section in the Functions develop chapter is missing. ### Modifications - Add encryption docs to Functions develop This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] huangdx0726 opened a new pull request #5783: [Issues 5763] Broker graceful shutdown when znode of loadbalance disappeared
huangdx0726 opened a new pull request #5783: [Issues 5763] Broker graceful shutdown when znode of loadbalance disappeared URL: https://github.com/apache/pulsar/pull/5783 [Issues 5763] Broker graceful shutdown when znode of loadbalance disappeared Fixes #5763 ### Motivation fix 5763 ### Modifications Called ``` createZPathIfNotExists ```before calling ```zkClient.setData``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng edited a comment on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink
tuteng edited a comment on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink URL: https://github.com/apache/pulsar/pull/5635#issuecomment-560311429 Please check unit test: ``` 2019-12-02\T\05:14:57.921 [ERROR] testWriteGoodMessage(org.apache.pulsar.io.mongodb.MongoSinkTest) Time elapsed: 1.027 s <<< FAILURE! Wanted but not invoked: record.ack(); -> at org.apache.pulsar.io.mongodb.MongoSinkTest.testWriteGoodMessage(MongoSinkTest.java:137) However, there were exactly 2 interactions with this mock: record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) at org.apache.pulsar.io.mongodb.MongoSinkTest.testWriteGoodMessage(MongoSinkTest.java:137) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54) at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2019-12-02\T\05:14:57.923 [ERROR] testWriteMultipleMessages(org.apache.pulsar.io.mongodb.MongoSinkTest) Time elapsed: 1.014 s <<< FAILURE! Wanted but not invoked: record.ack(); -> at org.apache.pulsar.io.mongodb.MongoSinkTest.testWriteMultipleMessages(MongoSinkTest.java:151) However, there were exactly 6 interactions with this mock: record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.write(MongoSink.java:111) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) record.getValue(); -> at org.apache.pulsar.io.mongodb.MongoSink.flush(MongoSink.java:148) ``` You can try command: ``` cd pulsar-io mvn test -Dtest=org.apache.pulsar.io.mongodb.MongoSinkTest#testWriteGoodMessage -pl mongo ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ChangWinde edited a comment on issue #5580: error when using pulsar-client with Kerberos
ChangWinde edited a comment on issue #5580: error when using pulsar-client with Kerberos URL: https://github.com/apache/pulsar/issues/5580#issuecomment-560308786 > @ChangWinde Have you configure the broker? You can follow this document https://pulsar.apache.org/docs/en/security-kerberos/ to config the broker. If you using proxy service, you need to configure the proxy too. thanks for your reply.it seems something wrong with my kerberos, and it can work correctly~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy closed issue #5580: error when using pulsar-client with Kerberos
wolfstudy closed issue #5580: error when using pulsar-client with Kerberos URL: https://github.com/apache/pulsar/issues/5580 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5580: error when using pulsar-client with Kerberos
wolfstudy commented on issue #5580: error when using pulsar-client with Kerberos URL: https://github.com/apache/pulsar/issues/5580#issuecomment-560359459 @ChangWinde Thanks for your reply, I will close this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on a change in pull request #5714: Add more info in the error messages
zymap commented on a change in pull request #5714: Add more info in the error
messages
URL: https://github.com/apache/pulsar/pull/5714#discussion_r352654157
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
##
@@ -1466,7 +1480,10 @@ public void seek(long timestamp) throws
PulsarClientException {
seekFuture.complete(null);
}).exceptionally(e -> {
log.error("[{}][{}] Failed to reset subscription: {}", topic,
subscription, e.getCause().getMessage());
-seekFuture.completeExceptionally(e.getCause());
+seekFuture.completeExceptionally(
+new PulsarClientException.WrapperException(
Review comment:
Because the A is thrown by `ClientCnx`. If we reconstruct the A it might
still change the client's behavior.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] zymap commented on a change in pull request #5714: Add more info in the error messages
zymap commented on a change in pull request #5714: Add more info in the error
messages
URL: https://github.com/apache/pulsar/pull/5714#discussion_r352654157
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
##
@@ -1466,7 +1480,10 @@ public void seek(long timestamp) throws
PulsarClientException {
seekFuture.complete(null);
}).exceptionally(e -> {
log.error("[{}][{}] Failed to reset subscription: {}", topic,
subscription, e.getCause().getMessage());
-seekFuture.completeExceptionally(e.getCause());
+seekFuture.completeExceptionally(
+new PulsarClientException.WrapperException(
Review comment:
because there are some methods when it uses the exception it will use the
`unwrap` method to get the real exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] Fomal-haut commented on issue #5755: UnAckedMessages admin information not correct
Fomal-haut commented on issue #5755: UnAckedMessages admin information not correct URL: https://github.com/apache/pulsar/issues/5755#issuecomment-560453013 Yes, a little weird... it can be reproduced on my side every time. Looks like you have changed some parameters, 1 out of 2 message is acknowledged in your code.(Mine is 1 out of 1000). Not sure whether this matters. Another thing is that have you changed "maxUnackedMessagesPerConsumer" to 0 in standalone config file? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on a change in pull request #5714: Add more info in the error messages
zymap commented on a change in pull request #5714: Add more info in the error
messages
URL: https://github.com/apache/pulsar/pull/5714#discussion_r352676175
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
##
@@ -1466,7 +1480,10 @@ public void seek(long timestamp) throws
PulsarClientException {
seekFuture.complete(null);
}).exceptionally(e -> {
log.error("[{}][{}] Failed to reset subscription: {}", topic,
subscription, e.getCause().getMessage());
-seekFuture.completeExceptionally(e.getCause());
+seekFuture.completeExceptionally(
+new PulsarClientException.WrapperException(
Review comment:
But it seems if I want to add the error message there are two ways to
update. One is updating the error message before the exception throws. Another
is unwrapping the wrapper when the method needs to handle the exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jiazhai edited a comment on issue #5687: Failed to enable State API in K8s installation
jiazhai edited a comment on issue #5687: Failed to enable State API in K8s installation URL: https://github.com/apache/pulsar/issues/5687#issuecomment-560376782 @pedro93, Thanks for the clarification. and sorry for the delay. `apply-config-from-env.py` will use the env vars, which passed into docker, to replace the config file(no mater the value is setting or not in the file). so your change to `deployment/terraform-ansible/templates/bookkeeper.conf` or `conf/bookkeeper.conf` will not take effect. In your example, the value in `values-mini.yaml` will take effect. Tried your example: ``` extraServerComponents: "org.apache.bookkeeper.stream.server.StreamStorageLifecycleComponent" ``` Indeed the value is not replaced in config file: env:  conf file:  start log:  This is because the change of #3827, and made user have to provide a pre-fix before the env. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5784: Create load balance znode of broker when znode missed.
codelipenghui commented on issue #5784: Create load balance znode of broker when znode missed. URL: https://github.com/apache/pulsar/pull/5784#issuecomment-560438621 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on a change in pull request #5714: Add more info in the error messages
zymap commented on a change in pull request #5714: Add more info in the error
messages
URL: https://github.com/apache/pulsar/pull/5714#discussion_r352654157
##
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
##
@@ -1466,7 +1480,10 @@ public void seek(long timestamp) throws
PulsarClientException {
seekFuture.complete(null);
}).exceptionally(e -> {
log.error("[{}][{}] Failed to reset subscription: {}", topic,
subscription, e.getCause().getMessage());
-seekFuture.completeExceptionally(e.getCause());
+seekFuture.completeExceptionally(
+new PulsarClientException.WrapperException(
Review comment:
if the A is thrown by `ClientCnx`, we reconstruct the A it might still
change the client's behavior.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5668: bookie server's log error: Failed to restore rocksdb 000000000000000000/000000000000000001/000000000000000000
jiazhai commented on issue #5668: bookie server's log error: Failed to restore rocksdb 00/01/00 URL: https://github.com/apache/pulsar/issues/5668#issuecomment-560454915 @kaykillyou Sorry for the late. If you were starting broker by `./pulsar-daemon start broker`, then I guess you were running in a cluster mode. In cluster mode '-nss' will not take effect. besides `./pulsar-daemon start broker`, was there any other conf file changes for `broker.conf`, and which command is used to start a bookie, was there any change for `bookkeeper.conf`? looked into the code, seems by default, cluster mode will not start function worker and storage store. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #4510: A host can't connect and won't try other hosts again in multi-hosts service url configuration
jiazhai commented on issue #4510: A host can't connect and won't try other hosts again in multi-hosts service url configuration URL: https://github.com/apache/pulsar/issues/4510#issuecomment-560393966 I would like to contribute to this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5785: Modify the logical for getting schema compatibility strategy
congbobo184 commented on issue #5785: Modify the logical for getting schema compatibility strategy URL: https://github.com/apache/pulsar/pull/5785#issuecomment-560395936 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
