[GitHub] jerrypeng commented on issue #2234: add stats entry for active consumer name and endpoint to get leader of functions cluster
jerrypeng commented on issue #2234: add stats entry for active consumer name and endpoint to get leader of functions cluster URL: https://github.com/apache/incubator-pulsar/pull/2234#issuecomment-407985376 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cckellogg commented on a change in pull request #2237: [website] Fix security page links and includes
cckellogg commented on a change in pull request #2237: [website] Fix security
page links and includes
URL: https://github.com/apache/incubator-pulsar/pull/2237#discussion_r205314170
##
File path: site2/website/scripts/replace.js
##
@@ -49,7 +49,7 @@ const versions = getVersions();
const latestVersion = versions[0];
const from = [
- /pulsar:version/g,
+ /{{pulsar:version}}/g,
Review comment:
{{}} are commonly used in templating engines so I was trying to avoid using
those for search and replace variables. I do see how it's confusing above.
Maybe we can do $pulsar:version or \ i'm not sure sure if
either of those are are any better, thoughts?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] cckellogg commented on a change in pull request #2237: [website] Fix security page links and includes
cckellogg commented on a change in pull request #2237: [website] Fix security
page links and includes
URL: https://github.com/apache/incubator-pulsar/pull/2237#discussion_r205314170
##
File path: site2/website/scripts/replace.js
##
@@ -49,7 +49,7 @@ const versions = getVersions();
const latestVersion = versions[0];
const from = [
- /pulsar:version/g,
+ /{{pulsar:version}}/g,
Review comment:
{{}} are commonly used in templating engines so I was trying to avoid using
those for search and replace variables. I do see how it's confusing above.
Maybe we can do $pulsar:version or i'm not sure sure if either
of those are are any better, thoughts?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie commented on a change in pull request #2237: [website] Fix security page links and includes
sijie commented on a change in pull request #2237: [website] Fix security page
links and includes
URL: https://github.com/apache/incubator-pulsar/pull/2237#discussion_r205310024
##
File path: site2/website/scripts/replace.js
##
@@ -49,7 +49,7 @@ const versions = getVersions();
const latestVersion = versions[0];
const from = [
- /pulsar:version/g,
+ /{{pulsar:version}}/g,
Review comment:
the main reason I changed to `{{}}` is because of this line:
https://github.com/apache/incubator-pulsar/pull/2237/files#diff-2e5bd7bb5ee452df3f4d1c90b6cfff0bL20
it is a bit confusing with `pulsar:pulsar:version`. I thought it is clear to
have `{{}}`. I changed `pulsar:version` in this PR for now. if we agreed that
is the direction we should head down, I can change the others to make things
consistent and clearer.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie commented on issue #2238: [website] Expand the includes `admin-api-partitioned-topics.md`
sijie commented on issue #2238: [website] Expand the includes `admin-api-partitioned-topics.md` URL: https://github.com/apache/incubator-pulsar/pull/2238#issuecomment-407949006 @cckellogg can you review this? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie opened a new pull request #2238: [website] Expand the includes `admin-api-partitioned-topics.md`
sijie opened a new pull request #2238: [website] Expand the includes `admin-api-partitioned-topics.md` URL: https://github.com/apache/incubator-pulsar/pull/2238 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cckellogg commented on a change in pull request #2237: [website] Fix security page links and includes
cckellogg commented on a change in pull request #2237: [website] Fix security
page links and includes
URL: https://github.com/apache/incubator-pulsar/pull/2237#discussion_r205309630
##
File path: site2/website/scripts/replace.js
##
@@ -49,7 +49,7 @@ const versions = getVersions();
const latestVersion = versions[0];
const from = [
- /pulsar:version/g,
+ /{{pulsar:version}}/g,
Review comment:
I'm curious as to why the {{}} were added? Should we be consistent with the
other variables (pulsar:variable_name)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie commented on issue #2237: [website] Fix security page links and includes
sijie commented on issue #2237: [website] Fix security page links and includes URL: https://github.com/apache/incubator-pulsar/pull/2237#issuecomment-407939948 @cckellogg ^^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie opened a new pull request #2237: [website] Fix security page links and includes
sijie opened a new pull request #2237: [website] Fix security page links and includes URL: https://github.com/apache/incubator-pulsar/pull/2237 ### Motivation There is a duplication in the security pages. That was introduced because security pages were relocated when developing new site. And a couple of includes links are not working. ### Changes - Fix all the security related links and removed unneeded `docs/administration-auth.md` page - Expanding the includes additionally: - fixing some anchors when modifying the pages - remove `pulsar io` and `pulsar functions` from concepts page since they are already top-level categories. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mykidong opened a new issue #2236: how to unsubscribe pulsar mailing list.
mykidong opened a new issue #2236: how to unsubscribe pulsar mailing list. URL: https://github.com/apache/incubator-pulsar/issues/2236 Sorry, it is not related to pulsar issue, but to pulsar mailing list unsubscription. Even though I have sent unsubscription emails to users-unsubscr...@pulsar.incubator.apache.org and dev-unsubscr...@pulsar.incubator.apache.org, I am stilling receiving mails from pulsar. I have no idea how to unsubscribe pulsar mailing list. - Kidong Lee. P.S: my email address: mykid...@gmail.com This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2182: Issue 2110: get-retention returns retentionSizeInMB as 0 when set to -1
sijie commented on issue #2182: Issue 2110: get-retention returns retentionSizeInMB as 0 when set to -1 URL: https://github.com/apache/incubator-pulsar/pull/2182#issuecomment-407926312 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie closed pull request #2199: Cleanup unnecessary casting and creating variables
sijie closed pull request #2199: Cleanup unnecessary casting and creating
variables
URL: https://github.com/apache/incubator-pulsar/pull/2199
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
index f70100d268..df9946068b 100644
---
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
+++
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
@@ -91,18 +91,13 @@ public void open(Map config, SourceContext
sourceContext) throws
org.apache.pulsar.client.api.Message message =
this.inputConsumer.receive();
String topicName;
-int partitionId;
// If more than one topics are being read than the Message return by
the consumer will be TopicMessageImpl
// If there is only topic being read then the Message returned by the
consumer wil be MessageImpl
if (message instanceof TopicMessageImpl) {
topicName = ((TopicMessageImpl) message).getTopicName();
-TopicMessageIdImpl topicMessageId = (TopicMessageIdImpl)
message.getMessageId();
-MessageIdImpl messageId = (MessageIdImpl)
topicMessageId.getInnerMessageId();
-partitionId = messageId.getPartitionIndex();
} else {
topicName =
this.pulsarSourceConfig.getTopicSerdeClassNameMap().keySet().iterator().next();
-partitionId = ((MessageIdImpl)
message.getMessageId()).getPartitionIndex();
}
Object object;
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-pulsar] branch master updated: Cleanup unnecessary casting and creating variables (#2199)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new bf56a8a Cleanup unnecessary casting and creating variables (#2199)
bf56a8a is described below
commit bf56a8ab168a2ec03ce44331ff7f95d687433f4a
Author: Rajan Dhabalia
AuthorDate: Wed Jul 25 16:11:01 2018 -0700
Cleanup unnecessary casting and creating variables (#2199)
### Motivation
I think due to recent refactoring some of the minor code which is not
necessary to keep. so, cleaning up unnecessary assignments.
---
.../main/java/org/apache/pulsar/functions/source/PulsarSource.java | 5 -
1 file changed, 5 deletions(-)
diff --git
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
index 7903267..ec71450 100644
---
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
+++
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
@@ -103,18 +103,13 @@ public class PulsarSource implements Source {
org.apache.pulsar.client.api.Message message =
this.inputConsumer.receive();
String topicName;
-int partitionId;
// If more than one topics are being read than the Message return by
the consumer will be TopicMessageImpl
// If there is only topic being read then the Message returned by the
consumer wil be MessageImpl
if (message instanceof TopicMessageImpl) {
topicName = ((TopicMessageImpl) message).getTopicName();
-TopicMessageIdImpl topicMessageId = (TopicMessageIdImpl)
message.getMessageId();
-MessageIdImpl messageId = (MessageIdImpl)
topicMessageId.getInnerMessageId();
-partitionId = messageId.getPartitionIndex();
} else {
topicName =
this.pulsarSourceConfig.getTopicSerdeClassNameMap().keySet().iterator().next();
-partitionId = ((MessageIdImpl)
message.getMessageId()).getPartitionIndex();
}
Object object;
[incubator-pulsar] branch master updated: Fix Spelling Mistake in class (#2232)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 5b543ca Fix Spelling Mistake in class (#2232)
5b543ca is described below
commit 5b543ca8b87e464c725939788222974b61443f04
Author: Ali Ahmed
AuthorDate: Wed Jul 25 16:09:54 2018 -0700
Fix Spelling Mistake in class (#2232)
* Fix Spelling Mistake in class
* Update Docs
---
...NativeExclmationFunction.java => JavaNativeExclamationFunction.java} | 2 +-
site/docs/latest/functions/api.md | 2 +-
site2/docs/functions-api.md | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java
b/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java
similarity index 92%
rename from
pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java
rename to
pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java
index 4653e1b..f09a436 100644
---
a/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java
+++
b/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java
@@ -20,7 +20,7 @@ package org.apache.pulsar.functions.api.examples;
import java.util.function.Function;
-public class JavaNativeExclmationFunction implements Function {
+public class JavaNativeExclamationFunction implements Function
{
@Override
public String apply(String input) {
return String.format("%s!", input);
diff --git a/site/docs/latest/functions/api.md
b/site/docs/latest/functions/api.md
index e07479c..6dd7b75 100644
--- a/site/docs/latest/functions/api.md
+++ b/site/docs/latest/functions/api.md
@@ -209,7 +209,7 @@ In general, you should use native functions when you don't
need access to the fu
There is one example Java native function in [this
folder](https://github.com/apache/incubator-pulsar/tree/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples):
-*
[`JavaNativeExclmationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java)
+*
[`JavaNativeExclamationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java)
### Java SDK functions {#java-sdk}
diff --git a/site2/docs/functions-api.md b/site2/docs/functions-api.md
index c673070..f0f4f8c 100644
--- a/site2/docs/functions-api.md
+++ b/site2/docs/functions-api.md
@@ -211,7 +211,7 @@ In general, you should use native functions when you don't
need access to the fu
There is one example Java native function in [this
folder](https://github.com/apache/incubator-pulsar/tree/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples):
-*
[`JavaNativeExclmationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java)
+*
[`JavaNativeExclamationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java)
### Java SDK functions
[GitHub] sijie closed pull request #2232: Fix Spelling Mistake in class
sijie closed pull request #2232: Fix Spelling Mistake in class
URL: https://github.com/apache/incubator-pulsar/pull/2232
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java
b/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java
similarity index 92%
rename from
pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java
rename to
pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java
index 4653e1bba8..f09a43688d 100644
---
a/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java
+++
b/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java
@@ -20,7 +20,7 @@
import java.util.function.Function;
-public class JavaNativeExclmationFunction implements Function {
+public class JavaNativeExclamationFunction implements Function
{
@Override
public String apply(String input) {
return String.format("%s!", input);
diff --git a/site/docs/latest/functions/api.md
b/site/docs/latest/functions/api.md
index e07479c9a8..6dd7b75459 100644
--- a/site/docs/latest/functions/api.md
+++ b/site/docs/latest/functions/api.md
@@ -209,7 +209,7 @@ In general, you should use native functions when you don't
need access to the fu
There is one example Java native function in [this
folder](https://github.com/apache/incubator-pulsar/tree/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples):
-*
[`JavaNativeExclmationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java)
+*
[`JavaNativeExclamationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java)
### Java SDK functions {#java-sdk}
diff --git a/site2/docs/functions-api.md b/site2/docs/functions-api.md
index c673070767..f0f4f8c325 100644
--- a/site2/docs/functions-api.md
+++ b/site2/docs/functions-api.md
@@ -211,7 +211,7 @@ In general, you should use native functions when you don't
need access to the fu
There is one example Java native function in [this
folder](https://github.com/apache/incubator-pulsar/tree/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples):
-*
[`JavaNativeExclmationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclmationFunction.java)
+*
[`JavaNativeExclamationFunction`](https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/JavaNativeExclamationFunction.java)
### Java SDK functions
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie closed pull request #2231: Add search bar to new site.
sijie closed pull request #2231: Add search bar to new site.
URL: https://github.com/apache/incubator-pulsar/pull/2231
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/site2/website/siteConfig.js b/site2/website/siteConfig.js
index a996ebfb16..30ca549552 100644
--- a/site2/website/siteConfig.js
+++ b/site2/website/siteConfig.js
@@ -63,9 +63,10 @@ const siteConfig = {
{doc: 'client-libraries', label: 'Client libraries'},
{href: '#community', label: 'Community'},
{href: '#apache', label: 'Apache'},
-{ search: true },
+// Determines search bar position among links
+//{ search: true },
// Determines language drop down position among links
-// { languages: true }
+//{ languages: true }
],
// If you have users set above, you add it here:
@@ -75,6 +76,10 @@ const siteConfig = {
headerIcon: 'img/pulsar.svg',
footerIcon: 'img/pulsar.svg',
favicon: 'img/pulsar.ico',
+ algolia: {
+apiKey: 'd226a455cecdd4bc18a554c1b47e5b52',
+indexName: 'apache_pulsar',
+ },
/* colors for website */
colors: {
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-pulsar] branch master updated: Add search bar to new site. (#2231)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new cca58fc Add search bar to new site. (#2231)
cca58fc is described below
commit cca58fc03f4f1da0d7c44b286b025bb250f13c23
Author: cckellogg
AuthorDate: Wed Jul 25 16:07:51 2018 -0700
Add search bar to new site. (#2231)
Add doc search to site version 2.
---
site2/website/siteConfig.js | 9 +++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/site2/website/siteConfig.js b/site2/website/siteConfig.js
index a996ebf..30ca549 100644
--- a/site2/website/siteConfig.js
+++ b/site2/website/siteConfig.js
@@ -63,9 +63,10 @@ const siteConfig = {
{doc: 'client-libraries', label: 'Client libraries'},
{href: '#community', label: 'Community'},
{href: '#apache', label: 'Apache'},
-{ search: true },
+// Determines search bar position among links
+//{ search: true },
// Determines language drop down position among links
-// { languages: true }
+//{ languages: true }
],
// If you have users set above, you add it here:
@@ -75,6 +76,10 @@ const siteConfig = {
headerIcon: 'img/pulsar.svg',
footerIcon: 'img/pulsar.svg',
favicon: 'img/pulsar.ico',
+ algolia: {
+apiKey: 'd226a455cecdd4bc18a554c1b47e5b52',
+indexName: 'apache_pulsar',
+ },
/* colors for website */
colors: {
[incubator-pulsar] branch master updated: [integration tests] Return exit code as part of execution result of running a command in testcontainers (#2233)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 596d681 [integration tests] Return exit code as part of execution
result of running a command in testcontainers (#2233)
596d681 is described below
commit 596d68170172937f92bcc0876eb36a42a99c0793
Author: Sijie Guo
AuthorDate: Wed Jul 25 16:06:43 2018 -0700
[integration tests] Return exit code as part of execution result of
running a command in testcontainers (#2233)
### Motivation
The default `ExecResult` in testcontainers doesn't include the exit code of
running a command in a test container.
### Changes
- Add a class `ContainerExecResult` to represent the result of executing a
command in a container, including exitCode, stdout and stderr
- Improve the existing util method in `DockerUtils` to return
`ContainerExecResult`
- Improve all the integration tests on validating exit codes before
validating outputs
---
.../pulsar/tests/integration/cli/CLITest.java | 39 ++--
.../integration/compaction/TestCompaction.java | 51 +++--
.../integration/containers/ChaosContainer.java | 19 +++-
.../integration/docker/ContainerExecResult.java| 33 ++
.../integration/functions/PulsarFunctionsTest.java | 10 +++--
.../runtime/PulsarFunctionsRuntimeTest.java| 17 ---
.../tests/integration/io/PulsarIOSinkTest.java | 25 +++
.../tests/integration/io/PulsarIOSourceTest.java | 24 ++
.../pulsar/tests/integration/smoke/SmokeTest.java | 17 ---
.../integration/topologies/PulsarCluster.java | 12 ++---
.../tests/integration/utils/DockerUtils.java | 52 --
11 files changed, 196 insertions(+), 103 deletions(-)
diff --git
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
index 7b4d0d0..e69b0dd 100644
---
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
+++
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
@@ -18,14 +18,15 @@
*/
package org.apache.pulsar.tests.integration.cli;
+import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertFalse;
+import static org.testng.Assert.assertNotEquals;
import static org.testng.Assert.assertTrue;
import org.apache.pulsar.tests.integration.containers.BrokerContainer;
+import org.apache.pulsar.tests.integration.docker.ContainerExecResult;
import org.apache.pulsar.tests.integration.topologies.PulsarCluster;
import org.apache.pulsar.tests.integration.topologies.PulsarClusterTestBase;
-import org.testcontainers.containers.Container.ExecResult;
-import org.testng.Assert;
import org.testng.annotations.Test;
/**
@@ -37,7 +38,8 @@ public class CLITest extends PulsarClusterTestBase {
public void testDeprecatedCommands() throws Exception {
String tenantName = "test-deprecated-commands";
-ExecResult result = pulsarCluster.runAdminCommandOnAnyBroker("--help");
+ContainerExecResult result =
pulsarCluster.runAdminCommandOnAnyBroker("--help");
+assertEquals(0, result.getExitCode());
assertFalse(result.getStdout().isEmpty());
assertFalse(result.getStdout().contains("Usage: properties "));
result = pulsarCluster.runAdminCommandOnAnyBroker(
@@ -63,7 +65,7 @@ public class CLITest extends PulsarClusterTestBase {
int i = 0;
for (BrokerContainer container : pulsarCluster.getBrokers()) {
-ExecResult result = container.execCmd(
+ContainerExecResult result = container.execCmd(
PulsarCluster.ADMIN_SCRIPT,
"persistent",
"create-subscription",
@@ -71,6 +73,7 @@ public class CLITest extends PulsarClusterTestBase {
"--subscription",
"" + subscriptionPrefix + i
);
+assertEquals(0, result.getExitCode());
assertTrue(result.getStdout().isEmpty());
assertTrue(result.getStderr().isEmpty());
i++;
@@ -81,7 +84,7 @@ public class CLITest extends PulsarClusterTestBase {
public void testTopicTerminationOnTopicsWithoutConnectedConsumers() throws
Exception {
String topicName =
"persistent://public/default/test-topic-termination";
BrokerContainer container = pulsarCluster.getAnyBroker();
-ExecResult result = container.execCmd(
+ContainerExecResult result = container.execCmd(
PulsarCluster.CLIENT_SCRIPT,
"produce",
"-m",
@@ -90,7 +93,8 @@ public class CLITest extends PulsarClusterTestBase {
"1",
[GitHub] sijie closed pull request #2233: [integration tests] Return exit code as part of execution result of running a command in testcontainers
sijie closed pull request #2233: [integration tests] Return exit code as part
of execution result of running a command in testcontainers
URL: https://github.com/apache/incubator-pulsar/pull/2233
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
index 7b4d0d00dd..e69b0dd4a5 100644
---
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
+++
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
@@ -18,14 +18,15 @@
*/
package org.apache.pulsar.tests.integration.cli;
+import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertFalse;
+import static org.testng.Assert.assertNotEquals;
import static org.testng.Assert.assertTrue;
import org.apache.pulsar.tests.integration.containers.BrokerContainer;
+import org.apache.pulsar.tests.integration.docker.ContainerExecResult;
import org.apache.pulsar.tests.integration.topologies.PulsarCluster;
import org.apache.pulsar.tests.integration.topologies.PulsarClusterTestBase;
-import org.testcontainers.containers.Container.ExecResult;
-import org.testng.Assert;
import org.testng.annotations.Test;
/**
@@ -37,7 +38,8 @@
public void testDeprecatedCommands() throws Exception {
String tenantName = "test-deprecated-commands";
-ExecResult result = pulsarCluster.runAdminCommandOnAnyBroker("--help");
+ContainerExecResult result =
pulsarCluster.runAdminCommandOnAnyBroker("--help");
+assertEquals(0, result.getExitCode());
assertFalse(result.getStdout().isEmpty());
assertFalse(result.getStdout().contains("Usage: properties "));
result = pulsarCluster.runAdminCommandOnAnyBroker(
@@ -63,7 +65,7 @@ public void testCreateSubscriptionCommand() throws Exception {
int i = 0;
for (BrokerContainer container : pulsarCluster.getBrokers()) {
-ExecResult result = container.execCmd(
+ContainerExecResult result = container.execCmd(
PulsarCluster.ADMIN_SCRIPT,
"persistent",
"create-subscription",
@@ -71,6 +73,7 @@ public void testCreateSubscriptionCommand() throws Exception {
"--subscription",
"" + subscriptionPrefix + i
);
+assertEquals(0, result.getExitCode());
assertTrue(result.getStdout().isEmpty());
assertTrue(result.getStderr().isEmpty());
i++;
@@ -81,7 +84,7 @@ public void testCreateSubscriptionCommand() throws Exception {
public void testTopicTerminationOnTopicsWithoutConnectedConsumers() throws
Exception {
String topicName =
"persistent://public/default/test-topic-termination";
BrokerContainer container = pulsarCluster.getAnyBroker();
-ExecResult result = container.execCmd(
+ContainerExecResult result = container.execCmd(
PulsarCluster.CLIENT_SCRIPT,
"produce",
"-m",
@@ -90,7 +93,8 @@ public void
testTopicTerminationOnTopicsWithoutConnectedConsumers() throws Excep
"1",
topicName);
-Assert.assertTrue(result.getStdout().contains("1 messages successfully
produced"));
+assertEquals(0, result.getExitCode());
+assertTrue(result.getStdout().contains("1 messages successfully
produced"));
// terminate the topic
result = container.execCmd(
@@ -98,7 +102,8 @@ public void
testTopicTerminationOnTopicsWithoutConnectedConsumers() throws Excep
"persistent",
"terminate",
topicName);
-Assert.assertTrue(result.getStdout().contains("Topic succesfully
terminated at"));
+assertEquals(0, result.getExitCode());
+assertTrue(result.getStdout().contains("Topic succesfully terminated
at"));
// try to produce should fail
result = pulsarCluster.getAnyBroker().execCmd(
@@ -109,6 +114,7 @@ public void
testTopicTerminationOnTopicsWithoutConnectedConsumers() throws Excep
"-n",
"1",
topicName);
+assertNotEquals(0, result.getExitCode());
assertTrue(result.getStdout().contains("Topic was already
terminated"));
}
@@ -117,7 +123,7 @@ public void testSchemaCLI() throws Exception {
BrokerContainer container = pulsarCluster.getAnyBroker();
String topicName = "persistent://public/default/test-schema-cli";
-ExecResult result = container.execCmd(
+ContainerExecResult result = container.execCmd(
[GitHub] sijie commented on a change in pull request #2233: [integration tests] Return exit code as part of execution result of running a command in testcontainers
sijie commented on a change in pull request #2233: [integration tests] Return
exit code as part of execution result of running a command in testcontainers
URL: https://github.com/apache/incubator-pulsar/pull/2233#discussion_r205288351
##
File path:
tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
##
@@ -162,16 +164,26 @@ public static String getContainerIP(DockerClient docker,
String containerId) {
throw new IllegalArgumentException("Container " + containerId + " has
no networks");
}
-public static String getContainerHostname(DockerClient docker, String
containerId) {
-return runCommand(docker, containerId, "hostname").trim();
+public static ContainerExecResult runCommand(DockerClient docker, String
containerId, String... cmd)
+throws Exception {
+return runCommand(docker, containerId, false, cmd);
}
-public static String runCommand(DockerClient docker, String containerId,
String... cmd) {
+public static ContainerExecResult runCommand(DockerClient docker,
+ String containerId,
+ boolean ignoreError,
Review comment:
good idea. will work on that in the next PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] aahmed-se commented on issue #2232: Fix Spelling Mistake in class
aahmed-se commented on issue #2232: Fix Spelling Mistake in class URL: https://github.com/apache/incubator-pulsar/pull/2232#issuecomment-407924049 (re)run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] zhaijack commented on issue #2219: Cpp client: add PatternMultiTopicsConsumerImpl to support regex subscribe
zhaijack commented on issue #2219: Cpp client: add PatternMultiTopicsConsumerImpl to support regex subscribe URL: https://github.com/apache/incubator-pulsar/pull/2219#issuecomment-407915589 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly commented on issue #890: Intermittent test failure : org.apache.pulsar.broker.service.ReplicatorTest.testCloseReplicatorStartProducer
ivankelly commented on issue #890: Intermittent test failure : org.apache.pulsar.broker.service.ReplicatorTest.testCloseReplicatorStartProducer URL: https://github.com/apache/incubator-pulsar/issues/890#issuecomment-407907639 Instead of fixing as it is now, this should be converted to an integration test first. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: Add script and README for publishing `pulsar-build` image (#2222)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2d1b100 Add script and README for publishing `pulsar-build` image
(#)
2d1b100 is described below
commit 2d1b100ba282f2cb983889fd80c7de1b2305a6d3
Author: Sijie Guo
AuthorDate: Wed Jul 25 14:21:25 2018 -0700
Add script and README for publishing `pulsar-build` image (#)
*Motivation*
The instructions are missing for publishing `pulsar-build` image.
*Changes*
- Add a script for publishing `pulsar-build` image
- Add a README for the instructions to do so
---
build/docker/README.md | 17 +++
build/docker/publish.sh | 57 +
2 files changed, 74 insertions(+)
diff --git a/build/docker/README.md b/build/docker/README.md
index 7a01d2c..bc2fe53 100644
--- a/build/docker/README.md
+++ b/build/docker/README.md
@@ -29,3 +29,20 @@ docker build -t pulsar-build .
The image is already available at
https://hub.docker.com/r/apachepulsar/pulsar-build
Example: `apachepulsar/pulsar-build:ubuntu-16.04`
+
+## Build and Publish pulsar-build image
+
+> Only committers have permissions on publishing pulsar images to
`apachepulsar` docker hub.
+
+### Build pulsar-build image
+
+
+```shell
+docker build -t apachepulsar/pulsar-build:ubuntu-16.04 .
+```
+
+### Publish pulsar-build image
+
+```shell
+publish.sh
+```
diff --git a/build/docker/publish.sh b/build/docker/publish.sh
new file mode 100755
index 000..6bfa56b
--- /dev/null
+++ b/build/docker/publish.sh
@@ -0,0 +1,57 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+ROOT_DIR=$(git rev-parse --show-toplevel)
+cd $ROOT_DIR/build/docker
+
+if [ -z "$DOCKER_USER" ]; then
+echo "Docker user in variable \$DOCKER_USER was not set. Skipping image
publishing"
+exit 1
+fi
+
+if [ -z "$DOCKER_PASSWORD" ]; then
+echo "Docker password in variable \$DOCKER_PASSWORD was not set. Skipping
image publishing"
+exit 1
+fi
+
+DOCKER_ORG="${DOCKER_ORG:-apachepulsar}"
+
+docker login ${DOCKER_REGISTRY} -u="$DOCKER_USER" -p="$DOCKER_PASSWORD"
+if [ $? -ne 0 ]; then
+echo "Failed to loging to Docker Hub"
+exit 1
+fi
+
+if [[ -z ${DOCKER_REGISTRY} ]]; then
+docker_registry_org=${DOCKER_ORG}
+else
+docker_registry_org=${DOCKER_REGISTRY}/${DOCKER_ORG}
+echo "Starting to push images to ${docker_registry_org}..."
+fi
+
+set -x
+
+# Fail if any of the subsequent commands fail
+set -e
+
+# Push all images and tags
+docker push ${docker_registry_org}/pulsar-build:ubuntu-16.04
+
+echo "Finished pushing images to ${docker_registry_org}"
[GitHub] sijie closed pull request #2222: Add script and README for publishing `pulsar-build` image
sijie closed pull request #: Add script and README for publishing
`pulsar-build` image
URL: https://github.com/apache/incubator-pulsar/pull/
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build/docker/README.md b/build/docker/README.md
index 7a01d2cceb..bc2fe532f7 100644
--- a/build/docker/README.md
+++ b/build/docker/README.md
@@ -29,3 +29,20 @@ docker build -t pulsar-build .
The image is already available at
https://hub.docker.com/r/apachepulsar/pulsar-build
Example: `apachepulsar/pulsar-build:ubuntu-16.04`
+
+## Build and Publish pulsar-build image
+
+> Only committers have permissions on publishing pulsar images to
`apachepulsar` docker hub.
+
+### Build pulsar-build image
+
+
+```shell
+docker build -t apachepulsar/pulsar-build:ubuntu-16.04 .
+```
+
+### Publish pulsar-build image
+
+```shell
+publish.sh
+```
diff --git a/build/docker/publish.sh b/build/docker/publish.sh
new file mode 100755
index 00..6bfa56bace
--- /dev/null
+++ b/build/docker/publish.sh
@@ -0,0 +1,57 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+ROOT_DIR=$(git rev-parse --show-toplevel)
+cd $ROOT_DIR/build/docker
+
+if [ -z "$DOCKER_USER" ]; then
+echo "Docker user in variable \$DOCKER_USER was not set. Skipping image
publishing"
+exit 1
+fi
+
+if [ -z "$DOCKER_PASSWORD" ]; then
+echo "Docker password in variable \$DOCKER_PASSWORD was not set. Skipping
image publishing"
+exit 1
+fi
+
+DOCKER_ORG="${DOCKER_ORG:-apachepulsar}"
+
+docker login ${DOCKER_REGISTRY} -u="$DOCKER_USER" -p="$DOCKER_PASSWORD"
+if [ $? -ne 0 ]; then
+echo "Failed to loging to Docker Hub"
+exit 1
+fi
+
+if [[ -z ${DOCKER_REGISTRY} ]]; then
+docker_registry_org=${DOCKER_ORG}
+else
+docker_registry_org=${DOCKER_REGISTRY}/${DOCKER_ORG}
+echo "Starting to push images to ${docker_registry_org}..."
+fi
+
+set -x
+
+# Fail if any of the subsequent commands fail
+set -e
+
+# Push all images and tags
+docker push ${docker_registry_org}/pulsar-build:ubuntu-16.04
+
+echo "Finished pushing images to ${docker_registry_org}"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie closed pull request #2230: Make doc image links work when there is a base path.
sijie closed pull request #2230: Make doc image links work when there is a base
path.
URL: https://github.com/apache/incubator-pulsar/pull/2230
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/site2/docs/administration-geo.md b/site2/docs/administration-geo.md
index 08c7b3ee29..c50563968f 100644
--- a/site2/docs/administration-geo.md
+++ b/site2/docs/administration-geo.md
@@ -10,7 +10,7 @@ sidebar_label: Geo-replication
The diagram below illustrates the process of geo-replication across Pulsar
clusters:
-
+
In this diagram, whenever producers **P1**, **P2**, and **P3** publish
messages to the topic **T1** on clusters **Cluster-A**, **Cluster-B**, and
**Cluster-C**, respectively, those messages are instantly replicated across
clusters. Once replicated, consumers **C1** and **C2** can consume those
messages from their respective clusters.
diff --git a/site2/docs/administration-zk-bk.md
b/site2/docs/administration-zk-bk.md
index cf0eb3a079..79ee037a0f 100644
--- a/site2/docs/administration-zk-bk.md
+++ b/site2/docs/administration-zk-bk.md
@@ -329,6 +329,6 @@ PersistencePolicies policies =
admin.namespaces().getPersistence(namespace);
This diagram illustrates the role of ZooKeeper and BookKeeper in a Pulsar
cluster:
-
+
Each Pulsar cluster consists of one or more message brokers. Each broker
relies on an ensemble of bookies.
diff --git a/site2/docs/cookbooks-encryption.md
b/site2/docs/cookbooks-encryption.md
index 1bf22c..e11b1310bb 100644
--- a/site2/docs/cookbooks-encryption.md
+++ b/site2/docs/cookbooks-encryption.md
@@ -19,10 +19,10 @@ A message can be encrypted with more than one key. Any one
of the keys used for
Pulsar does not store the encryption key anywhere in the pulsar service. If
you lose/delete the private key, your message is irretrievably lost, and is
unrecoverable
## Producer
-
+
## Consumer
-
+
## Here are the steps to get started:
diff --git a/site2/docs/cookbooks-tiered-storage.md
b/site2/docs/cookbooks-tiered-storage.md
index 792592e4a2..003cd14a7e 100644
--- a/site2/docs/cookbooks-tiered-storage.md
+++ b/site2/docs/cookbooks-tiered-storage.md
@@ -14,7 +14,7 @@ Tiered storage should be used when you have a topic for which
you want to keep a
A topic in Pulsar is backed by a log, known as a managed ledger. This log is
composed of an ordered list of segments. Pulsar only every writes to the final
segment of the log. All previous segments are sealed. The data within the
segment is immutable. This is known as a segment oriented architecture.
-
+
{% include figure.html src="/img/pulsar-tiered-storage.png" alt="Tiered
Storage" width="80" %}
diff --git a/site2/docs/deploy-bare-metal.md b/site2/docs/deploy-bare-metal.md
index 6fb1f1d412..8b3497fc7b 100644
--- a/site2/docs/deploy-bare-metal.md
+++ b/site2/docs/deploy-bare-metal.md
@@ -26,7 +26,7 @@ Each machine in your cluster will need to have [Java
8](http://www.oracle.com/te
Here's a diagram showing the basic setup:
-
+
In this diagram, connecting clients need to be able to communicate with the
Pulsar cluster using a single URL, in this case `pulsar-cluster.acme.com`, that
abstracts over all of the message-handling brokers. Pulsar message brokers run
on machines alongside BookKeeper bookies; brokers and bookies, in turn, rely on
ZooKeeper.
diff --git a/site2/docs/deploy-dcos.md b/site2/docs/deploy-dcos.md
index 62dfd8150d..64b3d962b5 100644
--- a/site2/docs/deploy-dcos.md
+++ b/site2/docs/deploy-dcos.md
@@ -48,69 +48,69 @@ This command will deploy Docker container instances in
three groups, which toget
After executing the `dcos` command above, click on the **Services** tab in the
DC/OS [GUI interface](https://docs.mesosphere.com/latest/gui/), which you can
access at [http://m1.dcos](http://m1.dcos) in this example. You should see
several applications in the process of deploying.
-![DC/OS command
[GitHub] sijie commented on issue #2231: Add search bar to new site.
sijie commented on issue #2231: Add search bar to new site. URL: https://github.com/apache/incubator-pulsar/pull/2231#issuecomment-407899542 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: Fix TestS3Offload (#2227)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 9273c28 Fix TestS3Offload (#2227)
9273c28 is described below
commit 9273c282f4d29b16366262d1d8d96d875076792c
Author: Sijie Guo
AuthorDate: Wed Jul 25 14:13:38 2018 -0700
Fix TestS3Offload (#2227)
* Fix TestS3Offload
### Motivation
s3 mock service doesn't need credential. so we will fail to load
credentials on s3 integration tests.
that is expected. However there is a regression when introducing GCS
support in #2151:
https://github.com/apache/incubator-pulsar/pull/2151/files#diff-9160ffc9df7afcf42a8335f214393c6dR188
### Changes
* Don't throw exception when failed to load aws credentials
* change `log.error` to `log.warn`
---
.../pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java| 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
index 418c14f..fa0647c 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
@@ -184,8 +184,7 @@ public class BlobStoreManagedLedgerOffloader implements
LedgerOffloader {
credentials = creds.getCredentials();
} catch (Exception e) {
// allowed, some mock s3 service not need credential
-log.error("Exception when get credentials for s3 ", e);
-throw new PulsarServerException(e);
+log.warn("Exception when get credentials for s3 ", e);
}
String id = "accesskey";
[GitHub] sijie closed pull request #2227: Fix TestS3Offload
sijie closed pull request #2227: Fix TestS3Offload
URL: https://github.com/apache/incubator-pulsar/pull/2227
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
index 418c14f04a..fa0647c31e 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
@@ -184,8 +184,7 @@ public static Credentials getCredentials(String driver,
ServiceConfiguration con
credentials = creds.getCredentials();
} catch (Exception e) {
// allowed, some mock s3 service not need credential
-log.error("Exception when get credentials for s3 ", e);
-throw new PulsarServerException(e);
+log.warn("Exception when get credentials for s3 ", e);
}
String id = "accesskey";
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie commented on issue #2235: [website] Fix content inconsistency between site and site2
sijie commented on issue #2235: [website] Fix content inconsistency between site and site2 URL: https://github.com/apache/incubator-pulsar/pull/2235#issuecomment-407897009 @cckellogg ^^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie opened a new pull request #2235: [website] Fix content inconsistency between site and site2
sijie opened a new pull request #2235: [website] Fix content inconsistency between site and site2 URL: https://github.com/apache/incubator-pulsar/pull/2235 ### Motivation There were a few changes merged to site after site2 is merged. This PR is to address the content inconsistency between site and site2 ### Changes Fix content inconsistency This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jerrypeng opened a new pull request #2234: add stats entry for active consumer name and endpoint to get leader of functions cluster
jerrypeng opened a new pull request #2234: add stats entry for active consumer name and endpoint to get leader of functions cluster URL: https://github.com/apache/incubator-pulsar/pull/2234 ### Motivation Currently there is no way to determine the active consumer of a topic (failover or exclusive). It would be nice for debugging purposes to be able to see which consumer is the active consumer. Also in a functions cluster, it would also be nice to now which worker is the leader ### Modifications Added an entry in subscriptions stats for the active consumer name. Added an endpoint that returns the leader of the functions cluster This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: Set empty response to complete grpc txn (#2225)
This is an automated email from the ASF dual-hosted git repository.
rdhabalia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 55c8dc0 Set empty response to complete grpc txn (#2225)
55c8dc0 is described below
commit 55c8dc0519608a755509cdc445496140b38d98b3
Author: Rajan Dhabalia
AuthorDate: Wed Jul 25 13:07:52 2018 -0700
Set empty response to complete grpc txn (#2225)
---
.../java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
b/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
index 0c10fdb..966740d 100644
---
a/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
+++
b/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
@@ -370,9 +370,10 @@ public class JavaInstanceMain implements AutoCloseable {
if (runtime != null) {
try {
runtime.resetMetrics().get();
+
responseObserver.onNext(com.google.protobuf.Empty.getDefaultInstance());
responseObserver.onCompleted();
} catch (InterruptedException | ExecutionException e) {
-log.error("Exception in JavaInstance doing
getAndResetMetrics", e);
+log.error("Exception in JavaInstance doing resetMetrics",
e);
throw new RuntimeException(e);
}
}
[GitHub] rdhabalia commented on issue #2225: Set empty response to complete grpc txn
rdhabalia commented on issue #2225: Set empty response to complete grpc txn URL: https://github.com/apache/incubator-pulsar/pull/2225#issuecomment-407878846 rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia closed pull request #2225: Set empty response to complete grpc txn
rdhabalia closed pull request #2225: Set empty response to complete grpc txn
URL: https://github.com/apache/incubator-pulsar/pull/2225
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
b/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
index 0c10fdbcb8..966740da98 100644
---
a/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
+++
b/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceMain.java
@@ -370,9 +370,10 @@ public void resetMetrics(com.google.protobuf.Empty request,
if (runtime != null) {
try {
runtime.resetMetrics().get();
+
responseObserver.onNext(com.google.protobuf.Empty.getDefaultInstance());
responseObserver.onCompleted();
} catch (InterruptedException | ExecutionException e) {
-log.error("Exception in JavaInstance doing
getAndResetMetrics", e);
+log.error("Exception in JavaInstance doing resetMetrics",
e);
throw new RuntimeException(e);
}
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie commented on issue #2233: [integration tests] Return exit code as part of execution result of running a command in testcontainers
sijie commented on issue #2233: [integration tests] Return exit code as part of execution result of running a command in testcontainers URL: https://github.com/apache/incubator-pulsar/pull/2233#issuecomment-407874899 for reviewers: this PR is based on #2227 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie opened a new pull request #2233: [integration tests] Return exit code as part of execution result of running a command in testcontainers
sijie opened a new pull request #2233: [integration tests] Return exit code as part of execution result of running a command in testcontainers URL: https://github.com/apache/incubator-pulsar/pull/2233 ### Motivation The default `ExecResult` in testcontainers doesn't include the exit code of running a command in a test container. ### Changes - Add a class `ContainerExecResult` to represent the result of executing a command in a container, including exitCode, stdout and stderr - Improve the existing util method in `DockerUtils` to return `ContainerExecResult` - Improve all the integration tests on validating exit codes before validating outputs This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] aahmed-se opened a new pull request #2232: Fix Spelling Mistake in class
aahmed-se opened a new pull request #2232: Fix Spelling Mistake in class URL: https://github.com/apache/incubator-pulsar/pull/2232 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: More detailed documentation to config file for TLS (#2228)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new ebb2822 More detailed documentation to config file for TLS (#2228) ebb2822 is described below commit ebb2822e611009662dca7b4a3ed1309f039a7f14 Author: Ivan Kelly AuthorDate: Wed Jul 25 20:42:09 2018 +0100 More detailed documentation to config file for TLS (#2228) This patch adds more detail for TLS options in the broker, proxy, and client configuration files which were previously ambiguous. It also adds some options which were missing, and suggests some defaults in the client configuration. --- conf/broker.conf | 15 --- conf/client.conf | 43 ++- conf/proxy.conf | 12 3 files changed, 62 insertions(+), 8 deletions(-) diff --git a/conf/broker.conf b/conf/broker.conf index 11a87d5..a639b85 100644 --- a/conf/broker.conf +++ b/conf/broker.conf @@ -211,15 +211,24 @@ tlsCertificateFilePath= # Path for the TLS private key file tlsKeyFilePath= -# Path for the trusted TLS certificate file +# Path for the trusted TLS certificate file. +# This cert is used to verify that any certs presented by connecting clients +# are signed by a certificate authority. If this verification +# fails, then the certs are untrusted and the connections are dropped. tlsTrustCertsFilePath= -# Accept untrusted TLS certificate from client +# Accept untrusted TLS certificate from client. +# If true, a client with a cert which cannot be verified with the +# 'tlsTrustCertsFilePath' cert will allowed to connect to the server, +# though the cert will not be used for client authentication. tlsAllowInsecureConnection=false -# Specify whether Client certificates are required for TLS +# Trusted client certificates are required for to connect TLS # Reject the Connection if the Client Certificate is not trusted. +# In effect, this requires that all connecting clients perform TLS client +# authentication. tlsRequireTrustedClientCertOnConnect=false + ### --- Authentication --- ### # Enable authentication diff --git a/conf/client.conf b/conf/client.conf index f9e0bc3..93c99e5 100644 --- a/conf/client.conf +++ b/conf/client.conf @@ -17,12 +17,45 @@ # under the License. # -# Pulsar Client and pulsar-admin configuration +# Configuration for pulsar-client and pulsar-admin CLI tools + +# URL for Pulsar REST API (for admin operations) +# For TLS: +# webServiceUrl=https://localhost:8443/ webServiceUrl=http://localhost:8080/ + +# URL for Pulsar Binary Protocol (for produce and consume operations) +# For TLS: +# brokerServiceUrl=pulsar+ssl://localhost:6651/ brokerServiceUrl=pulsar://localhost:6650/ -#authPlugin= -#authParams= -#useTls= + +# Authentication plugin to authenticate with servers +# e.g. for TLS +# authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls +authPlugin= + +# Parameters passed to authentication plugin. +# A comma separated list of key:value pairs. +# Keys depend on the configured authPlugin. +# e.g. for TLS +# authParams=tlsCertFile:/path/to/client-cert.pem,tlsKeyFile:/path/to/client-key.pem +authParams= + +# Whether to use TLS. Defaults to false. +useTls= + +# Allow TLS connections to servers whose certificate cannot be +# be verified to have been signed by a trusted certificate +# authority. tlsAllowInsecureConnection=false + +# Whether server hostname must match the common name of the certificate +# the server is using. tlsEnableHostnameVerification=false -#tlsTrustCertsFilePath + +# Path for the trusted TLS certificate file. +# This cert is used to verify that any cert presented by a server +# is signed by a certificate authority. If this verification +# fails, then the cert is untrusted and the connection is dropped. +tlsTrustCertsFilePath= + diff --git a/conf/proxy.conf b/conf/proxy.conf index 3bde301..7b7e690 100644 --- a/conf/proxy.conf +++ b/conf/proxy.conf @@ -97,6 +97,18 @@ tlsCertificateFilePath= # Path for the TLS private key file tlsKeyFilePath= +# Path for the trusted TLS certificate file. +# This cert is used to verify that any certs presented by connecting clients +# are signed by a certificate authority. If this verification +# fails, then the certs are untrusted and the connections are dropped. +tlsTrustCertsFilePath= + +# Accept untrusted TLS certificate from client. +# If true, a client with a cert which cannot be verified with the +# 'tlsTrustCertsFilePath' cert will allowed to connect to the server, +# though the cert will not be used for client authentication. +tlsAllowInsecureConnection=false + # Whether the hostname is validated when the proxy creates a TLS connection with brokers tlsHostnameVerificationEnabled=false
[GitHub] sijie closed pull request #2228: More detailed documentation to config file for TLS
sijie closed pull request #2228: More detailed documentation to config file for TLS URL: https://github.com/apache/incubator-pulsar/pull/2228 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/conf/broker.conf b/conf/broker.conf index 2e40d82a87..76b525b2aa 100644 --- a/conf/broker.conf +++ b/conf/broker.conf @@ -211,15 +211,24 @@ tlsCertificateFilePath= # Path for the TLS private key file tlsKeyFilePath= -# Path for the trusted TLS certificate file +# Path for the trusted TLS certificate file. +# This cert is used to verify that any certs presented by connecting clients +# are signed by a certificate authority. If this verification +# fails, then the certs are untrusted and the connections are dropped. tlsTrustCertsFilePath= -# Accept untrusted TLS certificate from client +# Accept untrusted TLS certificate from client. +# If true, a client with a cert which cannot be verified with the +# 'tlsTrustCertsFilePath' cert will allowed to connect to the server, +# though the cert will not be used for client authentication. tlsAllowInsecureConnection=false -# Specify whether Client certificates are required for TLS +# Trusted client certificates are required for to connect TLS # Reject the Connection if the Client Certificate is not trusted. +# In effect, this requires that all connecting clients perform TLS client +# authentication. tlsRequireTrustedClientCertOnConnect=false + ### --- Authentication --- ### # Enable authentication diff --git a/conf/client.conf b/conf/client.conf index f9e0bc303e..93c99e5984 100644 --- a/conf/client.conf +++ b/conf/client.conf @@ -17,12 +17,45 @@ # under the License. # -# Pulsar Client and pulsar-admin configuration +# Configuration for pulsar-client and pulsar-admin CLI tools + +# URL for Pulsar REST API (for admin operations) +# For TLS: +# webServiceUrl=https://localhost:8443/ webServiceUrl=http://localhost:8080/ + +# URL for Pulsar Binary Protocol (for produce and consume operations) +# For TLS: +# brokerServiceUrl=pulsar+ssl://localhost:6651/ brokerServiceUrl=pulsar://localhost:6650/ -#authPlugin= -#authParams= -#useTls= + +# Authentication plugin to authenticate with servers +# e.g. for TLS +# authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls +authPlugin= + +# Parameters passed to authentication plugin. +# A comma separated list of key:value pairs. +# Keys depend on the configured authPlugin. +# e.g. for TLS +# authParams=tlsCertFile:/path/to/client-cert.pem,tlsKeyFile:/path/to/client-key.pem +authParams= + +# Whether to use TLS. Defaults to false. +useTls= + +# Allow TLS connections to servers whose certificate cannot be +# be verified to have been signed by a trusted certificate +# authority. tlsAllowInsecureConnection=false + +# Whether server hostname must match the common name of the certificate +# the server is using. tlsEnableHostnameVerification=false -#tlsTrustCertsFilePath + +# Path for the trusted TLS certificate file. +# This cert is used to verify that any cert presented by a server +# is signed by a certificate authority. If this verification +# fails, then the cert is untrusted and the connection is dropped. +tlsTrustCertsFilePath= + diff --git a/conf/proxy.conf b/conf/proxy.conf index 3bde30118e..7b7e690478 100644 --- a/conf/proxy.conf +++ b/conf/proxy.conf @@ -97,6 +97,18 @@ tlsCertificateFilePath= # Path for the TLS private key file tlsKeyFilePath= +# Path for the trusted TLS certificate file. +# This cert is used to verify that any certs presented by connecting clients +# are signed by a certificate authority. If this verification +# fails, then the certs are untrusted and the connections are dropped. +tlsTrustCertsFilePath= + +# Accept untrusted TLS certificate from client. +# If true, a client with a cert which cannot be verified with the +# 'tlsTrustCertsFilePath' cert will allowed to connect to the server, +# though the cert will not be used for client authentication. +tlsAllowInsecureConnection=false + # Whether the hostname is validated when the proxy creates a TLS connection with brokers tlsHostnameVerificationEnabled=false This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie closed pull request #2229: C++ should default not allow insecure connections with TLS
sijie closed pull request #2229: C++ should default not allow insecure
connections with TLS
URL: https://github.com/apache/incubator-pulsar/pull/2229
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/pulsar-client-cpp/lib/ClientConfigurationImpl.h
b/pulsar-client-cpp/lib/ClientConfigurationImpl.h
index 7160a6592b..cb23a14088 100644
--- a/pulsar-client-cpp/lib/ClientConfigurationImpl.h
+++ b/pulsar-client-cpp/lib/ClientConfigurationImpl.h
@@ -44,7 +44,7 @@ struct ClientConfigurationImpl {
concurrentLookupRequest(5),
logConfFilePath(),
useTls(false),
- tlsAllowInsecureConnection(true),
+ tlsAllowInsecureConnection(false),
statsIntervalInSeconds(600), // 10 minutes
loggerFactory() {}
};
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-pulsar] branch master updated: C++ should default not allow insecure connections with TLS (#2229)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 4e217dd C++ should default not allow insecure connections with TLS
(#2229)
4e217dd is described below
commit 4e217dde805014da49ba147889e1e7ee224f7dd0
Author: Ivan Kelly
AuthorDate: Wed Jul 25 20:40:34 2018 +0100
C++ should default not allow insecure connections with TLS (#2229)
If tlsAllowInsecureConnection is true, then the client doesn't
validate the server's TLS cert, allowing for MITM attacks. This
shouldn't be the default option.
---
pulsar-client-cpp/lib/ClientConfigurationImpl.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pulsar-client-cpp/lib/ClientConfigurationImpl.h
b/pulsar-client-cpp/lib/ClientConfigurationImpl.h
index 7160a65..cb23a14 100644
--- a/pulsar-client-cpp/lib/ClientConfigurationImpl.h
+++ b/pulsar-client-cpp/lib/ClientConfigurationImpl.h
@@ -44,7 +44,7 @@ struct ClientConfigurationImpl {
concurrentLookupRequest(5),
logConfFilePath(),
useTls(false),
- tlsAllowInsecureConnection(true),
+ tlsAllowInsecureConnection(false),
statsIntervalInSeconds(600), // 10 minutes
loggerFactory() {}
};
[GitHub] sijie commented on issue #2227: Fix TestS3Offload
sijie commented on issue #2227: Fix TestS3Offload URL: https://github.com/apache/incubator-pulsar/pull/2227#issuecomment-407870742 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2230: Make doc image links work when there is a base path.
sijie commented on issue #2230: Make doc image links work when there is a base path. URL: https://github.com/apache/incubator-pulsar/pull/2230#issuecomment-407870272 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cckellogg opened a new pull request #2231: Add search bar to new site.
cckellogg opened a new pull request #2231: Add search bar to new site. URL: https://github.com/apache/incubator-pulsar/pull/2231 Add doc search to site version 2. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2227: Fix TestS3Offload
sijie commented on issue #2227: Fix TestS3Offload URL: https://github.com/apache/incubator-pulsar/pull/2227#issuecomment-407863353 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2230: Make doc image links work when there is a base path.
sijie commented on issue #2230: Make doc image links work when there is a base path. URL: https://github.com/apache/incubator-pulsar/pull/2230#issuecomment-407854865 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: fix bug in FunctionRuntimeManager involving not cleaning up old invalid assignments (#2223)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 4d631c0 fix bug in FunctionRuntimeManager involving not cleaning up
old invalid assignments (#2223)
4d631c0 is described below
commit 4d631c0608fdeb605dd39822e2abc2cc57c7b37b
Author: Boyang Jerry Peng
AuthorDate: Wed Jul 25 11:37:48 2018 -0700
fix bug in FunctionRuntimeManager involving not cleaning up old invalid
assignments (#2223)
### Motivation
Old invalid assignments of functions are not cleaned up and left over after
a function is rescheduled.
Getting the status of functions rely on assignments to route the get status
to the correct worker running the function. If the assignments are not
correct, get status requests will be routed incorrectly.
Also fixed a test that was asserting the incorrect value. The test would
have caught this problem if it was asserting the correct value.
---
.../pulsar/functions/worker/FunctionRuntimeManager.java | 12 ++--
.../pulsar/functions/worker/FunctionRuntimeManagerTest.java | 4 +---
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
index 05a79d8..5e1995e 100644
---
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
+++
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
@@ -82,7 +82,6 @@ public class FunctionRuntimeManager implements AutoCloseable{
private MembershipManager membershipManager;
private final ConnectorsManager connectorsManager;
-
public FunctionRuntimeManager(WorkerConfig workerConfig,
PulsarClient pulsarClient,
Namespace dlogNamespace,
@@ -354,6 +353,7 @@ public class FunctionRuntimeManager implements
AutoCloseable{
* @param assignmentsUpdate the assignment update
*/
public synchronized void processAssignmentUpdate(MessageId messageId,
AssignmentsUpdate assignmentsUpdate) {
+
if (assignmentsUpdate.getVersion() > this.currentAssignmentVersion) {
Map assignmentMap = new HashMap<>();
@@ -430,8 +430,16 @@ public class FunctionRuntimeManager implements
AutoCloseable{
newFunctionRuntimeInfo.setFunctionInstance(assignment.getInstance());
this.insertStartAction(newFunctionRuntimeInfo);
this.setFunctionRuntimeInfo(fullyQualifiedInstanceId,
newFunctionRuntimeInfo);
-this.setAssignment(assignment);
}
+
+// find existing assignment
+Assignment existing_assignment =
this.findAssignment(assignment);
+if (existing_assignment != null) {
+// delete old assignment that could have old data
+this.deleteAssignment(existing_assignment);
+}
+// set to newest assignment
+this.setAssignment(assignment);
}
}
diff --git
a/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
b/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
index 8ab7473..4f618c4 100644
---
a/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
+++
b/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
@@ -378,12 +378,10 @@ public class FunctionRuntimeManagerTest {
.build();
Assert.assertEquals(functionRuntimeManager.functionRuntimeInfoMap.size(), 2);
-
-
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments.size(), 2);
+
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments.size(), 1);
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments
.get("worker-1").get("test-tenant/test-namespace/func-1:0"),
assignment1);
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments
.get("worker-1").get("test-tenant/test-namespace/func-2:0"),
assignment3);
}
-
}
[GitHub] sijie closed pull request #2223: fix bug in FunctionRuntimeManager involving not cleaning up old invalid assignments
sijie closed pull request #2223: fix bug in FunctionRuntimeManager involving
not cleaning up old invalid assignments
URL: https://github.com/apache/incubator-pulsar/pull/2223
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
index 05a79d8da5..5e1995e633 100644
---
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
+++
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionRuntimeManager.java
@@ -82,7 +82,6 @@
private MembershipManager membershipManager;
private final ConnectorsManager connectorsManager;
-
public FunctionRuntimeManager(WorkerConfig workerConfig,
PulsarClient pulsarClient,
Namespace dlogNamespace,
@@ -354,6 +353,7 @@ public synchronized void
removeAssignments(Collection assignments) {
* @param assignmentsUpdate the assignment update
*/
public synchronized void processAssignmentUpdate(MessageId messageId,
AssignmentsUpdate assignmentsUpdate) {
+
if (assignmentsUpdate.getVersion() > this.currentAssignmentVersion) {
Map assignmentMap = new HashMap<>();
@@ -430,8 +430,16 @@ public synchronized void processAssignmentUpdate(MessageId
messageId, Assignment
newFunctionRuntimeInfo.setFunctionInstance(assignment.getInstance());
this.insertStartAction(newFunctionRuntimeInfo);
this.setFunctionRuntimeInfo(fullyQualifiedInstanceId,
newFunctionRuntimeInfo);
-this.setAssignment(assignment);
}
+
+// find existing assignment
+Assignment existing_assignment =
this.findAssignment(assignment);
+if (existing_assignment != null) {
+// delete old assignment that could have old data
+this.deleteAssignment(existing_assignment);
+}
+// set to newest assignment
+this.setAssignment(assignment);
}
}
diff --git
a/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
b/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
index 8ab7473eed..4f618c41b4 100644
---
a/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
+++
b/pulsar-functions/worker/src/test/java/org/apache/pulsar/functions/worker/FunctionRuntimeManagerTest.java
@@ -378,12 +378,10 @@ public boolean matches(Object o) {
.build();
Assert.assertEquals(functionRuntimeManager.functionRuntimeInfoMap.size(), 2);
-
-
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments.size(), 2);
+
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments.size(), 1);
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments
.get("worker-1").get("test-tenant/test-namespace/func-1:0"),
assignment1);
Assert.assertEquals(functionRuntimeManager.workerIdToAssignments
.get("worker-1").get("test-tenant/test-namespace/func-2:0"),
assignment3);
}
-
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie commented on issue #2222: Add script and README for publishing `pulsar-build` image
sijie commented on issue #: Add script and README for publishing `pulsar-build` image URL: https://github.com/apache/incubator-pulsar/pull/#issuecomment-407853554 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2225: Set empty response to complete grpc txn
sijie commented on issue #2225: Set empty response to complete grpc txn URL: https://github.com/apache/incubator-pulsar/pull/2225#issuecomment-407853728 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2199: Cleanup unnecessary casting and creating variables
sijie commented on issue #2199: Cleanup unnecessary casting and creating variables URL: https://github.com/apache/incubator-pulsar/pull/2199#issuecomment-407853295 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: Pull docker images before running integration tests (#2212)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new be1eaac Pull docker images before running integration tests (#2212) be1eaac is described below commit be1eaac9f4bd7a8c8804d77bd1935411f70955ab Author: Sijie Guo AuthorDate: Wed Jul 25 11:32:54 2018 -0700 Pull docker images before running integration tests (#2212) *Motivation* Integration tests sometime failed with "No such image:". *Changes* This change attempts to pull docker images before running integration tests. Hopefully it can improve the stability on running integration tests. --- tests/scripts/pre-integ-tests.sh | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/scripts/pre-integ-tests.sh b/tests/scripts/pre-integ-tests.sh index e5dec32..ab83c53 100755 --- a/tests/scripts/pre-integ-tests.sh +++ b/tests/scripts/pre-integ-tests.sh @@ -26,4 +26,7 @@ df -h ps -eo euser,pid,ppid,pgid,start,pcpu,pmem,cmd docker network prune -f --filter name=pulsarnet_* docker system events > docker.debug-info & echo $! > docker-log.pid - +docker pull apachepulsar/s3mock:latest +docker pull alpine/socat:latest +docker pull cassandra:3 +docker pull confluentinc/cp-kafka:4.0.0
[GitHub] sijie closed pull request #2212: Pull docker images before running integration tests
sijie closed pull request #2212: Pull docker images before running integration tests URL: https://github.com/apache/incubator-pulsar/pull/2212 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tests/scripts/pre-integ-tests.sh b/tests/scripts/pre-integ-tests.sh index e5dec320d2..ab83c53b84 100755 --- a/tests/scripts/pre-integ-tests.sh +++ b/tests/scripts/pre-integ-tests.sh @@ -26,4 +26,7 @@ df -h ps -eo euser,pid,ppid,pgid,start,pcpu,pmem,cmd docker network prune -f --filter name=pulsarnet_* docker system events > docker.debug-info & echo $! > docker-log.pid - +docker pull apachepulsar/s3mock:latest +docker pull alpine/socat:latest +docker pull cassandra:3 +docker pull confluentinc/cp-kafka:4.0.0 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie closed pull request #2192: Rename Netty native libs to reflect shading prefix
sijie closed pull request #2192: Rename Netty native libs to reflect shading
prefix
URL: https://github.com/apache/incubator-pulsar/pull/2192
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/pulsar-broker-shaded/pom.xml b/pulsar-broker-shaded/pom.xml
index 7b3b9669da..1adc84f0f7 100644
--- a/pulsar-broker-shaded/pom.xml
+++ b/pulsar-broker-shaded/pom.xml
@@ -345,6 +345,30 @@
+
+
+exec-maven-plugin
+org.codehaus.mojo
+
+
+rename-epoll-library
+package
+
+ exec
+
+
+
${project.parent.basedir}/src/rename-netty-native-libs.sh
+
+${project.artifactId}
+
+
+
+
+
diff --git a/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
b/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
index 3ba94e9aaf..397ec475a7 100644
--- a/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
+++ b/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
@@ -223,6 +223,30 @@
+
+
+exec-maven-plugin
+org.codehaus.mojo
+
+
+rename-epoll-library
+package
+
+ exec
+
+
+
${project.parent.basedir}/../src/rename-netty-native-libs.sh
+
+${project.artifactId}
+
+
+
+
+
diff --git a/pulsar-client-shaded/pom.xml b/pulsar-client-shaded/pom.xml
index 8e7eb98ee3..180e7f6693 100644
--- a/pulsar-client-shaded/pom.xml
+++ b/pulsar-client-shaded/pom.xml
@@ -207,6 +207,31 @@
+
+
+
+exec-maven-plugin
+org.codehaus.mojo
+
+
+rename-epoll-library
+package
+
+ exec
+
+
+
${project.parent.basedir}/src/rename-netty-native-libs.sh
+
+${project.artifactId}
+
+
+
+
+
diff --git a/src/rename-netty-native-libs.sh b/src/rename-netty-native-libs.sh
new file mode 100755
index 00..136f82156f
--- /dev/null
+++ b/src/rename-netty-native-libs.sh
@@ -0,0 +1,54 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -e
+
+ARTIFACT_ID=$1
+JAR_PATH="$PWD/target/$ARTIFACT_ID.jar"
+
+FILE_PREFIX='META-INF/native'
+
+FILES_TO_RENAME=(
+'libnetty_transport_native_epoll_x86_64.so
liborg_apache_pulsar_shade_netty_transport_native_epoll_x86_64.so'
+'libnetty_tcnative_linux_x86_64.so
liborg_apache_pulsar_shade_netty_tcnative_linux_x86_64.so'
+)
+
+echo "- Renaming epoll lib in $JAR_PATH --"
+TMP_DIR=`mktemp -d`
+unzip -q $JAR_PATH -d $TMP_DIR
+
+pushd $TMP_DIR
+
+for line in "${FILES_TO_RENAME[@]}"; do
+read -r -a A <<< "$line"
+FROM=${A[0]}
+TO=${A[1]}
+
+if [ -f $FILE_PREFIX/$FROM ]; then
+echo "Renaming $FROM -> $TO"
+mv $FILE_PREFIX/$FROM $FILE_PREFIX/$TO
+fi
+done
+
+# Overwrite the original ZIP archive
+zip -q -r $JAR_PATH .
+popd
+
+rm -rf $TMP_DIR
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-pulsar] branch master updated: Rename Netty native libs to reflect shading prefix (#2192)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 04e38c7 Rename Netty native libs to reflect shading prefix (#2192)
04e38c7 is described below
commit 04e38c7e66a9712ca35cfccc81e5a0041e640e5f
Author: Matteo Merli
AuthorDate: Wed Jul 25 11:32:28 2018 -0700
Rename Netty native libs to reflect shading prefix (#2192)
### Motivation
When using shaded artifacts, the Netty epoll and OpenSSL libraries are not
being loaded correctly and it falls back to Java based implementations.
The problem is that when the Java classes are shaded, Netty expects the
native library also to be prefixed with the same name.
Since there is no easy way to do the renaming in Maven itself, I've added
here a bash script that gets executed before the install phase.
---
pulsar-broker-shaded/pom.xml | 24 ++
.../pulsar-client-kafka-shaded/pom.xml | 24 ++
pulsar-client-shaded/pom.xml | 25 ++
src/rename-netty-native-libs.sh| 54 ++
4 files changed, 127 insertions(+)
diff --git a/pulsar-broker-shaded/pom.xml b/pulsar-broker-shaded/pom.xml
index 7b3b966..1adc84f 100644
--- a/pulsar-broker-shaded/pom.xml
+++ b/pulsar-broker-shaded/pom.xml
@@ -345,6 +345,30 @@
+
+
+exec-maven-plugin
+org.codehaus.mojo
+
+
+rename-epoll-library
+package
+
+ exec
+
+
+
${project.parent.basedir}/src/rename-netty-native-libs.sh
+
+${project.artifactId}
+
+
+
+
+
diff --git a/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
b/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
index 3ba94e9..397ec47 100644
--- a/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
+++ b/pulsar-client-kafka-compat/pulsar-client-kafka-shaded/pom.xml
@@ -223,6 +223,30 @@
+
+
+exec-maven-plugin
+org.codehaus.mojo
+
+
+rename-epoll-library
+package
+
+ exec
+
+
+
${project.parent.basedir}/../src/rename-netty-native-libs.sh
+
+${project.artifactId}
+
+
+
+
+
diff --git a/pulsar-client-shaded/pom.xml b/pulsar-client-shaded/pom.xml
index 8e7eb98..180e7f6 100644
--- a/pulsar-client-shaded/pom.xml
+++ b/pulsar-client-shaded/pom.xml
@@ -207,6 +207,31 @@
+
+
+
+exec-maven-plugin
+org.codehaus.mojo
+
+
+rename-epoll-library
+package
+
+ exec
+
+
+
${project.parent.basedir}/src/rename-netty-native-libs.sh
+
+${project.artifactId}
+
+
+
+
+
diff --git a/src/rename-netty-native-libs.sh b/src/rename-netty-native-libs.sh
new file mode 100755
index 000..136f821
--- /dev/null
+++ b/src/rename-netty-native-libs.sh
@@ -0,0 +1,54 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -e
+
+ARTIFACT_ID=$1
+JAR_PATH="$PWD/target/$ARTIFACT_ID.jar"
+
+FILE_PREFIX='META-INF/native'
+
+FILES_TO_RENAME=(
+'libnetty_transport_native_epoll_x86_64.so
liborg_apache_pulsar_shade_netty_transport_native_epoll_x86_64.so'
+'libnetty_tcnative_linux_x86_64.so
liborg_apache_pulsar_shade_netty_tcnative_linux_x86_64.so'
+)
+
+echo "- Renaming epoll lib in $JAR_PATH --"
+TMP_DIR=`mktemp -d`
+unzip -q $JAR_PATH -d $TMP_DIR
+
+pushd $TMP_DIR
+
+for line in "${FILES_TO_RENAME[@]}"; do
+read -r -a A <<< "$line"
+FROM=${A[0]}
+TO=${A[1]}
+
+if [ -f $FILE_PREFIX/$FROM ]; then
[GitHub] sijie closed pull request #2189: dumpContainerDirToTargetCompressed should copy files to a containerName named directory
sijie closed pull request #2189: dumpContainerDirToTargetCompressed should copy
files to a containerName named directory
URL: https://github.com/apache/incubator-pulsar/pull/2189
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
index d3d7ba251d..51fb8d8790 100644
---
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
+++
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
@@ -113,7 +113,7 @@ public static void
dumpContainerDirToTargetCompressed(DockerClient dockerClient,
// docker api returns names prefixed with "/", it's part of it's
legacy design,
// this removes it to be consistent with what docker ps shows.
final String containerName =
inspectContainerResponse.getName().replace("/","");
-File output = new File(getTargetDirectory(containerId),
+File output = new File(getTargetDirectory(containerName),
(path.replace("/", "-") +
".tar.gz").replaceAll("^-", ""));
try (InputStream dockerStream =
dockerClient.copyArchiveFromContainerCmd(containerId, path).exec();
OutputStream os = new GZIPOutputStream(new
FileOutputStream(output))) {
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[incubator-pulsar] branch master updated: dumpContainerDirToTargetCompressed should copy files to a containerName named directory (#2189)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new d57f400 dumpContainerDirToTargetCompressed should copy files to a
containerName named directory (#2189)
d57f400 is described below
commit d57f400121e5d0a232092b67ad7b96f734d9d6e7
Author: Sijie Guo
AuthorDate: Wed Jul 25 11:31:28 2018 -0700
dumpContainerDirToTargetCompressed should copy files to a containerName
named directory (#2189)
*Motivation*
dumpContainerLogToTarget copies the files to a containerName named
directory.
However dumpContainerDirToTargetCompressed doesn't.
*Changes*
Change `dumpContainerDirToTargetCompressed` to copy the tarball to a
containerName named directory.
---
.../java/org/apache/pulsar/tests/integration/utils/DockerUtils.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
index d3d7ba2..51fb8d8 100644
---
a/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
+++
b/tests/integration/src/test/java/org/apache/pulsar/tests/integration/utils/DockerUtils.java
@@ -113,7 +113,7 @@ public class DockerUtils {
// docker api returns names prefixed with "/", it's part of it's
legacy design,
// this removes it to be consistent with what docker ps shows.
final String containerName =
inspectContainerResponse.getName().replace("/","");
-File output = new File(getTargetDirectory(containerId),
+File output = new File(getTargetDirectory(containerName),
(path.replace("/", "-") +
".tar.gz").replaceAll("^-", ""));
try (InputStream dockerStream =
dockerClient.copyArchiveFromContainerCmd(containerId, path).exec();
OutputStream os = new GZIPOutputStream(new
FileOutputStream(output))) {
[GitHub] jerrypeng commented on issue #2199: Cleanup unnecessary casting and creating variables
jerrypeng commented on issue #2199: Cleanup unnecessary casting and creating variables URL: https://github.com/apache/incubator-pulsar/pull/2199#issuecomment-407845448 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jerrypeng commented on issue #2225: Set empty response to complete grpc txn
jerrypeng commented on issue #2225: Set empty response to complete grpc txn URL: https://github.com/apache/incubator-pulsar/pull/2225#issuecomment-407842062 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jerrypeng removed a comment on issue #2225: Set empty response to complete grpc txn
jerrypeng removed a comment on issue #2225: Set empty response to complete grpc txn URL: https://github.com/apache/incubator-pulsar/pull/2225#issuecomment-407841960 rerun tests please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jerrypeng commented on issue #2225: Set empty response to complete grpc txn
jerrypeng commented on issue #2225: Set empty response to complete grpc txn URL: https://github.com/apache/incubator-pulsar/pull/2225#issuecomment-407841960 rerun tests please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
svn commit: r28350 - /dev/incubator/pulsar/KEYS
Author: sijie Date: Wed Jul 25 17:54:28 2018 New Revision: 28350 Log: add Sijie's KEY to pulsar KEYS file Modified: dev/incubator/pulsar/KEYS Modified: dev/incubator/pulsar/KEYS == --- dev/incubator/pulsar/KEYS (original) +++ dev/incubator/pulsar/KEYS Wed Jul 25 17:54:28 2018 @@ -439,3 +439,61 @@ SOSVDXIqtOATMZyU6FYxbzHubC//P74ABTvSkast 74x7h/7mjFNKlg== =A5zg -END PGP PUBLIC KEY BLOCK- +pub rsa4096 2017-03-28 [SC] [expires: 2021-03-28] + DA4DE82038672568BC834EE05B3FBDB4FD74402C +uid [ unknown] Sijie Guo +sig 35B3FBDB4FD74402C 2017-03-28 Sijie Guo +sub rsa4096 2017-03-28 [E] [expires: 2021-03-28] +sig 5B3FBDB4FD74402C 2017-03-28 Sijie Guo + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQINBFjaDcQBEAClZS2iJD/MpTx2j1SydCCEWa9lp6z2wHc8b962EFt2Pe8HkWbt +KGKgST9cOOR/fpgsoSNNsNalf6JoCIfimBZsK9L5iULqTgi34TMO/jvp38xEzACO +1BPav15HsAc5/yFUqoeKnAOkbbrWoXMhuPlbsdO2jsjPmlRAM0ocu3o2+Mv5lUPO +P3XLLzQTXEip6sKGl3T90fOrWMJneyTjO6rc5Vqg1V4hbc6XJWXmmrGrJk/tVB8Q +b4AbRiptelqROyqQqG6gaHK1KVGs65C2lYdur9KhsRT+z3NtIo8y9R+yTzwPkcgO +rWfeg0x+M6uZNPNk8wLrAlgUVJtMK2YfN1qlQ0uhj1TRHocy5/eNmMlLzVyH7xTK +C6oTi4zhCWIc02WCn3yVJ5IfzrrY7ACIbl3fKGTm9fTLypFFKt3SwQos3LHaDAsc +fgrx4ePwe1Ys74xlxz7ML7DxnurC8fqIA8UBmyeHj2wUOJrKBZLpfBsCrdRn4yN3 +1xcUwFN9w8r3Mdx3gHSXz+CN3Ng2LitkXUAArzwXsr0VHMwO5Asdc7rPLJYP9ymP +CUGReKrKA+RwMX+CyauSc7VobLYAT9kyJWMEuzGljbqn9/SRf8YP75Rpij4Lo7Zp +U/mS+xWVTpuW2RFeWjBaXTEY9+yFSQVqTw9TmED2TPuYDH1471RkTZxUvQARAQAB +tBxTaWppZSBHdW8gPHNpamllQGFwYWNoZS5vcmc+iQI9BBMBCgAnBQJY2g3EAhsD +BQkHhh+ABQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJEFs/vbT9dEAs1pQQAKIi +QfgYkC3kjGThCGbo5f7o1bw8+wqRWEiELmcjmKSnJoIDAby2mfq40/HoLRUu3svy +erRyUq22f9mULUG377k29e07g1oGJGPQUGbFkmd+nYpSIbunCtrCcUFq/wGgFHGE +HhXWOJts91zpCvAuAcFasVgo7pB3YS4IGzH9lBMktw/92XYoxmLzW5Z4ln6jkmf1 +yk7JOEYcDCugfnkO+FebtzLn+HkZWuPBXKufcesdKBR8FL5gx0awRVB1vMz0YyJ9 +hOJeY0kADnm5Y8QU8zDv4IU5oQbLLVbcmZJrgR+t+GMdx1Vf/miluX6nQmUULXtp +CtXJ0w9Tl1MPRIbOwFBgDjnIPu+CdZbdZDzszG6cqSxICVkkzJZ6Adz1Y3+Pbvp7 +iEgU/RjROWl2cccEo3DZX+8nUTOeIsbvd1nxA+b0A1jzalIn7IfFH0iPZY3wK7M8 +o0QbOsVt1SCugwTRcgRmg7xNhWDq1mxVMLCwE9p4u4XushfwDthzZ3iKVd3F5huU +ANEyb6moeTRZVPijv3TKScagci3+ld53qSn0ZCQYhlPmnDcvcMJRXpZbZfuGJgSM +KfSpM6XkS/4Q2eXM82Nu7Z0ZjLm7MMcyD6ldq95jxIMlvy+XKT1XbuJgLFdnojSU +0+Q1Qe95QpRhcID1zhuuZNMkxohv9rShOtDZwcPYuQINBFjaDcQBEADJqU6EfBgz +GWYRe8FZPs3aTrmWxlN5vGw0z3mJ8Ug56PRtYB3DMl6fnE9W9lRcqYQaKcxP3NdG +hdFIrEV+H8ROT2+jobwdLFqnhboscP7MUOyzWe9kuSwVbHh8L9i/g2aSETB3Ixea +nGUGd0+1aJimzu8uwcMCEE9TXb5JrCH2Q/4O/5i1FuFVQdAgb1XANIFUGPm+Q47b +Njz087oe0T/dJyqWwWiU0EsiUnS83WJfucPn61uF7shONGfxRoQ0mb88qGxS8cSU +y/emwl/5Vg7hWbTJy871ihhqMfn0Dq2kJn5zjcMJSTb6PN5yl/F5x5H/IvCEkoVL +vmP/n4sy2BjzQdy5vyZdFqmnYXpg8CBS0Jb2z2zN+uT2pVrF6AKh56/O9E3myD4F +i1kt7mH21rdytdn0QxaNdzDMkRhDAI0925yfv65NtceJvtLb+Kcb2NkKZ06Wi6yj +GjbGlVp3jmpywnXWSgaRjnU+pf+cekMqbHa0A8gPAhBqPGHR9f4GXVl2c96kPA0V +T4XKZ1z2fhASHvuas7jXAvtMPhw0Lv2HdUmDAab93JxTfO/7z35gcJ658RJqmKZP +aGwr8GIesa5ElIJW3Ck8IEjBCrb59ghxx7yvlAm/IgbV6APkqViz6rsogBKDjZz2 +MF4BEv6KkIrxXQWRlO4eEwhLbZctkqEJrQARAQABiQIlBBgBCgAPBQJY2g3EAhsM +BQkHhh+AAAoJEFs/vbT9dEAsscYP/2UAcCKfdvf3g1+PsoZsunxSmFrzUOut/iye +v7xlrCWV0oPwdWqcG6Rjj3FBxgM8+wHOELDMpqGL87Mfp8YCsp32yTHStx8dMPME +8vrdJ6oct259Z2nPBotv2SHk1EXsIW7SAhxyf9T0RBJQZIVqL5GJWfchbvW8L4JU +ZPqdXQ3yjjfynVdh7TbFlkl1SkW0MSXabSxih0ngLTCitqp5f6tBqmfJYfJjBKVT +f9iyScWVxm3fOiWSnMVdCHCVZZ93sbCLhIPBG2xUIhtTATSnfdPr1N/Qz9ByhDk5 +ncCwR8w13caDxbAf2K2AWI9POwo4880RpD4ctlErSg2Nl6OuT38PAOculvb5wEgo +eHAxETJBIAx9Naceli9jmm4IsuMResqjps8wGRUCnxIYPliKYnGjyztcHCLbVu9D +0NfCjPAM89kZcnEgBbfBi682a0+nz6fmsN3E3zsUQ1Z/jNKkrp4B4cV08OsT4JI2 +Ufo4H11bM8eCjk7k/BIKHtvQyjlcxPzCFY6zdQOv4fkRaO/dZDaCe+5gHcBzyb+h +23GCL/0fnAPKOneLikVX4aHa+cmmICQnOABe33f9i84S3BSqQ9zRrkL0Y4bGKWEt +X4rdhQleSYhAFmk7XDX48Nf2GZCt2zw42GsZMNgXc5n1w5bRWsHMEA7JaGl4eODH +1gpK8c3N +=8Bl9 +-END PGP PUBLIC KEY BLOCK-
[GitHub] rdhabalia removed a comment on issue #2225: Set empty response to complete grpc txn
rdhabalia removed a comment on issue #2225: Set empty response to complete grpc txn URL: https://github.com/apache/incubator-pulsar/pull/2225#issuecomment-407839334 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on issue #2225: Set empty response to complete grpc txn
rdhabalia commented on issue #2225: Set empty response to complete grpc txn URL: https://github.com/apache/incubator-pulsar/pull/2225#issuecomment-407839334 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on a change in pull request #2227: Fix TestS3Offload
sijie commented on a change in pull request #2227: Fix TestS3Offload
URL: https://github.com/apache/incubator-pulsar/pull/2227#discussion_r205202954
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
##
@@ -185,7 +185,6 @@ public static Credentials getCredentials(String driver,
ServiceConfiguration con
} catch (Exception e) {
// allowed, some mock s3 service not need credential
log.error("Exception when get credentials for s3 ", e);
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #2220: Rework TLS docs to separate concerns
merlimat commented on a change in pull request #2220: Rework TLS docs to separate concerns URL: https://github.com/apache/incubator-pulsar/pull/2220#discussion_r205179518 ## File path: site/docs/latest/security/tls.md ## @@ -23,139 +23,134 @@ title: Encryption and Authentication using TLS --> -## TLS Overview +## TLS Authentication Overview -With [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) authentication, the server authenticates the client (also called “2-way authentication”). -Since TLS authentication requires TLS encryption, this page shows you how to configure both at the same time. +TLS authentication is an extension of [TLS transport encryption](../tls-transport), but instead of only servers having keys and certs which the client uses the verify the server's identity, clients also have keys and certs which the server uses to verify the client's identity. You must have TLS transport encryption configured on your cluster before you can use TLS authentication. This guide assumes you already have TLS transport encryption configured. -By default, Apache Pulsar communicates in plain text service url, which means that all data is sent in the clear. -To encrypt communication, it is recommended to configure all the Apache Pulsar components in your deployment to use TLS encryption. +### Creating client certificates -TLS can be configured for encryption or authentication. You may configure just TLS encryption -(by default TLS encryption includes certificate authentication of the server) and independently choose a separate mechanism -for client authentication, e.g. TLS, [Athenz](../athenz), etc. Note that TLS encryption, technically speaking, already enables -1-way authentication in which the client authenticates the server certificate. So when referring to TLS authentication, it is really -referring to 2-way authentication in which the broker also authenticates the client certificate. +Client certificates are generated using the same certificate authority as was used to generate the server certificates. -> Note that enabling TLS may have a performance impact due to encryption overhead. +The biggest difference between client certs and server certs is that the **common name** for the client certificate is the **role token** which that client will be authenticated as. -## Creating TLS Certificates - -Creating TLS certificates for Pulsar involves creating a [certificate authority](#certificate-authority) (CA), [broker certificate](#broker-certificate), and [client certificate](#client-certificate). +First generate the key. +```bash +$ openssl genrsa -out admin.key.pem 2048 +``` -### Certificate authority +Similar to the broker, the client expects the key to be in [PKCS 8](https://en.wikipedia.org/wiki/PKCS_8) format, so convert it. -The first step is to create the certificate for the CA. The CA will be used to sign both the broker and client certificates, in order to ensure that each party will trust the others. +```bash +$ openssl pkcs8 -topk8 -inform PEM -outform PEM \ + -in admin.key.pem -out admin.key-pk8.pem -nocrypt +``` - Linux +Generate the certificate request. When asked for a **common name**, enter the **role token** which you want this key pair to authenticate a client as. ```bash -$ CA.pl -newca +$ openssl req -config openssl.cnf \ + -key admin.key.pem -new -sha256 -out admin.cert.pem ``` - macOS +Sign with request with the certificate authority. Note that that client certs uses the **usr_cert** extension, which allows the cert to be used for client authentication. ```bash -$ /System/Library/OpenSSL/misc/CA.pl -newca +$ openssl ca -config openssl.cnf -extensions usr_cert \ + -days 1000 -notext -md sha256 \ + -in admin.csr.pem -out admin.cert.pem ``` -After answering the question prompts, this will store CA-related files in the `./demoCA` directory. Within that directory: +This will give you a cert, `admin.cert.pem`, and a key, `admin.key-pk8.pem`, which, with `ca.cert.pem`, can be used by clients to authenticate themselves to brokers and proxies as the role token ``admin``. -* `demoCA/cacert.pem` is the public certificate. It is meant to be distributed to all parties involved. -* `demoCA/private/cakey.pem` is the private key. This is only needed when signing a new certificate for either broker or clients and it must be safely guarded. +## Enabling TLS Authentication ... -### Broker certificate +### ... on Brokers -Once a CA certificate has been created, you can create certificate requests and sign them with the CA. +To configure brokers to authenticate clients, put the following in `broker.conf`, alongside [the configuration to enable tls transport](../tls-transport#broker-configuration): -The following commands will ask you a few questions and then create the certificates. When asked for the common name, you need to match the hostname of the broker.
[GitHub] cckellogg opened a new pull request #2230: Make doc image links work when there is a base path.
cckellogg opened a new pull request #2230: Make doc image links work when there is a base path. URL: https://github.com/apache/incubator-pulsar/pull/2230 ### Motivation Make images work in the docs when there is a base path. Example can be seen here. https://cckellogg.github.io/incubator-pulsar/docs/functions-overview This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dsambandam commented on issue #2195: Can't start Global Zookeeper in a Multi-cluster setup
dsambandam commented on issue #2195: Can't start Global Zookeeper in a Multi-cluster setup URL: https://github.com/apache/incubator-pulsar/issues/2195#issuecomment-407767339 15:00:23.240 [NIOServerCxn.Factory:::/0:0:0:0:0:0:0:0:2181] INFO org.apache.zookeeper.server.ZooKeeperServer - Refusing session request for client /Zk2:44714 as it has seen zxid 0xe0057 our last zxid is 0x1 client must try another server Any suggestions @sijie @merlimat - This happened after copying over local meta data from Local to Global-Zk This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly commented on a change in pull request #2227: Fix TestS3Offload
ivankelly commented on a change in pull request #2227: Fix TestS3Offload
URL: https://github.com/apache/incubator-pulsar/pull/2227#discussion_r205101931
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/BlobStoreManagedLedgerOffloader.java
##
@@ -185,7 +185,6 @@ public static Credentials getCredentials(String driver,
ServiceConfiguration con
} catch (Exception e) {
// allowed, some mock s3 service not need credential
log.error("Exception when get credentials for s3 ", e);
Review comment:
log.warn
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] jnioche commented on issue #2210: PulsarSpout to use consumer.redeliverUnacknowledgedMessages() ?
jnioche commented on issue #2210: PulsarSpout to use consumer.redeliverUnacknowledgedMessages() ? URL: https://github.com/apache/incubator-pulsar/issues/2210#issuecomment-407734907 > Probably it would even be better to use the method at #2211 so that we can precisely control the redelivery. We need both _redeliverUnacknowledgedMessages()_ and #2211. The latter could be used for failed messages but we'd still need to keep track of the number of times it failed and a delay to apply. This won't help for messages which were being processed when the JVM died, which is why we need _redeliverUnacknowledgedMessages()_ at init time. > When subscribing the unacked messages are automatically re-delivered. I'm confused. Do you mean that this is what it does even without using _redeliverUnacknowledgedMessages_? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly removed a comment on issue #1134: Docker compose files for pulsar
ivankelly removed a comment on issue #1134: Docker compose files for pulsar URL: https://github.com/apache/incubator-pulsar/pull/1134#issuecomment-407719477 @sijie @merlimat can I get a yea or nah on this? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly edited a comment on issue #1134: Docker compose files for pulsar
ivankelly edited a comment on issue #1134: Docker compose files for pulsar URL: https://github.com/apache/incubator-pulsar/pull/1134#issuecomment-407719330 @sijie @merlimat can I get a yea or nay on this? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly commented on issue #1134: Docker compose files for pulsar
ivankelly commented on issue #1134: Docker compose files for pulsar URL: https://github.com/apache/incubator-pulsar/pull/1134#issuecomment-407719561 retest this please // integration test failed before anything even ran This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly commented on issue #1134: Docker compose files for pulsar
ivankelly commented on issue #1134: Docker compose files for pulsar URL: https://github.com/apache/incubator-pulsar/pull/1134#issuecomment-407719477 @sijie @merlimat can I get a yea or nah on this? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly commented on issue #1134: Docker compose files for pulsar
ivankelly commented on issue #1134: Docker compose files for pulsar URL: https://github.com/apache/incubator-pulsar/pull/1134#issuecomment-407719330 @sijie @merlimat can I get a yea or nah on this? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: Fix implementation of equals() for MessageId (#2226)
This is an automated email from the ASF dual-hosted git repository.
nkurihar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new a186672 Fix implementation of equals() for MessageId (#2226)
a186672 is described below
commit a1866721d9e9aa85fb913bc6a0ea165ca0f7ab3d
Author: massakam
AuthorDate: Wed Jul 25 19:53:08 2018 +0900
Fix implementation of equals() for MessageId (#2226)
---
.../pulsar/client/impl/BatchMessageIdImpl.java | 4 +++-
.../apache/pulsar/client/impl/MessageIdImpl.java | 8
.../pulsar/client/impl/BatchMessageIdImplTest.java | 24 ++
3 files changed, 31 insertions(+), 5 deletions(-)
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
index 78109c4..d66e242 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
@@ -92,7 +92,9 @@ public class BatchMessageIdImpl extends MessageIdImpl {
return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex
&& batchIndex == other.batchIndex;
} else if (obj instanceof MessageIdImpl) {
-return batchIndex == NO_BATCH && obj.equals(this);
+MessageIdImpl other = (MessageIdImpl) obj;
+return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex
+&& batchIndex == NO_BATCH;
}
return false;
}
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
index 41cc9c9..3686b12 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
@@ -70,12 +70,12 @@ public class MessageIdImpl implements MessageId {
@Override
public boolean equals(Object obj) {
-if (obj instanceof MessageIdImpl) {
-MessageIdImpl other = (MessageIdImpl) obj;
-return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex;
-} else if (obj instanceof BatchMessageIdImpl){
+if (obj instanceof BatchMessageIdImpl) {
BatchMessageIdImpl other = (BatchMessageIdImpl) obj;
return other.equals(this);
+} else if (obj instanceof MessageIdImpl) {
+MessageIdImpl other = (MessageIdImpl) obj;
+return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex;
}
return false;
}
diff --git
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
index a3b5f72..af21fcf 100644
---
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
+++
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
@@ -44,4 +44,28 @@ public class BatchMessageIdImplTest {
Assert.assertTrue(batchMsgId1.hashCode() != batchMsgId2.hashCode());
}
+
+@Test
+public void equalsTest() {
+BatchMessageIdImpl batchMsgId1 = new BatchMessageIdImpl(0, 0, 0, 0);
+BatchMessageIdImpl batchMsgId2 = new BatchMessageIdImpl(1, 1, 1, 1);
+BatchMessageIdImpl batchMsgId3 = new BatchMessageIdImpl(0, 0, 0, 1);
+BatchMessageIdImpl batchMsgId4 = new BatchMessageIdImpl(0, 0, 0, -1);
+MessageIdImpl msgId = new MessageIdImpl(0, 0, 0);
+
+Assert.assertTrue(batchMsgId1.equals(batchMsgId1));
+Assert.assertFalse(batchMsgId1.equals(batchMsgId2));
+Assert.assertFalse(batchMsgId1.equals(batchMsgId3));
+Assert.assertFalse(batchMsgId1.equals(batchMsgId4));
+Assert.assertFalse(batchMsgId1.equals(msgId));
+
+Assert.assertTrue(msgId.equals(msgId));
+Assert.assertFalse(msgId.equals(batchMsgId1));
+Assert.assertFalse(msgId.equals(batchMsgId2));
+Assert.assertFalse(msgId.equals(batchMsgId3));
+Assert.assertTrue(msgId.equals(batchMsgId4));
+
+Assert.assertTrue(batchMsgId4.equals(msgId));
+}
+
}
[GitHub] nkurihar closed pull request #2226: Fix implementation of equals() for MessageId
nkurihar closed pull request #2226: Fix implementation of equals() for MessageId
URL: https://github.com/apache/incubator-pulsar/pull/2226
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
index 78109c490c..d66e242feb 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java
@@ -92,7 +92,9 @@ public boolean equals(Object obj) {
return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex
&& batchIndex == other.batchIndex;
} else if (obj instanceof MessageIdImpl) {
-return batchIndex == NO_BATCH && obj.equals(this);
+MessageIdImpl other = (MessageIdImpl) obj;
+return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex
+&& batchIndex == NO_BATCH;
}
return false;
}
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
index 41cc9c9d50..3686b124a8 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageIdImpl.java
@@ -70,12 +70,12 @@ public int hashCode() {
@Override
public boolean equals(Object obj) {
-if (obj instanceof MessageIdImpl) {
-MessageIdImpl other = (MessageIdImpl) obj;
-return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex;
-} else if (obj instanceof BatchMessageIdImpl){
+if (obj instanceof BatchMessageIdImpl) {
BatchMessageIdImpl other = (BatchMessageIdImpl) obj;
return other.equals(this);
+} else if (obj instanceof MessageIdImpl) {
+MessageIdImpl other = (MessageIdImpl) obj;
+return ledgerId == other.ledgerId && entryId == other.entryId &&
partitionIndex == other.partitionIndex;
}
return false;
}
diff --git
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
index a3b5f72995..af21fcfde6 100644
---
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
+++
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BatchMessageIdImplTest.java
@@ -44,4 +44,28 @@ public void hashCodeTest() {
Assert.assertTrue(batchMsgId1.hashCode() != batchMsgId2.hashCode());
}
+
+@Test
+public void equalsTest() {
+BatchMessageIdImpl batchMsgId1 = new BatchMessageIdImpl(0, 0, 0, 0);
+BatchMessageIdImpl batchMsgId2 = new BatchMessageIdImpl(1, 1, 1, 1);
+BatchMessageIdImpl batchMsgId3 = new BatchMessageIdImpl(0, 0, 0, 1);
+BatchMessageIdImpl batchMsgId4 = new BatchMessageIdImpl(0, 0, 0, -1);
+MessageIdImpl msgId = new MessageIdImpl(0, 0, 0);
+
+Assert.assertTrue(batchMsgId1.equals(batchMsgId1));
+Assert.assertFalse(batchMsgId1.equals(batchMsgId2));
+Assert.assertFalse(batchMsgId1.equals(batchMsgId3));
+Assert.assertFalse(batchMsgId1.equals(batchMsgId4));
+Assert.assertFalse(batchMsgId1.equals(msgId));
+
+Assert.assertTrue(msgId.equals(msgId));
+Assert.assertFalse(msgId.equals(batchMsgId1));
+Assert.assertFalse(msgId.equals(batchMsgId2));
+Assert.assertFalse(msgId.equals(batchMsgId3));
+Assert.assertTrue(msgId.equals(batchMsgId4));
+
+Assert.assertTrue(batchMsgId4.equals(msgId));
+}
+
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ivankelly commented on issue #2220: Rework TLS docs to separate concerns
ivankelly commented on issue #2220: Rework TLS docs to separate concerns URL: https://github.com/apache/incubator-pulsar/pull/2220#issuecomment-407710801 run integration tests // PulsarIOSinkTest.setupCluster This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly opened a new pull request #2229: C++ should default not allow insecure connections with TLS
ivankelly opened a new pull request #2229: C++ should default not allow insecure connections with TLS URL: https://github.com/apache/incubator-pulsar/pull/2229 If tlsAllowInsecureConnection is true, then the client doesn't validate the server's TLS cert, allowing for MITM attacks. This shouldn't be the default option. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly opened a new pull request #2228: More detailed documentation to config file for TLS
ivankelly opened a new pull request #2228: More detailed documentation to config file for TLS URL: https://github.com/apache/incubator-pulsar/pull/2228 This patch adds more detail for TLS options in the broker, proxy, and client configuration files which were previously ambiguous. It also adds some options which were missing, and suggests some defaults in the client configuration. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie opened a new pull request #2227: Fix TestS3Offload
sijie opened a new pull request #2227: Fix TestS3Offload URL: https://github.com/apache/incubator-pulsar/pull/2227 ### Motivation s3 mock service doesn't need credential. so we will fail to load credentials on s3 integration tests. that is expected. However there is a regression when introducing GCS support in #2151: https://github.com/apache/incubator-pulsar/pull/2151/files#diff-9160ffc9df7afcf42a8335f214393c6dR188 ### Changes Don't throw exception when failed to load aws credentials This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] massakam commented on issue #2226: Fix implementation of equals() for MessageId
massakam commented on issue #2226: Fix implementation of equals() for MessageId URL: https://github.com/apache/incubator-pulsar/pull/2226#issuecomment-407699839 rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ivankelly commented on a change in pull request #2152: GCS offload support(4): add documentations for GCS
ivankelly commented on a change in pull request #2152: GCS offload support(4):
add documentations for GCS
URL: https://github.com/apache/incubator-pulsar/pull/2152#discussion_r205039932
##
File path: site/docs/latest/cookbooks/tiered-storage.md
##
@@ -17,42 +19,63 @@ A topic in Pulsar is backed by a log, known as a managed
ledger. This log is com
The Tiered Storage offloading mechanism takes advantage of this segment
oriented architecture. When offloading is requested, the segments of the log
are copied, one-by-one, to tiered storage. All segments of the log, apart from
the segment currently being written to can be offloaded.
-## Amazon S3
+On the broker, the administrator must configure the bucket or credentials for
the cloud storage service. The configured bucket must exist before attempting
to offload. If it does not exist, the offload operation will fail.
-Tiered storage currently supports S3 for long term storage. On the broker, the
administrator must configure a S3 bucket and the AWS region where the bucket
exists. Offloaded data will be placed into this bucket.
+Pulsar users multi-part objects to update the segment data. It is possible
that a broker could crash while uploading the data. We recommend you add a life
cycle rule your bucket to expire incomplete multi-part upload after a day or
two to avoid getting charged for incomplete uploads.
-The configured S3 bucket must exist before attempting to offload. If it does
not exist, the offload operation will fail.
+## Configuring for S3 and GCS in the broker
-Pulsar users multipart objects to update the segment data. It is possible that
a broker could crash while uploading the data. We recommend you add a lifecycle
rule your S3 bucket to expire incomplete multipart upload after a day or two to
avoid getting charged for incomplete uploads.
+Offloading is configured in ```broker.conf```.
-### Configuring the broker
+At a minimum, the administrator must configure the driver, the bucket and the
authenticating. There is also some other knobs to configure, like the bucket
regions, the max block size in backed storage, etc.
-Offloading is configured in ```broker.conf```.
+### Configure the driver
-At a minimum, the user must configure the driver, the region and the bucket.
+Currently we support driver of types: { "S3", "aws-s3", "google-cloud-storage"
},
+{% include admonition.html type="warning" content="The chars are case ignored
for driver's name. "s3" and "aws-s3" are similar, with "aws-s3" you just don't
need to define the url of the endpoint because it will know to use
`s3.amazonaws.com`." %}
```conf
managedLedgerOffloadDriver=S3
-s3ManagedLedgerOffloadRegion=eu-west-3
+```
+
+### Configure the Bucket
+
+On the broker, the administrator must configure the bucket or credentials for
the cloud storage service. The configured bucket must exist before attempting
to offload. If it does not exist, the offload operation will fail.
+
+- Regarding driver type "S3" or "aws-s3", the administrator should configure
`s3ManagedLedgerOffloadBucket`.
+
+```conf
s3ManagedLedgerOffloadBucket=pulsar-topic-offload
```
-It is also possible to specify the s3 endpoint directly, using
```s3ManagedLedgerOffloadServiceEndpoint```. This is useful if you are using a
non-AWS storage service which provides an S3 compatible API.
+- While regarding driver type "google-cloud-storage", the administrator should
configure `gcsManagedLedgerOffloadBucket`.
+```conf
+gcsManagedLedgerOffloadBucket=pulsar-topic-offload
+```
-{% include admonition.html type="warning" content="If the endpoint is
specified directly, then the region must _not_ be set." %}
+### Configure the Bucket Region
-{% include admonition.html type="warning" content="The broker.conf of all
brokers must have the same configuration for driver, region and bucket for
offload to avoid data becoming unavailable as topics move from one broker to
another." %}
+Bucket Region is the region where bucket located.
-Pulsar also provides some knobs to configure the size of requests sent to S3.
+Regarding AWS S3, the default region is `US East (N. Virginia)`. Page [AWS
Regions and
Endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html) contains
more information.
Review comment:
The user doesn't care if there is duplication. They care how much they need
to read to get up and running. And how much of what they read is useful to
them. With the current layout, 50% of what they read is useless to them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] sijie closed pull request #2224: Fixed ack message tracker cleanup for single message batches
sijie closed pull request #2224: Fixed ack message tracker cleanup for single
message batches
URL: https://github.com/apache/incubator-pulsar/pull/2224
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
index 0b5126b00f..da53760d2b 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
@@ -19,6 +19,8 @@
package org.apache.pulsar.client.impl;
import static org.testng.Assert.assertEquals;
+import static org.testng.Assert.assertFalse;
+import static org.testng.Assert.assertTrue;
import java.util.HashSet;
import java.util.concurrent.TimeUnit;
@@ -30,6 +32,7 @@
import org.apache.pulsar.client.api.MessageRoutingMode;
import org.apache.pulsar.client.api.Producer;
import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.client.api.Schema;
import org.apache.pulsar.client.api.SubscriptionType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -386,4 +389,33 @@ public void testCheckUnAcknowledgedMessageTimer() throws
PulsarClientException,
assertEquals(consumer.getUnAckedMessageTracker().size(), 0);
}
+
+@Test
+public void testSingleMessageBatch() throws Exception {
+String topicName = "prop/ns-abc/topic-estSingleMessageBatch";
+
+Producer producer = pulsarClient.newProducer(Schema.STRING)
+.topic(topicName)
+.enableBatching(true)
+.batchingMaxPublishDelay(10, TimeUnit.SECONDS)
+.create();
+
+Consumer consumer = pulsarClient.newConsumer(Schema.STRING)
+.topic(topicName)
+.subscriptionName("subscription")
+.ackTimeout(1, TimeUnit.HOURS)
+.subscribe();
+
+// Force the creation of a batch with a single message
+producer.sendAsync("hello");
+producer.flush();
+
+Message message = consumer.receive();
+
+assertFalse(((ConsumerImpl)
consumer).getUnAckedMessageTracker().isEmpty());
+
+consumer.acknowledge(message);
+
+assertTrue(((ConsumerImpl)
consumer).getUnAckedMessageTracker().isEmpty());
+}
}
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
index 6f781196d5..1c69c754a8 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
@@ -385,11 +385,6 @@ boolean markAckForBatchMessage(BatchMessageIdImpl
batchMessageId, AckType ackTyp
log.debug("[{}] [{}] can ack message to broker {}, acktype {},
cardinality {}, length {}", subscription,
consumerName, batchMessageId, ackType,
outstandingAcks, batchSize);
}
-// increment Acknowledge-msg counter with number of messages in
batch only if AckType is Individual.
-// CumulativeAckType is handled while sending ack to broker
-if (ackType == AckType.Individual) {
-stats.incrementNumAcksSent(batchSize);
-}
return true;
} else {
if (AckType.Cumulative == ackType
@@ -433,11 +428,16 @@ boolean markAckForBatchMessage(BatchMessageIdImpl
batchMessageId, AckType ackTyp
Map
properties) {
MessageIdImpl msgId = (MessageIdImpl) messageId;
-
if (ackType == AckType.Individual) {
-unAckedMessageTracker.remove(msgId);
-// increment counter by 1 for non-batch msg
-if (!(messageId instanceof BatchMessageIdImpl)) {
+if (messageId instanceof BatchMessageIdImpl) {
+BatchMessageIdImpl batchMessageId = (BatchMessageIdImpl)
messageId;
+
+stats.incrementNumAcksSent(batchMessageId.getBatchSize());
+unAckedMessageTracker.remove(new
MessageIdImpl(batchMessageId.getLedgerId(),
+batchMessageId.getEntryId(),
batchMessageId.getPartitionIndex()));
+} else {
+// increment counter by 1 for non-batch msg
+unAckedMessageTracker.remove(msgId);
stats.incrementNumAcksSent(1);
}
} else if (ackType == AckType.Cumulative) {
@@ -717,16 +717,16 @@ void messageReceived(MessageIdData messageId,
[GitHub] sijie closed issue #2221: Getting UnAckedMessageTracker messages have timed-out without a reason
sijie closed issue #2221: Getting UnAckedMessageTracker messages have timed-out without a reason URL: https://github.com/apache/incubator-pulsar/issues/2221 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch master updated: Fixed ack message tracker cleanup for single message batches (#2224)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 3c2e3e2 Fixed ack message tracker cleanup for single message batches
(#2224)
3c2e3e2 is described below
commit 3c2e3e2e3fc3d2cce75dd3e42a77c2d7b3bc0499
Author: Matteo Merli
AuthorDate: Wed Jul 25 02:00:16 2018 -0700
Fixed ack message tracker cleanup for single message batches (#2224)
### Motivation
Fixes #2221
The issue is that we were removing a batch message id impl from the
tracking set which contains the batch-wide ids.
---
.../impl/UnAcknowledgedMessagesTimeoutTest.java| 32 ++
.../apache/pulsar/client/impl/ConsumerImpl.java| 28 +--
2 files changed, 46 insertions(+), 14 deletions(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
index 0b5126b..da53760 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java
@@ -19,6 +19,8 @@
package org.apache.pulsar.client.impl;
import static org.testng.Assert.assertEquals;
+import static org.testng.Assert.assertFalse;
+import static org.testng.Assert.assertTrue;
import java.util.HashSet;
import java.util.concurrent.TimeUnit;
@@ -30,6 +32,7 @@ import org.apache.pulsar.client.api.MessageId;
import org.apache.pulsar.client.api.MessageRoutingMode;
import org.apache.pulsar.client.api.Producer;
import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.client.api.Schema;
import org.apache.pulsar.client.api.SubscriptionType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -386,4 +389,33 @@ public class UnAcknowledgedMessagesTimeoutTest extends
BrokerTestBase {
assertEquals(consumer.getUnAckedMessageTracker().size(), 0);
}
+
+@Test
+public void testSingleMessageBatch() throws Exception {
+String topicName = "prop/ns-abc/topic-estSingleMessageBatch";
+
+Producer producer = pulsarClient.newProducer(Schema.STRING)
+.topic(topicName)
+.enableBatching(true)
+.batchingMaxPublishDelay(10, TimeUnit.SECONDS)
+.create();
+
+Consumer consumer = pulsarClient.newConsumer(Schema.STRING)
+.topic(topicName)
+.subscriptionName("subscription")
+.ackTimeout(1, TimeUnit.HOURS)
+.subscribe();
+
+// Force the creation of a batch with a single message
+producer.sendAsync("hello");
+producer.flush();
+
+Message message = consumer.receive();
+
+assertFalse(((ConsumerImpl)
consumer).getUnAckedMessageTracker().isEmpty());
+
+consumer.acknowledge(message);
+
+assertTrue(((ConsumerImpl)
consumer).getUnAckedMessageTracker().isEmpty());
+}
}
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
index 6f78119..1c69c75 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
@@ -385,11 +385,6 @@ public class ConsumerImpl extends ConsumerBase
implements ConnectionHandle
log.debug("[{}] [{}] can ack message to broker {}, acktype {},
cardinality {}, length {}", subscription,
consumerName, batchMessageId, ackType,
outstandingAcks, batchSize);
}
-// increment Acknowledge-msg counter with number of messages in
batch only if AckType is Individual.
-// CumulativeAckType is handled while sending ack to broker
-if (ackType == AckType.Individual) {
-stats.incrementNumAcksSent(batchSize);
-}
return true;
} else {
if (AckType.Cumulative == ackType
@@ -433,11 +428,16 @@ public class ConsumerImpl extends ConsumerBase
implements ConnectionHandle
Map
properties) {
MessageIdImpl msgId = (MessageIdImpl) messageId;
-
if (ackType == AckType.Individual) {
-unAckedMessageTracker.remove(msgId);
-// increment counter by 1 for non-batch msg
-if (!(messageId instanceof BatchMessageIdImpl)) {
+if (messageId instanceof BatchMessageIdImpl) {
+BatchMessageIdImpl batchMessageId = (BatchMessageIdImpl)
messageId;
+
+
[GitHub] aahmed-se commented on issue #2224: Fixed ack message tracker cleanup for single message batches
aahmed-se commented on issue #2224: Fixed ack message tracker cleanup for single message batches URL: https://github.com/apache/incubator-pulsar/pull/2224#issuecomment-407683851 rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-pulsar] branch asf-site updated: Updated site at revision 2ad89f4
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new b83c777 Updated site at revision 2ad89f4 b83c777 is described below commit b83c7775756689c783d130169e0f390ea7936ed1 Author: jenkins AuthorDate: Wed Jul 25 08:35:24 2018 + Updated site at revision 2ad89f4 --- .../docs/latest/adaptors/PulsarSpark/index.html| 10 +- .../docs/latest/adaptors/PulsarStorm/index.html| 8 content/docs/latest/admin-api/overview/index.html | 18 +- content/docs/latest/clients/Cpp/index.html | 8 content/docs/latest/clients/Java/index.html| 16 content/docs/latest/clients/Python/index.html | 10 +- content/docs/latest/clients/WebSocket/index.html | 8 content/docs/latest/clients/go/index.html | 6 +++--- .../docs/latest/cookbooks/Encryption/index.html| 6 +++--- .../latest/cookbooks/PartitionedTopics/index.html | 16 .../latest/cookbooks/RetentionExpiry/index.html| 16 .../docs/latest/cookbooks/compaction/index.html| 8 .../cookbooks/message-deduplication/index.html | 12 ++-- .../docs/latest/cookbooks/message-queue/index.html | 16 .../latest/cookbooks/tiered-storage/index.html | 6 +++--- .../docs/latest/deployment/Kubernetes/index.html | 4 ++-- .../docs/latest/deployment/aws-cluster/index.html | 6 +++--- content/docs/latest/deployment/cluster/index.html | 6 +++--- content/docs/latest/deployment/instance/index.html | 6 +++--- .../ConceptsAndArchitecture/index.html | 4 ++-- .../latest/getting-started/LocalCluster/index.html | 4 ++-- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 ++-- .../docs/latest/project/BinaryProtocol/index.html | 4 ++-- content/docs/latest/project/CompileCpp/index.html | 8 .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 ++-- content/docs/latest/reference/CliTools/index.html | 22 +++--- .../docs/latest/security/authorization/index.html | 14 +++--- content/docs/latest/security/encryption/index.html | 6 +++--- content/ja/adaptors/PulsarSpark/index.html | 8 content/ja/adaptors/PulsarStorm/index.html | 6 +++--- content/ja/admin/AdminInterface/index.html | 12 ++-- content/ja/admin/Authz/index.html | 12 ++-- content/ja/admin/ClustersBrokers/index.html| 6 +++--- content/ja/admin/PropertiesNamespaces/index.html | 6 +++--- content/ja/advanced/PartitionedTopics/index.html | 12 ++-- content/ja/advanced/RetentionExpiry/index.html | 12 ++-- content/ja/clients/Cpp/index.html | 6 +++--- content/ja/clients/Java/index.html | 8 content/ja/clients/Python/index.html | 8 content/ja/clients/WebSocket/index.html| 8 content/ja/deployment/InstanceSetup/index.html | 6 +++--- content/ja/deployment/Kubernetes/index.html| 4 ++-- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 ++-- content/ja/project/BinaryProtocol/index.html | 4 ++-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 +- 49 files changed, 202 insertions(+), 202 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index 9471cd3..ae1dfd8 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1086,9 +1086,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1330,9 +1330,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1540,8 +1540,6 @@ - - The Pulsar Java client @@ -1620,6 +1618,8 @@ + + Authentication and authorization in Pulsar @@ -1674,9 +1674,9 @@ - The Pulsar admin interface + The Pulsar admin interface diff --git
[GitHub] IdanFridman commented on issue #2221: Getting UnAckedMessageTracker messages have timed-out without a reason
IdanFridman commented on issue #2221: Getting UnAckedMessageTracker messages have timed-out without a reason URL: https://github.com/apache/incubator-pulsar/issues/2221#issuecomment-407674300 @merlimat Can you tell if this issue can harm us in anyway with the current pulsar version? or can we just ignore it and move on untill this fix will be submitted ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2189: dumpContainerDirToTargetCompressed should copy files to a containerName named directory
sijie commented on issue #2189: dumpContainerDirToTargetCompressed should copy files to a containerName named directory URL: https://github.com/apache/incubator-pulsar/pull/2189#issuecomment-407660951 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] massakam commented on issue #2226: Fix implementation of equals() for MessageId
massakam commented on issue #2226: Fix implementation of equals() for MessageId URL: https://github.com/apache/incubator-pulsar/pull/2226#issuecomment-407648444 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2189: dumpContainerDirToTargetCompressed should copy files to a containerName named directory
sijie commented on issue #2189: dumpContainerDirToTargetCompressed should copy files to a containerName named directory URL: https://github.com/apache/incubator-pulsar/pull/2189#issuecomment-407646602 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] zhaijack commented on issue #2152: GCS offload support(4): add documentations for GCS
zhaijack commented on issue #2152: GCS offload support(4): add documentations for GCS URL: https://github.com/apache/incubator-pulsar/pull/2152#issuecomment-407646535 rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] sijie commented on issue #2224: Fixed ack message tracker cleanup for single message batches
sijie commented on issue #2224: Fixed ack message tracker cleanup for single message batches URL: https://github.com/apache/incubator-pulsar/pull/2224#issuecomment-407646271 rerun java8 tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
