Re: Non-persistent topics

2018-06-20 Thread Sijie Guo
2.1.0-SNAPSHOT is published to apache staging nightly. I will try to include this fix for 2.1.0 release in this week. So it can go out with 2.1 release. On Tue, Jun 19, 2018 at 11:40 PM Geoffroy Fouquier < geoffroy.fouqu...@exensa.com> wrote: > On 19/06/2018 21:58, Sijie Guo wrote: > > I put up

Re: Non-persistent topics

2018-06-19 Thread Geoffroy Fouquier
On 19/06/2018 21:58, Sijie Guo wrote: I put up a fix - https://github.com/apache/incubator-pulsar/pull/1994 verified that it fixes the problem. however I would like to get feedback before I added unit tests / integration tests. - Sijie Great, thanks ! Is there some nightly build which produce

Re: Non-persistent topics

2018-06-19 Thread Sijie Guo
he server. For >>>> a given pulsar version, my test script download an archive and make all >>>> tests using this archive. >>>> >>>> > -Ali >>>> > >>>> > >>>> > On Tue, Jun 19, 2018 at 7:47 AM, Geoffroy

Re: Non-persistent topics

2018-06-19 Thread Sijie Guo
>>> >>> > -Ali >>> > >>> > >>> > On Tue, Jun 19, 2018 at 7:47 AM, Geoffroy Fouquier < >>> > geoffroy.fouqu...@exensa.com> wrote: >>> > >>> >> >>> >> I recently describe on pulsar-users

Re: Non-persistent topics

2018-06-19 Thread Sijie Guo
a given pulsar version, my test script download an archive and make all >> tests using this archive. >> >> > -Ali >> > >> > >> > On Tue, Jun 19, 2018 at 7:47 AM, Geoffroy Fouquier < >> > geoffroy.fouqu...@exensa.com> wrote: >> > &g

Re: Non-persistent topics

2018-06-19 Thread Sijie Guo
uquier < > > geoffroy.fouqu...@exensa.com> wrote: > > > >> > >> I recently describe on pulsar-users my issue concerning non-persistent > >> topics. This time, I reproduce the faultly behaviour using only > >> pulsar-client and test the new pu

Re: Non-persistent topics

2018-06-19 Thread geoffroy . fouquier
AM, Geoffroy Fouquier < geoffroy.fouqu...@exensa.com> wrote: I recently describe on pulsar-users my issue concerning non-persistent topics. This time, I reproduce the faultly behaviour using only pulsar-client and test the new pulsar 2.0.1 release, but my problem remains. It's qu

Re: Non-persistent topics

2018-06-19 Thread Ali Ahmed
Is the server version is both cases 2.0 ? -Ali On Tue, Jun 19, 2018 at 7:47 AM, Geoffroy Fouquier < geoffroy.fouqu...@exensa.com> wrote: > > I recently describe on pulsar-users my issue concerning non-persistent > topics. This time, I reproduce the faultly behaviour using only

Non-persistent topics

2018-06-19 Thread Geoffroy Fouquier
I recently describe on pulsar-users my issue concerning non-persistent topics. This time, I reproduce the faultly behaviour using only pulsar-client and test the new pulsar 2.0.1 release, but my problem remains. It's quite easy to reproduce this behaviour:  - I made tests with

[GitHub] merlimat closed pull request #538: PIP-2: Introduce non-persistent topics

2017-07-28 Thread git
merlimat closed pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129984310 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129984310 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129983782 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129974898 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129978136 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129953704 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318472991 @rdhabalia One other thing is what happens when a subscriptions has no more consumers available. Right now the subscription stays around

[GitHub] rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318461106 > If I enable batching the drop-rate falls to 0.016 /s though I suspect it's only counting the number of dropped batches

[GitHub] rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318460224 > At the rate of 10K write/s, with no batching after adding more debug logs where we record-msg drop: found out that all

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318455797 @rdhabalia One other note: If I enable batching the drop-rate falls to 0.016 /s though I suspect it's only counting the number of dr

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129923821 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Producer.java ## @@ -56,6

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129925654 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129926873 ## File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/PublisherStats.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129925369 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129920713 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/PersistentTopics.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129924549 ## File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129926898 ## File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/ReplicatorStats.java

[GitHub] merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129926941 ## File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/SubscriptionStats.java

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318450195 This was even with 500K permits This is an automated message from the

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318450049 ```json "consumerName" : "28513", "availablePermits" : 417242,

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318449959 > I feel it's happening due to not available permits? Not sure, I increased to 50K receiver queue size and it still

[GitHub] rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318449580 > At the rate of 10K write/s, with no batching I feel it's happening due to [not available permits](https://github.com/r

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318443565 ```json $ bin/pulsar-admin non-persistent stats non-persistent://prop/cluster/ns/my-topic { "msgRateIn" : 1.31

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-27 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318443470 @rdhabalia I'm running a 1 prod - 1 cons test with non-persistent. Everything local. At the rate of 10K write/s, with no bat

[GitHub] rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-26 Thread git
rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318213106 > testClosingReplicationProducerTwice(org.apache.pulsar.broker.service.PersistentTopicTest) Time elapsed: 0.025 sec <<

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-26 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-318209178 @rdhabalia One test failure seems genuine : ``` Tests run: 521, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 996.983 sec

[GitHub] rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-25 Thread git
rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-317838210 > One small issue I'm seeing on the CLI tool: yes, thanks for catching it. I will fix it. >

[GitHub] rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-25 Thread git
rdhabalia commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-317838210 > One small issue I'm seeing on the CLI tool: yes, thanks for catching it. I will fix it. >

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-25 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-317836584 One more exception in broker : ``` 2017-07-25 11:53:32,957 - ERROR - [pulsar-stats-updater-73-1:PulsarStats@121] - Failed to

[GitHub] merlimat commented on issue #538: PIP-2: Introduce non-persistent topics

2017-07-25 Thread git
merlimat commented on issue #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#issuecomment-317831724 @rdhabalia One small issue I'm seeing on the CLI tool: ```shell $ bin/pulsar-admin non-persistent stats non-persistent:/

[GitHub] rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-25 Thread git
rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129388965 ## File path: pulsar-broker/src/test/java/org/apache/pulsar/client/api/NonPersistentTopicTest.java

[GitHub] rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics

2017-07-25 Thread git
rdhabalia commented on a change in pull request #538: PIP-2: Introduce non-persistent topics URL: https://github.com/apache/incubator-pulsar/pull/538#discussion_r129388496 ## File path: pulsar-broker/src/test/java/org/apache/pulsar/client/api/NonPersistentTopicTest.java