Re: [VOTE] Release Apache RocketMQ 4.9.8 RC1

[Zhanhui Li]

+1

On Sun, Feb 18, 2024 at 10:29 AM ShannonDing  wrote:

> +1
>
>
>
>
>
> At 2024-02-13 18:17:38, "jinrongtong"  wrote:
> >Hello RocketMQ Community,
> >
> >This is the vote for 4.9.8 of Apache RocketMQ.
> >
> >This release fixes some bugs to enhance stability.
> >
> >The artifacts:
> >https://dist.apache.org/repos/dist/dev/rocketmq/4.9.8-rc1/
> >
> >The staging repo:
> >https://repository.apache.org/content/repositories/orgapacherocketmq-1309
> >
> >Git tag for the release:
> >https://github.com/apache/rocketmq/tree/release-4.9.8
> >
> >Hash for the release tag:
> >8b287928fb4c8c565282c019ef668c43e6eb3289
> >
> >Release Notes:
> >https://rocketmq.apache.org/zh/release-notes/2024/01/29/4.9.8/
> >
> >
> >
> >The artifacts have been signed with Key :
> >EC9F268B4C20590138B11FE701420E4292296EAE, which can be found in the keys
> file:
> >https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> >
> >Checklist for reference,
> >Note that this is not official policy but may help with checking releases.
> >Fill in the following:
> >
> >[]  Checksums and PGP signatures are valid for Source package.
> >[]  Checksums and PGP signatures are valid for Binary package.
> >[]  Source code artifacts have correct names matching the current release.
> >[]  Binary artifacts have correct names matching the current release.
> >[]  License and Notice are correct in source package.
> >[]  License and Notice are correct in binary package.
> >[]  All files have license headers in source package if necessary.
> >[]  No compiled archives bundled in source archive.
> >[]  Hash and Tag in GitHub repo is matching the current artifacts.
> >[]  The code can be built success in the source package.
> >[]  Start nameserver and broker according to the quick-start
> >[]  Run clusterList command to see if the version is correct
> >
> >And any other check point is welcomed and please feel free to reply to
> this email, we sincerely hope to your feedback.
> >
> >The vote will be open for at least 72 hours or until necessary number of
> votes are reached.
> >
> >Please vote accordingly:
> >
> >[ ] +1 approve
> >
> >[ ] +0 no opinion
> >
> >[ ] -1 disapprove with the reason
> >
> >
> >
> >Thanks,
> >The Apache RocketMQ Team
> >
>

Re: [VOTE] Release Apache RocketMQ EventBridge 1.1.0

[Zhanhui Li]

+1

Checked
   SHA512 of source and bin zip packages are correct;
   PGP signature;
   License is Apache Version 2.0;
  Notice existence;

BTW, the bin/source release zip packages contain macOS-specific, yet
irrelevant, files:

 lizhanhui@ZHANHUILI-MB0 apache % unzip -l
rocketmq-eventbridge-1.1.0-bin-release.zip | grep __M

  176  02-17-2024 22:23
__MACOSX/rocketmq-eventbridge-1.1.0-bin-release/bin/._eventbridge.sh

  176  02-17-2024 22:52
__MACOSX/rocketmq-eventbridge-1.1.0-bin-release/config/._application.properties

  120  02-17-2024 22:57
__MACOSX/rocketmq-eventbridge-1.1.0-bin-release/plugin/._.DS_Store

  120  02-17-2024 22:32
__MACOSX/rocketmq-eventbridge-1.1.0-bin-release/lib/._.DS_Store

lizhanhui@ZHANHUILI-MB0 apache % unzip -l
rocketmq-eventbridge-1.1.0-source-release.zip | grep __M

  120  02-17-2024 23:08
__MACOSX/rocketmq-eventbridge-1.1.0-source-release/._.DS_Store

  120  02-17-2024 23:08
__MACOSX/rocketmq-eventbridge-1.1.0-source-release/supports/._.DS_Store

  120  02-17-2024 23:08
__MACOSX/rocketmq-eventbridge-1.1.0-source-release/test/._.DS_Store

  120  02-17-2024 23:08
__MACOSX/rocketmq-eventbridge-1.1.0-source-release/start/._.DS_Store

  120  02-17-2024 23:08
__MACOSX/rocketmq-eventbridge-1.1.0-source-release/adapter/._.DS_Store

  120  02-17-2024 23:07
__MACOSX/rocketmq-eventbridge-1.1.0-source-release/start/src/._.DS_Store

  120  02-17-2024 23:07
__MACOSX/rocketmq-eventbridge-1.1.0-source-release/start/src/main/._.DS_Store


IMO, these release tar/zip packages shall be generated via CI pipeline.
This would help us verify these source files matches given git commit
conveniently.


Zhanhui Li

On Sun, Feb 18, 2024 at 9:03 AM 沈林 <2011shen...@gmail.com> wrote:

> Hello RocketMQ Community,
>
> This is the vote for 1.1.0 release of Apache RocketMQ EventBridge. In this
> version, We will be releasing some functionality of EventBridge. Including:
> [+] The runtime of eventbridge.
> [+] Built in demo use case: send events,filter,transform and write to file.
> [+] Supports built-in databases without the need to build MySQL.
> [+] Add file sink connector.
> [+] Add e2e test case.
> [+] Add git workflows check when creating the  pull request.
>
> The artifact:
>
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-eventbridge/1.1.0-rc1/
>
>
> Git tag for the release:
>
> https://github.com/apache/rocketmq-eventbridge/releases/tag/rocketmq-eventbridge-1.1.0
>
> Hash for the release tag: 6a852f22fb55a2c8c3e80070100547eb9b1a9b51
>
> Relate Notes:
>
> https://github.com/apache/rocketmq-eventbridge/releases/tag/rocketmq-eventbridge-1.1.0
>
>
> The artifacts have been signed with Key :
> F1CF7B3B63003270B3EB5B0B66BFD9C8461F1A4A, which can be found in the keys
> file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
>
>
> Fill in the following:
> [+]  check LICENSE, should be Apache V2
> [+]  check NOTICE, should have a notice for third-party dependency if
> necessary
> [+]  extract the zip and check if the source version is correct
> [+]  verify the asc(PGP sign),SHA512
>
>
> The vote will be open for at least 72 hours or until the necessary number
> of votes are reached.
> Please vote accordingly:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
> Thanks,
> The Apache RocketMQ Team
>

Re: [VOTE] Release Apache RocketMQ 5.2.0 RC1

[Zhanhui Li]

+1

I checked:
  1)  checksum;
  2)  license;
  3) notice;
  4) signature;

On Tue, Feb 13, 2024 at 6:34 PM fuyou  wrote:

> +1
>
>=
>
>   fuyou001
> Best Regards
>
>
>
> jinrongtong 于2024年2月13日 周二18:18写道：
>
>> Hello RocketMQ Community,
>>
>> This is the vote for 5.2.0 RC1 of Apache RocketMQ.
>>
>> This release includes fixes for many issues, enhancements to stability,
>> and also introduces some significant features, including the million-level
>> queue and the implementation of the jraft controller.
>> The artifacts:
>> https://dist.apache.org/repos/dist/dev/rocketmq/5.2.0-rc1/
>>
>> The staging repo:
>> https://repository.apache.org/content/repositories/orgapacherocketmq-1303/
>>
>> Git tag for the release:
>> https://github.com/apache/rocketmq/releases/tag/rocketmq-all-5.2.0
>>
>> Hash for the release tag:
>> 281a7b32f002019525cfa4dd4a4b983b653f070d
>>
>> Release Notes:
>> https://rocketmq.apache.org/zh/release-notes/2024/01/30/5.2.0/
>>
>> The artifacts have been signed with Key :
>> EC9F268B4C20590138B11FE701420E4292296EAE, which can be found in the keys
>> file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
>>
>> Checklist for reference,
>> Note that this is not an official policy but may help with checking
>> releases.
>>
>> Fill in the following:
>> [ ]  check LICENSE, should be Apache V2
>> [ ]  check NOTICE, should have a notice for third-party dependency if
>> necessary
>> [ ]  extract the zip and check if the source version is correct
>> [ ]  verify the asc(PGP sign),SHA512
>> [ ]  build the source, start nameserver and broker according to the
>> quick-start
>> [ ]  run clusterList command to see if the version is correct
>>
>> The vote will be open for at least 72 hours or until the necessary number
>> of votes are reached.
>>
>> Please vote accordingly:
>>
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove with the reason
>>
>> Thanks
>> The Apache RocketMQ Team
>>
>

Re: [VOTE] Release gRPC Java Client for Apache RocketMQ 5.0.6 RC1

[Zhanhui Li]

Hi, tiger lee,

I find the source package at apache release SVN repository
https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-java/5.0.6-rc1/rocketmq-client-java-5.0.6-source-release.zip
differs from github release/tag
https://github.com/apache/rocketmq-clients/archive/refs/tags/java-5.0.6.zip

How can we ensure the consistency? For example, source package in SVN
repository has this file 'rocketmq-client-java-parent-5.0.6/DEPENDENCIES'
while there is no such file in the git repository.

Zhanhui Li

On Mon, Jan 22, 2024 at 9:38 PM aaron ai  wrote:

> +1
>
> tiger lee 于2024年1月22日 周一21:30写道：
>
>> Hello RocketMQ Community,
>>
>> This is the vote for gRPC Java Client for Apache RocketMQ 5.0.6 RC1.
>>
>> Apache RocketMQ gRPC clients 5.x series follow the specs of rocketmq-apis
>> <https://github.com/apache/rocketmq-apis>;, and are built on top of
>> Protocol
>> Buffer and gRPC.
>>
>> The artifact url:
>>
>> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-java/5.0.6-rc1/
>>
>> The staging repo for maven:
>> https://repository.apache.org/content/repositories/orgapacherocketmq-1249/
>>
>> Git tag for the release:
>> https://github.com/apache/rocketmq-clients/releases/tag/java-5.0.6
>>
>> Hash for the release tag:
>> 5ea91c3a23616cd7e1cc36c6a1dc753d8ab58dd3
>>
>> Release Notes:
>> https://github.com/apache/rocketmq-clients/releases/tag/java-5.0.6
>>
>> The artifact have been signed with
>> Key: F32A2D46, which can be found in the
>> keys file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
>>
>> Fill in the following:
>>
>> [+]  check LICENSE, should be Apache V2
>> [+]  check NOTICE, should have a notice for third-party dependency if
>> necessary
>> [+]  extract the zip and check if the source version is correct
>> [+]  verify the asc(PGP sign),SHA512
>>
>> The vote will be open for at least 72 hours or until the necessary number
>> of votes are reached.
>> Please vote accordingly:
>>
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove with the reason
>>
>> Thanks
>> The Apache RocketMQ Team
>>
>

Re: [Vote] Release gRPC Go Client for Apache RocketMQ 5.1.0 RC1

[Zhanhui Li]

+1

Verified hash and tag in the repository;
Verified sha512 checksum;


On Mon, Jan 22, 2024 at 11:39 AM Zhouxiang Zhan  wrote:

> +1
>
> On Tue, Jan 16, 2024 at 4:30 PM jinrongtong 
> wrote:
>
> > +1, I checked[OK] Hash and Tag in GitHub repo is matching the
> current
> > artifacts[OK] The release note is clear
> > At 2024-01-16 16:17:11, "Mad" <1094592...@qq.com.INVALID> wrote:
> > >Hello RocketMQ Community,
> > >
> > >
> > >This is the vote for 5.1.0 RC1 of Apache RocketMQ client for golang.
> > >
> > >
> > >Apache RocketMQ clients 5.x series follow the specs of rocketmq-apis
> > > > Protocol
> > >Buffer and gRPC.
> > >
> > >
> > >The artifact:
> > >
> >
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-golang/5.1.0-rc1/
> > >
> > >
> > >Git tag for the release:
> > >https://github.com/apache/rocketmq-clients/tree/golang/v5.1.0-rc.1
> > >
> > >
> > >Hash for the release tag: 0aaf9f9763d4353f09f1b04f297bf7e0bc24ad90
> > >
> > >
> > >Relate Notes:
> > >
> >
> https://github.com/apache/rocketmq-clients/releases/tag/golang%2Fv5.1.0-rc.1
> > >
> > >
> > >The artifacts have been signed with Key :
> > >
> > >
> > >EC9F268B4C20590138B11FE701420E4292296EAE, which can be found in the keys
> > >file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> > >
> > >
> > >Fill in the following:
> > >
> > >
> > >[+] check LICENSE, should be Apache V2
> > >[+] check NOTICE, should have a notice for third-party dependency
> if
> > >necessary
> > >[+] extract the zip and check if the source version is correct
> > >[+] verify the asc(PGP sign),SHA512
> > >
> > >
> > >
> > >
> > >The vote will be open for at least 72 hours or until the necessary
> number
> > >of votes are reached.
> > >Please vote accordingly:
> > >
> > >
> > >
> > >
> > >[ ] +1 approve
> > >[ ] +0 no opinion
> > >[ ] -1 disapprove with the reason
> > >
> > >
> > >
> > >
> > >Thanks
> > >The Apache RocketMQ Team
> >
>

Re: [DISCUSS][RIP-67] RocketMQ ACL 2.0

[Zhanhui Li]

Hi Ding,

The storage of ACL is primarily kept locally in the Broker, and there are
> two main options: file storage and RocksDB-based storage. In this case,
> RocksDB is used for storage, while file storage is provided as an example
> for reference purposes.


Either embedded RocksDB or yaml file is stored per node.  This raises two
concerns: how the metadata will be propagated to newly added nodes? How to
avoid partial failure and resulting inconsistency among nodes?

There are consensus efforts ongoing, will this this feature be built on top
of them?

Li Zhanhui

On Wed, Nov 15, 2023 at 8:32 PM qiaoao  wrote:

> Good to know! Will the ACL 2.0 support ACL in NameServer?
>
> > 2023年11月15日 19:50，丁双喜  写道：
> >
> > Hi, RocketMQ Community
> >
> > RocketMQ ACL is crucial for ensuring the security of message data. We aim
> > to enhance its capabilities and are preparing to upgrade it to a newer
> > version.
> >
> > We have written the proposal and you can see it by the link below:
> > https://github.com/apache/rocketmq/issues/7560
> >
> > Please reply to this email if you have any questions
> > Best Regards
>
>

Re: [VOTE] RocketMQ-Client-Go 2.1.2 Release

[Zhanhui Li]

+1

On Tue, Aug 8, 2023 at 1:31 PM cserwen  wrote:

> +1
>
> > 在 2023年8月8日，13:28，tiger lee  写道：
> >
> > Hello RocketMQ Community,
> >
> > This is the vote for the RocketMQ-Client-Go 2.1.2 Release.
> > Github repo:
> https://github.com/apache/rocketmq-client-go/tree/v2.1.2.rc2
> >
> > The current version updates mainly:
> > 1. Add filter hooks in consumer
> > 2. Add consumer and producer client add unit support (WithUnitName)
> > 3. Add api GetOffsetDiffMap
> > 4. Add param which like consumeThreadMin of java sdk to control
> consumption
> > rate
> > 5. Add user defined topic dimension consumer limiter
> > 6. Fix goroutine leak
> > 7. Fix query not found
> > 8. Fix consumer doesn't consume message because of blocked on Lock
> > 9. Fix the groupName takes the namespace prefix
> > 10. Fix pq.mutex lock blocked
> >
> > more updates to see :
> > https://github.com/apache/rocketmq-client-go/releases/tag/v2.1.2.rc2
> >
> > The vote will be open for at least 72 hours or until a necessary number
> of
> > votes are reached.
> > Please vote accordingly:
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> > Thanks,
> > The Apache RocketMQ Team
>
>

Re: [VOTE] Release gRPC Rust Client for Apache RocketMQ 5.0.0 RC1

[Zhanhui Li]

+1 binding

I checked license, source tagging, and PGP signature.

On Fri, Jul 28, 2023 at 1:37 PM SSpirits  wrote:

> Hello RocketMQ Community,
>
> This is the vote for 5.0.0 RC1 of Apache RocketMQ client for Rust.
>
> Apache RocketMQ clients 5.x series follow the specs of rocketmq-apis<
> https://github.com/apache/rocketmq-apis>, and are built on top of
> Protocol Buffer and gRPC.
>
> The artifact url:
>
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-rust/5.0.0-rc1/
>
> Crates url:
> https://crates.io/crates/rocketmq/5.0.0-rc1
>
> Git tag for the release:
> https://github.com/apache/rocketmq-clients/releases/tag/rust-5.0.0-rc1
>
> Hash for the release tag:
> 1e7aa3327a056296a887ff2e1d12fee45651fd3f
>
> Release Notes:
> https://github.com/apache/rocketmq-clients/releases/tag/rust-5.0.0-rc1
>
> The artifact have been signed with Key:
> FE25109A4201D6ABA11BB013F0D8856C50824735, which can be found in the keys
> file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
>
> Fill in the following:
>
> [+]  check LICENSE, should be Apache V2
> [+]  check NOTICE, should have a notice for third-party dependency if
> necessary
> [+]  extract the zip and check if the source version is correct
> [+]  verify the asc(PGP sign),SHA512
>
> The vote will be open for at least 72 hours or until the necessary number
> of votes are reached.
> Please vote accordingly:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
> Thanks
> The Apache RocketMQ Team
>

Re: [VOTE]: Release Apache RocketMQ 5.1.0 RC1

[Zhanhui Li]

+1

Verified checksum, signature.

On Sat, Feb 18, 2023 at 2:12 PM Xinyu Zhou  wrote:

> +1 Big features are introduced since this release. Thanks to all the
> contributors.
>
> By the way, your key has been copied to the release KEYS file for your next
> steps as a release manager.
>
>
> On Fri, Feb 17, 2023 at 6:42 PM SSpirits  wrote:
>
> > ＋1
> >
> > Your response would be highly appreciated.
> > SSpirits
> > Email：ad...@lv5.moe
> > 在 2023年2月17日+0800 PM6:34，Qiang Wang ，写道：
> > +1 binding
> >
> > 1. Compiling passed
> > 2. asc checked.
> > 3. shasum exists
> > 4. version correct.
> >
> > Good luck.
> >
> > jinrongtong  于2023年2月17日周五 17:55写道：
> >
> > +1, I Checked[OK] verify the asc(PGP sign),SHA512[OK] maven
> > jar in nexus repo is right
> > At 2023-02-17 17:28:38, "Zhouxiang Zhan"  wrote:
> > Hello RocketMQ Community,
> >
> >
> > This is the vote for 5.1.0 RC1 of Apache RocketMQ.
> >
> >
> > This release contains some important new features, including supporting
> > remoting protocol in proxy, transaction message improvement, tiered
> > storage
> > and etc.
> >
> >
> > *The artifacts:*
> >
> > https://dist.apache.org/repos/dist/dev/rocketmq/5.1.0-rc1
> >
> >
> > *The staging repo:*
> >
> >
> >
> >
> https://repository.apache.org/content/repositories/orgapacherocketmq-1148/org/apache/rocketmq/
> >
> >
> > *Git tag for the release:*
> >
> > https://github.com/apache/rocketmq/tree/rocketmq-all-5.1.0
> >
> >
> > *Hash for the release tag:*
> >
> > b8e9334d2a1ed8868a3b15b63b8ef3198fcb81ce
> >
> >
> > *Release Notes:*
> >
> > https://rocketmq.apache.org/release-notes/2023/02/16/5.1.0/
> >
> >
> > The artifacts have been signed with
> > Key 8213C5B0D057B71E4DFD41806688AEA6B96540B2 which can be found in the
> > keys file:
> >
> > https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> >
> >
> > The vote will be open for at least 72 hours or until necessary number of
> > votes are reached.
> >
> >
> > Please vote accordingly:
> >
> > [ ] +1 approve
> >
> > [ ] +0 no opinion
> >
> > [ ] -1 disapprove with the reason
> >
> >
> > Thanks,
> >
> > The Apache RocketMQ Team
> >
> >
>

Re: [DISCUSS] [RIP-61] RocketMQ EventBridge Runtime

[Zhanhui Li]

+1

On Mon, Jan 30, 2023 at 7:53 PM Xin Wang  wrote:

> +1 Nick work. Thanks shenlin.
>
> Xinyu Zhou  于2023年1月29日周日 22:34写道：
>
> > +1 to start this RIP.
> >
> > In most cases, users need an embedded and stateless runtime to adopt
> > eventbridge in a single-tenant scenario.
> > For large-scale use cases, the rocketmq-connect runtime is another
> option.
> >
> > Regards,
> > yukon
> >
> > On Fri, Jan 27, 2023 at 5:16 PM 沈林 <2011shen...@gmail.com> wrote:
> >
> > > Hello RocketMQ Community:
> > >
> > > We have plans to add Runtime to EventBridge to reduce the dependence on
> > > external services and improve the service experience of EDA
> integration.
> > >
> > > We have written the proposal, and you can see it by the link below:
> > >
> https://github.com/apache/rocketmq-eventbridge/wiki/EventBridge-Runtime
> > >
> > > Please reply to this email if you have any suggestions.
> > >
> > > Your response would be highly appreciated.
> > >
> >
>
>
> --
> Thanks,
> Xin
>

Re: [VOTE] [RIP-57] Tiered storage for RocketMQ

[Zhanhui Li]

+1

On Thu, Dec 15, 2022 at 2:43 PM cserwen  wrote:

> +1
>
> > 在 2022年12月15日，10:53，SSpirits  写道：
> >
> > Hello RocketMQ Community,
> >
> > As discussed in the previous email, we discussed introducing tiered
> storage for RocketMQ, and it's time to start an email thread to enter the
> voting process.
> >
> > Links:
> >
> https://github.com/apache/rocketmq/wiki/RIP-57-Tiered-storage-for-RocketMQ
> >
> > The vote will be open for at least 72 hours or until the necessary
> > number of votes are reached.
> >
> > Please vote accordingly:
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> > Thanks
> >
> > Your response would be highly appreciated.
> > SSpirits
> > Email：ad...@lv5.moe
>

Re: [VOTE] Release gRPC Java Client for Apache RocketMQ 5.0.3 RC1

[Zhanhui Li]

+1

On Thu, Nov 24, 2022 at 12:57 PM lollipop  wrote:

> +1
>
> I have checked all of the four items.
>
> On Wed, Nov 23, 2022 at 11:52 AM aaron ai  wrote:
>
> > Hello RocketMQ Community,
> >
> >
> > This is the vote for 5.0.3 RC1 of Apache RocketMQ client for java.
> >
> >
> > Apache RocketMQ clients 5.x series follow the specs of rocketmq-apis
> > , and are built on top of
> > Protocol
> > Buffer and gRPC.
> >
> >
> > The artifact:
> >
> >
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-java/5.0.3-rc1/
> >
> >
> > The staging repo for maven:
> >
> https://repository.apache.org/content/repositories/orgapacherocketmq-1128/
> >
> >
> > Git tag for the release:
> > https://github.com/apache/rocketmq-clients/releases/tag/java-5.0.3
> >
> >
> > Hash for the release tag: ba5c8a718bbd67d03dad0460df1507c7e88cd0f7
> >
> >
> > Relate Notes:
> > https://github.com/apache/rocketmq-clients/releases/tag/java-5.0.3
> >
> >
> > The artifacts have been signed with Key :
> > 3A11FEBBD64C807A233BF1F27C46C79BD4D29011, which can be found in the keys
> > file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> >
> >
> > Fill in the following:
> >
> >
> > [+]  check LICENSE, should be Apache V2
> >
> >
> > [+]  check NOTICE, should have a notice for third-party dependency if
> > necessary
> >
> >
> > [+]  extract the zip and check if the source version is correct
> >
> >
> > [+]  verify the asc(PGP sign),SHA512
> >
> >
> >
> >
> >
> >
> >
> >
> > The vote will be open for at least 72 hours or until the necessary number
> > of votes are reached.
> >
> >
> > Please vote accordingly:
> >
> >
> >
> >
> >
> > [ ] +1 approve
> >
> >
> > [ ] +0 no opinion
> >
> >
> > [ ] -1 disapprove with the reason
> >
> >
> >
> >
> >
> > Thanks
> >
> >
> > The Apache RocketMQ Team
> >
>

Re: [VOTE][RIP-56] Replace Logging Module with Shaded Logback

[Zhanhui Li]

+1

On Tue, Oct 25, 2022 at 11:07 AM Xinyu Zhou  wrote:

> +1
>
> On Mon, Oct 24, 2022 at 2:58 PM lollipop  wrote:
>
> > +1
> >
> > On Mon, Oct 24, 2022 at 1:53 PM aaron ai  wrote:
> >
> > > Hi, RocketMQ Community,
> > >
> > > As discussed in the previous email, this is the vote for *RIP-56
> Replace
> > > Logging Module with Shaded Logback*.
> > >
> > > links:
> > >
> > >
> >
> https://www.yuque.com/docs/share/b2594b77-25f1-40a9-aada-5afe5488ce83?#%20%E3%80%8ARIP-56%20Replace%20Logging%20Module%20with%20Shaded%20logback%E3%80%8B
> > >
> > > The vote will be open for at least 72 hours or until a necessary number
> > of
> > > votes are reached.
> > >
> > > Please vote accordingly:
> > >
> > > [ ] +1 approve
> > > [ ] +0 no opinion
> > > [ ] -1 disapprove with the reason
> > >
> > >
> > > Best Regards!
> > >
> > > jinrongtong  于2022年10月21日周五 20:31写道：
> > >
> > > > +1
> > > > 在 2022-10-21 17:13:59，"fuyou"  写道：
> > > > >+1
> > > > >
> > > > >zhimin li  于2022年10月21日周五 15:38写道：
> > > > >
> > > > >> +1，We should focus on rocketmq itself
> > > > >>
> > > > >> > 2022年10月21日 11:49，aaron ai  写道：
> > > > >> >
> > > > >> > Hi, RocketMQ Community：
> > > > >> >
> > > > >> > We noticed that the existing logging module is unnecessary, so
> > there
> > > > is a
> > > > >> > proposal to replace it with shaded logback.
> > > > >> >
> > > > >> > You can see more details from
> > > > >> >
> > > >
> > https://www.yuque.com/docs/share/b2594b77-25f1-40a9-aada-5afe5488ce83?#
> > > > >> > 《RIP-56 Replace Logging Module with Shaded logback》
> > > > >> >
> > > > >> > Please reply to this email if you have any questions.
> > > > >>
> > > > >>
> > > > >
> > > > >--
> > > > >   =
> > > > >
> > > > >  fuyou001
> > > > >Best Regards
> > > >
> > >
> >
>

Re: [VOTE][RIP-54] Backporting of Remoting-based Client

[Zhanhui Li]

+1

On Tue, Oct 25, 2022 at 11:07 AM Xinyu Zhou  wrote:

> +1, make the remoting client more stable.
>
> On Mon, Oct 24, 2022 at 1:52 PM aaron ai  wrote:
>
> > Hi, RocketMQ Community,
> >
> > As discussed in the previous email, this is the vote for* RIP-54
> > Backporting of Remoting-based Client*.
> >
> > links:
> > https://www.yuque.com/docs/share/03d824be-8510-4ab4-ad81-d04969174f81?#
> >
> > The vote will be open for at least 72 hours or until a necessary number
> of
> > votes are reached.
> >
> > Please vote accordingly:
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> >
> > Best Regards!
> >
> > aaron ai  于2022年10月19日周三 15:13写道：
> >
> > > Due to the maintenance of shimo docs, I can not get the public share
> > link.
> > > Here is the alternative solutions:
> > >
> > > * google docs:
> > >
> >
> https://docs.google.com/document/d/1Go60u-jPwEx4pfXwRagJQg6qbxAwnufF9IuRcM33suk/edit?usp=sharing
> > > * yuque:
> > >
> https://www.yuque.com/docs/share/03d824be-8510-4ab4-ad81-d04969174f81?#
> > >
> > > aaron ai  于2022年10月19日周三 14:49写道：
> > >
> > >>
> > >>
> > >> aaron ai  于2022年10月19日周三 14:47写道：
> > >>
> > >>> Hi, RocketMQ Community.
> > >>>
> > >>> We intent to migrate some bugfix and several optimizations to
> RocketMQ
> > >>> remoting-based SDK.
> > >>>
> > >>> We have written the proposal and you can see it by the link below:
> > >>> https://shimo.im/docs/Ee32MG7bOEURnZA2
> > >>>
> > >>> Please reply to this email if you have any suggestions.
> > >>>
> > >>
> >
>

Re: [VOTE] [RIP-52] Optimize Building ConsumeQueue

[Zhanhui Li]

Hi,

The previous discussion thread is:
https://lists.apache.org/list?dev@rocketmq.apache.org:lte=1M:Optimize%20Building%20ConsumeQueue

+1


On Mon, Oct 24, 2022 at 10:03 AM nowin...@tom.com  wrote:

> Hello RocketMQ Community,
>
> We discussed Optimize Building ConsumeQueue, and it's time to start an
> email thread to enter the voting process.
>
> Wiki Links:
> https://github.com/apache/rocketmq/wiki/RIP-52-Optimize-Building-ConsumeQueue
> Latest English version:
> https://yu7y22ce7k.feishu.cn/docx/doxcn9A5lPA05AD0oISoB11FL2f
> Latest Chinese version:
> https://yu7y22ce7k.feishu.cn/docx/doxcnltrB7VzUKCqx0yxqMgJ4RM
>
> The vote will be open for at least 72 hours or until a necessary number of
> votes is reached.
>
> Please vote accordingly:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
> Thanks!
>
>
>
> nowin...@tom.com
>

Re: [VOTE] Release Apache RocketMQ EventBridge 1.0.0

[Zhanhui Li]

+1

On Tue, Oct 25, 2022 at 11:06 AM Xinyu Zhou  wrote:

> +1
>
> Remember to copy your key to
> https://dist.apache.org/repos/dist/release/rocketmq/KEYS when this release
> is finished.
>
> Regards,
>
> On Mon, Oct 24, 2022 at 12:33 PM aaron ai  wrote:
>
> > +1
> >
> > lollipop  于2022年10月24日周一 11:16写道：
> >
> > > +1
> > >
> > > I checked:
> > > [OK ]  Checksums and PGP signatures are valid for the source package.
> > > [OK ]  Source code artifacts have correct names matching the current
> > > release.
> > > [OK ]  License and Notice are correct in the source package.
> > >
> > >
> > > On Wed, Oct 19, 2022 at 6:02 PM 沈林 <2011shen...@gmail.com> wrote:
> > >
> > > > Hello RocketMQ Community,
> > > >
> > > > This is the vote for the 1.0.0 release of Apache RocketMQ
> EventBridge.
> > > Also
> > > > it will be the first release of EventBridge. In this version, We will
> > be
> > > > releasing the overall functionality of EventBridge for the first
> time.
> > > > Including:
> > > > 1) Lifecycle management of core resources, including EventSource,
> > > EventBus
> > > > and EventRule.
> > > > 2) The event api gateway for putting the event to EventBridge.
> > > > 3) The Source Runner and Target Runner on RocketMQ Connect.
> > > > 4) The Source and Target class register.
> > > > 5) The Filter and Transform RocketMQ Connect plugin, before
> delivering
> > > the
> > > > event to the target.
> > > > and so on. (Detail: https://github.com/apache/rocketmq-eventbridge)
> > > >
> > > >
> > > > The artifact:
> > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-eventbridge/1.0.0-rc1/
> > > >
> > > >
> > > > Git tag for the release:
> > > >
> > > >
> > >
> >
> https://github.com/apache/rocketmq-eventbridge/releases/tag/rocketmq-eventbridge-1.0.0
> > > >
> > > > Hash for the release tag: e11edf726ddb564c1fc4ade00fc3103c798c1291
> > > >
> > > > Relate Notes:
> > > >
> > > >
> > >
> >
> https://github.com/apache/rocketmq-eventbridge/releases/tag/rocketmq-eventbridge-1.0.0
> > > >
> > > >
> > > > The artifacts have been signed with Key :
> > > > F1CF7B3B63003270B3EB5B0B66BFD9C8461F1A4A, which can be found in the
> > keys
> > > > file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> > > >
> > > >
> > > > Fill in the following:
> > > > [+]  check LICENSE, should be Apache V2
> > > > [+]  check NOTICE, should have a notice for third-party dependency if
> > > > necessary
> > > > [+]  extract the zip and check if the source version is correct
> > > > [+]  verify the asc(PGP sign),SHA512
> > > >
> > > >
> > > > The vote will be open for at least 72 hours or until the necessary
> > number
> > > > of votes are reached.
> > > > Please vote accordingly:
> > > > [ ] +1 approve
> > > > [ ] +0 no opinion
> > > > [ ] -1 disapprove with the reason
> > > >
> > > > Thanks,
> > > > The Apache RocketMQ Team
> > > >
> > >
> >
>

Re: [VOTE] [RIP-48] Enhance server-side offset management ability

[Zhanhui Li]

+1

On Tue, Oct 18, 2022 at 3:46 PM aaron ai  wrote:

> +1
>
> Hu Zongtang  于2022年10月18日周二 14:14写道：
>
> > +1
> > 
> > 发件人: L K 
> > 发送时间: 2022年10月18日 10:43
> > 收件人: dev@rocketmq.apache.org 
> > 主题: 回复: [VOTE] [RIP-48] Enhance server-side offset management ability
> >
> > +1
> > 
> > 发件人: Xinyu Zhou 
> > 发送时间: 2022年10月17日 2:26
> > 收件人: dev@rocketmq.apache.org 
> > 主题: Re: [VOTE] [RIP-48] Enhance server-side offset management ability
> >
> > +1
> >
> > Thanks for this RIP.
> >
> > On Mon, Oct 17, 2022 at 10:23 AM zhimin li  wrote:
> >
> > > Hello RocketMQ Community,
> > >
> > > We discussed enhancing server side offset management ability for
> > > RocketMQ, and it's time to start an email thread to enter the voting
> > > process.
> > > Links:
> > >
> >
> https://github.com/apache/rocketmq/wiki/RIP-48-Enhance-server-side-offset-management-ability
> > >
> > > The vote will be open for at least 72 hours or until the necessary
> > > number of votes are reached.
> > >
> > > Please vote accordingly:
> > > [ ] +1 approve
> > > [ ] +0 no opinion
> > > [ ] -1 disapprove with the reason
> > >
> > > Thanks
> > >
> > > aaron ai  于2022年10月17日周一 10:16写道：
> > > >
> > > > Pleased to see this proposal! The offset management of broker should
> be
> > > > improved urgently.
> > > >
> > > > zhimin li  于2022年10月9日周日 14:29写道：
> > > >
> > > > > Hi, RocketMQ Community,
> > > > >
> > > > > There are some problems in the current implementation of rocketmq
> > > offset
> > > > > management. We propose some solutions to optimize.
> > > > >
> > > > > We have written the proposal and you and see it by the link below:
> > > > >
> > > > > https://shimo.im/docs/XKq4MwKxQESr7akN/ 「RIP 48 Enhance
> server-side
> > > offset
> > > > > management ability」
> > > > >
> > > > > Please reply to this email if you have any questions
> > > > >
> > >
> >
>

Re: Re:Re: [VOTE][RIP-50]RocketMQ Transaction Message Improvement

[Zhanhui Li]

+1

On Tue, Oct 18, 2022 at 4:45 PM fuyou  wrote:

> +1
>
> Hu Zongtang  于2022年10月18日周二 14:14写道：
>
> > +1
> > 
> > 发件人: jinrongto...@163.com  代表 jinrongtong <
> > jinrongt...@apache.org>
> > 发送时间: 2022年10月18日 10:20
> > 收件人: dev@rocketmq.apache.org 
> > 主题: Re:Re: [VOTE][RIP-50]RocketMQ Transaction Message Improvement
> >
> > +1
> > 在 2022-10-18 10:09:20，"heng du"  写道：
> > >+1
> > >
> > >Xinyu Zhou  于2022年10月18日周二 10:05写道：
> > >
> > >> +1
> > >>
> > >> Thanks for this RIP!
> > >>
> > >> On Mon, Oct 17, 2022 at 6:51 PM tihong ruan 
> > wrote:
> > >>
> > >> > As discussed in the previous email, we discussed RocketMQ
> Transaction
> > >> > Message Improvement, and it's time to start an email thread to enter
> > the
> > >> > voting process.
> > >> >
> > >> > Links:
> > >> >
> > >> >
> > >>
> >
> https://github.com/apache/rocketmq/wiki/RIP-50-RocketMQ-Transaction-Message-Improvement
> > >> >
> > >> > The vote will be open for at least 72 hours or until a necessary
> > number
> > >> of
> > >> > votes is reached.
> > >> >
> > >>
> >
>
>
> --
>=
>
>   fuyou001
> Best Regards
>

Re: [DISCUSS] [RIP-52 Optimize Building ConsumeQueue]

[Zhanhui Li]

Hi aaron.ai,

What kind of specific performance profiling data do you expect? Basically,
this RIP is changing single thread ReputMessageService to
multi-stage-multi-threaded version. Something similar in the Linux world is
multiple-queuing block IO mechanism
https://docs.kernel.org/block/blk-mq.html

@nowin...@tom.com   Can we, preliminarily, decompose the
dispatch request time into multiple slices, like Linux Kernel IO
stages(await time, service time...)? This way, guys would roughly get goals
of the optimization.
For example, prepare a handful of commit log files without associated
consume queues, run the recovery process. During the recovery, collect
statistics of each DispatchRequest, including duration in queue, time for
writing consume queue logs

Zhanhui Li

On Mon, Oct 17, 2022 at 5:03 PM aaron ai  wrote:

> Hi, Could you provide more data(performance profiling or another way) to
> support this proposal ?
>
> fuyou  于2022年10月17日周一 15:33写道：
>
> > good idea.
> >
> > but i think Introducing bitmap is complicated, so what's about using hash
> > for topic and share some queue,ReputService access queue to build Consume
> > Queue.
> >
> > Zhanhui Li  于2022年10月17日周一 15:04写道：
> >
> > > Hey, RocketMQ developers and users,
> > >
> > > It's glad and encouraging to see improvement in the building of
> > > ConsumeQueue, which is essential to keep end-to-end latency low, even
> if
> > > workload of RocketMQ broker is pretty heavy. This is one of the places
> > > where RocketMQ really shines among other messaging/streaming products.
> > >
> > > Questions, suggestions and comments are all welcome!
> > >
> > > Best Regards!
> > >
> > > Zhanhui Li
> > >
> > > On Mon, Oct 17, 2022 at 2:55 PM nowin...@tom.com 
> > wrote:
> > >
> > > > Hi, RocketMQ Community
> > > >
> > > > The generation of messages depend on the ReputMessageService single
> > > > thread. When the message production traffic is heavy, the
> ConsumeQueue
> > > > messages are generated slowly, resulting in a delay in message
> > > consumption.
> > > >
> > > > We need to process CommitLog messages concurrently to speed up the
> > > > generation of ConsumeQueue messages.
> > > >
> > > > [RIP-52 Optimize Building ConsumeQueue]
> > > > Wiki <
> > > >
> > >
> >
> https://github.com/apache/rocketmq/wiki/RIP-52-Optimize-Building-ConsumeQueue
> > > > >
> > > > Chinese version <
> > > > https://yu7y22ce7k.feishu.cn/docx/doxcnltrB7VzUKCqx0yxqMgJ4RM>
> > > > English version <
> > > > https://yu7y22ce7k.feishu.cn/docx/doxcn9A5lPA05AD0oISoB11FL2f>
> > > >
> > > > Please reply to this email if you have any questions
> > > > Best Regards
> > > >
> > > >
> > > > nowin...@tom.com
> > > >
> > >
> >
> >
> > --
> >=
> >
> >   fuyou001
> > Best Regards
> >
>

Re: [DISCUSS] [RIP-52 Optimize Building ConsumeQueue]

[Zhanhui Li]

Hey, RocketMQ developers and users,

It's glad and encouraging to see improvement in the building of
ConsumeQueue, which is essential to keep end-to-end latency low, even if
workload of RocketMQ broker is pretty heavy. This is one of the places
where RocketMQ really shines among other messaging/streaming products.

Questions, suggestions and comments are all welcome!

Best Regards!

Zhanhui Li

On Mon, Oct 17, 2022 at 2:55 PM nowin...@tom.com  wrote:

> Hi, RocketMQ Community
>
> The generation of messages depend on the ReputMessageService single
> thread. When the message production traffic is heavy, the ConsumeQueue
> messages are generated slowly, resulting in a delay in message consumption.
>
> We need to process CommitLog messages concurrently to speed up the
> generation of ConsumeQueue messages.
>
> [RIP-52 Optimize Building ConsumeQueue]
> Wiki <
> https://github.com/apache/rocketmq/wiki/RIP-52-Optimize-Building-ConsumeQueue
> >
> Chinese version <
> https://yu7y22ce7k.feishu.cn/docx/doxcnltrB7VzUKCqx0yxqMgJ4RM>
> English version <
> https://yu7y22ce7k.feishu.cn/docx/doxcn9A5lPA05AD0oISoB11FL2f>
>
> Please reply to this email if you have any questions
> Best Regards
>
>
> nowin...@tom.com
>

Re: [DISCUSS] [RIP-49] Some Remoting module improvement

[Zhanhui Li]

+1

On Mon, Oct 17, 2022 at 10:55 AM zhimin li  wrote:

> Hello RocketMQ Community,
>
> I have proposed some remoting module improvement in RIP, and it's time to
> start an email thread to enter the voting process.
> Links:
> https://github.com/apache/rocketmq/wiki/RIP-49-RocketMQ-remoting-module-improvement
>
> The vote will be open for at least 72 hours or until the necessary number
> of votes are reached.
> Please vote accordingly:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
> Thanks
>
> 在 2022/10/17 10:37，“aaron ai” 写入:
>
> +1, Even though gRPC is widely applied in proxy and client,
> remoting-based
> protocol will not be discarded in the foreseeable future.
>
> SSpirits  于2022年10月16日周日 12:31写道：
>
> > +1, moving to vote
> > On Oct 9, 2022, 15:05 +0800, zhimin li , wrote:
> > Hi, RocketMQ Community,
> >
> > Remoting module is a very important component in RocketMQ, which is
> used
> > for component communication. We want to do some optimizations on this
> > module.
> >
> >
> > We have written the proposal and you can see it by the link below:
> >
> > https://shimo.im/docs/B1Aw1lnx7giVJLqm/ 「RIP 49 Some Remoting module
> > improvement」
> >
> >
> > Please reply to this email if you have any questions
> >
>
>
>

Re: [DISCUSSION] RIP-47

[Zhanhui Li]

> IMO, the main reason for storing broker ip is that messages are bound to
broker, not topic.

I am not sure if this the main reason or not... binding messages to node
not topic is obviously a design flaw.

> We must find out which broker the message is stored on before we fetch
the message.  And it is no way to migrate messages from their original
broker to another.

It is indeed something we need to improve. But it deserves a dedicated RIP.


> The new storage layer design should consider this problem, at least not
to add new obstacles.

Definitely... The design of this RIP is to grant maximum freedom to the
business layer and make the storage flexible and adaptable enough.

> Eg. Where to store the mapping of topic name and topic identity? Is this
mapping consistent across brokers?

Ideally, the mapping should be accessible to the whole cluster. The
mapping, at least, needs to be consistent among brokers of the same group.
In the perspective of the storage layer, it expects the topic ID on topic
creation from broker module.


On Sun, Oct 16, 2022 at 11:57 AM SSpirits  wrote:

> Hi Zhanhui,
>
> IMO, the main reason for storing broker ip is that messages are bound to
> broker, not topic. We must find out which broker the message is stored on
> before we fetch the message.  And it is no way to migrate messages from
> their original broker to another.
>
> The new storage layer design should consider this problem, at least not to
> add new obstacles. Eg. Where to store the mapping of topic name and topic
> identity? Is this mapping consistent across brokers?
>
>
> On Oct 9, 2022, 21:52 +0800, Zhanhui Li , wrote:
> Hi,
>
> RocketMQ has stuck to its data format for many years and we are
> experiencing many blocking problems originating from this design.
>
> RIP-47 https://github.com/apache/rocketmq/wiki/RIP-47-Data-Layout-V2 is to
> designed to solve the mentioned issues.
>
> Please review the design. Constructive feedback and comments are
> anticipated.
>
> It is a major RIP to RocketMQ, multiple critical data paths are involved.
> It's welcome to join this RIP if you are interested.
>
> Zhanhui Li
>

Re: [VOTE] Release Apache RocketMQ APIs 2.0.1 RC1

[Zhanhui Li]

+1

On Fri, Oct 14, 2022 at 10:36 AM lollipop  wrote:

> +1,
> I checked:
>
> [OK]  Extract the zip and check if the source version is correct with
> 2.0.1;
> [OK]  The Apache V2 license file and Notice file is included in the
> package.
>
> On Thu, Oct 13, 2022 at 7:48 PM aaron ai  wrote:
>
> > Hello RocketMQ Community,
> >
> > This is the vote for 2.0.1 RC1 of Apache RocketMQ APIs.
> >
> > RocketMQ APIs use Protocol Buffers version 3 (proto3) as their Interface
> > Definition Language (IDL) to define the API interface and the structure
> of
> > the payload messages.
> >
> > The artifacts:
> >
> > https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-proto/2.0.1-rc1
> >
> > The staging repo:
> >
> >
> https://repository.apache.org/content/repositories/orgapacherocketmq-1107/
> >
> > Git tag for the release:
> >
> > https://github.com/apache/rocketmq-apis/releases/tag/v2.0.1
> >
> >
> https://github.com/apache/rocketmq-apis/releases/tag/rocketmq-proto-2.0.1
> >
> > Hash for the release tag:
> >
> > 7ed3e267292ea971064d86acee47a8ef60181121
> >
> > Relate Notes:
> >
> > https://github.com/apache/rocketmq-apis/releases/tag/v2.0.1
> >
> >
> https://github.com/apache/rocketmq-apis/releases/tag/rocketmq-proto-2.0.1
> >
> > The artifacts have been signed with Key :
> >
> > 3A11FEBBD64C807A233BF1F27C46C79BD4D29011, which can be found in the keys
> > file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> >
> > Fill in the following:
> >
> > [+]  check LICENSE, should be Apache V2
> >
> > [+]  check NOTICE, should have a notice for third-party dependency if
> > necessary
> >
> > [+]  extract the zip and check if the source version is correct
> >
> > [+]  verify the asc(PGP sign),SHA512
> >
> >
> >
> >
> >
> >
> >
> > The vote will be open for at least 72 hours or until the necessary number
> > of votes are reached.
> >
> > Please vote accordingly:
> >
> >
> >
> >
> > [ ] +1 approve
> >
> > [ ] +0 no opinion
> >
> > [ ] -1 disapprove with the reason
> >
> >
> >
> >
> > Thanks
> >
> > The Apache RocketMQ Team
> >
>

Re: [DISCUSSION] RIP-47

[Zhanhui Li]

Hi fuyou,

Thanks for your feedback. Basically, if both broker and clients support v2
serialization, we can minimize the cost of serialization and
deserialization a lot. If clients remained v1 and brokers upgraded, extra
overhead would be minimal, within an acceptable boundary.

Here are a few blogs discussing pros and cos of this style of serde, among
implementations.

https://engineering.fb.com/2015/07/31/android/improving-facebook-s-performance-on-android-with-flatbuffers/
https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-sbe.html

Zhanhui Li

On Tue, Oct 11, 2022 at 10:02 AM fuyou  wrote:

> Great RIP,
> one question:
> we thought about  zero copy when consumers pull messages,if we support
> v2,zero copy can't support.
>
> Zhanhui Li  于2022年10月10日周一 15:57写道：
>
> > Hi zhimin,
> >
> > >> 1. About the length of Topic, 128 can meet most scenarios. Even if the
> > >> storage module supports store long topic data,
> > >>I still recommend that the length of this non-system topic‘s
> > >> length should not exceed the 128 limit.
> > >>For system topics, such as the retry topic name of pop consumption
> > >> can increase the upper limit.
> >
> > Storage layer, IMO, is not the best place to apply such limit. Broker
> > module might be a better option; Normally, we should store a logical,
> fixed
> > width number in the storage system and let the upper layer to interpret
> the
> > actual name.
> >
> >
> > >>  For the system properties in the message, there should be its own
> > >> namespace to distinguish it from the user's properties and protect it
> > >> from being modified by the user.
> >
> > Yes! This is what this RIP is going to do.
> >
> > >> Is it necessary to store the server IP in every message? This
> > >> design will waste more storage space.
> >
> > Yes, this is indeed an issue. Actually, two questions need to be
> answered:
> > 1, how does store IP help; 2, How to define store IP?  The first node
> that
> > saves the message or the node that serves the message pull request.
> >
> > >> The "topic remapping" needs to consider the compatibility with the
> > >> existing design, It is recommended to discuss it as another
> > >> improvement.
> >
> > As pointed out in the previous section, most storage system stores
> logical,
> > fixed width number and let the business layer interpret the actual topic
> > name. This matters in this RIP as it decides what to store in the storage
> > tier. Logical fixed with number or string name or both during migration.
> >
> > >> We should consider compression and columnar storage in the new
> > >> storage format.
> >
> > Compression can be made transparent, similar to what RocsDB/LevelDB does
> > for SST files at the bottom most level.
> >
> >
> > On Mon, Oct 10, 2022 at 2:50 PM zhimin li  wrote:
> >
> > > This is a great improvement plan, I have the following thoughts:
> > >
> > > 1. About the length of Topic, 128 can meet most scenarios. Even if the
> > > storage module supports store long topic data,
> > > I still recommend that the length of this non-system topic‘s
> > > length should not exceed the 128 limit.
> > > For system topics, such as the retry topic name of pop consumption
> > > can increase the upper limit.
> > >
> > > 2. For the system properties in the message, there should be its own
> > > namespace to distinguish it from the user's properties and protect it
> > > from being modified by the user.
> > >
> > > 3. Is it necessary to store the server IP in every message? This
> > > design will waste more storage space.
> > >
> > > 4. The "topic remapping" needs to consider the compatibility with the
> > > existing design, It is recommended to discuss it as another
> > > improvement.
> > >
> > > 5. We should consider compression and columnar storage in the new
> > > storage format.
> > >
> >
>
>
> --
>=
>
>   fuyou001
> Best Regards
>

Re: [DISCUSSION] RIP-47

[Zhanhui Li]

Hi zhimin,

>> 1. About the length of Topic, 128 can meet most scenarios. Even if the
>> storage module supports store long topic data,
>>I still recommend that the length of this non-system topic‘s
>> length should not exceed the 128 limit.
>>For system topics, such as the retry topic name of pop consumption
>> can increase the upper limit.

Storage layer, IMO, is not the best place to apply such limit. Broker
module might be a better option; Normally, we should store a logical, fixed
width number in the storage system and let the upper layer to interpret the
actual name.


>>  For the system properties in the message, there should be its own
>> namespace to distinguish it from the user's properties and protect it
>> from being modified by the user.

Yes! This is what this RIP is going to do.

>> Is it necessary to store the server IP in every message? This
>> design will waste more storage space.

Yes, this is indeed an issue. Actually, two questions need to be answered:
1, how does store IP help; 2, How to define store IP?  The first node that
saves the message or the node that serves the message pull request.

>> The "topic remapping" needs to consider the compatibility with the
>> existing design, It is recommended to discuss it as another
>> improvement.

As pointed out in the previous section, most storage system stores logical,
fixed width number and let the business layer interpret the actual topic
name. This matters in this RIP as it decides what to store in the storage
tier. Logical fixed with number or string name or both during migration.

>> We should consider compression and columnar storage in the new
>> storage format.

Compression can be made transparent, similar to what RocsDB/LevelDB does
for SST files at the bottom most level.


On Mon, Oct 10, 2022 at 2:50 PM zhimin li  wrote:

> This is a great improvement plan, I have the following thoughts:
>
> 1. About the length of Topic, 128 can meet most scenarios. Even if the
> storage module supports store long topic data,
> I still recommend that the length of this non-system topic‘s
> length should not exceed the 128 limit.
> For system topics, such as the retry topic name of pop consumption
> can increase the upper limit.
>
> 2. For the system properties in the message, there should be its own
> namespace to distinguish it from the user's properties and protect it
> from being modified by the user.
>
> 3. Is it necessary to store the server IP in every message? This
> design will waste more storage space.
>
> 4. The "topic remapping" needs to consider the compatibility with the
> existing design, It is recommended to discuss it as another
> improvement.
>
> 5. We should consider compression and columnar storage in the new
> storage format.
>

Re: [DISCUSSION] RIP-47

[Zhanhui Li]

Hi Zhendong,

Nice to see your feedback.

In terms of Batched Messages. I believe we can solve it this way

In message headers IDL, we have an enum, flagging if the data layout is
single or batch/composite or other format. If the enum says single, then
the body is user data. If enum says composite, then body of the frame
should be considered as several nested messages. The format of the nested
messages may be some compact representation.  We may add one more task,
tackling this issue.

Zhanhui Li


On Mon, Oct 10, 2022 at 11:13 AM dongeforever 
wrote:

> Nice try.
> Looking forward to the detailed layout.
>
> By the way, the new data format may need to consider more about the batch
> record, several messages in one record end to end (from producers to
> consumers).
>
>
> Zhanhui Li  于2022年10月9日周日 21:52写道：
>
> > Hi,
> >
> > RocketMQ has stuck to its data format for many years and we are
> > experiencing many blocking problems originating from this design.
> >
> > RIP-47 https://github.com/apache/rocketmq/wiki/RIP-47-Data-Layout-V2 is
> to
> > designed to solve the mentioned issues.
> >
> > Please review the design. Constructive feedback and comments are
> > anticipated.
> >
> > It is a major RIP to RocketMQ, multiple critical data paths are involved.
> > It's welcome to join this RIP if you are interested.
> >
> > Zhanhui Li
> >
>

[DISCUSSION] RIP-47

[Zhanhui Li]

Hi,

RocketMQ has stuck to its data format for many years and we are
experiencing many blocking problems originating from this design.

RIP-47 https://github.com/apache/rocketmq/wiki/RIP-47-Data-Layout-V2 is to
designed to solve the mentioned issues.

Please review the design. Constructive feedback and comments are
anticipated.

It is a major RIP to RocketMQ, multiple critical data paths are involved.
It's welcome to join this RIP if you are interested.

Zhanhui Li

Re: [Vote] Release gRPC Java Client for Apache RocketMQ 5.0.2 RC1

[Zhanhui Li]

+1

I verified all of the four items.

On Sat, Oct 8, 2022 at 11:32 AM Hu Zongtang  wrote:

> +1
> 
> 发件人: aaron ai 
> 发送时间: 2022年9月30日 11:00
> 收件人: dev@rocketmq.apache.org 
> 主题: Re: [Vote] Release gRPC Java Client for Apache RocketMQ 5.0.2 RC1
>
> OK
>
> Xinyu Zhou  于2022年9月29日周四 17:30写道：
>
> > +1
> >
> > All is fine, please don't forget to add the release records here:
> >
> >
> https://github.com/apache/rocketmq-site/blob/new-official-website/src/pages/download.md
> >
> > On Wed, Sep 28, 2022 at 4:02 PM aaron ai  wrote:
> >
> > > Hello RocketMQ Community,
> > >
> > >
> > > This is the vote for 5.0.2 RC1 of Apache RocketMQ client for java.
> > >
> > >
> > > Apache RocketMQ clients 5.x series follow the specs of rocketmq-apis
> > > , and are built on top of
> > > Protocol
> > > Buffer and gRPC.
> > >
> > >
> > > The artifact:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-java/5.0.2-rc1/
> > >
> > >
> > > The staging repo for maven:
> > >
> >
> https://repository.apache.org/content/repositories/orgapacherocketmq-1106/
> > >
> > >
> > > Git tag for the release:
> > > https://github.com/apache/rocketmq-clients/tree/java-5.0.2
> > >
> > >
> > > Hash for the release tag: 1cda3319b637f225bdcdce9683ddbde2bdb6038e
> > >
> > >
> > > Relate Notes:
> > > https://github.com/apache/rocketmq-clients/releases/tag/java-5.0.2
> > >
> > >
> > > The artifacts have been signed with Key :
> > > 3A11FEBBD64C807A233BF1F27C46C79BD4D29011, which can be found in the
> keys
> > > file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> > >
> > >
> > > Fill in the following:
> > >
> > >
> > > [+]  check LICENSE, should be Apache V2
> > >
> > >
> > > [+]  check NOTICE, should have a notice for third-party dependency if
> > > necessary
> > >
> > >
> > > [+]  extract the zip and check if the source version is correct
> > >
> > >
> > > [+]  verify the asc(PGP sign),SHA512
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > The vote will be open for at least 72 hours or until the necessary
> number
> > > of votes are reached.
> > >
> > >
> > > Please vote accordingly:
> > >
> > >
> > >
> > >
> > >
> > > [ ] +1 approve
> > >
> > >
> > > [ ] +0 no opinion
> > >
> > >
> > > [ ] -1 disapprove with the reason
> > >
> > >
> > >
> > >
> > >
> > > Thanks
> > >
> > >
> > > The Apache RocketMQ Team
> > >
> >
>

Re: [DISCUSS][RIP-46] Observability improvement for RocketMQ

[Zhanhui Li]

Hi,

>  this RIP is based on the rocketmq-exporter and Prometheus, will you work
focus on the metrics collector and specification of metric?

No. As the goal section of RIP-46 states, it is a redesign, aiming to
provide systematic observability in metric dimensions.

>  And I suggest you can implment the elasticity of broker and proxy pod in
the k8s cluster base on the some other component, such as HPA;

Not quite understand your concern here. Could you please elaborate it
further?

> And we can disscuss about this part in online meeting of Apache rocketmq!

Best to have the discussion in the mailing list following Apache Way.  *Apache
projects use mailing lists to coordinate development of their software and
administer their organization. Mailing lists also serve as a primary
support channel where users can help each other learn to use the
software.  *https://www.apache.org/foundation/mailinglists.html

Not everyone has the schedule to attend online meetings.  It's OK to have
online meetings, be private or public. But they should not be considered
formal and necessary.

Zhanhui Li


On Mon, Sep 26, 2022 at 3:58 PM Hu Zongtang  wrote:

> I look your desgin document of RIP-46 and have some serveral questions
> below:
>
> (1)Actually, this RIP is based on the rocketmq-exporter and Prometheus,
> will you work focus on the metrics collector and specification of metric?
> (2)And I suggest you can implment the elasticity of broker and proxy pod
> in the k8s cluster base on the some other component, such as HPA; And we
> can disscuss about this part in online meeting of Apache rocketmq!
>
> 
> 发件人: SSpirits 
> 发送时间: 2022年9月26日 13:38
> 收件人: dev@rocketmq.apache.org 
> 主题: [DISCUSS][RIP-46] Observability improvement for RocketMQ
>
> Hi, RocketMQ Community：
>
> We have a plan to improve the observability of RocketMQ. Specifically, we
> hope to introduce native metrics for RocketMQ.
>
> We have written the proposal, and you can see it by the link below:
> https://shimo.im/docs/m4kML2VzdESO9eqD
>
> Please reply to this email if you have any suggestions.
>
> SSpirits
> Email：ad...@lv5.moe<mailto:ad...@lv5.moe>
>

[ANNOUNCEMENT] Release RocketMQ Client Golang 5.0.0 & Release RocketMQ Client C# 5.0.0

[Zhanhui Li]

Hi RocketMQ community,

This is to announce the releases of RocketMQ Client Golang 5.0.0 and the
Release of RocketMQ Client C# 5.0.0

Released artifacts can be found here:
https://dist.apache.org/repos/dist/release/rocketmq/rocketmq-clients/rocketmq-client-csharp/5.0.0/
https://dist.apache.org/repos/dist/release/rocketmq/rocketmq-clients/rocketmq-client-golang/5.0.0/

General change logs may be  found as below:
https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-golang-5.0.0
https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-csharp-5.0.0

Zhanhui Li

Re: Re: Re: Re: [VOTE]: Release Apache RocketMQ C# SDK 5.0.0 RC1

[Zhanhui Li]

Hi RocketMQ community,

The vote is now closed, collecting 3 "+1" votes.  One withdrawn -1 is
reported.

Three positive votes are:
yukong (binding),
hill(binding),
arron.ai(binding),

One withdrawn -1 from jinrngtong.

According to https://www.apache.org/foundation/voting.html
"Votes on whether a package is ready to release use majority approval
<https://www.apache.org/foundation/glossary.html#MajorityApproval> -- i.e.
at least three PMC members must vote affirmatively for release, and there
must be more positive than negative votes. Releases may not be vetoed.
Generally
the community will cancel the release vote if anyone identifies serious
problems, but in most cases the ultimate decision lies with the individual
serving as release manager. The specifics of the process may vary from
project to project, but the 'minimum quorum of three +1 votes' rule is
universal."

As this release manager, I decide to go and release it.

The release will be published soon.

On Wed, Sep 21, 2022 at 3:01 PM jinrongtong  wrote:

> I see. I don't know much about C #, but if most users can download the
> correct package by other ways, then I believe it is acceptable and would
> withdraw my vote. In addition, I don't think this will block the release
> process too much. It is fast to repackage and restart an RC2 process.
> At 2022-09-21 09:56:23, "Xinyu Zhou"  wrote:
> >Hi, rongtong,
> >
> >I also noticed this issue, but don't let it block our release process,
> it's
> >not a big deal, we can fix it in the next release since many users are
> >waiting for this release and most of them will download the release
> >from NuGet. Does that make sense to you?
> >
> >Regards,
> >Xinyu Zhou
> >
> >On Tue, Sep 20, 2022 at 3:17 PM jinrongtong 
> wrote:
> >
> >> -1, IMO, it is inappropriate to include the implementation of other
> >> languages in the source code package when releasing C# SDK, which will
> >> mislead users. It would be better to repackage.
> >> 在 2022-09-19 16:31:02，"aaron ai"  写道：
> >> >+1
> >> >
> >> >hill  于2022年9月19日周一 16:30写道：
> >> >
> >> >> +1
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> 在 2022-09-19 15:25:52，"Xinyu Zhou"  写道：
> >> >> >+1
> >> >> >
> >> >> >On Sat, Sep 17, 2022 at 1:15 AM Konstantin Kolinko <
> >> >> knst.koli...@gmail.com>
> >> >> >wrote:
> >> >> >
> >> >> >> BCC d...@community.apache.org
> >> >> >>
> >> >> >> Forwarding to the correct mailing list.
> >> >> >>
> >> >> >> -- Forwarded message -
> >> >> >> От: Zhanhui Li 
> >> >> >> Date: пт, 16 сент. 2022 г. в 18:17
> >> >> >> Subject: [VOTE]: Release Apache RocketMQ C# SDK 5.0.0 RC1
> >> >> >> To: 
> >> >> >>
> >> >> >>
> >> >> >> Hello RocketMQ Community,
> >> >> >>
> >> >> >> This is the vote for 5.0.0 of Apache RocketMQ C# SDK.
> >> >> >>
> >> >> >> C# SDK 5.x series are built on top of
> >> >> >> https://github.com/apache/rocketmq-apis, following the same
> concept
> >> and
> >> >> >> model with those of Java/C++ 5.x;
> >> >> >>
> >> >> >>
> >> >> >> The artifacts:
> >> >> >> https//
> >> >> >>
> >> >> >>
> >> >>
> >>
> dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-csharp/5.0.0-rc1
> >> >> >>
> >> >> >> Git tag for the release:
> >> >> >>
> >> >> >>
> >> >>
> >>
> https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-csharp-5.0.0-rc1
> >> >> >>
> >> >> >> Hash for the release tag:
> >> >> >> 3914b66539ca5216f9f82332769dc76ddac27f5d
> >> >> >>
> >> >> >> Release Notes:
> >> >> >> [1] Support publishing standard/FIFO/timed messages;
> >> >> >>
> >> >> >> [2] Support consuming messages through SimpleConsumer;
> >> >> >>
> >> >> >> [3] Support OpenTelemetry compatible metrics exporting;
> >> >> >>
> >> >> >> The artifacts have been signed by Key:
> >> >> >> 462BE3C1C0231675374D0D93B96ECC1229F96FDD, which can be found in
> the
> >> key
> >> >> >> file:
> >> >> >> https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> >> >> >>
> >> >> >> The vote will be open for at least 72 hours or until the necessary
> >> >> number
> >> >> >> of votes is collected.
> >> >> >>
> >> >> >> Please vote accordingly:
> >> >> >>
> >> >> >> [ ] +1 approve
> >> >> >> [ ] +0 no opinion
> >> >> >> [ ] -1 disapprove with the reason
> >> >> >>
> >> >> >> Thanks,
> >> >> >> The Apache RocketMQ Team
> >> >> >>
> >> >>
> >>
>

Re: [VOTE]: Release Apache RocketMQ Golang SDK 5.0.0 RC1

[Zhanhui Li]

Hi RocketMQ community,

The vote is now closed, collecting 5 "+1" votes.  No 0 or -1 is reported.

Five positive votes are:
yukong (binding),
hill(binding),
arron.ai(binding),
heng du(binding),
lollipop(binding).


The release will be published soon.

Zhanhui Li



On Tue, Sep 20, 2022 at 10:02 AM lollipop  wrote:

> +1
>
> On Mon, Sep 19, 2022 at 5:07 PM heng du  wrote:
>
> > +1
> > I checked：
> > (1).License and Notice are correct in the source package.
> > (2).All files have license headers in source package if necessary.
> > (3)Hash and Tag in GitHub repo is matching the current artifacts.
> >
> > Xinyu Zhou  于2022年9月19日周一 16:05写道：
> >
> > > +1
> > >
> > > A small problem: the artifact contains other language implementations,
> > > remember to remove it in the next release.
> > >
> > > On Fri, Sep 16, 2022 at 11:20 PM Zhanhui Li 
> wrote:
> > >
> > > > Hello RocketMQ Community,
> > > >
> > > > This is the vote for 5.0.0 of Apache RocketMQ Golang SDK.
> > > >
> > > > Golang SDK 5.x series are built on top of
> > > > https://github.com/apache/rocketmq-apis, following the same concept
> > and
> > > > model as those of Java/C++ 5.x;
> > > >
> > > > The artifacts:
> > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-golang/5.0.0-rc1
> > > >
> > > > Git tag for the release:
> > > >
> > > >
> > >
> >
> https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-golang-5.0.0-rc1
> > > >
> > > > Hash for the release tag:
> > > > 3914b66539ca5216f9f82332769dc76ddac27f5d
> > > >
> > > > Release Notes:
> > > > [1] Support publishing standard/FIFO/timed messages;
> > > >
> > > > [2] Support consuming messages through SimpleConsumer;
> > > >
> > > > [3] Support OpenTelemetry compatible metrics exporting;
> > > >
> > > > The artifacts have been signed by Key:
> > > > 462BE3C1C0231675374D0D93B96ECC1229F96FDD, which can be found in the
> key
> > > > file:
> > > > https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> > > >
> > > > The vote will be open for at least 72 hours or until the necessary
> > number
> > > > of votes is collected.
> > > >
> > > > Please vote accordingly:
> > > >
> > > > [ ] +1 approve
> > > > [ ] +0 no opinion
> > > > [ ] -1 disapprove with the reason
> > > >
> > > > Thanks,
> > > > The Apache RocketMQ Team
> > > >
> > >
> >
>

[VOTE]: Release Apache RocketMQ Golang SDK 5.0.0 RC1

[Zhanhui Li]

Hello RocketMQ Community,

This is the vote for 5.0.0 of Apache RocketMQ Golang SDK.

Golang SDK 5.x series are built on top of
https://github.com/apache/rocketmq-apis, following the same concept and
model as those of Java/C++ 5.x;

The artifacts:
https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-golang/5.0.0-rc1

Git tag for the release:
https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-golang-5.0.0-rc1

Hash for the release tag:
3914b66539ca5216f9f82332769dc76ddac27f5d

Release Notes:
[1] Support publishing standard/FIFO/timed messages;

[2] Support consuming messages through SimpleConsumer;

[3] Support OpenTelemetry compatible metrics exporting;

The artifacts have been signed by Key:
462BE3C1C0231675374D0D93B96ECC1229F96FDD, which can be found in the key
file:
https://dist.apache.org/repos/dist/dev/rocketmq/KEYS

The vote will be open for at least 72 hours or until the necessary number
of votes is collected.

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Thanks,
The Apache RocketMQ Team

Re: [DISSCUSS] New Apache RocketMQ official website released

[Zhanhui Li]

>  And a vote will be launched in the near future.

This is NOT how Apache Way works. Vote BEFORE act, not AFTER.

Also, it's a good manner to clarify concerns in the discussion before
actually conducting the proposed changes.
It takes time and efforts to figure out issues and most of them are
constructive. Silently ignoring them is not consistent with the way we
interact with the community.

On Tue, Sep 6, 2022 at 7:05 PM 周波  wrote:

> The new official website has been designed and developed for a period of
> time and has been pre-launched recently. And a vote will be launched in the
> near future.
>
> The problems found in the early stage have also been repaired. The
> community found that there are indeed some problems in the new official
> website. The problem of sending letters after the launch has also been
> urgently repaired, and the official website will continue to be optimized.
> In addition, not all problems are bugs. For example, the search function
> needs to be officially launched and bound to a domain name before it can be
> used. The default display is Chinese, because the community found that
> there are many Chinese developers. To view the English official website,
> you only need to switch the language. This is also the new official
> website. ability to support. Although the Chinese and English documents are
> not complete, they are more complete than the old documents, and the
> documents will continue to be improved in the community.
>
> If you have better ideas and designs, you can also directly submit pr.
>
> heng du  于2022年9月6日周二 17:42写道：
>
> > @Zhanhui Li
> >
> > The new official website provides support for both Chinese and English,
> > https://rocketmq.apache.org/en/.
> >
> > The new official website is based on retaining the original content. It
> not
> > only brings a brand new interface but also adds a lot of documents. It
> was
> > completed by the community developers.  At present, there are indeed
> > problems of one kind or another. The above problems you mentioned can be
> > improved by submitting PRs, and more developers will be involved to help
> > this website mature. Our goal is to make things happen. happen and move
> in
> > the right direction
> >
> > Zhanhui Li  于2022年9月5日周一 11:10写道：
> >
> > > Hi,
> > >
> > > I found the preview website has been brought online without a vote and
> > > dozens of bugs are not fixed, which causes RocketMQ inaccessible to
> > > non-Chinese speakers.
> > >
> > > I suggest to revert the online action, postpone the online action until
> > the
> > > new website is mature enough.
> > >
> > > Zhanhui Li
> > >
> > > On Mon, Aug 22, 2022 at 10:25 AM Zhanhui Li 
> wrote:
> > >
> > > > After a quick preview of the proposing sites, I found a few issues:
> > > > 1. https://rocketmq.netlify.app/en/docs/ the comparison table should
> > be
> > > > transposed.
> > > > 2. "Contribute Instruction" should be "Contribution Guide"
> > > > 3. "Contribute" points to the Team panels, the menu item is
> > inconsistent
> > > > with its content;
> > > > 4.  Guess it is a good idea to have a breadcrumb section
> > > > 5. Download page, mixing 4.x and 5.x releases looks less neat. Better
> > to
> > > > separate them;
> > > > 6. When English is selected, a lot of Chinese menus/titles are still
> > > > shown;
> > > > 7. Search is not working yet; Search field is editable;
> > > >
> > > > Zhanhui Li
> > > >
> > > > On Mon, Aug 22, 2022 at 10:13 AM 周波  wrote:
> > > >
> > > >> Thank you for your affirmation and suggestions for the work of the
> new
> > > >> website. The new website has not been completed. The link provided
> is
> > > only
> > > >> a demo, not the final version. It will be checked before it goes
> > online.
> > > >> Please continue to pay attention
> > > >>
> > > >> Xiaorui Wang  于2022年8月21日周日 22:27写道：
> > > >>
> > > >> > Thank you all for contributing to the new website project. I am
> > > >> delighted
> > > >> > to see the new RocketMQ website.
> > > >> >
> > > >> >
> > > >> > I thoroughly read every page on the new site, which features a
> > dynamic
> > > >> > design, rich content and easy navigation to visitors. BTW, I
> didn't
> > > s

Re: [DISSCUSS] New Apache RocketMQ official website released

[Zhanhui Li]

Hi,

I found the preview website has been brought online without a vote and
dozens of bugs are not fixed, which causes RocketMQ inaccessible to
non-Chinese speakers.

I suggest to revert the online action, postpone the online action until the
new website is mature enough.

Zhanhui Li

On Mon, Aug 22, 2022 at 10:25 AM Zhanhui Li  wrote:

> After a quick preview of the proposing sites, I found a few issues:
> 1. https://rocketmq.netlify.app/en/docs/ the comparison table should be
> transposed.
> 2. "Contribute Instruction" should be "Contribution Guide"
> 3. "Contribute" points to the Team panels, the menu item is inconsistent
> with its content;
> 4.  Guess it is a good idea to have a breadcrumb section
> 5. Download page, mixing 4.x and 5.x releases looks less neat. Better to
> separate them;
> 6. When English is selected, a lot of Chinese menus/titles are still
> shown;
> 7. Search is not working yet; Search field is editable;
>
> Zhanhui Li
>
> On Mon, Aug 22, 2022 at 10:13 AM 周波  wrote:
>
>> Thank you for your affirmation and suggestions for the work of the new
>> website. The new website has not been completed. The link provided is only
>> a demo, not the final version. It will be checked before it goes online.
>> Please continue to pay attention
>>
>> Xiaorui Wang  于2022年8月21日周日 22:27写道：
>>
>> > Thank you all for contributing to the new website project. I am
>> delighted
>> > to see the new RocketMQ website.
>> >
>> >
>> > I thoroughly read every page on the new site, which features a dynamic
>> > design, rich content and easy navigation to visitors. BTW, I didn't see
>> the
>> > Origin page [1] which was on the official website before. I'm not sure
>> if
>> > you accidentally missed it. While the original page is helpful for
>> learning
>> > more about RocketMQ, I still advise combining it with team menu.
>> >
>> > [1] https://rocketmq.apache.org/about/origin/
>> >
>> > Best regards,
>> >
>> > Xiaorui Wang[#] - Apache RocketMQ PMC chair
>> > [#] https://github.com/vintagewang
>> >
>> >
>> > On Sun, Aug 21, 2022 at 6:32 PM 周波  wrote:
>> >
>> > > Hi, RocketMQ Community:
>> > >
>> > > The Apache RocketMQ official website has been with us for many years
>> and
>> > > has become an important channel for many developers to know,
>> understand
>> > and
>> > > learn RocketMQ. We are well aware of the importance of the official
>> > website
>> > > to RocketMQ. With the continuous evolution and iteration of the
>> RocketMQ
>> > > version, we will soon usher in the release of the major version 5.0.
>> The
>> > > Apache RocketMQ official website has also ushered in a major upgrade,
>> > with
>> > > a new homepage, new multi-language and multi-version support, and the
>> new
>> > > official website will be released soon, stay tuned.
>> > >
>> > > Early access to the latest official website.
>> > > https://rocketmq.netlify.app/
>> > >
>> > > Thanks
>> > > zhou bo
>> > >
>> >
>>
>

Re: Re:Re: [Vote] Release gRPC Java Client for Apache RocketMQ 5.0.1 RC1

[Zhanhui Li]

+1

On Wed, Aug 31, 2022 at 9:45 PM jinrongtong  wrote:

> +1, I checked[OK] Hash and Tag in GitHub repo is matching the
> current artifacts[OK] Maven jar in nexus repo is
> right[OK] The release note is clear[OK] verify the
> asc(PGP sign)
> At 2022-08-31 14:52:22, "hill"  wrote:
> >+1
> >
> >At 2022-08-31 13:32:54, "Xiaorui Wang"  wrote:
> >>+1
> >>
> >>yukon 于2022年8月31日 周三14:19写道：
> >>
> >>> +1
> >>>
> >>> On Mon, Aug 29, 2022 at 10:55 AM aaron ai 
> wrote:
> >>>
> >>> > Hello RocketMQ Community,
> >>> >
> >>> > This is the vote for 5.0.1 RC1 of Apache RocketMQ client for java.
> >>> >
> >>> > Apache RocketMQ clients 5.x series follow the specs of rocketmq-apis
> >>> > , and are built on top of
> >>> > Protocol
> >>> > Buffer and gRPC.
> >>> >
> >>> > The artifact:
> >>> >
> >>> >
> >>>
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-clients/rocketmq-client-java/5.0.1-rc1/
> >>> >
> >>> > The staging repo for maven:
> >>> >
> >>>
> https://repository.apache.org/content/repositories/orgapacherocketmq-1097/
> >>> >
> >>> > Git tag for the release:
> >>> >
> >>>
> https://github.com/apache/rocketmq-clients/tree/rocketmq-client-java-5.0.1
> >>> >
> >>> > Hash for the release tag: 62d0deb2acfc7adf827f7d636581b6652b1e3b53
> >>> >
> >>> > Relate Notes:
> >>> >
> >>> >
> >>>
> https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-java-5.0.1
> >>> >
> >>> > The artifacts have been signed with Key :
> >>> >
> >>> > 3A11FEBBD64C807A233BF1F27C46C79BD4D29011, which can be found in the
> keys
> >>> > file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> >>> >
> >>> > Fill in the following:
> >>> >
> >>> > [+]  check LICENSE, should be Apache V2
> >>> >
> >>> > [+]  check NOTICE, should have a notice for third-party dependency if
> >>> > necessary
> >>> >
> >>> > [+]  extract the zip and check if the source version is correct
> >>> >
> >>> > [+]  verify the asc(PGP sign),SHA512
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > The vote will be open for at least 72 hours or until the necessary
> number
> >>> > of votes are reached.
> >>> >
> >>> > Please vote accordingly:
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > [ ] +1 approve
> >>> >
> >>> > [ ] +0 no opinion
> >>> >
> >>> > [ ] -1 disapprove with the reason
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > Thanks
> >>> >
> >>> > The Apache RocketMQ Team
> >>> >
> >>>
> >>--
> >>
> >>Best regards,
> >>
> >>Xiaorui Wang[#] - Apache RocketMQ PMC chair
> >>[#] https://github.com/vintagewang
>

Re: [DISSCUSS] New Apache RocketMQ official website released

[Zhanhui Li]

After a quick preview of the proposing sites, I found a few issues:
1. https://rocketmq.netlify.app/en/docs/ the comparison table should be
transposed.
2. "Contribute Instruction" should be "Contribution Guide"
3. "Contribute" points to the Team panels, the menu item is inconsistent
with its content;
4.  Guess it is a good idea to have a breadcrumb section
5. Download page, mixing 4.x and 5.x releases looks less neat. Better to
separate them;
6. When English is selected, a lot of Chinese menus/titles are still shown;
7. Search is not working yet; Search field is editable;

Zhanhui Li

On Mon, Aug 22, 2022 at 10:13 AM 周波  wrote:

> Thank you for your affirmation and suggestions for the work of the new
> website. The new website has not been completed. The link provided is only
> a demo, not the final version. It will be checked before it goes online.
> Please continue to pay attention
>
> Xiaorui Wang  于2022年8月21日周日 22:27写道：
>
> > Thank you all for contributing to the new website project. I am delighted
> > to see the new RocketMQ website.
> >
> >
> > I thoroughly read every page on the new site, which features a dynamic
> > design, rich content and easy navigation to visitors. BTW, I didn't see
> the
> > Origin page [1] which was on the official website before. I'm not sure if
> > you accidentally missed it. While the original page is helpful for
> learning
> > more about RocketMQ, I still advise combining it with team menu.
> >
> > [1] https://rocketmq.apache.org/about/origin/
> >
> > Best regards,
> >
> > Xiaorui Wang[#] - Apache RocketMQ PMC chair
> > [#] https://github.com/vintagewang
> >
> >
> > On Sun, Aug 21, 2022 at 6:32 PM 周波  wrote:
> >
> > > Hi, RocketMQ Community:
> > >
> > > The Apache RocketMQ official website has been with us for many years
> and
> > > has become an important channel for many developers to know, understand
> > and
> > > learn RocketMQ. We are well aware of the importance of the official
> > website
> > > to RocketMQ. With the continuous evolution and iteration of the
> RocketMQ
> > > version, we will soon usher in the release of the major version 5.0.
> The
> > > Apache RocketMQ official website has also ushered in a major upgrade,
> > with
> > > a new homepage, new multi-language and multi-version support, and the
> new
> > > official website will be released soon, stay tuned.
> > >
> > > Early access to the latest official website.
> > > https://rocketmq.netlify.app/
> > >
> > > Thanks
> > > zhou bo
> > >
> >
>

Re: [ANNOUNCE]New Committers of Apache RocketMQ: Zhangheng Huang(hzh0425)

[Zhanhui Li]

Congratulations!

Zhanhui Li

On Thu, Aug 11, 2022 at 10:24 AM yukon  wrote:

> Congrats, Zhangheng
>
> Regards,
> yukon
>
> On Thu, Aug 11, 2022 at 10:17 AM Hu Zongtang 
> wrote:
>
> > Congrats, guys :-)
> > 
> > 发件人: Xiaorui Wang 
> > 发送时间: 2022年8月8日 11:36
> > 收件人: dev@rocketmq.apache.org 
> > 抄送: us...@rocketmq.apache.org 
> > 主题: Re: [ANNOUNCE]New Committers of Apache RocketMQ: Zhangheng
> > Huang(hzh0425)
> >
> > Congratulations to be a member of the RocketMQ community！
> > Looking forward to building the community together and developing our
> > products a step further！
> >
> > Best regards,
> >
> > Xiaorui Wang[#] - Apache RocketMQ PMC chair
> > [#] https://github.com/vintagewang
> >
> >
> > On Thu, Aug 4, 2022 at 10:50 PM jinrongtong 
> > wrote:
> >
> > > Hi Apache RocketMQ Community,
> > >
> > >
> > >
> > > The Project Management Committee (PMC) for Apache RocketMQ has invited
> > > Zhangheng Huang (apache id: hzh0425, github id:hzh0425)
> > > to become committers, and we are pleased to announce that he has
> > accepted.
> > >
> > >
> > > Congrats, guys :-)
> > >
> > >
> > > Best Wishes,
> > > RongtongJin
> >
>

Re: [ANNOUNCE] Release gRPC Client for Apache RocketMQ 5.0.0

[Zhanhui Li]

Congratulations, we finally reach this milestone!

I am suggesting C/C++ users to try this new C++ SDK in that

1. It's designed and implemented from scratch, memory safety is prioritized
and guaranteed with address sanitizer, thread safety is ensured by thread
sanitizer (https://clang.llvm.org/docs/ThreadSanitizer.html),  in addition
to unit tests;
2. For the first time, RocketMQ C++ Client SDK builds, works and formally
tested on Windows, Linux and Mac;
3. It's verified working on CPUs with instruction set of x86, amd64 and ARM;
4. We support build tools Modern CMake and Bazel. Build your own binary, be
it static and shared.

On Wed, Aug 10, 2022 at 11:24 AM aaron ai  wrote:

> Hi all,
>
> The Apache RocketMQ team would like to announce the release of Apache
> RocketMQ gRPC Client 5.0.0.
>
> Apache RocketMQ clients 5.x series follow the specs of rocketmq-apis
> , and are built on top of
> Protocol Buffer and gRPC.
>
> The release artifacts can be downloaded here:
> https://dist.apache.org/repos/dist/release/rocketmq/rocketmq-clients/
>
> The release notes can be found here:
>
> https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-java-5.0.0
> https://github.com/apache/rocketmq-clients/releases/tag/cpp-5.0.0
>
> Thanks,
> The Apache RocketMQ Team
>

Re: [Vote] Release gRPC Client for Apache RocketMQ 5.0.0 RC1

[Zhanhui Li]

+1

I checked the license, notice and checksum, GPG signature.

I also validated compilation instructions of the CPP SDK on a clean docker.

On Mon, Aug 1, 2022 at 8:39 PM aaron ai  wrote:

> Hello RocketMQ Community,
>
> This is the vote for 5.0.0 RC1 of Apache RocketMQ clients.
>
> Apache RocketMQ clients 5.x series follow the specs of rocketmq-apis
> , and are built on top of
> Protocol Buffer and gRPC.
>
> The artifacts:
>
>-
>
>Java:
>
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-client-java/5.0.0-rc1/
>-
>
>CPP:
>
> https://dist.apache.org/repos/dist/dev/rocketmq/rocketmq-client-cpp/5.0.0-rc1/
>
>
> The staging repo for maven:
>
>
> https://repository.apache.org/content/repositories/orgapacherocketmq-1094/
>
> Git tag for the release:
>
>-
>
>Java:
>https://github.com/apache/rocketmq-clients/tree/rocketmq-client-java-5.0.0
>-
>
>CPP: https://github.com/apache/rocketmq-clients/tree/cpp-5.0.0-rc1
>
>
> Hash for the release tag:
>
>-
>
>Java: db031977d2e0a1e0d64e43f3daf10eab080a13ac
>-
>
>CPP: 6fcff3207d292d925d06f44148934b8d79c1ab4a
>
>
> Relate Notes:
>
>-
>
>Java:
>
> https://github.com/apache/rocketmq-clients/releases/tag/rocketmq-client-java-5.0.0
>-
>
>CPP:
>https://github.com/apache/rocketmq-clients/releases/tag/cpp-5.0.0-rc1
>
>
> The artifacts have been signed with Key :
>
> 3A11FEBBD64C807A233BF1F27C46C79BD4D29011, which can be found in the keys
> file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
>
> Fill in the following:
>
> [+]  check LICENSE, should be Apache V2
>
> [+]  check NOTICE, should have a notice for third-party dependency if
> necessary
>
> [+]  extract the zip and check if the source version is correct
>
> [+]  verify the asc(PGP sign),SHA512
>
>
>
>
>
>
>
> The vote will be open for at least 72 hours or until the necessary number
> of votes are reached.
>
> Please vote accordingly:
>
>
>
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove with the reason
>
>
>
>
> Thanks
>
> The Apache RocketMQ Team
>
>
>

[DISCUSS] Support Graceful Shutdown

[Zhanhui Li]

Hi guys,

RocketMQ follows the paradigm of long connections. Aka, the connections
will be multiplexed for multiple remoting commands. This mechanism works
well for regular cases. Unfortunately, when it comes to the server-offline
use case, clients are frequently interrupted, witnessing a number of
errors, and exceptions before the offline servers are passively detected
and removed.


As a matter of fact, all paradigms of long connections would witness the
same issue, for example, HTTP 2.0. And this is also the reason why it has a
GOAWAY frame https://datatracker.ietf.org/doc/html/rfc7540#section-6.8


I've scratched a RIP:
https://docs.google.com/document/d/1yoNtFFz91z1_SEkmadCURJL2-EdrNkG-3pFrzXnrz-k/edit?usp=sharing


Let me know what you think.


Zhanhui Li

Re: [VOTE]Release Apache RocketMQ APIs 2.0.0 RC1

[Zhanhui Li]

+1 binding

I checked

1. Checksum and PGP signature;
2. License headers are included in Protobuf files;
3. The Apache V2 license file is included in the package;


On Mon, Jul 18, 2022 at 11:07 AM lollipop  wrote:

> +1，checked：
>
> [OK ]  Checksums and PGP signatures are valid for source package.
> [OK ]  License and Notice are correct in source package.
>
> On Fri, Jul 15, 2022 at 11:13 PM aaron ai  wrote:
>
> > Hello RocketMQ Community,
> >
> > This is the vote for 2.0.0 RC1 of Apache RocketMQ APIs.
> >
> > RocketMQ APIs use Protocol Buffers version 3 (proto3) as their Interface
> > Definition Language (IDL) to define the API interface and the structure
> of
> > the payload messages.
> >
> > The artifacts:
> >
> > https://dist.apache.org/repos/dist/dev/rocketmq/2.0.0-rc1
> >
> > The staging repo:
> >
> >
> https://repository.apache.org/content/repositories/orgapacherocketmq-1091/
> >
> > Git tag for the release:
> >
> > https://github.com/apache/rocketmq-apis/releases/tag/v2.0
> >
> >
> https://github.com/apache/rocketmq-apis/releases/tag/rocketmq-proto-2.0.0
> >
> > Hash for the release tag:
> >
> > 303962118171075c3a5a8c395822fecfdf492d6f
> >
> > Relate Notes:
> >
> > https://github.com/apache/rocketmq-apis/releases/tag/v2.0
> >
> >
> https://github.com/apache/rocketmq-apis/releases/tag/rocketmq-proto-2.0.0
> >
> > The artifacts have been signed with Key :
> >
> > 3A11FEBBD64C807A233BF1F27C46C79BD4D29011, which can be found in the keys
> > file: https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> >
> > Fill in the following:
> >
> > [+]  check LICENSE, should be Apache V2
> >
> > [+]  check NOTICE, should have a notice for third-party dependency if
> > necessary
> >
> > [+]  extract the zip and check if the source version is correct
> >
> > [+]  verify the asc(PGP sign),SHA512
> >
> >
> >
> >
> >
> >
> >
> > The vote will be open for at least 72 hours or until the necessary number
> > of votes are reached.
> >
> > Please vote accordingly:
> >
> >
> >
> >
> > [ ] +1 approve
> >
> > [ ] +0 no opinion
> >
> > [ ] -1 disapprove with the reason
> >
> >
> >
> >
> > Thanks
> >
> > The Apache RocketMQ Team
> >
>

Re: [DISCUSS][RIP-42]Support Timing Messages with Arbitrary Time Delay

[Zhanhui Li]

Is it possible to implement hierarchical timing wheels solution:
http://www.cs.columbia.edu/~nahum/w6998/papers/sosp87-timing-wheels.pdf
True that your proposal is relatively easy to implement, but it introduces
too many random reads. Operating systems normally read data in pages(at
least 4k) with potential read-ahead, making page cache pollution more
serious.

The hierarchical timing wheels solution, however, is capable of making
sequential reads/writes, thus much more performant.  As a matter of fact,
Aache Kafka has already implemented this solution:
https://www.confluent.io/blog/apache-kafka-purgatory-hierarchical-timing-wheels/
According
to their description, the engineer complexity is contained and worthwhile.


On Tue, Jun 21, 2022 at 12:01 PM yukon  wrote:

> The long-awaited feature for our community, this RIP will make our timer
> message more competitive, and make the new SimpleConsumer API
> `changeInvisibleDuration` more flexible.
>
> On Tue, Jun 21, 2022 at 11:42 AM dongeforever 
> wrote:
>
> > Good Job.
> >
> > It will be nice if there are some performance test reports.
> >
> > 季俊涛 <3160102...@zju.edu.cn> 于2022年6月20日周一 17:28写道：
> >
> > > Hi, RocketMQ Community:
> > >
> > > Currently, RocketMQ's delay message feature only supports delayed
> > delivery
> > > for specific time levels. Such delay message feature(only support
> > specific
> > > levels of delay time) is no longer enough to support the flexible usage
> > of
> > > rocketmq. Therefore, we need a delay message feature that supports
> > > arbitrary delay time. So I would like to start an email thread to
> discuss
> > > RIP-42 Support Timing Messages with Arbitrary Time Delay.
> > >
> > > I have written my proposal and you can click on the link below：
> > > Google Doc:
> > >
> >
> https://docs.google.com/document/d/1D6XWwY39p531c2aVi5HQll9iwzTUNT1haUFHqMoRkT0/edit?usp=sharing
> > > Shimo:https://shimo.im/docs/gXqme9PKKpIeD7qo/
> > >
> > > If you have any questions or suggestions, please reply to this email or
> > > comment on the proposal.
> > >
> > > Thanks
> > > Juntao Ji
> > >
> > >
> >
>

Re: [DISCUSS] [RIP-39] Support gRPC protocol

[Zhanhui Li]

+1

On Mon, Mar 14, 2022 at 12:45 PM Xiaorui Wang 
wrote:

> +1
>
> Best regards,
>
> Xiaorui Wang[1] - Apache RocketMQ PMC chair
> [1] https://github.com/vintagewang
>
>
> On Wed, Mar 9, 2022 at 4:32 PM 陈仲良  wrote:
>
> > Currently the .NET environment  SDK is very lacking. I hope the new sdk
> > based on gRPC protocol could solve the problem.
> >
>

Re: [DISCUSS][RIP-37] New and unified APIs

[Zhanhui Li]

+1


On Mon, Mar 14, 2022 at 10:53 AM yukon  wrote:

> +1, and let's start the further discussions on new APIs.
>
> On Mon, Mar 14, 2022 at 10:32 AM aaron ai  wrote:
>
> > Hi, RocketMQ Community,
> >
> > As discussed in the previous email, we launched a new RIP to establish
> new
> > and unified APIs and it's time to start an email thread to enter the
> voting
> > process.
> >
> > links:
> > https://shimo.im/docs/m5kv92OeRRU8olqX
> >
> > The vote will be open for at least 72 hours or until a necessary number
> of
> > votes are reached.
> >
> > Please vote accordingly:
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> >
> > Best Regards!
> >
> > On Sat, Mar 12, 2022 at 2:32 PM yuzhou  wrote:
> >
> > > Thanks, glad to see that weakly typed topic will keep exist.
> > >
> > > On 2022/03/10 08:01:46 yukon wrote:
> > > > Hi,
> > > >
> > > > A weakly typed topic that supports all kinds of messages has
> > > > many advantages, it's easy and flexible, while a strongly typed topic
> > > also
> > > > has other advantages:
> > > >
> > > > 1. Reinforce the mind that rocketmq supports many integration
> patterns
> > > > which could simplify the development of business applications.
> > > > 2. Fail fast if developers send wrong typed messages to a strongly
> > typed
> > > > topic.
> > > > 3. Developers could arrange their applications by topics of different
> > > > types, actually, it's a best practice of rocketmq
> > > > 4. RocketMQ has a chance to provide more competitive features for
> > > different
> > > > topic types separately.
> > > >
> > > > And, we won't disable the weakly typed topic, from an implementation
> > > > perspective, we just add an attribute for the topic to indicate
> whether
> > > > it's a strongly typed topic, and a strongly typed topic can be
> > converted
> > > to
> > > > a weakly typed topic easily.
> > > >
> > > > Regards,
> > > > yukon
> > > >
> > > > On Mon, Mar 7, 2022 at 2:04 PM aaron ai 
> wrote:
> > > >
> > > > > Well, The new design about APIs allows us to focus more on the
> > feature
> > > > > itself, rather than the underlying implementation.
> > > > >
> > > > > It seems that topic type creates more limitations to users,
> actually
> > it
> > > > > simplifies operation of users, we think it is more friendly to
> users.
> > > > >
> > > > > On Mon, Mar 7, 2022 at 10:59 AM yuzhou  wrote:
> > > > >
> > > > > > Hi, aaron:
> > > > > >
> > > > > > It is a great improvement, especially for some of features such
> as
> > > the
> > > > > new
> > > > > > constructor use
> > > > > > builder pattern, unified 3 kinds of consumers, unified exception
> > > types,
> > > > > > transaction API
> > > > > > improvement.
> > > > > >
> > > > > > IMHO, many user scenarios have mixed message types, for example,
> > > delay
> > > > > and
> > > > > > normal
> > > > > > message in the same topic, other cases use transaction and normal
> > > message
> > > > > > in the same
> > > > > > topic. Do we have specail reason to split them into defferent
> > topics?
> > > > > >
> > > > > >
> > > > > > On 2022/03/06 08:10:55 aaron ai wrote:
> > > > > > > Hi, RocketMQ Community：
> > > > > > >
> > > > > > > Regarding the design of RocketMQ APIs, we have put forward some
> > new
> > > > > > ideas,
> > > > > > > hoping to make the definition of messaging model and behavior
> > more
> > > > > clear.
> > > > > > >
> > > > > > > We have written the proposal and you can see it by the link
> > below:
> > > > > > > https://shimo.im/docs/m5kv92OeRRU8olqX
> > > > > > >
> > > > > > > Please reply to this email if you have any suggestions.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] [RIP-39] Support gRPC protocol

[Zhanhui Li]

It is completely possible to maintain API compatible while transport and
codec are replaced with gRPC.  However, another RIP is also proposing
amendment to the existing API, with strong focus on consistency,
immutability, and ease of use.

Considering the well received SemVer paradigm (see https://semver.org/),
wondering that we should upgrade the major version and keep the new SDK
capable of co-existence with current ones. That is, the new SDK is able to
run alongside 4.x version in the same JVM, allowing partial migration to
the new SDK. In the meantime, maintaining version 4.x for some time until
it reached end-of-life.


On Wed, Mar 9, 2022 at 1:35 PM Xiaorui Wang  wrote:

> I think it is a right choice to support gRPC. However, I have a concern for
> the client compatibility, whether the benefits of gRPC can be used by
> upgrading SDK. If so, the cost of the upgrade should be taken into account.
>
> I look forward to receiving your reply on the questions I raised.
>
> Cheers
>
> Xiaorui Wang 王小瑞
> Apache RocketMQ PMC chair
>
>
> On Tue, Mar 8, 2022 at 3:11 PM Zhouxiang Zhan  wrote:
>
> > Hi. RocketMQ Community:
> >
> > We are proposing to implement the gRPC protocol on RocketMQ.
> >
> > Detailed proposal: https://shimo.im/docs/gXqmeEPYgdUw5bqo/
> 「RIP-39：Support
> > gRPC protocol」
> >
> > Looking forward to more detailed discussions.
> >
>

Re: [VOTE] [RIP-35]Use rocketmq instead of mysql as state store

[Zhanhui Li]

+1

On Fri, Feb 18, 2022 at 3:07 PM vongosling  wrote:

> +1
>
> nize  于2022年2月18日周五 10:06写道：
>
> > Hi, RocketMQ Community,
> >
> > As discussed in the previous email, we launched a new RIP to replace
> mysql
> > with RocketMQ as a state store in RocketMQ-streams.Now the
> > shepherd @duhenglucky and @von Gosling is willing to support the RIP, so
> I
> > think it is time to start an email thread to enter the voting process.
> >
> > The vote will be open for at least 72 hours or until a necessary number
> of
> > votes are reached.
> >
> > Please vote accordingly:
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> > Best Regards!
> >
> > links
> >
> >
> https://docs.google.com/document/d/1gmwU4rC5wyG07R55jW5oN4GzxHSq8QoFNdoNL61xT0M/edit?usp=sharing
> >
>
>
> --
> Best Regards :-)
>

Re: [VOTE][RIP-34] Support quorum write and adaptive degradation in master-slave architecture

[Zhanhui Li]

Is there any discussion thread prior to this RIP? I got a few doubts to
clarify in terms of how auto degradation is supposed to work.

On Sat, Feb 12, 2022 at 10:48 AM jinrongtong  wrote:

> Hi, RocketMQ Community,
>
> As discussed in the previous email, we launched a new RIP to support
> quorum write and adaptive degradation in master-slave architecture. Now the
> shepherd @duhenglucky is willing to support the RIP, so I think it is time
> to start an email thread to enter the voting process.
>
> The vote will be open for at least 72 hours or until a necessary number of
> votes are reached.
>
> Please vote accordingly:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
>
> Best Regards!
> Rongtong Jin
>
>
> links:
>
> https://docs.google.com/document/d/1ENCJl83OGnmtPMAbxTqtEs9LOOIkMNkz3A9HZJHyI2o/edit?usp=sharing
> https://shimo.im/docs/WJptDyXgjc3h6C8R

Re: [VOTE][RIP-33] RocketMQ MQTT

[Zhanhui Li]

+1

On Thu, Feb 10, 2022 at 5:43 PM 刘振东  wrote:

> +1
>
> 发自我的 iPhone
>
> > 在 2022年2月10日，上午8:45，vongosling  写道：
> >
> > +1
> >
> > Ping Wang  于2022年2月9日周三 19:19写道：
> >
> >> Hi, RocketMQ Community,
> >>
> >> As discussed in the previous email, we launched a new RIP to support
> MQTT
> >> protocol. Now the shepherd @duhenglucky is willing to support the RIP,
> so I
> >> think it is time to start an email thread to enter the voting process.
> >>
> >> The vote will be open for at least 72 hours or until a necessary number
> of
> >> votes are reached.
> >>
> >> Please vote accordingly:
> >> [ ] +1 approve
> >> [ ] +0 no opinion
> >> [ ] -1 disapprove with the reason
> >>
> >>
> >> Best Regards!
> >> Ping Wang
> >>
> >>
> >> links:
> >>
> >>
> >>
> https://docs.google.com/document/d/1AD1GkV9mqE_YFA97uVem4SmB8ZJSXiJZvzt7-K6Jons/edit#heading=h.i42fwfxz62gw
> >>
> >
> >
> > --
> > Best Regards :-)
>

Thread Safety Analysis Project

[Zhanhui Li]

Hi RocketMQ CPP SDK developers,

Existing codebase makes heavy use of STL containers, which requires
external synchronization when concurrent read/write access happens.  Even
though we have employed technics like RAII(lock_guard) to ensure locks are
properly used and unlocked. Still there are potential risks uncovered. This
noticeably raises the bar for new contributors.

This challenge is not unique to us, instead, developers from various roles
are attempting to meet this challenge.
Guys from google published the following articles:
https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/42958.pdf
https://llvm.org/devmtg/2011-11/Hutchins_ThreadSafety.pdf
Then compilers including GCC and Clang supports static thread analysis.
Abseil, open-sourced common library at Google, includes tools and macros to
ease application of this feature:
https://github.com/abseil/abseil-cpp/blob/master/absl/base/thread_annotations.h
and this guide: https://abseil.io/docs/cpp/guides/synchronization

I am proposing annotate our containers and mutex, functions to perform
thread sanitizing at build time.

Any comment is welcome and looking for guys having interest in this.

Zhanhui Li

Support Bazel build

[Zhanhui Li]

Developers of RocketMQ-CPP-SDK,

I am writing to propose building the CPP SDK project using Bazel and
further utilizing buildbuddy.io to enhance external dependency management
and build speed.

At the moment, we are creating build.sh and build.bat to manually download,
verify and compile external dependencies. The process is tedious and
error-prone. Timeouts are frequently observed while performing
continuous integration.
https://app.travis-ci.com/github/apache/rocketmq-client-cpp/jobs/555011480

Bazel has grown mature enough that many popular projects adopt it,
including [gRPC](https://github.com/grpc/grpc), [envoy](
https://github.com/envoyproxy/envoy), and many other google owned projects:
abseil, benchmark, googletest.

It also has a very good ecology and community. Many of the commercial
enterprise
infrastructure are free to open-source projects, BuildBuddy, for example.
These infra provides us tools like RemoteBuildExecution(RBE hereafter),
which launches hundreds of build/test tasks in the build farm and thus
could significantly reduce build time to seconds in terms of reliable
incremental build and minutes in terms of fresh build.

 Note we rarely need a fresh new build because Bazel remote cache takes
file checksum, environment variables, compiler toolchain, etc, into account
before re-using cache. Take a look at
https://app.buildbuddy.io/invocation/6303894a-ecfc-4e7c-9869-b8e747a7e842

 As the initial step, I have  setup build scripts for core logics and
external dependencies including  boost, libevent, jsoncpp, googletests in
the following pull request:
https://github.com/apache/rocketmq-client-cpp/pull/394

At present, rules_foreign_cc are used for libevent, calling internal CMake
commands.  Native build file would be created in a second pull-request.
Once the build script is completed, I would setup to take advantage of
buildbuddy.io infra.

Tests and coverage are not yet migrated.

I am looking forward to feedback and potential joint efforts.

Zhanhui Li

Re: [DISCUSS] RIP-25 Ease of Use Improvements on RocketMQ Client API

[Zhanhui Li]

Agree, exiting API set require substantial improvement, to further refine
RocketMQ model.

On Fri, Oct 1, 2021 at 9:37 AM yukon  wrote:

> Great catch, we will draft a new version of client APIs in few days~
>
> On Wed, Sep 29, 2021 at 4:19 PM dongeforever 
> wrote:
>
> > The idea is OK.
> > And the same time, it is better to show more details.  Maybe drafting the
> > new API interface is a good way to dive deep.
> >
> > caigy  于2021年9月28日周二 下午5:40写道：
> >
> > > RIP-25 Ease of Use Improvements on RocketMQ Client API
> > >
> > >
> > >   * Current Status: Draft
> > >   * Authors: caigy https://github.com/caigy
> > >   * Shepherds: duhengforever duhengfore...@apache.org
> > >   * Mailing List discussion: dev@rocketmq.apache.org
> > >   * Pull Request: #PR_NUMBER
> > >   * Released: 
> > >
> > >
> > > Background & Motivation
> > > 
> > >
> > > What do we need to do
> > >  * will we add a new module? No.
> > >  * will we add new APIs? Yes.
> > >  * will we add new feature? No.
> > >
> > >
> > > Why should we do that
> > >  * Are there any problems of our current project?
> > > Currently, RocketMQ's client API is a bit complex, and there exists
> > > some unreasonable encapsulation. For example:
> > > ** There are 20 methods for sending messages in DefaultProducer,
> with
> > > lack of encapsulation, which increases complexity in usage.
> > > ** DefaultMQPushConsumer provides 10 constructors，which may make
> user
> > > difficult to choose one.
> > > ** Class Message provides constructors with and without arguments,
> > and
> > > also provides getters/setters to operate fields of it,  which lacks
> good
> > > separation of required and non-required arguments.
> > > API is important media for users to interact with RocketMQ, and it
> is
> > > worth investing effort in optimizing its design.
> > >
> > > * What can we benefit proposed changes?
> > > Provide clear and easier-to-understand API of RocketMQ, make it
> > easier
> > > to use, especially for beginners.
> > >
> > >
> > >
> > > Goals
> > > 
> > >
> > > * What problem is this proposal designed to solve?
> > >Optimize RocketMQ client APIs, including Producer, Consumer and
> > > Message, remove unnecessary APIs, and provide better encapsulation.
> > Current
> > > unreasonable APIs will be marked deprecated and will be removed in the
> > > future. After this optimization, the APIs should keep as stable as
> > possible.
> > >
> > > * To what degree should we solve the problem?
> > > We wish developers can use RocketMQ more easily with new APIs.
> > >
> > >
> > > Non-Goals
> > > 
> > >
> > > * What problem is this proposal NOT designed to solve?
> > >This proposal will NOT change feature and performance.
> > >
> > > * Are there any limits of this proposal?
> > >   Nothing specific.
> > >
> > >
> > >
> > > Changes
> > > 
> > >
> > > * Architecture
> > >  No architecture changes in this proposal.
> > >
> > > * Interface Design/Change
> > >** Method signature changes. Yes.
> > >For example:
> > >*** Add generic to support data type of message body in Message.
> > >*** Support chain-call instantiation of Producer, Consumer,
> > > Message, eg：Message.builder().topic("msg  topic").body("msg
> > > body").build();
> > >*** Producer provides unified and stable send() API, which is
> like
> > > write() method of UNIX file I/O.
> > >*** Add asynchronous message consumption API based on
> > > CompletableFuture in Consumer.
> > >*** etc...
> > >
> > >** Method behavior changes. No.  CLI command changes. No.
> > >** Log format or content changes. No.
> > >
> > > * Compatibility, Deprecation, and Migration Plan
> > >** Are backward and forward compatibility taken into consideration?
> > >   Optimized APIs should NOT affect compatibility, some APIs would
> be
> > > marked deprecated. New features will use optimized APIs, encouraging
> > users
> > > not to use deprecated APIs.
> > >
> > >** Are there deprecated APIs?
> > >   Yes. Some unreasonable APIs will be deprecated in this RIP.
> > >
> > >   ** How do we do migration?
> > >   Unreasonable APIs are marked deprecated in this proposal, and
> will
> > > be removed in the future.
> > >
> > > * Implementation Outline
> > >   We will implement the proposed changes by 1 phase.
> > >   Phase 1
> > >  1. Collect pain points in using RocketMQ client APIs and redesign
> > > them.
> > >  2. Optimize those APIs.
> > >  3. Mark previous APIs deprecated.
> > >
> > >
> >
>

Re: [VOTE] RIP23: Support gRPC protocol

[Zhanhui Li]

+1

I partially agree with vongosling that this proposal requires more feedback
before initiating a vote.  But taking advantage of gRPC does save a lot of
work, especially those blocking issues caused by FastJSON due to its
non-strict conformance to JSON standards. Review how cumbersome it is to
implement the reset-offset-to-playback feature for Non-Java SDK. In
addition, ProtoBuffer supports protocol evolution pretty well in the long
term.

Actually, this proposal, IMO, is not to replace existing SDKs, instead, it
is to make the transport and serialization layer pluggable and thus
configurable, which is similar to MySQL X Protocol:
https://dev.mysql.com/doc/internals/en/x-protocol.html



On Wed, Jun 16, 2021 at 11:14 AM vongosling  wrote:

> -1  I think some pro and cron including tech details should be clarified.
> I would like to get feedback on this RIP proposal.  不要着急投票 ~
>
>
>
> 我其实是部分支持这个Proposal的，但是新协议如何和老协议兼容，这是一个令人望而却步的工作。我之前提过，如果希望解决多语言的问题，可以参考下dapr目前的做法。在云原生时代，坦率的说，我们不需要富客户端，但是瘦客户端能满足我们80%的主要场景吗？这就跟之前我们一直在强调pull的好处，而远离push，结果呢？如果我们能够以始为终认真看待这个问题，
> 近2年多年来
> ，社区前仆后继接近30号同学一直致力于多语言的改进优化，我希望看到工作的延续性。这里，我希望能够慎重的看到这个问题，希望看到更多类似Yin
> James的分享和讨论。
>
> heng du  于2021年6月15日周二 上午11:37写道：
>
> > Hi RocketMQ Community,
> >
> > This is the vote for the kickoff of RIP-23 RocketMQ gRPC protocol
> support.
> >
> > On one hand RocketMQ. remoting module is too complicated to maintain,
> gRPC
> > makes it easier to establish a robust communication layer, the current
> > remoting module would be simplified radically.
> > On the other hand, gRPC has been the de-facto standard in CloudNative,
> > service mesh would be easily applied if gRPC is enabled.
> >
> > So in this RIP, we want to support gRPC's protocol, simplify the current
> > communication layer of RocketMQ, make it easier to adapt to other
> > languages, which is not limited to CPP/GO/C#/GO。
> >
> > The vote will be open for at least 72 hours or until a necessary number
> > of votes are reached.
> >
> > Please vote accordingly:
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> >
> > i yangkun  于2021年6月8日周二 下午8:28写道：
> >
> > > Background & Motivation
> > > What do we need to do
> > >
> > >
> > >   *   Will we add a new module?
> > > maybe.
> > >   *   Will we add new APIs?
> > > Yes.
> > >
> > >   *   Will we add new feature?
> > > Yes.
> > >
> > >
> > > Why should we do that
> > >
> > >
> > >   *   Are there any problems of our current project?
> > >
> > > a. Remoting module is too complicated to maintain, gRPC makes it easier
> > to
> > > establish a robust communication layer, the current remoting module
> would
> > > be simplified radically.
> > >
> > > b. gRPC has been the de-facto standard in CloudNative, service mesh
> would
> > > be easily applied if gRPC is enabled.
> > >
> > > c. The private protocol of RocketMQ depends on the FastJson, it is
> > > difficult to adapt for other language.
> > >
> > > On the other side, since the pop consumer has been merged, we could
> > > implement new SDK based on gRPC and pop, which is easier to develop and
> > > maintain.
> > >
> > > Chinese Version:
> > >
> > > a. Remoting 模块对于长期的维护而言过于复杂了，我们可以使用 gRPC 更轻松地建立起一个健壮的通信层，这会使得现有的
> remoting
> > > 模块从根本上得到简化。
> > >
> > > b. gRPC 目前已经是云原生时代的事实标准，使用 gRPC 可以使得我们天然获取一些云原生的能力，比如 Service Mesh。
> > >
> > > c. 目前 RocketMQ 的私有协议强烈依赖 FastJson，多语言的适配将会变得困难。
> > >
> > >
> > > 从另外一个角度来说，鉴于 pop 消费者已经被合并，我们可以基于 gRPC 和 pop 实现新的 SDK，新的 SDK
> 将会更加易于开发和维护。
> > >
> > > Goals
> > >
> > >
> > >   *   What problem is this proposal designed to solve?
> > >
> > > Support gRPC's protocol, simplify current communication layer oof
> > > RocketMQ, make it easier to adapt for other language, which is not
> > limited
> > > to CPP/GO/C#/GO。
> > >
> > > Chinese Version:
> > >
> > > 支持 gRPC 协议，简化 RocketMQ 现有的通信层，复用 gRPC 的能力，简化多语言适配成本，不限于 CPP/GO/C#/GO。
> > >
> > >   *   To what degree should we solve the problem?
> > > This RIP must guarantee below point:
> > >
> > >   1.  Compatibility: Both of gRPC and RemotingCommand should be
> > supported.
> > >   2.  High performance: This implementation does not affects latency
> and
> > > throughput.
> > >
> > >
> > > Chinese Version:
> > >
> > > 新方案需要保证两点：
> > >
> > >   1.  兼容性：同时支持 gRPC 和 RemotingCommand 协议，不影响现有功能。
> > >   2.  高性能：基于 gRPC 的实现不影响整理的延时和吞吐量。
> > >
> > >
> > > Non-Goals
> > >
> > >
> > >   *   What problem is this proposal NOT designed to solve?
> > > Nothing specific.
> > >   *   Are there any limits of this proposal?
> > > Nothing specific.
> > >
> > >
> > > Changes
> > > Architecture
> > >
> > >
> > > Current broker processor and client.
> > >
> > > [
> > >
> >
> https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142547507-128b85f5-98f4-4568-85f8-28ef32982b7c.png
> > > ]
> > >
> > > Proposed gRPC processor and client.
> > >
> > > [
> > >
> >
> https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142552491-a7f58ac0-cd7d-4ddd-936e-fb296b667196.png
> > > ]
> > >
> > > 

Re: RIP23: Support gRPC protocol

[Zhanhui Li]

Hi,
This proposal, in general, is in the right direction that helps RocketMQ
provide full-fledged SDK for popular languages and platforms. Taking full
advantage of gRPC does save a lot of effort in terms of serialization and
RPC tiers. Obviously, this proposal also brings complexities and potential
compatibility issues.

One of the potential issues is that gRPC does not expose Channel in the
implementation while RocketMQ processors make heavy use of it, even if both
of them are built on top of Netty 4.x.  Will this an issue when reuse
existing code?

Zhanhui Li

On Tue, Jun 8, 2021 at 8:28 PM i yangkun  wrote:

> Background & Motivation
> What do we need to do
>
>
>   *   Will we add a new module?
> maybe.
>   *   Will we add new APIs?
> Yes.
>
>   *   Will we add new feature?
> Yes.
>
>
> Why should we do that
>
>
>   *   Are there any problems of our current project?
>
> a. Remoting module is too complicated to maintain, gRPC makes it easier to
> establish a robust communication layer, the current remoting module would
> be simplified radically.
>
> b. gRPC has been the de-facto standard in CloudNative, service mesh would
> be easily applied if gRPC is enabled.
>
> c. The private protocol of RocketMQ depends on the FastJson, it is
> difficult to adapt for other language.
>
> On the other side, since the pop consumer has been merged, we could
> implement new SDK based on gRPC and pop, which is easier to develop and
> maintain.
>
> Chinese Version:
>
> a. Remoting 模块对于长期的维护而言过于复杂了，我们可以使用 gRPC 更轻松地建立起一个健壮的通信层，这会使得现有的 remoting
> 模块从根本上得到简化。
>
> b. gRPC 目前已经是云原生时代的事实标准，使用 gRPC 可以使得我们天然获取一些云原生的能力，比如 Service Mesh。
>
> c. 目前 RocketMQ 的私有协议强烈依赖 FastJson，多语言的适配将会变得困难。
>
>
> 从另外一个角度来说，鉴于 pop 消费者已经被合并，我们可以基于 gRPC 和 pop 实现新的 SDK，新的 SDK 将会更加易于开发和维护。
>
> Goals
>
>
>   *   What problem is this proposal designed to solve?
>
> Support gRPC's protocol, simplify current communication layer oof
> RocketMQ, make it easier to adapt for other language, which is not limited
> to CPP/GO/C#/GO。
>
> Chinese Version:
>
> 支持 gRPC 协议，简化 RocketMQ 现有的通信层，复用 gRPC 的能力，简化多语言适配成本，不限于 CPP/GO/C#/GO。
>
>   *   To what degree should we solve the problem?
> This RIP must guarantee below point:
>
>   1.  Compatibility: Both of gRPC and RemotingCommand should be supported.
>   2.  High performance: This implementation does not affects latency and
> throughput.
>
>
> Chinese Version:
>
> 新方案需要保证两点：
>
>   1.  兼容性：同时支持 gRPC 和 RemotingCommand 协议，不影响现有功能。
>   2.  高性能：基于 gRPC 的实现不影响整理的延时和吞吐量。
>
>
> Non-Goals
>
>
>   *   What problem is this proposal NOT designed to solve?
> Nothing specific.
>   *   Are there any limits of this proposal?
> Nothing specific.
>
>
> Changes
> Architecture
>
>
> Current broker processor and client.
>
> [
> https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142547507-128b85f5-98f4-4568-85f8-28ef32982b7c.png
> ]
>
> Proposed gRPC processor and client.
>
> [
> https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142552491-a7f58ac0-cd7d-4ddd-936e-fb296b667196.png
> ]
>
> Broker would provide a protocol negotiate procedure to distinguish
> RemotingCommand from gRPC protocol. two kinds or processor in broker would
> re-use the same port to serve for RPC from different SDK.
>
>
> Chinese Version:
>
> broker 本身提供协议协商机制用于区分 RemotingCommnad 和 gRPC 协议，broker 针对 gRPC 和
> RemotingCommand 提供不同的 processor 为各自的 SDK 服务。
>
> Interface Design/Change
>
>
>   *   Method signature changes
> Nothing specific.
>   *   Method behavior changes
> Nothing specific.
>
>   *   CLI command changes
> Nothing specific.
>   *   Log format or content changes
> Nothing specific.
>
>
> Compatibility, Deprecation, and Migration Plan
>
>
>   *   Are backward and forward compatibility taken into consideration?
>
> Broker support processor of RemotingCommand and gRPC simultaneously, so
> there are one compatibility situations:
>
> If user migrates from original SDK to gRPC SDK in push mode, the
> re-balance policy should make sure that it would not cause repeated
> consumption for a lot of messages.
>
>   *   Are there deprecated APIs?
> Nothing specific.
>   *   How do we do migration?
> Nothing specific.
>
>
> Implementation Outline
>
>
> We will implement the proposed changes by 4 phases.
>
>
> Phase 1
>
>   1.  Provides gRPC protocol definition(IDL)
>
> Phase 2
>
>   1.  Implement gRPC processor of broker.
>   2.  Implement protocol negotiation of two kinds of protocol(gRPC and
> RemotingCommand)
>
> Phase 3
>
>   1.  Implement new JAVA and CPP native SDK based on gRPC
>
> Phase 4
>
>   1.  Implement native SDK base on gRPC for other language.
>
>
> Rejected Alternatives
>
>
> How does alternatives solve the issue you proposed?
>
>
> Thrift? not so much impact as gRPC in community.
>
>
> Pros and Cons of alternatives
>
>
> Nothing specific.
>
> Why should we reject above alternatives
>
>

[DISCUSS]Use timer-wheel to scan timeout of in-flight RPC

[Zhanhui Li]

Hi,

To detect in-flight RPC timeout, RocketMQ uses scheduler to periodically
scan all in-flight RPC requests in NettyRemotingAbstract#scanResponseTable.
This implementation is less effective in terms of resource overhead and
responsiveness in comparison with timer-wheel
<https://blog.acolyer.org/2015/11/23/hashed-and-hierarchical-timing-wheels/>.
I am suggesting to implement the timer-wheel scheme.

Any feedback is welcome. ^_^

Zhanhui Li

Re: [DISCUSS]Apache RocketMQ quarter‘s report

[Zhanhui Li]

Hi developers,
I have updated the report according to feedbacks received.

## Description:
 - Apache RocketMQ is a distributed messaging and streaming platform with
low latency, high performance and reliability, trillion-level capacity, and
flexible scalability.

## Issues:
 - There are no issues requiring board attention at this time.


## Membership Data:
Apache RocketMQ was founded 2017-09-20. At present, there are 31 committers
and 14 PMC members.
New committers joined this project in the last quarter are:
Houdao Chen  05/08
Weihe Yin04/29
Li Wei   04/28
Xu Jianhai   04/10
Xiangwang Cheng 04/10

New PCM Members are:
RongtongJin   04/01

## Project Activity:
Major activities during the last quarter are:
- ROCKETMQ-4.6.1 was released on 2020-02-24.
 * Fix CVE-2019-17572 and consolidate server-side validation;
- ROCKETMQ-4.7.0 was released on 03/17/2020.
 * Performance optimization and multiple bug fixes;
- RocketMQ Golang client version 2.0.0 was released on 03/31/2020.
 * Now product ready with full-fledged features supported, including
pub/sub messages, ACL, and message tracing.
- RocketMQ C++ Client reached version 2.1.0.
 * Add new features including message tracing, guard of
sending-message-back timeout;
 * Bugfix of a pointer use after free;

## Community Health
RocketMQ community health is overall good as we see multiple improvement
proposals are being discussed in parallel and most issues are timely
responded. C++/Golang client libraries are making gradual yet good progress
both in feature-rich and stability. Developers from different language
communities are starting to contribute and join.

In accordance with the cloud-native technical trend, RocketMQ also evolves
that way. RocketMQ gained initial support in Envoy in the last quarter.


## Security Vulnerabilities and Patches
In the last quarter, the following vulnerabilities are identified and fixed:
- CVE-2019-17572, a directory-traversal bug due to broker side validation
flaw;

On Mon, May 11, 2020 at 8:12 PM Zhanhui Li  wrote:

> Hi developers,
> I am drafting RocketMQ quarter's report which will be due in a few days.
> Please proof-read and feedback if there is a need.
>
>
> ## Description:
>  - Apache RocketMQ is a distributed messaging and streaming platform with
> low latency, high performance and reliability, trillion-level capacity, and
> flexible scalability.
>
> ## Issues:
>  - There are no issues requiring board attention at this time.
>
>
> ## Membership Data:
> Apache RocketMQ was founded 2017-09-20. At present, there are 31
> committers and 14 PMC members.
> New committers joined this project in the last quarter are:
> Houdao Chen  05/08
> Weihe Yin04/29
> Li Wei   04/28
> Xu Jianhai   04/10
> Xiangwang Cheng 04/10
>
> New PCM Members are:
> RongtongJin   04/01
>
> ## Project Activity:
> Major activities during the last quarter are:
> - ROCKETMQ-4.7.0 was released on 03/17/2020.
> - RocketMQ Golang client version 2.0.0 was released on 03/31/2020.
> - RocketMQ C++ Client reached version 2.1.0.
>
> ## Community Health
> RocketMQ community health is overall good as we see multiple improvement
> proposals are being discussed in parallel and most issues are timely
> responded. C++/Golang client libraries are making gradual yet good progress
> both in feature-rich and stability. Developers from different language
> communities are starting to contribute and join.
>
> In accordance with the cloud-native technical trend, RocketMQ also evolves
> that way. RocketMQ gained initial support in Envoy in the last quarter.
>
>

[DISCUSS]Apache RocketMQ quarter‘s report

[Zhanhui Li]

Hi developers,
I am drafting RocketMQ quarter's report which will be due in a few days.
Please proof-read and feedback if there is a need.


## Description:
 - Apache RocketMQ is a distributed messaging and streaming platform with
low latency, high performance and reliability, trillion-level capacity, and
flexible scalability.

## Issues:
 - There are no issues requiring board attention at this time.


## Membership Data:
Apache RocketMQ was founded 2017-09-20. At present, there are 31 committers
and 14 PMC members.
New committers joined this project in the last quarter are:
Houdao Chen  05/08
Weihe Yin04/29
Li Wei   04/28
Xu Jianhai   04/10
Xiangwang Cheng 04/10

New PCM Members are:
RongtongJin   04/01

## Project Activity:
Major activities during the last quarter are:
- ROCKETMQ-4.7.0 was released on 03/17/2020.
- RocketMQ Golang client version 2.0.0 was released on 03/31/2020.
- RocketMQ C++ Client reached version 2.1.0.

## Community Health
RocketMQ community health is overall good as we see multiple improvement
proposals are being discussed in parallel and most issues are timely
responded. C++/Golang client libraries are making gradual yet good progress
both in feature-rich and stability. Developers from different language
communities are starting to contribute and join.

In accordance with the cloud-native technical trend, RocketMQ also evolves
that way. RocketMQ gained initial support in Envoy in the last quarter.

Re: [VOTE]Open the ISSUE module for Apache RocketMQ Sub Project

[Zhanhui Li]

+1 

> 在 2019年6月4日，下午2:21，chengxiangw...@cmss.chinamobile.com 写道：
> 
> +1
> 
> 
> 
> 程 向往（Cheng Xiangwang）
> 中移（苏州）软件技术有限公司
> 云计算产品部
> 电话：18896725053
> 邮箱：chengxiangw...@cmss.chinamobile.com
> 地址：苏州市高新区科灵路78号苏高新软件园10幢
> 
> From: Gosling Von
> Date: 2019-06-04 13:57
> To: users
> CC: dev@rocketmq.apache.org
> Subject: Re: [VOTE]Open the ISSUE module for Apache RocketMQ Sub Project
> +1
> 
> On Jun 4, 2019, at 12:40 PM, ShannonDing  wrote:
> 
> Hello RocketMQ Community,
> 
> This is the vote for Opening the Issue module of RocketMQ’s subproject, and 
> it is list below:
> https://github.com/apache/rocketmq-docker 
> https://github.com/apache/rocketmq-exporter 
> https://github.com/apache/rocketmq-operator 
> 
> These projects are very important parts of Apache RocketMQ and enrich the 
> RocketMQ ecosystem.
> The issue module is an efficient way to manage the questions and features of 
> projects
> So we need to open the ISSUE modules of these projects.
> 
> The vote will be open for at least 72 hours or until a necessary number of 
> votes are reached.
> 
> Please vote accordingly:
> 
> [ ] +1 approve 
> [ ] +0 no opinion 
> [ ] -1 disapprove with the reason
> 
> Thanks,
> The Apache RocketMQ Team
> 

Re: [DISCUSS]Introduce a lightweight Apache RocketMQ client

[Zhanhui Li]

Looks it is the right direction to go. A simplified and unified set of API does 
lower the bar for new users in the community. 

> 在 2019年6月4日，下午12:26，heng du  写道：
> 
> Dear RocketMQ Community,
> 
> In order to enhance the usability of RocketMQ, a simple client API seems to
> be introduced into RocketMQ to further reduce the user threshold and
> lowering the probability of making mistakes.
> 
> Compared With RocketMQ's origin client, the new client can be seen as a
> more high-level API that not only provides a better abstraction, but also
> removes some dangerous interfaces, hides more implementation details, and
> reduces A cumbersome configuration. At the same time, the original client
> will be retained as a low-level API to meet the additional needs of some
> experienced users, providing higher control ability for them.
> 
> In the cloud-native era, messaging middleware improvements should not only
> be stayed in the use of cloud features (elasticity, scalability, etc.), but
> more importantly, it can provide users with a more concise and easy-to-use
> API to shield the difference  brought by different cloud vendors or
> different deployment methods , so I think this should also be a focus of
> the follow-up development of RocketMQ.
> 
> Looking forward to hearing your voice.
> 
> Best Regards,
> Henry

Re: [RESTART][VOTE][#1]RocketMQ spring 2.0.3 release.

[Zhanhui Li]

+1

On Mon, Jun 3, 2019 at 10:19 AM dongeforever 
wrote:

> +1
>
> heng du  于2019年5月31日周五 下午7:55写道：
>
> > +1
> >
> > ShannonDing  于2019年5月31日周五 下午7:46写道：
> >
> >>
> >> Hello RocketMQ Community,
> >>
> >>
> >> This is the vote for the 2.0.3 release of Apache rocketmq-spring
> >>
> >>
> >> Github repo:
> >>
> >> https://github.com/apache/rocketmq-spring
> >>
> >>
> >> The staging repo:
> >>
> >>
> https://repository.apache.org/content/repositories/orgapacherocketmq-1027/
> >>
> >>
> >> The main goal of this release is to align with RocketMQ Client 4.5.1 and
> >>
> >> supports the new features, e.g. connect name server with domain and set
> >> delay level in async model. and besides this,
> >>
> >> this release also will bring some enhancements for usability.
> >>
> >>
> >> New Feature:
> >>
> >> *  [Feature-36](
> https://github.com/apache/rocketmq-spring/issues/36)Support
> >> delay level configuration in async mode
> >>
> >> *  [Feature-46](
> https://github.com/apache/rocketmq-spring/issues/46)Support
> >> multiple RocketMQTemplate & name-server overrides Consumer Listener
> >>
> >> *  [Feature-59](
> https://github.com/apache/rocketmq-spring/issues/59)Sending
> >> batch messages with RocketMQTemplate
> >>
> >>
> >>  Improvement:
> >>
> >> *  [ISSUE-61](https://github.com/apache/rocketmq-spring/pull/61)
> Generate
> >> spring-configuration-metadata.json for easy configuration tips.
> >>
> >> *  [ISSUE-63](https://github.com/apache/rocketmq-spring/issues/63)
> Update
> >> README_zh_CN.md
> >>
> >> *  [ISSUE-74](https://github.com/apache/rocketmq-spring/issues/74) Add
> >> acl examples compile in the project
> >>
> >> *  [ISSUE-78](https://github.com/apache/rocketmq-spring/issues/78) Add
> >> new accessChannel property for both producer/consumer @since RMQ 4.5.1
> >>
> >>
> >>
> >> Bug
> >>
> >> *  [ISSUE-53](https://github.com/apache/rocketmq-spring/issues/53)
> Raise
> >> a warning log message if no property 'rocketmq.name-server' is defined
> in
> >> application.property
> >>
> >>  *  [ISSUE-67](https://github.com/apache/rocketmq-spring/issues/67) Fix
> >> typo in example readme file
> >>
> >>
> >> The vote will be open for at least 72 hours or until a necessary number
> >> of votes are reached.
> >>
> >>
> >> Please vote accordingly:
> >>
> >>
> >> [ ] +1 approve
> >>
> >> [ ] +0 no opinion
> >>
> >> [ ] -1 disapprove with the reason
> >>
> >>
> >> Thanks,
> >>
> >> The Apache RocketMQ Team
> >>
> >>
> >> Shannon
> >> ding...@apache.org
> >>
> >> <
> https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1=Shannon=libya_003%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fwzpmmc%2F1e46309f40a02fa0cc021079ff4afff2.jpg=%5B%22dinglei%40apache.org%22%5D
> >
> >> 签名由 网易邮箱大师  定制
> >>
> >>
>

Re: [VOTE] Release Apache RocketMQ 4.4.0 RC1.

[Zhanhui Li]

+1

ACL and tracing are really good features to have.

> 在 2019年1月21日，上午11:53，heng du  写道：
> 
> +1
> 
> Kevin Wang mailto:wiseking...@gmail.com>> 
> 于2019年1月21日周一 上午11:44写道：
> +1
> 
>> 在 2019年1月21日，上午10:35，huzongt...@cmss.chinamobile.com 
>>  写道：
>> 
>> +1，I'm very pleased to participate in developping acl and msg trace feature.
>> We test some cases of acl feature,such as global white address,local white 
>> address,access key/secret key and using apache rocketmq client to link to 
>> Cloud MQ.
>> And we alse test the msg trace features in rocketmq cluster enviroment and 
>> CloudMQ.
>> Our testing result pictures are in the email attachment.
>> 
>>  
>> 
>> huzongt...@cmss.chinamobile.com 
>>  
>> 发件人： lemonzone2...@163.com 
>> 发送时间： 2019-01-21 10:22
>> 收件人： Shannon ; dev@rocketmq.apache.org 
>> 
>> 抄送： us...@rocketmq.apache.org 
>> 主题： 回复: [VOTE] Release Apache RocketMQ 4.4.0 RC1.
>> +1
>>  
>> lemonzone2...@163.com 
>>  
>> 发件人： Shannon 
>> 发送时间： 2019-01-18 18:43
>> 收件人： dev@rocketmq.apache.org 
>> 抄送： us...@rocketmq.apache.org 
>> 主题： [VOTE] Release Apache RocketMQ 4.4.0 RC1.
>> Hello RocketMQ Community,
>> 
>> 
>> 
>> 
>> This is the vote for 4.4.0 of Apache RocketMQ.
>> 
>> 
>> 
>> 
>> This release implemented three new features such as ACL, message trace, and 
>> fixed some bugs,
>> 
>> further improved the stability of RocketMQ.
>> 
>> 
>> 
>> 
>> The artifacts:
>> 
>> https://dist.apache.org/repos/dist/dev/rocketmq/4.4.0-rc1/ 
>> 
>> 
>> 
>> 
>> The staging repo:
>> 
>> https://repository.apache.org/content/repositories/orgapacherocketmq-1015/ 
>> 
>> 
>> 
>> 
>> Git tag for the release:
>> 
>> https://github.com/apache/rocketmq/tree/release-4.4.0 
>> 
>> 
>> 
>> 
>> Hash for the release tag:
>> 
>> f61fd5c73f58589c47061bb78fc6693b6402ecc3
>> 
>> 
>> 
>> 
>> Release Notes:
>> 
>> https://rocketmq.apache.org/release_notes/release-notes-4.4.0/ 
>> 
>> 
>> 
>> 
>> The artifacts have been signed with Key :
>> 
>> CFBA901D7BC4227CE9AAE5B4BA7F56BDB18F8F5B, which can be found in the keys
>> 
>> file:
>> 
>> https://dist.apache.org/repos/dist/dev/rocketmq/KEYS 
>> 
>> 
>> 
>> 
>> The vote will be open for at least 72 hours or until the necessary number of
>> 
>> votes are reached.
>> 
>> 
>> 
>> 
>> Please vote accordingly:
>> 
>> 
>> 
>> 
>> [ ] +1 approve
>> 
>> [ ] +0 no opinion
>> 
>> [ ] -1 disapprove with the reason
>> 
>> 
>> 
>> 
>> Thanks,
>> 
>> The Apache RocketMQ Team
> 

Re: [VOTE] RocketMQ-client-python 1.2.0 Release And Graduation

[Zhanhui Li]

+1

On Thu, Dec 6, 2018 at 7:34 PM liqipeng  wrote:

> +1
> > 在 2018年12月6日，下午7:14，lollipop  写道：
> >
> > +1
> >
> > I've checked  the LICENSE, NOTICE and Copyright, LGTM.
> >
> > Thanks
> >
> > On Thu, Dec 6, 2018 at 11:34 AM heng du  wrote:
> >
> >> Hello RocketMQ Community,
> >>
> >> This is the vote for the 1.2.0 release and the graduation of Apache
> >> RocketMQ-client-python.
> >>
> >> Github repo: https://github.com/apache/rocketmq-client-python <
> >> https://github.com/apache/rocketmq-client-python>
> >>
> >> The version released this time is the first initial Python client of
> >> rocketmq. It is based on the kernel of the CPP client and uses boost
> python
> >> to encapsulate the API Implementation of C. You can make Python modules
> >> with a simple command.
> >> The current version provides the following functions:
> >> 1.support reliable synchronous sending of messages;
> >> 2.support reliable push consumption model;
> >> 3.support default cluster consumption;
> >> 4.support delayed messages;
> >> 5.support custom message properties;
> >> 6.support message Compression.
> >> The vote will be open for at least 72 hours or until a necessary number
> of
> >> votes are reached.
> >>
> >> Please vote accordingly:
> >>
> >> [ ] +1 approve
> >> [ ] +0 no opinion
> >> [ ] -1 disapprove with the reason
> >>
> >> Thanks,
> >> The Apache RocketMQ Team
>
>
>

Re: [VOTE] RocketMQ-client-cpp 1.2.0 Release And Graduation

[Zhanhui Li]

+1

Glad to see the improvement of build scripts of the codebase and look
forward to code refinement in the next release as C++11 is officially
adopted.

On Thu, Dec 6, 2018 at 7:16 PM lollipop  wrote:

> + 1
>
> On Thu, Dec 6, 2018 at 11:21 AM heng du  wrote:
>
> > Hello RocketMQ Community,
> >
> > This is the vote for the 1.2.0 release and the graduation of Apache
> > RocketMQ-client-cpp.
> >
> > Github repo: https://github.com/apache/rocketmq-client-cpp <
> > https://github.com/apache/rocketmq-client-cpp>
> > This release is a huge step forward in cpp 1.0.1. It 
> > provides two kinds of interfaces in C++ and C-style, and a bunch of other
> > language bindings will be built on top of it, including python,
> > Node.js,C#,golang and so on.We simplified the process of project
> > compilation in this version,and you can compile the whole project with
> just
> > one command.
> >
> > new feature
> >C-Style API was added to support multilingual client, such as
> > python,node.js, golang, which has not yet implemented all the functions
> of
> > C++ API. The following are the major functions of C-Style API :
> >
> >
> > 1 Support  reliable synchronous,  one-way  and order transmission
> > 2 Support push consumer and pull consumer.
> > 3 Support setDelayTimeLevel.
> > 4 Support message compression.
> > 5 Support setting up message consumption mode.
> > Improvement:
> > 1  Enhanced the capability of multi-platform support, such as linux,
> > window,macOS.
> > 2  The compilation method of the project is simplified. You can compile
> > the entire project with one command.
> > Bug:
> > 1 Fix an issue about the unique key. When a message is sent, the producer
> > cannot get unique.
> >
> > The vote will be open for at least 72 hours or until a necessary number
> of
> > votes are reached.
> >
> > Please vote accordingly:
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> > Thanks,
> > The Apache RocketMQ Team
>

Re: [VOTE]: Release Apache RocketMQ 4.3.1 RC1

[Zhanhui Li]

+1

On Thu, Aug 30, 2018 at 1:05 PM yukon  wrote:

> +1
>
> On Wed, Aug 29, 2018 at 8:40 PM Shannon  wrote:
>
> >
> >
> > +1
> >
> >
> >
> >
> >
> >
> > At 2018-08-29 20:36:19, "heng du"  wrote:
> > >Hello RocketMQ Community,
> > >
> > >This is the vote for 4.3.1 of Apache RocketMQ. We now kindly request the
> > >RocketMQ community review and vote on this release artifact.
> > >
> > >Apache RocketMQ is a distributed messaging and streaming platform with
> low
> > >latency, high performance and reliability, trillion-level capacity and
> > >flexible scalability.
> > >
> > >This release is an enhancement and fix for the previous version, mainly
> > >enhanced the compatibility of producer and fixed some issues, related
> > >details could be found in the release notes.
> > >
> > >The artifacts:
> > >https://dist.apache.org/repos/dist/dev/rocketmq/4.3.1-rc1/
> > >
> > >The staging repo:
> > >
> https://repository.apache.org/content/repositories/orgapacherocketmq-1013
> > >
> > >Git tag for the release:
> > >https://github.com/apache/rocketmq/tree/rocketmq-all-4.3.1
> > >
> > >Hash for the release tag: 4cef43670f3d28367569fe089ce300666b14416b
> > >
> > >Release Notes:
> > >https://rocketmq.apache.org/release_notes/release-notes-4.3.1/
> > >
> > >The artifacts have been signed with Key : 133C87CA, which can be found
> in
> > >the keys file:
> > >https://dist.apache.org/repos/dist/dev/rocketmq/KEYS
> > >
> > >The vote will be open for at least 72 hours or until necessary number of
> > >votes are reached.
> > >
> > >Please vote accordingly:
> > >
> > >[ ] +1 approve
> > >[ ] +0 no opinion
> > >[ ] -1 disapprove with the reason
> > >
> > >Thanks,
> > >The Apache RocketMQ Team
> >
>

Re: [GitHub] zangyk opened a new issue #333: PullMessageService 是否可以用多个线程拉取消息？

[Zhanhui Li]

>> Your question is: Can PullMessageService make most of multi-thread while 
>> pulling messages from brokers?

The answer is: No multi-thread is needed. Pulling message from brokers is 
completely asynchronous, therefore, introducing multi-thread does not make 
sense here.

Zhanhui Li

> 在 2018年6月5日，下午5:16，GitBox  写道：
> 
> zangyk opened a new issue #333: PullMessageService 是否可以用多个线程拉取消息？
> URL: https://github.com/apache/rocketmq/issues/333
> 
> 
>   PullMessageService 是否可以用多个线程拉取消息？
> 
> 
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on GitHub and use the
> URL above to go to the specific comment.
> 
> For queries about this service, please contact Infrastructure at:
> us...@infra.apache.org
> 
> 
> With regards,
> Apache Git Services

Re: [GSoC][ROCKETMQ-379][Support AMQP protocol for RocketMQ] Weekly meeting

[Zhanhui Li]

@Von, @yukong

Bump it up in case you guys missed this email.

> 在 2018年5月20日，上午10:44，Kasthuriraajan Ratnasingam  
> 写道：
> 
> Hi Von,
> 
> What will be the communication medium for today's meeting(Project progress
> weekly meeting)?
> 
> At what time we will start the meeting?
> 
> Thanks ,
> 
> Best regards,
> R.Kasthuriraajan.
> 
> -- 
> R.Kasthuriraajan.
> Undergraduate student | Department of Computer Science,
> Faculty of Science | University of Jaffna.
> 
> LinkedIn: https://www.linkedin.com/in/ratnasingam-kasthuriraajan-2a6892121/
> 
> stackoverflow:
> https://stackoverflow.com/users/8870269/kasthuriraajan?tab=profile
> GitHub: https://github.com/kasthuriraajan
> Medium: https://medium.com/@Kasthuriraajan

Re: [GSOC|ROCKETMQ-124] Support non-redundant message delivery mechanism

[Zhanhui Li]

Hi Sohaib,

I have been sort of busy this these days. Sorry to reply you so late!

So sure what “deadline” you are referring to. If this is part of a campaign, I 
have to admit I am not aware of the regulations and what kind of help I should 
offer to maintain fairness considering other arising similar issues.

Regards!

Zhanhui Li


> 在 2018年3月1日，上午3:43，Sohaib Iftikhar <sohaib1...@gmail.com> 写道：
> 
> Hi guys,
> 
> Would be nice to have some feedback on this as the deadline is not too far :)
> 
> Thanks,
> Sohaib
> 
> Regards,
> Sohaib Iftikhar
> 
> -- Man is still the most extraordinary computer of all.--
> 
> 
> On Mon, Feb 26, 2018 at 10:36 AM, Sohaib Iftikhar <sohaib1...@gmail.com 
> <mailto:sohaib1...@gmail.com>> wrote:
> Thank you for the pointers to the code. This was super helpful. The multiple 
> keys can probably be serialized better than separating them with a space but 
> that is already legacy I suppose.
> 
> Firstly filters like bloom or cuckoo are heuristic. They can help make things 
> faster but definitely cannot be used as the only solution. Hence, in the end, 
> we will still need a persistent keystore/distributed set. My plan was to have 
> this keystore as distributed (raft guarantee etc.). The keystore can also 
> hold a persistent filter on its end. If a broker collapses it can 
> renew/refresh its filter from the keystore. Hence eliminating the problems 
> about crashes that you mention. The problem here could be in maintaining 
> performance for filters in case of removals from the keystore (for eg: 
> sliding windows as mentioned in my previous mail). Periodic refreshal of 
> filters can help solve this but I am open to suggestions on how to make this 
> better.
> 
> I think implementing a distributed set on the client cluster has its caveats. 
> The way I understand RocketMQ is that we do not have control over the 
> diskspace/memory on the client end. So we probably only have a constant 
> amount. A distributed set on the client would also need to be persistent. For 
> eg: if a client restarts/recovers etc. This basically means we need a 
> keystore on the client instead of the broker cluster. This probably puts too 
> much responsibility on the client cluster. A different approach would be to 
> ensure that the offsets are always in sync with the broker. Since the broker 
> only serves unique messages (based on the proposed solution on the 
> producer/broker end) all we need to ensure is that a client does not consume 
> messages with the same offset twice.
> 
> Please suggest improvements if this does not look like the correct approach. 
> Also would be great if someone can come up with a completely different 
> approach so that we can weigh up pros and cons.
> 
> Thanks for reading this through and looking forward to your opinions.
> 
> Regards,
> Sohaib
> 
> Regards,
> Sohaib Iftikhar
> 
> -- Man is still the most extraordinary computer of all.--
> 
> 
> On Mon, Feb 26, 2018 at 3:58 AM, Zhanhui Li <lizhan...@gmail.com 
> <mailto:lizhan...@gmail.com>> wrote:
> Hi Sohaib,
> 
> About multiple key support, the following code snippet should clarify your 
> doubt:
> org.apache.rocketmq.common.message.Message class has overloaded setKeys 
> methods, allowing your to set multiple keys via string(separated by 
> space…sorry, we have not yet unified all separators, hoping this does not 
> confuse you) or collection.
> 
> 
> When broker tries to build index for the message with multiple keys, multiple 
> index entries are inserted into the indexing file. 
> See org.apache.rocketmq.store.index.IndexService#buildIndex
> 
> 
> In terms of eliminating message duplication, personally, I wish we have a 
> feature of exactly-once semantic covering the whole cluster and the complete 
> send-store-consume processes. A rough idea is route the message according to 
> its unique key to a broker according to a rule; The serving broker ensures 
> uniqueness of the message according to the key( as you said, 
> bloom-filter/cuckoo-filter, etc);  Things might looks simple, but issues 
> resides in scenarios where cluster is experiencing membership changes: for 
> example, what if a broker crashed down? We might need propagate bloom-filter 
> bitset synchronously to other brokers having the same topics; What if a new 
> broker joins in the cluster and starts to serve? I do not mean this is too 
> complex to implement. Instead, this is a pretty interesting topic and fancy 
> feature to have. Alternatively, we might defer eliminating duplicates to the 
> consumption phase using kind of distributed set. For sure, my proposing idea 
> suffers the same challenges including membership changes. 
> 
&g

Re: [GSOC|ROCKETMQ-124] Support non-redundant message delivery mechanism

[Zhanhui Li]

Hi Sohaib,

Happy to know you are interested in RocketMQ.

First, let me answer questions you raised.

— can there be multiple tags?
No. At present, the storage engine allows single tag only. Subscriptions are 
allowed to use combination of tags. The current model should meet your business 
development. If not, please let us know.


— key (Similar question to above.)
RocketMQ builds index using message keys. A single message may have multiple 
keys.

— About redundant message
From my understanding, you are trying to eliminate duplicate messages. True 
there are various reasons which may cause message duplication, ranging from 
message delivery and consumption. Discussion on this topic is warmly welcome.  
Had you had any idea to contribute on this issue, the developer board is happy 
to discuss.

Zhanhui Li




> 在 2018年2月24日，上午11:17，Sohaib Iftikhar <sohaib1...@gmail.com> 写道：
> 
> My earlier email message seems to have gotten lost. So I will try again.
> Please see the original message for the discussion.
> 
> Regards,
> Sohaib Iftikhar
> 
> -- Man is still the most extraordinary computer of all.--
> 
> On Tue, Feb 20, 2018 at 1:54 AM, Sohaib Iftikhar <sohaib1...@gmail.com>
> wrote:
> 
>> Hi,
>> 
>> I am interested in working on this issue (https://issues.apache.org/
>> jira/browse/ROCKETMQ-124) as part of GSOC-18. I have a few questions for
>> the same. I am not sure if this discussion needs to be on the JIRA issue or
>> here. Feel free to correct me if this is the wrong platform. Also while I
>> have worked with distributed pub-sub systems I am still fairly new to
>> Rocket-MQ so maybe my understanding of it is incorrect. I apologise if that
>> is the case and would be happy to stand corrected.
>> 
>> Following are my questions:
>> 1. What defines a redundant message?
>>The constructor that I see for a message is as follows:
>>Message(String topic, String tags, String keys, int flag, byte[] body,
>> boolean waitStoreMsgOK)
>>Possible candidates to me are topic, tags (can there be multiple tags?
>> I could not find an example for this. If yes how are they separated?), keys
>> (Similar question to above.) and of course the body. Is there something
>> that I have missed in this? Is there something that we do not need to
>> consider?
>> 2. Is their a timeline on the redundant messages? What I mean by this is
>> that is there a time limit after which a message with similar content is
>> allowed. From what I gather there was no such thing mentioned. This would
>> mean storing all the messages. Depending on the requirements this may or
>> may not be the best solution. It might be desirable that no duplicates are
>> needed within a certain time window (sliding). This allows ignoring of
>> duplicate messages that were generated very close to each other (or in the
>> window indicated). Depending on this requirement implementation may become
>> a little bit more involved.
>> 
>> For now, these are the only questions. I have ideas that need review about
>> possible implementations but I will mention them once the specifications
>> are clear to me. As an end question, I would at some point like to post
>> design ideas to this problem privately to get it reviewed by the
>> development community but not make it publicly available so that it cannot
>> be plagiarised. What platform/method can I use to do that? Or is submitting
>> a draft to the Google platform the only possible way to accomplish this?
>> 
>> Thanks a lot for reading this through and looking forward to your inputs.
>> 
>> Regards,
>> Sohaib Iftikhar
>> 

Re: [DISCUSS] Migrate JIRA to Github Issue

[Zhanhui Li]

I am +1 to migration.

Creating a ticket on current JIRA is too slow.  

Zhanhui Li

> 在 2018年1月5日，下午10:54，yukon <yu...@apache.org> 写道：
> 
> Hi all,
> 
> Please note that we cannot migrate any existing Jira tickets to Github
> Issues. We need to start fresh and migrate the incomplete tickets to
> Github manually.
> 
> I will open a migrate ticket to INFRA next week if there is no opposite
> voice.
> 
> Regards,
> yukon
> 
> On Fri, Jan 5, 2018 at 5:39 PM, Xin Wang <data.xinw...@gmail.com> wrote:
> 
>> +1 for moving issues to github due to the bad network.
>> Besides we still need handle issues from Jira carefully; indeed active
>> community is one of the most important things.
>> 
>> Thanks,
>> Xin
>> 
>> 2018-01-04 11:47 GMT+08:00 dongeforever <dongefore...@apache.org>:
>> 
>>> +1
>>> 
>>> The JIRA system is truly slow in China, which makes it difficult to
>> create
>>> or update an ISSUE.
>>> It would be great to migrate the issues to github.
>>> 
>>> 
>>> Best Regards
>>> dongeforever
>>> 
>>> 
>>> 2018-01-02 21:24 GMT+08:00 yukon <yu...@apache.org>:
>>> 
>>>> Hi Justin,
>>>> 
>>>> RocketMQ is hosted on Gitbox now, and every software engineer has a
>>> proper
>>>> way to reach GitHub, no need to worry about it :)
>>>> 
>>>> Regards,
>>>> yukon
>>>> 
>>>> On Tue, Jan 2, 2018 at 8:22 PM, Justin Mclean <
>> jus...@classsoftware.com>
>>>> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>>> Consider that most of users and contributors are from China while
>> the
>>>>> JIRA system has a slow response for us, so I propose that RocketMQ
>>> should
>>>>> migrate from JIRA to Github issue for working efficiency.
>>>>>> 
>>>>>> Furthermore, the Github issue has a better integration with pull
>>>>> requests, could speed up the merge process.
>>>>> 
>>>>> Other Apache projects do the so it’s certainly possible. I’m not from
>>>>> China so forgive my ignorance on this matter but is there a concern
>>> that
>>>>> GitHub could be possibly blocked? I seem to remember it being blocked
>>>> for a
>>>>> time in 2016. Using Apache gitbox service [1] may get around that??
>>>>> 
>>>>> Thank,
>>>>> Justin
>>>>> 
>>>>> 1.  https://gitbox.apache.org
>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>> 
>> 
>> --
>> Thanks,
>> Xin
>>