This is an automated email from the ASF dual-hosted git repository. chenguangsheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-eventmesh-site.git
commit b6e9ef3fffc2e01d459a004010cecec3e4be9b49 Author: qqeasonchen <[email protected]> AuthorDate: Thu Jan 12 20:40:17 2023 +0800 update docs --- docs/README.md | 111 ++++ docs/contribute/01-release.md | 731 +++++++++++++++++++++++ docs/contribute/02-write-unit-test.md | 77 +++ docs/contribute/03-new-contributor-guidelines.md | 145 +++++ docs/contribute/_category_.json | 5 + docs/design-document/06-cloudevents.md | 106 ++++ docs/design-document/07-tracing.md | 87 +++ docs/design-document/09-event-bridge.md | 156 +++++ docs/design-document/10-knative-connector.md | 93 +++ docs/design-document/11-prometheus.md | 24 + docs/design-document/12-zipkin.md | 38 ++ docs/design-document/13-jaeger.md | 44 ++ docs/design-document/webhook.md | 270 +++++++++ docs/instruction/01-store-with-docker.md | 71 +++ docs/instruction/01-store.md | 44 ++ docs/instruction/02-runtime-with-docker.md | 107 ++++ docs/instruction/02-runtime.md | 107 ++++ docs/instruction/03-demo.md | 152 +++++ docs/instruction/_category_.json | 5 + docs/sdk-java/intro.md | 29 + 20 files changed, 2402 insertions(+) diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..aa3bd9a --- /dev/null +++ b/docs/README.md @@ -0,0 +1,111 @@ +<div align="center"> + +<br /><br /> +<img src="docs/images/logo.png" width="256"> +<br /> + +[](https://github.com/apache/incubator-eventmesh/actions/workflows/ci.yml) +[](https://codecov.io/gh/apache/incubator-eventmesh) +[](https://lgtm.com/projects/g/apache/incubator-eventmesh/context:java) +[](https://lgtm.com/projects/g/apache/incubator-eventmesh/alerts/) + +[](https://www.apache.org/licenses/LICENSE-2.0.html) +[](https://github.com/apache/incubator-eventmesh/releases) +[](https://join.slack.com/t/apacheeventmesh/shared_invite/zt-1hgto0wcm-KXKz8pJGUwrWdk00juaQSw) + +[📦 Documentation](https://eventmesh.apache.org/docs/introduction) | +[📔 Examples](https://github.com/apache/incubator-eventmesh/tree/master/eventmesh-examples) | +[⚙️ Roadmap](https://github.com/apache/incubator-eventmesh/blob/master/docs/en/roadmap.md) | +[🌐 简体中文](README.zh-CN.md) +</div> + + +# Apache EventMesh (Incubating) + +**Apache EventMesh (Incubating)** is a dynamic [event-driven](https://en.wikipedia.org/wiki/Event-driven_architecture) application multi-runtime used to decouple the application and backend middleware layer, which supports a wide range of use cases that encompass complex multi-cloud, widely distributed topologies using diverse technology stacks. + +### EventMesh Architecture + + + +### EventMesh Dashboard + + +## Features + +Apache EventMesh (Incubating) has a vast amount of features to help users achieve their goals. Let us share with you some of the key features EventMesh has to offer: + +- Built around the [CloudEvents](https://cloudevents.io) specification. +- Rapidty extendsible language sdk around [gRPC](https://grpc.io) protocols. +- Rapidty extendsible middleware by connectors such as [Apache RocketMQ](https://rocketmq.apache.org), [Apache Kafka](https://kafka.apache.org), [Apache Pulsar](https://pulsar.apache.org), [RabbitMQ](https://rabbitmq.com), [Redis](https://redis.io), [Pravega](https://cncf.pravega.io), and [RDMS](https://en.wikipedia.org/wiki/Relational_database)(in progress) using [JDBC](https://en.wikipedia.org/wiki/Java_Database_Connectivity). +- Rapidty extendsible controller such as [Consul](https://consulproject.org/en/), [Nacos](https://nacos.io), [ETCD](https://etcd.io) and [Zookeeper](https://zookeeper.apache.org/). +- Guaranteed at-least-once delivery. +- Deliver events between multiple EventMesh deployments. +- Event schema management by catalog service. +- Powerful event orchestration by [Serverless workflow](https://serverlessworkflow.io/) engine. +- Powerful event filtering and transformation. +- Rapid, seamless scalability to add new machines. + +## Roadmap + +Please go to the [roadmap](https://github.com/apache/incubator-eventmesh/blob/master/docs/en/roadmap.md) to get the release history and new features of Apache EventMesh (Incubating). + +## Quick start +Here are the guidelines: + +[Step 1: Deploy eventmesh-store](docs/en/instruction/01-store.md) + +[Step 2: Start eventmesh-runtime](docs/en/instruction/02-runtime.md) + +[Step 3: Run our demos](docs/en/instruction/03-demo.md) + +Besides, we also provide the docker-version guidelines for you if you prefer Docker: + +[Step 1: Deploy eventmesh-store using docker](docs/en/instruction/01-store-with-docker.md) + +[Step 2: Start eventmesh-runtime using docker](docs/en/instruction/02-runtime-with-docker.md) + +[Step 3: Run our demos](docs/en/instruction/03-demo.md) + +## Contributing + +Each contributor has played an important role in promoting the robust development of Apache EventMesh (Incubating). We sincerely appreciate all contributors who have contributed code and documents. + +- [Contributing Guideline](https://github.com/apache/incubator-eventmesh/blob/master/docs/en/contribute/03-new-contributor-guidelines.md) +- [Good First Issues](https://github.com/apache/incubator-eventmesh/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22) + +Here is the [List of Contributors](https://github.com/apache/incubator-eventmesh/graphs/contributors), thank you all! :) + +<a href="https://github.com/apache/incubator-eventmesh/graphs/contributors";> + <img src="https://contrib.rocks/image?repo=apache/incubator-eventmesh"; /> +</a> + + +## CNCF Landscape + +<div align="center"> + +<img src="https://landscape.cncf.io/images/left-logo.svg"; width="150"/> +<img src="https://landscape.cncf.io/images/right-logo.svg"; width="200"/> + +Apache EventMesh (Incubating) enriches the <a href="https://landscape.cncf.io/serverless?license=apache-license-2-0";>CNCF Cloud Native Landscape.</a> + +</div> + +## License + +Apache EventMesh (Incubating) is licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html). + +## Community + +|WeChat Assistant|WeChat Public Account|Slack| +|-|-|-| +|<img src="docs/images/contact/wechat-assistant.jpg" width="128"/>|<img src="docs/images/contact/wechat-official.jpg" width="128"/>|[Join Slack Chat](https://join.slack.com/t/apacheeventmesh/shared_invite/zt-1hgto0wcm-KXKz8pJGUwrWdk00juaQSw)| + +### Mailing List + +|Name|Description|Subscribe|Unsubscribe|Archive +|-|-|-|-|-| +|Users|User discussion|[Subscribe](mailto:[email protected])|[Unsubscribe](mailto:[email protected])|[Mail Archives](https://lists.apache.org/[email protected])| +|Development|Development discussion (Design Documents, Issues, etc.)|[Subscribe](mailto:[email protected])|[Unsubscribe](mailto:[email protected])|[Mail Archives](https://lists.apache.org/[email protected])| +|Commits|Commits to related repositories| [Subscribe](mailto:[email protected]) |[Unsubscribe](mailto:[email protected]) |[Mail Archives](https://lists.apache.org/[email protected])| diff --git a/docs/contribute/01-release.md b/docs/contribute/01-release.md new file mode 100644 index 0000000..0810a75 --- /dev/null +++ b/docs/contribute/01-release.md @@ -0,0 +1,731 @@ +# Release Creation Process + +:::caution +The documentation of Release Creation Process is WIP (Work-in-Progress). +::: + +## 理解 Apache 发布的内容和流程 + +Source Release 是 Apache 关注的重点，也是发布的必须内容；而 Binary Release 是可选项， + +请参考以下链接，找到更多关于 ASF 的发布指南: + +- [Apache Release Guide](http://www.apache.org/dev/release-publishing) +- [Apache Release Policy](http://www.apache.org/dev/release.html) +- [Maven Release Info](http://www.apache.org/dev/publishing-maven-artifacts.html) + + +## 本地构建环境准备 + +主要包括签名工具、Maven 仓库认证相关准备 + +### 1.安装GPG + +在[GnuPG官网](https://www.gnupg.org/download/index.html)下载安装包。GnuPG的1.x版本和2.x版本的命令有细微差别，下列说明以**GnuPG-2.x**版本为例 + +```sh +$ gpg --version #检查版本，应该为2.x +``` + +### 2.用gpg生成key + +根据提示，生成 key + +> 注意：请使用Apache邮箱生成GPG的Key + +```shell +$ gpg --full-gen-key +gpg (GnuPG) 2.0.12; Copyright (C) 2009 Free Software Foundation, Inc. +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. + +Please select what kind of key you want: + (1) RSA and RSA (default) + (2) DSA and Elgamal + (3) DSA (sign only) + (4) RSA (sign only) +Your selection? 1 +RSA keys may be between 1024 and 4096 bits long. +What keysize do you want? (2048) 4096 +Requested keysize is 4096 bits +Please specify how long the key should be valid. + 0 = key does not expire + <n> = key expires in n days + <n>w = key expires in n weeks + <n>m = key expires in n months + <n>y = key expires in n years +Key is valid for? (0) +Key does not expire at all +Is this correct? (y/N) y + +GnuPG needs to construct a user ID to identify your key. + +Real name: ${输入用户名} +Email address: ${邮箱地址} +Comment: CODE SIGNING KEY +You selected this USER-ID: + "${输入用户名} (CODE SIGNING KEY) <${邮箱地址}>" + +Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O +You need a Passphrase to protect your secret key. # 填入密码，以后打包过程中会经常用到 +``` + +### 3.查看 key + +```shell +$ gpg --list-keys +pub rsa4096/579C25F5 2021-04-26 # 579C25F5就是key id +uid [ultimate] ${输入用户名} <${邮箱地址}> +sub rsa4096 2021-04-26 + +# 通过key id发送public key到keyserver +$ gpg --keyserver pgpkeys.mit.edu --send-key 579C25F5 +# 其中，pgpkeys.mit.edu为随意挑选的keyserver，keyserver列表为：https://sks-keyservers.net/status/，相互之间是自动同步的，选任意一个都可以。 +$ gpg --keyserver hkp://pgpkeys.mit.edu --recv-keys 579C25F5 # 验证是否同步到公网，网络不好可能需多试几次 +``` + +**注：如果有多个 public key，设置默认 key。**修改`~/.gnupg/gpg.conf` + +```sh +# If you have more than 1 secret key in your keyring, you may want to +# uncomment the following option and set your preferred keyid. +default-key 28681CB1 +``` + +**如果有多个 public key, 也可以删除无用的 key：** + +```shell +$ gpg --delete-secret-keys 29BBC3CB # 先删除私钥，指明key id +gpg (GnuPG) 2.2.27; Copyright (C) 2021 g10 Code GmbH +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. + +sec rsa4096/EE8DAE7D29BBC3CB 2021-04-27 mikexue <[email protected]> + +Delete this key from the keyring? (y/N) y +This is a secret key! - really delete? (y/N) y +``` + +```shell +$ gpg --delete-keys 29BBC3CB # 删除公钥，指明key id +gpg (GnuPG) 2.2.27; Copyright (C) 2021 g10 Code GmbH +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. + + +pub rsa4096/EE8DAE7D29BBC3CB 2021-04-27 mikexue <[email protected]> + +Delete this key from the keyring? (y/N) y +``` + +由于公钥服务器没有检查机制，任何人都可以用你的名义上传公钥，所以没有办法保证服务器上的公钥的可靠性。 通常，你可以在网站上公布一个公钥指纹，让其他人核对下载到的公钥是否为真。 + +```shell +# fingerprint参数生成公钥指纹： +$gpg --fingerprint mikexue +pub rsa4096 2021-04-26 [SCA] + F84A 0041 D70B 37AF 9C7B F0B3 39F4 29D7 579C 25F5 +uid [ultimate] mikexue <[email protected]> +sub rsa4096 2021-04-26 [E] +``` + +登录 [https://id.apache.org](https://id.apache.org/), 将上面的 fingerprint （即 F84A 0041 D70B 37AF 9C7B F0B3 39F4 29D7 579C 25F5） 粘贴到自己的用户信息中 OpenPGP Public Key Primary Fingerprint + + + +## 发布Apache Maven仓库 + +> 注：EventMesh使用Gradle构建，需修改gradle相关配置 + +### 1.导出私钥文件 + +```shell +$ gpg --export-secret-keys -o secring.gpg #私钥文件妥善保管，后面配置需要 +``` + +### 2.准备分支 + +从主干分支拉取新分支作为发布分支，如现在要发布$`{release_version}`版本，则从develop分支拉出新分支`${release_version}-release`，此后`${release_version}` Release Candidates涉及的修改及打标签等都在`${release_version}-release`分支进行，最终发布完成后合入主干分支。 + +### 3.更新版本说明 + +更新官网项目的如下文件，并提交至master分支： + +https://github.com/apache/incubator-eventmesh-site/tree/master/events/release-notes + +### 4.配置根项目下gradle.properties文件 + +```shell +group=org.apache.eventmesh +version=1.2.0-release +#40位公钥的最后8位 +signing.keyId=579C25F5 +#生成密钥时填的passphrase +signing.password= +#导出的私钥文件secring.gpg路径 +signing.secretKeyRingFile=../secring.gpg +#apache 账号 +apacheUserName= +#apache 密码 +apachePassWord= +``` + +### 5.检查子模块下gradle.properties文件 + +```shell +group=org.apache.eventmesh +version=${release_version} +``` + +### 6.检查并配置根项目下build.gradle文件 + +```shell +publishing { + publications { + mavenJava(MavenPublication) { + from components.java + artifact packageSources + artifact packageJavadoc + versionMapping { + usage('java-api') { + fromResolutionOf('runtimeClasspath') + } + usage('java-runtime') { + fromResolutionResult() + } + } + pom { + name = 'EventMesh' + description = 'Apache EventMesh' + url = 'https://github.com/apache/incubator-eventmesh' + licenses { + license { + name = 'The Apache License, Version 2.0' + url = 'http://www.apache.org/licenses/LICENSE-2.0.txt' + } + } + developers { + developer { + id = 'Apache EventMesh(incubating)' + name = 'Apache EventMesh(incubating) of ASF' + url = 'https://eventmesh.apache.org/' + } + } + scm { + connection = 'scm:git:[email protected]:apache/incubator-eventmesh.git' + developerConnection = 'scm:git:[email protected]:apache/incubator-eventmesh.git' + url = 'https://github.com/apache/incubator-eventmesh' + } + } + } + } + repositories { + maven { + def releasesRepoUrl = 'https://repository.apache.org/service/local/staging/deploy/maven2/' + def snapshotsRepoUrl = 'https://repository.apache.org/content/repositories/snapshots/' + url = version.endsWith('SNAPSHOT') ? snapshotsRepoUrl : releasesRepoUrl + credentials { + username apacheUserName + password apachePassWord + } + + } + } +} + +signing { + sign publishing.publications.mavenJava +} +``` + +### 7.上传发布包 + +执行如下命令，需要对jar、源码包、doc和pom等文件签名加密 + +```shell +$ gradle signMavenJavaPublication publish +``` + +上述命令执行成功后，待发布版本会自动上传到Apache的临时筹备仓库(staging repository)。所有被deploy到远程[maven仓库](http://repository.apache.org/)的Artifacts都会处于staging状态，访问https://repository.apache.org/#stagingRepositories, 使用Apache的LDAP账户登录后，就会看到上传的版本，`Repository`列的内容即为${STAGING.REPOSITORY}。 点击`Close`来告诉Nexus这个构建已经完成，只有这样该版本才是可用的。 如果电子签名等出现问题，`Close`会失败，可以通过`Activity`查看失败信息。 + + + +## 发布Apache SVN仓库 + +### 1.准备svn本机环境（Apache使用svn托管项目的发布内容） + +### 2.checkout到本地目录 + +```shell +$ svn checkout https://dist.apache.org/repos/dist/dev/incubator/eventmesh/ +# 假定本地目录为 ~/apache/eventmesh +``` + +### 3.添加gpg公钥 + +添加public key到[KEYS](https://dist.apache.org/repos/dist/dev/incubator/eventmesh/KEYS)文件并提交到SVN仓库（第一次做发布的人需要做这个操作，具体操作参考KEYS文件里的说明）。KEYS主要是让参与投票的人在本地导入，用来校验sign的正确性 + +Windows + +```sh +$ gpg --list-sigs <your name> | out-file -append KEYS -encoding utf8 +$ gpg --armor --export <your name> | out-file -append KEYS -encoding utf8 +``` + +> Mac OS/Linux + +```sh +$ (gpg --list-sigs <your name> && gpg --armor --export <your name>) >> KEYS +``` + +### 4.添加待发布内容到SVN目录 + +```shell +$ cd ~/apache/eventmesh # eventmesh svn根目录 +$ mkdir ${release_version}-${rc_version} +``` + +#### 4.1 创建tag + +在`${release_version}-release`分支上创建tag，需带有rc版本，为预发布版本 + +```shell +$ git tag -a v{$release_version}-{$rc_version} -m "Tagging the ${release_version} first Release Candidate (Candidates start at zero)" +$ git push origin --tags +``` + +#### 4.2 打包源码 + +检查项目源码命名，将源码命名为`apache-eventmesh-${release_version}-incubating-src`，将源码打包为tar.gz格式 + +```shell +$ tar -czvf apache-eventmesh-${release_version}-incubating-source.tar.gz apache-eventmesh-${release_version}-incubating-src +``` + +#### 4.3 打包二进制 + +> 编译上一步打包的源码 + +检查编译后的文件命名，将二进制文件命名为`apache-eventmesh-${release_version}-incubating` + +> 注：需将源码根目录下的`NOTICE`文件，`DISCLAIMER-WIP`文件以及`tools/third-party-licenses`目录下的`LICENSE`文件拷贝到二进制的包中 + +```shell +$ gradle clean jar dist && gradle installPlugin && gradle tar -x test +$ tar -czvf apache-eventmesh-${release_version}-incubating-bin.tar.gz apache-eventmesh-${release_version}-incubating +``` + +压缩source包、bin包，并将相关的压缩包拷贝到svn本地仓库下`/apache/eventmesh/${release_version}-${rc_version}` + +### 5.生成签名/sha512文件 + +> 针对源码包与二进制包生成签名/sha512文件 + +```shell +$ for i in *.tar.gz; do echo $i; gpg --print-md SHA512 $i > $i.sha512 ; done #计算sha512 +$ for i in *.tar.gz; do echo $i; gpg --armor --output $i.asc --detach-sig $i ; done #计算签名 +``` + +### 6.提交到Apache svn + +```shell +$ cd ~/apache/eventmesh # eventmesh svn根目录 +$ svn status +$ svn commit -m 'prepare for ${release_version}-${rc_version}' +``` + + + +## 验证Release Candidates + +详细检查列表请参考官方的[check list](https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist) + +从以下地址下载要发布的Release Candidates到本地环境： + +```shell +https://dist.apache.org/repos/dist/dev/incubator/eventmesh/${release_version}-${rc_version}/ +``` + +然后开始验证环节，验证包含但不限于以下内容和形式 + +### 1.检查签名和hash等信息 + +> 由于操作系统不同，检查的命令或有差异，具体可参考[官方检查步骤](https://www.apache.org/info/verification.html) + +#### 1.1检查sha512哈希 + +> Mac OS/Linux + +```shell +$ shasum -a apache-eventmesh-${release_version}-incubating-source.tar.gz +#并将输出内容与 apache-eventmesh-${release_version}-${rc_version}-incubating-source.tar.gz.sha512文件内容作对比 +$ shasum -a apache-eventmesh-${release_version}-incubating-bin.tar.gz +#并将输出内容与 apache-eventmesh-${release_version}-${rc_version}-incubating-bin.tar.gz.sha512文件内容作对比 +``` + +> Windows + +```shell +$ certUtil -hashfile apache-eventmesh-${release_version}-incubating-source.tar.gz SHA512 +#并将输出内容与 apache-eventmesh-${release_version}-${rc_version}-incubating-source.tar.gz.sha512文件内容作对比 +$ certUtil -hashfile apache-eventmesh-${release_version}-incubating-bin.tar.gz SHA512 +#并将输出内容与 apache-eventmesh-${release_version}-${rc_version}-incubating-bin.tar.gz.sha512文件内容作对比 +``` + +#### 1.2检查gpg签名 + +首先导入发布人公钥。从svn仓库导入KEYS到本地环境。（发布版本的人不需要再导入，帮助做验证的人需要导入，用户名填发版人的即可） + +```shell +$ curl https://dist.apache.org/repos/dist/dev/incubator/eventmesh/KEYS >> KEYS +$ gpg --import KEYS +$ gpg --edit-key "${发布人的gpg用户名}" + > trust + +Please decide how far you trust this user to correctly verify other users' keys +(by looking at passports, checking fingerprints from different sources, etc.) + + 1 = I don't know or won't say + 2 = I do NOT trust + 3 = I trust marginally + 4 = I trust fully + 5 = I trust ultimately + m = back to the main menu + +Your decision? 5 + + > save +``` + +然后使用如下命令检查签名 + +```shell +$ gpg --verify apache-eventmesh-${release_version}-incubating-source.tar.gz.asc apache-eventmesh-${release_version}-incubating-source-tar.gz +$ gpg --verify apache-eventmesh-${release_version}-incubating-bin.tar.gz.asc apache-eventmesh-${release_version}-incubating-bin.tar.gz +``` + +### 2.检查源码包的文件内容 + +解压缩`apache-eventmesh-${release_version}-incubating-source-tar.gz`，进行如下检查: + +- 检查源码包是否包含由于包含不必要文件，致使tar包过于庞大 +- 文件夹包含单词`incubating` +- 存在`LICENSE`和`NOTICE`文件 +- 存在`DISCLAIMER`文件 +- `NOTICE`文件中的年份正确 +- 只存在文本文件，不存在二进制文件 +- 所有文件的开头都有ASF许可证 +- 能够正确编译，单元测试可以通过 (./gradle build) (目前支持JAVA 8/gradle 7.0/idea 2021.1.1及以上) +- 检查是否有多余文件或文件夹，例如空文件夹等 + +### 3.检查二进制包的文件内容 + +- 文件夹包含单词`incubating` +- 存在`LICENSE`和`NOTICE`文件 +- 存在`DISCLAIMER`文件 +- `NOTICE`文件中的年份正确 +- 所有文本文件开头都有ASF许可证 +- 检查第三方依赖许可证： + - 第三方依赖的许可证兼容 + - 所有第三方依赖的许可证都在`LICENSE`文件中声名 + - 依赖许可证的完整版全部在`license`目录 + - 如果依赖的是Apache许可证并且存在`NOTICE`文件，那么这些`NOTICE`文件也需要加入到版本的`NOTICE`文件中 + +你可以参考此文章：[ASF第三方许可证策](https://apache.org/legal/resolved.html) + +## 发起投票 + +> EventMesh 仍在孵化阶段，需要进行两次投票 + +- EventMesh社区投票，发送邮件至：`[email protected]` +- incubator社区投票，发送邮件至：`[email protected]` EventMesh毕业后，只需要在EventMesh社区投票 + +### 1.EventMesh社区投票阶段 + +1. EventMesh社区投票，发起投票邮件到`[email protected]`。PMC需要先按照文档检查版本的正确性，然后再进行投票。 经过至少72小时并统计到3个`+1 PMC member`票后，即可进入下一阶段的投票。 +2. 宣布投票结果,发起投票结果邮件到`[email protected]`。 + +### 2.EventMesh社区投票模板 + +标题： + +``` +[VOTE] Release Apache EventMesh (incubating) ${release_version} ${rc_version} +``` + +正文： + +``` +Hello EventMesh Community, + + This is a call for vote to release Apache EventMesh (incubating) version ${release_version}-${rc_version}. + + Release notes: + https://github.com/apache/incubator-eventmesh/releases/tag/v${release_version}-${rc_version} + + The release candidates: + https://dist.apache.org/repos/dist/dev/incubator/eventmesh/${release_version}-${rc_version}/ + + Maven artifacts are available in a staging repository at: + https://repository.apache.org/content/repositories/orgapacheeventmesh-{staging-id} + + Git tag for the release: + https://github.com/apache/incubator-eventmesh/tree/v${release_version}-${rc_version} + + Keys to verify the Release Candidate: + https://downloads.apache.org/incubator/eventmesh/KEYS + + Hash for the release tag: + #hashCode of this release tag + + GPG user ID: + ${YOUR.GPG.USER.ID} + + The vote will be open for at least 72 hours or until necessary number of votes are reached. + + Please vote accordingly: + + [ ] +1 approve + + [ ] +0 no opinion + + [ ] -1 disapprove with the reason + + Checklist for reference: + + [ ] Download links are valid. + + [ ] Checksums and PGP signatures are valid. + + [ ] Source code distributions have correct names matching the current release. + + [ ] LICENSE and NOTICE files are correct for each EventMesh repo. + + [ ] All files have license headers if necessary. + + [ ] No compiled archives bundled in source archive. + + More detail checklist please refer: + https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist + +Thanks, +Your EventMesh Release Manager +``` + +### 3.宣布投票结果模板 + +标题： + +``` +[RESULT][VOTE] Release Apache EventMesh (incubating) ${release_version} ${rc_version} +``` + +正文： + +``` +Hello Apache EventMesh PPMC and Community, + + The vote closes now as 72hr have passed. The vote PASSES with + + xx (+1 non-binding) votes from the PPMC, + xx (+1 binding) votes from the IPMC, + xx (+1 non-binding) votes from the rest of the developer community, + and no further 0 or -1 votes. + + The vote thread: {vote_mail_address} + + I will now bring the vote to [email protected] to get approval by the IPMC. + If this vote passes also, the release is accepted and will be published. + +Thank you for your support. +Your EventMesh Release Manager +``` + +### 4.Incubator社区投票阶段 + +1. Incubator社区投票，发起投票邮件到`[email protected]`，需3个 `+1 IPMC Member`投票，方可进入下一阶段。 +2. 宣布投票结果,发起投票结果邮件到`[email protected]` 并抄送至`[email protected]`。 + +### 5.Incubator社区投票模板 + +标题： + +``` +[VOTE] Release Apache EventMesh (incubating) ${release_version} ${rc_version} +``` + +内容： + +``` +Hello Incubator Community, + + This is a call for a vote to release Apache EventMesh(Incubating) version ${release_version} ${rc_version} + + The Apache EventMesh community has voted on and approved a proposal to release + Apache EventMesh(Incubating) version ${release_version} ${rc_version} + + We now kindly request the Incubator PMC members review and vote on this + incubator release. + + EventMesh community vote thread: + • [投票链接] + + Vote result thread: + • [投票结果链接] + + The release candidate: + •https://dist.apache.org/repos/dist/dev/incubator/eventmesh/${release_version}-${rc_version}/ + + Git tag for the release: + • https://github.com/apache/incubator-eventmesh/tree/${release_version}-${rc_version} + Release notes: + • https://github.com/apache/incubator-eventmesh/releases/tag/${release_version}-${rc_version} + + The artifacts signed with PGP key [填写你个人的KEY], corresponding to [填写你个人的邮箱], that can be found in keys file: + • https://downloads.apache.org/incubator/eventmesh/KEYS + + The vote will be open for at least 72 hours or until necessary number of votes are reached. + + Please vote accordingly: + + [ ] +1 approve + [ ] +0 no opinion + [ ] -1 disapprove with the reason + +Thanks, +On behalf of Apache EventMesh(Incubating) community +``` + +### 6.宣布投票结果模板 + +标题： + +``` +[RESULT][VOTE] Release Apache EventMesh (incubating) ${release_version} ${rc_version} +``` + +内容： + +``` +Hi all, + + Thanks for reviewing and voting for Apache EventMesh(Incubating) version ${release_version} ${rc_version} release, I am happy to announce the release voting has passed with [投票结果数] binding votes, no +0 or -1 votes. + + Binding votes are from IPMC + - xxx + - xxx + - xxx + + Non-binding votes: + +1 xxx + +0 xxx + -1 xxx + + The voting thread is: + • [投票结果链接] + + Many thanks for all our mentors helping us with the release procedure, and all IPMC helped us to review and vote for Apache EventMesh(Incubating) release. I will be working on publishing the artifacts soon. + +Thanks, +On behalf of Apache EventMesh(Incubating) community +``` + +## 正式发布 + +### 1.合并分支 + +合并`${release_version}-release`分支的改动到`master`分支，合并完成后删除`release`分支 + +```shell +$ git checkout master +$ git merge origin/${release_version}-release +$ git pull +$ git push origin master +$ git push --delete origin ${release_version}-release +$ git branch -d ${release_version}-release +``` + +### 2.迁移源码与二进制包 + +将源码和二进制包从svn的`dev`目录移动到`release`目录 + +```shell +$ svn mv https://dist.apache.org/repos/dist/dev/incubator/eventmesh/${release_version}-${rc_version} https://dist.apache.org/repos/dist/release/incubator/eventmesh/ -m "transfer packages for ${release_version}-${rc_version}" #移动源码包与二进制包 +$ svn delete https://dist.apache.org/repos/dist/release/incubator/eventmesh/KEYS -m "delete KEYS" #清除原有release目录下的KEYS +$ svn cp https://dist.apache.org/repos/dist/dev/incubator/eventmesh/KEYS https://dist.apache.org/repos/dist/release/incubator/eventmesh/ -m "transfer KEYS for ${release_version}-${rc_version}" #拷贝dev目录KEYS到release目录 +``` + +### 3.确认dev和release下的包是否正确 + +- 确认[dev](https://dist.apache.org/repos/dist/dev/incubator/eventmesh/)下的`${release_version}-${rc_version}`已被删除 +- 删除[release](https://dist.apache.org/repos/dist/release/incubator/eventmesh/)目录下上一个版本的发布包，这些包会被自动保存在[这里](https://archive.apache.org/dist/incubator/eventmesh/) + +```shell +$ svn delete https://dist.apache.org/repos/dist/release/incubator/eventmesh/${last_release_version} -m "Delete ${last_release_version}" +``` + +### 4.在Apache Staging仓库发布版本 + +- 登录http://repository.apache.org , 使用Apache账号登录 +- 点击左侧的Staging repositories， +- 搜索EventMesh关键字，选择你最近上传的仓库，投票邮件中指定的仓库 +- 点击上方的`Release`按钮，这个过程会进行一系列检查 + +> 等仓库同步到其他数据源，一般需要24小时 + +### 5.GitHub版本发布 + +1.Tag the commit (on which the vote happened) with the release version without `-${RELEASE_CANDIDATE}`. 例如：after a successful vote on `v1.2-rc5`, the hash will be tagged again with `v1.2` only. + +2.在 [GitHub Releases](https://github.com/apache/incubator-eventmesh/releases) 页面的 `${release_version}` 版本上点击 `Edit` + +编辑版本号及版本说明，并点击 `Publish release` + +### 6.更新下载页面 + +等待并确认新的发布版本同步至 Apache 镜像后，更新如下页面： + +https://eventmesh.apache.org/download/ + +https://eventmesh.apache.org/zh/download/ + +GPG签名文件和哈希校验文件的下载连接应该使用这个前缀：`https://downloads.apache.org/incubator/eventmesh/` + +> 注意：项目下载链接应该使用 https://www.apache.org/dyn/closer.lua 而不是 closer.cgi 或者 mirrors.cgi + +### 7.邮件通知版本发布完成 + +> 请确保Apache Staging仓库已发布成功，一般是在该步骤的24小时后发布邮件 + +发邮件到 `[email protected]` 、 `[email protected]`和`[email protected]` + +标题： + +``` +[ANNOUNCE] Apache EventMesh (incubating) ${release_version} available +``` + +正文： + +``` +Hi all, + +Apache EventMesh (incubating) Team is glad to announce the new release of Apache EventMesh (incubating) ${release_version}. + +Apache EventMesh (incubating) is a dynamic cloud-native eventing infrastructure used to decouple the application and backend middleware layer, which supports a wide range of use cases that encompass complex multi-cloud, widely distributed topologies using diverse technology stacks. + +Download Links: https://eventmesh.apache.org/projects/eventmesh/download/ + +Release Notes: https://eventmesh.apache.org/events/release-notes/v${release_version}/ + +Website: https://eventmesh.apache.org/ + +EventMesh Resources: +- Issue: https://github.com/apache/incubator-eventmesh/issues +- Mailing list: [email protected] + + + +Apache EventMesh (incubating) Team +``` + diff --git a/docs/contribute/02-write-unit-test.md b/docs/contribute/02-write-unit-test.md new file mode 100644 index 0000000..0c48e46 --- /dev/null +++ b/docs/contribute/02-write-unit-test.md @@ -0,0 +1,77 @@ +# Unit Test Requirement + +- Each unit test case should use assertions instead of `System.out` output or `if` statement +- Each unit test case shouldn't call other cases or depend on the order of execution. +- Each unit test case should be repeatable and not depend on the external environment because the test might be executed in the continuous integration. +- The scope of each unit test should be small enough to help locate the problem at the method level. + +## Location and Naming Rules + +- The unit test should be placed in `src/test/java`. +- The unit test configuration file should be placed in `src/test/resources`. For example: + - Class to be tested: `src/main/java/org/apache/eventmesh/common/protocol/http/body/BaseResponseBody.java` + - Unit test: `src/test/java/org/apache/eventmesh/common/protocol/http/body/BaseResponseBodyTest.java` + - Unit test configuration: `src/test/resources/configuration.properties` +- The package name of the unit test class should be identical to the class to be tested. +- The name of the unit test class should be `{class or interface to be tested}Test`. For example: + - Class to be tested: `EventMeshUtil` + - Unit test class: `EventMeshUtilTest` +- The name of each test case should be `test{method name}`. For example: + - Method to be tested: `addProp(String key, String val)` + - Unit test case: `testAddProp` + +## Assertion Usage + +### Common Assertion + +| Methods | Instructions | +| :-------------- | :-------------- | +| `assertEquals` | Determines whether two objects or primitive types are equal | +| `assertNotEquals` | Determines whether two objects or primitive types are not equal | +| `assertTrue` | Determines whether the given Boolean value is `true` | +| `assertFalse` | Determines whether the given Boolean value is `false` | +| `assertNull` | Determines whether the given object reference is `null` | +| `assertNotNull` | Determines whether the given object reference is not `null` | +| `assertAll` | When multiple decision logic are processed together if only one error is reported, the whole test will fail | + +### Example + +#### `assertEquals()` + +```java +configuration.init(); +Assert.assertEquals("value1", configuration.eventMeshEnv); +``` + +#### `assertTrue()` + +```java +BaseResponseHeader header = BaseResponseHeader.buildHeader("200"); +Assert.assertTrue(header.toMap().containsKey(ProtocolKey.REQUEST_CODE)); +``` + +#### `assertFalse()` + +```java +Class<NacosRegistryService> nacosRegistryServiceClass = NacosRegistryService.class; +Field initStatus = nacosRegistryServiceClass.getDeclaredField("INIT_STATUS"); +initStatus.setAccessible(true); +Object initStatusField = initStatus.get(nacosRegistryService); +Assert.assertFalse((Boolean.parseBoolean(initStatusField.toString()))); +``` + +#### `assertNull()` + +```java +DefaultFullHttpResponse response = httpCommand.httpResponse(); +Assert.assertNull(response); +``` + +#### `assertNotNull()` + +```java +Codec.Decoder cd = new Codec.Decoder(); +ArrayList<Object> result = new ArrayList<>(); +cd.decode(null, buf, result); +Assert.assertNotNull(result.get(0)); +``` diff --git a/docs/contribute/03-new-contributor-guidelines.md b/docs/contribute/03-new-contributor-guidelines.md new file mode 100644 index 0000000..1099734 --- /dev/null +++ b/docs/contribute/03-new-contributor-guidelines.md @@ -0,0 +1,145 @@ + +# How to Contribution + +If you are a new contributor who wants to contribute to the eventmesh community, please read this document, which describes how to contribute to the community, and if you find any questions in the document, feel free to leave comments or suggestions. + +## Preparation + +### Development environment + +- You should have the JDK installed in your development environment. + +### Code Style + +Import [EventMesh CheckStyle](https://github.com/apache/incubator-eventmesh/blob/master/style/checkStyle.xml) file to your IDEA. + +For IDEA, you can import check style file by: +```shell + Editor -> Code Style -> Java -> Scheme -> Import Scheme -> CheckStyle Configuration +``` + +If you can't see CheckStyle Configuration section under Import Scheme, you can install CheckStyle-IDEA plugin first, and you will see it. + +You can also use `./gradlew check` to check the code style. +(Note: this command will check all file in project, when you submit a pr, the ci will only check the file has been changed in this pr). + +### Workflow + +Here are the workflow for contributors: + +1. Fork to your own + +2. Clone fork to local repository +```git +git clone [email protected]:yourgithub/incubator-eventmesh.git +``` + +3. Create a new branch and work on it +```git +git checkout -b fix_patch_xx +``` + +4. Keep your branch in sync +```git +git remote add upstream [email protected]:apache/incubator-eventmesh.git +git fetch upstream master:upstream_master +git rebase upstream_master +``` + +5. Commit your changes (make sure your commit message concise) + +6. Push your commits to your forked repository + +7. Create a pull request + +## Explanation + +The original warehouse: https://github.com/apache/incubator-eventmesh The apache warehouse of eventmesh is called the original warehouse in the text. + +The Fork library: From https://github.com/apache/eventmesh fork to your own personal repository to become a fork library. + +So fork the original EventMesh repository into your own repository. + +## Development branch + +**The current development branch of eventmesh is Master. Please submit PR to this branch.** + +- We recommend that you create a new branch in your repository for development and submit the branch to the master branch of eventmesh. + +## Contribution Categories + +### Bug feedback or bug fixes + +- Whether it's a bug feedback or a fix, an issue needs to be created first to describe the bug in detail, so that the community can easily find and view the problem and code through the issue record. bug feedback issues usually need to contain a complete description of the bug information and reproducible scenarios. + +### Implementation of functions, refactoring + +- If you plan to implement a new feature (or refactoring), be sure to communicate with the eventmesh core development team via an Issue or other means, and describe the new feature (or refactoring), mechanism and scenario in detail during the communication process. + +### Documentation Improvement + +- You can find the eventmesh documentation at [eventmesh-docs](https://github.com/apache/incubator-eventmesh/tree/master/docs), and the documentation is supplemented or improved in a way that is also essential for eventmesh. + +## Contribution method + +There are two ways for new contributors to contribute to the eventmesh community: + +- If you find a bug in the eventmesh code that you want to fix, or if you provide a new feature for the eventmesh, submit an issue in the eventmesh and submit a pr to the eventmesh. + +- Other contributors in the eventmesh community have raised issues, the [`issue for first-time contributors`](https://github.com/apache/incubator-eventmesh/issues/888) sorted out by the community here are relatively simple PR, which can help you familiarize yourself with the process of making contributions to the eventmesh community. + +## Submit issue guidelines + +- If you don't know how to raise an issue on eventmesh, please read [about the issue](https://docs.github.com/cn/issues/tracking-your-work-with-issues/quickstart). + +- In the eventmesh community, there are issue templates that can be used for reference, if the type matches please use the template, if the issue template does not meet your requirements, you can open an empty issue template, for the issue please bring its matching feature labels. + +- For the name of the issue, please briefly describe your question or purpose in one sentence, and write in English for better global communication. + +## pull request (pr) submission guidelines + +- If you don't know how to initiate a pr for eventmesh, please see [about pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request). + +- Whether it's a bug fix, or a new feature development (if this pr is a new feature development, then documentation updates about the new feature should be included in this pr), please submit a PR to the current development branch master. + +- The pr submission should follow the template provided by eventmesh as well as the need to write the submission information, a brief description of what the pr you are submitting does is sufficient, please see the [template for details](https://github.com/apache/incubator-eventmesh/blob/master/.github/PULL_REQUEST_TEMPLATE.md). + +- The pr you submit needs to be associated with the issue you are fixing, or the issue you are raising,so your PR title should start with [ISSUE #xx]. + +- If your change is about a typo or small optimize, you needn't create an Issue, just submit a PR and title with [MINOR]. + +**Note:** + + - A single pull request should not be too large. If heavy changes are required, it's better to separate the changes to a few individual PRs. + + - After creating a PR, one or more committers will help to review the pull request, after approve, this PR will be merged in to eventmesh repository, and the related Issue will be closed. + +## review + +### PR review + +All code should be well reviewed by one or more committers. Some principles: + +- Readability: Important code should be well-documented. Comply with our [code style](https://github.com/apache/incubator-eventmesh/blob/master/style/checkStyle.xml). + +- Elegance: New functions, classes or components should be well-designed. + +- Testability: Important code should be well-tested (high unit test coverage). + +### License review + +EventMesh follows [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0.html) policy. All source files should +have the Apache License header added to the file header. EventMesh uses the [apache/skywalking-eyes](https://github.com/apache/skywalking-eyes) to check +the source file header. + +### PR merge + +After a PR is approved by at least one committer, it can be merged. Before the merge, the committer can make changes to the commits message, requiring the commits +message to be clear without duplication, and use Squash and Merge to make sure one PR should only contain one commits. +For large multi-person PR, use Merge to merge, and fix the commits by rebase before merging. + +## Community + +### Contact us + +Mail: [email protected] \ No newline at end of file diff --git a/docs/contribute/_category_.json b/docs/contribute/_category_.json new file mode 100644 index 0000000..56d36f9 --- /dev/null +++ b/docs/contribute/_category_.json @@ -0,0 +1,5 @@ +{ + "position": 5, + "label": "Contribute", + "collapsed": false +} diff --git a/docs/design-document/06-cloudevents.md b/docs/design-document/06-cloudevents.md new file mode 100644 index 0000000..20a99d1 --- /dev/null +++ b/docs/design-document/06-cloudevents.md @@ -0,0 +1,106 @@ +# CloudEvents Integration + +## Introduction + +[CloudEvents](https://github.com/cloudevents/spec) is a specification for describing event data in common formats to provide interoperability across services, platforms and systems. + +As of May 2021, EventMesh contains the following major components: `eventmesh-runtime`, `eventmesh-sdk-java` and `eventmesh-connector-rocketmq`. +For a customer to use EventMesh, `eventmesh-runtime` can be deployed as microservices to transmit +customer's events between event producers and consumers. Customer's applications can then interact +with `eventmesh-runtime` using `eventmesh-sdk-java` to publish/subscribe for events on given topics. + +CloudEvents support has been a highly desired feature by EventMesh users. There are many reasons +for users to prefer using a SDK with CloudEvents support: + +- CloudEvents is a more widely accepted and supported way to describe events. `eventmesh-sdk-java` + currently uses the `LiteMessage` structure to describe events, which is less standardized. +- CloudEvents's Java SDK has a wider range of distribution methods. For example, EventMesh users + currently need to use the SDK tarball or build from source for every EventMesh release. With + CloudEvents support, it's easier for users to take a dependency on EventMesh's SDK using CloudEvents's public distributions (e.g. through a Maven configuration). +- CloudEvents's SDK supports multiple languages. Although EventMesh currently only supports a Java SDK, in future if more languages need to be supported, the extensions can be easier with experience on binding Java SDK with CloudEvents. + +## Requirements + +### Functional Requirements + +| Requirement ID | Requirement Description | Comments | +| -------------- | ----------------------- | -------- | +| F-1 | EventMesh users should be able to depend on a public SDK to publish/subscribe events in CloudEvents format | Functionality | +| F-2 | EventMesh users should continue to have access to existing EventMesh client features (e.g. load balancing) with an SDK that supports CloudEvent | Feature Parity | +| F-3 | EventMesh developers should be able to sync `eventmesh-sdk-java` and an SDK with CloudEvents support without much effort/pain | Maintainability | +| F-4 | EventMesh support pluggable protocols for developers integrate other protocols (e.g. CloudEvents\EventMesh Message\OpenMessage\MQTT ...) | Functionality | +| F-5 | EventMesh support the unified api for publish/subscribe events to/from event store | Functionality | + +### Performance Requirements + +| Requirement ID | Requirement Description | Comments | +| -------------- | ----------------------- | -------- | +| P-1 | Client side latency for SDK with CloudEvents support should be similar to current SDK | | + +## Design Details + +Binding with the CloudEvents Java SDK (similar to what Kafka already did, see Reference for more details) +should be an easy way to achieve the requirements. + +### Pluggable Protocols + + + +### Process of CloudEvents under EventMesh + +#### For TCP + +##### SDK side for publish + +- add the CloudEvents identifier in `package` header +- use `CloudEventBuilder` build the CloudEvent and put it into the `package` body + +##### SDK side for subscribe + +- add `convert` function under the `ReceiveMsgHook` interface, for converting the `package` body to the specific protocol with the identifier in `package` header +- different protocols should implement the `ReceiveMsgHook` interface + +##### Server side for publish + +- design the protocol convert api contains `decodeMessage` interface which convert the package's body to CloudEvent +- update `Session.upstreamMsg()` in `MessageTransferTask` change the input parameter Message to CloudEvent, the CloudEvent use the last step `decodeMessage` api convert +- update `SessionSender.send()` change the input parameter `Message` to `CloudEvent` +- update `MeshMQProducer` api support send `CloudEvents` in runtime +- support the implementation in `connector-plugin` for send `CloudEvents` to EventStore + +##### Server side for subscribe + +- support change the `RocketMessage` to `CloudEvent` in connector-plugin + +- overwrite the `AsyncMessageListener.consume()` function, change the input parameter `Message` to `CloudEvent` + +- update the `MeshMQPushConsumer.updateOffset()` implementation change the the input parameter `Message` to `CloudEvent` + +- update `DownStreamMsgContext` , change the input parameter `Message` to `CloudEvent`, update the `DownStreamMsgContext.ackMsg` + +#### For HTTP + +##### SDK side for publish + +- support `LiteProducer.publish(cloudEvent)` +- add the CloudEvents identifier in http request header + +##### SDK side for subscribe + +##### Server side for publish + +- support build the `HttpCommand.body` by pluggable protocol plugins according the protocol type in `HttpCommand` header +- support publish the CloudEvent in message processors + +##### Server side for subscribe + +- update the `EventMeshConsumer.subscribe()` + +- update `HandleMsgContext` , change the input parameter `Message` to `CloudEvent` +- update `AsyncHttpPushRequest.tryHTTPRequest()` + +## Appendix + +### References + +- <https://cloudevents.github.io/sdk-java/kafka> diff --git a/docs/design-document/07-tracing.md b/docs/design-document/07-tracing.md new file mode 100644 index 0000000..1b19196 --- /dev/null +++ b/docs/design-document/07-tracing.md @@ -0,0 +1,87 @@ +# Distributed Tracing + +## Overview of OpenTelemetry + +OpenTelemetry is a collection of tools, APIs, and SDKs. You can use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) for analysis in order to understand your software's performance and behavior. + +## Requirements + +- set tracer +- different exporter +- start and end span in server + +## Design Details + +- SpanProcessor: BatchSpanProcessor + +- Exporter: log(default), would be changed from properties + +```java +// Configure the batch spans processor. This span processor exports span in batches. +BatchSpanProcessor batchSpansProcessor = + BatchSpanProcessor.builder(exporter) + .setMaxExportBatchSize(512) // set the maximum batch size to use + .setMaxQueueSize(2048) // set the queue size. This must be >= the export batch size + .setExporterTimeout( + 30, TimeUnit.SECONDS) // set the max amount of time an export can run before getting + // interrupted + .setScheduleDelay(5, TimeUnit.SECONDS) // set time between two different exports + .build(); +OpenTelemetrySdk.builder() + .setTracerProvider( + SdkTracerProvider.builder().addSpanProcessor(batchSpansProcessor).build()) + .build(); +``` + +1. When using the method 'init()' of the class "EventMeshHTTPServer", the class "AbstractHTTPServer” will get the tracer + +```java +super.openTelemetryTraceFactory = new OpenTelemetryTraceFactory(eventMeshHttpConfiguration); +super.tracer = openTelemetryTraceFactory.getTracer(this.getClass().toString()); +super.textMapPropagator = openTelemetryTraceFactory.getTextMapPropagator(); +``` + +2. then the trace in class "AbstractHTTPServer” will work. + +## Problems + +### How to set different exporter in class 'OpenTelemetryTraceFactory'? (Solved) + +After I get the exporter type from properties, how to deal with it. + +The 'logExporter' only needs to new it. + +But the 'zipkinExporter' needs to new and use the "getZipkinExporter()" method. + +## Solutions + +### Solution of different exporter + +Use reflection to get an exporter. + +First of all, different exporter must implement the interface 'EventMeshExporter'. + +Then we get the exporter name from the configuration and reflect to the class. + +```java +//different spanExporter +String exporterName = configuration.eventMeshTraceExporterType; +//use reflection to get spanExporter +String className = String.format("org.apache.eventmesh.runtime.exporter.%sExporter",exporterName); +EventMeshExporter eventMeshExporter = (EventMeshExporter) Class.forName(className).newInstance(); +spanExporter = eventMeshExporter.getSpanExporter(configuration); +``` + +Additional, this will surround with try catch.If the specified exporter cannot be obtained successfully, the default exporter log will be used instead + +#### Improvement of different exporter + +SPI (To be completed) + +## Appendix + +### References + +<https://github.com/open-telemetry/docs-cn/blob/main/QUICKSTART.md> + +<https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/netty> diff --git a/docs/design-document/09-event-bridge.md b/docs/design-document/09-event-bridge.md new file mode 100644 index 0000000..b74b78e --- /dev/null +++ b/docs/design-document/09-event-bridge.md @@ -0,0 +1,156 @@ +# Event Bridge + + + +Event Bridge 可以支持跨mesh集群的消息投递，下面展示这一功能的详细设计与体验步骤 + + + +> 注：在本地体验这一功能时需要启动两台eventmesh实例，同时要修改`eventmesh-runtime`目录下的`eventmesh.properties`文件中的端口配置，避免端口冲突。便于下文描述，event-bridge特性按照上图信息进行表述。 + +## 01 远程订阅 + +**描述**：向cluster2 eventmesh发起远程订阅指令，cluster2 eventmesh收到指令后会携带订阅信息调用cluster1 eventmesh的本地订阅接口 + +**URL**: http://{cluster2 address}/eventmesh/subscribe/remote + +**请求方式**：POST + +**请求参数：**application/json 格式 + +| 参数名 | 类型 | 是否必填 | 说明 | +| ------------- | ------ | -------- | ------------------------------------------------------------ | +| url | String | 是 | 标识订阅url信息，暂时无用，后续可移除，目前仅为强校验，实际会被（/eventmesh/bridge/publish）替换 | +| consumerGroup | String | 是 | 标识消费组信息，实际会被cluster2的eventmesh配置信息替换 | +| topic | List | 是 | 标识订阅信息列表 | +| mode | String | 是 | 标识消费模式，分为集群模式和广播模式 | +| topic | String | 是 | 标识订阅的topic | +| type | String | 是 | 标识消费类型，分为同步和异步 | +| remoteMesh | String | 否 | 标识远程mesh地址，优先根据topic从注册中心获取，获取不到使用该字段替换 | + +**请求样例：** + +```json +{ + "url": "http://127.0.0.1:8088/sub/test";, + "consumerGroup": "TEST-GROUP", + "topic": [ + { + "mode": "CLUSTERING", + "topic": "TEST-TOPIC-HTTP-ASYNC", + "type": "ASYNC" + } + ], + "remoteMesh" : "http://127.0.0.1:10105/eventmesh/subscribe/local"; +} +``` + +## 02 本地订阅 + +**描述**：向cluster2的EventMesh实例发起本地订阅指令，cluster2的EventMesh收到订阅指令后会启动本地监听从event store收下来的消息，并推送给订阅信息中的url。 + +**URL**: http://{cluster2 address}/eventmesh/subscribe/local + +**请求方式**：POST + +**请求参数：**application/json 格式 + +| 参数名 | 类型 | 是否必填 | 说明 | +| ------------- | ------ | -------- | ------------------------------------ | +| url | String | 是 | 标识订阅url信息 | +| consumerGroup | String | 是 | 标识消费组信息 | +| topic | List | 是 | 标识订阅信息列表 | +| mode | String | 是 | 标识消费模式，分为集群模式和广播模式 | +| topic | String | 是 | 标识订阅的topic | +| type | String | 是 | 标识消费类型，分为同步和异步 | + +**请求样例：** + +```JSON +{ + "url": "http://127.0.0.1:8088/sub/test";, + "consumerGroup": "TEST-GROUP", + "topic": [ + { + "mode": "CLUSTERING", + "topic": "TEST-TOPIC-HTTP-ASYNC", + "type": "ASYNC" + } + ] +} +``` + +## 03 发送消息 + +**描述**：向cluster1的EventMesh实例发送消息，cluster1的EventMesh收到消息后会发送到event store，再从event store收下来消息推送给cluster2的EventMesh url `/eventmesh/bridge/publish`。 + +**URL**: http://{cluster1 address}/eventmesh/publish/TEST-TOPIC-HTTP-ASYNC + +**请求方式**：POST + +**请求参数：**application/json 格式 + +**请求样例：** + +```json +{ + "name":"test", + "age":"19" +} +``` + +## 04远程去订阅 + +**描述**：向cluster2的EventMesh实例发送去除订阅指令，cluster2的EventMesh收到指令后会发送cluster1的EventMesh，cluster1的EventMesh会本地执行去除订阅 + +**URL**: http://{cluster2 address}/eventmesh/unsubscribe/remote + +**请求方式**：POST + +**请求参数：**application/json 格式 + +| 参数名 | 类型 | 是否必填 | 说明 | +| ------------- | ------ | -------- | ------------------------------------------------------------ | +| url | String | 是 | 标识要去除订阅url信息，暂时无用，后续可移除，目前仅为强校验，实际会被（/eventmesh/bridge/publish）替换 | +| consumerGroup | String | 是 | 标识要去除的消费组信息，实际会使用EventMesh cluster2的group信息替换 | +| topic | List | 是 | 标识订阅topic信息列表 | + +**请求样例：** + +```json +{ + "consumerGroup": "EventMeshTest-consumerGroup", + "url": "http://127.0.0.1:8088/sub/test";, + "topic": [ + "TEST-TOPIC-HTTP-ASYNC" + ] +} +``` + +## 05本地去订阅 + +**描述**：向cluster2的EventMesh实例发送去除订阅指令，cluster2的EventMesh收到指令后会本地执行去除订阅 + +**URL**: http://{cluster2 address}/eventmesh/unsubscribe/local + +**请求方式**：POST + +**请求参数：**application/json 格式 + +| 参数名 | 类型 | 是否必填 | 说明 | +| ------------- | ------ | -------- | ---------------------- | +| url | String | 是 | 标识要去除订阅url信息 | +| consumerGroup | String | 是 | 标识要去除的消费组信息 | +| topic | List | 是 | 标识订阅topic信息列表 | + +**请求样例：** + +```json +{ + "consumerGroup": "EventMeshTest-consumerGroup", + "url": "http://127.0.0.1:8088/sub/test";, + "topic": [ + "TEST-TOPIC-HTTP-ASYNC" + ] +} +``` \ No newline at end of file diff --git a/docs/design-document/10-knative-connector.md b/docs/design-document/10-knative-connector.md new file mode 100644 index 0000000..2c98e14 --- /dev/null +++ b/docs/design-document/10-knative-connector.md @@ -0,0 +1,93 @@ +# Knative Connector + +## Prerequisite +### Create Knative Source and Sink +We use the *cloudevents-player* [Knative service](https://knative.dev/docs/serving/) as an example. If you do not know how to create *cloudevents-player* Knative service as source and sink, please follow the steps in this [link](https://knative.dev/docs/getting-started/first-source/#creating-your-first-source). + + +### Set up EventMesh Configuration +- Add the following lines to [eventmesh-starter/build.gradle](https://github.com/apache/incubator-eventmesh/blob/master/eventmesh-starter/build.gradle) file. +``` +plugins { + id 'application' +} + +application { + mainClass = project.hasProperty("mainClass") ? project.getProperty("mainClass") : 'org.apache.eventmesh.starter.StartUp' + applicationDefaultJvmArgs = [ + '-Dlog4j.configurationFile=../eventmesh-runtime/conf/log4j2.xml', '-Deventmesh.log.home=../eventmesh-runtime/logs', '-Deventmesh.home=../eventmesh-runtime', '-DconfPath=../eventmesh-runtime/conf' + ] +} + +dependencies { + implementation project(":eventmesh-connector-plugin:eventmesh-connector-knative") + implementation project(":eventmesh-runtime") +} +``` +- Add the following lines to [eventmesh-examples/build.gradle](https://github.com/apache/incubator-eventmesh/blob/master/eventmesh-examples/build.gradle) file. +``` +plugins { + id 'application' +} + +application { + mainClass = project.hasProperty("mainClass") ? project.getProperty("mainClass") : 'NULL' +} +``` +- Set ```eventMesh.connector.plugin.type=knative``` in [eventmesh-runtime/conf/eventmesh.properties](https://github.com/pchengma/incubator-eventmesh/blob/master/eventmesh-runtime/conf/eventmesh.properties) file. + +## Demo +### Publish an Event Message from Knative and Subscribe from EventMesh +#### Step 1: Start an Eventmesh-Runtime Server +```bash +$ cd eventmesh-starter +$ ../gradlew -PmainClass=org.apache.eventmesh.starter.StartUp run +``` + +#### Step 2: Publish an Event Message from Knative +```bash +$ curl -i http://cloudevents-player.default.127.0.0.1.sslip.io -H "Content-Type: application/json" -H "Ce-Id: 123456789" -H "Ce-Specversion: 1.0" -H "Ce-Type: some-type" -H "Ce-Source: command-line" -d '{"msg":"Hello CloudEvents!"}' +``` + +#### Step 3: Subscribe from an EventMesh +- Set ```public static final String EVENTMESH_HTTP_ASYNC_TEST_TOPIC = "messages";``` in [ExampleConstants.java](https://github.com/apache/incubator-eventmesh/blob/master/eventmesh-examples/src/main/java/org/apache/eventmesh/common/ExampleConstants.java) file. +```bash +$ cd eventmesh-examples +$ ../gradlew -PmainClass=org.apache.eventmesh.http.demo.sub.SpringBootDemoApplication run +``` + +#### Expected Result +The following message with ```data``` field as ```Hello CloudEvents!``` will be printed on the console of EventMesh server. +```bash +2022-09-05 16:37:58,237 INFO [eventMesh-clientManage-] DefaultConsumer(DefaultConsumer.java:60) - \ +[{"event":{"attributes":{"datacontenttype":"application/json","id":"123456789","mediaType":"application/json",\ +"source":"command-line","specversion":"1.0","type":"some-type"},"data":{"msg":"Hello CloudEvents!"},"extensions":{}},\ +"id":"123456789","receivedAt":"2022-09-05T10:37:49.537658+02:00[Europe/Madrid]","type":"RECEIVED"}] +``` + +### Publish an Event Message from EventMesh and Subscribe from Knative +#### Step 1: Start an Eventmesh-Runtime Server +```bash +$ cd eventmesh-starter +$ ../gradlew -PmainClass=org.apache.eventmesh.starter.StartUp run +``` + +#### Step 2: Publish an Event Message from EventMesh +We use a test program to demo this function. +```bash +$ cd eventmesh-connector-plugin/eventmesh-connector-knative +$ ../../gradlew clean test --tests KnativeProducerImplTest.testPublish +``` + +#### Step 3: Subscribe from Knative +```bash +$ curl http://cloudevents-player.default.127.0.0.1.sslip.io/messages +``` + +#### Expected Result +The following message with ```data``` field as ```Hello Knative from EventMesh!``` will be printed on the console of EventMesh server. +```bash +2022-09-05 16:52:41,633 INFO [eventMesh-clientManage-] DefaultConsumer(DefaultConsumer.java:60) - \ +[{"event":{"attributes":{"datacontenttype":"application/json","id":"1234","mediaType":"application/json",\ +"source":"java-client","specversion":"1.0","type":"some-type"},"data":{"msg":["Hello Knative from EventMesh!"]},"extensions":{}},"id":"1234","receivedAt":"2022-09-05T10:52:32.999273+02:00[Europe/Madrid]","type":"RECEIVED"}] +``` \ No newline at end of file diff --git a/docs/design-document/11-prometheus.md b/docs/design-document/11-prometheus.md new file mode 100644 index 0000000..1bc6e0e --- /dev/null +++ b/docs/design-document/11-prometheus.md @@ -0,0 +1,24 @@ +# Observe Metrics with Prometheus + +## Prometheus + +[Prometheus](https://prometheus.io/docs/introduction/overview/) is an open-source system monitoring and alerting toolkit that collects and stores the metrics as time-series data. EventMesh exposes a collection of metrics data that could be scraped and analyzed by Prometheus. Please follow [the "First steps with Prometheus" tutorial](https://prometheus.io/docs/introduction/first_steps/) to download and install the latest release of Prometheus. + +## Edit Prometheus Configuration + +The `eventmesh-runtime/conf/prometheus.yml` configuration file specifies the port of the metrics HTTP endpoint. The default metrics port is `19090`. + +```properties +eventMesh.metrics.prometheus.port=19090 +``` + +Please refer to [the Prometheus configuration guide](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) to add the EventMesh metrics as a scrape target in the configuration file. Here's the minimum configuration that creates a job with the name `eventmesh` and the endpoint `http://localhost:19090`: + +```yaml +scrape_configs: + - job_name: "eventmesh" + static_configs: + - targets: ["localhost:19090"] +``` + +Please navigate to the Prometheus dashboard (e.g. `http://localhost:9090`) to view the list of metrics exported by EventMesh, which are prefixed with `eventmesh_`. diff --git a/docs/design-document/12-zipkin.md b/docs/design-document/12-zipkin.md new file mode 100644 index 0000000..f4d0725 --- /dev/null +++ b/docs/design-document/12-zipkin.md @@ -0,0 +1,38 @@ +# Collect Trace with Zipkin + +## Zipkin + +Distributed tracing is a method used to profile and monitor applications built with microservices architecture. Distributed tracing helps pinpoint where failures occur and what causes poor performance. + +[Zipkin](https://zipkin.io) is a distributed tracing system that helps collect timing data needed to troubleshoot latency problems in service architectures. EventMesh exposes a collection of trace data that could be collected and analyzed by Zipkin. Please follow [the "Zipkin Quickstart" tutorial](https://zipkin.io/pages/quickstart.html) to download and install the latest release of Zipkin. + +## Configuration + +To enable the trace exporter of EventMesh Runtime, set the `eventMesh.server.trace.enabled` field in the `conf/eventmesh.properties` file to `true`. + +```conf +# Trace plugin +eventMesh.server.trace.enabled=true +eventMesh.trace.plugin=zipkin +``` + +To customize the behavior of the trace exporter such as timeout or export interval, edit the `exporter.properties` file. + +```conf +# Set the maximum batch size to use +eventmesh.trace.max.export.size=512 +# Set the queue size. This must be >= the export batch size +eventmesh.trace.max.queue.size=2048 +# Set the max amount of time an export can run before getting(TimeUnit=SECONDS) +eventmesh.trace.export.timeout=30 +# Set time between two different exports (TimeUnit=SECONDS) +eventmesh.trace.export.interval=5 +``` + +To send the exported trace data to Zipkin, edit the `eventmesh.trace.zipkin.ip` and `eventmesh.trace.zipkin.port` fields in the `conf/zipkin.properties` file to match the configuration of the Zipkin server. + +```conf +# Zipkin's IP and Port +eventmesh.trace.zipkin.ip=localhost +eventmesh.trace.zipkin.port=9411 +``` diff --git a/docs/design-document/13-jaeger.md b/docs/design-document/13-jaeger.md new file mode 100644 index 0000000..e9452c3 --- /dev/null +++ b/docs/design-document/13-jaeger.md @@ -0,0 +1,44 @@ +# Collect Trace with Jaeger + +## Jaeger + +[Jaeger](https://www.jaegertracing.io/), inspired by [Dapper](https://research.google.com/pubs/pub36356.html) and [OpenZipkin](https://zipkin.io/), is a distributed tracing platform created by [Uber Technologies](https://uber.github.io/) and donated to [Cloud Native Computing Foundation](https://cncf.io/). It can be used for monitoring microservices-based distributed systems. + +For the installation of Jaeger, you can refer to the [official documentation](https://www.jaegertracing.io/docs/latest/getting-started/) of Jaeger. It is recommended to use docker image `jaegertracing/all-in-one` to quickly build the environment for testing. + +## Configuration + +To enable the trace exporter of EventMesh Runtime, set the `eventMesh.server.trace.enabled` field in the `conf/eventmesh.properties` file to `true`. + +```conf +# Trace plugin +eventMesh.server.trace.enabled=true +eventMesh.trace.plugin=jaeger +``` + +To customize the behavior of the trace exporter such as timeout or export interval, edit the `exporter.properties` file. + +```conf +# Set the maximum batch size to use +eventmesh.trace.max.export.size=512 +# Set the queue size. This must be >= the export batch size +eventmesh.trace.max.queue.size=2048 +# Set the max amount of time an export can run before getting(TimeUnit=SECONDS) +eventmesh.trace.export.timeout=30 +# Set time between two different exports (TimeUnit=SECONDS) +eventmesh.trace.export.interval=5 +``` + +To send the exported trace data to Jaeger, edit the `eventmesh.trace.jaeger.ip` and `eventmesh.trace.jaeger.port` fields in the `conf/jaeger.properties` file to match the configuration of the Jaeger server. + +```conf +# Jaeger's IP and Port +eventmesh.trace.jaeger.ip=localhost +eventmesh.trace.jaeger.port=14250 +``` + +## Migrating from Zipkin + +Collector service exposes Zipkin compatible REST API `/api/v1/spans` which accepts both Thrift and JSON. Also there is `/api/v2/spans` for JSON and Proto. + +So you can also use the `eventmesh-trace-zipkin` plugin to collect trace with Jaeger. Please refer to the `eventmesh-trace-zipkin` documentation for the specific configuration. By default this feature in Jaeger is disabled. It can be enabled with `--collector.zipkin.host-port=:9411`. \ No newline at end of file diff --git a/docs/design-document/webhook.md b/docs/design-document/webhook.md new file mode 100644 index 0000000..13ec4de --- /dev/null +++ b/docs/design-document/webhook.md @@ -0,0 +1,270 @@ + +## Webhook usage process +#### The first step: Configure webhook related information in eventmesh and start + +##### Configuration +``` +# Whether to start the webhook admin service +eventMesh.webHook.admin.start=true + +# webhook event configuration storage mode. But currently only supports file and nacos +eventMesh.webHook.operationMode=file + +# The file path of fileMode. If you write #{eventMeshHome}, in the eventMesh root directory +eventMesh.webHook.fileMode.filePath= #{eventMeshHome}/webhook + +# The nacos storage mode. The configuration naming rule is eventMesh.webHook.nacosMode.{nacos native configuration key} For the specific configuration, please see [nacos github api](https://github.com/alibaba/nacos/blob/develop/api/src/main/java /com/alibaba/nacos/api/SystemPropertyKeyConst.java) +## address of nacos +eventMesh.webHook.nacosMode.serverAddr=127.0.0.1:8848 + +# webhook eventcloud send mode. Same as eventMesh.connector.plugin.type configuration +eventMesh.webHook.producer.connector=standalone +``` + +#### The second step: Add webhook configuration information + +Configuration information description + +```java + /** + * The path called by the manufacturer. Manufacturer event call address, [http or https]://[domain or IP]:[port]/webhook/[callbackPath] + * for example: http://127.0.0.1:10504/webhook/test/event , The full url needs to be filled in the manufacturer call input + * callbackPath is the only + */ + private String callbackPath; + + /** + * manufacturer name, like github + */ + private String manufacturerName; + + /** + * webhook event name, like rep-push + */ + private String manufacturerEventName; + + /** + * http header content type + */ + private String contentType = "application/json"; + + /** + * description of this WebHookConfig + */ + private String description; + + /** + * secret key, for authentication + */ + private String secret; + + /** + * userName, for HTTP authentication + */ + private String userName; + + /** + * password, for HTTP authentication + */ + private String password; + + + /** + * roll out event name, like topic to mq + */ + private String cloudEventName; + + /** + * roll out data format -> CloudEvent serialization mode + * If HTTP protocol is used, the request header contentType needs to be marked + */ + private String dataContentType = "application/json";; + + /** + * source of event + */ + private String cloudEventSource; + + /** + * id of cloudEvent, like uuid/manufacturerEventId + */ + private String cloudEventIdGenerateMode; + +``` + +##### Add webhook config + +path: /webhook/insertWebHookConfig +method: POST +contentType: application/json + +input params: +| field | desc | type | necessary | default | +| -- | -- | -- | -- | -- | +| callbackPath | call address, unique address | string | Y | null | +| manufacturerName | manufacturer name | string | Y | null | +| manufacturerEventName | manufacturer EventName | string | Y | null | +| contentType | http connettype | string | N | application/json | +| description | configuration instructions | string | N | null | +| secret | signature string | string | N | null | +| userName | username | string | N | null | +| password | password | string | N | null | +| cloudEventName | cloudEvent name | string | Y | null | +| cloudEventSource | cloudEvent source | string | Y | null | +| cloudEventIdGenerateMode | cloudEvent event object identification method, uuid or event id | string | N manufacturerEventId | + +E.g: + +```json + +{ + "callbackPath":"/webhook/github/eventmesh/all", + "manufacturerName":"github", + "manufacturerEventName":"all", + "secret":"eventmesh", + "cloudEventName":"github-eventmesh", + "cloudEventSource":"github" +} + +``` + +Output params: 1 for success, 0 for failure + +##### delete webhook config +path: /webhook/deleteWebHookConfig +method: POST +contentType： application/json + +input params: +| field | desc | type | necessary | default | +| -- | -- | -- | -- | -- | +| callbackPath | call address, unique address | string | Y | null | + + +E.g: + +```json + +{ + "callbackPath":"/webhook/github/eventmesh/all" +} + +``` + +Output params: 1 for success, 0 for failure + +##### select WebHookConfig by callbackPath +path: /webhook/queryWebHookConfigById +method: POST +contentType： application/json + +input params: +| field | desc | type | necessary | default | +| -- | -- | -- | -- | -- | +| callbackPath | call address, unique address | string | Y | null | + + +E.g: + +```json + +{ + "callbackPath":"/webhook/github/eventmesh/all" +} + +``` + +Output params: +| field | desc | type | necessary | default | +| -- | -- | -- | -- | -- | +| callbackPath | call address, unique address | string | Y | null | +| manufacturerName | manufacturer name | string | Y | null | +| manufacturerEventName | manufacturer event name | string | Y | null | +| contentType | http connettype | string | N | application/json | +| description | configuration instructions | string | N | null | +| secret | signature key | string | N | null | +| userName | user name | string | N | null | +| password | password | string | N | null | +| cloudEventName | cloudEvent name | string | Y | null | +| cloudEventSource | cloudEvent source | string | Y | null | +| cloudEventIdGenerateMode | cloudEvent event object identification method, uuid or event id | string | N | manufacturerEventId | + + +##### 通过manufacturer查询WebHookConfig列表 +path: /webhook/queryWebHookConfigByManufacturer +method: POST +contentType： application/json + +input params: +| field | desc | type | necessary | default | +| -- | -- | -- | -- | -- | +| manufacturerName | manufacturer name | string | Y | null | + + +E.g: + +```json + +{ + "manufacturerName":"github" +} + +``` + +Output params: +| field | desc | type | necessary | default | +| -- | -- | -- | -- | -- | +| callbackPath | call address, unique address | string | Y | null | +| manufacturerName | manufacturer name | string | Y | null | +| manufacturerEventName | manufacturer event name | string | Y | null | +| contentType | http connettype | string | N | application/json | +| description | configuration instructions | string | N | null | +| secret | signature key | string | N | null | +| userName | user name | string | N | null | +| password | password | string | N | null | +| cloudEventName | cloudEvent name | string | Y | null | +| cloudEventSource | cloudEvent source | string | Y | null | +| cloudEventIdGenerateMode | cloudEvent event object identification method, uuid or event id | string | N | manufacturerEventId | + + +#### The third step: Check if the configuration is successful + +1. file storage mode. Please go to the eventMesh.webHook.fileMode.filePath directory to view. filename callbackPath. + +2. nacos storage mode. Please go to the nacos service configured by eventMesh.webHook.nacosMode.serverAddr to see. + +#### The fourth step: Configure the consumer of cloudevent + +#### The fifth step: Configure webhook related information in the manufacturer + +> For manufacturer's operation, please refer to [Manufacturer's webhook operation instructions] . + +## Manufacturer's webhook operation instructions + +### github sign up + +#### The first step: Enter the corresponding project + +#### The second step: click setting + + + +#### The third step: click Webhooks + + + +#### The fourth step: Click on Add webhook + + + +#### The fifth step: Fill in the webhook information + + + +Payload URL: Service address and pahts. [http or https]://[domain or IP]:[port]/webhook/[callbackPath] +Content type: http header content type +secret: signature string + + + + diff --git a/docs/instruction/01-store-with-docker.md b/docs/instruction/01-store-with-docker.md new file mode 100644 index 0000000..8c102b2 --- /dev/null +++ b/docs/instruction/01-store-with-docker.md @@ -0,0 +1,71 @@ +# Guidelines of eventmesh-store with Docker + +## Dependencies + +``` +64-bit OS，we recommend Linux/Unix； +64-bit JDK 1.8+; +Gradle 7.0+, we recommend 7.0.* +4g+ available disk to deploy eventmesh-store +If you choose standalone mode, you could skip this file and go to the next step: Start Eventmesh-Runtime; if not, you could choose RocketMQ as the store layer. +``` + + +## Download + +Download the Binary code (recommended: 4.9.*) from [RocketMQ Official](https://rocketmq.apache.org/dowloading/releases/). Here we take 4.9.2 as an example. + +``` +unzip rocketmq-all-4.9.2-bin-release.zip +cd rocketmq-4.9.2/ +``` + + +## Deploy + +- #### Start Name Server + +``` +nohup sh bin/mqnamesrv & +tail -f ~/logs/rocketmqlogs/namesrv.log +``` + +- #### Start Broker + +``` +nohup sh bin/mqbroker -n localhost:9876 & +tail -f ~/logs/rocketmqlogs/broker.log +``` + +The deployment of eventmesh-store has finished, please go to the next step: [Start Eventmesh-Runtime](02-runtime.md) + + + +## Deploy +Pull RocketMQ image from Docker Hub： + +```shell +#获取namesrv镜像 +sudo docker pull rocketmqinc/rocketmq-namesrv:4.5.0-alpine +#获取broker镜像 +sudo docker pull rocketmqinc/rocketmq-broker:4.5.0-alpine +``` + +Start namesrv and broker + +```shell +#运行namerv容器 +sudo docker run -d -p 9876:9876 -v `pwd`/data/namesrv/logs:/root/logs -v `pwd`/data/namesrv/store:/root/store --name rmqnamesrv rocketmqinc/rocketmq-namesrv:4.5.0-alpine sh mqnamesrv + +#运行broker容器 +sudo docker run -d -p 10911:10911 -p 10909:10909 -v `pwd`/data/broker/logs:/root/logs -v `pwd`/data/broker/store:/root/store --name rmqbroker --link rmqnamesrv:namesrv -e "NAMESRV_ADDR=namesrv:9876" rocketmqinc/rocketmq-broker:4.5.0-alpine sh mqbroker -c ../conf/broker.conf +``` + +Please note that the **rocketmq-broker ip** is **pod ip**. If you want to modify this ip, you can set it your custom value in **broker.conf**。 + + +By now, the deployment of eventmesh-store has finished, please go to the next step: [Start Eventmesh-Runtime Using Docker](02-runtime-with-docker.md) + + +## Reference +For more details about RocketMQ，please refer to <https://rocketmq.apache.org/docs/quick-start/> diff --git a/docs/instruction/01-store.md b/docs/instruction/01-store.md new file mode 100644 index 0000000..ab281c7 --- /dev/null +++ b/docs/instruction/01-store.md @@ -0,0 +1,44 @@ +# Guidelines of eventmesh-store + +## Dependencies + +``` +64-bit OS，we recommend Linux/Unix； +64-bit JDK 1.8+; +Gradle 7.0+, we recommend 7.0.* +4g+ available disk to deploy eventmesh-store +If you choose standalone mode, you could skip this file and go to the next step: Start Eventmesh-Runtime; if not, you could choose RocketMQ as the store layer. +``` + + +## Download + +Download the Binary code (recommended: 4.9.*) from [RocketMQ Official](https://rocketmq.apache.org/dowloading/releases/). Here we take 4.9.2 as an example. + +``` +unzip rocketmq-all-4.9.2-bin-release.zip +cd rocketmq-4.9.2/ +``` + + +## Deploy + +- #### Start Name Server + +``` +nohup sh bin/mqnamesrv & +tail -f ~/logs/rocketmqlogs/namesrv.log +``` + +- #### Start Broker + +``` +nohup sh bin/mqbroker -n localhost:9876 & +tail -f ~/logs/rocketmqlogs/broker.log +``` + +The deployment of eventmesh-store has finished, please go to the next step: [Start Eventmesh-Runtime](02-runtime.md) + + +## Reference +For more details about RocketMQ, please refer to <https://rocketmq.apache.org/docs/quick-start/> diff --git a/docs/instruction/02-runtime-with-docker.md b/docs/instruction/02-runtime-with-docker.md new file mode 100644 index 0000000..7da3e2a --- /dev/null +++ b/docs/instruction/02-runtime-with-docker.md @@ -0,0 +1,107 @@ +# EventMesh Runtime (Docker) + +The documentation introduces the steps to install the latest release of EventMesh Runtime with Docker and connect to Apache RocketMQ. It's recommended to use a Linux-based system with [Docker Engine](https://docs.docker.com/engine/install/). Please follow the [Docker tutorial](https://docs.docker.com/get-started/) to get familiar with the basic concepts (registry, volume, etc.) and commands of Docker. + + +## Dependencies +``` +64-bit OS，we recommend Linux/Unix； +64-bit JDK 1.8+; +Gradle 7.0+, we recommend 7.0.* +4g+ available disk to deploy eventmesh-store +If you choose standalone mode, you could skip this file and go to the next step: Start Eventmesh-Runtime; if not, you could choose RocketMQ as the store layer. +``` + +## Pull EventMesh Image + +Download the pre-built image of [`eventmesh`](https://hub.docker.com/r/eventmesh/eventmesh) from Docker Hub with `docker pull`: + +```console +$ sudo docker pull eventmesh/eventmesh:v1.4.0 +``` + +To verify that the `eventmesh/eventmesh` image is successfully installed, list the downloaded images with `docker images`: + +```console +$ sudo docker images +eventmesh/eventmesh v1.4.0 6e2964599c78 2 weeks ago 937MB +``` + +## Edit Configuration + +Edit the `eventmesh.properties` to change the configuration (e.g. TCP port, client blacklist) of EventMesh Runtime. To integrate RocketMQ as a connector, these two configuration files should be created: `eventmesh.properties` and `rocketmq-client.properties`. + +```shell +sudo mkdir -p /data/eventmesh/rocketmq/conf +cd /data/eventmesh/rocketmq/conf +sudo touch eventmesh.properties +sudo touch rocketmq-client.properties +``` + +### `eventmesh.properties` + +The `eventmesh.properties` file contains the properties of EventMesh runtime environment and integrated plugins. Please refer to the [default configuration file](https://github.com/apache/incubator-eventmesh/blob/master/eventmesh-runtime/conf/eventmesh.properties) for the available configuration keys. + +```shell +sudo vim eventmesh.properties +``` + +| Configuration Key | Default Value | Description | +|-|-|-| +| `eventMesh.server.http.port` | 10105 | EventMesh HTTP server port | +| `eventMesh.server.tcp.port` | 10000 | EventMesh TCP server port | +| `eventMesh.server.grpc.port` | 10205 | EventMesh gRPC server port | + +### `rocketmq-client.properties` + +The `rocketmq-client.properties` file contains the properties of the Apache RocketMQ nameserver. + +```shell +sudo vim rocketmq-client.properties +``` + +Please refer to the [default configuration file](https://github.com/apache/incubator-eventmesh/blob/1.3.0/eventmesh-runtime/conf/rocketmq-client.properties) and change the value of `eventMesh.server.rocketmq.namesrvAddr` to the nameserver address of RocketMQ. + +| Configuration Key | Default Value | Description | +|-|-|-| +| `eventMesh.server.rocketmq.namesrvAddr` | `127.0.0.1:9876;127.0.0.1:9876` | The address of RocketMQ nameserver | + +## Run and Manage EventMesh Container + +Run an EventMesh container from the `eventmesh/eventmesh` image with the `docker run` command. The `-p` option of the command binds the container port with the host machine port. The `-v` option of the command mounts the configuration files from files in the host machine. + +```shell +sudo docker run -d -p 10000:10000 -p 10105:10105 \ +-v /data/eventmesh/rocketmq/conf/eventmesh.properties:/data/app/eventmesh/conf/eventmesh.properties \ +-v /data/eventmesh/rocketmq/conf/rocketmq-client.properties:/data/app/eventmesh/conf/rocketmq-client.properties \ +eventmesh/eventmesh:v1.4.0 +``` + +The `docker ps` command lists the details (id, name, status, etc.) of the running containers. The container id is the unique identifier of the container. + +```console +$ sudo docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +<container_id> eventmesh/eventmesh:v1.4.0 "/bin/sh -c 'sh star…" About a minute ago Up About a minute 0.0.0.0:10000->10000/tcp, :::10000->10000/tcp, 0.0.0.0:10105->10105/tcp, :::10105->10105/tcp <container_name> +``` + +To connect to the EventMesh container: + +```shell +sudo docker exec -it [container id or name] /bin/bash +``` + +To read the log of the EventMesh container: + +```shell +tail -f ../logs/eventmesh.out +``` + +To stop or remove the container: + +```shell +sudo docker stop [container id or name] + +sudo docker rm -f [container id or name] +``` + diff --git a/docs/instruction/02-runtime.md b/docs/instruction/02-runtime.md new file mode 100644 index 0000000..66bfca7 --- /dev/null +++ b/docs/instruction/02-runtime.md @@ -0,0 +1,107 @@ +# EventMesh Runtime + +EventMesh Runtime is the core component of Apache EventMesh (Incubating). It is the middleware that transmits events between producers and consumers. The documentation introduces the step to install and start the latest release of EventMesh Runtime in the local or test environment. The EventMesh Runtime requires a Linux-based system with JDK (Java Development Kit) 8+. + +Here, we take JDK 8 as an example. JDK 8 could be installed with the system package manager or the [openjdk:8-jdk](https://hub.docker.com/_/openjdk) Docker image. + + +## 1 Run on your local machine + +### 1.1 Dependencies + +``` +64-bit OS，we recommend Linux/Unix； +64-bit JDK 1.8+; +Gradle 7.0+, we recommend 7.0.* +4g+ available disk to deploy eventmesh-store +If you choose standalone mode, you could skip this file and go to the next step: Start Eventmesh-Runtime; if not, you could choose RocketMQ as the store layer. +``` + +### 1.2 Download Source Code + +Gradle is the build automation tool used by Apache EventMesh (Incubating). Please refer to the [offical guide](https://docs.gradle.org/current/userguide/installation.html) to install the latest release of Gradle. + +Download and extract the source code of the latest release from [EventMesh download](https://eventmesh.apache.org/download). + +```console +wget https://dlcdn.apache.org/incubator/eventmesh/{version}-incubating/apache-eventmesh-{version}-incubating-source.tar.gz + +tar -xvzf apache-eventmesh-1.5.0-incubating-source.tar.gz +``` + +Build the source code with Gradle. + +```console +cd apache-eventmesh-1.5.0-incubating-source +gradle clean dist +``` + +Edit the `eventmesh.properties` to change the configuration (e.g. TCP port, client blacklist) of EventMesh Runtime. + +```console +cd dist +vim conf/eventmesh.properties +``` + +Execute the `start.sh` script to start the EventMesh Runtime server. + +```console +bash bin/start.sh +``` + +### 1.3 Build and Load Plugins + +Apache EventMesh (Incubating) introduces the SPI (Service Provider Interface) mechanism, which enables EventMesh to discover and load the plugins at runtime. The plugins could be installed with these methods: + +- Gradle Dependencies: Declare the plugins as the build dependencies in `eventmesh-starter/build.gradle`. + +```gradle +dependencies { + implementation project(":eventmesh-runtime") + + // Example: Load the RocketMQ plugin + implementation project(":eventmesh-connector-plugin:eventmesh-connector-rocketmq") +} +``` + +- Plugin directory: EventMesh loads the plugins in the `dist/plugin` directory based on `eventmesh.properties`. The `installPlugin` task of Gradle builds and moves the plugins into the `dist/plugin` directory. + +```console +gradle installPlugin +``` + + +## 2 Remote deployment +### 2.1 Dependencies + +``` +64-bit OS，we recommend Linux/Unix； +64-bit JDK 1.8+; +Gradle 7.0+, we recommend 7.0.* +4g+ available disk to deploy eventmesh-store +If you choose standalone mode, you could skip this file and go to the next step: Start Eventmesh-Runtime; if not, you could choose RocketMQ as the store layer. +``` + +### 2.2 Download +Download and extract the executable binaries of the latest release from [EventMesh download](https://eventmesh.apache.org/download). + +```console +wget https://dlcdn.apache.org/incubator/eventmesh/1.5.0-incubating/apache-eventmesh-1.5.0-incubating-bin.tar.gz + +tar -xvzf apache-eventmesh-1.5.0-incubating-bin.tar.gz +``` + +### 2.3 Deploy +Edit the `eventmesh.properties` to change the configuration (e.g. TCP port, client blacklist) of EventMesh Runtime. The executable binaries contain all plugins in the bundle, thus there's no need to build them from source code. + +```console +cd apache-eventmesh-1.5.0-incubating +vim conf/eventmesh.properties +``` + +Execute the `start.sh` script to start the EventMesh Runtime server. + +```console +bash bin/start.sh +``` + diff --git a/docs/instruction/03-demo.md b/docs/instruction/03-demo.md new file mode 100644 index 0000000..0b99d47 --- /dev/null +++ b/docs/instruction/03-demo.md @@ -0,0 +1,152 @@ +# Run our demos + +[](https://maven-badges.herokuapp.com/maven-central/org.apache.eventmesh/eventmesh-sdk-java) + +> EventMesh-sdk-java as the client，and comminucate with eventmesh-runtime，to finish the message sub and pub +> +> EventMesh-sdk-java support both async and broadcast. +> +> EventMesh-sdk-java support HTTP, TCP and gRPC. + +The test demos of TCP, HTTP 和 GRPC are in the module **eventmesh-examples** + +## 1 TCP DEMO + +### 1.1 ASYNC + +- Start consumer to subscribe the topic (we have created the TEST-TOPIC-TCP-ASYNC by default, you can also create other topic to test) + +``` +Run the main method of org.apache.eventmesh.tcp.demo.sub.eventmeshmessage.AsyncSubscribe +``` + +- Start producer to publish async message + +``` +Run the main method of org.apache.eventmesh.tcp.demo.pub.eventmeshmessage.AsyncPublish +``` + +### 1.2 BROADCAST + +- Start subscriber to subscribe the topic (we have created the TEST-TOPIC-TCP-BROADCAST by default, you can also create other topic to test) + +``` +Run the main method of org.apache.eventmesh.tcp.demo.sub.eventmeshmessage.AsyncSubscribeBroadcast +``` + +- Start publisher to publish async message + +``` +Run the main method of org.apache.eventmesh.tcp.demo.pub.eventmeshmessage.AsyncPublishBroadcast +``` + +More information about EventMesh-TCP, please refer to [EventMesh TCP](/docs/en/sdk-java/03-tcp.md) + + +## 2 HTTP DEMO + + +### 2.1 ASYNC + +- The subscriber is a SpringBoot demo, so run this demo to start subscriber (we have created the topic TEST-TOPIC-HTTP-ASYNC by default, you can also create other topic to test) + +``` +Run the main method of org.apache.eventmesh.http.demo.sub.SpringBootDemoApplication +``` + +- Start publisher to publish message + +``` +Run the main method of org.apache.eventmesh.http.demo.pub.eventmeshmessage.AsyncPublishInstance +``` +More information about EventMesh-HTTP, please refer to [EventMesh HTTP](/docs/en/sdk-java/02-http.md) + +## 3 GRPC DEMO + +### 3.1 ASYNC PUBLISH & WEBHOOK SUBSCRIBE </h4> + +- Start publisher to publish message (we have created the topic TEST-TOPIC-GRPC-ASYNC by default, you can also create other topic to test) + +``` +Run the main method of org.apache.eventmesh.grpc.pub.eventmeshmessage.AsyncPublishInstance +``` + +- Start webhook subscriber + +``` +Run the main method of org.apache.eventmesh.grpc.sub.app.SpringBootDemoApplication +``` + +### 3.2 SYNC PUBLISH & STREAM SUBSCRIBE + +- Start Request-Reply publisher to publish message (we have created the topic TEST-TOPIC-GRPC-RR by default, you can also create other topic to test) + +``` +Run the main method of org.apache.eventmesh.grpc.pub.eventmeshmessage.RequestReplyInstance +``` + +- Start stream subscriber + +``` +Run the main method of org.apache.eventmesh.grpc.sub.EventmeshAsyncSubscribe +``` + +### 3.3 PUBLISH BATCH MESSAGE + +- Start publisher to publish batch message (we have created the TEST-TOPIC-GRPC-ASYNC by default, you can also create other topic to test) + +``` +Run the main method of org.apache.eventmesh.grpc.pub.eventmeshmessage.BatchPublishInstance +``` + +More information about EventMesh-gRPC, please refer to [EventMesh gRPC](/docs/en/sdk-java/04-grpc.md) + +## 4 Run these demos by yourself + +Please refer to [EventMesh Store](/docs/en/instruction/01-store.md) and [EventMesh Runtime](/docs/en/instruction/02-runtime.md) to finish the necessary deployment before try our demo + +After finishing the deployment of store and runtime, you can run our demos in module `eventmesh-examples`: + +### TCP Sub + + ```shell + cd bin + sh tcp_eventmeshmessage_sub.sh + ``` + +### TCP Pub + + ```shell + cd bin + sh tcp_pub_eventmeshmessage.sh + ``` + +### TCP Sub Broadcast + + ```shell + cd bin + sh tcp_sub_eventmeshmessage_broadcast.sh + ``` + +### TCP Pub Broadcast + + ```shell + cd bin + sh tcp_pub_eventmeshmessage_broadcast.sh + ``` + +### HTTP Sub + + ```shell + cd bin + sh http_sub.sh + ``` + +### HTTP Pub + + ```shell + cd bin + sh http_pub_eventmeshmessage.sh + ``` + +You can review the log in the folder `/logs` diff --git a/docs/instruction/_category_.json b/docs/instruction/_category_.json new file mode 100644 index 0000000..690c6eb --- /dev/null +++ b/docs/instruction/_category_.json @@ -0,0 +1,5 @@ +{ + "position": 2, + "label": "Installation and Deployment", + "collapsed": false +} diff --git a/docs/sdk-java/intro.md b/docs/sdk-java/intro.md new file mode 100644 index 0000000..42b9c41 --- /dev/null +++ b/docs/sdk-java/intro.md @@ -0,0 +1,29 @@ +# Installation + +[](https://maven-badges.herokuapp.com/maven-central/org.apache.eventmesh/eventmesh-sdk-java) + +EventMesh SDK for Java is a collection of Java libraries to integrate EventMesh in a Java application. The SDK supports sending and receiving synchronous messages, asynchronous messages, and broadcast messages in TCP, HTTP, and gRPC protocols. The SDK implements EventMesh Message, CloudEvents, and OpenMessaging formats. The demo project is available in the [`eventmesh-example`](https://github.com/apache/incubator-eventmesh/tree/master/eventmesh-examples) module. + +## Gradle + +To install EventMesh SDK for Java with Gradle, declare `org.apache.eventmesh:eventmesh-sdk-java` as `implementation` in the dependencies block of the module's `build.gradle` file. + +```groovy +dependencies { + implementation 'org.apache.eventmesh:eventmesh-sdk-java:1.4.0' +} +``` + +## Maven + +To install EventMesh SDK for Java with Maven, declare `org.apache.eventmesh:eventmesh-sdk-java` as a dependency in the dependencies block of the project's `pom.xml` file. + +```xml +<dependencies> + <dependency> + <groupId>org.apache.eventmesh</groupId> + <artifactId>eventmesh-sdk-java</artifactId> + <version>1.4.0</version> + </dependency> +</dependencies> +``` --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://landscape.cncf.io/images/left-logo.svg"](https://landscape.cncf.io/images/left-logo.svg"){kind=link}
![https://landscape.cncf.io/images/right-logo.svg"](https://landscape.cncf.io/images/right-logo.svg"){kind=link}
