This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7af7e52 Automated deployment: Thu Jul 8 14:26:03 UTC 2021
b5791c53df5403e2ea5c20f174d0143a781c501d
7af7e52 is described below
commit 7af7e524f37c25ddbb07e6ab8329ad8a3dfba9fb
Author: dockerzhang <[email protected]>
AuthorDate: Thu Jul 8 14:26:04 2021 +0000
Automated deployment: Thu Jul 8 14:26:03 UTC 2021
b5791c53df5403e2ea5c20f174d0143a781c501d
---
docs/en-us/modules/agent/quick_start.md | 13 +++++++--
docs/en-us/modules/dataproxy/quick_start.md | 4 +--
.../modules/tubemq/tubemq-manager/quick_start.md | 8 +++---
docs/zh-cn/modules/agent/quick_start.md | 31 +++++++++++++++-------
docs/zh-cn/modules/dataproxy/quick_start.md | 5 ++--
.../modules/tubemq/tubemq-manager/quick_start.md | 7 ++---
en-us/docs/modules/agent/quick_start.html | 9 ++++++-
en-us/docs/modules/agent/quick_start.json | 2 +-
en-us/docs/modules/agent/quick_start.md | 13 +++++++--
en-us/docs/modules/dataproxy/quick_start.html | 4 +--
en-us/docs/modules/dataproxy/quick_start.json | 2 +-
en-us/docs/modules/dataproxy/quick_start.md | 4 +--
.../modules/tubemq/tubemq-manager/quick_start.html | 7 ++---
.../modules/tubemq/tubemq-manager/quick_start.json | 2 +-
.../modules/tubemq/tubemq-manager/quick_start.md | 8 +++---
zh-cn/docs/modules/agent/quick_start.html | 23 ++++++++++------
zh-cn/docs/modules/agent/quick_start.json | 2 +-
zh-cn/docs/modules/agent/quick_start.md | 31 +++++++++++++++-------
zh-cn/docs/modules/dataproxy/quick_start.html | 4 +--
zh-cn/docs/modules/dataproxy/quick_start.json | 2 +-
zh-cn/docs/modules/dataproxy/quick_start.md | 5 ++--
.../modules/tubemq/tubemq-manager/quick_start.html | 7 ++---
.../modules/tubemq/tubemq-manager/quick_start.json | 2 +-
.../modules/tubemq/tubemq-manager/quick_start.md | 7 ++---
24 files changed, 131 insertions(+), 71 deletions(-)
diff --git a/docs/en-us/modules/agent/quick_start.md
b/docs/en-us/modules/agent/quick_start.md
index 667d7bb..667d342 100644
--- a/docs/en-us/modules/agent/quick_start.md
+++ b/docs/en-us/modules/agent/quick_start.md
@@ -32,6 +32,7 @@ Create a new folder named .inlong in the agent directory, and
create a new bid+.
write:
```ini
{"cluster_id":1,"isInterVisit":1,"size":1,"address": [{"port":write proxy
port,"host":"write proxy ip"}], "switch":0}
+Among them, cluster_id, isInterVisit, and switch are reserved fields, please
fill in the default values
```
### 2.3 local job configuration
@@ -110,7 +111,7 @@ agent.http.port=Available ports
},
"name": "fileAgentTest",
"source": "org.apache.inlong.agent.plugin.sources.TextFileSource",
- "sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink",
+ "sink": "org.apache.inlong.agent.plugin.sinks.ProxySink",
"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel"
},
"bus": {
@@ -128,4 +129,12 @@ After decompression, run the following command
```bash
sh agent.sh start
-```
\ No newline at end of file
+```
+
+## 4、eg for directory config
+
+ E.g:
+ /data/inlong-agent/test.log //Represents reading the new file test.log in
the inlong-agent folder
+ /data/inlong-agent/test[0-9]{1} // means to read the new file test in the
inlong-agent folder followed by a number at the end
+ /data/inlong-agent/test //If test is a directory, it means to read all new
files under test
+ /data/inlong-agent/^\\d+(\\.\\d+)? // Start with one or more digits,
followed by. or end with one. or more digits (? stands for optional, can match
Examples: "5", "1.5" and "2.21"
\ No newline at end of file
diff --git a/docs/en-us/modules/dataproxy/quick_start.md
b/docs/en-us/modules/dataproxy/quick_start.md
index d556fbf..da17b2c 100644
--- a/docs/en-us/modules/dataproxy/quick_start.md
+++ b/docs/en-us/modules/dataproxy/quick_start.md
@@ -34,8 +34,8 @@ notice that conf/flume.conf FLUME_HOME is proxy the directory
for proxy inner da
### config manager
vim conf/common.properties
write：
- cluster_id=1
- manager_hosts=ip:port
+ cluster_id=1 // cluter id is for future use please write 1
+ manager_hosts=ip:port // manager open api
## 3、run
sh bin/start.sh
diff --git a/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md
b/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md
index 3b4ea81..f965707 100644
--- a/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md
+++ b/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md
@@ -6,8 +6,10 @@
environment requirements：
-mysql
-java(1.8+)
+ mysql
+ java(1.8+)
+
+
In the InLong/inlong-tubemq/tubemq-manager/dist/ directory, you can find the
file apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
tar -zxvf apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
@@ -126,7 +128,7 @@ POST
return json
{ "result":true, "errCode":0, "errMsg":"OK", }
- { "result":false, "errCode": 100, "errMsg":"topic thirty5is not writable"}
+ { "result":false, "errCode": 100, "errMsg":"topic test is not writable"}
{ "result":false, "errCode": 101, "errMsg":"no such topic in master"}
result is false as not writable
diff --git a/docs/zh-cn/modules/agent/quick_start.md
b/docs/zh-cn/modules/agent/quick_start.md
index 073ce2c..a6baa2b 100644
--- a/docs/zh-cn/modules/agent/quick_start.md
+++ b/docs/zh-cn/modules/agent/quick_start.md
@@ -1,4 +1,4 @@
-## ~~~~一、安装
+## 一、安装
环境要求：
java JDK 1.8
@@ -25,18 +25,18 @@ agent.manager.vip.http.host=manager open api host
agent.manager.vip.http.port=manager open api port
```
-### 2.2 Proxy 相关设置
+### 2.2 DataProxy 相关设置
在agent目录下新建.inlong文件夹，内部新建bid+.local文件，例如设置的发送bid为a, 则新建a.local文件
内部将写入：
```ini
{"cluster_id":1,"isInterVisit":1,"size":1,"address": [{"port":写入proxy
port,"host":"写入proxy ip"}], "switch":0}
+其中cluster_id, isInterVisit，switch为预留字段，请填写默认值
```
### 2.3 agent本地运行job配置
如果不使用线上模式，可以使用本地文件新建读取任务
-在conf中新建目录jobs，
-vim job1.json：
+在conf中新建目录jobs，jobs内部vim job1.json：
写入
```json
@@ -78,7 +78,7 @@ vim job1.json：
- job.dir.pattern: 配置读取的文件路径，可包含正则表达式
- job.trigger: 触发器名称，默认为DirectoryTrigger，功能为监听文件夹下的文件产生事件，任务运行时已有的文件不会读取
- job.source: 使用的数据源类型，默认为TextFileSource，读取文本文件
-- job.sink：使用的写入器类型，默认为ProxySink，发送消息到proxy中
+- job.sink：使用的写入器类型，默认为ProxySink，发送消息到dataproxy中
- proxy.bid: 写入proxy时使用的bid类型
### 2.4 实时添加job配置
@@ -93,7 +93,7 @@ agent.http.port=可用端口
#### 2.4.2 执行如下命令：
```bash
-curl --location --request POST 'http://localhost:8129/config/job' \
+curl --location --request POST 'http://localhost:8018/config/job' \
--header 'Content-Type: application/json' \
--data '{
"job": {
@@ -110,7 +110,7 @@ curl --location --request POST
'http://localhost:8129/config/job' \
},
"name": "fileAgentTest",
"source": "org.apache.inlong.agent.plugin.sources.TextFileSource",
-"sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink",
+"sink": "org.apache.inlong.agent.plugin.sinks.ProxySink",
"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel"
},
"proxy": {
@@ -119,13 +119,24 @@ curl --location --request POST
'http://localhost:8129/config/job' \
"op": "add"
}'
```
-
+
其中参数分别为：
pattern: 代表读取/data/inlong-agent/test.log文件，需要新建任务后再创建文件
- proxy.bid: 代表发送到proxy使用的bid
+ proxy.bid: 代表发送到dataproxy使用的bid
## 三、运行
解压后如下命令运行
```bash
sh agent.sh start
-```
\ No newline at end of file
+```
+
+
+## 四、可支持的路径配置方案
+
+ 例如：
+ /data/inlong-agent/test.log //代表读取inlong-agent文件夹下的的新增文件test.log
+ /data/inlong-agent/test[0-9]{1} //代表读取inlong-agent文件夹下的新增文件test后接一个数字结尾
+ /data/inlong-agent/test //如果test为目录，则代表读取test下的所有新增文件
+ /data/inlong-agent/^\\d+(\\.\\d+)? //
以一个或多个数字开头,之后可以是.或者一个.或多个数字结尾，?代表可选,可以匹配的实例："5", "1.5" 和 "2.21"
+
+
diff --git a/docs/zh-cn/modules/dataproxy/quick_start.md
b/docs/zh-cn/modules/dataproxy/quick_start.md
index 07b0882..c4d031c 100644
--- a/docs/zh-cn/modules/dataproxy/quick_start.md
+++ b/docs/zh-cn/modules/dataproxy/quick_start.md
@@ -20,7 +20,6 @@
sed -i 's/TUBE_LIST/ip1:port,ip2:port/g' conf/flume.conf
-
修改以下内容为proxy端口号
agent1.sources.tcp-source.port = 46801
@@ -35,8 +34,8 @@ sed -i 's/TUBE_LIST/ip1:port,ip2:port/g' conf/flume.conf
vim conf/common.properties
写入以下配置：
- cluster_id=1
- manager_hosts=ip:port // 这里写入的是manager_api地址
+ cluster_id=1 // 预留字段，请填写默认值1
+ manager_hosts=ip:port // 这里写入的是manager open api地址
## 三、启动
sh bin/start.sh
diff --git a/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md
b/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md
index 560c75a..c83473e 100644
--- a/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md
+++ b/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md
@@ -6,8 +6,9 @@
环境要求：
-mysql
-java(1.8+)
+ mysql
+ java(1.8+)
+
在InLong/inlong-tubemq/tubemq-manager/dist/目录当中，能够找到文件为apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
tar -zxvf apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
@@ -129,7 +130,7 @@ POST
返回json格式样例
{ "result":true, "errCode":0, "errMsg":"OK", }
- { "result":false, "errCode": 100, "errMsg":"topic thirty5is not writable"}
+ { "result":false, "errCode": 100, "errMsg":"topic test is not writable"}
{ "result":false, "errCode": 101, "errMsg":"no such topic in master"}
result为false为不可写
diff --git a/en-us/docs/modules/agent/quick_start.html
b/en-us/docs/modules/agent/quick_start.html
index 13abacb..2417de7 100644
--- a/en-us/docs/modules/agent/quick_start.html
+++ b/en-us/docs/modules/agent/quick_start.html
@@ -35,6 +35,7 @@ The tgz installation package can be found in the target under
the agent-release
<p>Create a new folder named .inlong in the agent directory, and create a new
bid+.local file inside. For example, if the sending bid is set to a, then
create a new file a.local</p>
<p>write:</p>
<pre><code
class="language-ini">{"cluster_id":1,"isInterVisit":1,"size":1,"address":
[{"port":write proxy port,"host":"write proxy ip"}], "switch":0}
+Among them, cluster_id, isInterVisit, and switch are reserved fields, please
fill in the default values
</code></pre>
<h3>2.3 local job configuration</h3>
<p>If you do not use the online mode, you can use the local file to create a
new read task
@@ -106,7 +107,7 @@ vim job1.json：</p>
},
"name": "fileAgentTest",
"source": "org.apache.inlong.agent.plugin.sources.TextFileSource",
- "sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink",
+ "sink": "org.apache.inlong.agent.plugin.sinks.ProxySink",
"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel"
},
"bus": {
@@ -122,6 +123,12 @@ vim job1.json：</p>
<p>After decompression, run the following command</p>
<pre><code class="language-bash">sh agent.sh start
</code></pre>
+<h2>4、eg for directory config</h2>
+<pre><code>E.g:
+/data/inlong-agent/test.log //Represents reading the new file test.log in the
inlong-agent folder
+/data/inlong-agent/test[0-9]{1} // means to read the new file test in the
inlong-agent folder followed by a number at the end
+/data/inlong-agent/test //If test is a directory, it means to read all new
files under test
+/data/inlong-agent/^\\d+(\\.\\d+)? // Start with one or more digits, followed
by. or end with one. or more digits (? stands for optional, can match Examples:
"5", "1.5" and "2.21"</code></pre>
</div></section><footer class="footer-container"><div class="footer-body"><img
src="/img/incubator-logo.svg"/><div class="cols-container"><div class="col
col-24"><p>Apache InLong (incubating) is an effort undergoing incubation at The
Apache Software Foundation (ASF), sponsored by Incubator. Incubation is
required of all newly accepted projects until a further review indicates that
the infrastructure, communications, and decision making process have stabilized
in a manner consistent with [...]
<script
src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js";></script>
<script
src="https://f.alicdn.com/react/15.4.1/react-dom.min.js";></script>
diff --git a/en-us/docs/modules/agent/quick_start.json
b/en-us/docs/modules/agent/quick_start.json
index 06f2354..561d344 100644
--- a/en-us/docs/modules/agent/quick_start.json
+++ b/en-us/docs/modules/agent/quick_start.json
@@ -1,6 +1,6 @@
{
"filename": "quick_start.md",
- "__html": "<h2>1、install</h2>\n<pre><code>env requirements：\njava JDK
1.8\nMaven 3.6 +\n\n\nmvn clean package -DskipTests\n\nThe tgz installation
package can be found in the target under the
agent-release\n</code></pre>\n<h2>2、Configuration</h2>\n<p>The agent supports
two modes of operation: local operation and online operation</p>\n<h3>2.1 Agent
configuration</h3>\n<p>Online operation needs to pull the configuration from
inlong-manager, the configuration conf/agent.properties is as fo [...]
+ "__html": "<h2>1、install</h2>\n<pre><code>env requirements：\njava JDK
1.8\nMaven 3.6 +\n\n\nmvn clean package -DskipTests\n\nThe tgz installation
package can be found in the target under the
agent-release\n</code></pre>\n<h2>2、Configuration</h2>\n<p>The agent supports
two modes of operation: local operation and online operation</p>\n<h3>2.1 Agent
configuration</h3>\n<p>Online operation needs to pull the configuration from
inlong-manager, the configuration conf/agent.properties is as fo [...]
"link": "/en-us/docs/modules/agent/quick_start.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/modules/agent/quick_start.md
b/en-us/docs/modules/agent/quick_start.md
index 667d7bb..667d342 100644
--- a/en-us/docs/modules/agent/quick_start.md
+++ b/en-us/docs/modules/agent/quick_start.md
@@ -32,6 +32,7 @@ Create a new folder named .inlong in the agent directory, and
create a new bid+.
write:
```ini
{"cluster_id":1,"isInterVisit":1,"size":1,"address": [{"port":write proxy
port,"host":"write proxy ip"}], "switch":0}
+Among them, cluster_id, isInterVisit, and switch are reserved fields, please
fill in the default values
```
### 2.3 local job configuration
@@ -110,7 +111,7 @@ agent.http.port=Available ports
},
"name": "fileAgentTest",
"source": "org.apache.inlong.agent.plugin.sources.TextFileSource",
- "sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink",
+ "sink": "org.apache.inlong.agent.plugin.sinks.ProxySink",
"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel"
},
"bus": {
@@ -128,4 +129,12 @@ After decompression, run the following command
```bash
sh agent.sh start
-```
\ No newline at end of file
+```
+
+## 4、eg for directory config
+
+ E.g:
+ /data/inlong-agent/test.log //Represents reading the new file test.log in
the inlong-agent folder
+ /data/inlong-agent/test[0-9]{1} // means to read the new file test in the
inlong-agent folder followed by a number at the end
+ /data/inlong-agent/test //If test is a directory, it means to read all new
files under test
+ /data/inlong-agent/^\\d+(\\.\\d+)? // Start with one or more digits,
followed by. or end with one. or more digits (? stands for optional, can match
Examples: "5", "1.5" and "2.21"
\ No newline at end of file
diff --git a/en-us/docs/modules/dataproxy/quick_start.html
b/en-us/docs/modules/dataproxy/quick_start.html
index dce5d67..98357e7 100644
--- a/en-us/docs/modules/dataproxy/quick_start.html
+++ b/en-us/docs/modules/dataproxy/quick_start.html
@@ -38,8 +38,8 @@ cd ..
<h3>config manager</h3>
<pre><code>vim conf/common.properties
write：
-cluster_id=1
-manager_hosts=ip:port
+cluster_id=1 // cluter id is for future use please write 1
+manager_hosts=ip:port // manager open api
</code></pre>
<h2>3、run</h2>
<pre><code>sh bin/start.sh
diff --git a/en-us/docs/modules/dataproxy/quick_start.json
b/en-us/docs/modules/dataproxy/quick_start.json
index 26f523f..06c3674 100644
--- a/en-us/docs/modules/dataproxy/quick_start.json
+++ b/en-us/docs/modules/dataproxy/quick_start.json
@@ -1,6 +1,6 @@
{
"filename": "quick_start.md",
- "__html": "<h2>1、install</h2>\n<pre><code>mvn clean package
-DskipTests\n</code></pre>\n<p>The tgz installation package
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz can be found under
the inlong-dataproxy/dataproxy-dist/target</p>\n<p>un tar
it</p>\n<pre><code>tar -zxvf
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>2、configuration</h2>\n<pre><code>cd
apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>config
tube</h3> [...]
+ "__html": "<h2>1、install</h2>\n<pre><code>mvn clean package
-DskipTests\n</code></pre>\n<p>The tgz installation package
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz can be found under
the inlong-dataproxy/dataproxy-dist/target</p>\n<p>un tar
it</p>\n<pre><code>tar -zxvf
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>2、configuration</h2>\n<pre><code>cd
apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>config
tube</h3> [...]
"link": "/en-us/docs/modules/dataproxy/quick_start.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/modules/dataproxy/quick_start.md
b/en-us/docs/modules/dataproxy/quick_start.md
index d556fbf..da17b2c 100644
--- a/en-us/docs/modules/dataproxy/quick_start.md
+++ b/en-us/docs/modules/dataproxy/quick_start.md
@@ -34,8 +34,8 @@ notice that conf/flume.conf FLUME_HOME is proxy the directory
for proxy inner da
### config manager
vim conf/common.properties
write：
- cluster_id=1
- manager_hosts=ip:port
+ cluster_id=1 // cluter id is for future use please write 1
+ manager_hosts=ip:port // manager open api
## 3、run
sh bin/start.sh
diff --git a/en-us/docs/modules/tubemq/tubemq-manager/quick_start.html
b/en-us/docs/modules/tubemq/tubemq-manager/quick_start.html
index bad2adf..054d3e2 100644
--- a/en-us/docs/modules/tubemq/tubemq-manager/quick_start.html
+++ b/en-us/docs/modules/tubemq/tubemq-manager/quick_start.html
@@ -17,9 +17,10 @@
</code></pre>
<h2>deploy</h2>
<p>environment requirements：</p>
-<p>mysql
+<pre><code>mysql
java(1.8+)
-In the InLong/inlong-tubemq/tubemq-manager/dist/ directory, you can find the
file apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>
+</code></pre>
+<p>In the InLong/inlong-tubemq/tubemq-manager/dist/ directory, you can find
the file apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>
<pre><code>tar -zxvf
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
</code></pre>
<p>Add mysql information in conf/application.properties:</p>
@@ -111,7 +112,7 @@ user (Required) After the access authorization
verification needs to verify the
</code></pre>
<p>return json</p>
<pre><code>{ "result":true, "errCode":0,
"errMsg":"OK", }
-{ "result":false, "errCode": 100,
"errMsg":"topic thirty5is not writable"}
+{ "result":false, "errCode": 100,
"errMsg":"topic test is not writable"}
{ "result":false, "errCode": 101,
"errMsg":"no such topic in master"}
</code></pre>
<p>result is false as not writable</p>
diff --git a/en-us/docs/modules/tubemq/tubemq-manager/quick_start.json
b/en-us/docs/modules/tubemq/tubemq-manager/quick_start.json
index f7da12b..daf4e29 100644
--- a/en-us/docs/modules/tubemq/tubemq-manager/quick_start.json
+++ b/en-us/docs/modules/tubemq/tubemq-manager/quick_start.json
@@ -1,6 +1,6 @@
{
"filename": "quick_start.md",
- "__html": "<h2>compile</h2>\n<pre><code>mvn clean package
\n</code></pre>\n<h2>deploy</h2>\n<p>environment
requirements：</p>\n<p>mysql\njava(1.8+)\nIn the
InLong/inlong-tubemq/tubemq-manager/dist/ directory, you can find the file
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<pre><code>tar
-zxvf
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<p>Add
mysql information in
conf/application.properties:</p>\n<pre><code>spring.jpa.hibernat [...]
+ "__html": "<h2>compile</h2>\n<pre><code>mvn clean package
\n</code></pre>\n<h2>deploy</h2>\n<p>environment
requirements：</p>\n<pre><code>mysql\njava(1.8+)\n</code></pre>\n<p>In the
InLong/inlong-tubemq/tubemq-manager/dist/ directory, you can find the file
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<pre><code>tar
-zxvf
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<p>Add
mysql information in conf/application.properties:</p>\n<pre [...]
"link": "/en-us/docs/modules/tubemq/tubemq-manager/quick_start.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/modules/tubemq/tubemq-manager/quick_start.md
b/en-us/docs/modules/tubemq/tubemq-manager/quick_start.md
index 3b4ea81..f965707 100644
--- a/en-us/docs/modules/tubemq/tubemq-manager/quick_start.md
+++ b/en-us/docs/modules/tubemq/tubemq-manager/quick_start.md
@@ -6,8 +6,10 @@
environment requirements：
-mysql
-java(1.8+)
+ mysql
+ java(1.8+)
+
+
In the InLong/inlong-tubemq/tubemq-manager/dist/ directory, you can find the
file apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
tar -zxvf apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
@@ -126,7 +128,7 @@ POST
return json
{ "result":true, "errCode":0, "errMsg":"OK", }
- { "result":false, "errCode": 100, "errMsg":"topic thirty5is not writable"}
+ { "result":false, "errCode": 100, "errMsg":"topic test is not writable"}
{ "result":false, "errCode": 101, "errMsg":"no such topic in master"}
result is false as not writable
diff --git a/zh-cn/docs/modules/agent/quick_start.html
b/zh-cn/docs/modules/agent/quick_start.html
index e6dc4a1..3398bd3 100644
--- a/zh-cn/docs/modules/agent/quick_start.html
+++ b/zh-cn/docs/modules/agent/quick_start.html
@@ -12,7 +12,7 @@
<link rel="stylesheet" href="/build/documentation.css" />
</head>
<body>
- <div id="root"><div class="documentation-page"
data-reactroot=""><header class="header-container header-container-normal"><div
class="header-body"><a href="/zh-cn/index.html"><a href="//www.apache.org"><img
class="logo apache" style="width:120px" src="/img/asf_logo.svg"/></a><div
class="logo-split"></div><img class="logo tube"
style="width:120px;top:12px;position:absolute" src="/img/Tube
logo.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class= [...]
+ <div id="root"><div class="documentation-page"
data-reactroot=""><header class="header-container header-container-normal"><div
class="header-body"><a href="/zh-cn/index.html"><a href="//www.apache.org"><img
class="logo apache" style="width:120px" src="/img/asf_logo.svg"/></a><div
class="logo-split"></div><img class="logo tube"
style="width:120px;top:12px;position:absolute" src="/img/Tube
logo.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class= [...]
<pre><code>环境要求：
java JDK 1.8
Maven 3.6 +
@@ -31,15 +31,15 @@ mvn clean package -DskipTests
<span class="hljs-attr">agent.manager.vip.http.host</span>=manager open api
host
<span class="hljs-attr">agent.manager.vip.http.port</span>=manager open api
port
</code></pre>
-<h3>2.2 Proxy 相关设置</h3>
+<h3>2.2 DataProxy 相关设置</h3>
<p>在agent目录下新建.inlong文件夹，内部新建bid+.local文件，例如设置的发送bid为a, 则新建a.local文件</p>
<p>内部将写入：</p>
<pre><code
class="language-ini">{"cluster_id":1,"isInterVisit":1,"size":1,"address":
[{"port":写入proxy port,"host":"写入proxy ip"}], "switch":0}
+其中cluster_id, isInterVisit，switch为预留字段，请填写默认值
</code></pre>
<h3>2.3 agent本地运行job配置</h3>
<p>如果不使用线上模式，可以使用本地文件新建读取任务
-在conf中新建目录jobs，
-vim job1.json：</p>
+在conf中新建目录jobs，jobs内部vim job1.json：</p>
<p>写入</p>
<pre><code class="language-json"> {
@@ -78,7 +78,7 @@ vim job1.json：</p>
<li>job.dir.pattern: 配置读取的文件路径，可包含正则表达式</li>
<li>job.trigger: 触发器名称，默认为DirectoryTrigger，功能为监听文件夹下的文件产生事件，任务运行时已有的文件不会读取</li>
<li>job.source: 使用的数据源类型，默认为TextFileSource，读取文本文件</li>
-<li>job.sink：使用的写入器类型，默认为ProxySink，发送消息到proxy中</li>
+<li>job.sink：使用的写入器类型，默认为ProxySink，发送消息到dataproxy中</li>
<li>proxy.bid: 写入proxy时使用的bid类型</li>
</ul>
<h3>2.4 实时添加job配置</h3>
@@ -89,7 +89,7 @@ vim job1.json：</p>
<span class="hljs-attr">agent.http.port</span>=可用端口
</code></pre>
<h4>2.4.2 执行如下命令：</h4>
-<pre><code class="language-bash">curl --location --request POST <span
class="hljs-string">'http://localhost:8129/config/job'</span> \
+<pre><code class="language-bash">curl --location --request POST <span
class="hljs-string">'http://localhost:8018/config/job'</span> \
--header <span class="hljs-string">'Content-Type: application/json'</span> \
--data <span class="hljs-string">'{
"job": {
@@ -106,7 +106,7 @@ vim job1.json：</p>
},
"name": "fileAgentTest",
"source": "org.apache.inlong.agent.plugin.sources.TextFileSource",
-"sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink",
+"sink": "org.apache.inlong.agent.plugin.sinks.ProxySink",
"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel"
},
"proxy": {
@@ -117,12 +117,19 @@ vim job1.json：</p>
</code></pre>
<pre><code>其中参数分别为：
pattern: 代表读取/data/inlong-agent/test.log文件，需要新建任务后再创建文件
-proxy.bid: 代表发送到proxy使用的bid
+proxy.bid: 代表发送到dataproxy使用的bid
</code></pre>
<h2>三、运行</h2>
<p>解压后如下命令运行</p>
<pre><code class="language-bash">sh agent.sh start
</code></pre>
+<h2>四、可支持的路径配置方案</h2>
+<pre><code>例如：
+/data/inlong-agent/test.log //代表读取inlong-agent文件夹下的的新增文件test.log
+/data/inlong-agent/test[0-9]{1} //代表读取inlong-agent文件夹下的新增文件test后接一个数字结尾
+/data/inlong-agent/test //如果test为目录，则代表读取test下的所有新增文件
+/data/inlong-agent/^\\d+(\\.\\d+)? //
以一个或多个数字开头,之后可以是.或者一个.或多个数字结尾，?代表可选,可以匹配的实例："5", "1.5" 和
"2.21"
+</code></pre>
</div></section><footer class="footer-container"><div class="footer-body"><img
src="/img/incubator-logo.svg"/><div class="cols-container"><div class="col
col-24"><p>Apache InLong (incubating) is an effort undergoing incubation at The
Apache Software Foundation (ASF), sponsored by Incubator. Incubation is
required of all newly accepted projects until a further review indicates that
the infrastructure, communications, and decision making process have stabilized
in a manner consistent with [...]
<script
src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js";></script>
<script
src="https://f.alicdn.com/react/15.4.1/react-dom.min.js";></script>
diff --git a/zh-cn/docs/modules/agent/quick_start.json
b/zh-cn/docs/modules/agent/quick_start.json
index ab5e9ef..a4c9fb0 100644
--- a/zh-cn/docs/modules/agent/quick_start.json
+++ b/zh-cn/docs/modules/agent/quick_start.json
@@ -1,6 +1,6 @@
{
"filename": "quick_start.md",
- "__html": "<h2>~~~~一、安装</h2>\n<pre><code>环境要求：\njava JDK 1.8\nMaven 3.6
+\n\ninlong目录下执行命令\nmvn clean package
-DskipTests\n\n在项目下的agent-release/target里面可以找到tgz安装包\n</code></pre>\n<h2>二、配置</h2>\n<p>agent
支持两种运行模式：本地运行以及线上运行</p>\n<h3>2.1 Agent
线上运行相关设置</h3>\n<p>线上运行需要从inlong-manager拉取配置，配置conf/agent.properties如下：</p>\n<pre><code
class=\"language-ini\"><span
class=\"hljs-attr\">agent.fetcher.classname</span>=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher
(设置任务获取的类名，默认为ManagerFetche [...]
+ "__html": "<h2>一、安装</h2>\n<pre><code>环境要求：\njava JDK 1.8\nMaven 3.6
+\n\ninlong目录下执行命令\nmvn clean package
-DskipTests\n\n在项目下的agent-release/target里面可以找到tgz安装包\n</code></pre>\n<h2>二、配置</h2>\n<p>agent
支持两种运行模式：本地运行以及线上运行</p>\n<h3>2.1 Agent
线上运行相关设置</h3>\n<p>线上运行需要从inlong-manager拉取配置，配置conf/agent.properties如下：</p>\n<pre><code
class=\"language-ini\"><span
class=\"hljs-attr\">agent.fetcher.classname</span>=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher
(设置任务获取的类名，默认为ManagerFetcher）\n [...]
"link": "/zh-cn/docs/modules/agent/quick_start.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/modules/agent/quick_start.md
b/zh-cn/docs/modules/agent/quick_start.md
index 073ce2c..a6baa2b 100644
--- a/zh-cn/docs/modules/agent/quick_start.md
+++ b/zh-cn/docs/modules/agent/quick_start.md
@@ -1,4 +1,4 @@
-## ~~~~一、安装
+## 一、安装
环境要求：
java JDK 1.8
@@ -25,18 +25,18 @@ agent.manager.vip.http.host=manager open api host
agent.manager.vip.http.port=manager open api port
```
-### 2.2 Proxy 相关设置
+### 2.2 DataProxy 相关设置
在agent目录下新建.inlong文件夹，内部新建bid+.local文件，例如设置的发送bid为a, 则新建a.local文件
内部将写入：
```ini
{"cluster_id":1,"isInterVisit":1,"size":1,"address": [{"port":写入proxy
port,"host":"写入proxy ip"}], "switch":0}
+其中cluster_id, isInterVisit，switch为预留字段，请填写默认值
```
### 2.3 agent本地运行job配置
如果不使用线上模式，可以使用本地文件新建读取任务
-在conf中新建目录jobs，
-vim job1.json：
+在conf中新建目录jobs，jobs内部vim job1.json：
写入
```json
@@ -78,7 +78,7 @@ vim job1.json：
- job.dir.pattern: 配置读取的文件路径，可包含正则表达式
- job.trigger: 触发器名称，默认为DirectoryTrigger，功能为监听文件夹下的文件产生事件，任务运行时已有的文件不会读取
- job.source: 使用的数据源类型，默认为TextFileSource，读取文本文件
-- job.sink：使用的写入器类型，默认为ProxySink，发送消息到proxy中
+- job.sink：使用的写入器类型，默认为ProxySink，发送消息到dataproxy中
- proxy.bid: 写入proxy时使用的bid类型
### 2.4 实时添加job配置
@@ -93,7 +93,7 @@ agent.http.port=可用端口
#### 2.4.2 执行如下命令：
```bash
-curl --location --request POST 'http://localhost:8129/config/job' \
+curl --location --request POST 'http://localhost:8018/config/job' \
--header 'Content-Type: application/json' \
--data '{
"job": {
@@ -110,7 +110,7 @@ curl --location --request POST
'http://localhost:8129/config/job' \
},
"name": "fileAgentTest",
"source": "org.apache.inlong.agent.plugin.sources.TextFileSource",
-"sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink",
+"sink": "org.apache.inlong.agent.plugin.sinks.ProxySink",
"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel"
},
"proxy": {
@@ -119,13 +119,24 @@ curl --location --request POST
'http://localhost:8129/config/job' \
"op": "add"
}'
```
-
+
其中参数分别为：
pattern: 代表读取/data/inlong-agent/test.log文件，需要新建任务后再创建文件
- proxy.bid: 代表发送到proxy使用的bid
+ proxy.bid: 代表发送到dataproxy使用的bid
## 三、运行
解压后如下命令运行
```bash
sh agent.sh start
-```
\ No newline at end of file
+```
+
+
+## 四、可支持的路径配置方案
+
+ 例如：
+ /data/inlong-agent/test.log //代表读取inlong-agent文件夹下的的新增文件test.log
+ /data/inlong-agent/test[0-9]{1} //代表读取inlong-agent文件夹下的新增文件test后接一个数字结尾
+ /data/inlong-agent/test //如果test为目录，则代表读取test下的所有新增文件
+ /data/inlong-agent/^\\d+(\\.\\d+)? //
以一个或多个数字开头,之后可以是.或者一个.或多个数字结尾，?代表可选,可以匹配的实例："5", "1.5" 和 "2.21"
+
+
diff --git a/zh-cn/docs/modules/dataproxy/quick_start.html
b/zh-cn/docs/modules/dataproxy/quick_start.html
index a49642a..2276f2e 100644
--- a/zh-cn/docs/modules/dataproxy/quick_start.html
+++ b/zh-cn/docs/modules/dataproxy/quick_start.html
@@ -39,8 +39,8 @@ cd ..
<pre><code>vim conf/common.properties
写入以下配置：
-cluster_id=1
-manager_hosts=ip:port // 这里写入的是manager_api地址
+cluster_id=1 // 预留字段，请填写默认值1
+manager_hosts=ip:port // 这里写入的是manager open api地址
</code></pre>
<h2>三、启动</h2>
<pre><code>sh bin/start.sh
diff --git a/zh-cn/docs/modules/dataproxy/quick_start.json
b/zh-cn/docs/modules/dataproxy/quick_start.json
index fe1d524..a902792 100644
--- a/zh-cn/docs/modules/dataproxy/quick_start.json
+++ b/zh-cn/docs/modules/dataproxy/quick_start.json
@@ -1,6 +1,6 @@
{
"filename": "quick_start.md",
- "__html": "<h2>一、安装</h2>\n<pre><code>mvn clean package
-DskipTests\n</code></pre>\n<p>在inlong-dataproxy/dataproxy-dist/target项目下可以找到tgz安装包apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<p>解压</p>\n<pre><code>tar
-zxvf
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>二、配置</h2>\n<pre><code>cd
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>配置tube地址和端口号</h3>\n<pre><code>执行以下替换命令,其中ip:port为tube地址\n</code></pre>\n<p
[...]
+ "__html": "<h2>一、安装</h2>\n<pre><code>mvn clean package
-DskipTests\n</code></pre>\n<p>在inlong-dataproxy/dataproxy-dist/target项目下可以找到tgz安装包apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<p>解压</p>\n<pre><code>tar
-zxvf
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>二、配置</h2>\n<pre><code>cd
apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>配置tube地址和端口号</h3>\n<pre><code>执行以下替换命令,其中ip:port为tube地址\n</code></pre>\n<p
[...]
"link": "/zh-cn/docs/modules/dataproxy/quick_start.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/modules/dataproxy/quick_start.md
b/zh-cn/docs/modules/dataproxy/quick_start.md
index 07b0882..c4d031c 100644
--- a/zh-cn/docs/modules/dataproxy/quick_start.md
+++ b/zh-cn/docs/modules/dataproxy/quick_start.md
@@ -20,7 +20,6 @@
sed -i 's/TUBE_LIST/ip1:port,ip2:port/g' conf/flume.conf
-
修改以下内容为proxy端口号
agent1.sources.tcp-source.port = 46801
@@ -35,8 +34,8 @@ sed -i 's/TUBE_LIST/ip1:port,ip2:port/g' conf/flume.conf
vim conf/common.properties
写入以下配置：
- cluster_id=1
- manager_hosts=ip:port // 这里写入的是manager_api地址
+ cluster_id=1 // 预留字段，请填写默认值1
+ manager_hosts=ip:port // 这里写入的是manager open api地址
## 三、启动
sh bin/start.sh
diff --git a/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.html
b/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.html
index 28c8433..e7ffb5b 100644
--- a/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.html
+++ b/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.html
@@ -17,9 +17,10 @@
</code></pre>
<h2>部署</h2>
<p>环境要求：</p>
-<p>mysql
+<pre><code>mysql
java(1.8+)
-在InLong/inlong-tubemq/tubemq-manager/dist/目录当中，能够找到文件为apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>
+</code></pre>
+<p>在InLong/inlong-tubemq/tubemq-manager/dist/目录当中，能够找到文件为apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>
<pre><code>tar -zxvf
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
</code></pre>
<p>在conf/application.properties中添加mysql信息：</p>
@@ -113,7 +114,7 @@ user (必填) 之后接入权限验证需要验证用户，这里预留出来
</code></pre>
<p>返回json格式样例</p>
<pre><code>{ "result":true, "errCode":0,
"errMsg":"OK", }
-{ "result":false, "errCode": 100,
"errMsg":"topic thirty5is not writable"}
+{ "result":false, "errCode": 100,
"errMsg":"topic test is not writable"}
{ "result":false, "errCode": 101,
"errMsg":"no such topic in master"}
</code></pre>
<p>result为false为不可写</p>
diff --git a/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.json
b/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.json
index 59ad09b..9e95b19 100644
--- a/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.json
+++ b/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.json
@@ -1,6 +1,6 @@
{
"filename": "quick_start.md",
- "__html": "<h2>编译</h2>\n<pre><code>mvn clean package
\n</code></pre>\n<h2>部署</h2>\n<p>环境要求：</p>\n<p>mysql\njava(1.8+)\n在InLong/inlong-tubemq/tubemq-manager/dist/目录当中，能够找到文件为apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<pre><code>tar
-zxvf
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<p>在conf/application.properties中添加mysql信息：</p>\n<pre><code>spring.jpa.hibernate.ddl-auto=update\n#
configuration for manager\nspring.datasource.url=j [...]
+ "__html": "<h2>编译</h2>\n<pre><code>mvn clean package
\n</code></pre>\n<h2>部署</h2>\n<p>环境要求：</p>\n<pre><code>mysql\njava(1.8+)\n</code></pre>\n<p>在InLong/inlong-tubemq/tubemq-manager/dist/目录当中，能够找到文件为apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<pre><code>tar
-zxvf
apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<p>在conf/application.properties中添加mysql信息：</p>\n<pre><code>spring.jpa.hibernate.ddl-auto=update\n#
configuration for manage [...]
"link": "/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.md
b/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.md
index 560c75a..c83473e 100644
--- a/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.md
+++ b/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.md
@@ -6,8 +6,9 @@
环境要求：
-mysql
-java(1.8+)
+ mysql
+ java(1.8+)
+
在InLong/inlong-tubemq/tubemq-manager/dist/目录当中，能够找到文件为apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
tar -zxvf apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
@@ -129,7 +130,7 @@ POST
返回json格式样例
{ "result":true, "errCode":0, "errMsg":"OK", }
- { "result":false, "errCode": 100, "errMsg":"topic thirty5is not writable"}
+ { "result":false, "errCode": 100, "errMsg":"topic test is not writable"}
{ "result":false, "errCode": 101, "errMsg":"no such topic in master"}
result为false为不可写
