dockerzhang commented on code in PR #381: URL: https://github.com/apache/incubator-inlong-website/pull/381#discussion_r875400188
########## docs/modules/sort/overview.md: ########## @@ -9,24 +9,34 @@ Inlong-sort is simply an Flink application, and relys on Inlong-manager to manag # features +## supported transforms +- String split index +- String regular replace +- String regular replace the first value +- Data filter +- Data distinct +- Regular join + ## supported sources -- pulsar +- Pulsar +- MySQL +- Kafka ## supported storages Review Comment: storages -> Load Node ########## docs/modules/sort/quick_start.md: ########## @@ -32,35 +30,17 @@ Notice： - `inlong-sort/sort-[version].jar` is the compiled jar ## Necessary configurations -- `--cluster-id ` represent a specified inlong-sort application, same as the configuration of `sort.appName` in inlong-manager -- `--dataflow.info.file` dataflow configuration file path -- `--source.type` source of the application, currently "pulsar" is supported -- `--sink.type` sink of the application, currently "clickhouse", "hive", "iceberg", "kafka" are supported -- `--metrics.audit.proxy.hosts` audit proxy host address for reporting audit metrics +- `--group.info.file` dataflow configuration file path +- `--lightweight` whether to use lightweight sort,default false Review Comment: we will use the lightweight version all the time, `--lightweight` is useless? ########## docs/modules/sort/overview.md: ########## @@ -9,24 +9,34 @@ Inlong-sort is simply an Flink application, and relys on Inlong-manager to manag # features +## supported transforms +- String split index +- String regular replace +- String regular replace the first value +- Data filter +- Data distinct +- Regular join + ## supported sources -- pulsar +- Pulsar +- MySQL +- Kafka ## supported storages -- hive（Currently we support parquet, orc and text file format） -- kafka -- clickhouse -- iceberg +- Hive +- Kafka +- ClickHouse +- Iceberg ## limitations Currently, we just support extracting specified fields in the stage of **Transform**. ## future plans ### More kinds of source systems -kafka and etc +Oracle, Mongo DB, SqlServer, and etc ### More kinds of storage systems Review Comment: storage systems -> Load Node ########## docs/modules/sort/overview.md: ########## @@ -9,24 +9,34 @@ Inlong-sort is simply an Flink application, and relys on Inlong-manager to manag # features +## supported transforms +- String split index +- String regular replace +- String regular replace the first value +- Data filter +- Data distinct +- Regular join + ## supported sources -- pulsar +- Pulsar +- MySQL +- Kafka ## supported storages -- hive（Currently we support parquet, orc and text file format） -- kafka -- clickhouse -- iceberg +- Hive +- Kafka +- ClickHouse +- Iceberg ## limitations Currently, we just support extracting specified fields in the stage of **Transform**. ## future plans ### More kinds of source systems Review Comment: source systems -> Extract Node ########## docs/modules/sort/overview.md: ########## @@ -9,24 +9,34 @@ Inlong-sort is simply an Flink application, and relys on Inlong-manager to manag # features +## supported transforms +- String split index +- String regular replace +- String regular replace the first value +- Data filter +- Data distinct +- Regular join + ## supported sources -- pulsar +- Pulsar +- MySQL +- Kafka ## supported storages Review Comment: storages -> Load Node ########## docs/modules/sort/overview.md: ########## @@ -9,24 +9,34 @@ Inlong-sort is simply an Flink application, and relys on Inlong-manager to manag # features +## supported transforms +- String split index +- String regular replace +- String regular replace the first value +- Data filter +- Data distinct +- Regular join + ## supported sources -- pulsar +- Pulsar +- MySQL +- Kafka ## supported storages -- hive（Currently we support parquet, orc and text file format） -- kafka -- clickhouse -- iceberg +- Hive +- Kafka +- ClickHouse +- Iceberg ## limitations Currently, we just support extracting specified fields in the stage of **Transform**. ## future plans ### More kinds of source systems -kafka and etc +Oracle, Mongo DB, SqlServer, and etc ### More kinds of storage systems Review Comment: storage systems -> Load Node ########## docs/modules/sort/quick_start.md: ########## @@ -32,35 +30,17 @@ Notice： - `inlong-sort/sort-[version].jar` is the compiled jar ## Necessary configurations -- `--cluster-id ` represent a specified inlong-sort application, same as the configuration of `sort.appName` in inlong-manager -- `--dataflow.info.file` dataflow configuration file path -- `--source.type` source of the application, currently "pulsar" is supported -- `--sink.type` sink of the application, currently "clickhouse", "hive", "iceberg", "kafka" are supported -- `--metrics.audit.proxy.hosts` audit proxy host address for reporting audit metrics +- `--group.info.file` dataflow configuration file path +- `--lightweight` whether to use lightweight sort,default false Review Comment: we will use the lightweight version all the time, `--lightweight` is useless? ########## docs/modules/sort/quick_start.md: ########## @@ -20,9 +20,7 @@ Now you can submit job to flink with the jar compiled, refer to [how to submit j Example： ``` ./bin/flink run -c org.apache.inlong.sort.singletenant.flink.Entrance inlong-sort/sort-[version].jar \ Review Comment: org.apache.inlong.sort.singletenant.flink.Entrance -> org.apache.inlong.sort.flink.Entrance singletenant have no meaning start from this version now, we should rename the entrance class name. ########## docs/modules/sort/overview.md: ########## @@ -9,24 +9,34 @@ Inlong-sort is simply an Flink application, and relys on Inlong-manager to manag # features +## supported transforms +- String split index +- String regular replace +- String regular replace the first value +- Data filter +- Data distinct +- Regular join + ## supported sources Review Comment: sources -> Extract Node ########## docs/modules/sort/quick_start.md: ########## @@ -20,9 +20,7 @@ Now you can submit job to flink with the jar compiled, refer to [how to submit j Example： ``` ./bin/flink run -c org.apache.inlong.sort.singletenant.flink.Entrance inlong-sort/sort-[version].jar \ Review Comment: org.apache.inlong.sort.singletenant.flink.Entrance -> org.apache.inlong.sort.flink.Entrance singletenant have no meaning start from this version now, we should rename the entrance class name. ########## docs/modules/sort/overview.md: ########## @@ -9,24 +9,34 @@ Inlong-sort is simply an Flink application, and relys on Inlong-manager to manag # features +## supported transforms +- String split index +- String regular replace +- String regular replace the first value +- Data filter +- Data distinct +- Regular join + ## supported sources -- pulsar +- Pulsar +- MySQL +- Kafka ## supported storages -- hive（Currently we support parquet, orc and text file format） -- kafka -- clickhouse -- iceberg +- Hive +- Kafka +- ClickHouse +- Iceberg ## limitations Currently, we just support extracting specified fields in the stage of **Transform**. ## future plans ### More kinds of source systems Review Comment: source systems -> Extract Node ########## docs/modules/sort/quick_start.md: ########## @@ -32,35 +30,17 @@ Notice： - `inlong-sort/sort-[version].jar` is the compiled jar ## Necessary configurations -- `--cluster-id ` represent a specified inlong-sort application, same as the configuration of `sort.appName` in inlong-manager -- `--dataflow.info.file` dataflow configuration file path -- `--source.type` source of the application, currently "pulsar" is supported -- `--sink.type` sink of the application, currently "clickhouse", "hive", "iceberg", "kafka" are supported -- `--metrics.audit.proxy.hosts` audit proxy host address for reporting audit metrics +- `--group.info.file` dataflow configuration file path +- `--lightweight` whether to use lightweight sort,default false **Example** ``` ---cluster-id debezium2kafka-canal --dataflow.info.file /YOUR_DATAFLOW_INFO_DIR/debezium-to-kafka-canal.json \ ---source.type pulsar --sink.type kafka +--lightweight true --group.info.file /YOUR_DATAFLOW_INFO_DIR/mysql-to-kafka.json Review Comment: ditto for `--lightweight` ########## docs/modules/sort/quick_start.md: ########## @@ -32,35 +30,17 @@ Notice： - `inlong-sort/sort-[version].jar` is the compiled jar ## Necessary configurations -- `--cluster-id ` represent a specified inlong-sort application, same as the configuration of `sort.appName` in inlong-manager -- `--dataflow.info.file` dataflow configuration file path -- `--source.type` source of the application, currently "pulsar" is supported -- `--sink.type` sink of the application, currently "clickhouse", "hive", "iceberg", "kafka" are supported -- `--metrics.audit.proxy.hosts` audit proxy host address for reporting audit metrics +- `--group.info.file` dataflow configuration file path +- `--lightweight` whether to use lightweight sort,default false **Example** ``` ---cluster-id debezium2kafka-canal --dataflow.info.file /YOUR_DATAFLOW_INFO_DIR/debezium-to-kafka-canal.json \ ---source.type pulsar --sink.type kafka +--lightweight true --group.info.file /YOUR_DATAFLOW_INFO_DIR/mysql-to-kafka.json Review Comment: ditto for `--lightweight` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
