dockerzhang commented on code in PR #941: URL: https://github.com/apache/inlong-website/pull/941#discussion_r1575549779
########## docs/design_and_concept/how_to_write_plugin_agent.md: ########## @@ -2,218 +2,156 @@ title: Agent Plugin sidebar_position: 3 --- +## Summary +In Standard Architecture, we can collect various types of data sources through the InLong Agent. The InLong Agent supports the extension of new collection types through plugins. This article will guide developers on how to customize the new Agent collection data source plugin. + +## Core concepts +### Task and Instance +Task and Instance are the two core concepts of Agent. Simple understanding: Task corresponds to a collection task configured on the management platform, while Instance is a specific collection instance generated by Task. For example, there is a collection task configuration on the management platform: `127.0.0.1 -> /data/log/YYMMDDhh.log._[0-9]+`, which means that the user needs to access the machine `127.0.0.1` collect data that conforms to the path rule `/data/log/YYMMDDhh.log._[0-9]+`. **This is a Task**. This Task will search for files that meet the conditions based on this path rule, and generate a corresponding Instance for each file that meets the conditions, for example, `/data/log/2024040221.log.0, /data/log /2024040221.log.1, /data/log/2024040221.log.3` 3 files, then the Task will generate 3 Instances to collect data from these three files respectively. + + +### Source and Sink +Source and Sink are lower-level concepts of Instance. They can be simply understood as each Instance has a Source and a Sink. As the name suggests, Source is used to read data from the data source; Sink is used to write data to the target storage. + +## Development process (taking Pulsar as an example) +### Main process Review Comment: ```suggestion ### Process ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
