raminqaf commented on a change in pull request #114: URL: https://github.com/apache/bahir-flink/pull/114#discussion_r593945598
########## File path: flink-connector-influxdb2/README.md ########## @@ -0,0 +1,202 @@ +# Flink InfluxDB Connector + +This connector provides a Source that parses the [InfluxDB Line Protocol](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/) and a Sink that can write to [InfluxDB](https://www.influxdata.com/). The Source implements the unified [Data Source API](https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/sources.html). Our sink implements the unified [Sink API](https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API#FLIP143:UnifiedSinkAPI-SinkAPI). + +## Installation + +To use this connector, add the following dependency to your project: + +```xml= +<dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>flink-connector-influxdb2_2.12</artifactId> + <version>1.1-SNAPSHOT</version> +</dependency> +``` + +Note that the streaming connectors are not part of the binary distribution of Flink. You need to link them into your job jar for cluster execution. See how to link with them for cluster execution [here](https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/project-configuration.html#adding-connector-and-library-dependencies). + +## Compatibility + +This module is compatible with InfluxDB 2.x and InfluxDB 1.8+. See more information [here](https://github.com/influxdata/influxdb-client-java#influxdb-client-java). + +## Source + +The Source accepts data in the form of the [Line Protocol](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/). One HTTP server starts per SplitReader that parses HTTP requests to our Data Point class. That Data Point instance is deserialized by a user-provided implementation of our InfluxDBDataPointDeserializer and send to the next Flink operator. + +When using Telegraf, you have two choices to configure it. You can either configure its [InfluxDB v2 output plugin](https://docs.influxdata.com/telegraf/v1.17/plugins/#influxdb_v2) for writing to the running HTTP servers or use its [HTTP output plugin](https://docs.influxdata.com/telegraf/v1.17/plugins/#http) for that: + +```toml +[[outputs.influxdb_v2]] + urls = ["http://task-manager-1:8000";, "http:/task-manager-2:8000"] + +# or + +[[outputs.http]] + url = "http://task-manager-1:8000/api/v2/write"; + method = "POST" + data_format = "influx" +``` + + + +### Usage + +```java= +InfluxDBSource<Long> influxDBSource = InfluxBSource.<Long>builder() + .setDeserializer(new TestDeserializer()) + .build() + +// ... + +/** + * Implementation of InfluxDBDataPointDeserializer interface + * (dataPoint) -----> (element) + * test,longValue=1 fieldKey="fieldValue" -----------> 1L + * test,longValue=2 fieldKey="fieldValue" -----------> 2L + * test,longValue=3 fieldKey="fieldValue" -----------> 3L + */ +class TestDeserializer implements InfluxDBDataPointDeserializer<Long> { + @Override + public Long deserialize(final DataPoint dataPoint) { + return (Long) dataPoint.getField("longValue"); + } +} +``` + + +### Options + +| Option | Description | Default Value | +| ----------------- |-----------------|:-----------------:| +| ENQUEUE_WAIT_TIME | The time out in seconds for enqueuing an HTTP request to the queue. | 5 | +| INGEST_QUEUE_CAPACITY | Size of queue that buffers HTTP requests data points before fetching. | 1000 | +| MAXIMUM_LINES_PER_REQUEST | The maximum number of lines that should be parsed per HTTP request. | 10000 | +| PORT | TCP port on which the split reader's HTTP server is running on. | 8000 | + +### Supported Data Types in Field Set + +| Field Set | Support | +| ------------- |:-------------:| +| Float | ✅ | +| Integer | ✅ | +| UInteger | ❌ | Review comment: Here the issue: https://github.com/apache/druid/issues/10993 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
