[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424543=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424543 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 18/Apr/20 01:11 Start Date: 18/Apr/20 01:11 Worklog Time Spent: 10m Work Description: bipinupd commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424543) Time Spent: 18.5h (was: 18h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 18.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424538=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424538 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 18/Apr/20 00:44 Start Date: 18/Apr/20 00:44 Worklog Time Spent: 10m Work Description: bipinupd commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615523726 Thanks @iemejia ... I will try if did not work ... I will follow you second advice and tag you and mwalenia. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424538) Time Spent: 18h 20m (was: 18h 10m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 18h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424462=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424462 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 22:21 Start Date: 17/Apr/20 22:21 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615489475 Oups I accidentally closed it :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424462) Time Spent: 18h 10m (was: 18h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 18h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424461=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424461 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 22:21 Start Date: 17/Apr/20 22:21 Worklog Time Spent: 10m Work Description: bipinupd commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @iemejia emejia`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[![Build
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424459=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424459 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 22:21 Start Date: 17/Apr/20 22:21 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615489412 I don't understand what happened but it looks really broken. One alternative would be to run `git pull origin master --rebase` to move all your commits on top of the latest master, this will fix the issue but you have to deal with the rebase that does not look easy, the other is that you recreate the branch/PR from scratch based on the latest master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424459) Time Spent: 17h 40m (was: 17.5h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 17h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424460=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424460 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 22:21 Start Date: 17/Apr/20 22:21 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424460) Time Spent: 17h 50m (was: 17h 40m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 17h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424435 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 21:30 Start Date: 17/Apr/20 21:30 Worklog Time Spent: 10m Work Description: bipinupd commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615472538 > retest this please HI @iemejia Thanks. I have retested. The tests are passing For some reason while rebasing is pulling all these changes ... When I check the diff from my branch (https://github.com/bipinupd/beam/tree/BEAM-2546) where apache:master, I still see only my changes. I will appreciate any suggestion fix this.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424435) Time Spent: 17.5h (was: 17h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 17.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424245=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424245 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 14:59 Start Date: 17/Apr/20 14:59 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615292356 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424245) Time Spent: 17h 20m (was: 17h 10m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 17h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424244=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424244 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 14:59 Start Date: 17/Apr/20 14:59 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615293438 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424244) Time Spent: 17h 10m (was: 17h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 17h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424242 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 14:58 Start Date: 17/Apr/20 14:58 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615292356 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424242) Time Spent: 16h 50m (was: 16h 40m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 16h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424241=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424241 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 14:58 Start Date: 17/Apr/20 14:58 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615292249 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424241) Time Spent: 16h 40m (was: 16.5h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 16h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424243=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424243 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 14:58 Start Date: 17/Apr/20 14:58 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615292249 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424243) Time Spent: 17h (was: 16h 50m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 17h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=424073=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424073 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 17/Apr/20 09:42 Start Date: 17/Apr/20 09:42 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-615150628 It seems some files are missing the Apache license headers, you can fix that by running `./gradlew :sdks:java:io:influxdb:check spotlessApply`. Also it seems the state of the PR is a bit crazy and needs some rebasing, you can do that with `git pull origin master --rebase`. Can you please do both and verify that tests pass locally and ping me just when ready. I will be glad to do the review then. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 424073) Time Spent: 16.5h (was: 16h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 16.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=422355=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-422355 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 14/Apr/20 21:40 Start Date: 14/Apr/20 21:40 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-613695575 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 422355) Time Spent: 16h 20m (was: 16h 10m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 16h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=420987=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420987 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 12/Apr/20 19:26 Start Date: 12/Apr/20 19:26 Worklog Time Spent: 10m Work Description: bipinupd commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-612664265 @iemejia and @mwalenia Thanks for reviewing and providing suggestion and helpful comments. It's my first contribution to Beam project. Looking forward for your guidance and support. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 420987) Time Spent: 16h 10m (was: 16h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 16h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=420986=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420986
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 12/Apr/20 19:21
Start Date: 12/Apr/20 19:21
Worklog Time Spent: 10m
Work Description: bipinupd commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r407241372
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=420985=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420985
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 12/Apr/20 19:19
Start Date: 12/Apr/20 19:19
Worklog Time Spent: 10m
Work Description: bipinupd commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r407241190
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=420984=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420984
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 12/Apr/20 19:16
Start Date: 12/Apr/20 19:16
Worklog Time Spent: 10m
Work Description: bipinupd commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r407240851
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=420982=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420982 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 12/Apr/20 19:13 Start Date: 12/Apr/20 19:13 Worklog Time Spent: 10m Work Description: bipinupd commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#discussion_r407240538 ## File path: sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/package-info.java ## @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Transforms for reading and writing from/to InfluxDB. + * + * @see org.apache.beam.sdk.io.influxdb.InfluxDBIO + */ +package org.apache.beam.sdk.io.influxdb; Review comment: I have added in the tag in InfluxDBIO class. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 420982) Time Spent: 15.5h (was: 15h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 15.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417208=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417208
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404380343
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417168=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417168 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 06/Apr/20 22:05 Start Date: 06/Apr/20 22:05 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#discussion_r404226273 ## File path: sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java ## @@ -0,0 +1,709 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.beam.sdk.io.influxdb; + +import static org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument; + +import com.google.auto.value.AutoValue; +import java.io.Serializable; +import java.security.cert.CertificateException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Iterator; +import java.util.List; +import java.util.NoSuchElementException; +import javax.annotation.Nullable; +import javax.net.ssl.HostnameVerifier; +import javax.net.ssl.SSLContext; +import javax.net.ssl.SSLSession; +import javax.net.ssl.SSLSocketFactory; +import javax.net.ssl.TrustManager; +import javax.net.ssl.X509TrustManager; +import okhttp3.OkHttpClient; +import org.apache.beam.sdk.annotations.Experimental; +import org.apache.beam.sdk.coders.Coder; +import org.apache.beam.sdk.coders.SerializableCoder; +import org.apache.beam.sdk.io.BoundedSource; +import org.apache.beam.sdk.options.PipelineOptions; +import org.apache.beam.sdk.options.ValueProvider; +import org.apache.beam.sdk.transforms.DoFn; +import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.ParDo; +import org.apache.beam.sdk.transforms.SerializableFunction; +import org.apache.beam.sdk.transforms.display.DisplayData; +import org.apache.beam.sdk.transforms.display.HasDisplayData; +import org.apache.beam.sdk.values.PBegin; +import org.apache.beam.sdk.values.PCollection; +import org.apache.beam.sdk.values.PDone; +import org.influxdb.BatchOptions; +import org.influxdb.InfluxDB; +import org.influxdb.InfluxDBFactory; +import org.influxdb.dto.Query; +import org.influxdb.dto.QueryResult; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * IO to read and write to InfluxDB. + * + * Reading from InfluxDB datasource Review comment: remove `datasource` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 417168) Time Spent: 10h 10m (was: 10h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 10h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417180=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417180
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404227410
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
Review comment:
s/source/{@link #read()}
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417180)
Time Spent: 11h 40m (was: 11.5h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Time Spent: 11h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417173=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417173
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404360427
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417194=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417194
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404376980
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417204=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417204
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404410842
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417187=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417187
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404356103
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417177=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417177
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404374894
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417182=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417182
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404388021
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417169=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417169
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404229145
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
Review comment:
Jdbc :) (check this in other parts too)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417169)
Time Spent: 10h 20m (was: 10h 10m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Time Spent: 10h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417164=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417164 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 06/Apr/20 22:05 Start Date: 06/Apr/20 22:05 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#discussion_r404212913 ## File path: sdks/java/io/influxdb/Readme.md ## @@ -0,0 +1,6 @@ +To test the influxdb io plugin. Run the influxDB container. Review comment: Remove this file and move its contents into the `InfluxDBIOIT.java` javadoc This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 417164) Time Spent: 9h 40m (was: 9.5h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 9h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417202=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417202
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404388505
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417213=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417213
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404382511
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417210=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417210
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404376801
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417185=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417185
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404384086
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
Review comment:
s/Stringn/String
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417185)
Time Spent: 12h 10m (was: 12h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Time Spent: 12h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417184=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417184
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404390537
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417191=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417191
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404384582
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417186=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417186
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404376936
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417171=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417171
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404375365
##
File path:
sdks/java/io/influxdb/src/test/java/org/apache/beam/sdk/io/influxdb/Model.java
##
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import java.io.Serializable;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+import org.influxdb.dto.Point;
+
+class Model implements LineProtocolConvertable, Serializable {
+ private String measurement;
+ private Map tags;
+ private Map fields;
+ private Long time;
+ private TimeUnit timeUnit;
+
+ Model() {
+tags = new HashMap();
Review comment:
`new HashMap<>();` (same in the line below)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417171)
Time Spent: 10h 40m (was: 10.5h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Time Spent: 10h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417209=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417209
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404385733
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417166=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417166
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404220372
##
File path:
sdks/java/io/influxdb/src/test/java/org/apache/beam/sdk/io/influxdb/InfluxDBIOIT.java
##
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import java.util.Arrays;
+import org.apache.beam.sdk.PipelineResult;
+import org.apache.beam.sdk.io.common.IOTestPipelineOptions;
+import org.apache.beam.sdk.options.Default;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.testing.PAssert;
+import org.apache.beam.sdk.testing.TestPipeline;
+import org.apache.beam.sdk.transforms.Count;
+import org.apache.beam.sdk.transforms.Create;
+import org.apache.beam.sdk.values.PCollection;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.JUnit4;
+
+/**
+ * A test of {@link org.apache.beam.sdk.io.influxdb.InfluxDBIO} on an
independent InfluxDB instance.
+ *
+ * This test requires a running instance of InfluxDB. Pass in connection
information using
+ * PipelineOptions:
+ *
+ *
+ * ./gradlew integrationTest -p sdks/java/io/influxdb
-DintegrationTestPipelineOptions='[
+ * "--influxdburl=http://localhost:8086;,
+ * "--infuxDBDatabase=mypass",
+ * "--username=username"
+ * "--password=password"]'
+ * --tests org.apache.beam.sdk.io.influxdb.InfluxDBIOIT
+ * -DintegrationTestRunner=direct
+ *
+ */
+@RunWith(JUnit4.class)
+public class InfluxDBIOIT {
+
+ private static InfluxDBPipelineOptions options;
+
+ @Rule public final TestPipeline writePipeline = TestPipeline.create();
+ @Rule public final TestPipeline readPipeline = TestPipeline.create();
+
+ /** InfluxDBIO options. */
+ public interface InfluxDBPipelineOptions extends IOTestPipelineOptions {
+@Description("InfluxDB host (host name/ip address)")
+@Default.String("http://localhost:8086;)
+String getInfluxDBURL();
+
+void setInfluxDBURL(String value);
+
+@Description("Username for InfluxDB")
+@Default.String("superadmin")
+String getInfluxDBUserName();
+
+void setInfluxDBUserName(String value);
+
+@Description("Password for InfluxDB")
+@Default.String("supersecretpassword")
+String getInfluxDBPassword();
+
+void setInfluxDBPassword(String value);
+
+@Description("InfluxDB database name")
+@Default.String("db0")
+String getDatabaseName();
+
+void setDatabaseName(String value);
+ }
+
+ @BeforeClass
+ public static void setUp() {
+PipelineOptionsFactory.register(InfluxDBPipelineOptions.class);
+options =
TestPipeline.testingPipelineOptions().as(InfluxDBPipelineOptions.class);
+ }
+
+ @After
+ public void clear() {
+try (InfluxDB connection =
+InfluxDBFactory.connect(
+options.getInfluxDBURL(),
+options.getInfluxDBUserName(),
+options.getInfluxDBPassword())) {
+ connection.query(new Query("DROP DATABASE \"" +
options.getDatabaseName() + "\""));
+}
+ }
+
+ @Before
+ public void initTest() {
+try (InfluxDB connection =
+InfluxDBFactory.connect(
+options.getInfluxDBURL(),
+options.getInfluxDBUserName(),
+options.getInfluxDBPassword())) {
+ connection.query(new Query("CREATE DATABASE \"" +
options.getDatabaseName() + "\""));
+}
+ }
+
+ @Test
+ public void testWriteAndRead() {
+final int noofElementsToReadAndWrite = 1000;
+writePipeline
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417172=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417172 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 06/Apr/20 22:05 Start Date: 06/Apr/20 22:05 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#discussion_r404221870 ## File path: settings.gradle ## @@ -118,6 +119,7 @@ include ":sdks:java:io:thrift" include ":sdks:java:io:tika" include ":sdks:java:io:xml" include ":sdks:java:io:synthetic" +include ":sdks:java:io:influxdb" Review comment: remove this second ocurrence This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 417172) Time Spent: 10h 50m (was: 10h 40m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 10h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417163=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417163 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 06/Apr/20 22:05 Start Date: 06/Apr/20 22:05 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#discussion_r404218067 ## File path: sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/package-info.java ## @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Transforms for reading and writing from/to InfluxDB. + * + * @see org.apache.beam.sdk.io.influxdb.InfluxDBIO + */ +package org.apache.beam.sdk.io.influxdb; Review comment: Please add `@Experimental(Kind.SOURCE_SINK)` annotation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 417163) Time Spent: 9.5h (was: 9h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417181=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417181
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404381249
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417198=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417198
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404402426
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417183=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417183
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404376571
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417192=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417192
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404400145
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417212=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417212
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404388315
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417167=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417167
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404223761
##
File path: sdks/java/io/influxdb/build.gradle
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+plugins { id 'org.apache.beam.module' }
+applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.influxdb')
+provideIntegrationTestingDependencies()
+enableJavaPerformanceTesting()
+
+description = "Apache Beam :: SDKs :: Java :: IO :: InfluxDB"
+ext.summary = "IO to read and write on JDBC datasource."
+
+dependencies {
+ compile library.java.vendored_guava_26_0_jre
+ compile project(path: ":sdks:java:core", configuration: "shadow")
+ compile library.java.slf4j_api
+ compile group: 'org.influxdb', name: 'influxdb-java', version: '2.15'
Review comment:
Prefer the inlined version `compile
"org.influxdb:influxdb-java:$influxdb_version"`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417167)
Time Spent: 10h (was: 9h 50m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Time Spent: 10h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417193=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417193
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404396609
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417199=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417199
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404378445
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417197=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417197
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404387662
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417195=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417195
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404378613
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417189
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404382238
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417178=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417178
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404380078
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417203
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404399019
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417207=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417207
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404410295
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417206=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417206
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404404249
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417205
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404406106
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417188=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417188
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404385611
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417170=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417170
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404225264
##
File path: sdks/java/io/influxdb/build.gradle
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+plugins { id 'org.apache.beam.module' }
+applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.influxdb')
+provideIntegrationTestingDependencies()
+enableJavaPerformanceTesting()
+
+description = "Apache Beam :: SDKs :: Java :: IO :: InfluxDB"
+ext.summary = "IO to read and write on JDBC datasource."
+
Review comment:
Add def influxdb_version="2.15" (maybe 2.17 better now).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417170)
Time Spent: 10.5h (was: 10h 20m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Time Spent: 10.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417196=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417196
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404385490
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417162=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417162
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404218978
##
File path:
sdks/java/io/influxdb/src/test/java/org/apache/beam/sdk/io/influxdb/InfluxDBIOIT.java
##
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import java.util.Arrays;
+import org.apache.beam.sdk.PipelineResult;
+import org.apache.beam.sdk.io.common.IOTestPipelineOptions;
+import org.apache.beam.sdk.options.Default;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.testing.PAssert;
+import org.apache.beam.sdk.testing.TestPipeline;
+import org.apache.beam.sdk.transforms.Count;
+import org.apache.beam.sdk.transforms.Create;
+import org.apache.beam.sdk.values.PCollection;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.JUnit4;
+
+/**
+ * A test of {@link org.apache.beam.sdk.io.influxdb.InfluxDBIO} on an
independent InfluxDB instance.
+ *
+ * This test requires a running instance of InfluxDB. Pass in connection
information using
+ * PipelineOptions:
+ *
+ *
+ * ./gradlew integrationTest -p sdks/java/io/influxdb
-DintegrationTestPipelineOptions='[
+ * "--influxdburl=http://localhost:8086;,
+ * "--infuxDBDatabase=mypass",
+ * "--username=username"
+ * "--password=password"]'
+ * --tests org.apache.beam.sdk.io.influxdb.InfluxDBIOIT
+ * -DintegrationTestRunner=direct
+ *
+ */
+@RunWith(JUnit4.class)
+public class InfluxDBIOIT {
+
+ private static InfluxDBPipelineOptions options;
+
+ @Rule public final TestPipeline writePipeline = TestPipeline.create();
+ @Rule public final TestPipeline readPipeline = TestPipeline.create();
+
+ /** InfluxDBIO options. */
+ public interface InfluxDBPipelineOptions extends IOTestPipelineOptions {
+@Description("InfluxDB host (host name/ip address)")
+@Default.String("http://localhost:8086;)
+String getInfluxDBURL();
+
+void setInfluxDBURL(String value);
+
+@Description("Username for InfluxDB")
+@Default.String("superadmin")
+String getInfluxDBUserName();
+
+void setInfluxDBUserName(String value);
+
+@Description("Password for InfluxDB")
+@Default.String("supersecretpassword")
+String getInfluxDBPassword();
+
+void setInfluxDBPassword(String value);
+
+@Description("InfluxDB database name")
+@Default.String("db0")
+String getDatabaseName();
+
+void setDatabaseName(String value);
+ }
+
+ @BeforeClass
+ public static void setUp() {
+PipelineOptionsFactory.register(InfluxDBPipelineOptions.class);
+options =
TestPipeline.testingPipelineOptions().as(InfluxDBPipelineOptions.class);
+ }
+
+ @After
+ public void clear() {
+try (InfluxDB connection =
+InfluxDBFactory.connect(
+options.getInfluxDBURL(),
+options.getInfluxDBUserName(),
+options.getInfluxDBPassword())) {
+ connection.query(new Query("DROP DATABASE \"" +
options.getDatabaseName() + "\""));
+}
+ }
+
+ @Before
+ public void initTest() {
+try (InfluxDB connection =
+InfluxDBFactory.connect(
+options.getInfluxDBURL(),
+options.getInfluxDBUserName(),
+options.getInfluxDBPassword())) {
+ connection.query(new Query("CREATE DATABASE \"" +
options.getDatabaseName() + "\""));
+}
+ }
+
+ @Test
+ public void testWriteAndRead() {
+final int noofElementsToReadAndWrite = 1000;
+writePipeline
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417165=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417165
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404220859
##
File path:
sdks/java/io/influxdb/src/test/java/org/apache/beam/sdk/io/influxdb/InfluxDBIOIT.java
##
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import java.util.Arrays;
+import org.apache.beam.sdk.PipelineResult;
+import org.apache.beam.sdk.io.common.IOTestPipelineOptions;
+import org.apache.beam.sdk.options.Default;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.testing.PAssert;
+import org.apache.beam.sdk.testing.TestPipeline;
+import org.apache.beam.sdk.transforms.Count;
+import org.apache.beam.sdk.transforms.Create;
+import org.apache.beam.sdk.values.PCollection;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.JUnit4;
+
+/**
+ * A test of {@link org.apache.beam.sdk.io.influxdb.InfluxDBIO} on an
independent InfluxDB instance.
+ *
+ * This test requires a running instance of InfluxDB. Pass in connection
information using
+ * PipelineOptions:
+ *
+ *
+ * ./gradlew integrationTest -p sdks/java/io/influxdb
-DintegrationTestPipelineOptions='[
+ * "--influxdburl=http://localhost:8086;,
+ * "--infuxDBDatabase=mypass",
+ * "--username=username"
+ * "--password=password"]'
+ * --tests org.apache.beam.sdk.io.influxdb.InfluxDBIOIT
+ * -DintegrationTestRunner=direct
+ *
+ */
+@RunWith(JUnit4.class)
+public class InfluxDBIOIT {
+
+ private static InfluxDBPipelineOptions options;
+
+ @Rule public final TestPipeline writePipeline = TestPipeline.create();
+ @Rule public final TestPipeline readPipeline = TestPipeline.create();
+
+ /** InfluxDBIO options. */
+ public interface InfluxDBPipelineOptions extends IOTestPipelineOptions {
+@Description("InfluxDB host (host name/ip address)")
+@Default.String("http://localhost:8086;)
+String getInfluxDBURL();
+
+void setInfluxDBURL(String value);
+
+@Description("Username for InfluxDB")
+@Default.String("superadmin")
+String getInfluxDBUserName();
+
+void setInfluxDBUserName(String value);
+
+@Description("Password for InfluxDB")
+@Default.String("supersecretpassword")
+String getInfluxDBPassword();
+
+void setInfluxDBPassword(String value);
+
+@Description("InfluxDB database name")
+@Default.String("db0")
+String getDatabaseName();
+
+void setDatabaseName(String value);
+ }
+
+ @BeforeClass
+ public static void setUp() {
+PipelineOptionsFactory.register(InfluxDBPipelineOptions.class);
+options =
TestPipeline.testingPipelineOptions().as(InfluxDBPipelineOptions.class);
+ }
+
+ @After
+ public void clear() {
+try (InfluxDB connection =
+InfluxDBFactory.connect(
+options.getInfluxDBURL(),
+options.getInfluxDBUserName(),
+options.getInfluxDBPassword())) {
+ connection.query(new Query("DROP DATABASE \"" +
options.getDatabaseName() + "\""));
+}
+ }
+
+ @Before
+ public void initTest() {
+try (InfluxDB connection =
+InfluxDBFactory.connect(
+options.getInfluxDBURL(),
+options.getInfluxDBUserName(),
+options.getInfluxDBPassword())) {
+ connection.query(new Query("CREATE DATABASE \"" +
options.getDatabaseName() + "\""));
+}
+ }
+
+ @Test
+ public void testWriteAndRead() {
+final int noofElementsToReadAndWrite = 1000;
+writePipeline
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417174=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417174
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404232253
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
Review comment:
Remove ` datasource`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417175=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417175
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404227752
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
Review comment:
remove `To configure the InfluxDB source, `
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417175)
Time Spent: 11h 10m (was: 11h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Time Spent: 11h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417190=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417190
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404385677
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417201=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417201
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404230813
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
Review comment:
missing space and . (see also below)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417201)
Time Spent: 14h (was: 13h 50m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417179=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417179
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404231393
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
Review comment:
Read with query example
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417179)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417176=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417176
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404232037
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
Review comment:
Uppercase the query
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 417176)
Time Spent: 11h 20m (was: 11h 10m)
> Create
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417211=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417211
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404390643
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=417200=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417200
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 06/Apr/20 22:05
Start Date: 06/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r404377013
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,709 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.influxdb;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+import com.google.auto.value.AutoValue;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import javax.annotation.Nullable;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.SSLSocketFactory;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.transforms.SerializableFunction;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * IO to read and write to InfluxDB.
+ *
+ * Reading from InfluxDB datasource
+ *
+ * InfluxDBIO source returns a bounded collection of {@code String} as a
{@code
+ * PCollection}.
+ *
+ * To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+ *
+ * {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+ * Optionally, {@link DataSourceConfiguration#withUsername(String)} and {@link
+ * DataSourceConfiguration#withPassword(String)} allows you to define username
and password.
+ *
+ * For example:
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * For example (Read from query):
+ *
+ * {@code
+ * PCollection collection = pipeline.apply(InfluxDBIO.read()
+ * .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+ * "https://localhost:8086","username","password;))
+ * .withDatabase("metrics")
+ * .withQuery("Select * from cpu")
+ * .withRetentionPolicy("autogen")
+ * .withSslInvalidHostNameAllowed(true)
+ * withSslEnabled(true));
+ * }
+ *
+ * Writing to Influx datasource
+ *
+ * InfluxDB sink supports writing records into a database. It writes a
{@link PCollection} to the
+ * database by converting each T. The T should implement getLineProtocol()
from {@link
+ * LineProtocolConvertable}.
+ *
+ * Like the source, to configure the sink, you have to provide a {@link
DataSourceConfiguration}.
+ *
+ *
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=409831=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409831
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 25/Mar/20 21:19
Start Date: 25/Mar/20 21:19
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r397962741
##
File path: .test-infra/jenkins/job_PerformanceTests_InfluxDBIO_IT.groovy
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import CommonJobProperties as common
+import Kubernetes
+
+String jobName = "beam_PerformanceTests_InfluxDBIO_IT"
+
+job(jobName) {
+ common.setTopLevelMainJobProperties(delegate)
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java InfluxDBIO Performance Test',
+ 'Run Java InfluxDBIO Performance Test')
+
+ String namespace = common.getKubernetesNamespace(jobName)
+ String kubeconfigPath = common.getKubeconfigLocationForNamespace(namespace)
+ Kubernetes k8s = Kubernetes.create(delegate, kubeconfigPath, namespace)
+
+
k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/influxdb/influxdb.yml"))
+ String influxDBHostName = "LOAD_BALANCER_IP"
+ k8s.loadBalancerIP("influxdb-load-balancer-service", influxDBHostName)
+ Map pipelineOptions = [
+ influxDBURL : "http://\$${influxDBHostName}:8086;,
+ influxDBUserName : "superadmin",
+ influxDBPassword : "supersecretpassword",
+ databaseName : "db1"
+ ]
+
+ steps {
+gradle {
+ rootBuildScriptDir(common.checkoutDir)
+ common.setGradleSwitches(delegate)
+ switches("--info")
+
switches("-DintegrationTestPipelineOptions=\'${common.joinPipelineOptions(pipelineOptions)}\'")
+ switches("-DintegrationTestRunner=direct")
Review comment:
After looking a bit into the code we have a weird mix of Dataflow/Direct in
the runs, @mwalenia do you think there is a way we can do that 'choosable' when
running an IT test? (note that I agree with the idea of running on Dataflow,
but of course I understand contributors may not have access to it and it is
quicker to develop/do it with DirectRunner).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409831)
Time Spent: 9h 20m (was: 9h 10m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=409603=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409603
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 25/Mar/20 15:49
Start Date: 25/Mar/20 15:49
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r397962741
##
File path: .test-infra/jenkins/job_PerformanceTests_InfluxDBIO_IT.groovy
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import CommonJobProperties as common
+import Kubernetes
+
+String jobName = "beam_PerformanceTests_InfluxDBIO_IT"
+
+job(jobName) {
+ common.setTopLevelMainJobProperties(delegate)
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java InfluxDBIO Performance Test',
+ 'Run Java InfluxDBIO Performance Test')
+
+ String namespace = common.getKubernetesNamespace(jobName)
+ String kubeconfigPath = common.getKubeconfigLocationForNamespace(namespace)
+ Kubernetes k8s = Kubernetes.create(delegate, kubeconfigPath, namespace)
+
+
k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/influxdb/influxdb.yml"))
+ String influxDBHostName = "LOAD_BALANCER_IP"
+ k8s.loadBalancerIP("influxdb-load-balancer-service", influxDBHostName)
+ Map pipelineOptions = [
+ influxDBURL : "http://\$${influxDBHostName}:8086;,
+ influxDBUserName : "superadmin",
+ influxDBPassword : "supersecretpassword",
+ databaseName : "db1"
+ ]
+
+ steps {
+gradle {
+ rootBuildScriptDir(common.checkoutDir)
+ common.setGradleSwitches(delegate)
+ switches("--info")
+
switches("-DintegrationTestPipelineOptions=\'${common.joinPipelineOptions(pipelineOptions)}\'")
+ switches("-DintegrationTestRunner=direct")
Review comment:
After looking a bit into the code we have a weird mix of Dataflow/Direct in
the runs, do you think there is a way we can do that 'choosable' when running
an IT test? (note that I agree with the idea of running on Dataflow, but of
course I understand contributors may not have access to it and it is quicker to
develop/do it with DirectRunner).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409603)
Time Spent: 9h 10m (was: 9h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=409431=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409431
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 25/Mar/20 10:47
Start Date: 25/Mar/20 10:47
Worklog Time Spent: 10m
Work Description: mwalenia commented on pull request #11028: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/11028#discussion_r397759342
##
File path: .test-infra/jenkins/job_PerformanceTests_InfluxDBIO_IT.groovy
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import CommonJobProperties as common
+import Kubernetes
+
+String jobName = "beam_PerformanceTests_InfluxDBIO_IT"
+
+job(jobName) {
+ common.setTopLevelMainJobProperties(delegate)
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java InfluxDBIO Performance Test',
+ 'Run Java InfluxDBIO Performance Test')
+
+ String namespace = common.getKubernetesNamespace(jobName)
+ String kubeconfigPath = common.getKubeconfigLocationForNamespace(namespace)
+ Kubernetes k8s = Kubernetes.create(delegate, kubeconfigPath, namespace)
+
+
k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/influxdb/influxdb.yml"))
+ String influxDBHostName = "LOAD_BALANCER_IP"
+ k8s.loadBalancerIP("influxdb-load-balancer-service", influxDBHostName)
+ Map pipelineOptions = [
+ influxDBURL : "http://\$${influxDBHostName}:8086;,
+ influxDBUserName : "superadmin",
+ influxDBPassword : "supersecretpassword",
+ databaseName : "db1"
+ ]
+
+ steps {
+gradle {
+ rootBuildScriptDir(common.checkoutDir)
+ common.setGradleSwitches(delegate)
+ switches("--info")
+
switches("-DintegrationTestPipelineOptions=\'${common.joinPipelineOptions(pipelineOptions)}\'")
+ switches("-DintegrationTestRunner=direct")
Review comment:
I think this should be executed on Dataflow so that we can test integration
of a real runner with a real service.
Remember about the appropriate options relevant to the Dataflow runner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409431)
Time Spent: 9h (was: 8h 50m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Bipin Upadhyaya
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 9h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=409411=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409411 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 25/Mar/20 10:00 Start Date: 25/Mar/20 10:00 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-603750036 Sorry for the delay and thanks for your patience @bipinupd , my review queue at least is finally advancing. I added @mwalenia so he can help me verfify that the CI / IT part is consistent with Beam common patterns and it works. I will give a round to the code and comment later on today. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 409411) Time Spent: 8h 50m (was: 8h 40m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Fix For: 2.21.0 > > Time Spent: 8h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=398319=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398319 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 05/Mar/20 11:17 Start Date: 05/Mar/20 11:17 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-595176235 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 398319) Time Spent: 8h 20m (was: 8h 10m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Fix For: 2.21.0 > > Time Spent: 8h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=398320=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398320 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 05/Mar/20 11:17 Start Date: 05/Mar/20 11:17 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-595176325 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 398320) Time Spent: 8.5h (was: 8h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Fix For: 2.21.0 > > Time Spent: 8.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=398316=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398316 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 05/Mar/20 11:17 Start Date: 05/Mar/20 11:17 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-595176235 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 398316) Time Spent: 7h 50m (was: 7h 40m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Fix For: 2.21.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=398321=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398321 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 05/Mar/20 11:17 Start Date: 05/Mar/20 11:17 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-595176377 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 398321) Time Spent: 8h 40m (was: 8.5h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Fix For: 2.21.0 > > Time Spent: 8h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=398317=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398317 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 05/Mar/20 11:17 Start Date: 05/Mar/20 11:17 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-595176325 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 398317) Time Spent: 8h (was: 7h 50m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Fix For: 2.21.0 > > Time Spent: 8h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=398318=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398318 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 05/Mar/20 11:17 Start Date: 05/Mar/20 11:17 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-595176377 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 398318) Time Spent: 8h 10m (was: 8h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Bipin Upadhyaya >Priority: Major > Fix For: 2.21.0 > > Time Spent: 8h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=397069=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397069 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 03/Mar/20 22:13 Start Date: 03/Mar/20 22:13 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-594197974 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 397069) Time Spent: 7.5h (was: 7h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 7.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=397070=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397070 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 03/Mar/20 22:13 Start Date: 03/Mar/20 22:13 Worklog Time Spent: 10m Work Description: iemejia commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-594197974 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 397070) Time Spent: 7h 40m (was: 7.5h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 7h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=396891=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396891 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 03/Mar/20 17:51 Start Date: 03/Mar/20 17:51 Worklog Time Spent: 10m Work Description: bipinupd commented on issue #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028#issuecomment-594081070 HI @iemejia, I decided to close the previous PR (https://github.com/apache/beam/pull/10604) and opened a new one. Sorry for the inconvenience. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396891) Time Spent: 7h 20m (was: 7h 10m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 7h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=396890=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396890 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 03/Mar/20 17:50 Start Date: 03/Mar/20 17:50 Worklog Time Spent: 10m Work Description: bipinupd commented on pull request #11028: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/11028 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @iemejia emejia`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[![Build
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=396886=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396886 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 03/Mar/20 17:47 Start Date: 03/Mar/20 17:47 Worklog Time Spent: 10m Work Description: bipinupd commented on pull request #10604: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/10604 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396886) Time Spent: 7h (was: 6h 50m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 7h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=396885=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396885 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 03/Mar/20 17:47 Start Date: 03/Mar/20 17:47 Worklog Time Spent: 10m Work Description: bipinupd commented on issue #10604: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/10604#issuecomment-594079164 Closing this pull request. I did something wrong while rebasing. It is showing way too much file changed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396885) Time Spent: 6h 50m (was: 6h 40m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 6h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=396774=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396774 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 03/Mar/20 14:23 Start Date: 03/Mar/20 14:23 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10604: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/10604#issuecomment-593976013 Oups yes it looks pretty screwed up. You can fix those errors doing `git pull origin master --rebase` but you would have to deal with the merge conflicts, you can also open a new PR if you prefer just mention me there if you take that approach to continue the review there.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396774) Time Spent: 6h 40m (was: 6.5h) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 6h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=396190=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396190 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 02/Mar/20 16:34 Start Date: 02/Mar/20 16:34 Worklog Time Spent: 10m Work Description: bipinupd commented on issue #10604: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/10604#issuecomment-593492504 @iemejia I think I did something wrong while rebasing. It is showing way too much file changed, however the branch looks fine [only files related to my project are showing] (https://github.com/apache/beam/compare/master...bipinupd:BEAM-2546?expand=1) . Do you want me to open a new PR. Also, I did not find embedded/mock server or InfluxDB. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396190) Time Spent: 6.5h (was: 6h 20m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 6.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[ https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=392103=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392103 ] ASF GitHub Bot logged work on BEAM-2546: Author: ASF GitHub Bot Created on: 24/Feb/20 21:32 Start Date: 24/Feb/20 21:32 Worklog Time Spent: 10m Work Description: alexvanboxel commented on pull request #10604: BEAM-2546 Beam IO for InfluxDB URL: https://github.com/apache/beam/pull/10604#discussion_r383525711 ## File path: settings.gradle ## @@ -145,8 +149,8 @@ include ":sdks:python:test-suites:tox:py2" include ":sdks:python:test-suites:tox:py35" include ":sdks:python:test-suites:tox:py36" include ":sdks:python:test-suites:tox:py37" -include ":vendor:grpc-1_21_0" -include ":vendor:bytebuddy-1_9_3" +include ":vendor:grpc-1_26_0" +include ":vendor:bytebuddy-1_10_8" Review comment: The vendored versions should not be changed in this PR. Probably better rebase this PR. Focus in the settings on changes for the PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392103) Time Spent: 6h 20m (was: 6h 10m) > Create InfluxDbIO > - > > Key: BEAM-2546 > URL: https://issues.apache.org/jira/browse/BEAM-2546 > Project: Beam > Issue Type: New Feature > Components: io-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré >Priority: Major > Time Spent: 6h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391162=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391162
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:08
Start Date: 22/Feb/20 15:08
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382915322
##
File path: sdks/java/io/influxdb/build.gradle
##
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+plugins { id 'org.apache.beam.module' }
+applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.influxdb')
+provideIntegrationTestingDependencies()
+enableJavaPerformanceTesting()
+
+description = "Apache Beam :: SDKs :: Java :: IO :: InfluxDB"
+ext.summary = "IO to read and write on JDBC datasource."
+
+dependencies {
+ compile "com.google.auto.value:auto-value-annotations:1.6.6"
Review comment:
remove this so you use beam's default autovalue
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391162)
Time Spent: 6h 10m (was: 6h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>Priority: Major
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391160=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391160
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382917635
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,624 @@
+package org.apache.beam.sdk.io.influxdb;
+
+import com.google.auto.value.AutoValue;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.*;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.*;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.*;
+import org.influxdb.dto.Query;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+import javax.net.ssl.*;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.*;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+/**
+ * IO to read and write to InfluxDB.
+ Reading from InfluxDB datasource
+*
+* InfluxDBIO source returns a bounded collection of {@code String}
as a {@code PCollection}.
+*
+* To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+* {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+* Optionally, {@link DataSourceConfiguration#withUsername(String)} and
{@link
+* DataSourceConfiguration#withPassword(String)} allows you to define
username and password.
+ *
+ * For example:
+*
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+*
+* For example (Read from query):
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withQuery("Select * from cpu")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+* Writing to Influx datasource
+*
+* InfluxDB sink supports writing records into a database. It writes
a {@link PCollection} to the
+* database by converting each T. The T should implement
getLineProtocol() from {@link LineProtocolConvertable}.
+*
+* Like the source, to configure the sink, you have to provide a
{@link DataSourceConfiguration}.
+*
+* {@code
+* pipeline
+* .apply(...)
+* .apply(InfluxDb.write()
+*
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+*"https://localhost:8086","username","password;))
+* .withRetentionPolicy("autogen")
+* .withDatabase("metrics")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+*);
+* }
+ * *
+ */
+@Experimental(Experimental.Kind.SOURCE_SINK)
+public class InfluxDBIO {
+private static final Logger LOG =
LoggerFactory.getLogger(InfluxDBIO.class);
+
+public static Write write() {
+return new AutoValue_InfluxDBIO_Write.Builder()
+.build();
+}
+public static Read read() {
+return new AutoValue_InfluxDBIO_Read.Builder()
+.build();
+}
+
+@AutoValue
+public abstract static class Read extends PTransform> {
+@Nullable
+abstract Boolean sslInvalidHostNameAllowed();
+@Nullable
+abstract String retentionPolicy();
+@Nullable
+abstract String database();
+@Nullable
+
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391142=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391142
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382916097
##
File path:
sdks/java/io/influxdb/src/test/java/org/apache/beam/sdk/io/influxdb/GenerateData.java
##
@@ -0,0 +1,58 @@
+package org.apache.beam.sdk.io.influxdb;
+
+import java.time.*;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Random;
+import java.util.concurrent.TimeUnit;
+
+public class GenerateData {
+
+public static Random random = new Random();
Review comment:
remove public
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391142)
Time Spent: 4.5h (was: 4h 20m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391143=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391143
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382916120
##
File path:
sdks/java/io/influxdb/src/test/java/org/apache/beam/sdk/io/influxdb/GenerateData.java
##
@@ -0,0 +1,58 @@
+package org.apache.beam.sdk.io.influxdb;
+
+import java.time.*;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Random;
+import java.util.concurrent.TimeUnit;
+
+public class GenerateData {
+
+public static Random random = new Random();
+public static List getMetric(int number){
+List element = new ArrayList<>();
+for (int i = 0; i< number; i++){
+Model m = new Model();
+m.setMeasurement("test_m");
+m.addFeild("test1", random.nextInt(100));
+m.addFeild("test2", random.nextInt(100));
+LocalDateTime time = LocalDateTime.of(LocalDate.now(),
+LocalTime.of(random.nextInt(24), random.nextInt(60),
+random.nextInt(60), random.nextInt(9 +
1)));
+ZonedDateTime
+zdt = time.atZone(ZoneId.of("America/Los_Angeles"));
+m.setTime(zdt.toInstant().toEpochMilli());
+m.setTimeUnit(TimeUnit.MILLISECONDS);
+element.add(m.getLineProtocol());
+}
+return element;
+}
+
+public static List getMultipleMetric(int number){
Review comment:
remove public
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391143)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391151=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391151
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382917687
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,624 @@
+package org.apache.beam.sdk.io.influxdb;
+
+import com.google.auto.value.AutoValue;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.*;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.*;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.*;
+import org.influxdb.dto.Query;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+import javax.net.ssl.*;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.*;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+/**
+ * IO to read and write to InfluxDB.
+ Reading from InfluxDB datasource
+*
+* InfluxDBIO source returns a bounded collection of {@code String}
as a {@code PCollection}.
+*
+* To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+* {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+* Optionally, {@link DataSourceConfiguration#withUsername(String)} and
{@link
+* DataSourceConfiguration#withPassword(String)} allows you to define
username and password.
+ *
+ * For example:
+*
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+*
+* For example (Read from query):
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withQuery("Select * from cpu")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+* Writing to Influx datasource
+*
+* InfluxDB sink supports writing records into a database. It writes
a {@link PCollection} to the
+* database by converting each T. The T should implement
getLineProtocol() from {@link LineProtocolConvertable}.
+*
+* Like the source, to configure the sink, you have to provide a
{@link DataSourceConfiguration}.
+*
+* {@code
+* pipeline
+* .apply(...)
+* .apply(InfluxDb.write()
+*
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+*"https://localhost:8086","username","password;))
+* .withRetentionPolicy("autogen")
+* .withDatabase("metrics")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
Review comment:
missing '.'
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391151)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee:
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391136=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391136
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382915443
##
File path: sdks/java/io/influxdb/build.gradle
##
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+plugins { id 'org.apache.beam.module' }
+applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.influxdb')
+provideIntegrationTestingDependencies()
+enableJavaPerformanceTesting()
+
+description = "Apache Beam :: SDKs :: Java :: IO :: InfluxDB"
+ext.summary = "IO to read and write on JDBC datasource."
+
+dependencies {
+ compile "com.google.auto.value:auto-value-annotations:1.6.6"
+ annotationProcessor "com.google.auto.value:auto-value:1.6.6"
+ compile group: 'org.slf4j', name: 'slf4j-api', version: '1.7.28'
+ compile group: 'com.google.auto.value', name: 'auto-value-annotations',
version: '1.6.6'
+ compile group: 'com.google.auto.value', name: 'auto-value', version: '1.6.6'
+ compile group: 'org.apache.beam', name: 'beam-sdks-java-core', version:
'2.15.0'
+ compile group: 'org.apache.beam', name: 'beam-vendor-guava-20_0', version:
'0.1'
Review comment:
`compile library.java.vendored_guava_26_0_jre` and fix imports consequently
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391136)
Time Spent: 3h 50m (was: 3h 40m)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391128=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391128
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382915322
##
File path: sdks/java/io/influxdb/build.gradle
##
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+plugins { id 'org.apache.beam.module' }
+applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.influxdb')
+provideIntegrationTestingDependencies()
+enableJavaPerformanceTesting()
+
+description = "Apache Beam :: SDKs :: Java :: IO :: InfluxDB"
+ext.summary = "IO to read and write on JDBC datasource."
+
+dependencies {
+ compile "com.google.auto.value:auto-value-annotations:1.6.6"
Review comment:
remove thi sso you use beam's default autovalue
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391128)
Time Spent: 2h 40m (was: 2.5h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391145=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391145
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382918246
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,624 @@
+package org.apache.beam.sdk.io.influxdb;
+
+import com.google.auto.value.AutoValue;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.*;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.*;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.*;
+import org.influxdb.dto.Query;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+import javax.net.ssl.*;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.*;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+/**
+ * IO to read and write to InfluxDB.
+ Reading from InfluxDB datasource
+*
+* InfluxDBIO source returns a bounded collection of {@code String}
as a {@code PCollection}.
+*
+* To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+* {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+* Optionally, {@link DataSourceConfiguration#withUsername(String)} and
{@link
+* DataSourceConfiguration#withPassword(String)} allows you to define
username and password.
+ *
+ * For example:
+*
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+*
+* For example (Read from query):
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withQuery("Select * from cpu")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+* Writing to Influx datasource
+*
+* InfluxDB sink supports writing records into a database. It writes
a {@link PCollection} to the
+* database by converting each T. The T should implement
getLineProtocol() from {@link LineProtocolConvertable}.
+*
+* Like the source, to configure the sink, you have to provide a
{@link DataSourceConfiguration}.
+*
+* {@code
+* pipeline
+* .apply(...)
+* .apply(InfluxDb.write()
+*
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+*"https://localhost:8086","username","password;))
+* .withRetentionPolicy("autogen")
+* .withDatabase("metrics")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+*);
+* }
+ * *
+ */
+@Experimental(Experimental.Kind.SOURCE_SINK)
+public class InfluxDBIO {
+private static final Logger LOG =
LoggerFactory.getLogger(InfluxDBIO.class);
+
+public static Write write() {
+return new AutoValue_InfluxDBIO_Write.Builder()
+.build();
+}
+public static Read read() {
+return new AutoValue_InfluxDBIO_Read.Builder()
+.build();
+}
+
+@AutoValue
+public abstract static class Read extends PTransform> {
+@Nullable
+abstract Boolean sslInvalidHostNameAllowed();
+@Nullable
+abstract String retentionPolicy();
+@Nullable
+abstract String database();
+@Nullable
+
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391153=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391153
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382917709
##
File path:
sdks/java/io/influxdb/src/main/java/org/apache/beam/sdk/io/influxdb/InfluxDBIO.java
##
@@ -0,0 +1,624 @@
+package org.apache.beam.sdk.io.influxdb;
+
+import com.google.auto.value.AutoValue;
+import okhttp3.OkHttpClient;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.coders.Coder;
+import org.apache.beam.sdk.coders.SerializableCoder;
+import org.apache.beam.sdk.io.BoundedSource;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.ValueProvider;
+import org.apache.beam.sdk.transforms.*;
+import org.apache.beam.sdk.transforms.display.DisplayData;
+import org.apache.beam.sdk.transforms.display.HasDisplayData;
+import org.apache.beam.sdk.values.*;
+import org.influxdb.BatchOptions;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.*;
+import org.influxdb.dto.Query;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+import javax.net.ssl.*;
+import java.io.Serializable;
+import java.security.cert.CertificateException;
+import java.util.*;
+
+import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;
+
+/**
+ * IO to read and write to InfluxDB.
+ Reading from InfluxDB datasource
+*
+* InfluxDBIO source returns a bounded collection of {@code String}
as a {@code PCollection}.
+*
+* To configure the InfluxDB source, you have to provide a {@link
DataSourceConfiguration} using
+* {@link DataSourceConfiguration#create(String, String, String)}(durl,
username and password).
+* Optionally, {@link DataSourceConfiguration#withUsername(String)} and
{@link
+* DataSourceConfiguration#withPassword(String)} allows you to define
username and password.
+ *
+ * For example:
+*
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+*
+* For example (Read from query):
+* {@code
+* PCollection collection = pipeline.apply(InfluxDBIO.read()
+* .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+* "https://localhost:8086","username","password;))
+* .withDatabase("metrics")
+* .withQuery("Select * from cpu")
+* .withRetentionPolicy("autogen")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+* }
+* Writing to Influx datasource
+*
+* InfluxDB sink supports writing records into a database. It writes
a {@link PCollection} to the
+* database by converting each T. The T should implement
getLineProtocol() from {@link LineProtocolConvertable}.
+*
+* Like the source, to configure the sink, you have to provide a
{@link DataSourceConfiguration}.
+*
+* {@code
+* pipeline
+* .apply(...)
+* .apply(InfluxDb.write()
+*
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
+*"https://localhost:8086","username","password;))
+* .withRetentionPolicy("autogen")
+* .withDatabase("metrics")
+* .withSslInvalidHostNameAllowed(true)
+* withSslEnabled(true));
+*);
+* }
+ * *
+ */
+@Experimental(Experimental.Kind.SOURCE_SINK)
+public class InfluxDBIO {
+private static final Logger LOG =
LoggerFactory.getLogger(InfluxDBIO.class);
+
+public static Write write() {
+return new AutoValue_InfluxDBIO_Write.Builder()
+.build();
+}
+public static Read read() {
+return new AutoValue_InfluxDBIO_Read.Builder()
+.build();
+}
+
+@AutoValue
+public abstract static class Read extends PTransform> {
+@Nullable
+abstract Boolean sslInvalidHostNameAllowed();
+@Nullable
Review comment:
review which of these are really Nullable
[jira] [Work logged] (BEAM-2546) Create InfluxDbIO
[
https://issues.apache.org/jira/browse/BEAM-2546?focusedWorklogId=391144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391144
]
ASF GitHub Bot logged work on BEAM-2546:
Author: ASF GitHub Bot
Created on: 22/Feb/20 15:07
Start Date: 22/Feb/20 15:07
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10604: BEAM-2546
Beam IO for InfluxDB
URL: https://github.com/apache/beam/pull/10604#discussion_r382915976
##
File path:
sdks/java/io/influxdb/src/test/java/org/apache/beam/sdk/io/influxdb/Model.java
##
@@ -0,0 +1,75 @@
+package org.apache.beam.sdk.io.influxdb;
+
+import org.apache.beam.sdk.coders.AvroCoder;
+import org.apache.beam.sdk.coders.DefaultCoder;
+import org.influxdb.dto.Point;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+@DefaultCoder(AvroCoder.class)
+public class Model implements LineProtocolConvertable {
Review comment:
remove public, make it package private
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391144)
Time Spent: 4h 40m (was: 4.5h)
> Create InfluxDbIO
> -
>
> Key: BEAM-2546
> URL: https://issues.apache.org/jira/browse/BEAM-2546
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>Priority: Major
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
