[
https://issues.apache.org/jira/browse/BEAM-6347?focusedWorklogId=180954&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-180954
]
ASF GitHub Bot logged work on BEAM-6347:
----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Jan/19 03:52
Start Date: 04/Jan/19 03:52
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #7397:
[BEAM-6347] Add website page for developing I/O connectors for Java
URL: https://github.com/apache/beam/pull/7397#discussion_r245182395
##########
File path: website/src/documentation/io/developing-io-java.md
##########
@@ -0,0 +1,419 @@
+---
+layout: section
+title: "Apache Beam: Developing I/O connectors for Java"
+section_menu: section-menu/documentation.html
+permalink: /documentation/io/developing-io-java/
+redirect_from: /documentation/io/authoring-java/
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+# Developing I/O connectors for Java
+
+To connect to a data store that isn’t supported by Beam’s existing I/O
+connectors, you must create a custom I/O connector that usually consist of a
+source and a sink. All Beam sources and sinks are composite transforms;
however,
+the implementation of your custom I/O depends on your use case. See the [new
+I/O connector overview]({{ site.baseurl
}}/documentation/io/developing-io-overview/)
+for a general overview of developing a new I/O connector.
+
+This page describes implementation details for developing sources and sinks
+using Java. The Python SDK offers the same functionality, but uses a slightly
+different API. See [Developing I/O connectors for Python]({{ site.baseurl
}}/documentation/io/developing-io-python/)
+for information specific to the Python SDK.
+
+## Implementation options
+
+**Sources**
+
+For bounded (batch) sources, there are currently two options for creating a
Beam
+source:
+
+1. Use `ParDo` and `GroupByKey`.
+2. Use the `Source` interface and extend the `BoundedSource` abstract subclass.
+
+`ParDo` is the recommended option, as implementing a `Source` can be tricky.
+The [developing I/O connectors overview]({{ site.baseurl
}}/documentation/io/developing-io-overview/)
+covers using `ParDo`, and lists some use cases where you might want to use a
+Source (such as [dynamic work rebalancing]({{ site.baseurl
}}/blog/2016/05/18/splitAtFraction-method.html)).
+
+For unbounded (streaming) sources, you must use the `Source` interface and
extend
+the `UnboundedSource` abstract subclass. `UnboundedSource` supports features
that
+are useful for streaming pipelines such as checkpointing.
+
+Splittable DoFn is a new sources framework that is under development and will
+replace the other options for developing bounded and unbounded sources. For
more
+information, see the
+[roadmap for multi-SDK connector efforts]({{ site.baseurl
}}/roadmap/connectors-multi-sdk/).
+
+**Sinks**
+
+To create a Beam sink, we recommend that you use a single `ParDo` that writes
the
Review comment:
Please see 'FileBasedSinks' section if your sink is for writing files. For
other sinks please used ParDo, GroupByKey and other transforms offered by Beam
Java SDK. Please see \<overview page\> for more details
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 180954)
Time Spent: 40m (was: 0.5h)
> Add page for developing I/O connectors for Java
> -----------------------------------------------
>
> Key: BEAM-6347
> URL: https://issues.apache.org/jira/browse/BEAM-6347
> Project: Beam
> Issue Type: Bug
> Components: website
> Reporter: Melissa Pashniak
> Assignee: Melissa Pashniak
> Priority: Minor
> Time Spent: 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
