Strong +1 for using x-lang instead if re-implementing the ElasticSearch connector in Go.
Thanks, Cham On Fri, Jul 8, 2022 at 5:16 AM Yu Watanabe <[email protected]> wrote: > Hello Danny. > > Thank you for the details. I appreciate your message. > > I am a newbie around building io . So I will look into the links and first > build my knowledge. > > Thanks, > Yu Watanabe > > On Fri, Jul 8, 2022 at 8:28 PM Danny McCormick via user < > [email protected]> wrote: > >> Hey Yu, >> >> The guidance on that page should generally apply for Go as well, though >> we are missing an example transform; I filed >> https://github.com/apache/beam/issues/22194 to fix this, but a couple >> examples are our textio implementation >> <https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/io/textio/textio.go> >> and this native streaming example >> <https://github.com/apache/beam/blob/master/sdks/go/examples/native_wordcap/nativepubsubio/native.go> >> (non-productionized, but it shows a lot of good concepts for handling >> streaming sources). >> >> Another option would be to use Java's elasticsearch implementation >> <https://github.com/apache/beam/blob/master/sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java> >> with a cross language transform. If you're thinking of contributing this >> back to the main beam code base, I'd probably recommend that approach. In >> general, we're currently investing most heavily in cross language >> transforms because it's a much lower burden to build/support since it >> reuses the main components of the original transform. There are several >> examples of wrapped cross-language transforms in >> https://github.com/apache/beam/tree/master/sdks/go/pkg/beam/io/xlang. If >> you have specific questions about authoring IOs that aren't answered in the >> docs/by example, feel free to drop them in this thread as well! >> >> Thanks, >> Danny >> >> On Fri, Jul 8, 2022 at 3:51 AM Yu Watanabe <[email protected]> wrote: >> >>> Hello . >>> >>> Is there any guideline for building a go sdk connector ? >>> I was reviewing the document but I could not find one for golang. >>> >>> https://beam.apache.org/documentation/io/developing-io-overview/ >>> >>> I was thinking of building one for elasticsearch. >>> >>> Thanks, >>> Yu Watanabe >>> >>> -- >>> Yu Watanabe >>> >>> linkedin: www.linkedin.com/in/yuwatanabe1/ >>> twitter: twitter.com/yuwtennis >>> >>> >> > > -- > Yu Watanabe > > linkedin: www.linkedin.com/in/yuwatanabe1/ > twitter: twitter.com/yuwtennis > >
