[
https://issues.apache.org/jira/browse/BEAM-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566541#comment-16566541
]
Varsha Thanooj commented on BEAM-5059:
--------------------------------------
Yes, I am referring to fully hosted Amazon elasticsearch service.
As per the documentation ElasticsearchIO is not providing any way to input the
aws credentials.
I tried
{code:java}
final String[] es_host = new String[] {"https://XXXX"}
data.apply(ElasticsearchIO.write()
.withConnectionConfiguration(
ElasticsearchIO.ConnectionConfiguration.create(es_host, "XXX", "XXX")));
{code}
The console shows
{code:java}
Exception in thread "main"
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.util.concurrent.UncheckedExecutionException:
org.apache.beam.sdk.util.UserCodeException:
java.lang.IllegalArgumentException: Cannot get Elasticsearch version
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2214)
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache.get(LocalCache.java:4053)
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
at
org.apache.beam.runners.direct.DoFnLifecycleManager.get(DoFnLifecycleManager.java:61)
at
org.apache.beam.runners.direct.ParDoEvaluatorFactory.createEvaluator(ParDoEvaluatorFactory.java:124)
at
org.apache.beam.runners.direct.ParDoEvaluatorFactory.forApplication(ParDoEvaluatorFactory.java:78)
at
org.apache.beam.runners.direct.TransformEvaluatorRegistry.forApplication(TransformEvaluatorRegistry.java:178)
at
org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:118)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.beam.sdk.util.UserCodeException:
java.lang.IllegalArgumentException: Cannot get Elasticsearch version
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$Write$WriteFn$DoFnInvoker.invokeSetup(Unknown
Source)
at
org.apache.beam.runners.direct.DoFnLifecycleManager$DeserializingCacheLoader.load(DoFnLifecycleManager.java:102)
at
org.apache.beam.runners.direct.DoFnLifecycleManager$DeserializingCacheLoader.load(DoFnLifecycleManager.java:91)
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628)
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336)
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295)
at
org.apache.beam.repackaged.beam_runners_direct_java.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208)
... 13 more
Caused by: java.lang.IllegalArgumentException: Cannot get Elasticsearch version
at
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.getBackendVersion(ElasticsearchIO.java:1045)
at
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$Write$WriteFn.setup(ElasticsearchIO.java:925)
{code}
> Access Restful elasticsearch on aws from beam
> ---------------------------------------------
>
> Key: BEAM-5059
> URL: https://issues.apache.org/jira/browse/BEAM-5059
> Project: Beam
> Issue Type: Improvement
> Components: io-java-elasticsearch
> Affects Versions: 2.5.0
> Reporter: Varsha Thanooj
> Assignee: Ismaël Mejía
> Priority: Blocker
> Labels: newbie
>
> Gone through the documentation of ElasticsearchIO. But I haven't found any
> way to write records to aws elasticsearch.
> Is there a way to access Restful Elasticsearch hosted on aws from beam ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
