[
https://issues.apache.org/jira/browse/BEAM-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Jorgensen updated BEAM-14202:
------------------------------------
Description:
[com.google.cloud.spanner.SpannerOptions|https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java]
has to determine if the configuration should use the emulator or not.
Essentially it will look for SPANNER_EMULATOR_HOST in the environment and if a
spanner host is not explicitly set it will set the host as the emulator, [see
here|[https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java#L1143-L1153]
By Contrast in Beam, SpannerConfig by default has a host set ([see
here|[https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java#L83]).
When Beam goes to convert SpannerConfig to SpannerOptions in SpannerAccessor
it will always call setHost on SpannerOptions which will set the emulatorHost
to null, [see
here|https://github.com/apache/beam/blob/47ab260aaad6b1014d86dd6cf9274f1045690631/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java#L173-L176].
The consequence of this is the SpannerAccessor will always ignore the
SPANNER_EMULATOR_HOST environment variable unless withEmulatorHost is called
explicitly on SpannerConfig. Ultimately this pushes the responsibility onto the
user to manually check for the environment variable and conditionally set it
rather than relying on the functionality that already exists in the underlying
SpannerOptions provided by the spanner sdk.
I think the solution would be for Beam to also check to see if
SPANNER_EMULATOR_HOST is set and then fall back to the provided host or the
DEFAULT_HOST in the event that we are not connecting to the emulator.
was:
[com.google.cloud.spanner.SpannerOptions|https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java]
has to determine if the configuration should use the emulator or not.
Essentially it will look for SPANNER_EMULATOR_HOST in the environment and if a
spanner host is not explicitly set it will set the host as the emulator, [see
here|[https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java#L1143-L1153]
By Contrast in Beam, SpannerConfig by default has a host set ([see
here|[https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java#L83]).
When Beam goes to convert SpannerConfig to SpannerOptions in SpannerAccessor
it will always call setHost on SpannerOptions which will set the emulatorHost
to null, [see
here|https://github.com/apache/beam/blob/47ab260aaad6b1014d86dd6cf9274f1045690631/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java#L173-L176].
The consequence of this is the SpannerAccessor will always ignore the
SPANNER_EMULATOR_HOST environment variable unless withEmulatorHost is called
explicitly on SpannerConfig. Ultimately this pushes the responsibility onto the
user to manually check for the environment variable and conditionally set it
rather than relying on the functionality that already exists in the underlying
SpannerOptions provided by the spanner sdk.
I think the solution would be for Beam to also check to see if
SPANNER_EMULATOR_HOST is set and then fall back to the provided host or the
DEFAULT_HOST.
> SpannerConfig does not respect SPANNER_EMULATOR_HOST
> ----------------------------------------------------
>
> Key: BEAM-14202
> URL: https://issues.apache.org/jira/browse/BEAM-14202
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.37.0
> Reporter: Andrew Jorgensen
> Priority: P2
> Fix For: Not applicable
>
>
> [com.google.cloud.spanner.SpannerOptions|https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java]
> has to determine if the configuration should use the emulator or not.
> Essentially it will look for SPANNER_EMULATOR_HOST in the environment and if
> a spanner host is not explicitly set it will set the host as the emulator,
> [see
> here|[https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java#L1143-L1153]
> By Contrast in Beam, SpannerConfig by default has a host set ([see
> here|[https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java#L83]).
> When Beam goes to convert SpannerConfig to SpannerOptions in SpannerAccessor
> it will always call setHost on SpannerOptions which will set the emulatorHost
> to null, [see
> here|https://github.com/apache/beam/blob/47ab260aaad6b1014d86dd6cf9274f1045690631/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java#L173-L176].
>
> The consequence of this is the SpannerAccessor will always ignore the
> SPANNER_EMULATOR_HOST environment variable unless withEmulatorHost is called
> explicitly on SpannerConfig. Ultimately this pushes the responsibility onto
> the user to manually check for the environment variable and conditionally set
> it rather than relying on the functionality that already exists in the
> underlying SpannerOptions provided by the spanner sdk.
> I think the solution would be for Beam to also check to see if
> SPANNER_EMULATOR_HOST is set and then fall back to the provided host or the
> DEFAULT_HOST in the event that we are not connecting to the emulator.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
