mxtymoshyk commented on issue #35777:
URL: https://github.com/apache/beam/issues/35777#issuecomment-5483159198
@Abacn thanks for reopening this. I reproduced it against
`ghcr.io/goccy/bigquery-emulator` and the `DIRECT_READ` half that @hichmo
raised in the follow-up looks like a real gap, distinct from the `tempLocation`
part.
**The `tempLocation` half is WAI, as you said.** `BigQueryIO.java:1415`
already skips that check when the method is `DIRECT_READ`, so nothing needs to
change there:
```java
if (getMethod() != TypedRead.Method.DIRECT_READ) {
String tempLocation = bqOptions.getTempLocation();
checkArgument(!Strings.isNullOrEmpty(tempLocation), "BigQueryIO.Read needs
a GCS temp location ...");
```
## Why DIRECT_READ hangs
`StorageClientImpl` honors `--bigQueryEndpoint` but keeps the default gRPC
transport, which negotiates TLS (`BigQueryServicesImpl.java:1902-1913`):
```java
BigQueryReadSettings.Builder settingsBuilder =
BigQueryReadSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(options.getGcpCredential()))
.setTransportChannelProvider(
BigQueryReadSettings.defaultGrpcTransportProviderBuilder()
.setHeaderProvider(USER_AGENT_HEADER_PROVIDER)
.build())
.setReadRowsRetryAttemptListener(listener);
@Nullable String endpoint = options.getBigQueryEndpoint();
if (!Strings.isNullOrEmpty(endpoint)) {
settingsBuilder.setEndpoint(trimSchemaIfNecessary(endpoint));
}
```
The emulator's gRPC port is plaintext, so the handshake never completes. A/B
with the same client, same endpoint, only the transport differing:
| Transport | Result |
| --- | --- |
| `InstantiatingGrpcChannelProvider` + `setChannelConfigurator(c ->
c.usePlaintext())` | success in 0.4s, read session created, `streams=1` |
| `BigQueryReadSettings.defaultGrpcTransportProviderBuilder()` (what Beam
uses) | hangs; killed at 25s and again at 60s |
Parked here:
```
com.google.cloud.bigquery.storage.v1.BigQueryReadClient.createReadSession(BigQueryReadClient.java:259)
com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53)
```
The emulator logged nothing at all for the TLS attempt, i.e. the request
never got past the handshake. The plaintext row is the control: the emulator's
Storage Read API itself works.
In a real pipeline this is worse than the 25s above suggests.
`BigQueryServicesImpl.java:1918-1925` sets `createReadSession`'s initial, max
and total RPC timeout to `Duration.ofHours(2)`, so the user-visible symptom is
a silent two-hour stall rather than an error. That matches @hichmo's report
that the BigQuery job completes but the pipeline stays stuck.
`newBigQueryWriteClient` (`BigQueryServicesImpl.java:1786-1789`) has the
same gap, so `STORAGE_WRITE_API` against an emulator is blocked in the same way.
## A second, independent blocker: one option, two ports
Running an actual `DIRECT_READ` pipeline never even reaches the storage
client. With `--bigQueryEndpoint=http://localhost:9060/` (the value the gRPC
client needs), the REST client is pointed at the gRPC port and hangs there
first:
```
BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:703)
BigQueryServicesImpl.executeWithRetries(BigQueryServicesImpl.java:1720)
BigQueryStorageTableSource.getTargetTable(BigQueryStorageTableSource.java:220)
```
`bigQueryEndpoint` feeds three clients with incompatible requirements: the
REST client takes it verbatim as `setRootUrl` and needs `http://host:9050/`,
while both gRPC clients run it through `trimSchemaIfNecessary`
(`BigQueryServicesImpl.java:1803`) and need `host:9060`. The emulator serves
REST on 9050 and gRPC on 9060, so no single value works. Fixing plaintext alone
would not be enough.
## Precedent already in this module
`FirestoreStatefulComponentFactory.java:86-97` solves exactly this, in the
same package tree:
```java
if (emulatorHostPort != null) {
builder
.setCredentialsProvider(FixedCredentialsProvider.create(new
EmulatorCredentials()))
.setEndpoint(emulatorHostPort)
.setTransportChannelProvider(
InstantiatingGrpcChannelProvider.newBuilder()
.setEndpoint(emulatorHostPort)
.setChannelConfigurator(c -> c.usePlaintext())
.build());
}
```
`SpannerAccessor.java:275-277` does the same behind a `usePlaintext` flag.
## Question before anyone writes code
Which shape would you prefer?
1. A new `BigQueryOptions.getBigQueryEmulatorHost()` that points the two
gRPC clients at that host with plaintext and a no-op credential, leaving
`bigQueryEndpoint` alone for the REST client. Closest to the Firestore
precedent, and additive for existing users.
2. A separate gRPC endpoint option plus an explicit plaintext flag, closer
to the Spanner shape.
I have the reproduction set up and am happy to send a PR, but I did not want
to guess at the public option surface. Not claiming the issue in the meantime.
<sub>Reproduced on Beam 2.75.0, DirectRunner, emulator seeded with one 3-row
table. On 2.75.0 you also hit an NPE in
`BigQueryStorageTableSource.getEstimatedSizeBytes` when the emulator returns no
`numBytes`; that one is already fixed on master by 4e5ae91554e and is not part
of this issue.</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
