[
https://issues.apache.org/jira/browse/BEAM-14326?focusedWorklogId=760140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760140
]
ASF GitHub Bot logged work on BEAM-14326:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Apr/22 15:45
Start Date: 21/Apr/22 15:45
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on code in PR #17428:
URL: https://github.com/apache/beam/pull/17428#discussion_r855330571
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinationsTableRow.java:
##########
@@ -95,7 +96,14 @@ public MessageConverter<T> getMessageConverter(
+ "using a create disposition of CREATE_IF_NEEDED.");
}
}
+ } else {
+ // Make sure we register this schema with the cache, unless there's
already a more
+ // up-to-date schema.
+ tableSchema =
+ MoreObjects.firstNonNull(
+ SCHEMA_CACHE.putSchemaIfAbsent(tableReference, tableSchema),
tableSchema);
Review Comment:
Is this change to fix some exception that was raising in the daemon and
crashing it?
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableSchemaCache.java:
##########
@@ -271,10 +280,11 @@ public void refreshThread() {
if (timeRemaining.getMillis() > 0) {
Thread.sleep(timeRemaining.getMillis());
}
- } catch (InterruptedException e) {
- runUnderMonitor(() -> this.stopped = true);
- return;
- } catch (IOException e) {
+ } catch (Exception e) {
Review Comment:
Is it appropriate to swallow _all_ exceptions? Could there be cases where
the daemon needs to be monitored and restarted?
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableSchemaCache.java:
##########
@@ -271,10 +280,11 @@ public void refreshThread() {
if (timeRemaining.getMillis() > 0) {
Thread.sleep(timeRemaining.getMillis());
}
- } catch (InterruptedException e) {
- runUnderMonitor(() -> this.stopped = true);
- return;
- } catch (IOException e) {
+ } catch (Exception e) {
+ // Since this is a daemon thread, don't exit until it is explicitly shut
down. Exiting early
+ // can cause the
+ // pipeline to stall.
+ LOG.error("Caught exception: " + e);
Review Comment:
Should this include some message about the context in which the exception
was raised?
Issue Time Tracking
-------------------
Worklog Id: (was: 760140)
Time Spent: 0.5h (was: 20m)
> Pipelines using BigQueryIO connector with one of the Storage Write API
> methods don't exit the launch process
> ------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-14326
> URL: https://issues.apache.org/jira/browse/BEAM-14326
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Sergei Lilichenko
> Assignee: Reuven Lax
> Priority: P1
> Fix For: 2.39.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The pipeline starts, but the launch process doesn't exit.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
