nuggetwheat commented on PR #25108:
URL: https://github.com/apache/beam/pull/25108#issuecomment-1400709831
When reading from change streams there are two databases involved, 1) the
primary database that contains the change stream, and 2) a metadata database
(maintained by Beam) that holds partition progress state so a job can be
restarted and pick up where it left off. To create a spanner config to access
the metadata database, the following pattern is used:
```
SpannerConfig metadataConfig = spannerConfig.toBuilder()
.setInstanceId(metadataInstanceId)
.setDatabaseId(metadataDatabaseId)
.build();
```
The metadata config (`metadataConfig`) is initialized using the primary
database config (`spannerConfig`) since most of the fields are the same and
then the ones that are different get overwritten. With the Fine-Grained Access
Control project, we've added a new config field _databaseRole_ that should only
be set for primary database access. Unfortunately the pattern above blindly
copies this field to the metadata config. A better approach might be to write
a metadata config create function that copies a whitelisted set of fields from
the primary config and leaves all of the other fields empty. I can implement
something like this now if you'd like. Otherwise, I can create an issue and
assign it to someone on the change stream team who maintains this code. What
are your thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
