tenthe commented on PR #3168:
URL: https://github.com/apache/streampipes/pull/3168#issuecomment-2301274682
@IsaakKrut thank you for this PR. The code looks good, but before we
proceed, I think we should clarify some conceptual ideas in StreamPipes that
are not quite clear yet.
### General Considerations:
- **Adapters vs. Data Streams:** These are two distinct elements in our
architecture, but they have interdependencies. We are currently not fully
consistent with the concept due to these dependencies.
### Topic 1:
- **Data Source Changes:** Your PR modifies the Data Source only. When we
change Data Sources, we may also need to update the Pipelines. The
functionality for handling this in Adapters can be found in
`AdapterResource.performPipelineMigrationPreflight`. A similar approach might
be required for Data Source updates.
### Topic 2:
- **Event Schema Handling:**
- This is not directly related to your PR, but it is an open issue in
our system that your PR brought to light.
- **Problem:** An Adapter creates a Data Stream. A user then changes the
Event Schema of this Data Stream via the API. This can lead to a situation
where the Adapter and the Data Source have different Event Schemas.
- I'm not yet sure how to resolve this. We can discuss this separately
if needed.
I think **Topic 2** should be discussed in a separate thread, as there are
additional points that need clarification.
How should we proceed with **Topic 1**? We could either:
- **Option 1**: Address it in this PR by implementing the necessary updates
for Data Source changes.
- **Option 2**: Leave it for now and create a separate issue and PR after we
have a solution for **Topic 2**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
