turbaszek commented on pull request #8:
URL: https://github.com/apache/kibble/pull/8#issuecomment-851616956
> how do you see this working for example for our project mailing lists?
Would each list the be a datasource and the conversations the datatype?
That's a very good question @sharanf!
I would lean to what you've written. Datasource does not only represent an
"external service" entity but "account/organization within an external
service". So, in case of mailing list each Apache project would required
configuring their own data source.
For example:
```yaml
data_sources:
- name: asf_mails_kibble
class: kibble.data_sources.pony_mail.PonyMailDataSource
config:
project_name: kibble
enabled_data_types:
- mails
- name: asf_mails_kafka
class: kibble.data_sources.pony_mail.PonyMailDataSource
config:
project_name: kafka
enabled_data_types:
- mails
- name: asf_mails_pulsar
class: kibble.data_sources.pony_mail.PonyMailDataSource
config:
project_name: pulsar
enabled_data_types:
- mails
```
While there's a bit of duplication in configuration it allow more
granularity. In case of ASF the config will be big and repeatable but for
smaller Kibble deployments it would be smaller and more configuration maybe an
advantage.
Additionally this additional granularity is useful in case of sources that
need authorisation. In such cases we may want to store the credentials in
different way or use different auth methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
