Hi Youcef,
On Fri, Aug 24, 2018 at 5:11 PM Youcef HILEM <youcef.hi...@laposte.fr> wrote: > Thank you very much. > We will therfore implement two solutions: > - One based on S3 with a Cassandra back-end > (https://github.com/exoscale/pithos) > Interesting. I'm curious why not use AWS S3 for storage as well? Data confidentiality concerns? Does this also mean the ArtifactoryRepository is written independently of data storage? For people who are already on AWS, using AWS S3 object storage would make more sense. > - Another based on kafka with compacted topics > (http://cloudurable.com/blog/kafka-architecture-log-compaction/index.html) > and Kafka mirroring (MirrorMaker) > (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330 > ) > > The implementation is less complex with kafka. Moreover the updates will be > in "real time". > > The deployment task runs every 15seconds, and that's when the manager node will check for local changes, and check-in/commit the changes to the external repo. If there were indeed local file changes, carbon sends a depsync cluster message to rest of the nodes asking to update themselves. The cluster message is sent almost real-time. Btw, with S3 sync, you can just commit and retrieve the file diffs. This is important because you don't want to update the timestamps of unchanged files. That'll lead to carbon deploying all the depl artifacts again and again. How do you plan to send just file diffs with Kafka? [1] https://github.com/kasunbg/carbon-kernel-4.x/blob/master/core/org.wso2.carbon.core/src/main/java/org/wso2/carbon/core/deployment/CarbonDeploymentSchedulerTask.java#L141-L147 > Regards > Youcef HILEM > > > > -- > Sent from: > http://wso2-oxygen-tank.10903.n7.nabble.com/WSO2-Development-f3.html > _______________________________________________ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > -- *Kasun Gajasinghe*Technical Lead, WSO2 Inc. email: kasung AT spamfree wso2.com linked-in: http://lk.linkedin.com/in/gajasinghe blog: http://kasunbg.org phone: +1 650-745-4499, 77 678 0813
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
