> This is a little inferior to the production version as it is unable to use production kafka and if it has any outages it will lose data.
EventStreams isn’t as good as using Kafka, but an outage shouldn’t be a reason to lose data. Store the Last-Event-ID <https://wikitech.wikimedia.org/wiki/EventStreams#Format> that EventStreams gives you, and you can use it when the service starts back up to start from where you left off. > and maybe, as others have mentioned, a good reason to get production Kafka events flowing into Cloud VPS backed projects. Def not opposed to a Kafka cluster in Cloud mirroring from Prod. :) BTW, this is a wee relevant: https://wikitech.wikimedia.org/wiki/User:Ottomata/Stream_Data_Platform This is a draft! I’m shopping this around as a program for next FY. We will see! On Tue, Dec 12, 2017 at 4:16 PM, Gergo Tisza <[email protected]> wrote: > On Tue, Dec 12, 2017 at 12:12 PM, Jon Robson <[email protected]> wrote: > > > This is a little inferior to the production version as it is unable to > use > > production kafka and if it has any outages it will lose data. > > > > Hopefully that gets fixed soon, Cloud VPS / Toolforge is the foundation for > out volunteer tool developer community who really shouldn't be treated as > second-class citizens. > > Other than that, moving to the Cloud is not a bad thing for an experimental > project IMO. It makes it easier to experiment with minimal risk, and it > makes it easy to add co-collaborators to your project without having to get > prod access for them. > > I'm hoping to get this onto IFTTT <https://ifttt.com/wikipedia> with help > > from Stephen Laporte in my volunteer time, as I think this feature is a > > pretty powerful one which has failed to find its use case in the wiki > > world. As Kaldari points out it's incredibly good at detecting edit wars > > and I personally have learned a lot about what our editors see as > important > > and notable in the world (our editors really seem to like wrestling). I > > think there are ample and exciting things people could build on top of > this > > api. > > > > Yeah a "give me push notifications about ongoing edit wars" tool for admins > sounds really cool. Although you'd probably want to look at revert trends > (or both edit and revert trends) for that. > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
