GWicke added a comment. > For starters, it means that we have alternatives for environments where Kafka > is overkill (small third-party installations, dev environments, mw-vagrant, > etc). Using, for example, sqlite instead of Kafka is already something > supported.
As far as I can see, there is no support for using any database as a queue / log in a way that would give us a light-weight alternative to Kafka. There is generally no support for streaming from a database in EventLogging, and separate tables are created whenever a schema is changed. We'll have to implement this either way. We do have fairly nice async table abstractions for sqlite and cassandra that we could reuse for this in node. Both already implement retention policies. Python has sqlalchemy, which is a pretty nice way to interface with dbs. Retention policies would have to be implemented manually. TASK DETAIL https://phabricator.wikimedia.org/T114443 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ottomata, GWicke Cc: mark, MZMcBride, Krinkle, EBernhardson, bd808, Joe, dr0ptp4kt, madhuvishy, Nuria, ori, faidon, aaron, GWicke, mobrovac, Eevans, Ottomata, Matanya, Aklapper, JAllemandou, jkroll, Smalyshev, Hardikj, Wikidata-bugs, Jdouglas, RobH, aude, Deskana, Manybubbles, daniel, JanZerebecki, RobLa-WMF, Jay8g, fgiunchedi, Dzahn, jeremyb, Legoktm, chasemp, Krenair _______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
