Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-09 Thread Damian Zaremba
On 01/03/2013 16:46, Asher Feldman wrote: I don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-03 Thread Antoine Musso
Le 01/03/13 09:47, Tim Starling a écrit : On 01/03/13 05:22, Petr Bena wrote: I believe it would require to create a new daemon (preferably written in c++) which I am willing to write, that could do similar what the ircd does right now. And that is delivery of new change to all connected

[Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
Hi, I think that irc feed of recent changes is working great, but there is still a lot of space for improvement. As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more advanced. My suggestion is to create some kind of feed that would be in

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Tyler Romeo
Hey, It sounds like an interesting idea. Actually, AWS (I've been working with it recently for Extension:AWS) has a similar architecture, where you establish a push notification service using their Simple Notification Service and have it send messages to a queue using their Simple Queue Service.

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
I believe it would require to create a new daemon (preferably written in c++) which I am willing to write, that could do similar what the ircd does right now. And that is delivery of new change to all connected clients. There would be preferably set of processes that are working together on this

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Daniel Friesen
We actually have an open RFC on this topic: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_data_push_notification_support_for_recent_changes -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
I see that the RFC is considering multiple formats, why not support all of them? We could make the client request the format they like, either XML or JSON, that would be a matter of dispatcher how it produce the output data. On Fri, Mar 1, 2013 at 2:35 PM, Daniel Friesen

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Happy Melon
Because we made that mistake with the API, and now we're stuck with a bunch of deadweight formats that do nothing other than increase maintenance costs. If your first preference as a client developer is for JSON, it's really not that hard for you to go get a library to receive it in XML instead,

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Tyler Romeo
The RFC doesn't seem to have gotten much interest (only a burst of edits from Krinkle in August and then it died). But interesting nonetheless. The one thing I do know is that if this were to be implemented, it would probably be pretty complex. It would have to support at least a couple of

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
The problem is that while XML is widely accepted standard supported on all platforms and languages, JSON, even if it might be better, is not so well supported in this moment. For this reason I think it would be cool to be able to offer multiple outputs. In the end, as you said, it's not that hard

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
OK Inserted this to hackaton topics as well... On Fri, Mar 1, 2013 at 3:02 PM, Tyler Romeo tylerro...@gmail.com wrote: The RFC doesn't seem to have gotten much interest (only a burst of edits from Krinkle in August and then it died). But interesting nonetheless. The one thing I do know is

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
I have not yet found a good and stable library for JSON parsing in c#, should you know some let me know :) However, I disagree with I feel like such a project would take an insane amount of resources to develop. If we wouldn't make it insanely complicated, it won't take insane amount of time ;).

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
I still don't see it as too much complex. Matter of month(s) for volunteers with limited time. However I quite don't see what is so complicated on last 2 points. Given the frequency of updates it's most simple to have the client (user / bot / service that need to read the feed) open the

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Yuvi Panda
0mq? RabbitMQ? Seem to fit the use case pretty well / closely. -- Yuvi Panda T http://yuvi.in/blog ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
Closely, but seems a bit overcomplicated to me. What I proposed is as simple as you could just use telnet to retrieve the last changes. In rabbitMQ for example you need to use 3rd libraries for client so that you can connect to server and obtain some data... But I don't have a problem with using

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Chad
On Fri, Mar 1, 2013 at 12:55 AM, Petr Bena benap...@gmail.com wrote: Hi, I think that irc feed of recent changes is working great, but there is still a lot of space for improvement. As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Asher Feldman
I don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The rate of uploads and edits at flickr in 2010

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Petr Bena
web frontend you say? if you compare the raw data of irc protocol (1 rc feed message) and raw data of a http request and response for one page consisting only of that 1 rc feed message, you will see a huge difference in size and performance. Also all kinds of authentication required doesn't seem

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Tyler Romeo
On Fri, Mar 1, 2013 at 11:46 AM, Asher Feldman afeld...@wikimedia.orgwrote: don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Asher Feldman
On Friday, March 1, 2013, Petr Bena wrote: web frontend you say? if you compare the raw data of irc protocol (1 rc feed message) and raw data of a http request and response for one page consisting only of that 1 rc feed message, you will see a huge difference in size and performance. I

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Asher Feldman
On Friday, March 1, 2013, Tyler Romeo wrote: On Fri, Mar 1, 2013 at 11:46 AM, Asher Feldman afeld...@wikimedia.orgjavascript:; wrote: don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Tim Starling
On 01/03/13 05:22, Petr Bena wrote: I believe it would require to create a new daemon (preferably written in c++) which I am willing to write, that could do similar what the ircd does right now. And that is delivery of new change to all connected clients. When this has been proposed in the

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Jay Ashworth
- Original Message - From: Tyler Romeo tylerro...@gmail.com I think a very light weight proxy that only passes subscribe commands to redis would work. A read only redis slave could be provided but I don't think it includes a way to limit what commands clients can run,

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Tim Starling
On 01/03/13 06:04, Petr Bena wrote: The problem is that while XML is widely accepted standard supported on all platforms and languages, JSON, even if it might be better, is not so well supported in this moment. For this reason I think it would be cool to be able to offer multiple outputs.

Re: [Wikitech-l] switching to something better than irc.wikimedia.org

2013-03-01 Thread Victor Vasiliev
On 03/01/2013 03:55 AM, Petr Bena wrote: As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more advanced. My suggestion is to create some kind of feed that would be in machine parseable format, like XML Whatever you do, please, don't use