Re: [Server-devel] Gadget fedora package
Martin Langhoff wrote: > But I'm working on this space at the moment - switching away from the > "all online users" patch that I suspect Guillaume is talking about and > shifting to a different strategy using PostgreSQL. > > So its "mostly behind us" (emphasis on mostly) as I mentioned but > things are looking promising. The strategies I'm planning to use are > in use by several large scale sites based on ejabberd with custom > roster behaviours. I thought about having the server subdivide people into smaller mutually visible groups, particularly in G1G1 when having magic groups which try and group by network proximity or random grouping is reasonable. It works just because that the n^2 scalability problems are avoided by simply making n smaller, but I still believe that approaches based around shared rosters aren't ultimately the right thing to do. Remember that what we're talking about isn't simply whether people can see each other's presence or not, but that we rely on a bidirectional presence subscription to use PEP to publish the extra OLPC-specific properties of a buddy, as well as their currently active activity, all of the activities they're running, and the properties of all of those activities. Problems with this: * It's further from what XMPP does normally because it continues and presumes/presupposes that the server knows best who should see each other. Gadget only publishes buddy information if they subscribe to it, and only publishes activities if it's invited to it, allowing fine-grained control, and for the other cases we should allow people to punch in JIDs or find people in activities/groups and make their own friendship subscriptions. * It's inefficient because every client in an activity ends up pushing PEP nodes with verbose details of their activities, when actually if the server had a concept of the activity it would only need to be aware of one copy of this information. That means when an activity changes its properties, you get the changed activity details N times, once for each participant. Gadget receives the activity details directly from the activity once only. * It's inefficient because the server pushes all of these to all clients in the mutually-visible group, even if that's too much information for the UI to display, or it's just not relevant because that view isn't open currently. Gadget pushes changes to only those people who are interested in a certain activity. * It precludes finer-grained visibility controls - you can't set up activities which are only visibile to certain groups of people without letting those people see all of your activities, or none. Although Gadget currently allows all people on a server to query an activity it's invited to, it gives us one place to implement functionality like sharing with certain groups or list of people. * You're limited to having subsets of people who are on one server, and because we need bi-directional subscriptions for PEP to work, both servers need to agree, so you really have very few options for including other people from other servers unless you start setting up a protocol by which servers inter-agree who should see each other on users' behalves. Seems pretty tricky... Further to this Gadget offers: * Searches for buddies and activities are made out of all information available to Gadget, rather than people being split into different "silos". So even if people in different classes are using some niche activity, they can still find each other. * If server to server connections are enabled, its possible to invite a Gadget from another server (a partner school, for example) into an activity to make it searchable by people in the other school. We could even look at gadget<->gadget propogation of activities and buddies if we wanted to. > cheers, > > martin Regards, Rob -- Robert McQueen +44 7876 562 564 Director, Collabora Ltd. http://www.collabora.co.uk ___ Server-devel mailing list server-de...@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Gadget fedora package
On Thu, Dec 11, 2008 at 3:01 PM, Martin Langhoff <[EMAIL PROTECTED]> wrote: > Note - I have *no* idea of what is that server running. > > But I'm working on this space at the moment - switching away from the > "all online users" patch that I suspect Guillaume is talking about and > shifting to a different strategy using PostgreSQL. > > So its "mostly behind us" (emphasis on mostly) as I mentioned but > things are looking promising. The strategies I'm planning to use are > in use by several large scale sites based on ejabberd with custom > roster behaviours. When you have something working, it would be good to setup a server with it so that we can play with it and see how reliable it is and how well it scales. Marco ___ Server-devel mailing list [EMAIL PROTECTED] http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Gadget fedora package
On Thu, Dec 11, 2008 at 11:54 AM, Marco Pesenti Gritti <[EMAIL PROTECTED]> wrote: > On Thu, Dec 11, 2008 at 2:51 PM, Marco Pesenti Gritti > <[EMAIL PROTECTED]> wrote: >>> With the "OMG! ejabberd's memory and roster mgmt are Out Of Control" >>> thing mostly behind us, I want to have a clear picture of why and how >>> Gadget fits into the picture. And at what (cpu, memory) cost, >>> specially for the 3K users scenarios we're looking at. > > Just to share my experience. We was testing with > schoolserver.media.mit.edu and we run into several presence > inconsistencies, which Guillame says are shared roster bugs (I don't > have the numbers anymore, unfortunately). Note - I have *no* idea of what is that server running. But I'm working on this space at the moment - switching away from the "all online users" patch that I suspect Guillaume is talking about and shifting to a different strategy using PostgreSQL. So its "mostly behind us" (emphasis on mostly) as I mentioned but things are looking promising. The strategies I'm planning to use are in use by several large scale sites based on ejabberd with custom roster behaviours. cheers, martin -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Server-devel mailing list [EMAIL PROTECTED] http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Gadget fedora package
On Thu, Dec 11, 2008 at 2:51 PM, Marco Pesenti Gritti <[EMAIL PROTECTED]> wrote: >> With the "OMG! ejabberd's memory and roster mgmt are Out Of Control" >> thing mostly behind us, I want to have a clear picture of why and how >> Gadget fits into the picture. And at what (cpu, memory) cost, >> specially for the 3K users scenarios we're looking at. Just to share my experience. We was testing with schoolserver.media.mit.edu and we run into several presence inconsistencies, which Guillame says are shared roster bugs (I don't have the numbers anymore, unfortunately). Marco ___ Server-devel mailing list [EMAIL PROTECTED] http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Gadget fedora package
This is really a question for Collabora :) Ccing them... On Thu, Dec 11, 2008 at 2:49 PM, Martin Langhoff <[EMAIL PROTECTED]> wrote: > On Thu, Dec 11, 2008 at 9:43 AM, Marco Pesenti Gritti > <[EMAIL PROTECTED]> wrote: >> Just fyi, I submitted a gadget fedora package for review. It's going >> to require ejabberd 2.0.2. >> >> https://bugzilla.redhat.com/show_bug.cgi?id=475971 > > When you have a bit of time, I am still keen on understanding how does > Gadget fit in the wider picture. > > With the "OMG! ejabberd's memory and roster mgmt are Out Of Control" > thing mostly behind us, I want to have a clear picture of why and how > Gadget fits into the picture. And at what (cpu, memory) cost, > specially for the 3K users scenarios we're looking at. > > In other words, we are finding that ejabberd is very amenable to > having its behaviour changed in interesting ways with > > - minimalistic Erlang plugins > - poking at its internal mnesia DB via the xml-rpc plugin > - using an external DB instead of Mnesia, and having external > programs manipulate the DB > > all of these things keep us on using standard XMPP (so our client and > server are more generic, and interchangeable) and from a scalability > POV avoid adding additional processes (except for the DB). > > OTOH, I'm not against Gadget. It's just that it's a big unknown to me > at a stage where -- without that much effort -- we seem to be getting > ejabberd under control and playing nice. > > cheers, > > > > m > -- > [EMAIL PROTECTED] > [EMAIL PROTECTED] -- School Server Architect > - ask interesting questions - don't get distracted with shiny stuff > - working code first > - http://wiki.laptop.org/go/User:Martinlanghoff > ___ Server-devel mailing list [EMAIL PROTECTED] http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Gadget fedora package
On Thu, Dec 11, 2008 at 9:43 AM, Marco Pesenti Gritti <[EMAIL PROTECTED]> wrote: > Just fyi, I submitted a gadget fedora package for review. It's going > to require ejabberd 2.0.2. > > https://bugzilla.redhat.com/show_bug.cgi?id=475971 When you have a bit of time, I am still keen on understanding how does Gadget fit in the wider picture. With the "OMG! ejabberd's memory and roster mgmt are Out Of Control" thing mostly behind us, I want to have a clear picture of why and how Gadget fits into the picture. And at what (cpu, memory) cost, specially for the 3K users scenarios we're looking at. In other words, we are finding that ejabberd is very amenable to having its behaviour changed in interesting ways with - minimalistic Erlang plugins - poking at its internal mnesia DB via the xml-rpc plugin - using an external DB instead of Mnesia, and having external programs manipulate the DB all of these things keep us on using standard XMPP (so our client and server are more generic, and interchangeable) and from a scalability POV avoid adding additional processes (except for the DB). OTOH, I'm not against Gadget. It's just that it's a big unknown to me at a stage where -- without that much effort -- we seem to be getting ejabberd under control and playing nice. cheers, m -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Server-devel mailing list [EMAIL PROTECTED] http://lists.laptop.org/listinfo/server-devel
[Server-devel] Gadget fedora package
Just fyi, I submitted a gadget fedora package for review. It's going to require ejabberd 2.0.2. https://bugzilla.redhat.com/show_bug.cgi?id=475971 Marco ___ Server-devel mailing list [EMAIL PROTECTED] http://lists.laptop.org/listinfo/server-devel
