Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-17 Thread Michael Stone
On Sun, Mar 15, 2009 at 06:30:17PM -0400, Daniel Drake wrote:
>2009/3/15 Martin Langhoff :
>> "Client" code for Gadget seems to be integrated in the Telepathy new
>> Sugar present on the SoaS images. The server side -- the proper gadget
>> code -- isn't on any XS, and I haven't seen or tested it (lack of time
>> :-( )
>>
>> Even if I had, it's a ton of new code, a lot more adventurous than
>> what we're doing w moodle. So short/midterm, following ejabberd+moodle
>> is lower risk from the perspective of a deployment today.
>
>One thing I still don't understand about gadget... how does it
>actually solve the problem? I'm assuming the problem it solves is lack
>of partitioning, and the fact that the neighborhood view becomes kind
>of impossible after 50 users, etc. Right?

Wrong. Gadget is primarily intended to reduce the bandwidth consumed by Gabble
under the load generated by Sugar.

>So what does gadget do? 

Think of it as a server-side keyword search engine which you can query for
lists of matching people and activities. The purported bandwidth reduction
comes from sending each client only what it asks for instead of everything,
which is what the shared roster hack does.

>Is there a new client side UI for electing groups? Who chooses, the kids or
>the teachers? etc.

Guillaume filed 

   https://dev.laptop.org/ticket/7711

eight months ago but the absence of comments in that ticket and the current
paucity of results in 

   http://dev.sugarlabs.org/search?q=gadget

suggests to me that the Sugar folks have completely ignored the necessary UI
work in favor of more pressing issues.

Regards,

Michael

---

To understand how Gadget works, read

   http://wiki.laptop.org/go/XMPP_Component_Protocol

and skim the contents of

   http://dev.laptop.org/git/projects/gadget/tree/gadget

paying particular attention to the automated tests. Then, if you're feeling
brave, read the Gabble source code:

   http://git.collabora.co.uk/?p=telepathy-gabble.git;a=tree;hb=master

paying particular attention to the files whose names contain 'olpc'.

___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-17 Thread Dave Bauer
On Tue, Mar 17, 2009 at 3:08 PM, Martin Langhoff
wrote:

> On Wed, Mar 18, 2009 at 8:04 AM, Dave Bauer  wrote:
> > Thanks, I guess I will try it out and let you know what I learn.
>
> Fantastic! One interesting data point would be whether the new
> software stack (sugar with the new telepathy-gadget) is backwards
> compat with an XS that has no gadget.
>

Preliminary tests show it works fine. We have been testing SoaS against XS
0.5.2 and it performs the same as jabber.sugarlabs.org (which I believe has
gadget installed).

Dave


>
> cheers,
>


>
>
>
>
> m
> --
>  martin.langh...@gmail.com
>  mar...@laptop.org -- School Server Architect
>  - ask interesting questions
>  - don't get distracted with shiny stuff  - working code first
>  - http://wiki.laptop.org/go/User:Martinlanghoff
>



-- 
Dave Bauer
d...@solutiongrove.com
http://www.solutiongrove.com
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-17 Thread Martin Langhoff
On Wed, Mar 18, 2009 at 8:04 AM, Dave Bauer  wrote:
> Thanks, I guess I will try it out and let you know what I learn.

Fantastic! One interesting data point would be whether the new
software stack (sugar with the new telepathy-gadget) is backwards
compat with an XS that has no gadget.

cheers,



m
-- 
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-17 Thread Dave Bauer
On Tue, Mar 17, 2009 at 2:57 PM, Martin Langhoff
wrote:

> 2009/3/18 Dave Bauer :
> > Martin, will this peacefully coexist with the Moodle/ejabberd work you
> are
> > doing?
>
> I am busy enough that I haven't had time to study that aspect. And
> won't for the next couple months I think :-/ there is more urgent work
> at hand.
>

Thanks, I guess I will try it out and let you know what I learn.

Dave


>
> cheers.,
>
>
> m
> --
>  martin.langh...@gmail.com
>  mar...@laptop.org -- School Server Architect
>  - ask interesting questions
>  - don't get distracted with shiny stuff  - working code first
>  - http://wiki.laptop.org/go/User:Martinlanghoff
>



-- 
Dave Bauer
d...@solutiongrove.com
http://www.solutiongrove.com
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-17 Thread Martin Langhoff
2009/3/18 Dave Bauer :
> Martin, will this peacefully coexist with the Moodle/ejabberd work you are
> doing?

I am busy enough that I haven't had time to study that aspect. And
won't for the next couple months I think :-/ there is more urgent work
at hand.

cheers.,


m
-- 
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-16 Thread Morgan Collett
On Mon, Mar 16, 2009 at 00:30, Daniel Drake  wrote:
> 2009/3/15 Martin Langhoff :
>> "Client" code for Gadget seems to be integrated in the Telepathy new
>> Sugar present on the SoaS images. The server side -- the proper gadget
>> code -- isn't on any XS, and I haven't seen or tested it (lack of time
>> :-( )
>>
>> Even if I had, it's a ton of new code, a lot more adventurous than
>> what we're doing w moodle. So short/midterm, following ejabberd+moodle
>> is lower risk from the perspective of a deployment today.
>
> One thing I still don't understand about gadget... how does it
> actually solve the problem? I'm assuming the problem it solves is lack
> of partitioning, and the fact that the neighborhood view becomes kind
> of impossible after 50 users, etc. Right?
>
> So what does gadget do? Is there a new client side UI for electing
> groups? Who chooses, the kids or the teachers? etc.

Gadget:
* Shows people you have explicitly friended (which is trivial, that's
just a conventional non-shared roster)
* Shows you some random people on the server so you can bootstrap your friends
* Has a search API so you can search for specific nicks (I don't know
if this is implemented client-side yet.)

The people you need to talk to about it are Dafydd Harries and
Guillaume Desmottes at Collabora. I don't know if they're still on
this list, but you can reach them through on the sugar-devel list.

Regards
Morgan
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-15 Thread Martin Langhoff
On Mon, Mar 16, 2009 at 11:30 AM, Daniel Drake  wrote:
> So what does gadget do? Is there a new client side UI for electing
> groups?

Good question. I think that

 - Gadget actually extends XMPP, so there are new XMPP msgs on the
wire -- which means you need new code on the client side -
Telepathy-Gadget

 - Gadget itself (which is on the server side) remembers who you've
interacted with recently, so above a certain threshold it'll show
those on the Network View

 - If you use the search on the network view, it'll also run a search
against Gadget so new nodes may appear in the Network View (not just
the greying out of non-matches)

there are other changes as well, I think. Right now a shared activity
is - on XMPP - a 'chatroom'. With Telepathy-Gadget and a Gadget server
they are implemented as something custom and specific. I don't know
what extra tricks you can get out of that ;-)

OTOH, it stops being generic XMPP so it kind of binds you to
ejabberd+Gadget (maybe other XMPP servers are supported as well?). And
you can no longer interop with any random XMPP client.

cheers,



martin
-- 
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-15 Thread Daniel Drake
2009/3/15 Martin Langhoff :
> "Client" code for Gadget seems to be integrated in the Telepathy new
> Sugar present on the SoaS images. The server side -- the proper gadget
> code -- isn't on any XS, and I haven't seen or tested it (lack of time
> :-( )
>
> Even if I had, it's a ton of new code, a lot more adventurous than
> what we're doing w moodle. So short/midterm, following ejabberd+moodle
> is lower risk from the perspective of a deployment today.

One thing I still don't understand about gadget... how does it
actually solve the problem? I'm assuming the problem it solves is lack
of partitioning, and the fact that the neighborhood view becomes kind
of impossible after 50 users, etc. Right?

So what does gadget do? Is there a new client side UI for electing
groups? Who chooses, the kids or the teachers? etc.

Daniel
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-15 Thread Martin Langhoff
On Wed, Mar 11, 2009 at 2:12 PM, Bryan Berry  wrote:
> Here are some notes from a short IRC conversation I had w/ Rob Mcqueen,
> the lead developer of Telepathy

Cool - thanks for posting it.

"Client" code for Gadget seems to be integrated in the Telepathy new
Sugar present on the SoaS images. The server side -- the proper gadget
code -- isn't on any XS, and I haven't seen or tested it (lack of time
:-( )

Even if I had, it's a ton of new code, a lot more adventurous than
what we're doing w moodle. So short/midterm, following ejabberd+moodle
is lower risk from the perspective of a deployment today.

OTOH, if you're feeling the experimental mood, the ejabberd we have
now on XS-0.5.2 is compatible with gadget. Go crazy ;-)

cheers,



m
-- 
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


Re: [Server-devel] notes on scaling ejabberd for the XO's

2009-03-11 Thread Dev Mohanty
More info available on the wiki at:
http://wiki.laptop.org/go/Ejabberd_Configuration#Patches

On 3/11/09, Bryan Berry  wrote:
>
> Here are some notes from a short IRC conversation I had w/ Rob Mcqueen,
> the lead developer of Telepathy
>
> transcript of conversation on #sugar
> bemasc: bernie: I am concerned about the fact that in the default
> schoolserver set up all users are in one giant shared roster
>  RESOLVED, ALREADYFIXED (but not in any deployments, or in the
> UI)
>  and the resulting chatter slows down the XO/Sugar
> considerably?
>  yes
>   Robot101 rwh
>   the latest versions of sugar and telepathy support using an XMPP
> component called gadget
>   instead of the shared roster
>  Robot101: so gadget fixes this?
>  yup
>   Robot101 rwh
>   you only receive push notifications about a) what Sugar has searched
> for/displaying on the neighborhood view, or b) your friends
> --> hgcphoenix (n=hc...@124.107.253.193) has joined #sugar
>  Robot101: neat, and does it work together w/ the XS?
>   Robot101: which version of sugar is it in?
> <-- hgcphoenix (n=hc...@124.107.253.193) has left #sugar
>  they went off on a complete tangent trying to hack shared
> rosters to have less mutually visible sets of people
>   we thought of that but also decided it was the bong, so we fixed it
> properly with gadget.
>  Robot101: what is the testing status of gadget?
>  it's deployed on jabber.sugarlabs.org (which is on
> collabora.co.uk)
>   seems to work fine, ejabberd seems to gradually leak memory though,
> which isn't too great
>   maybe a little much CPU usage on gadget, but nothing you couldn't
> profile
>   and I'm not familiar enough with the sugar release cycle to say where
> the support went in
>   Robot101 rwh
>   eu daytime is better to find the Sugar devs and the Collaborans who
> worked on Gadget
>   (cassidy, daf)
>  Robot101: ok, will talk w/ them later today
>  gadget was always our plan, it just took us a while to get to
> it
>  Robot101: by the way last year we tested ejabberd by streaming
> your video talk on Telepathy to 80 XO's
>  Robot101: I believe martin dropped the shared roster, and
> inside is simply using moodle to set all rosters directly.
>   s/inside/instead/
>   bemasc bernie benzea
>  bemasc: so it's still shared as in server-enforced mutual
> visibility, just in smaller groups.
>  right, but from ejabberd's perspective, it's individual rosters
>  that's exactly how shared rosters always work
>  the client thread gets a copy of the same roster at sign in
>  oh? I thought there was a patch to ejabberd required.
>  yes, he's patched it to source the shared roster from moodle,
> I'd imagine
>  martin seemed to say that he could use a totally stock ejabberd
>  oh, right. sql query or something. our patches were just
> extending the built-in shared roster to a) work properly (deal with
> dynamic additions and removals) and b) support a group of online users
> rather than everyone
>
>
> --
> Bryan W. Berry
> Technology Director
> OLE Nepal, http://www.olenepal.org
>
> ___
> Server-devel mailing list
> Server-devel@lists.laptop.org
> http://lists.laptop.org/listinfo/server-devel
>
>
___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel


[Server-devel] notes on scaling ejabberd for the XO's

2009-03-10 Thread Bryan Berry
Here are some notes from a short IRC conversation I had w/ Rob Mcqueen,
the lead developer of Telepathy

transcript of conversation on #sugar
bemasc: bernie: I am concerned about the fact that in the default
schoolserver set up all users are in one giant shared roster
 RESOLVED, ALREADYFIXED (but not in any deployments, or in the
UI)
 and the resulting chatter slows down the XO/Sugar
considerably?
 yes
 Robot101 rwh
 the latest versions of sugar and telepathy support using an XMPP
component called gadget
 instead of the shared roster
 Robot101: so gadget fixes this?
 yup
 Robot101 rwh
 you only receive push notifications about a) what Sugar has searched
for/displaying on the neighborhood view, or b) your friends
--> hgcphoenix (n=hc...@124.107.253.193) has joined #sugar
 Robot101: neat, and does it work together w/ the XS? 
 Robot101: which version of sugar is it in?
<-- hgcphoenix (n=hc...@124.107.253.193) has left #sugar
 they went off on a complete tangent trying to hack shared
rosters to have less mutually visible sets of people
 we thought of that but also decided it was the bong, so we fixed it
properly with gadget.
 Robot101: what is the testing status of gadget?
 it's deployed on jabber.sugarlabs.org (which is on
collabora.co.uk)
 seems to work fine, ejabberd seems to gradually leak memory though,
which isn't too great
 maybe a little much CPU usage on gadget, but nothing you couldn't
profile
 and I'm not familiar enough with the sugar release cycle to say where
the support went in
 Robot101 rwh
 eu daytime is better to find the Sugar devs and the Collaborans who
worked on Gadget
 (cassidy, daf)
 Robot101: ok, will talk w/ them later today
 gadget was always our plan, it just took us a while to get to
it
 Robot101: by the way last year we tested ejabberd by streaming
your video talk on Telepathy to 80 XO's
 Robot101: I believe martin dropped the shared roster, and
inside is simply using moodle to set all rosters directly.
 s/inside/instead/
 bemasc bernie benzea
 bemasc: so it's still shared as in server-enforced mutual
visibility, just in smaller groups.
 right, but from ejabberd's perspective, it's individual rosters
 that's exactly how shared rosters always work
 the client thread gets a copy of the same roster at sign in
 oh? I thought there was a patch to ejabberd required.
 yes, he's patched it to source the shared roster from moodle,
I'd imagine
 martin seemed to say that he could use a totally stock ejabberd
 oh, right. sql query or something. our patches were just
extending the built-in shared roster to a) work properly (deal with
dynamic additions and removals) and b) support a group of online users
rather than everyone


-- 
Bryan W. Berry
Technology Director
OLE Nepal, http://www.olenepal.org

___
Server-devel mailing list
Server-devel@lists.laptop.org
http://lists.laptop.org/listinfo/server-devel