Re: Future of jabberd

[Tomasz Sterna]

W dniu 30.05.2016, pon o godzinie 20∶05 +0200, użytkownik Tomasz Sterna
napisał:
> I am still not fond of the synchronous nature of storage interface,
> but changing this would require rewriting it from scratch.
> Also having an asynchronous interface for immediate in nature
> backends like file backend or BDB, would require arm twisting.

Let's have it in the open:
https://github.com/jabberd2/jabberd2/issues/120




-- 
smoku @ http://abadcafe.pl/ @ http://xiaoka.com/



signature.asc
Description: This is a digitally signed message part

Re: Future of jabberd

[Tomasz Sterna]

W dniu 31.05.2016, wto o godzinie 16∶31 +, użytkownik Shawn Debnath
napisał:
> Re 1. Merging separate daemons to one.
> I am not sure if merging them into one process is the best idea. It
> sure is convenient, but isolation is a nice thing to have. Specially,
> when you have unauthorized users hammering on C2S.

Oh. I wasn't clear on that.
You will have the option to choose which components you want to run in
process, so if you wish you can keep the current setup of having one
process for each component. Possibly on different machines.

I just want the simple setup to have the option to run all components
in one process.


-- 
smoku @ http://abadcafe.pl/ @ http://xiaoka.com/



signature.asc
Description: This is a digitally signed message part

Re: Future of jabberd

[Shawn Debnath]

I agree with Tomasz that there are changes that are definitely needed 
given how much technology has changed since jabberd2 was first released.

Re 1. Merging separate daemons to one.

I am not sure if merging them into one process is the best idea. It sure is
convenient, but isolation is a nice thing to have. Specially, when you
have unauthorized users hammering on C2S. To be honest, I believe, that
instead of going micro – go macro. Think distributed scalable server platform
rather than ease of running on one machine. In today’s services oriented world,
this approach will find a better home I believe. Which brings me to router. 
Today, it’s a bottleneck. There are several ways, as mentioned in Tomasz’s 
email, to fix this issue and something that would be great to tackle this as
part of this re-architecture. MoongooseIM is a good place to look at for
scalable XMPP platform architecture.

4. Configuration interface

Subscription based model works well and its okay to force developers
to write config change handlers. It’s 2016, time to not argue about this
point given the advantages.

5. JavaScript support.

Personally, C/C++ is fine, however, if folks insist, would be good to keep
the existing setup and allow for JS SM components as a compile
time option.

6. String handling

Something that would be nice to see is allowing SM modules or others
to pre-process the PKT to extract as much meaningful data once (in the
beginning) and then passing that object around. Currently, at each
stage of the SM module call chain, we have to re-process the PKT or NAD
to extract the same data over and over again. Unnecessary work that 
can be avoided easily. Easy to do in C++ but a bit awkward in C.


On 5/30/16, 1:31 AM, "Tomasz Sterna"  wrote:

>There are some things we already talked about on Gitter channel [1],
>but I would like to raise them on the ML for peer review.
>
>As you can see from late activity, jabberd2 project is far from dead.
>With the inclusion of new features like WebSocket support, C99 code
>compatibility, IPv6 improvements, modern TLS handling, SASL Anonymous,
>password hashing, CRAM-MD5 and more... it is not a stale codebase
>anymore.
>
>But it is far from modern too...
>There are some changes I would like to introduce in the near future and
>I would like to hear your thoughts about:
>
>1. Merging separate daemons to one.
>Current design of jabberd2 with separate router, sm, c2s, s2s processes
>is designed to allow nice separation of concerns and distribution of
>processing. Separate processes are proved to be better approach than
>threads too.
>But most installations of jabberd are not distributed, with one
>instance of each component. Especially when c2s and sm got vhost
>support and are able to handle more than one domain.
>Also, modern OS architectures are tuned for event processing rather
>than multithreading, so event based architecture is better suited for
>them. Even jabberd2 process internally is event based on MIO.
>So, it makes sense to allow for running all component instances in one
>process, especially on amateur, low load servers.
>Merging processes will allow for having one main loop only, so
>maintaining bugfixes in it will be easier (main.c of all processes is a
>copy-paste, with all the bugs, so bugs are also multiplied).
>
>2. Phasing out MIO.
>This is closely related to above. MIO used by jabberd2 does not have
>clerar main loop support, which is implemented separately in each
>component main.c and is hardly pluggable.
>Also, the way MIO is implemented (in .h file, with platform specific
>bits in .c) makes it a maintanance nightmare.
>I would really like to replace it with a modern, upstream maintained
>event library. The nicest one I know is libuv, which also gives us nice
>platform independence layer.
>I already have a working c2s port to libuv as a PoC.
>
>3. Phasing out router.
>router component is the one binding all the others.
>In current design it is the single point of failure. Other components
>already support multiple instances, but router proved to be difficult
>to multiply.
>The most radical, yet compelling solution to this problem is getting
>rid of the router at all. There are many cooked solutions for local
>packet distribution, which Local Message Bus [2] looks like most
>promising solution. I would see either Mbus [3] or NN_BUS [4] taking
>role of router component.
>The added advantage of using a Message Bus is the ability to connect to
>the bus with alternative implementations to perform own actions.
>i.e. having the ability to use CLI tools to eavesdrop and send messages
>to the bus proved to be priceless when I implemented a PoC of the Bus
>in experimental jabberd branch.
>Bus also solves the problem of distribution - it is up to the
>deployment administrator whether one sets up local, one-machine only
>bus or a network distributed one.
>
>4. Configuration interface.
>A the moment jabberd is configured with static XML files loaded at

Re: jabberd-2.4.0 release

[li...@lazygranch.com]

On Tue, 31 May 2016 09:52:38 +0200
Tomasz Sterna  wrote:

> W dniu 31.05.2016, wto o godzinie 00∶39 -0700, użytkownik
> li...@lazygranch.com napisał:
> > ./configure --with-extra-library-path /usr/local/lib --with-extra-
> > include-path /usr/local/include
> > yielded
> > 
> > checking build system type... /usr/local/lib
> > configure: error: invalid value of canonical build
> 
> Double-dash options format is:
> --long-option=value
> 
> So, you need:
> ./configure --with-extra-library-path=/usr/local/lib
> --with-extra-include-path=/usr/local/include
> 
> 

That works. It isn't finding the config files which I assume I can fix
on the command line. The point is the new binary is running. Thanks for
your help.

Re: jabberd-2.4.0 release

[Tomasz Sterna]

W dniu 31.05.2016, wto o godzinie 00∶39 -0700, użytkownik
li...@lazygranch.com napisał:
> ./configure --with-extra-library-path /usr/local/lib --with-extra-
> include-path /usr/local/include
> yielded
> 
> checking build system type... /usr/local/lib
> configure: error: invalid value of canonical build

Double-dash options format is:
--long-option=value

So, you need:
./configure --with-extra-library-path=/usr/local/lib 
--with-extra-include-path=/usr/local/include


-- 
 /o__ In case of injury notify your superior immediately. He'll kiss it and
(_<^' make it better.

Re: jabberd-2.4.0 release

[li...@lazygranch.com]

On Sat, 28 May 2016 08:16:50 +0200
Tomasz Sterna  wrote:

> W dniu 27.05.2016, pią o godzinie 19∶14 -0700, użytkownik
> li...@lazygranch.com napisał:
> > Do you have both expat and its headers installed?
> > Verify headers:
> > /usr/local/include
> > 
> > So is this a flag or environment variable I need to set?
> 
> Yes.
> 
> $ ./configure --help
> [...]
>   --with-extra-include-path
>   use additional include paths
>   --with-extra-library-path
>   use additional library paths (remember to
> update
>   /etc/ld.so.conf too)

I'm guessing the library command you suggested is ldconfig.

In any event, 
./configure --with-extra-library-path /usr/local/lib --with-extra-include-path 
/usr/local/include
yielded

checking build system type... /usr/local/lib
configure: error: invalid value of canonical build

I'm just going to wait for the freebsd port. 

Re: Future of jabberd

[Ed - 0x1b, Inc.]

On Mon, May 30, 2016 at 9:32 PM, Tomasz Sterna  wrote:
> W dniu 30.05.2016, pon o godzinie 10∶31 +0200, użytkownik Tomasz Sterna
> napisał:
>> 7. DBI interface to RDBM.
>
> Just one more question.
>
> Do you (ML) have a use case for having SM storage in SQL?
> Is it just for distributed SM only?
> Maybe it is not worth the effort and we should just drop it and embed
> something like LMDB [1] in?
>
> I do see value of having SQL backend for authreg, to integrate with
> existing userbase, but SM storage? Does it really need to be
> abstracted?
>
>
> [1] https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database
>
> --
>  /o__
> (_<^'  The best cure for insomnia is to get a lot of sleep. -W.C. Fields
>

Congratulations - you have Jabberd3 outlined, um.. so long as the name
isn't taken yet? In terms of "heavy", if it runs in OBSD on a
BeagleBone Black or Debian on a Rasperry Pi Zero, then I think you're
OK. Running on an Arduino is a special type of crazy - not that i
wouldn't mind seeing it done. Running Jabberd3* as part of a website
would be a new kind of distributed. If going full Javascript - this
might be interesting: http://electron.atom.io/

In any case - Thank You very much for a great XMPP server - keep having fun
*just trying it on - Jabberd11? too binary?