Jabber ODBC XDB module

1.4 Jabber ODBC module README (Lubos Pochman)

Introduction

Why a new xdb implementation?
Simple answer is that the default xdb module xdb_file is not very
scaleble and efficient. It uses directories for each service (agent, server),
and a file for each user registered for that service. That means more
than 1,000,000 file for 1,000,000 registered users. It is almost impossible
to have as many files in one directory on Linux (and most other Unix)
systems. Yet most sites who want to run Jabber IM have a lot of registed
users, and they want to allow them an access to Jabber IM using
their Jabber server.

Is there a better backend for xdb? Looks like using database as storage media
would lift most if not all the limitations of file system implementation of xdb.

1.0 Jabber server unsupported solution

To solve the problem of "too many registered user" unsuported ODBC module
was developed by Jon Simantov <jon@cybersites.com> and
Benjamin Kahn <xkahn@cybersites.com>, thanks guys. It worked quite well
for Jabber server 1.0, and I was able to port it to suppoprt most of the
Jabber server 1.2 protocol. It stores xdb data in RDB and uses ODBC to connect
to the DB. It stores data in multiple tables and it disects the Jabber
xdb requests protocol to store/retreive data to/from DB. This is very
efficient way to access xdb data, but has one major drawback.

As I learned the hard way by porting it from Jaber server 1.0 to 1.2 and then
to 1.4, porting is very difficult and time consuming task. Any xdb request
protocol change or a new xdb request protocol introduced requires change in
xdb ODBC implementation. All that with only Jabber server (and agents) source
code available as documentation.

After couple of weeks spent on porting ODBC module to Jabber server 1.4,
I decided to scratch the original design and try something simpler, and
not as dependent on xdb request protocol changes.

1.4 Jabber server solution

I have set follwing requirements for new xdb ODBC module:

  - lift file system number of users limitation
  - make it as independent on xdb request protocol changes as possible

I've used xdb file implementation as a base model for ODBC implementation,
and tried to treat content of the stored xdb data as a black box. 

XDB information for each server and user is stored in DB (accessed using ODBC)
in standard XML format (as varchar) as it is in file based module xdb_file.

To avoid problems with variable format data like blob, text etc. which are
hard to portably access using ODBC, instead the xml data are stored in varchar
chunks (each 8000 byte size), using ordinal number to address multiple chunks.
That means there is one or more records with the same server jid and user
name differenciated by ordinal number (nxml). When the xml data are retreived,
the xml chunks are joined together. When the xml data are stored, they are
split into n xml chunks.

Following tables are used:

STORAGE:                           - stores the xml chunks
	server_jid (varchar 255)   - server name (main or transport)
	username   (varchar 255)   - user name
	nxml       (int)           - ordinal number for multiple chunks
        xml        (varchar 8000)  - XML data chunk

STORAGE_COUNT:                     - stores current number of XML data chunks in STORAGE table
	server_jid (varchar 255)   - server name (main or transport)
	username   (varchar 255)   - user name
	xmlcount   (int)           - current number of XML chunks in STORAGE table

Use im_odbc_tables.sql file to create the tables above.

Following stored procedures are used:

GET_STORAGE_XML
SET_STORAGE_XML
GET_STORAGE_COUNT
SET_STORAGE_COUNT
INSERT_STORAGE_USER
PURGE_STORAGE_XML
REMOVE_STORAGE_USER

Use im_odbc_stored_procedures.sql to create the stored procedures above.


Creating ODBC connection to Jabber DB

First make sure you have a working ODBC environment.
If you can not get any ODBC connections to work with other
programs, this module won't work either.  

I suggest to use www.easysoft.com ODBC-ODBC bridge. I have tested it
for Linux (Jabber server) to Windows (SQLServer DB) with good results
(over 1,000,000 users in the storage, storage_count tables).

It should work for any RDB supporting ODBC.


Configuring Jabber server

To configure your jabber server to use xdb_odbc:

    - build xdb_odbc (check, modify the Makefile)
    - modify jabber.xml configuration file, replace xdb section
      with xdb node in jabber_odbc_sample.xml
    - fill approrpriate DB connection info (host, db, user, pass)


1.0 Jabber ODBC module README

	Following is the content of original Jabber 1.0 README file.

	This module has been only partially tested so far.  Althought it works
in many instances, it probably has not been tested in your configuration and
will probably crash, erase your hard drive, eat your dog, etc.  We hold no
responsibility for these disasters.

	If you are still willing to try it, make sure you have a working ODBC
environment.  If you can not get any ODBC connections to work with other
programs, this module won't work either.  

	I'll say that again.  MAKE SURE YOU HAVE A WORKING ODBC ENVIRONMENT!
This won't work until you have a test ODBC program working.  I can't help you
set this up.

	Next, make sure your ODBC driver is _*/NOT/*_ multi-threaded.  If it
is, this module WILL CRASH.  (This is a problem because Jabber uses pth as it's 
threading library, and your ODBC driver most certainly will not.)

	I'll say this again too.  If, at any time, you are using a library
which lists pthreads as a dependency, this extension will not work.  You can
check this by running "ldd" on your program or library.

	Next, you should edit the sample_config.xml file to match your database 
structure.  If you wish to simply test this module, load same_database.sql into 
your database.  Note that although SQL is a standard, not all databases support 
all sections; you may need to edit this file to make it work with your
database.

	Please see the LICENSE file for information about using this module.