madIS is an extensible relational database system built upon the SQLite
database and with extensions written in Python (via APSW SQLite
wrapper). Its is developed at:
http://madis.googlecode.com
Due to madIS’ SQLite core, the database format of madIS is exactly the
same as SQLite’s one. This means that all SQLite database files are
directly usable with madIS.
In usage, madIS, feels like a lightweight personal Hive+Hadoop
programming environment, without the distributed processing capabilities
of Hadoop. Nevertheless due to its low overhead while running on a
single computer (compared to Hadoop), madIS can easily handle tens of
millions of rows on a single desktop/laptop computer.
In version 1.3 of madIS:
- A great deal of testing has been done on Linux, Windows and Mac OSX.
- madIS now only depends on APSW on all these systems, so it is easier
to be installed
- XMLPARSE was added. XMLPARSE processes its input in a streaming
fashion, and has been tested with very large (~20+ GB) XML source files
without problems.
- JPACK functions were added. Jpacks are now the preferable way to
store a set of values into a single tuple. For easy viewing and
exporting of the jpacks, their format was based on the JSON format.
- Heavy testing under Windows and Mac OSX. CLIPBOARD and CLIPOUT
virtual tables work under all OSes.
- CLIPOUT and CLIPBOARD, have been tested with Excel, Libre/Open
Office Calc, and iWork Numbers.
- Functions that return tables, can easily be coded now, by using
Python's generators (yield)
- A lot of completions (via TAB) have been added to mterm. Mterm's
completion engine can automatically complete, tables, column names,
table index names and database filenames in attach database.
In detail:
MTERM changes:
While using madIS's terminal (mterm), mterm completes (via TAB) column
names, tables names, etc. of the opened and attached databases.
Also by default mterm colours column separators and if more than 4
columns are returned, mterm "tags" the columns with numbers:
mterm> select * from deficit;
[1|1 | People's Republic of China [3|272.500 |2010
[1|2 | Japan [3|166.500 |2010
[1|3 | Germany [3|162.300 |2010
--- Column names ---
[1|Rank [2|Country [3|CAB [4|Year
Note: In mterm the column number tags are coloured red in above example
XMLPARSE:
If i wished to retrieve the date and author of madIS project's Source
Changes ATOM feed:
mterm> select * from
(XMLPARSE
'tt'
select * from file('http://code.google.com/feeds/p/madis/hgchanges/basic')
) limit 3;
2011-07-25T14:07:07Z|est...@servum
2011-07-25T14:04:09Z|est...@servum
2011-07-22T14:08:11Z|est...@servum
--- Column names ---
[1|updated [2|author_name
Query executed in 0 min. 0 sec 543 msec
mterm>
In above query, XMLPARSE is used in an "inverted form" which is easier
to write, when chaining virtual tables.
JPACKS:
Frequently, the need to store multiple values into a tuple arises while
processing data. Previously in madIS a lot of formats were used to store
all these multiple values (space separated, comma separated, tab
separated). Now JPACKs are the recommended way to store multiple values.
JPACKs are based on the JSON format, with the exception that a JPACK of
a single value is itself. Some examples are presented below:
mterm> select jpack('a');
a
mterm> select jpack('a', 'b');
["a","b"]
mterm> select jsplit(jpack('a','b','c'));
a|b|c
mterm> select jsplitv(jpack('a','b','c'));
a
b
c
mterm> select jgroup(c1) from (select 'a' as c1 union select 'b');
["a","b"]
CLIPBOARD and CLIPOUT:
If a selection of data has been made in the web browser or a spreadsheet
program then by executing the following in mterm we could access the
clipboard data as a table:
mterm> select * from CLIPBOARD();
Rank↓ |Country↓ |CAB (billion US dollars)↓ |Year↓
1 | People's Republic of China |272.500 |2010
2 | Japan |166.500 |2010
3 | Germany |162.300 |2010
--- Column names ---
[1|C1 [2|C2 [3|C3 [4|C4
Query executed in 0 min. 0 sec 204 msec
* For above data i selected the top 3 rows while browsing
http://en.wikipedia.org/wiki/List_of_sovereign_states_by_current_account_balance
.
Similarly if i wished to process and send the data to a spreadsheet
program then i could execute the following in mterm:
mterm> CLIPOUT select * from deficit;
1
--- Column names ---
[1|return_value
Query executed in 0 min. 0 sec 111 msec
and paste in a spreadsheet program.
Both CLIPBOARD and CLIPOUT are virtual tables. CLIPOUT is used in an
"inverted form" which is easier to write, when chaining queries.
-- Lefteris
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
