Matt Maier wrote:
> According to this lesson, I can store an array as a custom property
> of a stack which can be saved and loaded.
>
http://lessons.livecode.com/m/4071/l/17375-how-do-i-save-custom-properties-in-a-standalone-application
>
> Something I've done myself already is to either save the array as
> JSON in a text file or just as a binary with encode.
>
> So, in the most naive sense, it looks like I can have a key-value
> store without installing and learning anything new. But databases
> have extra functionality to deal with stuff like multiple
> simultaneous users, data integrity, and performance optimization.
>
> Can I just setup an instance of Livecode to act like an in-memory,
> key-value database? Maybe have a standalone stack act as the
> controller and have it mirror it's state in a separate stack that
> can be saved and loaded.
>
> I don't know enough about database design to frame the tradeoffs. Can
> someone who groks both Livecode and databases comment on this idea?
I think you're on a very useful path.
TL/DR version:
LiveCode's encoded arrays are a good fit for many key-value storage needs.
Typically windy version:
RDBMSes are great when you have data too large to fit into RAM, or when
the nature of the data or its workflow lends itself to being expressed
through normalized relationality.
But for simple key-value stores they can be overkill. Simple arrays
work very well for those, but we need some form of persistence.
JSON is usually the go-to solution for serializing arrays, and it's
necessary to use either it or XML when transferring data between systems
that know little about each other, since both are widely-used standards.
But when data will only be used within LiveCode we have much broader
options.
I used to use stack files for data storage, but I've come to find them
cumbersome for two reasons:
1. Purging them from memory requires careful thinking (we have no "purge
stack" command, and "delete stack" is disconcerting to use because it
can only safely be used on mainstacks, but using it on a substack will
do what it says on the tin and delete the stack object from the stack file).
2. Given property sets and properties, array notation can only be used
natively on object properties with two levels of depth. It's possible
to store entire arrays as property values, but those require that you
first extract them into a variable, and if you're going to do that you
might as well bypass the concerns of #1 by using:
My favorite key-value format is LSON, my pet name for encoded LiveCode
arrays.
I call them LSON because everyone's familiar with JSON, some are
familiar with MongoDB's BSON ("Binary JSON"), and LiveCode's encoded
arrays are similar enough in structure and use case with BSON to merit a
similar label. I find when I use anything ending with "SON"
conversations with clients become much easier; they grasp the concept
and benefit immediately.
You can serialize an array with arrayEncode, and de-serialize it back
into an array with arrayDecode. Once serialized, it can be used like
any other data, saved to disk or sent over a network.
JSON is popular because JavaScript is popular, and JSON was designed to
be de-serialized with a single function call in the JS engine, eval.
But every language that isn't JavaScript has to go through the
cumbersome process of parsing it, notoriously slow compared to the
optimized handling provided by JavaScript's machine-code-compiled
engine. JSON is a glove that perfectly fits the hand of the JavaScript
engine, but any other engine needs to do quite a bit of work to emulate
what the JS engine does so easily with it.
So when data is to be used within LiveCode, LSON is by far the better
choice. It's functionally equivalent to JSON not only terms of
serializing hierarchical data, and also in terms of the performance and
robustnesss of being able to rely on a single engine
machine-code-compiled function to serialize and de-serialize it. LSON
is a glove that perfectly fits the hand of the LiveCode engine.
Extra bonus points:
Key-value stores needn't be limited by RAM.
Studying MongoDB, CouchDB, and other large-scale key-value stores can be
quite inspiring.
Of course if your collection is really large or intended for use by many
hundreds of concurrent users, you're probably better off just using
MongoDB or CouchDB. Greg Miller has a nice CouchDB library for LiveCode
on Github (background an detail in this forum thread:
<http://forums.livecode.com/viewtopic.php?f=12&t=24925>).
But for workgroup solutions, in which the number of concurrent users may
be only a few dozen and the collection size below several tens of
thousands, it's possible to build reasonably performant client-server
solutions using LSON as the storage format, even with the limitations of
CGI.
And if you're running a VPS or dedicated server the scale can be much
larger, since you're no longer bound by the CPU and RAM constraints
imposed by CGI. The MongoDB daemon, for example, keeps its index in
RAM, used to map access to the much large disk store. This can be done
with a LiveCode-based daemon as well. The experiments I've done along
those lines have gone very well, and I believe the backend for Canella
Software's LiveCloud works similarly.
Using LiveCode-native LSON for key-value serialization opens many doors
for data storage solutions, as efficiently as JSON does for
JavaScript-based systems.
--
Richard Gaskin
Fourth World Systems
Software Design and Development for the Desktop, Mobile, and the Web
____________________________________________________________________
ambassa...@fourthworld.com http://www.FourthWorld.com
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
