Tom Glod wrote:
> no reason why it wouldn't work... but keep 2 things in mind
>
> if you use community edition, the number of HHTP Requests you can make
> to the same domain at one time is exactly 1.
How many are needed from a single client?
While it's unfortunate the LiveCode Community Edition is the only open
source scripting language I know of that doesn't have CURL support
available in its community, that's a problem that can be fixed here as
it has been for others: it would be possible for someone in the
community to wrap the most relevant subset affordably. HTTPS alone
would take care of at least 80% of real-world needs in our API-driven world.
And in the meantime, the limit for domain access is in the scripted
libURL, so a modded version could up that (though I wouldn't take it too
high, for the same reasons browsers default to a small number of
connections to a single host).
> For a system like this, it could be better to save to an sql because
> seperate TXT files would be a lot of IO calls.
How would the method used for moving data from disk to socket affect
throughput for socket comms?
Ultimately all persistent storage needs to write to disk. MySQL is very
efficient, but does a lot of complex B-tree traversal. It's possible
(for _very_ limited use cases) to outperform it for some forms of simple
retrieval in LC script.
You'd never match even a small fraction of the full scope of DB features
without losing that advantage many times over. But if you knew in
advance that the only thing you ever needed to do was simple retrieval
your options are broad. Even writes aren't too bad under some circumstances.
Another concern for larger systems may be inode depletion: if you have
a lot of records and each record is a separate file, unless you tune the
file system from its defaults you're limited to roughly total-disk-kb/4
for the number of files (Ext4 and most others these days use a default
4k block size).
> I'd be curious to see the performance of using LC Arrays as Database.
Poor as CGI, promising as daemon.
The ability to serialize arrays in LC is very nice, but as intensive as
one would imagine for the task: beyond the ftstat and fread, it needs
to parse the data, extracting each element from length indicators,
translating numbers (which are serialized in binary form) by type
indicators, and tucking what it finds into an array, key and element by
key and element. Certainly faster in the engine's machine code than
trying to do it in script, but it's a lot of work no matter who's doing it.
I was doing some measurements on this the other day, exploring options
for server storage that might be reasonably performant while more
portable than MySQL (and unencumbered by GPL in case I decide to ship a
complete solution from it).
One file was plain text, with keys longer than we commonly find but much
shorter than the max of 255 (35 chars) as item 1 of each line, and a
10-character integer as the value in item 2.
The second file was that same data in array form, stored on disk as LSON.
The test was for CGI, so each iteration reads the file from disk and
obtains the value associated with what happens to be the last key in the
text file. I chose the last key specifically because it would be the
worst case for lineoffset, while of course for arrays it makes almost no
difference.
But even weighted against using lineoffset in a text file, the overhead
of arrayDecode more than ate up any benefits of using arrays for a
simple single lookup: the LSON file took nearly 8 times longer for
100,000 keys:
Text: 21.8 ms
LSON: 167.9 ms
All that said, the overhead of LSON only applies for CGIs, where each
request is effectively working from a cold boot, and any files used need
to be read and unpacked each time.
As a daemon, the array would already be in memory, completely avoiding
the overhead of deserialization.
In a broad sense that's more or less how MongoDB works: a key-value
store in which the index is RAM-bound, with data on disk found by
pointers in the index.
Using a CouchDB-like logfile method (append is among the faster disk
write options), one could get pretty good performance for storing any
arbitrary data; kinda like have one big array on disk, but with the
added benefit of built-in versioning.
But this is ideal only for limited use cases, in which both of these
conditions are met: shared hosting where you have no control over the
DBs you can use, a preference for document-style NonSQL storage.
If you're really concerned about C10k, you're probably not on a shared
host (or you'll soon find out why you don't want to be on a shared host
for that ).
And if you're on a well-equipped VPS or dedicated server, there's
probably no reason why you wouldn't just use MongoDB or CouchDB if you
prefer those. Compiled to machine code they'll give you not only far
better performance than any scripted solution, but far more
