Re: [sqlite] SQLite as a IPC mechanism
Joshua D. Boyd wrote: > I have a system that currently consists of 2 C programs and 3 python > programs. Currently the python programs transfer data between > themselves via pickles. The C programs transfer data between themselves > via streaming structs, and the C programs talk to one of the python > programs via a fairly ugly text over socket method. All of the programs > are threaded. Since you posted using a Linux mailer I'd assume your system is Linux based. I think you'll find that dbus is an excellent solution to your requirements and it works just fine with C and python based programs. http://en.wikipedia.org/wiki/D-Bus http://www.freedesktop.org/wiki/Software/dbus D-Bus deals with threading just fine. It also has a nice notification mechanism named 'signals'. Roger - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLite as a IPC mechanism
Joshua, I don't think sqlite was designed to be used that way. It would work very well for your persistent data, that is written to disk. However, not so well using it as an IPC. The database is locked, not the table, row or even page. The entire DB is locked when you write (insert, update or delete) data. IPC, is best done as either Message Queues depending upon your data volumes. Shared memory with appropriate semaphores/ Mutexes. Or even as one person posted Mmap files. You'll stilll need a mutex or a semaphore to write. Ken "Joshua D. Boyd" <[EMAIL PROTECTED]> wrote: I have a system that currently consists of 2 C programs and 3 python programs. Currently the python programs transfer data between themselves via pickles. The C programs transfer data between themselves via streaming structs, and the C programs talk to one of the python programs via a fairly ugly text over socket method. All of the programs are threaded. Of the data being communicated, some of it must also be saved to disk, and other pieces go away after a reset. All told there is only about 4 k of stuff saved. I am wondering about using SQLite to communicate between the programs. I'd use two databases. One on a flash disk for the data that needs to be saved, and the other database would somehow be in a ram disk. Each Db would have 1 table, and the fields would be key, type, val. Most fields would only be written to by one or two sources, but would be read from by nearly all processes. Is this a stupid use of SQLite? I can't quite seem to find anyone using it like this. I am a little concerned about page locking as opposed to row locking, but I think I can work around that. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLite as a IPC mechanism
Seems like it would work, but maybe overkill. What's stopping you from working out a fairly direct protocol to exchange data with? Sending key, type, val for all of your IPC seems reasonable. A mem-mapped file, a local socket or a network socket seem reasonable, depending on the structure of the system. Clay Joshua D. Boyd wrote: > I have a system that currently consists of 2 C programs and 3 python > programs. Currently the python programs transfer data between > themselves via pickles. The C programs transfer data between themselves > via streaming structs, and the C programs talk to one of the python > programs via a fairly ugly text over socket method. All of the programs > are threaded. > > Of the data being communicated, some of it must also be saved to disk, > and other pieces go away after a reset. All told there is only about 4 > k of stuff saved. > > I am wondering about using SQLite to communicate between the programs. > I'd use two databases. One on a flash disk for the data that needs to > be saved, and the other database would somehow be in a ram disk. Each > Db would have 1 table, and the fields would be key, type, val. Most > fields would only be written to by one or two sources, but would be read > from by nearly all processes. > > Is this a stupid use of SQLite? I can't quite seem to find anyone using > it like this. I am a little concerned about page locking as opposed to > row locking, but I think I can work around that. > > - > To unsubscribe, send email to [EMAIL PROTECTED] > - > > -- Lazarus Registration http://www.lazarusid.com/registration.shtml - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] SQLite as a IPC mechanism
I have a system that currently consists of 2 C programs and 3 python programs. Currently the python programs transfer data between themselves via pickles. The C programs transfer data between themselves via streaming structs, and the C programs talk to one of the python programs via a fairly ugly text over socket method. All of the programs are threaded. Of the data being communicated, some of it must also be saved to disk, and other pieces go away after a reset. All told there is only about 4 k of stuff saved. I am wondering about using SQLite to communicate between the programs. I'd use two databases. One on a flash disk for the data that needs to be saved, and the other database would somehow be in a ram disk. Each Db would have 1 table, and the fields would be key, type, val. Most fields would only be written to by one or two sources, but would be read from by nearly all processes. Is this a stupid use of SQLite? I can't quite seem to find anyone using it like this. I am a little concerned about page locking as opposed to row locking, but I think I can work around that. - To unsubscribe, send email to [EMAIL PROTECTED] -
