On 5/24/20 11:42 AM, Jan Lehnardt wrote:
On 24. May 2020, at 17:32, Miles Fidelman <[email protected]> wrote:
Thanks Jan, and a follow-up, below:
On 5/24/20 4:51 AM, Jan Lehnardt wrote:
On 23. May 2020, at 18:57, Miles Fidelman <[email protected]> wrote:
On 5/23/20 12:29 PM, Jan Lehnardt wrote:
On 23. May 2020, at 16:28, Miles Fidelman <[email protected]> wrote:
On 5/22/20 11:13 AM, Jan Lehnardt wrote:
On 22. May 2020, at 15:06, Miles Fidelman <[email protected]> wrote:
Hi Jan,
On 5/22/20 6:17 AM, Jan Lehnardt wrote:
Hi Miles,
I wanted to reply for a while, but struggled to find a good angle. I think I
finally figured out what I missed. I’m not sure I understand your deployment
scenario.
When I think conference app, I think folks having that on their mobile phones,
or tablets. Given that, you’d be using PouchDB (for web apps) or one of
Cloudant mobile sync libraries (if it is a native app). Either way, to get the
data to the devices, it’d have to come from somewhere, but then you say there
is no central server. Where does the data come from?
I was using that as an example - but I'm really thinking more the case of "smart
documents." Think of a plan (business, mission) or briefing package - and the whole
notion of "living documents.
not sure what these are and how they compare to CouchDB documents.
"Living Document" as in writerspeak - a binder full of documentation that is
continually being kept up to date, the latest version of a proposal, a book draft, a
theatrical script that's being marked up with commentary during rehearsal, etc.
Or think of marrying a wiki to a DCVS.
It sounds like you imagine the devices talking to each other in a replication
mesh kind of way. While technically possible from a CouchDB replication
protocol point of view, this approach isn’t very practical. For one, you won’t
be able to guarantee that all devices can talk to each other, especially when
you don’t control the wifi.
What’s more likely is that you’d have central CouchDB server that is connected
to the wifi network, either on site, or in a datacenter somewhere that all
devices connect to.
Having that many devices talk to a central database to get all the relatively
static data of the conference (schedule, info, etc), doesn’t sound like much of
a problem. Group interaction is a little more interesting. You could model this
a 1-db-per-group, and it would work okay, but there are some
devil-in-the-details-details to work out with access control and joining and
leaving a group, etc.
So overall:
- what architecture do you envision?
- I think this can be made to work.
- CouchDB definitely can handle 5000 intermittent clients. Depending on your
use-case, you may need more or less computing resources for this, but there
aren’t any fundamental blockers in CouchDB’s design that would prevent this
from being a success.
I keep coming back to the model of replicated copies, stored in something like a distributed
version control system, linked by a publish-subscribe network. (Not that much different than
the way large-scale modeling & sims are done - local "world models" linked by a
protocol like DIS or HLA.)
I've been wondering if Couch/Pouch might make a good platform - coupled with
something like NNTP for epidemic style distribution of changes.
If you squint a little, CouchDB fits your buzzword bill here, but without more
concrete descriptions of what you need (rather than less), we can’t help much.
And you didn’t address the network connectivity issue. If you plan to have
device-to-device communication, and you are not Apple or Google making those
devices and/or mobile operating systems, you won’t be having much luck,
especially on the web platform.
I'm thinking tactical environments & mesh networks, with intermittent
connectivity. Updates propagate slowly, but with either automatic fork/merge, or
with eventual consistency.
Right, so far the theory, which the CouchDB replication protocol supports. Now
in practice, say you are using PouchDB, how do you make one device’s browser
talk to another device’s browser?
Well... I would see this as a case of mesh networking. An instance of PouchDB,
on a phone, might sync to one (or more) instances of CouchDB - running on a
laptop, or a server. The basic model would be that of NNTP. Updates would
ripple through a mesh of instances. My assumption is, that any particular
instance of Couch of Pouch needs to know one or more other instances to connect
to - and from there, stuff moves around.
that is more concrete and we can talk about it now, thanks for clarifying.
Which comes back to... how well does this scale? What's the largest mesh
network of Pouch/Couch instances that anybody has run in practice?
since the setup is very non-standard, there isn’t a lot of existing setups to
look for.
I know of a PouchDB-running-inside-node.js on IoT devices scenario covering a
large geographic area with mesh wifi, collecting data and replicating sensory
data through the mesh up to one point that has an internet uplink, which then
sends the whole bunch up to the cloud. At the time I was involved, the target
for this was 1000s of devices, but I know little about the amount of documents
flowing through the mesh.
Again, the replication model fully support this, but given enough data, the
sensory data option might eventually be limited by data size / storage space
available on the devices, to hold, potentially, all data from all devices.
For your use-case, you have a lot less data coming in, so I wouldn’t see much
of an issue, overall.
Do you, perhaps, have a reference for that implementation?
I do not, my company consulted on this with a large technology company on this.
I’m not at liberty to disclose any details, or even names.
Ah well. Though I just spent some time nosing around "who's using
PouchDB" and there seem to be some folks doing large scale things.
Going to start digging in. (Searching on PouchDB +IoT got me there.)
Thanks!
Miles
--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra
Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined:
nothing works and no one knows why. ... unknown
