Re: Local-First apps and CouchDB

[ermouth]

> it also not feasible in practice for the same reasons ermouth outlines

If we shift the scope a bit, CRDT is ok for distributed collaborative
allocation/deallocation of quantitative resources or commutative numerical
entities of other kinds (ie likes or even spectral vectors).

Storing *changes* of tagged numbers or vectors instead of abs values, I
mean smth like +1, or [+0.2, -0.5, +0.1, ...], is more or less ok, esp if
we aggregate changes received from a single source (ie user or sensor) into
a single doc. The reducer for this case is obvious – built-in _sum for
instance. Garbage collection is also more or less obvious – collapsing
timeseries of a single doc into final result time to time, and then
compacting a bucket.

However, this solution is already well known and only deserves to be named
CRDT for hype or marketing.

ermouth

Re: Local-First apps and CouchDB

[Jan Lehnardt]

I came to similar conclusions while reading the JSON CRDT paper by the same
author. It closes with essentially “garbage collection is TBD”, which makes
it also not feasible in practice for the same reasons ermouth outlines.

Best
Jan
—

> On 7. Aug 2020, at 03:08, ermouth  wrote:
> 
> Both CRDT and OT (operational transform) are implementable over CouchDB.
> CRDT approach is ok for Couch/Pouch built-in MapReduce mechanics, esp if
> complicated combining reducers are executed client-side (ie no 64Mb for JS
> instance limitation). Best results however are achieved if a finalizing
> part of a reducer is put out of Couch/Pouch.
> 
> As anything ‘magic’, CRDT collab scheme, esp if there’s a goal to implement
> a text editor, has its price.
> 
> The entire text should be split into lower-level atomic blocks, say words,
> or even characters, and each should be stored as a separate entity inside
> JSON obj or array. Each action (insert/delete of an entity) is then stored
> as a separate doc, then distributed over collaborators using _changes, and
> then re-combined at each client. So, half a kilobyte text easily bloats
> into hundreds of docs and hundreds of kilobytes of JSON data.
> 
> Since we store each modification as a separate doc, there’s absolutely no
> way we can get conflicting doc revisions, so the problem of conflicting
> docs just never happens.
> 
> I’ve tested the CRDT collab approach over CouchDB, found it viable, however
> dropped it for enormous storage overhead. So, excessive storage is the
> problem, not conflicts or speed; however, bloating is inevitable with CRDT.
> 
> ermouth
> 
> 
> чт, 6 авг. 2020 г. в 01:09, Bill Stephenson :
> 
>> There have been 3 submissions of a paper about "Local-First" apps (
>> https://www.inkandswitch.com/local-first.html <
>> https://www.inkandswitch.com/local-first.html>) to Hacker News in the
>> past week. Twice it made it to the front page and the most recent was there
>> most all day and was still on the 2nd page the next morning.
>> 
>> The paper mentions and discusses CouchDB but I think it drifts quickly
>> from the subject of "Local-First” apps to conflict resolution and
>> overlooked using CouchDB as a data storage layer for Local-First apps.
>> 
>> I thought it’s worth mentioning here because CouchDB makes it easy for
>> both devs and users to create and install a Local-First app. A couple
>> months ago I made a very simple demo app that demonstrates one way to do
>> that. I posted a link to it in the comments on those HN posts and it seemed
>> to generate a little interest so I thought I’d share it here too...
>> 
>> https://cherrypc.com/app/editor/setup.html <
>> https://cherrypc.com/app/editor/setup.html>
>> 
>> --
>> 
>> Bill Stephenson
>> 
>> 
>> 
>> 
>> 
>> 

Re: Local-First apps and CouchDB

[ermouth]

Both CRDT and OT (operational transform) are implementable over CouchDB.
CRDT approach is ok for Couch/Pouch built-in MapReduce mechanics, esp if
complicated combining reducers are executed client-side (ie no 64Mb for JS
instance limitation). Best results however are achieved if a finalizing
part of a reducer is put out of Couch/Pouch.

As anything ‘magic’, CRDT collab scheme, esp if there’s a goal to implement
a text editor, has its price.

The entire text should be split into lower-level atomic blocks, say words,
or even characters, and each should be stored as a separate entity inside
JSON obj or array. Each action (insert/delete of an entity) is then stored
as a separate doc, then distributed over collaborators using _changes, and
then re-combined at each client. So, half a kilobyte text easily bloats
into hundreds of docs and hundreds of kilobytes of JSON data.

Since we store each modification as a separate doc, there’s absolutely no
way we can get conflicting doc revisions, so the problem of conflicting
docs just never happens.

I’ve tested the CRDT collab approach over CouchDB, found it viable, however
dropped it for enormous storage overhead. So, excessive storage is the
problem, not conflicts or speed; however, bloating is inevitable with CRDT.

ermouth


чт, 6 авг. 2020 г. в 01:09, Bill Stephenson :

> There have been 3 submissions of a paper about "Local-First" apps (
> https://www.inkandswitch.com/local-first.html <
> https://www.inkandswitch.com/local-first.html>) to Hacker News in the
> past week. Twice it made it to the front page and the most recent was there
> most all day and was still on the 2nd page the next morning.
>
> The paper mentions and discusses CouchDB but I think it drifts quickly
> from the subject of "Local-First” apps to conflict resolution and
> overlooked using CouchDB as a data storage layer for Local-First apps.
>
> I thought it’s worth mentioning here because CouchDB makes it easy for
> both devs and users to create and install a Local-First app. A couple
> months ago I made a very simple demo app that demonstrates one way to do
> that. I posted a link to it in the comments on those HN posts and it seemed
> to generate a little interest so I thought I’d share it here too...
>
> https://cherrypc.com/app/editor/setup.html <
> https://cherrypc.com/app/editor/setup.html>
>
> --
>
> Bill Stephenson
>
>
>
>
>
>

Re: Local-First apps and CouchDB

[Bill Stephenson]

Well that explains a lot. I’m not really surprised, but I am amazed he didn’t 
take advantage of the opportunity to learn from you. That just leaves me 
shaking my head.

--

Kindest Regards,

Bill Stephenson





> On Aug 6, 2020, at 10:41 AM, Jan Lehnardt  wrote:
> 
> The author of this sadly has demonstrated multiple times that they don’t 
> understand CouchDB very well, and despite my outreach, wasn’t open to 
> learning any more. I’m not sure if this is a personal choice, or a reflection 
> of few scientific papers existing around CouchDB, but it’s worth noting.
> 
> Best
> Jan
> —
> 
>> On 6. Aug 2020, at 00:09, Bill Stephenson  wrote:
>> 
>> There have been 3 submissions of a paper about "Local-First" apps 
>> (https://www.inkandswitch.com/local-first.html 
>> ) to Hacker News in the past 
>> week. Twice it made it to the front page and the most recent was there most 
>> all day and was still on the 2nd page the next morning. 
>> 
>> The paper mentions and discusses CouchDB but I think it drifts quickly from 
>> the subject of "Local-First” apps to conflict resolution and overlooked 
>> using CouchDB as a data storage layer for Local-First apps.
>> 
>> I thought it’s worth mentioning here because CouchDB makes it easy for both 
>> devs and users to create and install a Local-First app. A couple months ago 
>> I made a very simple demo app that demonstrates one way to do that. I posted 
>> a link to it in the comments on those HN posts and it seemed to generate a 
>> little interest so I thought I’d share it here too...
>> 
>> https://cherrypc.com/app/editor/setup.html 
>> 
>> 
>> --
>> 
>> Bill Stephenson
>> 
>> 
>> 
>> 
>> 

Re: Local-First apps and CouchDB

[Jan Lehnardt]

The author of this sadly has demonstrated multiple times that they don’t 
understand CouchDB very well, and despite my outreach, wasn’t open to learning 
any more. I’m not sure if this is a personal choice, or a reflection of few 
scientific papers existing around CouchDB, but it’s worth noting.

Best
Jan
—

> On 6. Aug 2020, at 00:09, Bill Stephenson  wrote:
> 
> There have been 3 submissions of a paper about "Local-First" apps 
> (https://www.inkandswitch.com/local-first.html 
> ) to Hacker News in the past 
> week. Twice it made it to the front page and the most recent was there most 
> all day and was still on the 2nd page the next morning. 
> 
> The paper mentions and discusses CouchDB but I think it drifts quickly from 
> the subject of "Local-First” apps to conflict resolution and overlooked using 
> CouchDB as a data storage layer for Local-First apps.
> 
> I thought it’s worth mentioning here because CouchDB makes it easy for both 
> devs and users to create and install a Local-First app. A couple months ago I 
> made a very simple demo app that demonstrates one way to do that. I posted a 
> link to it in the comments on those HN posts and it seemed to generate a 
> little interest so I thought I’d share it here too...
> 
> https://cherrypc.com/app/editor/setup.html 
> 
> 
> --
> 
> Bill Stephenson
> 
> 
> 
> 
> 

Re: Local-First apps and CouchDB

[Martin Broerse]

Hi Bill,

Very nice. Maybe for easy onboarding remove the need for setting up the
Local CouchDB in your example by using it on the free www.cloudstation.com
? We created the Free CloudStation to stimulate easy CouchDB onboarding,
with rewrite rules support, to enable hosting CouchDB on CouchDB like this
https://my.cloudstation.com/paint/_design/couch-paint/_rewrite/

- Martin



On Thu, 6 Aug 2020 at 00:09, Bill Stephenson 
wrote:

> There have been 3 submissions of a paper about "Local-First" apps (
> https://www.inkandswitch.com/local-first.html <
> https://www.inkandswitch.com/local-first.html>) to Hacker News in the
> past week. Twice it made it to the front page and the most recent was there
> most all day and was still on the 2nd page the next morning.
>
> The paper mentions and discusses CouchDB but I think it drifts quickly
> from the subject of "Local-First” apps to conflict resolution and
> overlooked using CouchDB as a data storage layer for Local-First apps.
>
> I thought it’s worth mentioning here because CouchDB makes it easy for
> both devs and users to create and install a Local-First app. A couple
> months ago I made a very simple demo app that demonstrates one way to do
> that. I posted a link to it in the comments on those HN posts and it seemed
> to generate a little interest so I thought I’d share it here too...
>
> https://cherrypc.com/app/editor/setup.html <
> https://cherrypc.com/app/editor/setup.html>
>
> --
>
> Bill Stephenson
>
>
>
>
>
>

Local-First apps and CouchDB

[Bill Stephenson]

There have been 3 submissions of a paper about "Local-First" apps 
(https://www.inkandswitch.com/local-first.html 
) to Hacker News in the past 
week. Twice it made it to the front page and the most recent was there most all 
day and was still on the 2nd page the next morning. 

The paper mentions and discusses CouchDB but I think it drifts quickly from the 
subject of "Local-First” apps to conflict resolution and overlooked using 
CouchDB as a data storage layer for Local-First apps.

I thought it’s worth mentioning here because CouchDB makes it easy for both 
devs and users to create and install a Local-First app. A couple months ago I 
made a very simple demo app that demonstrates one way to do that. I posted a 
link to it in the comments on those HN posts and it seemed to generate a little 
interest so I thought I’d share it here too...

https://cherrypc.com/app/editor/setup.html 


--

Bill Stephenson