Hello all,
I would like to start to think how we improve our back-end. With the
upcoming SDK updates we need to look at our gui implementation. We could
try a minimal update. but probably after all the time we could take a
broader view and look for the right target for the future. . I think we
should think about the high level approach strategy. It is not about
building or current actions, but thinking ahead and create a vision. The
effort part is only a secondary question. I would like to plan this and
i am happy if the plan is interesting to do.
I went through approaches i thought interesting. This is not so much
for tool selection. Tools mentioned are more like sample fillers, so we
have some idea what direction we talk about. I looked at flutter and a
bit into qt. First because it is interesting for Android and IOs and
latter because if you look into what supports Linux, Mac and Windows,
the leader is QT. The Approach would be similar to EFL. They are
different on the detail, but not different on the general approach. We
can dive there if someone wants to.
If you have other ideas then please mention them. If you have question
please ask. This should be an abstract discussion and i am not sure all
is clear. If you cant follow because it is to technical, then sum up
what you understand and i try to fill the gaps.
I dont want anyone to be left out. your thoughts are valuable.
Content:
1) Current situation
2) Modernizing the Architecture
3) Focus on one framework instead of different sets.
a) efl
b) The Linebender Ecosystem (a rust approach as a fancy alternative)
4) Multiuser approach (Moonshot, outlook)
a) concept - what we need to do to be able to support it.
b) an implementation approach for AOO as Tomcat Servlet
1) Current situation
Currently our architecture may look roughly like the following (ki
generated):
┌─────────────────────────────────────────────────────────┐
│ Application Layer │
│ Writer, Calc, Impress, Draw, Base │
├─────────────────────────────────────────────────────────┤
│ VCL Public API │
│ Window, Control, Dialog, Font, Bitmap │
├─────────────────────────────────────────────────────────┤
│ VCL Internals │
│ Event Loop │ Layout │ Rendering Primitives │ Printing │
├─────────────────────────────────────────────────────────┤
│ SAL (Abstraction Layer) │
│ SalInstance │ SalFrame │ SalGraphics │
│ ── Platform Contract ── │
├─────────────────────────────────────────────────────────┤
│ SAL Win32 │ SAL Aqua │ SAL Gen / GTK3 │
│ │ │ │
│ GDI / GDI+ │ CoreGraphics │ Cairo / X11 │
│ DirectWrite │ CoreText │ │
│ Direct2D │ Metal(stub) │ FreeType │
├─────────────────────────────────────────────────────────┤
│ OS / Hardware │
└─────────────────────────────────────────────────────────┘
So how does this work. the Application layer contains the document data,
and knows how to read write them. The VCL Public api defines a platform
independent way how everything looks on the screen.
The VCL internals are then description of those representations, which
use an abstraction layer. This abstraction layer maps then this to the
platform we are on.
2 Modernizing the Architecture
we could modernize the architecture into state of the art:
┌─────────────────────────────────────────┐
│ Application Layer │
│ (Writer, Calc, Impress - UNCHANGED) │
├─────────────────────────────────────────┤
│ VCL Public API │ ← Stable interface
│ (OutputDevice, Window, Event) │ applications depend on
├─────────────────────────────────────────┤
│ Scene Graph Layer │ ← NEW
│ Retained primitive tree │
├──────────────────┬──────────────────────┤
│ Layout Engine │ Animation Engine │ ← NEW
├──────────────────┴──────────────────────┤
│ Render Backend Interface │ ← NEW clean abstraction
├──────────┬──────────────┬───────────────┤
│ Skia │ Cairo │ Direct2D/ │ ← Backend
│ (GL/VK) │ (Software) │ CoreGraphics │ implementations
├──────────┴──────────────┴───────────────┤
│ Platform Abstraction (SAL) │ ← Modernized/thinned
│ Window | Input | Display | Font │
├──────────┬──────────────┬───────────────┤
│ Win32 │ Wayland/X11 │ Cocoa │
└──────────┴──────────────┴───────────────┘
following the principles:
|1. Clean separation of concerns 2. GPU-first rendering pipeline 3.
Retained mode scene graph 4. Immutable primitives 5. Clear ownership
semantics (modern C++) 6. Platform layer as thin as possible Imho this
may be modernized, but still quite complex approach. We have multiple
render engines to maintain. Even if we go down the path that LO went
focusing on skia, we have multiple platforms to manage in the SAL layer.
If we want to get rid of that the goto framework seems to be qt. It has
the best support it seems. |
|3)focus on one framework|
||
|3 a) EFL|
|Another Approach are the Enlightenment foundation Libraries, short efl.
From the git repo: |EFL is a collection of libraries for handling many common
tasks a
developer may have such as data structures, communication, rendering,
widgets and more. (read:https://www.enlightenment.org/about-efl)
Most of them have/BSD 2-Clause license some are LGPL ore GPL-2 license. /|EFL Architecture supports this: Ecore backends: ├── ecore_wl2 → Linux
Wayland MATURE ├── ecore_x → Linux X11 MATURE ├── ecore_win → Windows
EXISTS, needs work ├── ecore_cocoa → macOS EXISTS, needs work └──
ecore_drm2 → Linux bare metal MATURE Evas render engines: ├── OpenGL →
All platforms via OpenGL/ANGLE ├── Software → All platforms fallback └──
Vulkan → All platforms emerging|
Our Architecture would look like the following:
┌─────────────────────────────────────────┐
│ Application Layer │
│ (Writer, Calc, Impress - UNCHANGED) │
├─────────────────────────────────────────┤
│ VCL Public API │
├─────────────────────────────────────────┤
│ VCL Scene Graph │
├─────────────────────────────────────────┤
│ EFL Abstraction Backend │
│ (Single implementation of SAL) │
├──────────┬──────────────┬───────────────┤
│ EFL │ EFL │ EFL │
│ Linux │ Windows │ macOS │
│ (mature) │ (improve) │ (improve) │
└──────────┴──────────────┴───────────────┘
That would be very handy. we can profit from improvement on efl. Even if
the efl community is small.
3 b) |Linebender Ecosystem|
|┌─────────────────────────────────────────────────────────────────────┐
│ The Linebender Ecosystem (all Apache 2.0) │
├─────────────────────────────────────────────────────────────────────┤
│ │ │ kurbo → 2D geometry primitives (Bezier, Rect, Point) │ │ peniko →
graphics primitives (Color, Brush, Fill, Stroke) │ │ skrifa → font
loading and shaping │ │ parley → text layout engine │ │ vello → GPU
scene renderer │ │ masonry → widget layout and event system │ │ xilem →
reactive UI framework │ │ │ │ This entire stack is designed to work
together │ │ and replaces what AOO currently spreads across │ │ platform
specific GDI/Quartz/Cairo implementations │
└─────────────────────────────────────────────────────────────────────┘|
The really compelling argument for AOO specifically is that *Vello
already thinks in scene graphs and serialized draw commands* which is
exactly the abstraction AOO needs for both desktop rendering and web
streaming. You are not fighting the framework, you are aligning with it.
In this approach we would go through the default |C++ to Rust FFI Bridge
not using uno bridge. Or should we think about it? My guts say that this
is complicated without benefit.|
4) Multiuser approach (Moonshot, outlook)
4 a) concept - what we need to change change
One feature request that i hear on an off, is collaborative work on
documents. How could we do that.
We could Implement this in the VCL Scene Graph.
The architectural change needed in VCL:
|TODAY PROPOSED ───────────────────────── ───────────────────────── 1
document = 1 view 1 document = N views 1 view = 1 window 1 view = 1
scene graph node Scene graph → EFL/SAL Scene graph → serializable No
view identity Each view has session ID Input tied to window Input routed
by session ID |
The Scene Graph needs to become serializable:
|┌─────────────────────────────────────────────────────────┐ │ VCL Scene
Graph Node │ ├─────────────────────────────────────────────────────────┤
│ + serialize() → JSON / Protobuf / Binary │ │ + diff() → delta since
last frame │ │ + applyInput() → mouse / keyboard event │ │ + sessionId →
which viewer owns this view │ │ + dirtyRegion → what changed since last
push │ └─────────────────────────────────────────────────────────┘ |
Collaboration becomes natural:
|┌─────────────────────────────────────────────────────────┐ │
Collaboration Model │
├─────────────────────────────────────────────────────────┤ │ Document
Model → single source of truth │ │ Scene Graph → one node per active
viewer │ │ Change events → broadcast to all viewer nodes │ │ Cursor A →
visible in viewer B scene node │ │ Edit lock → managed at document model
level │ └─────────────────────────────────────────────────────────┘ |
4 b) AOO as Tomcat servlet
|If we have that we could do another crazy move and add an alternative
implementation that AOO works as a Tomcat servlet. |
This maps very cleanly to Tomcat concepts:
|Tomcat AOO VCL Servlet ──────────────────────────
────────────────────────── HttpServlet VCLSceneServlet HttpSession
ViewerSession ServletContext DocumentContext request/response input
event / scene delta war deployment document deployment thread per
request scene update per frame tick |The beauty is*the application layer does not need to know* about multi-viewer at all, that complexity lives entirely in the VCL scene graph session management layer
so this would look like:
|┌─────────────────────────────────────────────────────────────────────┐
│ VCL Scene Graph │ │ │ │
┌─────────────────────────────────────────────────────────────┐ │ │ │
Scene Graph Core │ │ │ │ serialize() │ diff() │ dirtyRegion() │ │ │ │
applyInput() │ sessionId │ viewerNode │ │ │
└─────────────────────────────────────────────────────────────┘ │ │ │ │
│ ┌──────────────────────────────┴──────────────────────────────┐ │ │ │
Collaboration Event Bus │ │ │ │ cursor positions │ edit locks │ change
broadcasts │ │ │
└─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│ ┌───────────────────┼───────────────────┐ │ │ ┌─────────▼──────────┐
┌──────────────▼──────────────┐ │ BUILD TARGET: │ │ BUILD TARGET: │ │
DESKTOP (EFL) │ │ WEB SERVLET │ └────────────────────┘
└─────────────────────────────┘ │ │ ┌─────────────────────────┐
┌─────────────────────────────┐ │ EFL Backend │ │ VCL Scene Servlet │ │
│ │ │ │ Ecore_Evas │ │ Tomcat / JBoss / Undertow │ │ Elementary widgets
│ │ HTTP2 / WebSocket │ │ Native input handling │ │ Session management │
│ Local window manager │ │ Scene delta streaming │
└─────────────────────────┘ └─────────────────────────────┘ │ │
┌─────────────────────────┐ ┌─────────────────────────────┐ │ P2P
Collaboration │ │ Server Collaboration │ │ Module (optional) │ │ (built
in) │ │ │ │ │ │ WebRTC data channel │ │ Document = single instance │ │
mDNS local discovery │ │ N viewers per document │ │ CRDT document sync │
│ Broadcast change events │ │ Cursor broadcast │ │ Edit lock management
│ │ No server needed │ │ Central authority model │
└─────────────────────────┘ └─────────────────────────────┘|
Here some more stuff the AI wrote:
||
The P2P module on desktop in detail:
|┌─────────────────────────────────────────────────────────────────────┐
│ Desktop P2P Collaboration Module │
├─────────────────────────────────────────────────────────────────────┤
│ │ │ Discovery Layer │ │
┌──────────────────────────────────────────────────────────────┐ │ │ │
mDNS/Bonjour → local network peer discovery │ │ │ │ DHT (Kademlia) →
internet peer discovery │ │ │ │ Manual invite → share link / QR code │ │
│ └──────────────────────────────────────────────────────────────┘ │ │ │
│ Transport Layer │ │
┌──────────────────────────────────────────────────────────────┐ │ │ │
WebRTC data channel → NAT traversal, encrypted │ │ │ │ fallback TURN
server → when direct P2P fails │ │ │
└──────────────────────────────────────────────────────────────┘ │ │ │ │
Sync Layer │ │
┌──────────────────────────────────────────────────────────────┐ │ │ │
CRDT on document model → conflict free edits │ │ │ │ Vector clocks →
change ordering │ │ │ │ Cursor sync → broadcast to all peers │ │ │ │
Edit lock negotiation → distributed mutex │ │ │
└──────────────────────────────────────────────────────────────┘ │ │ │ │
VCL Scene Bridge │ │
┌──────────────────────────────────────────────────────────────┐ │ │ │
Remote peer view → ghost cursor in local scene graph │ │ │ │ Change
events → apply to local document model │ │ │ │ Scene delta → send local
changes to all peers │ │ │
└──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘ |
The beautiful part is the hybrid scenario:
|┌─────────────────────────────────────────────────────────────────────┐
│ HYBRID SCENARIO │
├─────────────────────────────────────────────────────────────────────┤
│ │ │ Desktop User A ──P2P WebRTC──► Desktop User B │ │ │ │ │ │
└──────────► Servlet ◄──────────┘ │ │ │ │ │ Browser User C │ │ Browser
User D │ │ │ │ Desktop users sync P2P directly │ │ Servlet acts as
bridge for browser users │ │ Document model stays consistent across all
│ │ CRDT ensures no conflicts regardless of path │
└─────────────────────────────────────────────────────────────────────┘ |
All three modes share exactly one interface:
|┌─────────────────────────────────────────────────────────────────────┐
│ ICollaborationBackend (pure interface) │
├─────────────────────────────────────────────────────────────────────┤
│ + publishChange(delta) │ │ + subscribeChanges(callback) │ │ +
publishCursor(position, sessionId) │ │ + subscribeCursors(callback) │ │
+ requestEditLock(region) │ │ + releaseEditLock(region) │ │ + getPeers()
→ list of active sessions │
└─────────────────────────────────────────────────────────────────────┘
│ │ │ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────────┐
│ NullBackend │ │ P2PBackend │ │ ServletBackend │ │ (single user) │ │
(WebRTC/CRDT) │ │ (server broadcast) │ └──────────────────┘
└──────────────────┘ └──────────────────────┘ |
So a single user desktop gets |NullBackend| at zero cost, collaboration
is purely opt-in with no overhead when not used.
