[
https://issues.apache.org/jira/browse/GUACAMOLE-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18092745#comment-18092745
]
siva commented on GUACAMOLE-2292:
---------------------------------
This improvement will also enhance Network playback - Guacamole player can swap
file I/O calls to byte-range Network I/O calls reading and rendering playback
from blobs stored in cloud storage.
> Feature Request: Instant Seek for Session Recording Playback
> ------------------------------------------------------------
>
> Key: GUACAMOLE-2292
> URL: https://issues.apache.org/jira/browse/GUACAMOLE-2292
> Project: Guacamole
> Issue Type: New Feature
> Components: guacamole-client, guacamole-server
> Reporter: siva
> Priority: Minor
>
> h2. Summary
> Add indexing and display snapshot support to Guacamole session recordings,
> enabling O(log N) seek-to-position without sequential frame replay. All
> metadata is embedded inline in the recording file itself — no sidecar files,
> no post-processing.
> h2. Problem Statement
> The current Guacamole session recording player has no seek optimization. To
> jump to a position T in a recording, the player must:
> # Load the recording file from the beginning
> # Parse and replay every protocol instruction sequentially up to position T
> # Render all intermediate display state changes
> For a 1-hour recording, seeking to the 45-minute mark requires parsing and
> rendering ~75% of the file. This results in:
> * *Multi-second (or minute) seek times* for long recordings
> * *Poor user experience* — UI freezes during sequential replay
> * *No random access* — impossible to jump to an arbitrary point without
> sequential scan from the beginning
> * *CPU-intensive* — every frame between 0 and T must be fully rendered even
> though only the final state matters
> h2. Background: Guacamole Recording Format
> A Guacamole recording is a stream of length-prefixed protocol instructions:
>
> {{4.size,1.0,4.1920,4.1080;
> 3.img,1.0,2.12,2.-1,1.0,1.0,9.image/png;
> 4.blob,2.12,8.iVBOR...;
> 3.end,2.12;
> 4.sync,13.1719344000000;}}
> * Instructions are delimited by {{;}}
> * Each element is {{<length>.<content>}}
> * {{sync}} instructions mark frame boundaries (with Unix timestamp in ms)
> * The protocol has no built-in index, seek table, or keyframe concept
> * Unknown opcodes are ignored by standard clients (forward-compatible)
> h2. Proposed Solution
> h3. Approach: Inline Index with Reverse-Linked Back-Pointers + Inline
> Snapshots
> Emit two custom instruction types directly into the recording stream during
> capture:
> # *{{_snapshot}}* — Periodic full-display PNG capture (every ~10s)
> # *{{_index}}* — Per-frame metadata entry with a back-pointer to the
> previous index entry
> This creates a reverse-linked-list of index entries embedded in the file,
> enabling O(log N) binary search using HTTP Range requests. No separate files.
> No post-session processing.
> h3. Custom Protocol Instructions
> h4. {{_snapshot}} — Display Capture
>
> {{9._snapshot,<length>.<base64-encoded PNG data>;}}
> A full-screen PNG rendered by a server-side Cairo compositor that tracks
> display state from the protocol stream. Emitted every ~10 seconds at a
> {{sync}} boundary, *before* the {{sync}} instruction so its byte offset is
> known when writing the subsequent {{{}_index{}}}.
> h4. {{_index}} — Frame Index Entry
>
> {{6._index,<timestamp>,<frame_start>,<frame_end>,<snap_offset>,<snap_length>,<seq>,<prev_index_offset>;}}
> ||Field||Type||Description||
> |{{timestamp}}|int64|Unix timestamp in ms (from the {{sync}} instruction)|
> |{{frame_start}}|uint64|Byte offset where this frame's protocol data begins|
> |{{frame_end}}|uint64|Byte offset where this frame's protocol data ends|
> |{{snap_offset}}|uint64|Byte offset of the nearest preceding {{_snapshot}} (0
> if none yet)|
> |{{snap_length}}|uint64|Byte length of that {{{}_snapshot{}}}'s PNG data (0
> if none yet)|
> |{{seq}}|uint32|Monotonically increasing sequence number (0, 1, 2, ...)|
> |{{prev_index_offset}}|uint64|Byte offset of the previous {{_index}}
> instruction (0 for the first entry)|
> Emitted immediately after every {{sync}} instruction.
> h3. Recording File Structure
>
> {{[frame 0 protocol instructions...]
> 4.sync,13.1719344000000;
> 6._index,13.1719344000000,1.0,4.4523,1.0,1.0,1.0,1.0;
> [frame 1 protocol instructions...]
> 4.sync,13.1719344000500;
> 6._index,13.1719344000500,4.4523,5.12400,1.0,1.0,1.1,4.4524;
> [frame 2 protocol instructions...]
> 9._snapshot,65536.<base64 PNG data>;
> 4.sync,13.1719344010000;
> 6._index,13.1719344010000,5.12400,5.82000,5.12401,5.65540,1.2,5.12401;
> [frame 3 protocol instructions...]
> 4.sync,13.1719344010500;
> 6._index,13.1719344010500,5.82000,5.95000,5.12401,5.65540,1.3,5.82001;
> ...continues for entire session...}}
> Key properties:
> * Standard protocol instructions are unmodified
> * Custom instructions ({{{}_snapshot{}}}, {{{}_index{}}}) are interleaved
> * Back-pointers form a traversable chain from tail to head
> * Everything is written in a single forward pass — append only
> h2. Server-Side Implementation
> h3. Recording Pipeline
>
> {{Protocol Stream → Recording Socket Wrapper → File
> ↓
> Cairo Compositor (tracks display state)
> ↓
> Snapshot PNG (every ~10s)}}
> # *Intercept writes:* Socket wrapper observes all protocol bytes flowing to
> the recording file
> # *Parse instructions:* Lightweight state machine identifies opcodes and
> arguments (no full parser needed)
> # *Feed compositor:* Every instruction is forwarded to a Cairo-based display
> compositor that maintains a pixel-accurate framebuffer
> # *On {{sync}} boundary (every ~10s):* Render compositor surface to PNG,
> emit {{_snapshot}} instruction to file
> # *On every {{{}sync{}}}:* Emit {{_index}} instruction with frame
> boundaries, snapshot reference, sequence number, and back-pointer
> # *On session end:* Close file handle. Done.
> h3. Zero Post-Processing
> All index and snapshot data is written inline during the session. When the
> session ends, the server simply closes the file descriptor. No trailer
> writing, no index compilation, no file merging — the recording is immediately
> ready for playback with seek support.
> h3. Cairo Compositor
> The server maintains a lightweight display compositor that processes:
> * {{size}} — Resize layer 0 surface
> * {{img}} / {{blob}} / {{end}} — Decode PNG image streams and composite onto
> surface
> * {{copy}} — Region copy within the display
> * {{cfill}} — Color fill operations
> * {{rect}} / {{transfer}} — Additional drawing primitives
> This is a subset of the full Guacamole rendering instruction set — enough to
> produce accurate display snapshots.
> h2. Client-Side Implementation
> h3. Local File Seek (Current Player Architecture)
> The existing player operates on local files. With the inline index, the local
> player can:
> # *On file open:* Scan backward from EOF (~last 2KB) to find the last
> {{_index}} entry
> # *Walk the back-pointer chain:* Follow {{prev_index_offset}} links to build
> a complete frame table in memory — O(N) sequential reads but only touching
> {{_index}} entries, skipping all protocol/snapshot data
> # *On seek:* Look up the target frame in the in-memory table, {{fseek()}} to
> the snapshot offset, decode the PNG, then {{fseek()}} to the gap frames and
> replay ~10s of protocol data
> This is already a massive improvement over the current sequential replay:
> instead of rendering every frame from 0 to T, the player reads one snapshot +
> ~10 seconds of protocol data.
> h3. Complexity (Local File Playback)
> ||Operation||Cost||Notes||
> |Build frame table (one-time)|O(N) fseek + read|Only reads {{_index}}
> entries, skips bulk data|
> |Seek to any position|2 fseek + read|1 snapshot + gap frames|
> |Gap replay|O(frames in 10s window)|At most ~10s of protocol data|
> Compare to current: seek to 45min in a 1hr recording = parse and render every
> frame from the beginning (potentially minutes of CPU time).
> h3. Playback (Non-Seek)
> Normal frame-by-frame playback is unaffected. The player processes
> instructions sequentially and simply ignores {{_snapshot}} and {{_index}}
> opcodes (unknown opcode handling).
> h2. Backward Compatibility
> ||Client||Behavior||
> |Standard guacamole-common-js|Plays normally; skips unknown
> {{{}_snapshot{}}}/{{{}_index{}}} opcodes|
> |Enhanced player (this feature)|Uses inline index for instant seek|
> |guacd without this feature|Produces recordings without
> {{{}_snapshot{}}}/{{{}_index{}}} — player falls back to sequential playback|
> No protocol specification changes required. Custom instructions use {{_}}
> prefix to signal non-standard extensions.
> h2. Configuration
> ||Parameter||Default||Description||
> |{{recording-index-enabled}}|{{true}}|Emit {{_index}} instructions in
> recording|
> |{{recording-snapshots-enabled}}|{{true}}|Emit {{_snapshot}} instructions in
> recording|
> |{{recording-snapshot-interval}}|{{10000}}|Milliseconds between snapshots|
> |{{recording-snapshot-max-width}}|{{0}} (no limit)|Cap snapshot resolution to
> reduce file size|
> h2. File Size Impact
> For a typical 1-hour, 1920×1080 desktop recording:
> ||Component||Size||% of typical recording||
> |{{_index}} entries (~7200 frames)|~600 KB|< 0.1%|
> |{{_snapshot}} entries (~360 @ ~80KB avg)|~28 MB|~3-5%|
> |*Total overhead*|*~29 MB*|*~3-5%*|
> Snapshots dominate the overhead. The snapshot interval and resolution cap are
> tunable to trade seek granularity for file size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
