siva created GUACAMOLE-2292:
-------------------------------
Summary: Feature Request: Instant Seek for Session Recording
Playback
Key: GUACAMOLE-2292
URL: https://issues.apache.org/jira/browse/GUACAMOLE-2292
Project: Guacamole
Issue Type: New Feature
Components: guacamole-client, guacamole-server
Reporter: siva
h2. Summary
Add indexing and display snapshot support to Guacamole session recordings,
enabling O(log N) seek-to-position without sequential frame replay. All
metadata is embedded inline in the recording file itself — no sidecar files, no
post-processing.
h2. Problem Statement
The current Guacamole session recording player has no seek optimization. To
jump to a position T in a recording, the player must:
# Load the recording file from the beginning
# Parse and replay every protocol instruction sequentially up to position T
# Render all intermediate display state changes
For a 1-hour recording, seeking to the 45-minute mark requires parsing and
rendering ~75% of the file. This results in:
* *Multi-second (or minute) seek times* for long recordings
* *Poor user experience* — UI freezes during sequential replay
* *No random access* — impossible to jump to an arbitrary point without
sequential scan from the beginning
* *CPU-intensive* — every frame between 0 and T must be fully rendered even
though only the final state matters
h2. Background: Guacamole Recording Format
A Guacamole recording is a stream of length-prefixed protocol instructions:
{{4.size,1.0,4.1920,4.1080;
3.img,1.0,2.12,2.-1,1.0,1.0,9.image/png;
4.blob,2.12,8.iVBOR...;
3.end,2.12;
4.sync,13.1719344000000;}}
* Instructions are delimited by {{;}}
* Each element is {{<length>.<content>}}
* {{sync}} instructions mark frame boundaries (with Unix timestamp in ms)
* The protocol has no built-in index, seek table, or keyframe concept
* Unknown opcodes are ignored by standard clients (forward-compatible)
h2. Proposed Solution
h3. Approach: Inline Index with Reverse-Linked Back-Pointers + Inline Snapshots
Emit two custom instruction types directly into the recording stream during
capture:
# *{{_snapshot}}* — Periodic full-display PNG capture (every ~10s)
# *{{_index}}* — Per-frame metadata entry with a back-pointer to the previous
index entry
This creates a reverse-linked-list of index entries embedded in the file,
enabling O(log N) binary search using HTTP Range requests. No separate files.
No post-session processing.
h3. Custom Protocol Instructions
h4. {{_snapshot}} — Display Capture
{{9._snapshot,<length>.<base64-encoded PNG data>;}}
A full-screen PNG rendered by a server-side Cairo compositor that tracks
display state from the protocol stream. Emitted every ~10 seconds at a {{sync}}
boundary, *before* the {{sync}} instruction so its byte offset is known when
writing the subsequent {{{}_index{}}}.
h4. {{_index}} — Frame Index Entry
{{6._index,<timestamp>,<frame_start>,<frame_end>,<snap_offset>,<snap_length>,<seq>,<prev_index_offset>;}}
||Field||Type||Description||
|{{timestamp}}|int64|Unix timestamp in ms (from the {{sync}} instruction)|
|{{frame_start}}|uint64|Byte offset where this frame's protocol data begins|
|{{frame_end}}|uint64|Byte offset where this frame's protocol data ends|
|{{snap_offset}}|uint64|Byte offset of the nearest preceding {{_snapshot}} (0
if none yet)|
|{{snap_length}}|uint64|Byte length of that {{{}_snapshot{}}}'s PNG data (0 if
none yet)|
|{{seq}}|uint32|Monotonically increasing sequence number (0, 1, 2, ...)|
|{{prev_index_offset}}|uint64|Byte offset of the previous {{_index}}
instruction (0 for the first entry)|
Emitted immediately after every {{sync}} instruction.
h3. Recording File Structure
{{[frame 0 protocol instructions...]
4.sync,13.1719344000000;
6._index,13.1719344000000,1.0,4.4523,1.0,1.0,1.0,1.0;
[frame 1 protocol instructions...]
4.sync,13.1719344000500;
6._index,13.1719344000500,4.4523,5.12400,1.0,1.0,1.1,4.4524;
[frame 2 protocol instructions...]
9._snapshot,65536.<base64 PNG data>;
4.sync,13.1719344010000;
6._index,13.1719344010000,5.12400,5.82000,5.12401,5.65540,1.2,5.12401;
[frame 3 protocol instructions...]
4.sync,13.1719344010500;
6._index,13.1719344010500,5.82000,5.95000,5.12401,5.65540,1.3,5.82001;
...continues for entire session...}}
Key properties:
* Standard protocol instructions are unmodified
* Custom instructions ({{{}_snapshot{}}}, {{{}_index{}}}) are interleaved
* Back-pointers form a traversable chain from tail to head
* Everything is written in a single forward pass — append only
h2. Server-Side Implementation
h3. Recording Pipeline
{{Protocol Stream → Recording Socket Wrapper → File
↓
Cairo Compositor (tracks display state)
↓
Snapshot PNG (every ~10s)}}
# *Intercept writes:* Socket wrapper observes all protocol bytes flowing to
the recording file
# *Parse instructions:* Lightweight state machine identifies opcodes and
arguments (no full parser needed)
# *Feed compositor:* Every instruction is forwarded to a Cairo-based display
compositor that maintains a pixel-accurate framebuffer
# *On {{sync}} boundary (every ~10s):* Render compositor surface to PNG, emit
{{_snapshot}} instruction to file
# *On every {{{}sync{}}}:* Emit {{_index}} instruction with frame boundaries,
snapshot reference, sequence number, and back-pointer
# *On session end:* Close file handle. Done.
h3. Zero Post-Processing
All index and snapshot data is written inline during the session. When the
session ends, the server simply closes the file descriptor. No trailer writing,
no index compilation, no file merging — the recording is immediately ready for
playback with seek support.
h3. Cairo Compositor
The server maintains a lightweight display compositor that processes:
* {{size}} — Resize layer 0 surface
* {{img}} / {{blob}} / {{end}} — Decode PNG image streams and composite onto
surface
* {{copy}} — Region copy within the display
* {{cfill}} — Color fill operations
* {{rect}} / {{transfer}} — Additional drawing primitives
This is a subset of the full Guacamole rendering instruction set — enough to
produce accurate display snapshots.
h2. Client-Side Implementation
h3. Local File Seek (Current Player Architecture)
The existing player operates on local files. With the inline index, the local
player can:
# *On file open:* Scan backward from EOF (~last 2KB) to find the last
{{_index}} entry
# *Walk the back-pointer chain:* Follow {{prev_index_offset}} links to build a
complete frame table in memory — O(N) sequential reads but only touching
{{_index}} entries, skipping all protocol/snapshot data
# *On seek:* Look up the target frame in the in-memory table, {{fseek()}} to
the snapshot offset, decode the PNG, then {{fseek()}} to the gap frames and
replay ~10s of protocol data
This is already a massive improvement over the current sequential replay:
instead of rendering every frame from 0 to T, the player reads one snapshot +
~10 seconds of protocol data.
h3. Complexity (Local File Playback)
||Operation||Cost||Notes||
|Build frame table (one-time)|O(N) fseek + read|Only reads {{_index}} entries,
skips bulk data|
|Seek to any position|2 fseek + read|1 snapshot + gap frames|
|Gap replay|O(frames in 10s window)|At most ~10s of protocol data|
Compare to current: seek to 45min in a 1hr recording = parse and render every
frame from the beginning (potentially minutes of CPU time).
h3. Playback (Non-Seek)
Normal frame-by-frame playback is unaffected. The player processes instructions
sequentially and simply ignores {{_snapshot}} and {{_index}} opcodes (unknown
opcode handling).
h2. Backward Compatibility
||Client||Behavior||
|Standard guacamole-common-js|Plays normally; skips unknown
{{{}_snapshot{}}}/{{{}_index{}}} opcodes|
|Enhanced player (this feature)|Uses inline index for instant seek|
|guacd without this feature|Produces recordings without
{{{}_snapshot{}}}/{{{}_index{}}} — player falls back to sequential playback|
No protocol specification changes required. Custom instructions use {{_}}
prefix to signal non-standard extensions.
h2. Configuration
||Parameter||Default||Description||
|{{recording-index-enabled}}|{{true}}|Emit {{_index}} instructions in recording|
|{{recording-snapshots-enabled}}|{{true}}|Emit {{_snapshot}} instructions in
recording|
|{{recording-snapshot-interval}}|{{10000}}|Milliseconds between snapshots|
|{{recording-snapshot-max-width}}|{{0}} (no limit)|Cap snapshot resolution to
reduce file size|
h2. File Size Impact
For a typical 1-hour, 1920×1080 desktop recording:
||Component||Size||% of typical recording||
|{{_index}} entries (~7200 frames)|~600 KB|< 0.1%|
|{{_snapshot}} entries (~360 @ ~80KB avg)|~28 MB|~3-5%|
|*Total overhead*|*~29 MB*|*~3-5%*|
Snapshots dominate the overhead. The snapshot interval and resolution cap are
tunable to trade seek granularity for file size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
