tanonl opened a new pull request, #1124: URL: https://github.com/apache/guacamole-client/pull/1124
Note: This PR works in conjunction with the corresponding guacamole-server [PR](https://github.com/apache/guacamole-server/pull/624) for complete camera redirection functionality. ### Overview This PR implements client-side support for RDP Camera Redirection using the MS-RDPECAM protocol. This enables web browsers to stream local camera video (H.264-encoded) through Apache Guacamole to remote Windows sessions, with features including per-camera selection, persistent preferences, and audio/video synchronization controls. --- ## Problem Statement Users connecting to remote Windows desktops need seamless access to their local cameras for video conferencing, authentication, and other applications. This client-side implementation provides the browser-based camera capture, H.264 encoding, and user interface necessary to stream camera data to the guacamole-server RDPECAM implementation. --- ## Technical Implementation ### 1. **Camera Recording Foundation** (`07db985b4`) Generic, reusable camera recording infrastructure in `guacamole-common-js/src/main/webapp/modules/`: #### **`CameraRecorder.js`** - Abstract Camera Recorder Core recording engine with H.264 encoding: **Key Features:** - MediaRecorder API integration with H.264 codec support - Device enumeration via `navigator.mediaDevices.enumerateDevices()` - Configurable video constraints: - Resolution (width, height) - Frame rate (15-60 fps) - Bit rate (500kbps - 8Mbps) - Camera switching without stopping recording - Capability caching for performance **Design Philosophy:** - Protocol-agnostic (not specific to RDPECAM) - Reusable for any Guacamole protocol requiring camera access - Clean separation of concerns (capture vs. protocol handling) #### **`H264AnnexBUtil.js`** - H.264 Processing Utilities Low-level H.264 Annex-B format utilities: **Capabilities:** - NAL unit extraction from byte streams - Start code detection (`0x00000001`, `0x000001`) - SPS (Sequence Parameter Set) parsing - PPS (Picture Parameter Set) parsing - Frame boundary detection - Annex-B format validation **Use Case:** Essential for processing MediaRecorder output into proper H.264 frames for transmission via Guacamole protocol. ### 2. **RDPECAM Service** (`6efbab318`) Complete RDPECAM protocol implementation in `guacamole/src/main/frontend/src/app/client/services/`: #### **`guacRDPECAM.js`** - RDPECAM Service The centerpiece of the client implementation, providing: ##### **Core Protocol Features:** **Capability Negotiation:** - Parse device capabilities from RDPECAM channel - Match browser MediaRecorder capabilities with server requirements - Generate optimal video constraints (resolution, frame rate, format) **Camera Lifecycle Management:** - Initialize CameraRecorder with negotiated constraints - Start/stop camera streams - Handle camera switching - Device enumeration with capability prefetching - Error recovery and retry logic **Stream Management:** - H.264 frame packetization - Guacamole stream creation and management - Frame transmission via Guacamole protocol - Proper stream termination ##### **Video Delay Synchronization Feature:** Addresses audio/video synchronization issues in remote video calls: **Problem:** Video frames may arrive faster than audio, causing lip-sync issues in video conferencing applications. **Solution:** Configurable video delay (0-1000ms) that buffers outgoing video frames. **Features:** - Delay range: 0-1000ms (adjustable in 50ms increments) - Slider visible in left menu when camera in use - Visual toast notifications for delay changes - localStorage persistence (per-connection) - Real-time adjustment during active streaming ### 3. **UI Integration** (`07966fc90`, `d68ad85f8`) User interface for camera control in `guacamole/src/main/frontend/src/app/client/`: #### **`clientController.js`** (165 new lines) AngularJS controller integration: **Camera State Management:** ```javascript $scope.cameraState = { available: false, // RDPECAM supported active: false, // Camera streaming devices: [], // Available cameras selectedDevice: null, // Current camera videoDelay: 0 // Current A/V delay }; ``` #### **`client.html`** (58 lines modified) Template updates for camera controls: **Camera Selection Dropdown:** ```html <select ng-model="cameraState.selectedDevice" ng-change="selectCamera(cameraState.selectedDevice)" ng-options="device.deviceId as device.label for device in cameraState.devices"> </select> ``` **Status Indicators:** ```html <span class="camera-status" ng-show="cameraState.active"> <i class="icon-camera"></i> In Use </span> ``` **Video Delay Display:** ```html <div class="video-delay-notification" ng-show="videoDelayNotification.visible"> Video Delay: {{cameraState.videoDelay}}ms </div> ``` #### **`ManagedClient.js`** (69 lines modified) **Service Integration:** - Wire RDPECAM service into ManagedClient - Expose camera state to controllers - Event propagation for camera state changes ### 4. **Configuration Support** (`c59f1b9a5`) #### **`extensions/guacamole-auth-json/protocols/rdp.json`** Protocol configuration additions **Purpose:** - Declares H.264 video mimetype support - Enables video instruction handling - Configures RDPECAM feature availability #### **`guacamole-common-js/src/main/webapp/modules/Client.js`** Core client updates for video streaming: **Video Instruction Handler:** ```javascript Guacamole.Client.prototype.handleVideoInstruction = function(stream, mimetype, duration) { // Route to appropriate protocol handler if (this.onvideo) this.onvideo(stream, mimetype, duration); }; ``` **Stream Management:** - Create video streams for camera data - Handle mimetype negotiation - Manage stream lifecycle ### 5. **Preference Persistence** (`preferenceService.js` modifications) Camera selection persistence: **Benefits:** - Remember user's preferred cameras - Persist video delay settings - Reduce setup friction for returning users ### 6. **Polish and Refinements** #### **Error Handling** (`6839eadcb`) - Promise catch handlers for all async operations: - `enumerateDevices()` failures - `getUserMedia()` denials - Stream creation errors - Consistent error logging - User-friendly error messages --- ## Key Features ### For End Users ✅ **Zero installation** - Works entirely in browser, no plugins ✅ **Camera selection** - Choose from multiple cameras if available ✅ **Persistent preferences** - Remember camera choice per connection ✅ **Visual feedback** - Clear indicators for camera status ✅ **A/V sync control** - Adjust video delay for lip-sync (0-1000ms) ✅ **Real-time switching** - Change cameras without reconnecting ### Technical Capabilities - **H.264 encoding** via browser MediaRecorder API - **Automatic capability negotiation** with server - **Adaptive constraints** based on camera capabilities - **Frame packetization** for Guacamole protocol - **Protocol-agnostic foundation** (CameraRecorder reusable) - **Unit tested** H.264 utilities --- ## Browser Compatibility ### Requirements - **MediaRecorder API** support - **H.264 encoding** support - **getUserMedia** support - **WebRTC** capabilities ### Tested Browsers - ✅ Chrome 90+ (full support) - ✅ Edge 90+ (full support) - ✅ Firefox 100+ (requires H.264 configuration) - ⚠️ Safari 14+ (limited H.264 support) --- ## Known Limitations 1. **Browser support** - Requires MediaRecorder API with H.264 encoding 2. **H.264 only** - No fallback codecs currently supported 3. **Single streaming camera** - One streaming camera per connection 4. **No audio** - Video-only streaming (by design, audio handled separately in RDP) 5. **Manual sync** - Video delay adjustment is manual, not automatic --- ## Security Considerations - **Camera permissions** - Browser prompts for user consent - **HTTPS required** - getUserMedia requires secure context - **Privacy controls** - Clear visual indicators when camera active - **No persistent access** - Camera stops on disconnect/tab close --- **Ready for review and merge.** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
