bobbai00 opened a new issue, #3930:
URL: https://github.com/apache/texera/issues/3930
## Feature Summary
Enable users to access a terminal/shell interface for their Computing Units
(CUs) through the web interface. This feature provides SSH-like access to CUs,
allowing users to interact with the execution environment directly via a
web-based terminal.
**Problem it solves:**
- Users cannot install custom libraries or binaries needed for specific
workflows
- Limited flexibility when debugging workflow execution issues
- No way to inspect the runtime environment or installed dependencies
- Difficult to troubleshoot operator-specific problems that require shell
access
- Cannot perform ad-hoc tasks or run custom commands in the execution
environment
## Proposed Solution or Design
Implement web-based terminal access to Computing Units using ttyd (terminal
over HTTP/WebSocket) with the following components:
### Backend Changes
1. **Computing Unit Master Dockerfile**
(`bin/computing-unit-master.dockerfile`):
- Install ttyd package for web-based terminal
- Expose port 7681 for terminal access
- Run ttyd alongside the computing unit master process
2. **Kubernetes Service Configuration**
(`bin/k8s/templates/workflow-computing-units-service.yaml`):
- Add ttyd port (7681) to the service definition
- Enable routing to terminal endpoints
3. **Envoy Proxy Configuration** (`bin/k8s/templates/envoy-config.yaml`):
- Add routing rules for `/wsapi/cu-ssh` endpoint
- Configure proxy pass to computing unit terminal services
- Implement proper authentication and authorization
4. **Access Control**:
- Integrate with existing access control service
- Verify user has permission to access specific CU
- Use JWT tokens for authentication
### Frontend Changes
1. **Computing Unit SSH Service** (`computing-unit-ssh.service.ts`):
- Generate terminal URLs with proper authentication tokens
- Handle user and CU ID parameters
2. **Computing Unit Selection UI** (`computing-unit-selection.component`):
- Add "Open Terminal" or "SSH Access" button for each CU
- Display terminal in iframe or modal
- Show terminal status and connection info
### Security Considerations
- Terminal access requires valid authentication token
- Users can only access CUs they have permissions for
- Terminal sessions are scoped to the specific CU
- Network access is controlled through Envoy proxy
## Use Cases
**1. Installing Custom Libraries**
```bash
# User can install specific Python packages
pip install custom-ml-library==2.1.0
# Install system packages (if permissions allow)
apt-get install custom-binary-tool
```
**2. Debugging Workflow Execution**
```bash
# Inspect running processes
ps aux | grep python
# Check logs in real-time
tail -f /texera/amber/logs/worker.log
# Verify environment variables
env | grep TEXERA
```
**3. Runtime Environment Inspection**
```bash
# Check installed packages and versions
pip list
R --version
# Inspect data files
ls -lh /texera/amber/data/
# Monitor resource usage
top -n 1
```
**4. Custom Script Execution**
```bash
# Run preprocessing scripts
python /path/to/custom/preprocess.py
# Execute data validation
./validate_input_data.sh
```
## Impact / Priority
**(P1) High – significantly improves user experience**
This feature provides significant benefits for:
- **Data Scientists**: Install specialized ML/data science libraries
on-demand
- **Developers**: Debug and troubleshoot workflow execution issues
effectively
- **System Administrators**: Inspect and maintain computing unit environments
- **Power Users**: Greater flexibility and control over execution environment
## Affected Area
- **Deployment / Infrastructure** - Modifies Dockerfile, Kubernetes configs,
Envoy routing
- **Workflow Engine (Amber)** - Adds terminal access to computing unit
masters
- **Workflow UI** - New terminal interface in frontend
## Additional Context
**Technical Implementation Details:**
- Uses ttyd for web-based terminal over WebSocket
- Terminal runs on port 7681 within computing unit pods
- Access control integrated with existing JWT authentication
- Routing handled by Envoy proxy with `/wsapi/cu-ssh` endpoint
**Benefits:**
- No need to rebuild images for library installation
- Faster iteration during workflow development
- Better debugging capabilities
- Reduced dependency on pre-installed packages
- Enhanced user autonomy and flexibility
**Potential Extensions:**
- File upload/download through terminal interface
- Terminal session recording for audit purposes
- Resource limits for user-installed packages
- Shared terminal sessions for collaboration
By submitting this issue, you agree to follow the [Apache Code of
Conduct](https://www.apache.org/foundation/policies/conduct).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
