sadpandajoe commented on issue #34197:
URL: https://github.com/apache/superset/issues/34197#issuecomment-3268201602
Just in case you'd want a full project plan:
# Apache Superset: Cypress to Playwright Migration Project
## Project Overview
Migrate end-to-end testing from Cypress to Playwright while maintaining test
coverage and reliability. The migration will be conducted in phases to allow
both frameworks to run in parallel during the transition period.
## Current State Analysis
### Cypress Setup
- **Location**: `superset-frontend/cypress-base/`
- **Version**: Cypress 11.2.0
- **Test Count**: ~50+ test files
- **Coverage**: Dashboard, charts, SQL Lab, authentication, datasets
- **CI Parallelization**: 6-way split using custom Python script
(`scripts/cypress_run.py`)
- **Features**:
- Code coverage with @cypress/code-coverage
- Visual testing with Applitools Eyes
- Custom commands and utilities
- Docker database setup for tests
### Key Files
- `superset-frontend/cypress-base/cypress.config.ts` - Main configuration
- `scripts/cypress_run.py` - CI parallelization orchestration
- `.github/workflows/superset-e2e.yml` - GitHub Actions workflow
- `docker/docker-entrypoint-initdb.d/cypress-init.sh` - Database setup
## Phase Breakdown
### Development Principles
- **YAGNI (You Aren't Gonna Need It)**: Build only what's immediately needed
- **DRY (Don't Repeat Yourself)**: Reuse existing Superset test
infrastructure
- **KISS (Keep It Simple, Stupid)**: Prefer simple solutions over complex
ones
- **Iterative**: Each phase builds on proven success from previous phase
### Phase 1: Get One Test Working End-to-End (Week 1)
**Goal**: Minimal viable Playwright setup with one auth test running locally
and in CI
**YAGNI Principle**: Build only what's needed to get one test working
#### Tasks
- [ ] **1.1 Minimal Setup**
- Add Playwright to `superset-frontend/package.json`
- Create `superset-frontend/playwright.config.ts` (Chrome only initially)
- Create `superset-frontend/tests/auth/login.spec.ts` (port simplest
Cypress auth test)
- [ ] **1.2 Essential Utilities (Only What's Needed)**
- Create basic login helper function
- Add baseURL configuration pointing to localhost:8088
- Setup minimal TypeScript configuration
- [ ] **1.3 CI Integration**
- Add Playwright to existing `.github/workflows/superset-e2e.yml`
- Use same database and setup as Cypress (DRY principle)
- Run single auth test alongside Cypress tests
- Add basic artifact collection for failures
**Success Criteria (KISS - Keep It Simple)**:
- ✅ One auth test runs locally: `npx playwright test auth/login`
- ✅ Same test passes in GitHub Actions CI
- ✅ Test uses existing Superset test infrastructure (no duplication)
- ✅ Takes < 1 day to implement
### Phase 2: Expand Testing Infrastructure (Week 2)
**Goal**: Add necessary infrastructure for migrating more tests
**Build on Phase 1 success, add only what's proven needed**
#### Tasks
- [ ] **2.1 Test Structure (Based on Phase 1 learnings)**
- Create `tests/` directory structure as needed
- Add utilities discovered during auth test implementation
- Setup page object models only if reuse is obvious
- [ ] **2.2 CI Optimization**
- Add Playwright parallelization (native, not custom script)
- Configure browser caching
- Setup proper test result reporting
- Add 2-3 more simple tests to validate infrastructure
- [ ] **2.3 Developer Experience**
- Add Makefile targets based on actual usage patterns
- Document basic Playwright commands
- Setup debugging configuration
**Success Criteria**:
- Multiple test types working (auth + list view + simple dashboard)
- Playwright runs faster than equivalent Cypress tests
- Clear path for migrating remaining tests
- Developer documentation exists for common tasks
### Phase 3: Systematic Test Migration (Weeks 3-8)
**Goal**: Systematically migrate tests from Cypress to Playwright
#### Migration Strategy
**Priority Order**:
1. Authentication tests (simple, foundational)
2. List view tests (charts, dashboards, datasets)
3. Form interactions and modals
4. Dashboard functionality
5. Chart/explore functionality
6. SQL Lab functionality
7. Complex integration tests
#### Tasks by Priority
- [ ] **3.1 Authentication & Navigation (Week 3)**
- `cypress/e2e/auth/` → `playwright/tests/auth/`
- Basic login/logout functionality
- Menu navigation and permissions
- [ ] **3.2 List Views (Week 4)**
- `cypress/e2e/chart_list/` → `playwright/tests/chart_list/`
- `cypress/e2e/dashboard_list/` → `playwright/tests/dashboard_list/`
- `cypress/e2e/dataset/dataset_list.test.ts` → `playwright/tests/dataset/`
- Filtering, sorting, search functionality
- [ ] **3.3 Database & Connections (Week 4)**
- `cypress/e2e/database/` → `playwright/tests/database/`
- Modal interactions, form submissions
- [ ] **3.4 Dashboard Functionality (Week 5-6)**
- `cypress/e2e/dashboard/` → `playwright/tests/dashboard/`
- Dashboard loading, editing, filters
- Chart interactions within dashboards
- [ ] **3.5 Chart Explore Interface (Week 6-7)**
- `cypress/e2e/explore/` → `playwright/tests/explore/`
- Chart creation, visualization types
- Advanced analytics, annotations
- Control panels and configurations
- [ ] **3.6 SQL Lab (Week 7-8)**
- `cypress/e2e/sqllab/` → `playwright/tests/sqllab/`
- Query execution, result handling
- Tab management, saved queries
- [ ] **3.7 Visualization Tests (Week 8)**
- `cypress/e2e/explore/visualizations/` →
`playwright/tests/visualizations/`
- Chart-specific testing (big_number, table, pie, etc.)
#### Per-Test Migration Process
1. **Analysis**: Review Cypress test for custom commands and dependencies
2. **Conversion**: Rewrite using Playwright APIs and patterns
3. **Enhancement**: Add cross-browser testing where beneficial
4. **Validation**: Compare results with Cypress baseline
5. **Documentation**: Update test descriptions and comments
**Success Criteria**:
- Each migrated test passes consistently
- No increase in flakiness compared to Cypress equivalent
- Performance improvements in test execution time
### Phase 4: Validation & Optimization (Weeks 7-9)
**Goal**: Validate Playwright reliability and optimize performance
#### Tasks
- [ ] **4.1 Parallel Execution Monitoring**
- Run both Cypress and Playwright in CI for all PRs
- Compare via CI logs and GitHub test reports:
- Test execution times
- Flakiness rates (retry counts)
- Resource usage (CI job duration)
- Coverage metrics (existing tools)
- [ ] **4.2 Performance Optimization**
- Optimize Playwright configuration for fastest execution
- Implement parallel test execution best practices
- Configure browser reuse and context management
- Tune timeouts and retry settings
- [ ] **4.3 Cross-Browser Testing**
- Enable Firefox and Safari testing where appropriate
- Add mobile viewport testing for responsive features
- Configure browser-specific test exclusions
- [ ] **4.4 Developer Experience**
- Create debugging guides and best practices
- Setup VS Code integration and extensions
- Document common patterns and utilities
**Success Criteria**:
- Playwright tests run 30-50% faster than Cypress
- Zero increase in false positives/negatives
- Team comfortable with Playwright tooling
### Phase 5: Cypress Sunset (Weeks 9-10)
**Goal**: Remove Cypress infrastructure and complete migration
#### Tasks
- [ ] **5.1 Final Migration**
- Convert any remaining Cypress-specific tests
- Handle edge cases and complex integrations
- Migrate Applitools visual tests or find alternatives
- [ ] **5.2 Cleanup**
- Remove Cypress dependencies from package.json
- Delete `superset-frontend/cypress-base/` directory
- Remove Cypress GitHub Actions workflow
- Clean up Docker configuration
- [ ] **5.3 Documentation**
- Update CLAUDE.md and other contributor guides
- Create Playwright testing best practices document
- Archive Cypress-specific documentation
- Update README and development setup guides
- [ ] **5.4 Training & Knowledge Transfer**
- Conduct team training sessions on Playwright
- Create troubleshooting guides
- Document common migration patterns
**Success Criteria**:
- No Cypress dependencies remain
- All tests migrated and passing
- Team fully trained on Playwright
- Documentation complete and accurate
## Risk Mitigation
### Technical Risks
- **Test Coverage Gaps**: Maintain both frameworks during transition
- **Performance Regression**: Benchmark and optimize continuously
- **Team Adoption**: Provide training and clear migration guides
- **CI/CD Stability**: Gradual rollout with rollback capability
### Mitigation Strategies
- Keep Cypress as backup during entire migration
- Create comprehensive test result comparisons
- Implement feature flags for Playwright adoption
- Maintain detailed migration documentation
## Success Metrics
### Performance
- [ ] Test execution time reduced by 30-50%
- [ ] CI pipeline stability maintained or improved
- [ ] Resource usage (memory/CPU) optimized
### Quality
- [ ] Zero increase in test flakiness
- [ ] Maintained or improved test coverage
- [ ] Cross-browser testing coverage added
### Developer Experience
- [ ] Reduced test maintenance overhead
- [ ] Improved debugging capabilities
- [ ] Better local development experience
## Timeline
**Total Duration**: 9-10 weeks
- **Phase 1**: Week 1 (Get One Test Working - MVP)
- **Phase 2**: Week 2 (Expand Infrastructure Based on Learnings)
- **Phase 3**: Weeks 3-7 (Migration)
- **Phase 4**: Weeks 7-9 (Validation)
- **Phase 5**: Weeks 9-10 (Cleanup)
## Resources
### Documentation
- [Playwright Migration Guide](https://playwright.dev/docs/intro)
- [Cypress to Playwright Migration
Examples](https://playwright.dev/docs/migrations)
### Key Benefits Expected
- **Native Parallelization**: Remove custom Python orchestration script
- **Cross-Browser Testing**: Chrome, Firefox, Safari support out of the box
- **Better Debugging**: Trace viewer, inspector, and screenshot tools
- **Mobile Testing**: Device emulation capabilities
- **Improved Reliability**: Better wait mechanisms and retry logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
