hitk6 opened a new pull request, #520:
URL: https://github.com/apache/atlas/pull/520
## ๐ฏ Overview
This PR migrates the Claude AI-powered PR review system from direct
Anthropic API to our internal **LiteLLM proxy** for cost optimization and
centralized LLM infrastructure management.
---
## ๐ Architecture Change
### Before
```
GitHub Actions
โ
anthropics/claude-code-action@v1
โ
Anthropic API (api.anthropic.com)
โ
Claude 3.5 Sonnet
```
### After
```
GitHub Actions
โ
Custom Python Scripts
โ
LiteLLM Proxy (llmproxy.atlan.dev)
โ
Claude (via internal infrastructure)
```
---
## ๐ What's Added
### ๐ Python Scripts (`.github/scripts/`)
| File | Purpose | Lines |
|------|---------|-------|
| `claude_review.py` | Main PR review script | 286 |
| `test_analysis.py` | Integration test analyzer | 213 |
| `test_litellm.sh` | Validation script | 77 |
| `requirements.txt` | Dependencies | 5 |
| `README.md` | Complete documentation | 207 |
### ๐ New Workflow (`.github/workflows/`)
**`claude-litellm.yml`** (187 lines)
- โ
Auto PR review on open/sync
- โ
Test analysis after integration tests
- โ
Cost-optimized (master/staging only)
- โ
Same review quality as before
---
## โจ Features
### Implemented โ
1. **Auto PR Review**
- Triggers on PR open/update (non-draft)
- Reviews Java code quality, security, performance
- Posts comprehensive review comment
- Same focus areas as original workflow
2. **Integration Test Analysis**
- Triggers after integration tests complete
- Parses JUnit XML results
- Analyzes failures and coverage gaps
- Posts actionable recommendations
3. **Cost Optimization**
- Test analysis only for master/staging PRs
- ~$0.50-$1.00 saved per feature branch PR
### Not Implemented (Can Add Later) โณ
- Interactive `@claude` mentions
- `/claude-qa-pipeline` command
- Inline review comments (currently single summary)
---
## ๐ฐ Cost Benefits
| Item | Before | After |
|------|--------|-------|
| **Per PR Review** | $0.50-$1.00 | Internal infrastructure |
| **Per Test Analysis** | $0.30 | Internal infrastructure |
| **Billing** | Anthropic account | No direct API charges |
| **Visibility** | Anthropic dashboard | LiteLLM metrics |
---
## ๐ง Setup Required
### 1. Add GitHub Secret โ ๏ธ **ACTION REQUIRED**
```bash
gh secret set LITELLM_API_KEY --repo atlanhq/atlas-metastore
```
**Value:** Your LiteLLM API key (contact infrastructure team)
Or via GitHub UI:
๐ https://github.com/atlanhq/atlas-metastore/settings/secrets/actions
### 2. Validate LiteLLM Connection
Run the test script to validate:
```bash
cd .github/scripts
LITELLM_KEY="your-key" ./test_litellm.sh
```
This will:
- โ
Test connection to proxy
- โ
List available models
- โ
Validate chat completion
- โ
Recommend model name
### 3. Update Model Name (If Needed)
If test script shows different model name, update workflow:
```yaml
# In claude-litellm.yml, line ~42 and ~159
--model "claude" # Change to actual model name
```
### 4. Enable Workflow
**Option A: Side-by-side (Recommended)**
```bash
# Disable old workflow
mv .github/workflows/claude.yml .github/workflows/claude.yml.disabled
# Test new workflow on a few PRs
# Delete old workflow once confident
```
**Option B: Direct replacement**
```bash
rm .github/workflows/claude.yml
mv .github/workflows/claude-litellm.yml .github/workflows/claude.yml
```
---
## ๐งช Testing This PR
### Step 1: Validate LiteLLM Key
See detailed validation steps below in "LiteLLM Key Validation Guide"
### Step 2: Test PR Review (After Merge)
1. Open a test PR
2. Watch workflow run: Actions โ Claude PR Review (LiteLLM)
3. Check for review comment on PR
4. Verify review quality
### Step 3: Test Analysis (After Merge)
1. Open PR that changes Java code
2. Wait for integration tests to complete
3. Check for test analysis comment
4. Verify insights are helpful
---
## ๐ Review Checklist
- [x] Scripts follow Python best practices
- [x] Workflow uses Python 3.11
- [x] Cost optimization implemented
- [x] Same review guidelines as original
- [x] Comprehensive documentation
- [x] Test script provided
- [ ] **TODO:** Add `LITELLM_API_KEY` secret (before merge)
- [ ] **TODO:** Run validation test
- [ ] **TODO:** Test on sample PR
---
## ๐ Documentation
Complete migration guide: `.github/scripts/README.md`
Includes:
- Setup instructions
- Local testing commands
- Troubleshooting guide
- Feature comparison
- Migration checklist
---
## ๐ Migration Path
1. **Merge this PR** (after adding secret)
2. **Disable old workflow** (`claude.yml`)
3. **Monitor first few PRs** (validate quality)
4. **Delete old workflow** (once confident)
5. **Update team docs** (new secret requirement)
---
## ๐ฏ Success Criteria
- โ
PR reviews work on open/sync
- โ
Test analysis works after integration tests
- โ
Review quality matches original workflow
- โ
Cost reduced (no direct Anthropic charges)
- โ
Team can use internal infrastructure
---
## ๐ Related
- Original workflow: `.github/workflows/claude.yml`
- LiteLLM Proxy: https://llmproxy.atlan.dev
- Anthropic Action: https://github.com/anthropics/claude-code-action
---
## ๐ Notes
- Python 3.11+ available in GitHub Actions runners
- Scripts use only `requests` library (minimal dependencies)
- Same review prompts and guidelines as original
- Can add more features (interactive, QA pipeline) if needed
---
**Ready to review!** ๐
Let me know if you need any changes or have questions.
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
