amoghrajesh commented on PR #56762:
URL: https://github.com/apache/airflow/pull/56762#issuecomment-3430828820
## Test Setup
1. Testing baseline
```python
"""Test baseline: pydantic + httpx only"""
import pydantic
import httpx
from pydantic import BaseModel
class TestModel(BaseModel):
name: str
model = TestModel(name="test")
client = httpx.Client()
client.close()
print("Baseline: pydantic + httpx loaded")
```
2. Testing with tenacity
```python
"""Test tenacity with equivalent retry configuration"""
import pydantic
import httpx
import tenacity
from pydantic import BaseModel
from tenacity import retry, retry_if_exception, stop_after_attempt,
wait_exponential
class TestModel(BaseModel):
name: str
model = TestModel(name="test")
client = httpx.Client()
client.close()
# Equivalent retry configuration
@retry(
retry=retry_if_exception(lambda e: isinstance(e, Exception)),
stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=0.1, max=1.0),
reraise=True,
)
def dummy():
pass
print("Tenacity with equivalent config loaded")
```
3. Testing with stamina
```python
"""Test stamina with equivalent retry configuration"""
import pydantic
import httpx
import stamina
from pydantic import BaseModel
class TestModel(BaseModel):
name: str
model = TestModel(name="test")
client = httpx.Client()
client.close()
# Equivalent retry configuration
@stamina.retry(
on=Exception,
attempts=3,
wait_initial=0.1,
wait_max=1.0,
wait_jitter=1.0,
)
def dummy():
pass
print("Stamina with equivalent config loaded")
```
4. Testing with retryhttp
```python
"""Test retryhttp with equivalent retry configuration"""
import pydantic
import httpx
import retryhttp
from pydantic import BaseModel
from retryhttp import retry
class TestModel(BaseModel):
name: str
model = TestModel(name="test")
client = httpx.Client()
client.close()
# Equivalent retry configuration (retryhttp uses different API)
@retry(
max_attempt_number=3,
# retryhttp uses different wait strategies - using default exponential
)
def dummy():
pass
print("RetryHTTP with equivalent config loaded")
```
### Run profiling
```bash
memray run --output baseline.bin test_baseline.py
memray run --output tenacity.bin test_tenacity.py
memray run --output stamina.bin test_stamina.py
memray run --output retryhttp.bin test_retryhttp.py
memray summary baseline.bin
memray summary tenacity.bin
memray summary stamina.bin
memray summary retryhttp.bin
```
### Benchmark Results
| Library | Total Memory | Overhead | Savings vs RetryHTTP | % Reduction |
|---------|-------------|----------|---------------------|-------------|
| **Baseline** (pydantic + httpx) | 13.62 MB | - | - | - |
| **Tenacity** (equivalent config) | 17.03 MB | +3.41 MB | **4.22 MB** |
**50%** |
| **Stamina** (equivalent config) | 17.65 MB | +4.03 MB | **3.60 MB** |
**42%** |
| **RetryHTTP** (equivalent config) | 21.25 MB | +7.63 MB | - | - |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
