aglinxinyuan opened a new issue, #7124:
URL: https://github.com/apache/texera/issues/7124
### Task Summary
Two retry loops were left hand-rolled when #7119 consolidated the rest onto
`RetryUtil.withBackoff`, because converting them changes runtime behavior
rather than just structure. Each needs a decision before it can move.
**1. `URLFetchUtil.getInputStreamFromURL`** (`common/workflow-operator`)
```scala
def getInputStreamFromURL(urlObj: URL, retries: Int = 5): Option[InputStream]
```
- Retries 5x with **no delay at all**, and returns `Option` rather than
throwing.
- Adopting the util's backoff means a dead URL costs ~3 s (200+400+800+1600)
before giving up, where today it fails immediately. That is arguably the fix —
an unthrottled 5x retry against a remote host is a small retry storm — but it
changes how long a URL-scan operator takes to fail.
- The `Option` return is a one-line adapter (`Try(...).toOption`), so the
delay is the only real question.
**2. `PythonProxyClient`'s Flight-connect loop** (`amber`)
```scala
while (!connected && tryCount <= MAX_TRY_COUNT) { ...
Thread.sleep(UNIT_WAIT_TIME_MS) ... }
```
- Waits a **constant** `UNIT_WAIT_TIME_MS` between attempts, not a doubling
one, so the util needs a delay-multiplier knob (`1` = constant) before it can
host this.
- Closes the Flight client between attempts; that per-attempt cleanup has to
move into the operation body.
- Throws `WorkflowRuntimeException` on give-up, where the util wraps in
`RuntimeException`. Either the util grows a give-up hook, or this call site
accepts the different type — worth deciding deliberately, since worker-bring-up
failures are surfaced to users.
Both are candidates, not obligations: if the answer for either is "the
current behavior is what we want", closing this with that note recorded is a
fine outcome.
### Task Type
- [x] Refactor / Cleanup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
