This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6027-2c2b76b9ea8a0935aca6f798d35ad15baba26853 in repository https://gitbox.apache.org/repos/asf/texera.git
commit 3f877734610ec4ae077bf87ce2c653d7ff78a250 Author: Yicong Huang <[email protected]> AuthorDate: Mon Jun 29 16:54:11 2026 -0700 feat(local-dev): add --json status output and a non-TTY build heartbeat (#6027) ### What changes were proposed in this PR? Two small, agent/script-friendly additions to `bin/local-dev.sh` — no change to the human TTY experience. **1. `--json` machine-readable status.** `status --json` prints one JSON object on stdout (no colours, no table) and exits `0` iff every service is running, else `1`: ```json {"branch":"...","sha":"...","running":14,"total":14,"services":[ {"service":"texera-web","port":8080,"type":"jvm","pid":25823,"state":"running"}, ...]} ``` `up` and `down` also accept `--json`: human progress is routed to **stderr** (unbuffered) and the final status JSON goes to **stdout** (via a saved fd), so a caller can `up --json >state.json 2>progress.log` and parse stdout directly. **2. Non-TTY build heartbeat.** In non-TTY mode the spinner can't render in place, so a long silent step (`sbt dist`, output redirected to a log) used to print one line then go quiet for 25s+ — indistinguishable from "stuck" to a non-interactive caller. `tui_spinner` now emits `… still running (Ns)` every `TUI_HEARTBEAT_SECS` (default 15), polling at 1s so it still returns within ~1s of the job finishing (no trailing latency). ### Any related issues, documentation, discussions? Closes #6026. Usage banner (`--help`) updated to document `--json` on `status`/`up`/`down`. ### How was this PR tested? - `bash bin/local-dev/tests/test_local_dev_sh.sh` → **19 passed, 0 failed** (added 6: JSON shape/consistency, health-based exit code, unknown-flag negative case, `--help` coverage, heartbeat regression guard, up/down `--json` wiring). - `python -m pytest bin/local-dev/tests/` → **38 passed** (no regression). - Dogfooded end to end: `down --json` → stdout pure JSON (`running:0`, 14 `stopped`), exit 0; `up --json` → single-line JSON `running:14/14` on stdout, and stderr showed the new heartbeat across a ~60s build (`… still running (15s/30s/45s/60s)`), 14/14 healthy. - Verified stdout/stderr separation: `up --json >state.json 2>progress.log` yields parseable JSON in `state.json`. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) --- bin/local-dev/main.sh | 99 +++++++++++++++++++++++++++++--- bin/local-dev/tests/test_local_dev_sh.sh | 84 +++++++++++++++++++++++++++ 2 files changed, 175 insertions(+), 8 deletions(-) diff --git a/bin/local-dev/main.sh b/bin/local-dev/main.sh index 37290e6134..acc49362a4 100755 --- a/bin/local-dev/main.sh +++ b/bin/local-dev/main.sh @@ -29,14 +29,22 @@ # double-click for logs, ↑/↓ # history, Ctrl-C twice to quit). # Requires Python + textual. -# bin/local-dev.sh status same as no-arg invocation. -# bin/local-dev.sh up [--fresh|--build|--no-build] [--skip=svc1,svc2] +# bin/local-dev.sh status [--json] same as no-arg invocation. With +# --json, print one machine-readable +# JSON object (no table) and exit 0 +# iff every service is running — the +# contract for agents/scripts. +# bin/local-dev.sh up [--fresh|--build|--no-build] [--skip=svc1,svc2] [--json] # Default: skip build if no source/lock # changes since last build. --build forces # incremental sbt dist + yarn/bun install. # --fresh runs `sbt clean dist`. --no-build -# skips the build step entirely. -# bin/local-dev.sh down [--skip=svc1,svc2] stop every non-skipped service. +# skips the build step entirely. --json +# sends progress to stderr and the final +# status JSON to stdout. +# bin/local-dev.sh down [--skip=svc1,svc2] [--json] +# stop every non-skipped service +# (--json: summary JSON on stdout). # bin/local-dev.sh start <service> start one service (no rebuild). # bin/local-dev.sh stop <service> stop one service. # bin/local-dev.sh <service> rebuild only that service incrementally @@ -717,7 +725,24 @@ tui_state_color() { tui_spinner() { local pid="$1" msg="$2" if [[ ! -t 1 ]]; then - printf " ${BLUE}${SYM_PROG}${RESET} ${DIM}%s (no-TTY, no spinner)${RESET}\n" "$msg" + # No cursor control on a pipe, so we can't spin in place. Print one + # line up front, then a heartbeat every TUI_HEARTBEAT_SECS while the + # job runs — otherwise a long silent step (e.g. `sbt dist`, whose + # output is redirected to a log) looks hung to a non-interactive + # caller polling the stream. + printf " ${BLUE}${SYM_PROG}${RESET} ${DIM}%s (no-TTY)${RESET}\n" "$msg" + # Poll every 1s (so we return within ~1s of the job finishing — no + # trailing dead time) but only print a heartbeat every + # TUI_HEARTBEAT_SECS so the log stays readable. + local hb_start=$SECONDS hb_every="${TUI_HEARTBEAT_SECS:-15}" hb_last=0 hb_now=0 + while kill -0 "$pid" 2>/dev/null; do + sleep 1 + hb_now=$((SECONDS - hb_start)) + if (( hb_now - hb_last >= hb_every )); then + printf " ${BLUE}${SYM_PROG}${RESET} ${DIM}… still running (%ds)${RESET}\n" "$hb_now" + hb_last=$hb_now + fi + done return fi # Use an array (vs a single multibyte string + byte indexing) because @@ -1778,7 +1803,49 @@ refresh_node_deps() { } # --------- subcommands --------- +# Machine-readable counterpart to cmd_status: one JSON object on stdout, no +# colours, no decorative table. The stable contract for agents/scripts that +# would otherwise scrape the dashboard. Exit code mirrors health: 0 iff every +# service is running, else 1. +emit_status_json() { + local branch="" sha="" + branch=$(git -C "$REPO_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || echo "?") + sha=$(git -C "$REPO_ROOT" rev-parse --short HEAD 2>/dev/null || echo "?") + + local n_running=0 n_total=0 first=true svc="" type="" port="" state="" pid="" rows="" + for svc in "${SERVICES[@]}"; do + n_total=$((n_total+1)) + type=$(amap_get SVC_TYPE "$svc") + port=$(amap_get SVC_PORT "$svc") + pid="null" + if [[ "$type" == "docker" ]]; then + state=$(docker_svc_state "$svc") + case "$state" in running|exited) n_running=$((n_running+1)) ;; esac + else + local p="" + p=$(svc_running_pid "$svc") + if [[ -n "$p" ]]; then + state="running"; pid="$p"; n_running=$((n_running+1)) + else + state="stopped" + fi + fi + $first || rows+="," + first=false + rows+=$(printf '{"service":"%s","port":%s,"type":"%s","pid":%s,"state":"%s"}' \ + "$svc" "$port" "$type" "$pid" "$state") + done + printf '{"branch":"%s","sha":"%s","running":%d,"total":%d,"services":[%s]}\n' \ + "$branch" "$sha" "$n_running" "$n_total" "$rows" + (( n_running == n_total )) +} + cmd_status() { + case "${1:-}" in + --json) emit_status_json; return $? ;; + "") ;; + *) tui_err "unknown flag: $1" >&2; exit 2 ;; + esac local branch="" sha="" branch=$(git -C "$REPO_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || echo "?") sha=$(git -C "$REPO_ROOT" rev-parse --short HEAD 2>/dev/null || echo "?") @@ -1875,17 +1942,25 @@ cmd_up() { SKIP_LIST="" FRESH=false BUILD=auto # auto (skip if no source change) | force | no + JSON_OUT=false while [[ $# -gt 0 ]]; do case "$1" in --skip=*) SKIP_LIST="${1#--skip=}" ;; --fresh) FRESH=true; BUILD=force ;; --build) BUILD=force ;; --no-build) BUILD=no ;; + --json) JSON_OUT=true ;; *) tui_err "unknown flag: $1" >&2; exit 2 ;; esac shift done + # --json: the final summary on stdout must be pure JSON, so push all the + # human progress (banner, sections, in-place health panel) to stderr and + # keep the real stdout on fd 3 for emit_status_json. stderr is unbuffered, + # so a non-interactive caller still sees progress live on the side stream. + if $JSON_OUT; then exec 3>&1 1>&2; fi + local n_skip=0 [[ -n "$SKIP_LIST" ]] && n_skip=$(echo "$SKIP_LIST" | tr ',' '\n' | wc -l | tr -d ' ') local skip_label="none" @@ -1916,6 +1991,7 @@ cmd_up() { tui_ok "no source/lock changes since last build" tui_ok "all ${#SERVICES[@]} services already running" printf "\n ${BOLD}${GREEN}${SYM_OK} nothing to do${RESET} ${DIM}(use \`u --build\` to force a rebuild, or \`<svc>\` to bounce just one)${RESET}\n\n" + $JSON_OUT && { emit_status_json >&3 || true; } return 0 fi fi @@ -2011,7 +2087,7 @@ cmd_up() { fi printf "\n" - cmd_status + if $JSON_OUT; then emit_status_json >&3 || true; else cmd_status; fi [[ $ec -eq 0 ]] } @@ -2252,13 +2328,17 @@ cmd_auto() { cmd_down() { SKIP_LIST="" + JSON_OUT=false while [[ $# -gt 0 ]]; do case "$1" in --skip=*) SKIP_LIST="${1#--skip=}" ;; + --json) JSON_OUT=true ;; *) tui_err "unknown flag: $1" >&2; exit 2 ;; esac shift done + # See cmd_up: human progress to stderr, JSON summary on real stdout (fd 3). + if $JSON_OUT; then exec 3>&1 1>&2; fi tui_banner "Texera Local Dev — stopping stack" "skip=${SKIP_LIST:-none}" tui_section "Stopping (reverse order)" local svc="" @@ -2285,6 +2365,8 @@ cmd_down() { done $has_docker_targets && infra_down printf "\n" + $JSON_OUT && { emit_status_json >&3 || true; } + return 0 } cmd_update_one() { @@ -2556,7 +2638,8 @@ cmd_interactive() { _precompute_src_dirs case "${1:-}" in - ""|status) cmd_status ;; # default: one-shot dashboard (safe in scripts/CI) + "") cmd_status ;; # default: one-shot dashboard (safe in scripts/CI) + status) shift; cmd_status "$@" ;; # `status [--json]` -i|--interactive) cmd_interactive ;; # opt in to the live TUI up) shift; cmd_up "$@" ;; auto) shift; cmd_auto "$@" ;; @@ -2566,6 +2649,6 @@ case "${1:-}" in logs) shift; cmd_logs "${1:-}" ;; w|watch) shift; cmd_watch "${1:-2}" ;; version) printf "%s\n" "$TEXERA_VERSION" ;; - -h|--help) sed -n '18,67p' "$0" ;; + -h|--help) sed -n '18,75p' "$0" ;; *) cmd_update_one "$1" ;; esac diff --git a/bin/local-dev/tests/test_local_dev_sh.sh b/bin/local-dev/tests/test_local_dev_sh.sh index b19c77b54b..390b83fdbd 100755 --- a/bin/local-dev/tests/test_local_dev_sh.sh +++ b/bin/local-dev/tests/test_local_dev_sh.sh @@ -230,5 +230,89 @@ for fn in cmd_up cmd_auto; do fi done +# 12) `status --json` emits a single machine-readable JSON object — the stable +# contract for agents/scripts that would otherwise grep the dashboard. +# Must parse, expose running/total/services, list every service exactly +# once, and stay internally consistent (len(services)==total, running<=total). +if command -v python3 >/dev/null 2>&1; then + json_out=$("$SCRIPT" status --json 2>/dev/null) + if printf '%s' "$json_out" | python3 -c ' +import sys, json +d = json.load(sys.stdin) +assert isinstance(d["services"], list), "services not a list" +assert isinstance(d["running"], int) and isinstance(d["total"], int) +assert d["total"] == len(d["services"]), "total != len(services)" +assert 0 <= d["running"] <= d["total"], "running out of range" +names = {s["service"] for s in d["services"]} +need = {"texera-web", "frontend", "postgres"} +assert need <= names, f"missing services: {need - names}" +for s in d["services"]: + assert isinstance(s["port"], int), "port not int" + assert s["type"] in {"jvm", "docker", "yarn", "bun"}, "bad service type" + assert s["pid"] is None or isinstance(s["pid"], int), "pid not int|null" +' 2>/tmp/.local-dev-json.err; then + _pass "status --json emits valid, consistent JSON with all services" + else + _fail "status --json invalid/inconsistent" \ + "$(tail -1 /tmp/.local-dev-json.err 2>/dev/null); out=$(printf '%s' "$json_out" | head -c 160)" + fi + rm -f /tmp/.local-dev-json.err + + # 13) Exit code mirrors health: 0 iff running == total, else 1. Lets an + # agent gate on `if status --json; then` without parsing the body. + running=$(printf '%s' "$json_out" | python3 -c 'import sys,json;print(json.load(sys.stdin)["running"])' 2>/dev/null) + total=$(printf '%s' "$json_out" | python3 -c 'import sys,json;print(json.load(sys.stdin)["total"])' 2>/dev/null) + "$SCRIPT" status --json >/dev/null 2>&1; rc_json=$? + if { [[ "$running" == "$total" ]] && (( rc_json == 0 )); } \ + || { [[ "$running" != "$total" ]] && (( rc_json == 1 )); }; then + _pass "status --json exit code reflects health (running=$running total=$total rc=$rc_json)" + else + _fail "status --json exit code wrong" "running=$running total=$total rc=$rc_json" + fi +else + _pass "skip: python3 not on PATH (status --json shape check)" +fi + +# 14) Negative: an unknown flag to `status` must refuse with rc 2 and a clear +# message — bad input is not silently ignored. +out=$("$SCRIPT" status --definitely-bogus 2>&1) +rc=$? +if (( rc == 2 )) && [[ "$out" == *"unknown flag"* ]]; then + _pass "status rejects unknown flag (rc=2, clear error)" +else + _fail "status didn't reject unknown flag" "rc=$rc out=$(echo "$out" | head -1)" +fi + +# 15) `--help` documents --json so the contract is discoverable. +help_out=$("$SCRIPT" --help 2>&1) +if [[ "$help_out" == *"--json"* ]]; then + _pass "--help documents --json" +else + _fail "--help doesn't mention --json" +fi + +# 16) Regression: in non-TTY mode tui_spinner can't spin in place, so a long +# silent step (sbt dist → log) must emit a heartbeat or it looks hung to a +# non-interactive caller. Guard the sentinel inside the function body. +spinner_body=$(awk '/^tui_spinner\(\)/{f=1} f{print} f&&/^}/{exit}' "$REPO_ROOT/bin/local-dev/main.sh") +if [[ "$spinner_body" == *"! -t 1"* && "$spinner_body" == *"still running"* && "$spinner_body" == *"kill -0"* ]]; then + _pass "tui_spinner emits a non-TTY heartbeat (no silent long-running steps)" +else + _fail "tui_spinner missing non-TTY heartbeat loop" +fi + +# 17) `up` and `down` accept --json (route the human stream to stderr, emit the +# JSON summary on stdout). Structural guard — invoking them for real would +# build/stop the stack, out of scope here. +for fn in cmd_up cmd_down; do + body=$(awk -v fn="$fn" '$0 ~ "^" fn "\\(\\)" {f=1} f{print} f&&/^}/{exit}' \ + "$REPO_ROOT/bin/local-dev/main.sh") + if [[ "$body" == *"--json"* && "$body" == *"emit_status_json"* ]]; then + _pass "$fn accepts --json and emits JSON summary" + else + _fail "$fn doesn't wire up --json" + fi +done + printf "\n%d passed, %d failed\n" "$PASS" "$FAIL" (( FAIL == 0 ))
