airajena opened a new pull request, #69: URL: https://github.com/apache/fineract-business-intelligence/pull/69
## Summary - Two singular dbt tests referenced `fact_loan_snapshot` and raw tables as plain SQL instead of `ref()`/`source()`, so dbt couldn't schedule them correctly caused nondeterministic "relation does not exist" errors depending on build thread timing. - CI's integration-test job inlined a hand-copied, already-drifted view SQL block instead of calling `bootstrap_source.sh` now calls the real script, so CI and local dev can never diverge again. - `bootstrap_source.sh` / `bootstrap_fineract_source.sh` weren't safely re-runnable: `CREATE OR REPLACE VIEW` can't reorder a view's columns, so re-running against `batch_job_execution` failed. Wrapped in `BEGIN`/`COMMIT` and added `DROP VIEW IF EXISTS` first. - `run_pipeline.sh` had no protection against two runs overlapping added a `flock`-based lock, and auto-runs `dbt deps` so a fresh clone doesn't need a manual step. - `.gitignore` was missing `.venv/`. ## Test plan - [x] `bootstrap_source.sh` run 3x back-to-back : clean, idempotent - [x] `dbt build --full-refresh` : 116/116, deterministic across repeated runs - [x] Full backfill and incremental extraction verified against a live source DB, including delete propagation - [x] Raced two concurrent `run_pipeline.sh` invocations : second correctly rejected by the lock - [x] Full pipeline (backfill + incremental) completes end-to-end including Superset refresh - [x] RLS integration suite : 46/46 passing - [x] shellcheck + sqlfluff clean on all modified files - [x] `ci.yml` validated as syntactically correct YAML Fixes: #68 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
