Hi everyone, As we expand the Fineract Business Intelligence repository, we are looking to establish a robust, standardized testing suite.
We would like to get the community's feedback and suggestions on the ideal testing frameworks, tools, and paradigms to adopt. Our Current Tech Stack Our project currently comprises: 1. Extractor CLI (Python): Connects to the Fineract source database (PostgreSQL) using pg8000 to extract raw tables and load them into the raw layer of the analytics warehouse. 2. Transformation Layer (dbt & PostgreSQL): Runs dbt models (dbt build) to transform raw tables into staging views, dimensional tables, facts, and final business presentation tables. 3. Visualization Layer (Apache Superset): Orchestrates analytics dashboards, with security roles and asset imports bootstrapped via custom Python scripts. 4. CI/CD (GitHub Actions): Launches Docker containers, runs SQL schemas and seed scripts, executes the extractor backfill, and runs a series of custom shell-based SQL smoke assertions to check metric integrity. Questions for the Community To align on an ideal testing framework, we would love to hear your thoughts on the following areas: 1. Python Application Testing (Extractor & Superset Bootstrap) Proposal: Add a standard unit testing directory. Question: Should we adopt pytest along with pytest-mock for mocking database connections and API endpoints? (These are widely used in Python-centric Apache projects like Airflow and Superset). Or should we stick to standard Python unittest libraries to minimize dependencies? 2. dbt SQL Model & Schema Testing Proposal: Move beyond simple schema checks. Question: What are the best practices you've seen for database testing in dbt? Should we introduce native dbt unit tests (mocking staging tables with inline values to assert fact/mart logic)? Should we adopt packages like dbt-expectations for validation boundaries (e.g. asserting collection efficiency ratios fall strictly within 0% to 200%)? 3. Database / Integration Testing Proposal: Test actual query behaviors. Question: To verify incremental watermarks and query boundaries against live SQL, should we look into spinning up ephemeral postgres containers during tests using pytest-postgresql or testcontainers-python? 4. CI Smoke Tests Refactoring Proposal: Replace the current shell-based SQL validation steps in .github/workflows/ci.yml. Question: Is it cleaner to maintain these as custom SQL-based dbt data tests, or should we build a lightweight Python validation script (e.g., using pytest) that queries the warehouse and runs the assertions? We want to make sure all selected testing libraries strictly comply with the ASF Category A license policies (which we already audit via our automated license checking scripts). Looking forward to your suggestions, advice, and references to similar testing patterns in Apache! Repo Link: https://github.com/apache/fineract-business-intelligence Best regards, Aira Jena
