The GitHub Actions job "Required Checks" on texera.git/feat/scan-report-skipped-rows has failed. Run started by GitHub user eugenegujing (triggered by eugenegujing).
Head commit for run: 3ba7130a3022a70aae84f9630596d15eab300fa5 / eugenegujing <[email protected]> feat(workflow-operator): skip and report scan rows that do not match the inferred schema CSV/JSONL scan sources infer each column's type from only the first INFER_READ_LIMIT (=100) rows. When a later row held a value that did not parse as the inferred type, the execs silently dropped the entire row — no error, no warning, no count. Downstream counts, aggregates, and joins then ran on a silently truncated dataset. Per the consensus in discussion #6324 (skip + surface, instead of the fail-fast approach originally proposed in #6323), a scan now still skips the unparsable row — the run completes and existing workflows keep working — but each skipped row is surfaced to the user as a console warning naming the row number, offending value, column, and expected type, e.g.: WARNING: skipped row 150 — value '55.5' in column 'age' cannot be read as INTEGER. Column types were inferred from the first 100 rows of the file, and this value does not match. Changes: - Add ScanRowParseError, which builds the per-row warning by re-parsing the failing row's fields to identify the offending column, with a generic fallback when no single column is identifiable (e.g. a malformed JSON line). - Add SkippedRowReporter, shared by all scan variants: reports the first 100 skipped rows individually, then a single summary line with the true total, so a heavily malformed file cannot flood the console or exhaust memory. - Record-and-skip in CSVScanSourceOpExec, JSONLScanSourceOpExec, ParallelCSVScanSourceOpExec, and CSVOldScanSourceOpExec. Legitimate null paths (empty cell parsed to null, blank/all-null line, exhausted block) are untouched and not reported. Row numbers: CSV uses its existing counter, JSONL/csvOld gain absolute line/data-row numbers, ParallelCSV reports none (byte-partitioned across workers). - Add OperatorExecutor.getWarnings (defaults to empty), a generic non-fatal warning channel for executors. - Emit the warnings at FinalizeExecutor in DataProcessor as PRINT console messages via the new ErrorUtils.mkPrintConsoleMessage. Titles keep the "WARNING: " prefix the UI keys on. Unlike the executor exception path, this does not pause the run. Tests: scan exec specs cover skip-and-report (row/value/column/type), the 100-detail cap with summary, empty-cell and blank-line non-reporting, and the malformed-JSON fallback; DataProcessorSpec covers the PRINT emission at finalize and asserts the run is not paused. Closes #6279 Report URL: https://github.com/apache/texera/actions/runs/30308491583 With regards, GitHub Actions via GitBox
