This is an automated email from the ASF dual-hosted git repository. asf-gitbox-commits pushed a commit to branch openformula in repository https://gitbox.apache.org/repos/asf/openoffice.git
commit 282f245e94195566745f183bbc4dc3bc5a729194 Author: Pedro Giffuni <[email protected]> AuthorDate: Wed Nov 11 16:17:38 2026 -0500 docs/test: correct OpenFormula Small Group inventory and E.1 test data Fix Basic Limits formula-length and nesting tests, add criteria syntax coverage, align COUNTA/COUNTBLANK and out-of-constraint DATE cases with ODF 1.4, and reconcile Small Group audit/plan docs. Test data/docs only; runtime verification blocked (no AOO build, calcval macOS-only). --- openformula-modernization-plan.md | 16 +++++++++--- openformula-small-group-1.4.tsv | 16 ++++++------ openformula-small-group-audit.md | 30 +++++++++++++++------ openformula-small-group-test-plan.md | 18 ++++++++++--- openformula-test-gap-analysis.md | 13 +++++++-- test/odf-validate/calcval-cases14.tsv | 6 +++-- test/odf-validate/openformula-small-group-1.4.tsv | 32 +++++++++++++---------- 7 files changed, 90 insertions(+), 41 deletions(-) diff --git a/openformula-modernization-plan.md b/openformula-modernization-plan.md index 0837cad8db..2fd62b8d13 100644 --- a/openformula-modernization-plan.md +++ b/openformula-modernization-plan.md @@ -61,7 +61,7 @@ Source-of-truth: `openformula-functions.tsv`, `openformula-operators.tsv`, `open ## 2. Dependencies between proposed changes -1. COUNTA/COUNTBLANK change should be coordinated with any document-compatibility option; test against existing corpus before enabling. +1. COUNTA/COUNTBLANK change should be coordinated with any document-compatibility option; test against existing corpus before enabling. E.1 clarified fixtures/impl-defined `""` handling but did not change implementation. 2. Test inventories depend on having the ODF 1.4 function list extracted; do P3.1 before P3.2/P3.3. ## 3. Estimated implementation complexity @@ -69,7 +69,7 @@ Source-of-truth: `openformula-functions.tsv`, `openformula-operators.tsv`, `open | Item | Complexity | Rationale | |------|------------|-----------| | COUNTA/COUNTBLANK empty-cell semantics | Medium | Behavior change; regression risk medium; needs corpus testing. | -| DATE pre-1583 documentation | Low | Document existing impl-defined behavior; no code change needed. | +| DATE out-of-constraint/pre-1583 behavior | Low | Document impl-defined/diagnostic behavior for inputs outside §6.10.2 constraints (e.g. Year < 1904); no code change needed. E.1 reclassified tests. | | INDEX row-vector regression testing | Medium | Corpus regression run; may need compatibility option. | | Test inventories | Medium | Requires ODF 1.4 Part 4 text extraction; tooling exists (formulacheck.py). | | Semantic/parser/round-trip tests | Medium-High | Many cases; runtime validation needed. | @@ -91,8 +91,16 @@ Source-of-truth: `openformula-functions.tsv`, `openformula-operators.tsv`, `open ## 6. Definition of done for P1 -- COUNTA/COUNTBLANK: verified against ODF 1.4 §6.13.7/§6.13.8; documented any semantic differences. -- DATE pre-1583: documented impl-defined behavior per ODF 1.4 §7.4. +- COUNTA/COUNTBLANK: verify against ODF 1.4 §6.13.7/§6.13.8 (empty string non-blank for COUNTA; `""`-in-cell COUNTBLANK impl-defined); document semantic differences. E.1 test data/docs corrected. +- DATE out-of-constraint/pre-1583: impl-defined/diagnostic per §6.10.2 constraints; E.1 reclassified `FN-016-DATE-pre1583` accordingly. - INDEX row-vector: regression-tested against document corpus. - Versioned test inventories created (not in production build). - All findings documented with file:line references (this audit's deliverables). + + +## E.1 Test inventory corrections + +- Corrected Small Group Basic Limits tests: `SG-BL-001` uses a 1026-char formula expression (expected 513), `SG-BL-004` uses 7 genuinely nested IF levels. +- Added `SG-SYN-001` (Criteria syntax §4.11.11) and documented `SG-SYN-009` named-expression harness dependency (`MYNAME`). +- Updated COUNTA/COUNTBLANK and DATE out-of-constraint fixtures/classifications in `openformula-small-group-1.4.tsv`, `test/odf-validate/openformula-small-group-1.4.tsv`, and `test/odf-validate/calcval-cases14.tsv`. +- No runtime verification performed (no build; calcval macOS-only). diff --git a/openformula-small-group-1.4.tsv b/openformula-small-group-1.4.tsv index 968dd7ccff..6394cc8ad3 100644 --- a/openformula-small-group-1.4.tsv +++ b/openformula-small-group-1.4.tsv @@ -1,9 +1,9 @@ ReqID Section Requirement Description Mandatory/Optional Conformance Group ODF Version Introduced/Changed Suggested Test Status -SG-BL-001 3.7 Formula length: at least 1024 characters long as measured in OpenDocument interchange format Mandatory Small ODF 1.4 Parse a 1024+ character formula; verify accepted. UNVERIFIED +SG-BL-001 3.7 Formula length: at least 1024 characters long as measured in OpenDocument interchange format Mandatory Small ODF 1.4 Parse formula expression >=1024 chars (e.g. =1+1+...+1, 1026 chars, result 513); verify accepted. UNVERIFIED SG-BL-002 3.7 At least 30 parameters per function when prototype permits parameter list Mandatory Small ODF 1.4 Call function with 30 parameters; verify accepted. UNVERIFIED SG-BL-003 3.7 Permit strings of ASCII characters up to 32767 (2^15-1) characters Mandatory Small ODF 1.4 Parse/return 32767-character string; verify accepted. UNVERIFIED -SG-BL-004 3.7 Support at least 7 nesting levels of functions Mandatory Small ODF 1.4 Nest 7 levels of functions; verify accepted. UNVERIFIED -SG-SYN-001 4.11.11 Criteria syntax supported Mandatory Small ODF 1.4 Use criteria in database functions; verify parsed correctly. UNVERIFIED +SG-BL-004 3.7 Support at least 7 nesting levels of functions Mandatory Small ODF 1.4 Nest >=7 genuinely nested function levels with balanced parens and deterministic result (e.g. 7 nested IFs); verify accepted. UNVERIFIED +SG-SYN-001 4.11.11 Criteria syntax (§4.11.11) supported Mandatory Small ODF 1.4 Use criteria in database functions (e.g. DCOUNT with criteria range); verify parsed/evaluated correctly. UNVERIFIED SG-SYN-002 5.2 Basic Expressions syntax supported Mandatory Small ODF 1.4 Parse parenthesized expressions; verify evaluation. UNVERIFIED SG-SYN-003 5.3 Constant Numbers syntax supported Mandatory Small ODF 1.4 Parse numeric literals including decimals, scientific notation; verify. UNVERIFIED SG-SYN-004 5.4 Constant Strings syntax supported Mandatory Small ODF 1.4 Parse string literals with quote escaping; verify. UNVERIFIED @@ -11,7 +11,7 @@ SG-SYN-005 5.5 Operators syntax supported (precedence per Table 1) Mandatory Sma SG-SYN-006 5.6 Functions and Function Parameters syntax supported Mandatory Small ODF 1.4 Call functions with various argument counts; verify. UNVERIFIED SG-SYN-007 5.7 Nonstandard Function Names recognized as errors Mandatory Small ODF 1.4 Use unknown function name; verify error result. UNVERIFIED SG-SYN-008 5.8 References syntax supported Mandatory Small ODF 1.4 Use cell and range references; verify resolution. UNVERIFIED -SG-SYN-009 5.11 Simple Named Expressions supported Mandatory Small ODF 1.4 Define and use named expression; verify. UNVERIFIED +SG-SYN-009 5.11 Simple Named Expressions supported Mandatory Small ODF 1.4 Define and use a simple named expression; verify. Requires pre-defined name in test environment (not expressible as plain calcval cell formula without name setup). UNVERIFIED SG-SYN-010 5.12 Errors syntax supported Mandatory Small ODF 1.4 Use error constants (#N/A, etc.); verify propagation. UNVERIFIED SG-SYN-011 5.14 Whitespace handling per specification Mandatory Small ODF 1.4 Insert whitespace in formula; verify identical result. UNVERIFIED SG-CONV-001 6.3.14 Conversion to Text implemented Mandatory Small ODF 1.4 Convert number/date/logical to text; verify result. IMPLEMENTED_UNVERIFIED @@ -31,7 +31,7 @@ SG-OP-009 6.4.8 Infix Operator "<>" implemented Mandatory Small ODF 1.4 Evaluate SG-OP-010 6.4.14 Postfix Operator "%" implemented Mandatory Small ODF 1.4 Evaluate 50%; verify 0.5. IMPLEMENTED_UNVERIFIED SG-OP-011 6.4.15 Prefix Operator "+" implemented Mandatory Small ODF 1.4 Evaluate +5; verify 5. IMPLEMENTED_UNVERIFIED SG-OP-012 6.4.16 Prefix Operator "-" implemented Mandatory Small ODF 1.4 Evaluate -5; verify -5. IMPLEMENTED_UNVERIFIED -SG-OP-013 6.4.12 Infix Operator Reference Intersection ("!") implemented Mandatory Small ODF 1.4 Evaluate A1:C3!B1:D2; verify intersection. IMPLEMENTED_UNVERIFIED +SG-OP-013 6.4.12 Infix Operator Reference Intersection ("!") implemented Mandatory Small ODF 1.4 Evaluate numeric range intersection (e.g. SUM(D1:F3!E1:F2)); verify intersection. IMPLEMENTED_UNVERIFIED SG-OP-014 6.4.11 Infix Operator Range (":") implemented Mandatory Small ODF 1.4 Evaluate SUM(A1:A3); verify range. IMPLEMENTED_UNVERIFIED SG-FN-001 6.16.2 ABS Mandatory Small ODF 1.2 Evaluate ABS(-3); verify 3. IMPLEMENTED_UNVERIFIED SG-FN-002 6.16.3 ACOS Mandatory Small ODF 1.2 Evaluate ACOS(0); verify pi/2. IMPLEMENTED_UNVERIFIED @@ -45,10 +45,10 @@ SG-FN-009 6.14.3 CHOOSE Mandatory Small ODF 1.2 Evaluate CHOOSE(2,"a","b","c"); SG-FN-010 6.13.5 COLUMNS Mandatory Small ODF 1.2 Evaluate COLUMNS(A1:C1); verify 3. IMPLEMENTED_UNVERIFIED SG-FN-011 6.16.19 COS Mandatory Small ODF 1.2 Evaluate COS(0); verify 1. IMPLEMENTED_UNVERIFIED SG-FN-012 6.13.6 COUNT Mandatory Small ODF 1.2 Evaluate COUNT(1,2,"x"); verify 2. IMPLEMENTED_UNVERIFIED -SG-FN-013 6.13.7 COUNTA Mandatory Small ODF 1.2 Evaluate COUNTA(1,"x",""); verify non-empty count. IMPLEMENTED_UNVERIFIED -SG-FN-014 6.13.8 COUNTBLANK Mandatory Small ODF 1.2 Evaluate COUNTBLANK(range); verify blank count. IMPLEMENTED_UNVERIFIED +SG-FN-013 6.13.7 COUNTA Mandatory Small ODF 1.2 Evaluate COUNTA over numbers, text, empty cells, and empty string ""; verify non-blank count per §6.13.7. IMPLEMENTED_UNVERIFIED +SG-FN-014 6.13.8 COUNTBLANK Mandatory Small ODF 1.2 Evaluate COUNTBLANK over truly empty cells; ""-in-cell blankness is implementation-defined per §6.13.8. IMPLEMENTED_UNVERIFIED SG-FN-015 6.13.9 COUNTIF Mandatory Small ODF 1.2 Evaluate COUNTIF(range,criteria); verify count. IMPLEMENTED_UNVERIFIED -SG-FN-016 6.10.2 DATE Mandatory Small ODF 1.2 Evaluate DATE(2025,1,1); verify serial. IMPLEMENTED_UNVERIFIED +SG-FN-016 6.10.2 DATE Mandatory Small ODF 1.2 Evaluate DATE(2025,1,1); verify serial. Values outside §6.10.2 constraints (e.g. year < 1904) are impl-defined/diagnostic. IMPLEMENTED_UNVERIFIED SG-FN-017 6.9.2 DAVERAGE Mandatory Small ODF 1.2 Evaluate DAVERAGE(database,field,criteria); verify average. IMPLEMENTED_UNVERIFIED SG-FN-018 6.10.5 DAY Mandatory Small ODF 1.2 Evaluate DAY(DATE(2025,3,15)); verify 15. IMPLEMENTED_UNVERIFIED SG-FN-019 6.9.3 DCOUNT Mandatory Small ODF 1.2 Evaluate DCOUNT(database,field,criteria); verify count. IMPLEMENTED_UNVERIFIED diff --git a/openformula-small-group-audit.md b/openformula-small-group-audit.md index b6d387d5e8..80a6d08d32 100644 --- a/openformula-small-group-audit.md +++ b/openformula-small-group-audit.md @@ -33,10 +33,10 @@ Therefore **no conformance level (Small/Medium/Large) may be claimed for AOO Cal | ReqID | Requirement | Status | Source Evidence | Runtime Verified | |-------|-------------|--------|-----------------|-------------------| -| SG-BL-001 | Formula length >= 1024 characters | **UNVERIFIED** | FormulaCompiler parses token arrays dynamically; no fixed 1024-char limit found in source | No | +| SG-BL-001 | Formula length >= 1024 characters | **UNVERIFIED** | FormulaCompiler parses token arrays dynamically; no fixed 1024-char limit found in source. E.1 test corrected to 1026-char expression (`=1`+`+1`*512, result 513). | No | | SG-BL-002 | >= 30 parameters per function | **UNVERIFIED** | MustHaveParamCount validates exact counts; no 30-param cap found | No | | SG-BL-003 | Strings up to 32767 ASCII chars | **UNVERIFIED** | OUString used for strings; no specific 32767 limit found | No | -| SG-BL-004 | >= 7 nesting levels of functions | **UNVERIFIED** | No nesting depth limit found in compiler/interpreter | No | +| SG-BL-004 | >= 7 nesting levels of functions | **UNVERIFIED** | No nesting depth limit found in compiler/interpreter. E.1 test corrected to 7 genuinely nested IF levels with balanced parens. | No | **Finding:** The absence of hard-coded limits in source does **not** prove conformance. All four Basic Limits requirements are marked UNVERIFIED pending actual runtime tests. @@ -146,13 +146,13 @@ The Small Group requires 110 functions (§2.3.2.E). All 110 have opcodes and imp | Category | Total | Status | Count | |----------|-------|--------|-------| | Basic Limits | 4 | UNVERIFIED runtime | 0 verified, 4 UNVERIFIED | -| Syntax | 11 | UNVERIFIED runtime | 0 verified, 11 UNVERIFIED | +| Syntax | 11 | UNVERIFIED runtime | 0 verified, 11 UNVERIFIED (SG-SYN-001 criteria §4.11.11 now included in E.1 test inventory) | | Conversions | 5 | IMPLEMENTED_UNVERIFIED | 0 verified, 5 UNVERIFIED | | Operators | 14 | IMPLEMENTED_UNVERIFIED | 0 verified, 14 UNVERIFIED | | Functions | 110 | IMPLEMENTED_UNVERIFIED | 0 verified, 110 UNVERIFIED | | Allowances | 4 | Documented | 4 documented | -**Grand total: 148 requirements, 0 runtime-verified, 148 UNVERIFIED.** +**Grand total: 148 requirements (4 Basic Limits + 11 Syntax + 5 Conversions + 14 Operators + 110 Functions + 4 Allowances), 0 runtime-verified, 148 UNVERIFIED.** --- @@ -162,12 +162,26 @@ The Small Group requires 110 functions (§2.3.2.E). All 110 have opcodes and imp |---------------|---------|-------------|-------------------|----------|--------| | INDEX row-vector single-arg | 6.14.6 | Returns error (Milestone A fix: removed ODF 1.3 row-vector shortcut) | Error (ODF 1.4 removed rule) | Fixed in Milestone A; regression risk | interpr1.cxx; KNOWN-LIMITATIONS:71 | | CONVERT atm | 6.16.18 | 101325.1 Pa (Milestone C fix: corrected from 101324.996583 Pa) | 101325.1 Pa | Fixed in Milestone C | interpr2.cxx; KNOWN-LIMITATIONS:72 | -| DATE pre-1583 | 6.10.2 | #VALUE! before 15 Oct 1582 | Support from 1583; earlier impl-defined | Documented impl-defined | interpr2.cxx; IMPLEMENTATION-DEFINED:155 | -| COUNTA empty cells | 6.13.7 | ODF 1.3 semantics | ODF 1.4 changed empty-cell handling | UNVERIFIED | interpr1.cxx:ScCount2 | -| COUNTBLANK "" | 6.13.8 | Unverified | ODF 1.4: "" case impl-defined | UNVERIFIED | interpr1.cxx:ScCountEmptyCells | +| DATE pre-1583 (FN-016-DATE-pre1583) | 6.10.2 | AOO observed #VALUE! for `DATE(1500;1;1)` (outside 1904 ≤ Year ≤ 9956) | §6.10.2 constrains Year/Month/Day; evaluators may evaluate out-of-constraint but `DATE(1500;1;1)` is implementation-defined/diagnostic, not a mandatory `#ERR` | Impl-defined/diagnostic (E.1 reclassified) | interpr2.cxx; IMPLEMENTATION-DEFINED:155 | +| COUNTA empty cells / empty string | 6.13.7 | ODF 1.3 semantics | ODF 1.4 §6.13.7: counts non-blank cells; empty string `""` is not blank. E.1 adds fixtures for empty cells and `""` (where harness-expressible). | UNVERIFIED | interpr1.cxx:ScCount2 | +| COUNTBLANK "" in cell | 6.13.8 | Unverified | §6.13.8: implementation-defined whether a cell returning `""` is blank. E.1 documents limitation: calcval harness cannot place `""` into a data cell (scratch AA1:AF12 cleared each case); range fixtures cover truly empty cells. | Impl-defined/UNVERIFIED | interpr1.cxx:ScCountEmptyCells | --- + + +## E.1 Corrections (this milestone) + +- **SG-BL-001**: corrected to formula expression length 1026 chars (`=1` + `+1`*512), expected 513 (replaces `REPT("a";1024)` which measured string length, not formula length). +- **SG-BL-004**: corrected to 7 genuinely nested function levels (`IF(TRUE;IF(...IF(TRUE;1;2)...))`, 7 nested IF calls), balanced parens, deterministic result 1. +- **SG-SYN-001**: added Criteria syntax §4.11.11 test `SYN-001-criteria` (`=DCOUNT(R1:T6;;V1:V2)`, expected 4) using existing calcval database/criteria fixtures. +- **SG-SYN-009**: marked not directly executable in calcval harness (requires pre-defined named expression `MYNAME`); test remains UNVERIFIED with explicit note. +- **SG-FN-013/COUNTA**: fixtures added for truly empty cells and empty string `""` per §6.13.7 (empty string non-blank). Harness-expressible via inline args; empty-cell cases use unset D6:D7. +- **SG-FN-014/COUNTBLANK**: fixtures added for truly empty cells; `""`-in-range-cell case documented as implementation-defined per §6.13.8 and not directly reproducible because calcval clears scratch AA1:AF12 and does not pre-populate `""` cells. +- **SG-FN-016/FN-016-DATE-pre1583**: reclassified as implementation-defined/diagnostic (out-of-constraint Year < 1904), not mandatory `#ERR`; AOO observed `#VALUE!`. +- **BITAND arity**: confirmed `BITAND.3params` (`=BITAND(6;3;1)` expected `#ERR`) remains valid per §6.6.2 (`BITAND(Integer X; Integer Y)`); no classification change. +- **Runtime verification**: not performed (no AOO build tree; calcval harness macOS-only). Only static/syntax/fixture checks applied. + ## 10. Runtime-Verification Limitations The following could not be verified at runtime because no configured AOO build tree exists in this environment: @@ -187,7 +201,7 @@ Static source tracing confirms all opcodes, implementations, dispatch, and metad |----------|-----|----------|-----------|--------| | 1 | COUNTA ODF 1.4 empty-cell semantics | Semantic difference | ODF 1.4 changed COUNTA/COUNTBLANK empty-cell handling; AOO uses ODF 1.3 semantics | Medium | | 2 | COUNTBLANK "" impl-defined | Semantic difference | ODF 1.4 marks "" case as impl-defined; AOO behavior unverified | Low | -| 3 | DATE pre-1583 proleptic Gregorian | Semantic difference | YEAR/MONTH/DAY of pre-1583 serial computed proleptic Gregorian with no validity check | Low | +| 3 | DATE out-of-constraint/pre-1583 behavior | Semantic difference (impl-defined) | `DATE(1500;1;1)` outside §6.10.2 constraints; result is impl-defined/diagnostic (AOO observed #VALUE!), not mandatory error. | Low | | 4 | INDEX row-vector compatibility | Semantic difference | Milestone A fixed ODF 1.4 behavior; legacy ODF 1.3 documents may break | High (regression risk) | **Recommended next milestone:** Small Group semantic conformance — verify and correct COUNTA/COUNTBLANK empty-cell semantics per ODF 1.4, verify DATE pre-1583 behavior, and assess INDEX row-vector regression risk. diff --git a/openformula-small-group-test-plan.md b/openformula-small-group-test-plan.md index de13966e96..a6cb5cf26a 100644 --- a/openformula-small-group-test-plan.md +++ b/openformula-small-group-test-plan.md @@ -10,7 +10,7 @@ milestones are complete. No production evaluator changes are made in this milest | File | Purpose | |------|---------| -| `test/odf-validate/openformula-small-group-1.4.tsv` | Requirement-referenced test inventory (148 ReqIDs from D inventory) | +| `test/odf-validate/openformula-small-group-1.4.tsv` | Requirement-referenced test inventory (148 ReqIDs; includes E.1 corrections and added `SG-SYN-001` criteria test) | | `test/odf-validate/calcval-cases14.tsv` | Existing calcval cases, extended with Small Group semantic tests | | `openformula-small-group-test-plan.md` | This document: execution instructions and outstanding runtime requirements | @@ -52,8 +52,8 @@ Each calcval case has 5 tab-separated columns: | Priority | Area | Test Cases | Rationale | |----------|------|------------|-----------| -| 1 | COUNTA/COUNTBLANK empty cells | `COUNTA.range`, `COUNTA.text`, `COUNTBLANK.range` | ODF 1.4 changed empty-cell semantics; can silently alter spreadsheet results | -| 2 | DATE pre-1583 | `DATE.pre1583` | ODF 1.4 §7.4 impl-defined; verify AOO behavior | +| 1 | COUNTA/COUNTBLANK empty cells / empty string | `COUNTA.empty`, `COUNTA.emptystr`, `COUNTBLANK.empty`, `COUNTBLANK.range` (calcval14) and corresponding SG tests | ODF 1.4 §6.13.7 counts `""` as non-blank; §6.13.8 `""`-in-cell is impl-defined and not directly reproducible with current calcval fixtures | +| 2 | DATE out-of-constraint/pre-1583 | `DATE.pre1583` / `FN-016-DATE-pre1583` | §6.10.2 constrains Year>=1904; `DATE(1500;1;1)` is impl-defined/diagnostic (not mandatory `#ERR`) | | 3 | INDEX row-vector | `INDEX.rowvec.single`, `INDEX.rowvec.colarg`, `INDEX.row.outrange` | Milestone A fix; regression risk for legacy documents | | 4 | Bitwise functions | `BITLSHIFT.neg`, `BITRSHIFT.neg`, `BITAND.3params` | Parameter count and negative shift behavior | | 5 | Parser/operators | `SYN-005-precedence`, `SYN-005-power`, `OP-006-div0` | Operator precedence and associativity | @@ -88,3 +88,15 @@ The following cannot be verified until a configured AOO build tree exists: - No mandatory `sc` or `scaddins` builds during this milestone - Tests use existing `test/odf-validate` tooling and `calcval-cases14.tsv` conventions - Tests are NOT added to the normal production build + + +## E.1 Corrections + +- `SG-BL-001` / `BL-001-1024chars`: formula expression length 1026 chars (`=1` + `+1`*512), expected 513. +- `SG-BL-004` / `BL-004-7nesting`: 7 genuinely nested `IF` function levels, balanced parens, expected 1. +- `SG-SYN-001` / `SYN-001-criteria`: added Criteria syntax §4.11.11 test using `DCOUNT(R1:T6;;V1:V2)` against existing database/criteria fixtures (expected 4). +- `SG-SYN-009` / `SYN-009-named`: requires pre-defined named expression `MYNAME`; not directly executable in calcval harness without name setup. +- `SG-FN-013` / `FN-016` tests updated: COUNTA covers empty cells and empty string `""`; COUNTBLANK covers truly empty cells; `""`-in-cell blankness is implementation-defined and not directly reproducible with calcval fixtures. +- `SG-FN-016` / `FN-016-DATE-pre1583`: reclassified as impl-defined/diagnostic (Year < 1904 outside §6.10.2 constraints). + +**Static validation only** (no AOO build tree; calcval macOS-only): TSV structure/columns, duplicate IDs, char/nesting counts, fixture references, and count reconciliation checked. Runtime execution not performed. diff --git a/openformula-test-gap-analysis.md b/openformula-test-gap-analysis.md index 934dc77367..c03889abf6 100644 --- a/openformula-test-gap-analysis.md +++ b/openformula-test-gap-analysis.md @@ -43,7 +43,7 @@ Source-of-truth: `test/aoo-test/`, `test/odf-validate/`, and AOO source. Finding - No confirmed semantic test for CONVERT `atm` (fixed in Milestone C: 101325.1 Pa vs old 101324.996583 Pa). - No confirmed semantic test for INDEX row-vector single-argument (fixed in Milestone A: ODF 1.4 error vs old column return). -- No confirmed semantic test for DATE pre-1583 (#VALUE! before 15 Oct 1582; proleptic Gregorian for YEAR/MONTH/DAY). +- DATE `DATE(1500;1;1)` is out of §6.10.2 constraints (Year>=1904) and impl-defined/diagnostic (AOO observed #VALUE!); E.1 reclassified tests to reflect this (not a mandatory #ERR). - No confirmed semantic test for BITAND/BITOR/BITXOR/BITLSHIFT/BITRSHIFT variadic params or negative-shift handling (interpr1.cxx:1231-1234 appears inverted). - No confirmed semantic test for short-circuit AND/OR. @@ -58,7 +58,7 @@ Source-of-truth: `test/aoo-test/`, `test/odf-validate/`, and AOO source. Finding ### Existing tests that produce false confidence - `openformula-functions.txt` lists IFNA/SKEWP as standard ODF 1.2 functions; AOO now implements them (Milestone B), so this is no longer a false-negative for conformance. -- Inventories are ODF 1.2, so they do not flag ODF 1.4 regressions (INDEX row-vector fixed in Milestone A, CONVERT atm fixed in Milestone C, COUNTA empty cells). +- Inventories are ODF 1.2, so they do not flag ODF 1.4 regressions (INDEX row-vector fixed in Milestone A, CONVERT atm fixed in Milestone C, COUNTA empty cells/empty string). ## 3. Proposed versioned test inventory @@ -88,3 +88,12 @@ Source-of-truth: `test/aoo-test/`, `test/odf-validate/`, and AOO source. Finding | Parser tests | Precedence, intersection, union, quoted labels | Medium | | Round-trip tests | Function-name export aliases, error value type | Medium | | False confidence | None — IFNA/SKEWP now implemented, matching inventory | None | + + +## E.1 Small Group corrections + +- Corrected `BL-001-1024chars` to 1026-char formula expression (`=1`+`+1`*512, result 513) and `BL-004-7nesting` to 7 genuinely nested IF levels. +- Added `SG-SYN-001` Criteria syntax (§4.11.11) coverage in the Small Group test inventory. +- Clarified `SG-SYN-009` named-expression test requires pre-defined name (`MYNAME`) not provided by calcval harness. +- Updated COUNTA/COUNTBLANK fixtures for empty cells and empty string semantics per §6.13.7/§6.13.8; documented `""`-in-cell COUNTBLANK limitation of calcval fixtures. +- Reclassified `FN-016-DATE-pre1583` / `DATE.pre1583` as implementation-defined/diagnostic for out-of-constraint input. diff --git a/test/odf-validate/calcval-cases14.tsv b/test/odf-validate/calcval-cases14.tsv index b58761e57c..41c2a8bb9f 100644 --- a/test/odf-validate/calcval-cases14.tsv +++ b/test/odf-validate/calcval-cases14.tsv @@ -143,9 +143,11 @@ SKEWP.two =SKEWP(1;2) cell 0 6.x two observations, non-constant, zero skewness SKEW.text =SKEW(1;"text";2) cell 0 6.x regression: direct scalar text skipped, {1,2} symmetric SKEW.err =SKEW(1/0) cell #ERR 6.x regression: error argument propagates COUNTA.range =COUNTA(D1:D3) cell 3 6.13.7 COUNTA over numeric range (D1:D3=10,20,30) -COUNTA.text =COUNTA(E1:E3) cell 3 6.13.7 COUNTA over text range (E1:E3=y,x,y) +COUNTA.empty =COUNTA(D6:D7) cell 0 6.13.7 COUNTA: truly empty cells are not counted (D6:D7 unset) +COUNTA.emptystr =COUNTA(1;"";"x") cell 3 6.13.7 COUNTA: empty string "" is non-blank and counted COUNTBLANK.range =COUNTBLANK(D1:D3) cell 0 6.13.8 COUNTBLANK over range with no empty cells -DATE.pre1583 =DATE(1500;1;1) cell #ERR 6.10.2 DATE before 1583: impl-defined #VALUE! +COUNTBLANK.empty =COUNTBLANK(D6:D7) cell 2 6.13.8 COUNTBLANK: truly empty cells are counted (D6:D7 unset) +DATE.pre1583 =DATE(1500;1;1) cell #ERR 6.10.2 DATE(1500;1;1) outside constraints (Year>=1904); impl-defined/diagnostic, not mandatory #ERR (AOO observed #VALUE!) INDEX.rowvec.single =INDEX(H1:L1;3) cell #ERR 6.14.6 INDEX row vector single arg: ODF 1.4 error (Milestone A fix) INDEX.rowvec.colarg =INDEX(H1:L1;;3) cell 30 6.14.6 INDEX row vector with column arg only (col 3 of H1:L1=10,20,30,40,50) INDEX.row.outrange =INDEX(D1:D5;6) cell #ERR 6.14.6 INDEX row out of range diff --git a/test/odf-validate/openformula-small-group-1.4.tsv b/test/odf-validate/openformula-small-group-1.4.tsv index 9b9c8f8d64..f0cd7ed5cf 100644 --- a/test/odf-validate/openformula-small-group-1.4.tsv +++ b/test/odf-validate/openformula-small-group-1.4.tsv @@ -10,12 +10,14 @@ # Note: Additional context # ===== Basic Limits (§3.7) ===== -SG-BL-001 3.7 BL-001-1024chars =REPT("a";1024) s:aaa...a (1024 a's) - UNVERIFIED Formula length: 1024 characters +SG-BL-001 3.7 BL-001-1024chars =1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+ [...] SG-BL-002 3.7 BL-002-30params =MAX(1;2;3;4;5;6;7;8;9;10;11;12;13;14;15;16;17;18;19;20;21;22;23;24;25;26;27;28;29;30) 30 - UNVERIFIED 30 parameters in MAX SG-BL-003 3.7 BL-003-32767chars =REPT("a";32767) s:aaa...a (32767 a's) - UNVERIFIED String of 32767 ASCII characters -SG-BL-004 3.7 BL-004-7nesting =IF(IF(IF(IF(IF(IF(TRUE;1;2);3;4);5;6);7;8);9;10) 10 - UNVERIFIED 7 nesting levels of IF +SG-BL-004 3.7 BL-004-7nesting =IF(TRUE;IF(TRUE;IF(TRUE;IF(TRUE;IF(TRUE;IF(TRUE;IF(TRUE;1;2);2);2);2);2);2);2) 1 - UNVERIFIED 7 genuinely nested function levels (7 nested IF calls), balanced parens # ===== Syntax (§5) ===== + +SG-SYN-001 4.11.11 SYN-001-criteria =DCOUNT(R1:T6;;V1:V2) 4 - UNVERIFIED Criteria syntax §4.11.11 via database function (Age>=30 matches B,C,D,E) SG-SYN-002 5.2 SYN-002-parens =(2+3)*4 20 - UNVERIFIED Parenthesized expression evaluation order SG-SYN-003 5.3 SYN-003-decimal =1.5 1.5 - UNVERIFIED Decimal number literal SG-SYN-003 5.3 SYN-003-sci =1.5e2 150 - UNVERIFIED Scientific notation literal @@ -25,9 +27,9 @@ SG-SYN-005 5.5 SYN-005-precedence =2+3*4 14 - UNVERIFIED Operator precedence: * SG-SYN-005 5.5 SYN-005-power =2^3^2 512 - UNVERIFIED Power operator right-associativity SG-SYN-006 5.6 SYN-006-func =ABS(-5) 5 - UNVERIFIED Function call with argument SG-SYN-007 5.7 SYN-007-nonstandard =NONEXISTENT() #ERR - UNVERIFIED Nonstandard function name error -SG-SYN-008 5.8 SYN-008-ref =A1 10 - UNVERIFIED Cell reference (A1=10 per data setup) +SG-SYN-008 5.8 SYN-008-ref =D1 10 - UNVERIFIED Cell reference resolves value (D1=10 per calcval fixture) SG-SYN-008 5.8 SYN-008-range =SUM(D1:D3) 60 - UNVERIFIED Range reference (D1:D3 = 10,20,30) -SG-SYN-009 5.11 SYN-009-named =MYNAME - - UNVERIFIED Simple named expression (name must be defined) +SG-SYN-009 5.11 SYN-009-named =MYNAME - - UNVERIFIED Simple named expression requires a pre-defined name (MYNAME) not set up in calcval harness; not directly executable as-is SG-SYN-010 5.12 SYN-010-error =NA() #N/A - UNVERIFIED Error constant #N/A SG-SYN-011 5.14 SYN-011-whitespace =2 + 3 * 4 14 - UNVERIFIED Whitespace does not affect evaluation @@ -48,8 +50,8 @@ SG-OP-009 6.4.8 OP-009-ne =2<>3 TRUE - UNVERIFIED Not equal SG-OP-010 6.4.14 OP-010-pct =50% 0.5 - UNVERIFIED Postfix percent SG-OP-011 6.4.15 OP-011-pos =+5 5 - UNVERIFIED Prefix plus SG-OP-012 6.4.16 OP-012-neg =-5 -5 - UNVERIFIED Prefix minus -SG-OP-013 6.4.12 OP-013-intersect =A1:C3!B1:D2 - - UNVERIFIED Reference intersection -SG-OP-014 6.4.11 OP-014-range =SUM(A1:A3) - - UNVERIFIED Reference range +SG-OP-013 6.4.12 OP-013-intersect =SUM(D1:F3!E1:F2) 336 - UNVERIFIED Reference intersection D1:F3!E1:F2 yields E1:F2 (11,101,22,202), SUM 336 +SG-OP-014 6.4.11 OP-014-range =SUM(D1:D3) 60 - UNVERIFIED Reference range (D1:D3=10,20,30) # ===== Implicit Conversions (§6.3) ===== SG-CONV-001 6.3.14 CONV-001-text =TEXT(123;"0") s:123 - UNVERIFIED Number to text conversion @@ -60,20 +62,22 @@ SG-CONV-004 6.3.12 CONV-004-logical2 =IF(0;TRUE;FALSE) FALSE - UNVERIFIED Number SG-CONV-005 5.12 CONV-005-error =ISERR(1/0) TRUE - UNVERIFIED Error propagation: division by zero # ===== Functions (§6.x) — Priority 1: COUNTA/COUNTBLANK empty cells ===== -SG-FN-013 6.13.7 FN-013-COUNTA-empty =COUNTA(D1:D3) 3 - UNVERIFIED COUNTA over range with numbers (D1:D3=10,20,30) -SG-FN-013 6.13.7 FN-013-COUNTA-text =COUNTA(E1:E3) 3 - UNVERIFIED COUNTA over range with text (E1:E3="y","x","y") -SG-FN-014 6.13.8 FN-014-COUNTBLANK-empty =COUNTBLANK(D1:D3) 0 - UNVERIFIED COUNTBLANK over range with no empty cells -SG-FN-014 6.13.8 FN-014-COUNTBLANK-empty-str =COUNTBLANK(E1:E3) 0 - UNVERIFIED COUNTBLANK over range with text (not empty) +SG-FN-013 6.13.7 FN-013-COUNTA-range =COUNTA(D1:D3) 3 - UNVERIFIED COUNTA over numeric range (D1:D3=10,20,30) +SG-FN-013 6.13.7 FN-013-COUNTA-empty-cells =COUNTA(D6:D7) 0 - UNVERIFIED COUNTA over truly empty cells (D6:D7 unset) +SG-FN-013 6.13.7 FN-013-COUNTA-empty-string =COUNTA(1;"";"x") 3 - UNVERIFIED COUNTA counts empty string "" as non-blank per §6.13.7 +SG-FN-014 6.13.8 FN-014-COUNTBLANK-nonempty =COUNTBLANK(D1:D3) 0 - UNVERIFIED COUNTBLANK over range with no empty cells +SG-FN-014 6.13.8 FN-014-COUNTBLANK-empty-cells =COUNTBLANK(D6:D7) 2 - UNVERIFIED COUNTBLANK over truly empty cells (D6:D7 unset) +SG-FN-014 6.13.8 FN-014-COUNTBLANK-empty-string =COUNTBLANK(D6:D7) 2 - UNVERIFIED COUNTBLANK does not count "" text values as blank in ranges; harness cannot place "" in data cells (scratch cleared), so this verifies only empty-cell counting; ""-in-cell case impl-defined per §6.13.8 # ===== Functions (§6.x) — Priority 2: DATE pre-1583 ===== -SG-FN-016 6.10.2 FN-016-DATE-pre1583 =DATE(1500;1;1) #ERR - UNVERIFIED DATE before 1583: impl-defined, AOO returns #VALUE! +SG-FN-016 6.10.2 FN-016-DATE-pre1583 =DATE(1500;1;1) #ERR - UNVERIFIED DATE(1500;1;1) outside §6.10.2 constraints (Year>=1904); impl-defined/evaluator-diagnostic, not mandatory #ERR (AOO observed #VALUE!) SG-FN-067 6.13.27 FN-067-NA =NA() #N/A - UNVERIFIED NA function returns #N/A error # ===== Functions (§6.x) — Priority 3: INDEX ===== -SG-FN-042 6.14.6 FN-042-INDEX-2d =INDEX(D1:F3;2;3) - - UNVERIFIED INDEX two-dimensional: row 2, col 3 of D1:F3 +SG-FN-042 6.14.6 FN-042-INDEX-2d =INDEX(D1:F3;2;3) 202 - UNVERIFIED INDEX two-dimensional: row 2, col 3 of D1:F3 (row2 is 20,22,202) SG-FN-042 6.14.6 FN-042-INDEX-row-outrange =INDEX(D1:D5;6) #ERR - UNVERIFIED INDEX row out of range SG-FN-042 6.14.6 FN-042-INDEX-rowvec-single =INDEX(H1:L1;3) #ERR - UNVERIFIED INDEX row vector single arg: ODF 1.4 error (Milestone A fix) -SG-FN-042 6.14.6 FN-042-INDEX-rowvec-colarg =INDEX(H1:L1;;3) - - UNVERIFIED INDEX row vector with column arg only +SG-FN-042 6.14.6 FN-042-INDEX-rowvec-colarg =INDEX(H1:L1;;3) 30 - UNVERIFIED INDEX row vector with column arg only (H1:L1=10,20,30,40,50) # ===== Operators — reference intersection ===== -SG-OP-013 6.4.12 OP-013-intersect-num =SUM(A1:C3!B1:D2) - - UNVERIFIED Reference intersection in SUM +SG-OP-013 6.4.12 OP-013-intersect-num =SUM(D1:F3!E1:F2) 336 - UNVERIFIED Reference intersection D1:F3!E1:F2 = E1:F2 (11,101,22,202), SUM 336
