pvillard31 opened a new pull request, #10891:
URL: https://github.com/apache/nifi/pull/10891
# Summary
NIFI-15588 - Improve date, time, and timestamp type compatibility with ISO
8601 default formatters
## Summary
- Add default ISO 8601 formatter fallback to temporal compatibility checks
(`isDateTypeCompatible`, `isTimeTypeCompatible`, `isTimestampTypeCompatible`)
and to the field converters (`ObjectLocalDateFieldConverter`,
`ObjectLocalTimeFieldConverter`, `ObjectLocalDateTimeFieldConverter`), allowing
ISO-formatted strings to be recognized and converted without requiring an
explicit format pattern.
- Introduce a three-argument overload for each compatibility method with an
`allowDefaultFormats` parameter, enabling callers to opt out of the new ISO
fallback and retain the legacy epoch-only behavior.
- Apply `ResolverStyle.STRICT` to both the default ISO formatters and the
explicit format path in compatibility checks, improving date validation
strictness.
## Motivation
Prior to this change, the temporal compatibility methods
(`isDateTypeCompatible`, `isTimeTypeCompatible`, `isTimestampTypeCompatible`)
only accepted numeric (epoch) strings when no explicit format pattern was
provided. ISO 8601 strings such as `"2024-01-15"` or `"13:45:00"` were
rejected, even though they are the most common standard date/time
representations. Similarly, the field converters would throw a
`FieldConversionException` for ISO strings when no pattern was supplied,
falling through directly to the epoch-millis numeric parse.
Additionally, `isTimeTypeCompatible` and `isTimestampTypeCompatible` simply
delegated to `isDateTypeCompatible`, meaning all three types shared the same
set of accepted formats. This made it impossible to use type-specific default
formatters.
## Important notes
### `AbstractCSVRecordReader`
Updated `convertSimpleIfPossible` to use the three-argument overloads with
`allowDefaultFormats` set to `true` only when the corresponding format
(`dateFormat`, `timeFormat`, `timestampFormat`) is non-null and non-blank. This
preserves existing CSV reader behavior: without an explicit format configured,
only epoch-like numeric strings are coerced.
### `ExcelRecordReader`
Applied the same guarding pattern as `AbstractCSVRecordReader` to
`convertSimpleIfPossible`, preventing the new ISO fallback from changing
behavior for the Excel reader when no explicit temporal format is configured.
## Behavior Changes
| Scenario | Before | After |
|---|---|---|
| `isDateTypeCompatible("2024-01-15", null)` | `false` (epoch-only) | `true`
(ISO fallback) |
| `isTimeTypeCompatible("13:45:00", null)` | `false` (delegated to date
check) | `true` (ISO fallback) |
| `isTimestampTypeCompatible("2024-01-15T13:45:00Z", null)` | `false`
(delegated to date check) | `true` (ISO fallback) |
| `isDateTypeCompatible("2024-02-30", null)` | `false` | `false` (ISO STRICT
rejects) |
| `isDateTypeCompatible("2024-01-15", null, false)` | N/A (new overload) |
`false` (epoch-only) |
| CSV/Excel reader with no explicit format | epoch-only coercion |
epoch-only coercion (unchanged) |
| Converter with no pattern, ISO string input | `FieldConversionException` |
Parsed via ISO formatters |
| Blank format string `" "` passed to compatibility check |
`IllegalArgumentException` | Treated as absent format |
**Callers unaffected by the two-argument overload change:**
- `isCompatibleDataType(value, dataType)` — always passes non-null default
formats (`"yyyy-MM-dd"`, `"HH:mm:ss"`, `"yyyy-MM-dd HH:mm:ss"`), so the
null-format ISO fallback path is never reached.
- `CEFRecordReader` — always passes non-null format constants.
- `AbstractCSVRecordReader` and `ExcelRecordReader` — explicitly use the
three-argument overload to opt out.
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue
created
### Pull Request Tracking
- [ ] Pull Request title starts with Apache NiFi Jira issue number, such as
`NIFI-00000`
- [ ] Pull Request commit message starts with Apache NiFi Jira issue number,
as such `NIFI-00000`
- [ ] Pull request contains [commits
signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)
with a registered key indicating `Verified` status
### Pull Request Formatting
- [ ] Pull Request based on current revision of the `main` branch
- [ ] Pull Request refers to a feature branch with one commit containing
changes
# Verification
Please indicate the verification steps performed prior to pull request
creation.
### Build
- [ ] Build completed using `./mvnw clean install -P contrib-check`
- [ ] JDK 21
- [ ] JDK 25
### Licensing
- [ ] New dependencies are compatible with the [Apache License
2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License
Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE`
files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
