sshiv012 opened a new issue, #8638:
URL: https://github.com/apache/texera/issues/8638
### Feature Summary
Delimiter properties are plain text boxes. Users can't tell whether to press
Tab or type `\t`, and invalid values are accepted without any message. The two
kinds of operator have different flaws.
| Operator | What the value really is | What the UI shows |
|---|---|---|
| CSV File Scan, Parallel CSV File Scan, CSV (old) | **One character**;
readers use `customDelimiter.charAt(0)` | An empty-looking one-character box |
| Unnest String | **A Java regex**; split with `delimiter.r.split(...)` | A
box described as "string that separates the data" |
**CSV File Scan**
| | |
|---|---|
| <!-- drag before/B01-csv-stored-tab-looks-empty.png here --> _screenshot
B01_ | **A saved tab looks empty.** The tab character is stored, but it's
invisible, so the field looks unset. |
| <!-- drag before/B02-csv-typing-backslash-t-stops-at-backslash.png here
--> _screenshot B02_ | **Typing `\t` doesn't work.** The one-character limit
stops typing at `\`, so a backslash is saved and the file is split on the wrong
character. |
| <!-- drag before/B03-csv-legacy-multichar-no-error-until-touched.png here
--> _screenshot B03_ | **An old invalid value shows no error.** A value such as
`,;` saved before the limit existed shows no message, and only its first
character is used. |
**Unnest String**
| | |
|---|---|
| <!-- drag before/B05-regex-bare-pipe-accepted-silently.png here -->
_screenshot B05_ | **`\|` is accepted silently**, although as a regex it is an
empty alternation and splits between every character (`abc` → `a`, `b`, `c`).
The user almost certainly meant `\\|`. |
| <!-- drag before/B06-regex-unescaped-dot-accepted-silently.png here -->
_screenshot B06_ | **`.` is accepted silently**, although it matches every
character, so every piece is empty and the operator outputs no rows. |
| <!-- drag before/B07-regex-unclosed-group-no-inline-error.png here -->
_screenshot B07_ | **An invalid pattern, e.g. `(`, shows no error** and still
passes workflow compilation. It fails only at run time, with a
`PatternSyntaxException`. |
### Proposed Solution or Design
Replace the text box with a **delimiter picker**: a dropdown of common
delimiters by name, plus a **Custom…** option that opens a text box. Operators
choose one of two modes through a schema hint (`@JsonSchemaInject`).
```
CSV scans (char mode) Comma Tab Semicolon Pipe Space | Custom… (1
character, \t typed -> tab)
Unnest String (regex mode) Comma Tab Semicolon Pipe (\|) Whitespace
(\s+) New line | Custom regex…
```
- **The stored value doesn't change.** It's still the delimiter itself, so
existing workflows load unchanged, and a stored tab shows as "Tab".
- **Regex mode checks the pattern as you type:**
- it must compile;
- it must not match the empty string (`|`, `\s*`);
- it must not match every character (`.`, which gets "use `\.`" as the
hint).
- Java-only syntax (possessive quantifiers, inline flags) is left to the
backend.
- **Backend:** Unnest String compiles the pattern while propagating its
schema, so an invalid regex is reported as a compile error on the operator
instead of at run time.
- **Errors show immediately**, including for invalid values loaded from a
saved workflow.
### Affected Area
Workflow UI, Workflow Engine (Amber)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
