This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-7258-5e41efae2650c3e70d7490b15bb1fdf9b1983568 in repository https://gitbox.apache.org/repos/asf/texera.git
commit 389db604ae6dcccc498eb530afaaad98f9228f77 Author: Kary Zheng <[email protected]> AuthorDate: Fri Aug 7 17:20:52 2026 -0700 feat(visualization): declare the column types the charts require (#7258) ### What changes were proposed in this PR? Twelve column pickers across ten operators consume their column as a quantity or an instant but declared no `attributeTypeRules`. Each now declares what it requires, the way Bar Chart, Pie Chart, Hierarchy Chart, ECDF Plot and Quiver Plot already do, and the way Gantt Chart declares `timestamp` for its two ends. | Operator | Columns | Declared | | --- | --- | --- | | Contour Plot | `x`, `y`, `z` | numeric | | Dendrogram | `xVal`, `yVal` | numeric | | Ternary Contour | the three variables and the measured value | numeric | | Volcano Plot | `effectColumn`, `pvalueColumn` | numeric | | Parallel Coordinates | `color` | numeric or boolean | | Waterfall Chart | `yColumn` | numeric | | Bullet Chart | `value` | numeric | | Carpet Plot | `a`, `b`, `y` | numeric | | Polar Chart | `r`, `theta` | numeric | | Ternary Plot | the three variables | numeric | | Time Series | `timeColumn` / `valueColumn` | timestamp or string / numeric | Time Series' `timeColumn` admits a string as well as a timestamp: `pd.to_datetime` parses date text and a CSV source hands its dates over as STRING, so a timestamp-only rule would flag workflows that plot correctly today. A type rule cannot tell date text from an unparseable value, so that half stays with the coercion, which drops what it cannot read. Waterfall's `xColumn` and Time Series' category and facet columns are deliberately left alone: those are labels and grouping keys, and a column of years or ids is a legitimate choice for them. ### Why are the changes needed? Generating each operator's module on `main` and executing it against a six-row frame whose selected column is a STRING attribute: Contour, Dendrogram, Ternary Contour, Volcano and Parallel Coordinates fail the run; Waterfall fails on `f"{v:+}"`; Bullet, Carpet and Polar render an error page; and Ternary Plot and Time Series' `valueColumn` **render a finished-looking chart that means nothing**. Polar Chart is the clearest of them: it already checks `np.issubdtype(..., np.number)` and renders a message, so the operator had decided the column must be numeric and only the form did not know. ### Any related issues, documentation, discussions? Closes #7250, which is filed as a bug: every one of these pickers offers columns its own operator rejects at run time or silently misdraws. Related: #7210, on rules that silently no-op when a rule key does not name a real property. ### How was this PR tested? `WorkflowOperator/compile`, `scalafmtCheckAll` and `scalafixAll --check`, and the eleven operators' descriptor specs. `TimeSeriesOpDescSpec` gains three cases (16 tests total): the two rules read back out of the generated schema — the one the property editor validates against — and one pinning that unparseable text is coerced, dropped and reported rather than raised. Each of the other rules was likewise read back out of the generated schema and its keys checked against the declared property names; all match, so none can silently no-op the way #7210 describes. ### Does this PR introduce any user-facing change? Yes — selecting a column of the wrong type for these fields now shows a type warning in the form instead of failing, or silently misdrawing, at run time. Nothing changes for a column of the right type. Two of the twelve are a judgement rather than an observed failure: Ternary Plot's variables and Time Series' `valueColumn` render without error, so constraining them says a text column is not a meaningful ternary proportion or measurement. If you would rather leave those two unconstrained, they can be dropped without touching the rest. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> --- .../bulletChart/BulletChartOpDesc.scala | 11 ++++++- .../carpetPlot/CarpetPlotOpDesc.scala | 13 +++++++- .../contourPlot/ContourPlotOpDesc.scala | 13 +++++++- .../dendrogram/DendrogramOpDesc.scala | 12 +++++++- .../ParallelCoordinatesPlotOpDesc.scala | 7 ++++- .../polarChart/PolarChartOpDesc.scala | 13 +++++++- .../ternaryContour/TernaryContourOpDesc.scala | 14 ++++++++- .../ternaryPlot/TernaryPlotOpDesc.scala | 14 ++++++++- .../timeSeriesplot/TimeSeriesOpDesc.scala | 14 +++++++++ .../volcanoPlot/VolcanoPlotOpDesc.scala | 12 +++++++- .../waterfallChart/WaterfallChartOpDesc.scala | 11 ++++++- .../timeSeriesPlot/TimeSeriesOpDescSpec.scala | 35 ++++++++++++++++++++++ 12 files changed, 159 insertions(+), 10 deletions(-) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/bulletChart/BulletChartOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/bulletChart/BulletChartOpDesc.scala index 22bee1bf50..19d02af3e5 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/bulletChart/BulletChartOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/bulletChart/BulletChartOpDesc.scala @@ -21,7 +21,7 @@ package org.apache.texera.amber.operator.visualization.bulletChart import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} import com.fasterxml.jackson.databind.annotation.JsonDeserialize -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext import org.apache.texera.amber.pybuilder.PyStringTypes.{EncodableString, PythonLiteral} @@ -39,6 +39,15 @@ import scala.jdk.CollectionConverters._ * Visualization Operator to visualize results as a Bullet Chart */ +// type constraint: the measured value is read as float(row[value]), so it can only +// be a numeric column. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "value": { "enum": ["integer", "long", "double"] } + } +} +""") class BulletChartOpDesc extends PythonOperatorDescriptor { @JsonProperty(value = "value", required = true) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/carpetPlot/CarpetPlotOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/carpetPlot/CarpetPlotOpDesc.scala index 1429f2242c..6c7981390d 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/carpetPlot/CarpetPlotOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/carpetPlot/CarpetPlotOpDesc.scala @@ -20,7 +20,7 @@ package org.apache.texera.amber.operator.visualization.carpetPlot import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext import org.apache.texera.amber.pybuilder.PyStringTypes.EncodableString @@ -30,6 +30,17 @@ import org.apache.texera.amber.operator.metadata.annotations.AutofillAttributeNa import org.apache.texera.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo} import javax.validation.constraints.NotNull +// type constraint: a / b / y are cast with astype(float) to build the carpet grid, +// so they can only be numeric columns. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "a": { "enum": ["integer", "long", "double"] }, + "b": { "enum": ["integer", "long", "double"] }, + "y": { "enum": ["integer", "long", "double"] } + } +} +""") class CarpetPlotOpDesc extends PythonOperatorDescriptor { @JsonProperty(value = "a", required = true) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/contourPlot/ContourPlotOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/contourPlot/ContourPlotOpDesc.scala index d257409b3e..3ff542e5d0 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/contourPlot/ContourPlotOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/contourPlot/ContourPlotOpDesc.scala @@ -20,7 +20,7 @@ package org.apache.texera.amber.operator.visualization.contourPlot import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext import org.apache.texera.amber.pybuilder.PyStringTypes.EncodableString @@ -31,6 +31,17 @@ import org.apache.texera.amber.operator.metadata.{OperatorGroupConstants, Operat import javax.validation.constraints.NotNull +// type constraint: x / y / z are plotted on numeric axes and interpolated via +// scipy.griddata, so they can only be numeric columns. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "x": { "enum": ["integer", "long", "double"] }, + "y": { "enum": ["integer", "long", "double"] }, + "z": { "enum": ["integer", "long", "double"] } + } +} +""") class ContourPlotOpDesc extends PythonOperatorDescriptor { @JsonProperty(value = "x", required = true) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/dendrogram/DendrogramOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/dendrogram/DendrogramOpDesc.scala index f60eab6b3e..f80d7acb5b 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/dendrogram/DendrogramOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/dendrogram/DendrogramOpDesc.scala @@ -21,7 +21,7 @@ package org.apache.texera.amber.operator.visualization.dendrogram import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} import com.fasterxml.jackson.databind.annotation.JsonDeserialize -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext import org.apache.texera.amber.pybuilder.PyStringTypes.{EncodableString, PythonLiteral} @@ -33,6 +33,16 @@ import org.apache.texera.amber.pybuilder.PythonTemplateBuilder import javax.validation.constraints.NotNull +// type constraint: xVal / yVal are stacked into a numeric point matrix for +// hierarchical clustering, so they can only be numeric columns. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "xVal": { "enum": ["integer", "long", "double"] }, + "yVal": { "enum": ["integer", "long", "double"] } + } +} +""") class DendrogramOpDesc extends PythonOperatorDescriptor { @JsonProperty(value = "xVal", required = true) @JsonSchemaTitle("Value X Column") diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/parallelCoordinatesPlot/ParallelCoordinatesPlotOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/parallelCoordinatesPlot/ParallelCoordinatesPlotOpDesc.scala index ef7730e7cc..4af91c332d 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/parallelCoordinatesPlot/ParallelCoordinatesPlotOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/parallelCoordinatesPlot/ParallelCoordinatesPlotOpDesc.scala @@ -35,12 +35,17 @@ import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBui import javax.validation.constraints.{NotNull, Size} -// type constraint: value can only be numeric +// type constraint: value can only be numeric. The colour column too -- plotly maps it +// onto a continuous colorscale, so a string or timestamp raises instead of rendering, +// while a boolean renders as its 0/1 equivalent. @JsonSchemaInject(json = """ { "attributeTypeRules": { "dimensions": { "enum": ["integer", "long", "double"] + }, + "color": { + "enum": ["integer", "long", "double", "boolean"] } } } diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/polarChart/PolarChartOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/polarChart/PolarChartOpDesc.scala index 838304d6fb..707350497a 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/polarChart/PolarChartOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/polarChart/PolarChartOpDesc.scala @@ -20,7 +20,7 @@ package org.apache.texera.amber.operator.visualization.polarChart import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.core.workflow.OutputPort.OutputMode import org.apache.texera.amber.core.workflow.{InputPort, OutputPort, PortIdentity} @@ -32,6 +32,17 @@ import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBui import javax.validation.constraints.NotNull +// type constraint: r / theta are the polar coordinates, and the operator already +// refuses a non-numeric column at run time with np.issubdtype -- this states the same +// requirement where the form can act on it. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "r": { "enum": ["integer", "long", "double"] }, + "theta": { "enum": ["integer", "long", "double"] } + } +} +""") class PolarChartOpDesc extends PythonOperatorDescriptor { @JsonProperty(value = "r", required = true) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryContour/TernaryContourOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryContour/TernaryContourOpDesc.scala index 8ecbd945d8..93771d4e1b 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryContour/TernaryContourOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryContour/TernaryContourOpDesc.scala @@ -20,7 +20,7 @@ package org.apache.texera.amber.operator.visualization.ternaryContour import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.core.workflow.OutputPort.OutputMode import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext @@ -40,6 +40,18 @@ import javax.validation.constraints.NotNull * The points can optionally be color coded using a data field. */ +// type constraint: the three variables are compared against 0 and summed as ternary +// proportions, and the measured value is interpolated, so all four can only be numeric. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "firstVariable": { "enum": ["integer", "long", "double"] }, + "secondVariable": { "enum": ["integer", "long", "double"] }, + "thirdVariable": { "enum": ["integer", "long", "double"] }, + "fourthVariable": { "enum": ["integer", "long", "double"] } + } +} +""") class TernaryContourOpDesc extends PythonOperatorDescriptor { // Add annotations for the first variable diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryPlot/TernaryPlotOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryPlot/TernaryPlotOpDesc.scala index 24c1376dde..14e7c7fad1 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryPlot/TernaryPlotOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/ternaryPlot/TernaryPlotOpDesc.scala @@ -20,7 +20,7 @@ package org.apache.texera.amber.operator.visualization.ternaryPlot import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext import org.apache.texera.amber.pybuilder.PyStringTypes.EncodableString @@ -39,6 +39,18 @@ import javax.validation.constraints.NotNull * The points can optionally be color coded using a data field. */ +// type constraint: a ternary axis is a proportion, so the three variables can only be +// numeric. Unlike the operators that raise, plotly renders a text column here -- the +// chart is drawn from labels and means nothing, with nothing to tell the user so. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "firstVariable": { "enum": ["integer", "long", "double"] }, + "secondVariable": { "enum": ["integer", "long", "double"] }, + "thirdVariable": { "enum": ["integer", "long", "double"] } + } +} +""") class TernaryPlotOpDesc extends PythonOperatorDescriptor { // Add annotations for the first variable diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/timeSeriesplot/TimeSeriesOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/timeSeriesplot/TimeSeriesOpDesc.scala index 7d09972717..6e5e15f632 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/timeSeriesplot/TimeSeriesOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/timeSeriesplot/TimeSeriesOpDesc.scala @@ -30,6 +30,20 @@ import org.apache.texera.amber.operator.metadata.{OperatorGroupConstants, Operat import javax.validation.constraints.{NotBlank, NotNull} +// type constraint: the time axis is an instant and the plotted value is a measurement. +// Neither is enforced today: a text value column silently degenerates the y axis into +// a categorical one. The time axis also accepts a string, because pd.to_datetime +// parses date text and a CSV source hands its dates over as one; a type rule cannot +// tell such text from an unparseable value, so that half is left to the coercion, +// which drops what it cannot read. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "timeColumn": { "enum": ["timestamp", "string"] }, + "valueColumn": { "enum": ["integer", "long", "double"] } + } +} +""") class TimeSeriesOpDesc extends PythonOperatorDescriptor { @JsonProperty(value = "timeColumn", required = true) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/volcanoPlot/VolcanoPlotOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/volcanoPlot/VolcanoPlotOpDesc.scala index 7d8b80fa96..6fcc7085be 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/volcanoPlot/VolcanoPlotOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/volcanoPlot/VolcanoPlotOpDesc.scala @@ -20,7 +20,7 @@ package org.apache.texera.amber.operator.visualization.volcanoPlot import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext import org.apache.texera.amber.pybuilder.PyStringTypes.EncodableString @@ -31,6 +31,16 @@ import org.apache.texera.amber.operator.metadata.{OperatorGroupConstants, Operat import javax.validation.constraints.NotNull +// type constraint: the p-value is filtered with `> 0` and passed to np.log10, and the +// effect column is the quantitative x axis, so both can only be numeric. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "effectColumn": { "enum": ["integer", "long", "double"] }, + "pvalueColumn": { "enum": ["integer", "long", "double"] } + } +} +""") class VolcanoPlotOpDesc extends PythonOperatorDescriptor { @JsonProperty(required = true) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/waterfallChart/WaterfallChartOpDesc.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/waterfallChart/WaterfallChartOpDesc.scala index e5e1de11fd..87965a61d6 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/waterfallChart/WaterfallChartOpDesc.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/waterfallChart/WaterfallChartOpDesc.scala @@ -20,7 +20,7 @@ package org.apache.texera.amber.operator.visualization.waterfallChart import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription} -import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import com.kjetland.jackson.jsonSchema.annotations.{JsonSchemaInject, JsonSchemaTitle} import org.apache.texera.amber.core.tuple.{AttributeType, Schema} import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext import org.apache.texera.amber.pybuilder.PyStringTypes.EncodableString @@ -32,6 +32,15 @@ import org.apache.texera.amber.pybuilder.PythonTemplateBuilder import javax.validation.constraints.NotNull +// type constraint: each bar is a signed amount, formatted with f"{v:+}", so the y +// column can only be numeric. The x column is the category axis and stays unconstrained. +@JsonSchemaInject(json = """ +{ + "attributeTypeRules": { + "yColumn": { "enum": ["integer", "long", "double"] } + } +} +""") class WaterfallChartOpDesc extends PythonOperatorDescriptor { @JsonProperty(value = "xColumn", required = true) diff --git a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/timeSeriesPlot/TimeSeriesOpDescSpec.scala b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/timeSeriesPlot/TimeSeriesOpDescSpec.scala index 7a02ce2275..a6abb11bae 100644 --- a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/timeSeriesPlot/TimeSeriesOpDescSpec.scala +++ b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/timeSeriesPlot/TimeSeriesOpDescSpec.scala @@ -217,6 +217,41 @@ class TimeSeriesOpDescSpec extends AnyFunSuite { assert(py.contains(b64("date")) && py.contains(b64("value"))) } + // --- column type rules -------------------------------------------------------- + + /** The rule the property editor reads to decide which columns a picker offers. */ + private def attributeTypeRule(property: String): Set[String] = + OperatorMetadataGenerator + .generateOperatorJsonSchema(classOf[TimeSeriesOpDesc]) + .path("attributeTypeRules") + .path(property) + .path("enum") + .elements() + .asScala + .map(_.asText()) + .toSet + + test("the time column accepts a date column stored as text, not only a timestamp") { + // A CSV source hands its dates over as STRING, and pd.to_datetime parses them, + // so rejecting the type would invalidate workflows that plot today. + assert(attributeTypeRule("timeColumn") == Set("timestamp", "string")) + } + + test("the value column is restricted to the numeric types") { + assert(attributeTypeRule("valueColumn") == Set("integer", "long", "double")) + } + + test("text the parser cannot read is dropped and reported rather than raised") { + // The type rule admits any string, so unparseable text still reaches the + // generated code; coercion turns it into NaT, dropna removes the row, and an + // input whose every row goes that way ends in a message instead of a traceback. + val py = minimalOp().generatePythonCode() + + assert(py.contains("errors='coerce'")) + assert(dropnaSubset(py).contains(b64("date"))) + assert(py.contains("Table became empty after filtering.")) + } + // --- JSON round-trip --------------------------------------------------------- test("the descriptor round-trips all of its config fields through the polymorphic base") {
