kz930 opened a new issue, #7643: URL: https://github.com/apache/texera/issues/7643
### What happened? Both cases end the execution with an error from inside scikit-learn or from code generation, naming neither the column nor the field the user has to change. **A text column the user did not mean as a feature.** The operators take every column except the target as features, fixed at `X = table.drop(target, axis=1)` (`SklearnClassifierOpDesc.scala:43`, `SklearnTrainingOpDesc.scala:43`, `SklearnTestingOpDesc.scala:71`). A dataset that carries a text column beside its numeric ones ends the run with `ValueError: could not convert string to float`. A timestamp column fails the same way with `DTypePromotionError`. Boolean and integer columns are fine. Nothing in the configuration narrows the feature set, so the only way through is a Projection upstream, which the error does not suggest. Turning on `Count Vectorizer` is not a workaround: it replaces the feature set with that one text column and discards the numeric ones. **Count Vectorizer on with no Text Attribute.** `text` is not required, so the switch can be turned on with the field left empty. It then reaches code generation as a null and the operator's code becomes `#EXCEPTION DURING CODE GENERATION`, per `PythonOperatorDescriptor.scala:36-44`. `text` also accepts a column of any type, though `CountVectorizer` tokenizes documents and a numeric column raises from inside it. Expected: a configuration that cannot work is refused while it is being written, and a column an estimator cannot fit is dropped and named rather than ending the run. `text` is a string column, and is required exactly when `Count Vectorizer` is on. Both are statable in the descriptor's schema, the second as the conditional `required` the Aggregate operator already uses. For the feature set, keeping the columns an estimator can fit and printing the ones left out follows what the rest of the codebase does with data it cannot use, where twenty-four visualization operators drop missing values before plotting. ### How to reproduce? Upload a CSV whose numeric columns sit beside a text column: ``` x1,x2,note,y 0.0,0.0,good great,0 0.1,0.2,great nice,0 1.0,1.0,bad awful,1 0.9,0.8,awful poor,1 ``` Build `CSV File Scan` to `Decision Tree` from the Sklearn group, wire it to both the `training` and `testing` ports, set Target Attribute to `y`, leave Count Vectorizer off, and run. The execution stops on the ValueError. For the second case, turn Count Vectorizer on and leave Text Attribute empty. ### Version/Branch 1.3.0-incubating-SNAPSHOT (main) ### Relevant log output ```shell ValueError: could not convert string to float: 'good great' ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
