kz930 opened a new issue, #8010: URL: https://github.com/apache/texera/issues/8010
### What happened? Two families of sklearn operators let the column already chosen as the label be chosen a second time as an input column, and neither says anything while the workflow is being written. *Text Attribute, on the fifty-one operators of the Sklearn and Sklearn Training groups.* `Target Attribute` names the label and `Text Attribute` names the columns `Count Vectorizer` tokenizes. Both dropdowns offer every column of the input, so one column can be picked for both. The generated code drops the target before the pipeline reads its columns (`SklearnClassifierOpDesc.scala:44`, `X = table.drop(target, axis=1)`), so the `ColumnTransformer` asks for a column that is no longer there and the run ends with `ValueError: A given column is not a column of the dataframe`, naming neither of the two fields the user has to change. Failing is right here, since vectorizing the label would train a model on the answer. What is missing is refusing the configuration in the panel. *Selected Features, on the four advanced trainers* (KNN Classifier, KNN Regressor, SVC, SVR). `Ground Truth Attribute Column` names the label and `Selected Features` names the columns handed to `fit`. The same overlap is reachable, and here nothing fails at all: the generated code is `X_train = self.dataset[features]` with the label among `features` (`SklearnAdvancedBaseDesc.scala:120-121`), so the estimator is given the answer as an input. On two hundred rows whose label is random noise, a KNN classifier scores 0.685 on the two real features and 0.995 once the label is included. Nothing in the run says the label was among the features. Expected: a column already chosen as the label is refused as an input column, with a message naming both fields. `SklearnModelOpDesc.getOutputSchemas` already refuses one impossible pairing this way, the `Count Vectorizer` switch on the two Gaussian Naive Bayes operators, so the shared base has a place for it. The advanced trainers have no such check today. ### How to reproduce? For Text Attribute, upload a CSV with a text column and a label: ``` note,species great excellent good,1 awful terrible bad,0 ``` Build `CSV File Scan` to `Bernoulli Naive Bayes`, connect the scan to both the training and the testing port, set Target Attribute to `species`, turn Count Vectorizer on, and set Text Attribute to `species` as well. The panel accepts it. The run ends with the ValueError above. For Selected Features, upload a CSV with two numeric columns and a label, build `CSV File Scan` to `KNN Classifier`, connect the scan to both ports, set Ground Truth Attribute Column to `species`, set Selected Features to all three columns including `species`, and add one parameter row with `n_neighbors` = `5`. The panel accepts it and the run finishes clean, having trained on the label. ### Version/Branch 1.3.0-incubating-SNAPSHOT (main) ### Commit Hash (Optional) 319859571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
