kz930 opened a new issue, #7597: URL: https://github.com/apache/texera/issues/7597
### What happened? The two Gaussian Naive Bayes operators, `Gaussian Naive Bayes` and `Training: Gaussian Naive Bayes`, expose a `Count Vectorizer` switch that cannot succeed. Turning it on ends the execution with an error raised from inside scikit-learn: `TypeError: Sparse data was passed for X, but dense data is required. Use '.toarray()' to convert to a dense numpy array.` `CountVectorizer` emits a scipy sparse matrix, and `GaussianNB` validates its input without `accept_sparse` because it estimates a per-feature mean and variance, which reads every cell including the zeros. The refusal is deliberate on scikit-learn's side: densifying a text matrix with a large vocabulary is what would exhaust memory. This is specific to `GaussianNB`, not a property of the switch. Fitting `make_pipeline(CountVectorizer(), Estimator())` on the same documents across all 26 estimators the Sklearn and Sklearn Training groups use, only `GaussianNB` raises; the other 25, including `MultinomialNB`, `BernoulliNB` and `ComplementNB`, accept the sparse matrix. So exactly two operators carry a switch that has no working configuration. The switch is declared once on the shared base, `SklearnModelOpDesc.scala:54`, and both families splice it into the pipeline the same way (`SklearnClassifierOpDesc.scala:48`, `SklearnTrainingOpDesc.scala:47`), so every operator in both groups inherits it whether or not its estimator can use it. Expected: a combination that cannot work should not be offered. Hiding or disabling `Count Vectorizer` on these two operators would match how the family already handles an impossible pairing, since `Tfidf Transformer` and `Text Attribute` are hidden when `Count Vectorizer` is off. Failing at compile time with a message naming the estimator and pointing at the Multinomial, Bernoulli and Complement variants would also resolve it. Silently densifying would not: it reintroduces the memory cost scikit-learn refuses for a reason. <img width="988" height="931" alt="Image" src="https://github.com/user-attachments/assets/e45e20f6-6a2b-4841-bff3-6c7671c7e8b4"; /> ### How to reproduce? Upload a CSV with a text column and a label: ``` note,y great excellent good,1 awful terrible bad,0 ``` Build `CSV File Scan` to `Gaussian Naive Bayes`, set Target Attribute to `y`, turn on `Count Vectorizer`, set Text Attribute to `note`, and run. The execution stops and the operator console shows the TypeError. `Training: Gaussian Naive Bayes` fails the same way on the same file. ### Version/Branch 1.3.0-incubating-SNAPSHOT (main) ### Relevant log output ```shell TypeError: Sparse data was passed for X, but dense data is required. Use '.toarray()' to convert to a dense numpy array. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
