[
https://issues.apache.org/jira/browse/SPARK-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-11992.
-------------------------------
Resolution: Invalid
[~Bonsanto] only committers should set "Blocker" and a question can't be one.
Despite the existence of this type, questions are for [email protected]
rather than JIRA (or yes stackoverflow). I think you have to narrow down your
problem more; it's not clear it's a Spark problem.
> Severl numbers in my spark shell (pyspark)
> ------------------------------------------
>
> Key: SPARK-11992
> URL: https://issues.apache.org/jira/browse/SPARK-11992
> Project: Spark
> Issue Type: Question
> Components: MLlib, PySpark
> Affects Versions: 1.5.2
> Environment: Linux Ubuntu 14.04 LTS
> Jupyter
> Spark 1.5.2
> Reporter: Alberto Bonsanto
> Priority: Blocker
> Labels: newbie
>
> The problem is very weird, I am currently trying to fit some classifiers from
> mllib library (SVM, LogisticRegression, RandomForest, DecisionTree and
> NaiveBayes), so they might classify the data properly, I am trying to compare
> their performances evaluating their predictions using my current validation
> data (the typical pipeline), and the problem is that when I try to fit any of
> those, my spark-shell console prints millions and millions of entries, and
> after that the fitting process gets stopped, you can see it
> [here|http://i.imgur.com/mohLnwr.png]
> Some details:
> - My data has around 15M of entries.
> - I use LabeledPoints to represent each entry, where the features are
> SparseVectors and they have *104* features or dimensions.
> - I don't show many things in the console,
> [log4j.properties|https://gist.github.com/Bonsanto/c487624db805f56882b8]
> - The program is running locally in a computer with 16GB of RAM.
> I have already asked this, in StackOverflow, you can see it here [Crazy
> print|http://stackoverflow.com/questions/33807347/pyspark-shell-outputs-several-numbers-instead-of-the-loading-arrow]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
