Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/6744#discussion_r33252541
--- Diff: docs/mllib-linear-methods.md ---
@@ -768,6 +768,58 @@ will get better!
</div>
+<div data-lang="python" markdown="1">
+
+First, we import the necessary classes for parsing our input data and
creating the model.
+
+{% highlight python %}
+from pyspark.mllib.linalg import Vectors
+from pyspark.mllib.regression import LabeledPoint
+from pyspark.mllib.regression import StreamingLinearRegressionWithSGD
+{% endhighlight %}
+
+Then we make input streams for training and testing data. We assume a
StreamingContext `ssc`
+has already been created, see [Spark Streaming Programming
Guide](streaming-programming-guide.html#initializing)
+for more info. For this example, we use labeled points in training and
testing streams,
+but in practice you will likely want to use unlabeled vectors for test
data.
+
+{% highlight python %}
+def parse(lp):
+ label = float(lp[lp.find('(') + 1: lp.find(',')])
+ vec = Vectors.dense(lp[lp.find('[') + 1: lp.find(']')].split(','))
+ return LabeledPoint(label, vec)
+
+trainingData = ssc.textFileStream("/training/data/dir").map(parse).cache()
+testData = ssc.textFileStream("/testing/data/dir").map(parse)
+{% endhighlight %}
+
+We create our model by initializing the weights to 0
+
+{% highlight python %}
+numFeatures = 3
+model = StreamingLinearRegressionWithSGD()
+model.setInitialWeights([0.0, 0.0, 0.0])
+{% endhighlight %}
+
+Now we register the streams for training and testing and start the job.
+
+{% highlight scala %}
--- End diff --
Forgot to change while copy pasting :P
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
