Anyone has perfect solution for spark source code compilation issue on intellij

Hi folks, I often meet the spark compilation issue on intellij. It wastes me lots of time. I googled it and found someone else also meet similar issue, but seems no perfect solution for now. but still wondering anyone here has perfect solution for that. The issue happens sometimes, I don't know

Re: PMML export for LinearRegressionModel

The method toPMML is in trait PMMLExportable *LinearRegressionModel has this trait, you should be able to call * *LinearRegressionModel#toPMML* On Thu, Oct 15, 2015 at 5:25 PM, Fazlan Nazeem wrote: > Hi > > I am trying to export a LinearRegressionModel in PMML format.

Re: When does python program started in pyspark

com> wrote: > See PythonRunner @ > https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/PythonRunner.scala > > On Tue, Oct 13, 2015 at 7:50 PM, canan chen <ccn...@gmail.com> wrote: > >> I look at the source code of spark, but didn't find where

When does python program started in pyspark

I look at the source code of spark, but didn't find where python program is started in python. It seems spark-submit will call PythonGatewayServer, but where is python program started ? Thanks

Re: What's the best practice for developing new features for spark ?

: http://search-hadoop.com/m/q3RTtdZv0d1btRHl/Spark+build+modulesubj=Building+Spark+Building+just+one+module+ On Aug 19, 2015, at 1:44 AM, canan chen ccn...@gmail.com wrote: I want to work on one jira, but it is not easy to do unit test, because it involves different components especially UI

What's the best practice for developing new features for spark ?

I want to work on one jira, but it is not easy to do unit test, because it involves different components especially UI. spark building is pretty slow, I don't want to build it each time to test my code change. I am wondering how other people do ? Is there any experience can share ? Thanks