What's relationship between the TensorflowOnSpark core modules?

2018-02-23 Thread xiaobo
Hi, After reading the mnist example and the API of TensorflowOnSpark, I somehow got confused, here are some questions: 1、 What's the relationship between TFCluster/TFManager/TFNode and TFSparkNode modules. 2、The conversion guide says we should replace the main function with a main_fun, but the

Re: Does Pyspark Support Graphx?

2018-02-19 Thread xiaobo
icholas.hakob...@rallyhealth.com> Date: Tue,Feb 20,2018 3:37 AM To: xiaobo <guxiaobo1...@qq.com> Cc: Denny Lee <denny.g@gmail.com>, user@spark.apache.org <user@spark.apache.org> Subject: Re: Does Pyspark Support Graphx? If you copy the Jar file and all of the dependencies to the ma

Re: [graphframes]how Graphframes Deal With BidirectionalRelationships

2018-02-19 Thread xiaobo
So the question comes to does graphframes support bidirectional relationship natively with only one edge? -- Original -- From: Felix Cheung <felixcheun...@hotmail.com> Date: Tue,Feb 20,2018 10:01 AM To: xiaobo <guxiaobo1...@qq.com>, user@spark.apac

Re: Does Pyspark Support Graphx?

2018-02-18 Thread xiaobo
Another question is how to install graphframes permanently when the spark nodes can not connect to the internet. -- Original -- From: Denny Lee <denny.g@gmail.com> Date: Mon,Feb 19,2018 10:23 AM To: xiaobo <guxiaobo1...@qq.com> Cc: user@spar

[graphframes]how Graphframes Deal With Bidirectional Relationships

2018-02-18 Thread xiaobo
Hi, To represent a bidirectional relationship, one solution is to insert two edges for the vertices pair, my question is do the algorithms of graphframes still work when we doing this. Thanks

Re: Does Pyspark Support Graphx?

2018-02-18 Thread xiaobo
Hi Denny, The pyspark script uses the --packages option to load graphframe library, what about the SparkLauncher class? -- Original -- From: Denny Lee Date: Sun,Feb 18,2018 11:07 AM To: 94035420 Cc:

Can Precompiled Stand Alone Python Application Submitted To A Spark Cluster?

2018-02-17 Thread xiaobo
Hi, To pretect the IP of our software distributed to customers, one solution is to use precompiled python scriptes, but we are wondering whether this is a supported feature by pyspark. Thanks.

Re: Does Pyspark Support Graphx?

2018-02-17 Thread xiaobo
Thanks Denny, will it be supported in the near future? -- Original -- From: Denny Lee Date: Sun,Feb 18,2018 11:05 AM To: 94035420 Cc: user@spark.apache.org Subject: Re: Does Pyspark Support