016 at 7:58 PM, Daniel Darabos <
> daniel.dara...@lynxanalytics.com> wrote:
>
> Hi Shreya,
> GraphFrames just calls the GraphX strongly connected components code. (
> https://github.com/graphframes/graphframes/blob/release-0.2.0/src/main/scala/org/graphframes/lib/StronglyCo
daniel.dara...@lynxanalytics.com> wrote:
>
>> Hi Shreya,
>> GraphFrames just calls the GraphX strongly connected components code. (
>> https://github.com/graphframes/graphframes/blob/release-0.
>> 2.0/src/main/scala/org/graphframes/lib/StronglyConnec
>> tedCom
the exact same algo as is implemented in graphx i think. or is it
not?
On Fri, Nov 11, 2016 at 7:58 PM, Daniel Darabos <
daniel.dara...@lynxanalytics.com> wrote:
> Hi Shreya,
> GraphFrames just calls the GraphX strongly connected components code. (
> https://github.com/graphframes/gr
g; Denny Lee
<denny@microsoft.com>
Subject: Re: Strongly Connected Components
Hi Shreya,
GraphFrames just calls the GraphX strongly connected components code.
(https://github.com/graphframes/graphframes/blob/release-0.2.0/src/main/scala/org/graphframes/lib/StronglyConnectedCo
Hi Shreya,
GraphFrames just calls the GraphX strongly connected components code. (
https://github.com/graphframes/graphframes/blob/release-0.2.0/src/main/scala/org/graphframes/lib/StronglyConnectedComponents.scala#L51
)
For choosing the number of iterations: If the number of iterations is less
ct: RE: Strongly Connected Components
Yesterday's run died sometime during the night, without any errors. Today, I am
running it using GraphFrames instead. It is still spawning new tasks, so there
is progress.
From: Felix Cheung [mailto:felixcheun...@hotmail.com]
Sent: Thursday, November 10, 2016 7
; Shreya Agarwal <shrey...@microsoft.com>
Subject: Re: Strongly Connected Components
It is possible it is dead. Could you check the Spark UI to see if there is any
progress?
_
From: Shreya Agarwal <shrey...@microsoft.com<mailto:shrey...@microsoft.com>&
It is possible it is dead. Could you check the Spark UI to see if there is any
progress?
_
From: Shreya Agarwal <shrey...@microsoft.com<mailto:shrey...@microsoft.com>>
Sent: Thursday, November 10, 2016 12:45 AM
Subject: RE: Strongly Connected Componen
Bump. Anyone? Its been running for 10 hours now. No results.
From: Shreya Agarwal
Sent: Tuesday, November 8, 2016 9:05 PM
To: user@spark.apache.org
Subject: Strongly Connected Components
Hi,
I am running this on a graph with >5B edges and >3B edges and have 2 questions -
1.
Hi,
I am running this on a graph with >5B edges and >3B edges and have 2 questions -
1. What is the optimal number of iterations?
2. I am running it for 1 iteration right now on a beefy 100 node cluster,
with 300 executors each having 30GB RAM and 5 cores. I have persisted the graph
to
10 matches
Mail list logo