Re: DAG Scheduler deadlock when two RDDs reference each other, force Stages manually?

2015-09-14 Thread Petros Nyfantis
for which the M matrix entry is 1 and then mapping again to their sum, but this means I need a lot of space especially when the problem in hand could be very large, which is unfeasible, so I need to avoid this) Thanks in advance for your help! -- View this mess

Re: DAG Scheduler deadlock when two RDDs reference each other, force Stages manually?

2015-09-14 Thread Sean Owen
s to a List of all the B elements > for which the M > matrix entry is 1 and then mapping again to their sum, but this means I need > a lot of space > especially when the problem in hand could be very large, which is > unfeasible, so I need to avoid this) > > Thanks in advance

DAG Scheduler deadlock when two RDDs reference each other, force Stages manually?

2015-09-14 Thread petranidis
user-list.1001560.n3.nabble.com/DAG-Scheduler-deadlock-when-two-RDDs-reference-each-other-force-Stages-manually-tp24684.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user