Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23016
@idlevi Actually, input and output of the prefix span are RDD. Earlier
intermediate rdd was cached, now final rdd is cached, and materialized it. So,
if you materialize the model, earlier it will compute from the intermediate
level, now it directly get from the finalRdd.
I ran all the UTs in the prefixSpanSuite, and there is hardly any time
difference with/without the patch.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
