How about storing logical plans (or printDebugString, in case of RDD) to an external file on the driver?
On Tue, Apr 4, 2017 at 1:19 PM, kant kodali <kanth...@gmail.com> wrote: > Hi All, > > I am wondering if there a way to persist the lineages generated by spark > underneath? Some of our clients want us to prove if the result of the > computation that we are showing on a dashboard is correct and for that If > we can show the lineage of transformations that are executed to get to the > result then that can be the Q.E.D moment but I am not even sure if this is > even possible with spark? > > Thanks, > kant > -- Best Regards, Ayan Guha
