Re: 答复: 答复: 答复: 答复: t it does not stop at breakpoints which is in an anonymous function

Hello Felix, No, this line isn't the one that is triggering the execution of the function, the count does that, unless your count val is a lazy val. The count method is the one that retrieves the information of the rdd, it has do go through all of it's data do determine how many records the RDD

答复: 答复: 答复: 答复: t it does not stop at breakpoints which is in an anonymous function

Dear Dirceu, Thanks for your kind help. i cannot see any code line corresponding to ". retrieve the data from your DataFrame/RDDs". which you suggested in the previous replies. Later, I guess the line val test = count is the key point. without it, it would not stop at the

Re: 答复: 答复: 答复: t it does not stop at breakpoints which is in an anonymous function

Hi Felix, Are sure your n is greater than 0? Here it stops first at breakpoint 1, image attached. Have you got the count to see if it's also greater than 0? 2016-09-15 11:41 GMT-03:00 chen yong : > Dear Dirceu > > > Thank you for your help. > > > Acutally, I use Intellij IDEA

答复: 答复: 答复: t it does not stop at breakpoints which is in an anonymous function

Dear Dirceu Thank you for your help. Acutally, I use Intellij IDEA to dubug the spark code. Let me use the following code snippet to illustrate my problem. In the code lines below, I've set two breakpoints, breakpoint-1 and breakpoint-2. when i debuged the code, it did not stop at

答复: 答复: t it does not stop at breakpoints which is in an anonymous function

Dear Dirceu, thanks you again. Actually,I never saw it stopped at the breakpoints no matter how long I wait. It just skipped the whole anonymous function to direactly reach the first breakpoint immediately after the anonymous function body. Is that normal? I suspect sth wrong in my

答复: t it does not stop at breakpoints which is in an anonymous function

Thanks for your reply. you mean i have to insert some codes, such as x.count or x.collect, between the original spark code lines to invoke some operations, right? but, where is the right places to put my code lines? Felix 发件人: Dirceu Semighini Filho

Re: t it does not stop at breakpoints which is in an anonymous function

Hello Felix, Spark functions run lazy, and that's why it doesn't stop in those breakpoints. They will be executed only when you call some methods of your dataframe/rdd, like the count, collect, ... Regards, Dirceu 2016-09-14 11:26 GMT-03:00 chen yong : > Hi all, > > > > I am

t it does not stop at breakpoints which is in an anonymous function

Hi all, I am newbie to spark. I am learning spark by debugging the spark code. It is strange to me that it does not stop at breakpoints which is in an anonymous function, it is normal in ordianry function, though. It that normal. How to obverse variables in an anonymous function. Please