I am not using case when. It is mostly IF. By slow, I mean 6 min even for 10 records for 41 level nested ifs.
On Jan 11, 2017 3:31 PM, "Georg Heiler" <[email protected]> wrote: > I was using the dataframe api not sql. The main problem was that too much > code was generated. > Using an unforgettable turned out to be quicker as well. > Olivier Girardot <[email protected]> schrieb am Di. 10. > Jan. 2017 um 21:54: > >> Are you using the "case when" functions ? what do you mean by slow ? can >> you share a snippet ? >> >> >> >> On Tue, Jan 10, 2017 8:15 PM, Georg Heiler [email protected] >> wrote: >> >> Maybe you can create an UDF? >> >> Raghavendra Pandey <[email protected]> schrieb am Di., 10. >> Jan. 2017 um 20:04 Uhr: >> >> I have of around 41 level of nested if else in spark sql. I have >> programmed it using apis on dataframe. But it takes too much time. >> Is there anything I can do to improve on time here? >> >> >> >> *Olivier Girardot* | Associé >> [email protected] >> +33 6 24 09 17 94 >> >
