Re: How to populate all possible combination values in columns using Spark SQL

2020-05-09 Thread Edgardo Szrajber
Once you pivot you can use CASE WHEN to calculate the True/false, then you can agregate.Bentzi Sent from Yahoo Mail on Android On Sat, May 9, 2020 at 21:18, Aakash Basu wrote: I know how to pivot, but how to aggregate and pivot and build those True and False combination is the doubt. On

Re: How to populate all possible combination values in columns using Spark SQL

2020-05-09 Thread Aakash Basu
I know how to pivot, but how to aggregate and pivot and build those True and False combination is the doubt. On Fri 8 May, 2020, 1:31 PM Edgardo Szrajber, wrote: > Have you checked the pivot function? > Bentzi > > Sent from Yahoo Mail on Android >

Re: How to populate all possible combination values in columns using Spark SQL

2020-05-08 Thread Edgardo Szrajber
Have you checked the pivot function?Bentzi Sent from Yahoo Mail on Android On Thu, May 7, 2020 at 22:46, Aakash Basu wrote: Hi, I've updated the SO question with masked data, added year column and other requirement. Please take a look. Hope this helps in solving the problem. Thanks and

Re: How to populate all possible combination values in columns using Spark SQL

2020-05-07 Thread Aakash Basu
Hi, I've updated the SO question with masked data, added year column and other requirement. Please take a look. Hope this helps in solving the problem. Thanks and regards, AB On Thu 7 May, 2020, 10:59 AM Sonal Goyal, wrote: > As mentioned in the comments on SO, can you provide a (masked)

Re: How to populate all possible combination values in columns using Spark SQL

2020-05-06 Thread Sonal Goyal
As mentioned in the comments on SO, can you provide a (masked) sample of the data? It will be easier to see what you are trying to do if you add the year column Thanks, Sonal Nube Technologies On Thu, May 7, 2020 at 10:26 AM

How to populate all possible combination values in columns using Spark SQL

2020-05-06 Thread Aakash Basu
Hi, I've described the problem in Stack Overflow with a lot of detailing, can you kindly check and help if possible? https://stackoverflow.com/q/61643910/5536733 I'd be absolutely fine if someone solves it using Spark SQL APIs rather than plain spark SQL query. Thanks, Aakash.