That repartition seems to do nothing? But yes the key point is use col() On Thu, Jun 9, 2022, 9:41 PM Stelios Philippou <stevo...@gmail.com> wrote:
> Perhaps > > > finalDF.repartition(finalDF.rdd.getNumPartitions()).withColumn("status_for_batch > > To > > finalDF.repartition(finalDF.rdd.getNumPartitions()).withColumn(col("status_for_batch") > > > > > On Thu, 9 Jun 2022, 22:32 Sid, <flinkbyhe...@gmail.com> wrote: > >> Hi Experts, >> >> I am facing one problem while passing a column to the method. The >> problem is described in detail here: >> >> >> https://stackoverflow.com/questions/72565095/how-to-pass-columns-as-a-json-record-to-the-api-method-using-pyspark >> >> TIA, >> Sid >> >