how to do table partitioning efficiently?

[诺铁]

hi,

now I'm doing something like this on a data frame to make use of table
partitioning

df.filter($"sex" === "male").write.parquet("path/to/table/sex=male")
df.filter($"sex" === "female").write.parquet("path/to/table/sex=female")

this will filter dataset multiple times, are there better way to do this?


thanks.