Re: DataSourceV2 hangouts sync

2018-10-28 Thread Russell Spitzer
Responding for invite On Fri, Oct 26, 2018, 12:34 PM Ryan Blue wrote: > Looks like the majority opinion is for Wednesday. I've sent out an invite > to everyone that replied and will add more people as I hear more responses. > > Thanks, everyone! > > On Fri, Oct 26, 2018 at 3:23 AM Gengliang

Re: Helper methods for PySpark discussion

2018-10-28 Thread Reynold Xin
I agree - it is very easy for users to shoot themselves in the foot if we don't put in the safeguards, or mislead them by giving them the impression that operations are cheap. DataFrame in Spark isn't like a single node in-memory data structure. Note that the repr string work is very different.