i noticed much to my surprise that spark csv supports newlines inside quoted values. ok thats cool
but how does this work with splitting files when reading? i assume splitting is still simply done on newlines or something similar. wouldnt that potentially split in the middle of a record?
