Re: "Schemaless" Spark

2016-08-19 Thread Sebastian Piu
You can do operations without a schema just fine, obviously the more you know about your data the more tools you will have, it is hard without more context on what you are trying to achieve. On Fri, 19 Aug 2016, 22:55 Efe Selcuk, wrote: > Hi Spark community, > > This is a

"Schemaless" Spark

2016-08-19 Thread Efe Selcuk
Hi Spark community, This is a bit of a high level question as frankly I'm not well versed in Spark or related tech. We have a system in place that reads columnar data in through CSV and represents the data in relational tables as it operates. It's essentially schema-based ETL. This restricts our