Can some one help me here? Please On Sat, Jun 20, 2015 at 9:54 AM Sathish Kumaran Vairavelu < vsathishkuma...@gmail.com> wrote:
> Hi, > > In Spark SQL JDBC data source there is an option to specify upper/lower > bound and num of partitions. How Spark handles data distribution, if we do > not give the upper/lower/num of parititons ? Will all data from the > external data source skewed up in one executor? > > In many situations, we do not know the upper/lower bound of the underlying > dataset until the query is executed, so it is not possible to pass > upper/lower bound values. > > > Thanks > > Sathish >
