Please let me know if anybody has any thoughts on this issue?
On Thu, Mar 30, 2017 at 10:37 PM Sathish Kumaran Vairavelu <
vsathishkuma...@gmail.com> wrote:
> Also, is it possible to cache logical plan and parsed query so that in
> subsequent executions it can be reused. It would improve overall
Also, is it possible to cache logical plan and parsed query so that in
subsequent executions it can be reused. It would improve overall query
performance particularly in streaming jobs
On Thu, Mar 30, 2017 at 10:06 PM Sathish Kumaran Vairavelu <
vsathishkuma...@gmail.com> wrote:
> Hi Ayan,
>
> I h
Hi Ayan,
I have searched Spark configuration options but couldn't find one to pin
execution plans in memory. Can you please help?
Thanks
Sathish
On Thu, Mar 30, 2017 at 9:30 PM ayan guha wrote:
> I think there is an option of pinning execution plans in memory to avoid
> such scenarios
>
I think there is an option of pinning execution plans in memory to avoid
such scenarios
On Fri, Mar 31, 2017 at 1:25 PM, Sathish Kumaran Vairavelu <
vsathishkuma...@gmail.com> wrote:
> Hi Everyone,
>
> I have complex SQL with approx 2000 lines of code and works with 50+
> tables with 50+ left
Hi Everyone,
I have complex SQL with approx 2000 lines of code and works with 50+ tables
with 50+ left joins and transformations. All the tables are fully cached in
Memory with sufficient storage memory and working memory. The issue is
after the launch of the query for the execution; the query tak