Github user tgravescs commented on a diff in the pull request:
    --- Diff: docs/ ---
    @@ -1795,6 +1796,19 @@ Apart from these, the following properties are also 
available, and may be useful
         Lower bound for the number of executors if dynamic allocation is 
    +  <td><code>spark.dynamicAllocation.fullParallelismDivisor</code></td>
    +  <td>1</td>
    +  <td>
    +    By default, the dynamic allocation will request enough executors to 
maximize the 
    +    parallelism according to the number of tasks to process. While this 
minimizes the 
    +    latency of the job, with small tasks this setting wastes a lot of 
resources due to
    +    executor allocation overhead, as some executor might not even do any 
    +    This setting allows to set a divisor that will be used to reduce the 
number of
    +    executors w.r.t. full parallelism
    --- End diff --
    add period at end of parallelism


To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to