Marcelo Vanzin commented on SPARK-20327:

bq. I think the point is, without reflection, using 3.x+ APIs with 2.x results 
in complete failures at runtime

More than that, without reflection, the code will not compile against 2.x. And 
even when we add support for 3.x, we won't drop support for 2.x at the same 
time. So reflection is the only solution here (well, not the only, but the 

bq. then we handle this case as an error explicitly and inform the user about 

The way this was done in the past was to print a warning and continue without 
using the feature. Either way should be fine though.

> Add CLI support for YARN custom resources, like GPUs
> ----------------------------------------------------
>                 Key: SPARK-20327
>                 URL: https://issues.apache.org/jira/browse/SPARK-20327
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Shell, Spark Submit
>    Affects Versions: 2.1.0
>            Reporter: Daniel Templeton
>            Priority: Major
>              Labels: newbie
> YARN-3926 adds the ability for administrators to configure custom resources, 
> like GPUs.  This JIRA is to add support to Spark for requesting resources 
> other than CPU virtual cores and memory.  See YARN-3926.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to