Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/21060
  
    It looks to me this is a bug fix that can merit backporting, as 
QueryExecutionListener is also marked as experimental,
    
    In this case, I think @gatorsmile is worried one might have written a 
listener that enumerates the possible function names, and that listener will 
fail now with a new action name. I feel this is quite unlikely, but I also 
appreciate @gatorsmile's concern for backward compatibility, and I've certainly 
been wrong before when our fixes break existing workloads.
    
    (On the spectrum of being extremely conservative to extremely liberal, I 
think I'm in general more on the middle, whereas @gatorsmile probably leans 
more to the conservative side. There isn't really anything wrong with this, and 
it's good to have balancing forces in a project.)
    
    How about this, @HyukjinKwon -- for the 2.3.x backport, add a config that 
so it is possible to turn this off in production, if somebody actually has 
their job failed because of this? It's a small delta from what this PR already 
does, and that should alleviate the concerns @gatorsmile has. I'd also change 
the function doc for onSuccess/onFailure to make it clear that we will add new 
function names in the future, and users shouldn't expect a fixed list of 
function names.
    
    
    



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to