Dobiasd commented on issue #27128: [SPARK-30421][SQL] Dropped columns still 
available for filtering
URL: https://github.com/apache/spark/pull/27128#issuecomment-584630973
 
 
   For me, neither "Because the software has worked this way", nor "other 
similar software", are valid arguments. For me, it's just plainly wrong to be 
able to filter a dataframe on a column that does not exist in this dataframe.
   
   I think this behavior is an issue because it means one can not simply look 
at the schema of a dataframe to determine if an operation with it is valid. 
Instead one has to consider the whole history of how the dataframe was 
created/derived. This leads to the effect that refactorings, e.g., changing the 
way of creation of a dataframe, will break one's code, even though the 
refactoring should be totally OK because it results in the exact same dataframe 
schema.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to