i just noticed that spark 2.1.0 bring in a new transitive dependency on shapeless 2.0.0
shapeless is a popular library for scala users, and shapeless 2.0.0 is old (2014) and not compatible with more current versions. so this means a spark user that uses shapeless in his own development cannot upgrade safely from 2.0.0 to 2.1.0, i think. wish i had noticed this sooner