So the reason for that is you already have spark 1.6 installed and tried to upgrade to 2.1 by apt-get update.
You can see that spark-extras has been renamed to spark-externals: https://ci.bigtop.apache.org/view/Releases/job/Bigtop-1.2.0/OS=ubuntu-16.04/lastSuccessfulBuild/artifact/output/spark/ https://ci.bigtop.apache.org/view/Releases/job/Bigtop-1.2.0/OS=ubuntu-16.04/lastSuccessfulBuild/artifact/output/spark/ I'm not quite sure what's inside that package. But it seems to be out of dependency. In general, your path of upgrade is important and worth to be document. I think we can have an Upgrade guide for these issues. 2017-04-12 2:55 GMT+08:00 MrAsanjar . <[email protected]>: > How to reproduce: > 1) Add Bigtop 1.2 repository > wget -O- > http://archive.apache.org/dist/bigtop/bigtop-1.2.0/repos/GPG-KEY-bigtop | > sudo apt-key add - > wget -O /etc/apt/sources.list.d/bigtop-1.2.0.list > https://www.apache.org/dist/bigtop/bigtop-1.2.0/repos/ > ubuntu16.04/bigtop.list > sudo apt-get -y update > 2) Query for spark packages by executing command "apt list | grep spark" > result: > spark-core/stable,now 2.1.0-1 all [installed] > spark-datanucleus/stable 2.1.0-1 all > spark-external/stable 2.1.0-1 all > spark-extras/stable 1.6.2-1 all > spark-history-server/stable,now 2.1.0-1 all [installed] > spark-master/stable,now 2.1.0-1 all [installed] > spark-python/stable 2.1.0-1 all > spark-thriftserver/stable 2.1.0-1 all > spark-worker/stable,now 2.1.0-1 all [installed] > spark-yarn-shuffle/stable 2.1.0-1 all > > > > > On Tue, Apr 11, 2017 at 1:29 PM, Evans Ye <[email protected]> wrote: > > > We still packages spark1 to workaround some components compatibility > issue. > > I guess that's why you got 1.6.2 available from 1.2 repo. > > > > How you get these packages installed? > > > > 2017-04-12 0:53 GMT+08:00 MrAsanjar . <[email protected]>: > > > > > Hi team, > > > Could anyone explain why we have spark-extras , a spark 1.6 package, > > > added to the ubuntu-16.04 repository for Bigtop 1.2? It should be all > > > spark 2.1 packages and not 1.6. > > > > > > List of spark packages in Bigtop 1.2 repo: > > > > > > spark-core/stable,now 2.1.0-1 all [installed] > > > spark-datanucleus/stable 2.1.0-1 all > > > spark-external/stable 2.1.0-1 all > > > spark-extras/stable 1.6.2-1 all <<<<< it should not be there, correct? > > > spark-history-server/stable,now 2.1.0-1 all [installed] > > > spark-master/stable,now 2.1.0-1 all [installed] > > > spark-python/stable 2.1.0-1 all > > > spark-thriftserver/stable 2.1.0-1 all > > > spark-worker/stable,now 2.1.0-1 all [installed] > > > spark-yarn-shuffle/stable 2.1.0-1 all > > > > > >
