Many thanks Ted and thanks for heads up Jakob Just these two changes to dependencies
<dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core*_2.10*</artifactId> <version>1.5.1</version> </dependency> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-sql*_2.10*</artifactId> <version>1.5.1</version> </dependency> [DEBUG] endProcessChildren: artifact=spark:scala:jar:1.0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:04 min [INFO] Finished at: 2016-03-15T22:55:08+00:00 [INFO] Final Memory: 32M/1089M [INFO] ------------------------------------------------------------------------ Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 15 March 2016 at 22:18, Jakob Odersky <ja...@odersky.com> wrote: > Hi Mich, > probably unrelated to the current error you're seeing, however the > following dependencies will bite you later: > <artifactId>spark-hive_2.10</artifactId> > <artifactId>spark-csv_2.11</artifactId> > the problem here is that you're using libraries built for different > Scala binary versions (the numbers after the underscore). The simple > fix here is to specify the Scala binary version you're project builds > for (2.10 in your case, however note that version is EOL, you should > upgrade to scala 2.11.8 if possible). > > On a side note, sbt takes care of handling correct scala versions for > you (the double %% actually is a shorthand for appending > "_scalaBinaryVersion" to your dependency). It also enables you to > build and publish your project seamlessly against multiple versions. I > would strongly recommend to use it in Scala projects. > > cheers, > --Jakob > > > > On Tue, Mar 15, 2016 at 3:08 PM, Mich Talebzadeh > <mich.talebza...@gmail.com> wrote: > > Hi, > > > > I normally use sbt and using this sbt file works fine for me > > > > cat ImportCSV.sbt > > name := "ImportCSV" > > version := "1.0" > > scalaVersion := "2.10.4" > > libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.1" > > libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.5.1" > > libraryDependencies += "org.apache.spark" %% "spark-hive" % "1.5.1" > > libraryDependencies += "com.databricks" % "spark-csv_2.11" % "1.3.0" > > > > This is my first trial using Mavan and pom > > > > > > my pom.xml file looks like this but throws error at build > > > > > > [DEBUG] com.univocity:univocity-parsers:jar:1.5.1:compile > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] BUILD FAILURE > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Total time: 1.326 s > > [INFO] Finished at: 2016-03-15T22:17:29+00:00 > > [INFO] Final Memory: 14M/455M > > [INFO] > > ------------------------------------------------------------------------ > > [ERROR] Failed to execute goal on project scala: Could not resolve > > dependencies for project spark:scala:jar:1.0: The following artifacts > could > > not be resolved: org.apache.spark:spark-core:jar:1.5.1, > > org.apache.spark:spark-sql:jar:1.5.1: Failure to find > > org.apache.spark:spark-core:jar:1.5.1 in > > https://repo.maven.apache.org/maven2 was cached in the local repository, > > resolution will not be reattempted until the update interval of central > has > > elapsed or updates are forced -> [Help 1] > > > > > > My pom file is > > > > > > cat pom.xml > > <project xmlns="http://maven.apache.org/POM/4.0.0"; > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > > http://maven.apache.org/maven-v4_0_0.xsd";> > > <modelVersion>4.0.0</modelVersion> > > <groupId>spark</groupId> > > <version>1.0</version> > > <name>${project.artifactId}</name> > > > > <properties> > > <maven.compiler.source>1.7</maven.compiler.source> > > <maven.compiler.target>1.7</maven.compiler.target> > > <encoding>UTF-8</encoding> > > <scala.version>2.10.4</scala.version> > > <maven-scala-plugin.version>2.15.2</maven-scala-plugin.version> > > </properties> > > > > <dependencies> > > <dependency> > > <groupId>org.scala-lang</groupId> > > <artifactId>scala-library</artifactId> > > <version>2.10.2</version> > > </dependency> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-core</artifactId> > > <version>1.5.1</version> > > </dependency> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-sql</artifactId> > > <version>1.5.1</version> > > </dependency> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-hive_2.10</artifactId> > > <version>1.5.0</version> > > </dependency> > > <dependency> > > <groupId>com.databricks</groupId> > > <artifactId>spark-csv_2.11</artifactId> > > <version>1.3.0</version> > > </dependency> > > </dependencies> > > > > <build> > > <sourceDirectory>src/main/scala</sourceDirectory> > > <plugins> > > <plugin> > > <groupId>org.scala-tools</groupId> > > <artifactId>maven-scala-plugin</artifactId> > > <version>${maven-scala-plugin.version}</version> > > <executions> > > <execution> > > <goals> > > <goal>compile</goal> > > </goals> > > </execution> > > </executions> > > <configuration> > > <jvmArgs> > > <jvmArg>-Xms64m</jvmArg> > > <jvmArg>-Xmx1024m</jvmArg> > > </jvmArgs> > > </configuration> > > </plugin> > > <plugin> > > <groupId>org.apache.maven.plugins</groupId> > > <artifactId>maven-shade-plugin</artifactId> > > <version>1.6</version> > > <executions> > > <execution> > > <phase>package</phase> > > <goals> > > <goal>shade</goal> > > </goals> > > <configuration> > > <filters> > > <filter> > > <artifact>*:*</artifact> > > <excludes> > > <exclude>META-INF/*.SF</exclude> > > <exclude>META-INF/*.DSA</exclude> > > <exclude>META-INF/*.RSA</exclude> > > </excludes> > > </filter> > > </filters> > > <transformers> > > <transformer > > > implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> > > <mainClass>com.group.id.Launcher1</mainClass> > > </transformer> > > </transformers> > > </configuration> > > </execution> > > </executions> > > </plugin> > > </plugins> > > </build> > > > > <artifactId>scala</artifactId> > > </project> > > > > > > I am sure I have omitted something? > > > > > > Thanks > > > > > > Dr Mich Talebzadeh > > > > > > > > LinkedIn > > > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > > > > > > http://talebzadehmich.wordpress.com > > > > >
