Hi Mich, probably unrelated to the current error you're seeing, however the following dependencies will bite you later: <artifactId>spark-hive_2.10</artifactId> <artifactId>spark-csv_2.11</artifactId> the problem here is that you're using libraries built for different Scala binary versions (the numbers after the underscore). The simple fix here is to specify the Scala binary version you're project builds for (2.10 in your case, however note that version is EOL, you should upgrade to scala 2.11.8 if possible).
On a side note, sbt takes care of handling correct scala versions for you (the double %% actually is a shorthand for appending "_scalaBinaryVersion" to your dependency). It also enables you to build and publish your project seamlessly against multiple versions. I would strongly recommend to use it in Scala projects. cheers, --Jakob On Tue, Mar 15, 2016 at 3:08 PM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Hi, > > I normally use sbt and using this sbt file works fine for me > > cat ImportCSV.sbt > name := "ImportCSV" > version := "1.0" > scalaVersion := "2.10.4" > libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.1" > libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.5.1" > libraryDependencies += "org.apache.spark" %% "spark-hive" % "1.5.1" > libraryDependencies += "com.databricks" % "spark-csv_2.11" % "1.3.0" > > This is my first trial using Mavan and pom > > > my pom.xml file looks like this but throws error at build > > > [DEBUG] com.univocity:univocity-parsers:jar:1.5.1:compile > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 1.326 s > [INFO] Finished at: 2016-03-15T22:17:29+00:00 > [INFO] Final Memory: 14M/455M > [INFO] > ------------------------------------------------------------------------ > [ERROR] Failed to execute goal on project scala: Could not resolve > dependencies for project spark:scala:jar:1.0: The following artifacts could > not be resolved: org.apache.spark:spark-core:jar:1.5.1, > org.apache.spark:spark-sql:jar:1.5.1: Failure to find > org.apache.spark:spark-core:jar:1.5.1 in > https://repo.maven.apache.org/maven2 was cached in the local repository, > resolution will not be reattempted until the update interval of central has > elapsed or updates are forced -> [Help 1] > > > My pom file is > > > cat pom.xml > <project xmlns="http://maven.apache.org/POM/4.0.0"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > http://maven.apache.org/maven-v4_0_0.xsd";> > <modelVersion>4.0.0</modelVersion> > <groupId>spark</groupId> > <version>1.0</version> > <name>${project.artifactId}</name> > > <properties> > <maven.compiler.source>1.7</maven.compiler.source> > <maven.compiler.target>1.7</maven.compiler.target> > <encoding>UTF-8</encoding> > <scala.version>2.10.4</scala.version> > <maven-scala-plugin.version>2.15.2</maven-scala-plugin.version> > </properties> > > <dependencies> > <dependency> > <groupId>org.scala-lang</groupId> > <artifactId>scala-library</artifactId> > <version>2.10.2</version> > </dependency> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-core</artifactId> > <version>1.5.1</version> > </dependency> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-sql</artifactId> > <version>1.5.1</version> > </dependency> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-hive_2.10</artifactId> > <version>1.5.0</version> > </dependency> > <dependency> > <groupId>com.databricks</groupId> > <artifactId>spark-csv_2.11</artifactId> > <version>1.3.0</version> > </dependency> > </dependencies> > > <build> > <sourceDirectory>src/main/scala</sourceDirectory> > <plugins> > <plugin> > <groupId>org.scala-tools</groupId> > <artifactId>maven-scala-plugin</artifactId> > <version>${maven-scala-plugin.version}</version> > <executions> > <execution> > <goals> > <goal>compile</goal> > </goals> > </execution> > </executions> > <configuration> > <jvmArgs> > <jvmArg>-Xms64m</jvmArg> > <jvmArg>-Xmx1024m</jvmArg> > </jvmArgs> > </configuration> > </plugin> > <plugin> > <groupId>org.apache.maven.plugins</groupId> > <artifactId>maven-shade-plugin</artifactId> > <version>1.6</version> > <executions> > <execution> > <phase>package</phase> > <goals> > <goal>shade</goal> > </goals> > <configuration> > <filters> > <filter> > <artifact>*:*</artifact> > <excludes> > <exclude>META-INF/*.SF</exclude> > <exclude>META-INF/*.DSA</exclude> > <exclude>META-INF/*.RSA</exclude> > </excludes> > </filter> > </filters> > <transformers> > <transformer > implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> > <mainClass>com.group.id.Launcher1</mainClass> > </transformer> > </transformers> > </configuration> > </execution> > </executions> > </plugin> > </plugins> > </build> > > <artifactId>scala</artifactId> > </project> > > > I am sure I have omitted something? > > > Thanks > > > Dr Mich Talebzadeh > > > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > > http://talebzadehmich.wordpress.com > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
