Carbon don't support spark 2.x
------------------ Original ------------------ From: "??????";<[email protected]>; Date: Thu, Dec 15, 2016 07:45 PM To: "dev"<[email protected]>; Subject: Some questions about compiling carbondata Hi all, I've tried the following two ways to compile carbondata, but they finally all failed, if anyone could help me: 1. with the latest verson carbondata at github and spark 2.0.0, use the command mvn -DskipTests -Pspark-2.0 -Dspark.version=2.0.0 clean package and it compiled successly. but when I run the script ./bin/carbon-spark-shell , it turns out: ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory ./bin/carbon-spark-shell: line 78: /bin/spark-submit: No such file or directory and when I run the script ./bin/carbon-spark-sql, it turns out: $SPARK_HOME is not set ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory ./bin/carbon-spark-sql: line 79: cmd: command not found ./bin/carbon-spark-sql: line 89: /bin/spark-submit: No such file or directory ./bin/carbon-spark-sql: line 89: exec: /bin/spark-submit: cannot execute: No such file or directory 2.with the carbondata-0.2.0 and spark 1.5.0, use the command mvn -DskipTests -Pspark-1.5 -X -Dspark.version=1.5.0 clean package and it compiled failed. the error message is : [ERROR] /home/hadoop/carbondata-0.2.0/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonHiveMetadataUtil.scala:46: error: not found: value SqlParser [INFO] val tableIdent = SqlParser.parseTableIdentifier(tableWithDb) [INFO] ^ [WARNING] one warning found [ERROR] one error found [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache CarbonData :: Parent ........................ SUCCESS [ 0.971 s] [INFO] Apache CarbonData :: Common ........................ SUCCESS [ 1.702 s] [INFO] Apache CarbonData :: Core .......................... SUCCESS [ 4.428 s] [INFO] Apache CarbonData :: Processing .................... SUCCESS [ 1.587 s] [INFO] Apache CarbonData :: Hadoop ........................ SUCCESS [ 1.104 s] [INFO] Apache CarbonData :: Spark ......................... FAILURE [ 11.456 s] [INFO] Apache CarbonData :: Assembly ...................... SKIPPED [INFO] Apache CarbonData :: Examples ...................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 21.432 s [INFO] Finished at: 2016-12-15T19:41:29+08:00 [INFO] Final Memory: 69M/995M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project carbondata-spark: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project carbondata-spark: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.apache.maven.plugin.MojoExecutionException: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:350) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) ... 20 more Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:346) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:149) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:136) at org_scala_tools_maven_executions.JavaMainCallerByFork.run(JavaMainCallerByFork.java:80) at org_scala_tools_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:124) at org_scala_tools_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:80) at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:342) ... 22 more [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :carbondata-spark
