hi Tom Chan, i did tried with directory u told me to do. but got an error, i know where did that error emerging from, but as for the document i myself edited the appName to appId,
now the error is like this, darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio train SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/darshan/PredictionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/darshan/PredictionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] [INFO] [Runner$] Submission command: /home/darshan/PredictionIO/vendors//spark-1.6.3/bin/spark-submit --class org.apache.predictionio.workflow.CreateWorkflow --jars file:/home/darshan/PredictionIO/lib/mysql-connector-java-5.1.37.jar,file:/home/darshan/PredictionIO/tapster-episode-similar/target/scala-2.10/template-scala-parallel-similarproduct_2.10-0.1-SNAPSHOT.jar,file:/home/darshan/PredictionIO/tapster-episode-similar/target/scala-2.10/template-scala-parallel-similarproduct-assembly-0.1-SNAPSHOT-deps.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar --files file:/home/darshan/PredictionIO/conf/log4j.properties --driver-class-path /home/darshan/PredictionIO/conf:/home/darshan/PredictionIO/lib/mysql-connector-java-5.1.37.jar --driver-java-options -Dpio.log.dir=/home/darshan file:/home/darshan/PredictionIO/lib/pio-assembly-0.11.0-incubating.jar --engine-id org.example.similarproduct.SimilarProductEngine --engine-version 7ff66e8607f7d36b79cb9e9f3b97b53287e553f7 --engine-variant file:/home/darshan/PredictionIO/tapster-episode-similar/engine.json --verbosity 0 --json-extractor Both --env PIO_ENV_LOADED=1,PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/home/darshan/.pio_store,PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost:3306/pio,PIO_HOME=/home/darshan/PredictionIO,PIO_FS_ENGINESDIR=/home/darshan/.pio_store/engines,PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=MYSQL,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=MYSQL,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio,PIO_FS_TMPDIR=/home/darshan/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=MYSQL,PIO_CONF_DIR=/home/darshan/PredictionIO/conf [INFO] [Engine] Extracting datasource params... [INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be used. [INFO] [Engine] Datasource params: (,DataSourceParams(null)) [INFO] [Engine] Extracting preparator params... [INFO] [Engine] Preparator params: (,Empty) [INFO] [Engine] Extracting serving params... [INFO] [Engine] Serving params: (,Empty) [WARN] [Utils] Your hostname, darshu resolves to a loopback address: 127.0.0.1; using 192.168.2.103 instead (on interface wlx001ea6631cc7) [WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address [INFO] [Remoting] Starting remoting [INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp:// [email protected]:45309] [INFO] [Engine$] EngineWorkflow.train [INFO] [Engine$] DataSource: org.example.similarproduct.DataSource@168b4cb0 [INFO] [Engine$] Preparator: org.example.similarproduct.Preparator@45545e7a [INFO] [Engine$] AlgorithmList: List(org.example.similarproduct.ALSAlgorithm@23cbbd07) [INFO] [Engine$] Data sanity check is on. [ERROR] [Common$] Invalid app name null Exception in thread "main" java.lang.IllegalArgumentException: Invalid app name null at org.apache.predictionio.data.store.Common$$anonfun$appNameToId$2.apply(Common.scala:50) at org.apache.predictionio.data.store.Common$$anonfun$appNameToId$2.apply(Common.scala:48) at scala.Option.getOrElse(Option.scala:120) at org.apache.predictionio.data.store.Common$.appNameToId(Common.scala:48) at org.apache.predictionio.data.store.PEventStore$.aggregateProperties(PEventStore.scala:108) at org.example.similarproduct.DataSource.readTraining(DataSource.scala:31) at org.example.similarproduct.DataSource.readTraining(DataSource.scala:18) at org.apache.predictionio.controller.PDataSource.readTrainingBase(PDataSource.scala:40) at org.apache.predictionio.controller.Engine$.train(Engine.scala:644) at org.apache.predictionio.controller.Engine.train(Engine.scala:177) at org.apache.predictionio.workflow.CoreWorkflow$.runTrain(CoreWorkflow.scala:67) at org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:250) at org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) i m using MySQL as my repository. i followed the article from official website only, still i m not able to execute the pio build or pio train. My src/main/scala DataSource.scala program is still as it is in previously mentioned. Thanks, -DAN On Wed, Jul 26, 2017 at 3:22 PM, Tom Chan <[email protected]> wrote: > darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$ pio > train > > Can you try the command from the tapster-episode-similar directory? > > Tom > > On Jul 26, 2017 2:45 AM, "Darshan A.N." <[email protected]> wrote: > >> hi team, >> i am trying to install demo tapster. i followed >> http://predictionio.incubator.apache.org/demo/tapster/. It took more >> than a week to install predictionio. >> now that i m installed the PIO, its throwing me error while running the >> $pio build command. the error goes like this: >> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio build >> SLF4J: Class path contains multiple SLF4J bindings. >> SLF4J: Found binding in [jar:file:/home/darshan/Predic >> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja >> r!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in [jar:file:/home/darshan/Predic >> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp >> l/StaticLoggerBinder.class] >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> explanation. >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> [INFO] [Engine$] Using command '/home/darshan/PredictionIO/sbt/sbt' at >> /home/darshan/PredictionIO/tapster-episode-similar to build. >> [INFO] [Engine$] If the path above is incorrect, this process will fail. >> [INFO] [Engine$] Uber JAR disabled. Making sure >> lib/pio-assembly-0.11.0-incubating.jar is absent. >> [INFO] [Engine$] Going to run: /home/darshan/PredictionIO/sbt/sbt >> package assemblyPackageDependency in /home/darshan/PredictionIO/tap >> ster-episode-similar >> [ERROR] [Engine$] [error] /home/darshan/PredictionIO/tap >> ster-episode-similar/src/main/scala/DataSource.scala:63: not found: >> value eventsDb >> [ERROR] [Engine$] [error] val viewEventsRDD: RDD[ViewEvent] = >> eventsDb.find( >> [ERROR] [Engine$] [error] ^ >> [ERROR] [Engine$] [error] one error found >> [ERROR] [Engine$] [error] (compile:compileIncremental) Compilation failed >> [ERROR] [Engine$] [error] Total time: 5 s, completed 26 Jul, 2017 1:56:50 >> AM >> [ERROR] [Engine$] Return code of build command: >> /home/darshan/PredictionIO/sbt/sbt package assemblyPackageDependency is >> 1. Aborting. >> [INFO] [Engine$] Looking for an engine... >> [INFO] [Engine$] Found template-scala-parallel-simila >> rproduct_2.10-0.1-SNAPSHOT.jar >> [INFO] [Engine$] Found template-scala-parallel-simila >> rproduct-assembly-0.1-SNAPSHOT-deps.jar >> [INFO] [Engine$] Build finished successfully. >> [INFO] [Pio$] Your engine is ready for training. >> >> >> and the file DataSource.scala, >> >> >> package org.example.similarproduct >> >> import org.apache.predictionio.controller.PDataSource >> import org.apache.predictionio.controller.EmptyEvaluationInfo >> import org.apache.predictionio.controller.EmptyActualResult >> import org.apache.predictionio.controller.Params >> import org.apache.predictionio.data.storage.Event >> import org.apache.predictionio.data.store.PEventStore >> >> import org.apache.spark.SparkContext >> import org.apache.spark.SparkContext._ >> import org.apache.spark.rdd.RDD >> >> import grizzled.slf4j.Logger >> >> case class DataSourceParams(appName: String) extends Params >> >> class DataSource(val dsp: DataSourceParams) >> extends PDataSource[TrainingData, >> EmptyEvaluationInfo, Query, EmptyActualResult] { >> >> @transient lazy val logger = Logger[this.type] >> >> override >> def readTraining(sc: SparkContext): TrainingData = { >> >> // create a RDD of (entityID, User) >> val usersRDD: RDD[(String, User)] = PEventStore.aggregateProperties( >> appName = dsp.appName, >> entityType = "user" >> )(sc).map { case (entityId, properties) => >> val user = try { >> User() >> } catch { >> case e: Exception => { >> logger.error(s"Failed to get properties ${properties} of" + >> s" user ${entityId}. Exception: ${e}.") >> throw e >> } >> } >> (entityId, user) >> }.cache() >> >> // create a RDD of (entityID, Item) >> val itemsRDD: RDD[(String, Item)] = PEventStore.aggregateProperties( >> appName = dsp.appName, >> entityType = "item" >> )(sc).map { case (entityId, properties) => >> val item = try { >> // Assume categories is optional property of item. >> Item(categories = properties.getOpt[List[String]]("categories")) >> } catch { >> case e: Exception => { >> logger.error(s"Failed to get properties ${properties} of" + >> s" item ${entityId}. Exception: ${e}.") >> throw e >> } >> } >> (entityId, item) >> }.cache() >> >> // get all "user" "view" "item" events >> val viewEventsRDD: RDD[ViewEvent] = eventsDb.find( >> appId = dsp.appId, >> entityType = Some("user"), >> eventNames = Some(List("like")), >> // targetEntityType is optional field of an event. >> targetEntityType = Some(Some("item")))(sc) >> // eventsDb.find() returns RDD[Event] >> .map { event => >> val viewEvent = try { >> event.event match { >> case "like" => ViewEvent( >> user = event.entityId, >> item = event.targetEntityId.get, >> t = event.eventTime.getMillis) >> case _ => throw new Exception(s"Unexpected event ${event} is >> read.") >> } >> } catch { >> case e: Exception => { >> logger.error(s"Cannot convert ${event} to ViewEvent." + >> s" Exception: ${e}.") >> throw e >> } >> } >> viewEvent >> }.cache() >> >> new TrainingData( >> users = usersRDD, >> items = itemsRDD, >> viewEvents = viewEventsRDD >> ) >> } >> } >> >> case class User() >> >> case class Item(categories: Option[List[String]]) >> >> case class ViewEvent(user: String, item: String, t: Long) >> >> class TrainingData( >> val users: RDD[(String, User)], >> val items: RDD[(String, Item)], >> val viewEvents: RDD[ViewEvent] >> ) extends Serializable { >> override def toString = { >> s"users: [${users.count()} (${users.take(2).toList}...)]" + >> s"items: [${items.count()} (${items.take(2).toList}...)]" + >> s"viewEvents: [${viewEvents.count()}] (${viewEvents.take(2).toList}. >> ..)" >> } >> } >> >> >> >> >> while training, it gives me following error, >> >> >> >> >> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$ >> pio train >> SLF4J: Class path contains multiple SLF4J bindings. >> SLF4J: Found binding in [jar:file:/home/darshan/Predic >> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja >> r!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in [jar:file:/home/darshan/Predic >> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp >> l/StaticLoggerBinder.class] >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> explanation. >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> [WARN] [Template$] /home/darshan/PredictionIO/tap >> ster-episode-similar/src/main/scala/template.json does not exist. >> Template metadata will not be available. (This is safe to ignore if you are >> not working on a template.) >> Exception in thread "main" java.io.FileNotFoundException: >> /home/darshan/PredictionIO/tapster-episode-similar/src/main/scala/engine.json >> (No such file or directory) >> at java.io.FileInputStream.open0(Native Method) >> at java.io.FileInputStream.open(FileInputStream.java:195) >> at java.io.FileInputStream.<init>(FileInputStream.java:138) >> at scala.io.Source$.fromFile(Source.scala:90) >> at scala.io.Source$.fromFile(Source.scala:75) >> at org.apache.predictionio.tools.console.Console$.getEngineInfo >> (Console.scala:724) >> at org.apache.predictionio.tools.RunWorkflow$.runWorkflow(RunWo >> rkflow.scala:54) >> at org.apache.predictionio.tools.commands.Engine$.train(Engine. >> scala:186) >> at org.apache.predictionio.tools.console.Pio$.train(Pio.scala:85) >> at org.apache.predictionio.tools.console.Console$$anonfun$main$ >> 1.apply(Console.scala:626) >> at org.apache.predictionio.tools.console.Console$$anonfun$main$ >> 1.apply(Console.scala:611) >> at scala.Option.map(Option.scala:145) >> at org.apache.predictionio.tools.console.Console$.main(Console. >> scala:611) >> at org.apache.predictionio.tools.console.Console.main(Console.scala) >> >> >> >> i tried almost all the things, but could not find the proper solution. >> please help me... >> i know you may feel bit odd about this mail, but in need of your help. >> >> thanks, >> >> >> -DAN >> >
