PIO uses Postgres, MySQL or other JDBC database from the SQL DBs or (and I
always use this) HBase. Hbase is a high performance NoSQL DB that scales
indefinitely.
It is possible to use any DB if you write an EventStore class for it, wrapping
the DB calls with a virtualization API that is DB independent.
Memory is completely algorithm and data dependent but expect PIO, which uses
Sparkm which in turn gets it’s speed from keeping data in-memory, to use a lot
compared to a web server. PIO apps are often in the big data category and many
deployments require Spark clusters with many G per machine. It is rare to be
able to run PIO in production on a single machine.
Welcome to big data.
On Jan 11, 2018, at 6:23 PM, Rajesh Jangid <raje...@grazitti.com> wrote:
Hi,
Well with version PIO 10 I think some dependency is causing trouble in
linux, we have figured out a way using Pio for now, and everything is working
great.
Thanks for the support though.
Few question-
1.Does Pio latest support Mongodb or NoSQL?
2.Memory uses by Pio, Is there any max memory limit set, If need be can it be
set?
Thanks
Rajesh
On Jan 11, 2018 10:25 PM, "Pat Ferrel" <p...@occamsmachete.com
<mailto:p...@occamsmachete.com>> wrote:
The version in the artifact built by Scala should only have the major version
number so 2.10 or 2.11. PIO 0.10.0 needs 2.10. Where, and what variable did
you set to 2.10.4? That is the problem. There will never be a lib built for
2.10.4, it will always be 2.10.
On Jan 11, 2018, at 5:15 AM, Daniel O' Shaughnessy <danieljamesda...@gmail.com
<mailto:danieljamesda...@gmail.com>> wrote:
Basically you need to make sure all your lib dependencies in build.sbt work
together.
On Thu, 11 Jan 2018 at 13:14 Daniel O' Shaughnessy <danieljamesda...@gmail.com
<mailto:danieljamesda...@gmail.com>> wrote:
Maybe try v2.10.4 based on this line:
[INFO] [Console$] [error] com.chuusai:shapeless _2.10, _2.10.4
I'm unfamiliar with the ubuntu setup for pio so can't help you there I'm afraid.
On Thu, 11 Jan 2018 at 05:08 Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
I am trying to run this on ubuntu 16.04
On Thu, Jan 11, 2018 at 10:36 AM, Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
Hi,
I have tried once again with 2.10 as well but getting following dependency
error
[INFO] [Console$] [error] Modules were resolved with conflicting cross-version
suffixes in
{file:/home/integration/client/PredictionIO-0.10/Engines/MyRecommendation/}myrecommendation:
[INFO] [Console$] [error] com.chuusai:shapeless _2.10, _2.10.4
[INFO] [Console$] java.lang.RuntimeException: Conflicting cross-version
suffixes in: com.chuusai:shapeless
[INFO] [Console$] at scala.sys.package$.error(package.scala:27)
[INFO] [Console$] at
sbt.ConflictWarning$.processCrossVersioned(ConflictWarning.scala:46)
[INFO] [Console$] at sbt.ConflictWarning$.apply(ConflictWarning.scala:32)
[INFO] [Console$] at sbt.Classpaths$$anonfun$100.apply(Defaults.scala:1300)
[INFO] [Console$] at sbt.Classpaths$$anonfun$100.apply(Defaults.scala:1297)
[INFO] [Console$] at
scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
[INFO] [Console$] at
sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
[INFO] [Console$] at sbt.std.Transform$$anon$4.work
<http://4.work/>(System.scala:63)
[INFO] [Console$] at
sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[INFO] [Console$] at
sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[INFO] [Console$] at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[INFO] [Console$] at sbt.Execute.work
<http://sbt.execute.work/>(Execute.scala:237)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[INFO] [Console$] at
sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[INFO] [Console$] at
sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[INFO] [Console$] at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[INFO] [Console$] at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO] [Console$] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO] [Console$] at java.lang.Thread.run(Thread.java:745)
[INFO] [Console$] [error] (*:update) Conflicting cross-version suffixes in:
com.chuusai:shapeless
[INFO] [Console$] [error] Total time: 6 s, completed Jan 11, 2018 5:03:51 AM
[ERROR] [Console$] Return code of previous step is 1. Aborting.
On Wed, Jan 10, 2018 at 10:03 PM, Daniel O' Shaughnessy
<danieljamesda...@gmail.com <mailto:danieljamesda...@gmail.com>> wrote:
I've pulled down this version without any modifications and run with pio v0.10
on a mac and it builds with no issues.
However, when I add in scalaVersion := "2.11.8" to build.sbt I get a dependency
error.
pio v0.10 supports scala 2.10 so you need to switch to this to run!
On Wed, 10 Jan 2018 at 13:47 Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
Yes, v0.5.0
On Jan 10, 2018 7:07 PM, "Daniel O' Shaughnessy" <danieljamesda...@gmail.com
<mailto:danieljamesda...@gmail.com>> wrote:
Is this the template you're using?
https://github.com/apache/predictionio-template-ecom-recommender
<https://github.com/apache/predictionio-template-ecom-recommender>
On Wed, 10 Jan 2018 at 13:16 Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
Yes,
We have dependency with elastic and we have elastic 1.4.4 already running.
We Do not want to run another elastic instance.
Latest prediction IO does not support elastic 1.4.4
On Wed, Jan 10, 2018 at 6:25 PM, Daniel O' Shaughnessy
<danieljamesda...@gmail.com <mailto:danieljamesda...@gmail.com>> wrote:
Strange....do you absolutely need to run this with pio v0.10?
On Wed, 10 Jan 2018 at 12:50 Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
{"pio": {"version": { "min": "0.10.0-incubating" }}}
On Wed, Jan 10, 2018 at 6:16 PM, Daniel O' Shaughnessy
<danieljamesda...@gmail.com <mailto:danieljamesda...@gmail.com>> wrote:
OK that looks fine. What version is PredictionIO set to in template.json?
On Wed, 10 Jan 2018 at 12:38 Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
pwd output :-
/home/integration/client/PredictionIO-0.10/Engines/Recommender
ls output :-
build.sbt data engine.json importdata LICENSE.txt manifest.json pio.log
project README.md src target template.json
Following command is used to build the template:-
/home/integration/client/PredictionIO-0.10/predictionio-0.10.0-incubating/PredictionIO-0.10.0-incubating/bin/pio
build --verbose
On Wed, Jan 10, 2018 at 5:39 PM, Daniel O' Shaughnessy
<danieljamesda...@gmail.com <mailto:danieljamesda...@gmail.com>> wrote:
wherever you're starting your engine can you type pwd and report the result
back?
On Wed, 10 Jan 2018 at 11:40 Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
Hi,
I have tried different template as well but still unable to get it to work,
Getting same error log.
On Wed, Jan 10, 2018 at 2:52 PM, Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
Yes, I have modified engine.js file with app name and then executed command in
same folder.
On Wed, Jan 10, 2018 at 2:21 PM, Noelia Osés Fernández <no...@vicomtech.org
<mailto:no...@vicomtech.org>> wrote:
I've gotten this error when I've run pio build in a folder with no engine.json
file.
Have you checked you are running the command in the right folder? i.e. a folder
that has an engine.json file...
Regards,
Noelia
On 10 January 2018 at 07:34, Rajesh Jangid <raje...@grazitti.com
<mailto:raje...@grazitti.com>> wrote:
Hi,
I have been trying to setup prediction IO's ecommerce recommendation
template. I was able to test it on the latest version of prediction IO but for
some reason I needed to use 0.10 release of prediction IO, the template dosn't
seem to work with this release. Following is the verbose output while
building template ->
user@linux:/MyRecommendation#
../../predictionio-0.10.0-incubating/PredictionIO-0.10.0-incubating/bin/pio
build --verbose
[INFO] [Console$] Using existing engine manifest JSON at
/home/integration/client/PredictionIO-0.10/Engines/MyRecommendation/manifest.json
[INFO] [Console$] Using command
'/home/integration/client/PredictionIO-0.10/predictionio-0.10.0-incubating/PredictionIO-0.10.0-incubating/sbt/sbt'
at the current working directory to build.
[INFO] [Console$] If the path above is incorrect, this process will fail.
[INFO] [Console$] Uber JAR disabled. Making sure
lib/pio-assembly-0.10.0-incubating.jar is absent.
[INFO] [Console$] Going to run:
/home/integration/client/PredictionIO-0.10/predictionio-0.10.0-incubating/PredictionIO-0.10.0-incubating/sbt/sbt
package assemblyPackageDependency
[INFO] [Console$] [info] Loading project definition from
/home/integration/client/PredictionIO-0.10/Engines/MyRecommendation/project
[INFO] [Console$] [info] Set current project to
template-scala-parallel-ecommercerecommendation (in build
file:/home/integration/client/PredictionIO-0.10/Engines/MyRecommendation/)
[INFO] [Console$] [success] Total time: 0 s, completed Jan 10, 2018 6:04:02 AM
[INFO] [Console$] [info] Including from cache: scala-library-2.11.8.jar
[INFO] [Console$] [info] Checking every *.class/*.jar file's SHA-1.
[INFO] [Console$] [info] Merging files...
[INFO] [Console$] [warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[INFO] [Console$] [warn] Strategy 'discard' was applied to a file
[INFO] [Console$] [info] Assembly up to date:
/home/integration/client/PredictionIO-0.10/Engines/MyRecommendation/target/scala-2.11/template-scala-parallel-ecommercerecommendation-assembly-0.1-SNAPSHOT-deps.jar
[INFO] [Console$] [success] Total time: 1 s, completed Jan 10, 2018 6:04:03 AM
[INFO] [Console$] Build finished successfully.
[INFO] [Console$] Looking for an engine...
[ERROR] [Console$] No engine found. Your build might have failed. Aborting.
Versions used:
scala 2.11.8
spark 1.5.1
elastic 1.4.4
hbase 1.0.0
pio event server is running fine here is log
[INFO] [HBLEvents] The table pio_event:events_0 doesn't exist yet. Creating
now...
[INFO] [HBLEvents] Removing table pio_event:events_0...
[INFO] [Console$] (sleeping 5 seconds for all messages to show up...)
[INFO] [Console$] Your system is all ready to go.
1. Can somebody please help regarding the issue here?
2. Also Can you recommend or point me to documentation to build template for
older version of predictionIO?
