I'm lost as to how such direct manipulation of CLASSPATH is not appearing in the logged spark-submit command.
What could cause this!? I just pushed a version of the buildpack which should help debug. Assuming only a single buildpack is assigned to the app, here's how to set it: heroku buildpacks:set https://github.com/heroku/predictionio-buildpack#debug-custom-dist Then redeploy the engine an check the build log for the line: + PredictionIO ($URL) Please confirm that it is the URL of your custom PredictionIO dist. On Fri, Mar 9, 2018 at 2:47 PM, Shane Johnson <[email protected]> wrote: > Thanks Donald and Mars, > > I created a new distribution ( > <https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache-predictionio-0.12.0-incubating-bin.tar.gz> > https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache- > predictionio-0.12.0-incubating-bin.tar.gz) with the added CLASSPATH code > and pointed to the distribution with the PREDICTIONIO_DIST_URL variable > within the engine app in Heroku. > > CLASSPATH="/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar:$CLASSPATH" > echo "$CLASSPATH" > > It didn't seem to force the aws-java-sdk to load first as I reviewed the > release logs. Should the aws-java-sdk.jar show up as the first file within > the --jars section when this is added CLASSPATH="/app/PredictionIO-dist/ > lib/spark/aws-java-sdk.jar:$CLASSPATH". > > I'm still getting the NoSuchMethodError when the *aws-java-sdk.jar* loads > after the *pio-data-s3-assembly-0.12.0-incubating.jar**. *Do you have > other suggestions to try? I was also testing locally to change the order of > the --jars but changes to the compute-classpath.sh didn't seem to change > the order of the jars in the logs. > > Running train on release… > > Picked up JAVA_TOOL_OPTIONS: -Xmx12g -Dfile.encoding=UTF-8 > > [INFO] [Runner$] Submission command: > /app/PredictionIO-dist/vendors/spark-hadoop/bin/spark-submit --driver-memory > 13g --class org.apache.predictionio.workflow.CreateWorkflow --jars > file:/app/PredictionIO-dist/lib/postgresql_jdbc.jar,file:/app/target/scala-2.11/template-scala-parallel-liftscoring-assembly-0.1-SNAPSHOT-deps.jar,file:/app/target/scala-2.11/template-scala-parallel-liftscoring_2.11-0.1-SNAPSHOT.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-hdfs-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-localfs-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-elasticsearch-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/hadoop-aws.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-hbase-assembly-0.12.0-incubating.jar,*file:/app/PredictionIO-dist/lib/spark/pio-data-s3-assembly-0.12.0-incubating.jar*,file:/app/PredictionIO-dist/lib/spark/pio-data-jdbc-assembly-0.12.0-incubating.jar,*file:/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar* > --files > file:/app/PredictionIO-dist/conf/log4j.properties,file:/app/PredictionIO-dist/conf/core-site.xml > --driver-class-path > /app/PredictionIO-dist/conf:/app/PredictionIO-dist/conf:/app/PredictionIO-dist/lib/postgresql_jdbc.jar:/app/PredictionIO-dist/conf > --driver-java-options -Dpio.log.dir=/app > file:/app/PredictionIO-dist/lib/pio-assembly-0.12.0-incubating.jar > --engine-id org.template.liftscoring.LiftScoringEngine --engine-version > 0c35eebf403cf91fe77a64921d76aa1ca6411d20 --engine-variant > file:/app/engine.json --verbosity 0 --json-extractor Both --env > > > Error: > > Exception in thread "main" java.lang.NoSuchMethodError: > com.amazonaws.services.s3.transfer.TransferManager.<init>(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V > > at > org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:287) > > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669) > > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94) > > > > > > *Shane Johnson | LIFT IQ* > *Founder | CEO* > > *www.liftiq.com <http://www.liftiq.com/>* or *[email protected] > <[email protected]>* > mobile: (801) 360-3350 > LinkedIn <https://www.linkedin.com/in/shanewjohnson/> | Twitter > <https://twitter.com/SWaldenJ> | Facebook > <https://www.facebook.com/shane.johnson.71653> > > > > On Wed, Mar 7, 2018 at 1:01 PM, Mars Hall <[email protected]> > wrote: > >> Shane, >> >> On Wed, Mar 7, 2018 at 4:49 AM, Shane Johnson <[email protected]> wrote: >> >>> >>> Re: adding a line to ensure a jar is loaded first. Is this what you are >>> referring to...(line at the bottom in red)? >>> >> >> >> I believe the code would need to look like this to effect the output >> classpath as intended: >> >> >>> CLASSPATH="/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar:$CLASSPATH" >>> echo "$CLASSPATH" >>> >> >> >> aws-java-sdk.jar is already in the CLASSPATH though, So, the script will >> need to be skip or remove it first. >> >> -- >> *Mars Hall >> 415-818-7039 <(415)%20818-7039> >> Customer Facing Architect >> Salesforce Platform / Heroku >> San Francisco, California >> > > -- *Mars Hall 415-818-7039 Customer Facing Architect Salesforce Platform / Heroku San Francisco, California
