Thanks JB.
I'm submitting from the AWS Spark master node, the spark-default.conf is
pre-deployed by Amazon (attached) and there is no setting
for spark.yarn.keytab. Is there any doc for setting this up if required in
this scenario ?
Also, I if deploy-mode is switched from cluster to client on spark-submit then
the error no longer appears. Just wondering if there's any difference to
using client versus cluster mode if the submit is being done on the master
node.
Thanks for any suggestions,
--
Nick
________________________________________
From: Jean-Baptiste Onofré <j...@nanthrax.net>
Sent: Friday, December 11, 2015 1:01 PM
To: user@spark.apache.org
Subject: Re: Spark Submit - java.lang.IllegalArgumentException: requirement
failed
Hi Nick,
the localizedPath has to be not null, that's why the requirement fails.
In the SparkConf used by the spark-submit (default in
conf/spark-default.conf), do you have all properties defined, especially
spark.yarn.keytab ?
Thanks,
Regards
JB
On 12/11/2015 05:49 PM, Afshartous, Nick wrote:
>
> Hi,
>
>
> I'm trying to run a streaming job on a single node EMR 4.1/Spark 1.5
> cluster. Its throwing an IllegalArgumentException right away on the submit.
>
> Attaching full output from console.
>
>
> Thanks for any insights.
>
> --
>
> Nick
>
>
>
> 15/12/11 16:44:43 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 15/12/11 16:44:43 INFO client.RMProxy: Connecting to ResourceManager at
> ip-10-247-129-50.ec2.internal/10.247.129.50:8032
> 15/12/11 16:44:43 INFO yarn.Client: Requesting a new application from
> cluster with 1 NodeManagers
> 15/12/11 16:44:43 INFO yarn.Client: Verifying our application has not
> requested more than the maximum memory capability of the cluster (54272
> MB per container)
> 15/12/11 16:44:43 INFO yarn.Client: Will allocate AM container, with
> 11264 MB memory including 1024 MB overhead
> 15/12/11 16:44:43 INFO yarn.Client: Setting up container launch context
> for our AM
> 15/12/11 16:44:43 INFO yarn.Client: Setting up the launch environment
> for our AM container
> 15/12/11 16:44:43 INFO yarn.Client: Preparing resources for our AM container
> 15/12/11 16:44:44 INFO yarn.Client: Uploading resource
> file:/usr/lib/spark/lib/spark-assembly-1.5.0-hadoop2.6.0-amzn-1.jar ->
> hdfs://ip-10-247-129-50.ec2.internal:8020/user/hadoop/.sparkStaging/application_1447\
> 442727308_0126/spark-assembly-1.5.0-hadoop2.6.0-amzn-1.jar
> 15/12/11 16:44:44 INFO metrics.MetricsSaver: MetricsConfigRecord
> disabledInCluster: false instanceEngineCycleSec: 60
> clusterEngineCycleSec: 60 disableClusterEngine: false maxMemoryMb: 3072
> maxInstanceCount: 500\
> lastModified: 1447442734295
> 15/12/11 16:44:44 INFO metrics.MetricsSaver: Created MetricsSaver
> j-2H3BTA60FGUYO:i-f7812947:SparkSubmit:15603 period:60
> /mnt/var/em/raw/i-f7812947_20151211_SparkSubmit_15603_raw.bin
> 15/12/11 16:44:45 INFO metrics.MetricsSaver: 1 aggregated HDFSWriteDelay
> 1276 raw values into 1 aggregated values, total 1
> 15/12/11 16:44:45 INFO yarn.Client: Uploading resource
> file:/home/hadoop/spark-pipeline-framework-1.1.6-SNAPSHOT/workflow/lib/spark-kafka-services-1.0.jar
> -> hdfs://ip-10-247-129-50.ec2.internal:8020/user/hadoo\
> p/.sparkStaging/application_1447442727308_0126/spark-kafka-services-1.0.jar
> 15/12/11 16:44:45 INFO yarn.Client: Uploading resource
> file:/home/hadoop/spark-pipeline-framework-1.1.6-SNAPSHOT/conf/AwsCredentials.properties
> -> hdfs://ip-10-247-129-50.ec2.internal:8020/user/hadoop/.sparkSta\
> ging/application_1447442727308_0126/AwsCredentials.properties
> 15/12/11 16:44:45 WARN yarn.Client: Resource
> file:/home/hadoop/spark-pipeline-framework-1.1.6-SNAPSHOT/conf/AwsCredentials.properties
> added multiple times to distributed cache.
> 15/12/11 16:44:45 INFO yarn.Client: Deleting staging directory
> .sparkStaging/application_1447442727308_0126
> Exception in thread "main" java.lang.IllegalArgumentException:
> requirement failed
> at scala.Predef$.require(Predef.scala:221)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6$$anonfun$apply$2.apply(Client.scala:392)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6$$anonfun$apply$2.apply(Client.scala:390)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:390)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:388)
> at scala.collection.immutable.List.foreach(List.scala:318)
> at
> org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:388)
> at
> org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:629)
> at
> org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:119)
> at org.apache.spark.deploy.yarn.Client.run(Client.scala:907)
> at org.apache.spark.deploy.yarn.Client$.main(Client.scala:966)
> at org.apache.spark.deploy.yarn.Client.main(Client.scala)
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
spark.master yarn
spark.driver.extraClassPath
/etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-mapreduce/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-lzo/lib/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/*:/usr/lib/spark/lib/RedshiftJDBC41-1.1.7.1007.jar
spark.driver.extraLibraryPath
/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native
spark.executor.extraClassPath
/etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-mapreduce/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-lzo/lib/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/*:/usr/lib/spark/lib/RedshiftJDBC41-1.1.7.1007.jar
spark.executor.extraLibraryPath
/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native
spark.eventLog.enabled true
spark.eventLog.dir hdfs:///var/log/spark/apps
spark.history.fs.logDirectory hdfs:///var/log/spark/apps
spark.yarn.historyServer.address ip-10-247-129-50.ec2.internal:18080
spark.history.ui.port 18080
spark.shuffle.service.enabled true
spark.driver.extraJavaOptions
-Dlog4j.configuration=file:///etc/spark/conf/log4j.properties
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70
-XX:MaxHeapFreeRatio=70 -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=512M
-XX:OnOutOfMemoryError='kill -9 %p'
spark.executor.extraJavaOptions -verbose:gc -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70
-XX:+CMSClassUnloadingEnabled -XX:OnOutOfMemoryError='kill -9 %p'
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
