[ https://issues.apache.org/jira/browse/SPARK-29047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949182#comment-16949182 ]
Hyukjin Kwon commented on SPARK-29047: -------------------------------------- Thanks for your response, [~我心凉凉的]. > use spark-submit Not a file: hdfs:// > -------------------------------------- > > Key: SPARK-29047 > URL: https://issues.apache.org/jira/browse/SPARK-29047 > Project: Spark > Issue Type: Question > Components: Deploy, Spark Submit > Affects Versions: 2.1.3 > Environment: spark1.3 ;hadoop2.6 > Reporter: bailin > Priority: Critical > Attachments: hadfs file is already existed.png > > > when i submit a spark application : > {code:java} > /spark-submit --class com.yto.log.SparkStatCleanJobYARN --name > TopNStatJobYARN --master yarn --executor-memory 1G --num-executors 1 > /home/hadoop/lib/sql-1.0-jar-with-dependencies.jar > hdfs://hadoop001:8020/clean 20161110{code} > the hadfs file is already existed,but not recognition > > {code:java} > 19/09/11 14:18:50 INFO mapred.FileInputFormat: Total input paths to process : > 219/09/11 14:18:50 INFO mapred.FileInputFormat: Total input paths to process > : 219/09/11 14:18:50 ERROR datasources.FileFormatWriter: Aborting job > null.java.io.IOException: Not a file: hdfs://hadoop001:8020/clean/day= at > org.apache.hadoop.mapred.FileInputFormat.getSplits > {code} > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org