You can use RDD.wholeTextFiles(). For example, suppose all your files are under /tmp/ABC_input/,
val rdd = sc.wholeTextFiles("file:///tmp/ABC_input”) val rdd1 = rdd.flatMap { case (path, content) => val fileName = new java.io.File(path).getName content.split("\n").map { line => (line, fileName) } } val df = sqlContext.createDataFrame(rdd1).toDF("line", "file") > On Jun 2, 2016, at 03:13, Vikash Kumar <vikashsp...@gmail.com> wrote: > > 100,abc,299 > 200,xyz,499