Dear Spark users,
I am reading a set of json files to compile them to Parquet data format.
I am willing to mark the folders in some way after having read their
contents so that I do not read it again(e.g. I can changed the name of the
folder).
I use .textFile("path/to*/dir/*/*/*.js") *technique to* automatically
*detect
the files.
I cannot however, use the same notation* to rename them.*
Could you suggest how I can *get the names of these folders* so that I can
rename them using native hadoop libraries.
I am using Apache Spark 1.4.1
I look forward to hearing suggestions!!
yours,
Narek
Նարեկ Գալստյան
