Is 'hadoop' / 'hdfs' command accessible to your python script ?
If so, you can call 'hdfs dfs -ls' from python.
Cheers
On Sat, Jan 23, 2016 at 4:08 AM, Andrew Holway <
andrew.hol...@otternetworks.de> wrote:
> Hello,
>
> I would like to make a list of files (parquet or json) in a specific
> HDFS directory with python so I can do some logic on which files to
> load into a dataframe.
>
> Any ideas?
>
> Thanks,
>
> Andrew
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
