No filesystem found for scheme hdfs

Mon, 31 May 2021 02:41:01 -0700 

Hi

I am using Spark-runner, and when I am using Apache Beam TextIO to read a file 
from HDFS:

.apply(TextIO.read().from("hdfs://path-to-file")

I get:

Caused by: java.lang.IllegalArgumentException: No filesystem found for scheme 
hdfs
     at 
org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(FileSystems.java:463)
     at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:120)
     at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:141)
     at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:153)
     at org.apache.beam.sdk.io.FileBasedSource.split(FileBasedSource.java:262)
     at 
org.apache.beam.runners.spark.io.SourceRDD$Bounded.getPartitions(SourceRDD.java:115)


What is missing to read files from hdfs ?


Noam Gershi
ICG Technology - TLV Lab

   [http://www.citigroup.com/emeaemailresources/gra30973_EmailSignature.jpg]

Reply via email to