Re: Reading files from multiple subdirectories

2020-06-11 Thread Yun Gao
Hi Lorenzo, Read from a previouse thread [1] and the source code, I think you may set inputFormat.setNestedFileEnumeration(true) to also scan the nested files. Best, Yun [1]

Reading files from multiple subdirectories

2020-06-11 Thread Lorenzo Nicora
Hi, related to the same case I am discussing in another thread, but not related to AVRO this time :) I need to ingest files a S3 Sink Kafka Connector periodically adds to an S3 bucket. Files are bucketed by date time as it often happens. Is there any way, using Flink only, to monitor a