Hi Community, Is it possible to create /write HDFS dirs./folders every 15 mins?
Right now, we use /hdfs_sink_path/%Y/%m/%d/%H --- ie using every Hour, But, Is it possible to write every 15 mins? - not every minute... If this is not possible out of the box config, can DEV team please advise which code/java-file do we need to customize so Flume creates dirs. Every 15 mins? Like /%Y/%m/%d/%H/%15M ? Please help us, thanks. Here is our present 1 hour HDFS dir/folder creation config: airwave.sinks.sink1.type = hdfs airwave.sinks.sink1.channel = channel1 airwave.sinks.sink1.hdfs.path = /prod/hadoop/smallsite/airwave_flume_ingest_hak_3_amp_1/%Y/%m/%d/%H airwave.sinks.sink1.hdfs.fileType = DataStream airwave.sinks.sink1.hdfs.writeFormat = Text airwave.sinks.sink1.hdfs.filePrefix = Airwave_amp_1 airwave.sinks.sink1.hdfs.useLocalTimeStamp = true airwave.sinks.sink1.hdfs.round = true airwave.sinks.sink1.hdfs.roundValue = 1 airwave.sinks.sink1.hdfs.roundUnit = hour airwave.sinks.sink1.hdfs.callTimeout = 10000000000
