Good evening,
I'm attempting to stream a large CSV file, split on each row, aggregrate the
rows in batches, and insert the batched rows into a database table. My only
issue is that I can't figure out how to determine when I've reached the end
of the file (and therefore the end of the aggregations and inserts). Is
there a ready means of detecting the end of the file (and if not, perhaps a
more effective route design than mine)?
Here's my route:
from("file:///Documents/tmp?fileName=TEST.dat")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
dao.truncateTable();
dao.disableUniqueKeyIndex();
}
}).split(body().tokenize()).streaming().aggregate(header("CamelFileName"),
new AggregationStrategy() {
@SuppressWarnings("unchecked")
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (oldExchange == null) {
oldExchange = newExchange.copy();
oldExchange.getIn().setBody(new ArrayList());
}
List<String> rows = oldExchange.getIn().getBody(List.class);
String row = newExchange.getIn().getBody(String.class);
rows.add(row);
return oldExchange;
}
}).completionSize(1000).completionTimeout(30000).process(new Processor() {
public void process(Exchange exchange) throws Exception {
List<String> rows = exchange.getIn().getBody(List.class);
dao.loadDatFileRows(rows);
}
// read pretend "Status" property
}).end().filter(property("STATUS").isEqualTo("FINISHED")).process(new
Processor() {
public void process(Exchange exchange) throws Exception {
dao.rebuildUniqueKeyIndex();
dao.updateStatistics();
}
});
Thanks in advance for any guidance.
--
View this message in context:
http://camel.465427.n5.nabble.com/How-to-determine-when-one-s-streamed-to-the-end-of-a-file-tp511784p511784.html
Sent from the Camel - Users mailing list archive at Nabble.com.
