hello,every
I'm running a Beam pipeline which uses the TextIO read same text from
text file, PTransform peer line search hbase. result is
PCollection<PCollection<KV<String, RecordData>>>
@Override
public PCollection<PCollection<KV<String, RecordData>>>
expand(PCollection<String> lines) {
PCollection<PCollection<KV<String, RecordData>>> results = lines
.apply(ParDo.of(new DoFn<String, PCollection<KV<String, RecordData>>>() {
@ProcessElement
public void processElement(ProcessContext c) {
String vin = c.element();
Pipeline pipelineHbase = Pipeline.create(c.getPipelineOptions());
HBaseIO.Read read =
HBaseIO.read().withConfiguration(conf).withTableId(hbaseTable).withKeyRange(
Bytes.toBytes(String.format("%s-%s", vin, startTime)),
Bytes.toBytes(String.format("%s-%s", vin, endTime)));
PCollection<Result> results = pipelineHbase.apply(read);
PCollection<KV<String, RecordData>> recordResults = results
.apply(ParDo.of(new Result2RecordNoModifyDataFn()));
c.output(recordResults);
}
}));
return results;
}
what process PCollection<PCollection<KV<String, RecordData>>> ????
