Hi,
I'm playing with Beam pipeline. My goal is to merge two big files.
So I have source (one of two) file like:
column_one|colum_two
0099"|"0080199111"
...
My trivial pipeline is:
Beam File Input => Text file output
I created definition for Beam File Input: separator "|", column_one - string,
column_two - string
But in I get in result (Text file output):
column_one|colum_two
0|0|9|9|"|"|0|0|8|0|1|9|9|1|1|1|"
...
Why each character is separated by "|"?
I also get 51 result files. Even if I set 'Number of workers : 3' in Pipeline
Run Configuration for engine 'Beam Direct pipeline engine'
Also this source file is really big and building definition is quite time
consuming process - would be great such options like in Text file input where
Hop detects fields and is able to preview it.
Best
