Hello! Generally, for big data, line number (or similar "sequence id") is an antipattern.
Knowing any line number implies knowing the previous line... and even this simple relationship breaks the ability to process elements in parallel. It *should* be possible to efficiently know the starting byte position (not row number) of an element in a file, but none of the file format IOs (TextIO, AvroIO) expose this for now. Would that be an alternative to your use case? If your files are small enough to be processed in one task, you can use https://beam.apache.org/releases/javadoc/2.16.0/org/apache/beam/sdk/io/FileIO.html#readMatches-- and stream through the file yourself while counting rows. Best regards, Ryan On Wed, Nov 6, 2019 at 7:57 AM Gershi, Noam <[email protected]> wrote: > Hi, > > > > How can I read a file and having in transformation the line number > information? > > > > > > > > [image: citi_logo_mail][image: citi_logo_mail]*Noam Gershi* > > Software Developer > > *T*: +972 (3) 7405718 > > [image: Mail_signature_blue] > > >
