Hi, I am trying to process some of our access logs using beam. Log files are archived to a GCS bucket, but they are lz compressed and encrypted using gpg.
Any idea how I could load up the files in to a pipeline without decrypting, decompressing and staging the file before feeding it to a beam pipeline? I see that I could write a custom coder, but I could not figure our the specifics. Thanks Sachin
