Michal-Nguyen-airspace-intelligence commented on issue #31040: URL: https://github.com/apache/beam/issues/31040#issuecomment-2072115105
So I double checked and there are differences between your example and our case. - We use content encoding `gzip` while saving our files to GCS, you don't have encoding specified - This leads us to using `ReadAllFromText` with parameter `compression_type=CompressionTypes.UNCOMPRESSED` since the downloaded file seems to be already uncompressed (it doesn't work with `CompressionTypes.AUTO`), as in gcs policy - This further results in reading only fragment of the file Furthermore, after removing encoding type from our file and using `CompressionTypes.AUTO` on it worked properly. To get you example to represent our situation please add content encoding `gzip` to your file metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
