[
https://issues.apache.org/jira/browse/BEAM-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Webb updated BEAM-10268:
-----------------------------
Resolution: Won't Fix
Status: Resolved (was: Triage Needed)
old issue - resolving
> AssertionError('Missing boto3 requirement')
> -------------------------------------------
>
> Key: BEAM-10268
> URL: https://issues.apache.org/jira/browse/BEAM-10268
> Project: Beam
> Issue Type: Bug
> Components: build-system, sdk-py-core
> Reporter: Nikhil
> Priority: P3
> Labels: newbie
>
> *Error as:*
> apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions
> \{'s3://xxxxxx.csv': BeamIOError("exists() operation failed with exceptions
> {'s3://xxxxxx.csv': AssertionError('Missing boto3 requirement')}")}
>
> *Agenda:*
> I am writing a dataflow pipeline in python to import data from s3 bucket. I
> am running this code as dataflow runner.
> Could you please guide where I am going wrong?
>
> *I have written code as:*
> *class CustomPipelineOptions(PipelineOptions):*
> @classmethod
> def _add_argparse_args(cls, parser):
> parser.add_value_provider_argument('--input', help='Path of the file to
> read from')
> parser.add_value_provider_argument('--output',help='Output file to write
> results to.')
> parser.add_value_provider_argument('--source_bucket',help='AWS source
> bucket name.')
> parser.add_value_provider_argument('--aws_access_key_id',help='AWS access
> key id.')
> parser.add_value_provider_argument('--aws_secret_access_key',help='Your AWS
> secret access key.')
> parser.add_value_provider_argument('--aws_default_region',help='Sets STS
> endpoint resolution logic.')
>
> *if __name__ == '__main__':*
> parser = argparse.ArgumentParser()
> # Parse arguments from the command line.
> pipeline_args = parser.parse_known_args(argv)
> pipeline_options = PipelineOptions(pipeline_args)
> pipeline_options.view_as(SetupOptions).save_main_session = True
> p = beam.Pipeline(options=pipeline_options)
>
> p1 = (p | 'Read assetdb' >> beam.io.ReadFromText(custom_options.input))
> p.run().wait_until_finish()
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
