[
https://issues.apache.org/jira/browse/BEAM-9094?focusedWorklogId=504312&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504312
]
ASF GitHub Bot logged work on BEAM-9094:
----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Oct/20 20:15
Start Date: 23/Oct/20 20:15
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #13180:
URL: https://github.com/apache/beam/pull/13180#discussion_r511134918
##########
File path: sdks/python/apache_beam/options/pipeline_options.py
##########
@@ -1321,3 +1320,39 @@ def augment_options(cls, options):
for name, value in override.items():
setattr(options, name, value)
return options
+
+
+class S3Options(PipelineOptions):
+ @classmethod
+ def _add_argparse_args(cls, parser):
+ # These options are passed to the S3 IO Client
+ parser.add_argument(
+ '--aws_access_key_id',
+ default=None,
+ help='The secret key to use when creating the s3 client.')
+ parser.add_argument(
+ '--aws_secret_access_key',
+ default=None,
+ help='The secret key to use when creating the s3 client.')
+ parser.add_argument(
+ '--aws_session_token',
+ default=None,
+ help='The session token to use when creating the s3 client.')
+ parser.add_argument(
+ '--endpoint_url',
+ default=None,
+ help='The complete URL to use for the constructed s3 client.')
+ parser.add_argument(
+ '--region_name',
+ default=None,
+ help='The name of the region associated with the client.')
+ parser.add_argument(
+ '--api_version', default=None, help='The API version to use.')
+ parser.add_argument(
+ '--verify',
+ default=None,
+ help='Whether or not to verify SSL certificates.')
+ parser.add_argument(
+ '--use_ssl',
+ default=True,
+ help='Whether or not to use SSL. By default, SSL is used.')
Review comment:
+1. Please do not use endpoint_url, region_name, api_version, verify,
use_ssl and so on without a prefix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 504312)
Time Spent: 3h 20m (was: 3h 10m)
> Support setting some options such as endpoint_url and credential infos for
> AWS S3 Filesystem in Python SDKs
> -----------------------------------------------------------------------------------------------------------
>
> Key: BEAM-9094
> URL: https://issues.apache.org/jira/browse/BEAM-9094
> Project: Beam
> Issue Type: Improvement
> Components: io-ideas
> Affects Versions: 2.19.0
> Reporter: Keunhyun Oh
> Priority: P3
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> AWS S3 File System is implemented in BEAM-2572.
> To use local s3 like minio, It is need to support setting some options such
> as endpoint_url and credential infos.
> In my idea, it can be implemented to use environment variables.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
