steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-866694906
One thought here: would you ever want the s3a connector to fall back to that bundled region lookup sequence? I'm wondering in particular if it makes a difference in routing/billing on EC2 deployments? As of Hadoop 3.3.1 if region=null, endpoint=null the Ec2 metadata is used to provide the region info (this is new). With this patch. if endpoint = null we switch to saying region = us-east-1 will that do bad things for signing/routing HTTP connections in an EC2 deployment in different regions? For example, if I am running in AWS ireland, will this cause requests to go to us-east-1, even if they then end up redirected back to eu-west-1. this could mean connections are slower to set up, risk of remote data transfer and billing (though the redirections should fix that, right?), and if the rules for a deployment prevent out-of-region network traffic, will this break. I think we have hit problems related to this in the past. put differently: is anything special happening with the default "null" endpoint and Ec2 metadata region name provision which we need to know about and support? If so, we could allow the region to be set to "" or maybe "ec2" and have that revert to the resolve chain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
