Vira Vitanska created DLAB-89:
---------------------------------
Summary: [AWS]: Improve method to receive prices of spot instances
Key: DLAB-89
URL: https://issues.apache.org/jira/browse/DLAB-89
Project: Apache DLab
Issue Type: Bug
Reporter: Vira Vitanska
*Preconditions:*
1. Jupyter or Appache Zeppelin or Rstudio is created
*Steps to reproduce:*
1. Create EMR with spot
*Actual result:*
EMR creation fails
*Expected result:*
EMR is created
Currently method _*get_ec2_price*_ uses downloading of big json file (~484 Mb)
during creating EMR cluster with sport instances.
Python required ~ _*3.2Gb + 484*_ Mb on SSN node for 1 launch, which can cause
memory error on small shapes (like _*t2.medium*_).
Suggestion: rewrite method using python client to avoid such resource
(memory/cpu) overheads.
Links:
[https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeSpotPriceHistory.html]
[https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_spot_price_history]
[https://aws.amazon.com/blogs/aws/new-aws-price-list-api/]
[https://aws.amazon.com/blogs/aws/aws-price-list-api-update-new-query-and-metadata-functions/]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
