dylangovender opened a new issue, #1821:
URL: https://github.com/apache/incubator-datalab/issues/1821
Hi,
I have been running a Datalab instance but am not seeing any data under the
billing section in the front end. AWS is definitely incurring costs as a result
of Datalab ($40 for the month of June).
Is there some specific condition that needs to be fulfilled before billing
is populated?
This is the command I used to create the Datalab:
/usr/bin/python3
~/incubator-datalab/infrastructure-provisioning/scripts/deploy_datalab.py \
--conf_service_base_name datalab-base-name \
--conf_tag_resource_id datalab-resource-id \
--conf_os_family debian \
--key_path /home/ubuntu/.ssh/ \
--conf_key_name datalab \
--action create \
--keycloak_realm_name master \
--keycloak_user XXXXXXXX \
--keycloak_user_password XXXXXXXX \
--keycloak_auth_server_url http:\/\/XX.XX.XXX.XX:8080 \
'aws' \
--aws_region eu-west-1 \
--aws_zone eu-west-1a \
--aws_ssn_instance_size t2.medium \
--aws_billing_bucket datalabbilling \
--aws_account_id XXXXXXXXXXX \
--aws_access_key XXXXXXXXXXXXXXX \
--aws_secret_access_key XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
An excerpt from the logs in /var/opt/datalab/log/ssn/billing.log:
2023-07-03 11:25:30.107 INFO 94517 ---
[cluster-ClusterId{value='64a2b029b4d0cc7135f64a6e',
description='null'}-localhost:27017] org.mongodb.driver.cluster :
Discovered cluster type of STANDALONE
2023-07-03 11:25:30.919 INFO 94517 --- [main]
o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8088
(https) with context path '/api/billing'
2023-07-03 11:25:30.922 INFO 94517 --- [main]
com.epam.datalab.BillingAwsApplication : Started BillingAwsApplication in
21.602 seconds (JVM running for 23.418)
2023-07-03 11:25:30.926 DEBUG 94517 --- [main]
com.epam.datalab.BillingServiceImpl : Billing report configuration file:
/opt/datalab/conf/billing.yml
INFO [2023-07-03 11:30:00,751]
org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/api/billing]:
Initializing Spring DispatcherServlet 'dispatcherServlet'
INFO [2023-07-03 11:30:00,751]
org.springframework.web.servlet.DispatcherServlet: Initializing Servlet
'dispatcherServlet'
INFO [2023-07-03 11:30:00,760]
org.springframework.web.servlet.DispatcherServlet: Completed initialization in
9 ms
DEBUG [2023-07-03 11:30:01,109] com.epam.datalab.module.aws.AdapterS3File:
Adapter S3 will be opened for READ
DEBUG [2023-07-03 11:30:02,317] com.epam.datalab.module.aws.AdapterS3File:
New report files in bucket folder datalabbilling not found
DEBUG [2023-07-03 11:30:02,318] com.epam.datalab.module.aws.AdapterS3File:
Adapter S3 has been opened
DEBUG [2023-07-03 11:30:02,318] com.epam.datalab.core.parser.ParserByLine:
Source data has multy entry true
DEBUG [2023-07-03 11:45:00,243] com.epam.datalab.module.aws.AdapterS3File:
Adapter S3 will be opened for READ
DEBUG [2023-07-03 11:45:00,332] com.epam.datalab.module.aws.AdapterS3File:
New report files in bucket folder datalabbilling not found
DEBUG [2023-07-03 11:45:00,332] com.epam.datalab.module.aws.AdapterS3File:
Adapter S3 has been opened
DEBUG [2023-07-03 11:45:00,332] com.epam.datalab.core.parser.ParserByLine:
Source data has multy entry true
Excerpt from the logs in /var/opt/datalab/log/ssn/selfservice.log:
INFO [2023-07-03 11:25:30,534]
org.eclipse.jetty.server.handler.ContextHandler: Started
i.d.j.MutableServletContextHandler@5a654e05{/,null,AVAILABLE}
INFO [2023-07-03 11:25:30,559] org.eclipse.jetty.server.AbstractConnector:
Started application@359e27d2{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
INFO [2023-07-03 11:25:30,565] org.eclipse.jetty.server.AbstractConnector:
Started admin@277bc3a5{SSL,[ssl, http/1.1]}{0.0.0.0:8444}
INFO [2023-07-03 11:25:30,565] org.eclipse.jetty.server.Server: Started
@23057ms
INFO [2023-07-03 11:25:30,616]
com.epam.datalab.backendapi.dropwizard.listeners.MongoStartupListener:
Populating DataLab default roles into database
INFO [2023-07-03 11:25:30,648]
com.epam.datalab.backendapi.dropwizard.listeners.MongoStartupListener: Check
for connected endpoints:
connected endpoints: 1
connected clouds: [AWS]
INFO [2023-07-03 11:30:00,075]
com.epam.datalab.backendapi.schedulers.CheckInfrastructureStatusScheduler:
Trying to update infrastructure statuses
INFO [2023-07-03 11:30:00,194]
com.epam.datalab.backendapi.service.impl.InfrastructureInfoServiceImpl:
EnvResources is empty: EnvResourceList{host=[], cluster=[]} , didn't send
request to provisioning service
INFO [2023-07-03 11:30:00,208]
com.epam.datalab.backendapi.schedulers.billing.BillingScheduler: Trying to
update billing
INFO [2023-07-03 11:30:02,580]
com.epam.datalab.backendapi.service.impl.BillingServiceImpl: Updating billing
information for endpoint local. Billing data []
INFO [2023-07-03 11:45:00,064]
com.epam.datalab.backendapi.schedulers.CheckInfrastructureStatusScheduler:
Trying to update infrastructure statuses
INFO [2023-07-03 11:45:00,094]
com.epam.datalab.backendapi.service.impl.InfrastructureInfoServiceImpl:
EnvResources is empty: EnvResourceList{host=[], cluster=[]} , didn't send
request to provisioning service
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
