I'll give an obligatory mention to Google Cloud Dataflow <https://cloud.google.com/dataflow/> since you mention ETL, as this is the sort of thing it was designed for. It's worth taking a look at.
If you're just looking to manage compute engine instances programmatically, yes you can do this with the 'gcloud' commmand eg. from a shell script invoked from cron (see 'gcloud compute instances <https://cloud.google.com/sdk/gcloud/reference/compute/instances/>'). On Friday, July 1, 2016 at 4:59:36 PM UTC-4, Rafael Zubairov wrote: > > Hello! > > I need to run ETL like jobs in google compute engine. > The jobs itself compute intensive, and would require compute instances, > downloading data from network, processing and then storing the data into > google bigquery(other storage). > > Referring article ( > https://cloud.google.com/solutions/reliable-task-scheduling-compute-engine) > i need to run compute nodes all the time, though my load would happen only > once a day. > Can i use gcloud/API to spin on/off google compute instances on cron > trigger? > Are there any other possible ways of spinning several google compute nodes > at the specific time, and then turn them off? > > Thanks! > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/91acff9a-a4d5-4d67-843a-3ce486f72f34%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
