rohan472000 commented on issue #31058:
URL: https://github.com/apache/airflow/issues/31058#issuecomment-1534858099
You could use the `gcloud` command or direct `Dataproc` APIs to create the
cluster with the `ShieldedInstanceConfig` properties until this feature is
added to the Airflow operator.
Here is how you can create a cluster with `ShieldedInstanceConfig` using the
`gcloud` command:
gcloud dataproc clusters create my-cluster \
--region=us-central1 \
--subnet=default \
--zone=us-central1-a \
--master-machine-type=n1-standard-2 \
--worker-machine-type=n1-standard-2 \
--num-workers=2 \
--image-version=1.5-debian10 \
--project=my-project \
--shielded-instance-config enable-secure-boot enable-vtpm
enable-integrity-monitoring
And here is how you can create a cluster with `ShieldedInstanceConfig` using
the `Dataproc` APIs:
from google.cloud import dataproc_v1 as dataproc
from google.protobuf import json_format
project_id = 'my-project'
region = 'us-central1'
cluster_name = 'my-cluster'
client = dataproc.ClusterControllerClient()
cluster = {
'project_id': project_id,
'cluster_name': cluster_name,
'config': {
'gce_cluster_config': {
'metadata': {
'enable-pepperdata': 'false',
'ha_flag': 'no'
},
'shielded_instance_config': {
'enable_secure_boot': True,
'enable_vtpm': True,
'enable_integrity_monitoring': True
}
}
}
}
op = client.create_cluster(
request={'project_id': project_id, 'region': region, 'cluster':
cluster}
)
print(json_format.MessageToDict(op))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
