pabloem commented on a change in pull request #13217:
URL: https://github.com/apache/beam/pull/13217#discussion_r525289585
##########
File path: sdks/python/apache_beam/internal/metrics/metric.py
##########
@@ -136,3 +165,64 @@ def log_metrics(self, reset_after_logging=False):
self._last_logging_millis = current_millis
finally:
self._lock.release()
+
+
+class ServiceCallMetric(object):
+ """Metric class which records Service API call metrics.
+
+ This class will capture a request count metric for the specified
+ request_count_urn and base_labels.
+
+ When call() is invoked the status must be provided, which will
+ be converted to a canonical GCP status code, if possible.
+
+ TODO(ajamato): Add Request latency metric.
+ """
+ def __init__(self, request_count_urn, base_labels=None):
+ # type: (str, Optional[Dict[str, str]]) -> None
Review comment:
fyi - from now on, you can use type hints directly, not as comments (no
need to change anything atm)
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -566,10 +568,30 @@ def _insert_all_rows(
skipInvalidRows=skip_invalid_rows,
# TODO(silviuc): Should have an option for ignoreUnknownValues?
rows=rows))
+
+ resource = '//bigquery.googleapis.com/projects/%s/datasets/%s/tables/%s' %
(
+ project_id, dataset_id, table_id)
+
+ labels = {
+ # TODO(ajamato): Add Ptransform label.
+ monitoring_infos.SERVICE_LABEL: 'BigQuery',
+ monitoring_infos.METHOD_LABEL: 'BigQueryBatchWrite',
Review comment:
The method here would be a BatchWrite? Or a Streaming Insert maybe?
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -566,10 +568,30 @@ def _insert_all_rows(
skipInvalidRows=skip_invalid_rows,
# TODO(silviuc): Should have an option for ignoreUnknownValues?
rows=rows))
+
+ resource = '//bigquery.googleapis.com/projects/%s/datasets/%s/tables/%s' %
(
+ project_id, dataset_id, table_id)
Review comment:
Maybe add the resource calculation to a helper method? So it can be
reused later? (I imagine you'll add these metrics for other methods?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
