[ 
https://issues.apache.org/jira/browse/BEAM-7528?focusedWorklogId=295809&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-295809
 ]

ASF GitHub Bot logged work on BEAM-7528:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 15/Aug/19 22:29
            Start Date: 15/Aug/19 22:29
    Worklog Time Spent: 10m 
      Work Description: pabloem commented on pull request #8941: [BEAM-7528] 
Save load test metrics according to distribution name
URL: https://github.com/apache/beam/pull/8941#discussion_r314523476
 
 

 ##########
 File path: 
sdks/python/apache_beam/testing/load_tests/load_test_metrics_utils.py
 ##########
 @@ -75,17 +73,114 @@
 ]
 
 
-def get_element_by_schema(schema_name, insert_list):
-  for element in insert_list:
-    if element['label'] == schema_name:
-      return element['value']
+def parse_step(step_name):
+  """Replaces white spaces and removes 'Step:' label
+
+  Args:
+    step_name(str): step name passed in metric ParDo
+
+  Returns:
+    lower case step name without namespace and step label
+  """
+  return step_name.lower().replace(' ', '_').strip('step:_')
+
+
+def split_metrics_by_namespace_and_name(metrics, namespace, name):
+  """Splits metrics list namespace and name.
+
+  Args:
+    metrics: list of metrics from pipeline result
+    namespace(str): filter metrics by namespace
+    name(str): filter metrics by name
+
+  Returns:
+    two lists - one of metrics which are matching filters
+    and second of not matching
+  """
+  matching_metrics = []
+  not_matching_metrics = []
+  for dist in metrics:
+    if dist.key.metric.namespace == namespace\
+        and dist.key.metric.name == name:
+      matching_metrics.append(dist)
+    else:
+      not_matching_metrics.append(dist)
+  return matching_metrics, not_matching_metrics
+
+
+def get_generic_distributions(generic_dists, metric_id):
+  """Creates flatten list of distributions per its value type.
+  A generic distribution is the one which is not processed but saved in
+  the most raw version.
+
+  Args:
+    generic_dists: list of distributions to be saved
+    metric_id(uuid): id of the current test run
+
+  Returns:
+    list of dictionaries made from :class:`DistributionMetric`
+  """
+  return sum(
+      (get_all_distributions_by_type(dist, metric_id)
+       for dist in generic_dists),
+      []
+  )
+
+
+def get_all_distributions_by_type(dist, metric_id):
+  """Creates new list of objects with type of each distribution
+  metric value.
+
+  Args:
+    dist(object): DistributionMetric object to be parsed
+    metric_id(uuid): id of the current test run
+  Returns:
+    list of :class:`DistributionMetric` objects
+  """
+  submit_timestamp = time.time()
+  dist_types = ['mean', 'max', 'min', 'sum']
+  return [
+      get_distribution_dict(dist_type, submit_timestamp,
+                            dist, metric_id)
+      for dist_type in dist_types
+  ]
+
+
+def get_distribution_dict(metric_type, submit_timestamp, dist, metric_id):
+  """Function creates :class:`DistributionMetric`
+
+  Args:
+    metric_type(str): type of value from distribution metric which will
+      be saved (ex. max, min, mean, sum)
+    submit_timestamp: timestamp when metric is saved
+    dist(object) distribution object from pipeline result
+    metric_id(uuid): id of the current test run
+
+  Returns:
+    dictionary prepared for saving according to schema
+  """
+  return DistributionMetric(dist, submit_timestamp, metric_id,
 
 Review comment:
   We create a generic distribution here, right? Should we tag them as generic 
somehow? Maybe it's just the fact that it has a different name thatn the 
distributions we do care about. Just wondering.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 295809)
    Time Spent: 7h 50m  (was: 7h 40m)

> Save correctly Python Load Tests metrics according to it's namespace
> --------------------------------------------------------------------
>
>                 Key: BEAM-7528
>                 URL: https://issues.apache.org/jira/browse/BEAM-7528
>             Project: Beam
>          Issue Type: Bug
>          Components: testing
>            Reporter: Kasia Kucharczyk
>            Assignee: Kasia Kucharczyk
>            Priority: Major
>          Time Spent: 7h 50m
>  Remaining Estimate: 0h
>
> Load test framework considers all distribution metrics defined in a pipeline 
> as a `runtime` metric (which is defined by the loadtest framework), while 
> only  `runtime` distribution metric should be considered as runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Reply via email to