kennknowles opened a new issue, #19258: URL: https://github.com/apache/beam/issues/19258
We have no way to dump and update the metric fake data this test uses. We should consider writing a tool to do this properly. We can change the format as long as its still human readable. I tried and the following failed: (1) Using json.dumps on the job_metrics Fails for not being JSON serializable (2) Using the proto json_format.py tools to dump on the job_metrics Fails with exception: object has no attribute 'DESCRIPTOR' [https://github.com/googleapis/google-cloud-python/issues/3485](https://github.com/googleapis/google-cloud-python/issues/3485) (2) Using the proto text_format.py tools to dump on the job_metrics Seemed to dump an empty file? (3) Using this code to turn any object into a simple JSON style dict and print the JSON. This fails because it will print out all the private variables and other things we don't want. We could use this approach and allowlist the relevant fields. ``` def _create_simple_obj(self, obj): # If its a dictionary or defined class type if hasattr(obj, '__dict__') or isinstance(obj, dict): items = obj.items() if isinstance(obj, dict) else obj.__dict__.items() simple_dict = dict() for key, value in items: simple_dict[key] = self._create_simple_obj(value) return simple_dict elif isinstance(obj, (tuple, list, set, frozenset)): simple_list = [] for x in obj: simple_list.append(self._create_simple_obj(x)) return simple_list else: return obj def _pretty_format_job_metrics(self, job_metrics): job_metrics = self._create_simple_obj(obj) return json.dumps(obj, indent=4, sort_keys=True) ``` Or we can just give up and not try to dump new ones, just add some basic tests to make sure it can convert the metrics format to MetricResults. Then add more extensive testing on the MetricResults themselves with the matchers we are adding. Imported from Jira [BEAM-6734](https://issues.apache.org/jira/browse/BEAM-6734). Original Jira may contain additional context. Reported by: [email protected]. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
