Issue #6528 has been updated by Pieter van de Bruggen.
Since the scope of this ticket has been changed, this note should serve to document my understanding and plan w.r.t. the current work: * The test data should appear "real". * The user should be able to choose to generate these reports as inspect reports. * This includes schedule resources on every report and some number of resources of arbitrary types (file, user, package, and service for now). * There should be events and log entries for these resources. * Accurate metrics for changes, resources, and events should be included. * The test data should change in realistic ways from report to report. * Reports with no changes should be vastly preferred. * Reports with changes should contain some small percent of resources that were changed, pending, or failed. Longer term goals include: * Configurable resource types. * Configurable node statuses. Please fill any gaps in my understanding. ---------------------------------------- Feature #6528: Test data: robust, real-looking, repeatable https://projects.puppetlabs.com/issues/6528 Author: Randall Hansen Status: Accepted Priority: Normal Assignee: Category: Target version: Keywords: Branch: Affected URL: Affected Dashboard version: We need to be able to test Dashboard with test data that we control. We cannot use customer data for this due to the potential of a leak. Our community needs to be able to see and work with this data as well. This test data should be able to reflect sensible changes over time. That is, a node should have some number of resources which drift in and out of compliance. We need to be able to report on a node's compliance over time. Which resource properties change to affect this compliance drift are undefined. We need a script which generates a configurable number of reports that can be imported into Dashboard with `rake reports:import`. These reports should: * look like real data (e.g., all nodes and resources can't be called "test") * exercise all user-facing features of Dashboard (e.g., we need nodes and resources in all possible states) * be repeatable (e.g., tested) * date-sensitive (i.e., since we make decisions in Dashboard based on the last date a node reported, hard-coded dates will not work). We have a very good start at this, but it's not yet complete. At minimum (and I'm not convinced this list is complete) we need to be able to generate: * nodes (see "Node status proportions" below) * compliant/successful * failed * unresponsive * pending * resources of status * successful * pending * failed * types of resources (see "Resource types proportions" below) * file * user * package * service * schedule (Puppet built-in schedules are sufficient; no resources need be associated with them) * events for these resources * inspect reports **Node status proportions:** roughly 80% successful nodes (**configurable**), with the rest distributed roughly evenly. **Resource types proportions:** file, user, package, service should each appear in roughly equal proportion. This proportion should be configurable via environment flags, similar to "NUM_EVENTS" and friends. The use case is that I need to be able to generate reports which are 100% files, with no users, packages, or services; but this is an edge case and should not be default behavior. Also: * every failure event should generate a log entry * accurate metrics for: * changes * events * resources * do NOT need metrics for "Time" -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
