How will you know if your policy did what you meant it to?
Policies and programs always have an intent or purpose. You will probably have the clearest idea about that intent in the design or development phase. Even though the time for checking whether it worked is possibly months or years down the track, it makes sense to capture intent and measurement as part of development.
This means designing in your assessment metrics as part of the package and how you would collect the data. And you really want the data collected from day 1.
Of course your evaluation framework may change over time. Even so, having a plan that ties measurement to intent will make it easier for you and for those that come after you.
People change roles in policy jobs, sometimes even before a policy, program or decision is implemented. Having a clear statement will help the people who come after you to do the assessment. And then when you pick up a project that somebody else started, you will be incredibly grateful that there is a clear record of what was initially intended (we certainly have been!).
Reviews come about in lots of different ways. It may be through a program evaluation, or post-implementation review. In exceptional cases it might involve a Royal Commission or Parliamentary Inquiry. In every case, having at least basic data and information will make the process more useful.
Checklist to make sure your project can be reviewed
A review or evaluation of your policy or initiative will need these (and developing these may help in the development of your proposal):
- Objectives and purpose: Explanation of why this policy has been developed at this time.
- Theory of change. Explanation of how your policy is expected to work and how it is intended to influence people’s behaviour (a program logic is one example).
- Context. A summary of the key contextual factors for your policy at the time it was being developed (as the context will be different by the time it is evaluated)
- A clear description of the instrument or device that is being implemented
- Success measures. Measures and targets that you anticipate you will use to test the success of your policy. Covers both process and outcome.
- Data collection methods. How you will collate and report both monitoring data (in program) and outcome data.
- Stakeholders. A list of key stakeholders and contact details (this will change, but a good list is an incredibly useful starting point).
- Core documents. A master set of the core documents relating to your policy, such as discussion papers, final approval documents, Cabinet Minutes, etc. This should be the ‘file’ as the one source of truth, but files often accrete loads of irrelevant information. You will want to think about the electronic storage as well as electronic files can often be confusing (e.g. multiple versions in different folders; unclear folder structure).
