By Hannah Muirhead – CECAN Fellow
Integrated rainwater management (IRM) is an approach to managing rainwater more deliberately, more locally, and more strategically across the complex, interconnected socio-technical system it sits in – from where it lands and across the pathways it subsequently follows.
This fellowship began with the assumption that traditional evaluation methods were poorly suited to a system as complex and institutionally fragmented as IRM, and set out to understand what a more appropriate approach might look like.
A systems mapping exercise, informed by literature then refined through informal conversations with practitioners, helped surface some of the structural dynamics that make integrated outcomes difficult to achieve and evaluate.
The mapping began with the basic system, showing how rainfall moves across three hydrological pathways (the drainage network, sustainable drainage systems (SuDS) & infiltration, and storage & reuse) and how multiple institutions shape those pathways through the decisions they make based on some core incentives.
The base map (Figure 1) suggests that IRM outcomes are not owned by any single organisation, that the key decisions are frequently made early and shape what becomes possible long before impacts are visible, and that many decisions require coordination across institutional boundaries – with the interfaces between institutions often the most consequential and least governed parts of the system.
Figure 1: The base systems map (see key below)
Two analytical overlays were then developed. The first overlay (Figure 2) maps time horizon mismatch. The red dashed lines indicate where short-term incentives or day-to-day operational pressures intersect with decision points that are early-locking and difficult to reverse. In practice, this means places in the system where an institution is making a consequential, hard-to-undo decision (for example about drainage network capacity, building design or land use) while operating under incentives that are oriented towards the short term (cost certainty, regulatory compliance, incident avoidance, operational performance). The red dashed lines show this mismatch, and the key takeaway here is that they appear in nearly every part of the system. Where those short-term pressures dominate, long-term outcomes get underweighted, not because they aren’t important, but because the timing of decision-making is misaligned with the timing of impacts.
Figure 2: The base map overlaid with time horizons
The second overlay (Figure 3) maps risk asymmetry. Here, the thick double black lines indicate where institutions face significant downside exposure if decisions or outcomes are poor (such as high regulatory, reputational, financial, operational, or political risk). The dashed black lines indicate the comparatively limited pathways through which getting decisions right directly reduces future risk for those same actors. In other words, the consequences of getting things wrong are concentrated and very visible, while the rewards of getting things right are diffuse and weakly fed back. What this overlay shows is a structural imbalance where many institutions are operating under meaningful downside risk, but there are comparatively few pathways through which positive outcomes reduce their future exposure. This helps explain why the system may favour cautious, easily-defensible approaches (for example conventional engineered solutions that satisfy permit conditions reliably) even where more integrated approaches would deliver greater long-term value.
Figure 3: The base map overlaid with risk asymmetry
Together, these diagrams move us from just describing what we think is a complex system towards a more explicit diagnosis of the conditions that make IRM difficult to deliver and difficult to evaluate.
The problem with conventional evaluation
Visualising the system helps explain why conventional evaluation struggles here. Rainwater moves across multiple interacting physical pathways, where decisions affecting one frequently influence the performance of others in unpredictable ways, and seemingly small interventions can produce disproportionate effects elsewhere.
Those physical dynamics don’t operate in isolation. System performance depends on the interaction of multiple decisions made across different institutions and timescales, where causal effects are hard to isolate, responsibility for outcomes is diffuse, and many outcomes emerge over long time horizons and across organisational boundaries.
A further structural feature of the system is that not all forms of value carry equal weight in decision-making. Returns like flood attenuation and regulatory compliance are easy to justify in business cases, while wider benefits like biodiversity, place quality, and long-term resilience tend to have weaker influence at the point of decision even where they are recognised as important in principle.
When multiple stresses act on all of this simultaneously, their effects interact rather than just accumulate, creating the potential for tipping points at which separately manageable challenges produce disproportionate system failure.
Conventional evaluation is poorly matched to all of this. Designed around discrete interventions, stable causal chains, and attributable outcomes, it tends to assess performance at a level that is downstream (figuratively and literally) of the decisions, interactions, and structural tensions that actually determine whether integrated management of rainwater is succeeding.
A framework built around system capability
This fellowship developed a conceptual framework that starts from these structural features of the system itself and derives the evaluation focus from that diagnosis – asking whether the system is developing the capabilities needed to function coherently under conditions of complexity.
It is organised around five evaluation domains grouped in three layers.
The first layer, foundational governance, asks whether the institutional conditions for coordinated action actually exist. Without this, the other layers will be largely moot.
- Domain 1: Governance clarity and institutional alignment examines whether roles, responsibilities, and accountabilities are sufficiently clear to support coordinated action, particularly at the interfaces between institutions where the most consequential gaps tend to occur.
- Domain 2: Collaboration and partnership functioning asks if collaboration is embedded and durable – a genuine system capability rather than dependent on particular individuals, project funding, or favourable conditions.
The second layer, decision quality, examines how well the system makes the choices that shape long-term outcomes.
- Domain 3: System integration and coherence asks whether decisions across rainwater pathways are made with awareness of their interactions and cumulative effects, and whether integration is built into early, hard-to-reverse decisions rather than only in downstream coordination where the scope to act has already narrowed.
- Domain 4: Value flows, cost distribution, and investment logic examines whether the wider benefits of integrated approaches, such as improved biodiversity, long-term resilience and public health, have sufficient influence on investment decisions in practice, or whether they are recognised in principle but underweighted at the point of decision.
The third layer, adaptive capacity, asks whether the system is improving over time.
- Domain 5: Learning, adaptation, and long-term resilience looks at whether the system learns from experience and translates that learning into changes in how it operates, and whether it is developing the flexibility needed to maintain performance under compound and changing pressures.
These layers are deliberately sequenced. If governance is fragmented and collaboration is fragile, the quality of individual decisions and the system’s ability to learn are both constrained from the outset. The framework is designed to make those dependencies visible.
In the framework, each domain also sets out the evaluative questions an assessor should ask, the forms of evidence most likely to reveal how the system is functioning in practice, and some judgement anchors that distinguish weak, mixed, and strong system capability.
Why this matters
The framework can be applied in several ways, such as:
- A diagnostic tool to establish a baseline across the five domains
- Developmentally, alongside innovation programmes or governance reform initiatives
- Comparatively over time to track whether system capability is strengthening
- Prospectively, to assess whether a proposed policy or investment programme is designed with the structural realities of IRM in mind
While still conceptual and not yet tested in live application, its central contribution is perhaps the diagnostic move that underpins it – rather than starting from evaluation methods and asking how to apply them to this complex system, the framework starts from the structural features of the IRM system itself and derives the evaluation domains from that diagnosis.
What it offers across all of these applications is a more honest basis for evaluation, and one oriented towards the conditions that make integrated performance possible.
This shift in evaluative focus feels like a prerequisite for understanding whether what we are doing is actually working – and this fellowship has given me a much stronger basis for thinking about what meaningful evaluation in this space could look like.
Huge thank you to my mentors, Liz Varga and Stuart Astill, for their time and guidance, and to CECAN for the opportunity to significantly broaden my evaluative horizons through this fellowship
The full framework report is currently being fine-tuned and will be available shortly.
Key for the maps:
