Collective action brings together organisations with different capabilities, priorities and ways of working around a shared agenda. But how do we know whether working together is actually producing better results than organisations working independently?
Measuring collective impact requires looking beyond whether individual organisations are delivering what they committed to. Progress may include the outcomes a collective set out to achieve, but it can also involve less visible shifts in relationships, coordination, alignment, ownership and ways of working. These dimensions can determine whether a collective is moving towards its shared vision, yet they are often difficult to capture through conventional programme reporting.
This raises an important question: What should a collective measure to understand whether it is making progress, and how can evidence help it make better decisions along the way?
The fourth session of the #CollabMatters webinar series, held on July 29, explored measurement as a multidimensional task. The discussion examined how collectives can build evidence systems that capture partner-level delivery, the quality of collaboration and changes at the community and systems levels. It also explored how learning agendas, evaluation methods and decision-making cycles can be aligned so that evidence is not simply produced, but used to shape the collective's evolution.
To explore the role of evidence in collective action, Arjun Bahadur moderated the fourth session of the #CollabMatters webinar series. He was joined by an expert panel featuring Marianne McPherson (McPherson Collaborative), Dr Chandan Jain (International Initiative for Impact Evaluation, 3ie), and Shantanu Dubey (Saamuhika Shakti, Sattva Consulting).
Opening the conversation, Arjun Bahadur observed that building a meaningful measurement system is difficult in any programme, and harder still in collectives where multiple partners report differently, funders hold different expectations, and the context keeps shifting. A collective is not simply a bigger programme. It works differently, so conventional approaches to measurement often do not fit.
The tension underneath, he suggested, is how a collective knows whether working together is producing better results, or whether it is mainly absorbing the additional complexity that comes with collaborations. And that difficulty rarely begins at the point of method. It begins, in fact much earlier, with questions that sound basic but are not: what evidence does a collective actually need, at what point in its life is each kind worth building, who builds it, and how do you handle the constraints sitting underneath all of it — capacity, relationships, trust and power?
The discussion that follows therefore approached measurement as a way of understanding not just what is changing, but how the collective is functioning and whether working together is contributing to better outcomes.
Two broad dimensions need to be held together. The first is purpose: what has the collective come together to accomplish? For collectives seeking systems change, this purpose may operate at several levels, including the broader system, participating organisations, and the people and families within that system.
The second is the collectiveness of the effort: why have these organisations chosen to work together, how is that way of working progressing, and what difference is collaboration making?
A useful evidence base therefore needs to illuminate both the destination and the journey. Outcomes, processes and relationships are interrelated rather than separate categories of success.
This means evidence should help a collective move through three questions: What is happening? So what does the data mean? And now what should the collective do, or choose not to do, in response?
Answering these questions will usually require a combination of qualitative and quantitative evidence. Measuring only the final destination, or relying primarily on numbers, can cause a collective to lose sight of the relationships and processes that enable progress.
The questions should also matter to the collective as a whole while reflecting the contributions of individual partners. Drawing on the 100 Million Healthier Lives initiative, Marianne highlighted questions such as: Whose lives are getting better because we are here doing this work? and Who is not thriving, and what will it take to partner to change that?
These questions keep attention on who is experiencing change, who may be left behind, and how the collective will know.
Saamuhika Shakti initially followed a familiar approach, tracking what individual partners delivered through programme progress reporting and commissioning an independent impact evaluation to understand changes for the community. Over time, however, it became clear that outputs being delivered on schedule did not necessarily demonstrate progress in a collective-impact context.
The collective risked losing sight of the mechanism that was supposed to make the model distinctive: the collective itself.
This prompted a different set of questions: Were collaborations actually working? Was their quality improving? Were partners genuinely aligned around the common agenda? Were they reinforcing one another, or simply implementing activities in parallel?
These questions matter because collaboration can determine whether a collective achieves its shared vision, but it often remains invisible in conventional reporting. It exists in the spaces between individual partner reports.
Saamuhika Shakti therefore built its shared measurement system around three connected dimensions: partner-level delivery, the quality of collaboration, and changes at the community and systems levels. The dimension that needed to be consciously added and prioritised was collaboration.
Collectives often concentrate on the third question - impact on primary stakeholders - without examining collaboration with the same seriousness. This creates a blind spot when assessing cost-effectiveness.
A collective model may be expected to deliver interventions more efficiently than a series of isolated grants. But bringing organisations together requires time and effort, and participation can initially increase delivery costs as partners adapt to new structures, reporting requirements and a shared agenda.
Collectives should therefore ask whether coordination is increasing partners' operating costs, what challenges are affecting delivery, and whether the additional investment is producing greater value at the collective level.
Methods such as process tracing and process evaluation can help reveal the obstacles partners face, how those obstacles influence delivery, and what support is required for the collective to work more effectively.
This creates both relational and resource constraints. Collective action and measurement, evaluation and learning can feel uncomfortable, slow and counterintuitive to people accustomed to more familiar organisational models. While that discomfort can sometimes signal that a collective is doing necessary work, it can also push people back towards safer and more conventional approaches.
There is also pressure to demonstrate impact quickly - not only that the collective is moving towards its intended destination, but that its chosen way of getting there is already working.
The resources required for measurement extend beyond funding. They include time to generate and interpret evidence, time in the field, and the less visible costs of sustained attention, motivation and commitment. When these resources are scarce, collectives can default to measuring what is already familiar rather than what is most important.
Moving from this competitive orientation towards collaboration requires time and deliberate effort. It cannot be assumed simply because organisations have been brought into a shared structure.
Three additional challenges shape measurement. First, collective impact remains relatively unfamiliar in the Global South and particularly in India. There are fewer precedents and experienced practitioners to learn from, and often no shared language through which partners can describe what they are attempting together. Shared language is not cosmetic; it is connective tissue for the collective.
Second, there is a capacity constraint. Many organisations are not equipped to undertake this kind of measurement or invest in the capabilities required. This challenge can extend to funders and backbone organisations as well.
Third, evidence within a collective is never entirely neutral. Evidence used for learning can also influence funding decisions. Asking a partner to be candid about what is not working may therefore mean asking them to reveal vulnerability in a room that has a say in their future.
This can suppress precisely the learning that the collective needs most. Honest measurement requires acknowledging disagreement, contradictions and power relations rather than smoothing them over. Measurement is therefore relational work as much as technical work.
For example, an evaluation may require partners not to enter a control locality or engage control households. At the same time, partners may have implementation targets that incentivise them to expand into those areas. This does not necessarily represent a failure by the partner. Rather, it reflects a misalignment between programme incentives and evaluation requirements that can compromise the evaluation.
Another challenge is the absence of a clear learning agenda at the right time. A learning agenda should not be developed by a third party and handed to partners after the fact. Partners, the backbone organisation and the funder need to co-create it before implementation. This builds ownership and clarifies what each organisation needs to contribute to the collective's learning.
Different questions require different methods and rhythms. Partner-level delivery can be monitored continuously through programme MIS data, a shared measurement system and field-level verification. These sources can be triangulated to assess whether partners are delivering what was agreed.
The quality of coordination and collaboration is better examined through process evaluation and qualitative methods. A rapid assessment early in the collective's life can establish how partners understand the collective, their roles and the common agenda. Repeating the assessment later can show whether alignment has improved and whether deliberate efforts to strengthen collaboration have worked.
Population-level impact may require methods that support causal attribution. While an experimental design may be difficult in a multi-partner programme, quasi-experimental methods can sometimes use existing variation to establish a credible comparison group.
The sequence is important. Collectives can begin by commissioning an impact evaluation and only later realise that they also need to understand partner performance and collaboration. A more useful progression is to establish whether partners are delivering and the collective is functioning as intended before investing in an evaluation of its overall impact.
She shared an example from a regional initiative in Southern California involving around 40 organisations across health care, housing and community sectors. The initiative seeks to support older adults experiencing intersecting health and housing challenges by shifting from isolated efforts towards a connected community of care.
Its outcomes are organised around three areas. The first is improved health and housing outcomes for older adults, guided by the question: whose lives will get better because of this work?
The second is improved coordination within the system, with a longer-term test of whether partners can eventually say, “This is just how we work around here” - signalling that coordinated, population-centred and cost-effective practice has become normal rather than exceptional.
The third is stronger and more durable relationships. Here, the guiding statement is: “The way we work together is an example of what is possible in our system.”
Partners periodically use the Assessment for Advancing Community Transformation (AACT) to examine dimensions including collaboration, communication and the use of data for action. The collective's leadership then selects one or two areas on which to concentrate its improvement efforts.
Marianne also highlighted Well-Being in the Nation (WIN) Measures, a shared set of measures designed to support improvements in population health, wellbeing and equity across sectors and geographies in the United States. Co-designed with more than 100 communities, government agencies and national organisations, it combines a small set of core measures with leading indicators across areas such as food, housing, climate and environment, transportation, community infrastructure and health-care access.
Together, these examples illustrate how a collective can maintain rigour at the centre while allowing flexibility at the edges.
Evidence production should be aligned not only with the learning agenda but also with the collective's decision cycles. This makes measurement more intuitive and increases the likelihood that evidence will arrive when it can shape action.
Methods can also be combined where that helps answer the question. At Saamuhika Shakti, this led to the development of a collaborative activity-logging system, which records who is collaborating with whom, the purpose of the collaboration and the pathway or sub-pathway in the theory of change to which it contributes.
Over time, these logs show where collaboration is concentrated and whether patterns of joint activity support the hypothesis that collective action produces outcomes beyond parallel delivery.
Measurement also needs to account for learning questions that are partly planned and partly emergent. Saamuhika Shakti's study of early signals of systems change, for instance, examines shifts the collective may have contributed to within its geography. During its design, another question emerged: how was the collective's investment in community agency and leadership beginning to manifest?
A separate study was not feasible, but the systems-change study had deliberately selected the community as a central subsystem and included themes related to agency. It could therefore provide partial evidence on this emerging question.
The broader lesson is to design studies thoughtfully, segment learning needs and connect them to decision and programming cycles.
A collective can instead begin by asking what would be observably different in the system if collaboration were working well. If the goal is more coordinated services, for example, a reduction in the time people take to receive those services may reveal the practical consequences of collaboration more clearly than a perception score alone.
Collectives can also track the collaborations themselves: which shared activities are taking place, whether they are planned or spontaneous, and whether joint action is moving from something requested by the backbone organisation towards something partners initiate and own.
Qualitative enquiry remains essential. Conversations with partners can reveal not only what collaboration is occurring, but why it is or is not happening, what enables it, what obstructs it, and what the collective should change in response.
Reading & Resources
This session was part of the five-part #CollabMatters webinar series on collective action. Stay tuned for upcoming sessions on measuring what matters and unlocking capital that supports collective action! For updates on upcoming webinars, subscribe 👇
