Printer-friendly versionSend by emailPDF version

Featured Conversation: Citizen Engagement

Engaging with citizens is often associated with positive results such as increased transparency and accountability, as well as innovative delivery of public policies and services. However, creating channels for citizens to express their needs and preferences is only a part of the equation. Often misinterpreted are the steps necessary to design and implement engagement processes that ultimately produce desirable outcomes. For governments, NGOs, and donors alike, there is great interest in understanding what works and what doesn’t in public participation. Join our host, Tiago Peixoto of the World Bank, and three other citizen engagement experts, the OECD's Joanne Caddy, Vera Schattan Coelho from El Centro Brasileiro de Análise e Planejamento, and Involve's Deputy Director Edward Andersson, in a conversation about innovative participatory practices that promote the effectiveness of governments and development projects.
Photo © Copyright The World Bank. All rights reserved.

Tiago Peixoto, Open Government Specialist for The World Bank, moderator

Tiago Peixoto, The World BankCalls for increased citizen empowerment are heard from across the spectrum, ranging from governments and donors to CSOs and multilateral efforts such as the Open Government Partnership.

But while the claims for citizen engagement abound, less discussion is dedicated to how to design and implement participatory processes that deliver their expected benefits, such as increased accountability and better delivery of policies and services. As part of this problem, not enough attention is paid to the various outcomes that participatory processes may engender and what they mean for policy and development.

For instance, in some cases participation may lead to disappointing results, such as citizens' mistrust of government, elite capture and public opinion polarization. Conversely, participation can also be associated with surprisingly positive outcomes, such as increased levels of tax compliance and reduced infant mortality. But how can we explain these disparities in results?

Shedding light on the question of when, why and how participation works is precisely the objective of this conversation. Thus, to kick off the debate, I would like to start by considering seven questions:

  • How can we measure the success of citizen engagement initiatives?
  • How essential are processes of organisational and institutional change?
  • Can political will towards increased participation be stimulated?
  • What role does organized civil society play in citizen engagement processes?
  • How can we foster inclusiveness and what are the impacts of different methods of participant selection (e.g. open, randomized)?
  • Can we learn anything from the private sector about listening to external audiences?
  • What is the actual role of technology (if any) in participatory processes?

Needless to say, the issues raised by these questions are far from exhaustive. Maybe some are even secondary. But I believe that considering them might bring us closer to answering an even more fundamental question: that is, how can we leverage the dispersed knowledge of citizens to shape decisions that affect their lives?

To start the conversation, we've invited three people working in different areas: Joanne Caddy of the Organisation for Economic Co-operation and Development (OECD); Vera Schattan Coelho of the Brazilian Center for Analysis and Planning (CEBRAP); and Edward Andersson of Involve.

Joanne Caddy, Senior Policy Analyst, OECD Directorate for Education and Skills, on policies for better citizen feedback

Joanne Caddy, OECD Directorate for Education and SkillsGovernments alone cannot solve the complex, urgent and entrenched challenges facing our societies today. Climate change, soaring obesity levels, poverty and growing inequalities of income and opportunity all call for a new approach to improving policy outcomes. One which engages citizens in defining the problem to be solved, mobilises the best available data, thrives on transparency and is driven by dialogue. Crucially, this new approach can only be considered successful if it gets everyone — citizens, governments, businesses, households, civil society organisations — to move beyond dialogue to taking action.

My own home province of New Brunswick, Canada was witness to this kind of paradigm shift a few years ago, when the premier at the time launched the audacious goal of “self-sufficiency” for a peripheral, struggling province that had only ever survived on fiscal transfers from the federal government. What followed was a series of initiatives, large and small, which used public engagement as the main lever to get everyone — chambers of commerce, the pulp and paper industry, students, mayors — to identify a specific tractable problem together, and then commit to taking concrete action. In doing so, they would hold one another to fulfil their commitments in a new form of mutual accountability (check out "It's more than talk" by Don Lenihan for more details).

Innovations in this area emerge daily from across the globe, as a result of initiatives launched by governments, businesses and civil society organisations alike. Some of the most powerful approaches combine online and face-to-face engagement to mobilise previously "silent voices" or tap into existing decision-making processes in new ways.

Framed thus, public engagement becomes an essential feature of modern governance and a powerful means to achieve shared goals by mobilising the ideas, energies and resources of a wider public. It is high time we treated public engagement as a field of public policy that has come of age. Doing so would oblige us to guarantee enabling legal and policy frameworks, set clear objectives for public engagement, clarify expected outcomes, account for the public and private resources used and to ensure assessment, evaluation and accountability for results. Nothing less.

Vera Schattan Coelho, Senior Researcher, Centro Brasileiro de Análise e Planejamento (CEBRAP), on new forms of citizen participation and consultation

Vera Schattan Coelho, OrganizationSince the late 1980s, direct civil participation in public policies is embedded in the Brazilian Constitution as a mean to promote accountability, priority setting and local problem solving. This official policy has been implemented through a national framework of participatory forums linking the local, the state and the federal levels. Among these forums, the health councils have become one of the most widespread and influent. Their successes and shortcomings throughout the last three decades provide lots of evidence for analysis and raise many questions, two of which I highlight here:

  • How can effective participatory mechanisms be implemented?
  • How can the performance of these mechanisms be assessed?

Successful implementation depends on the combination of several elements: committed public officials, mobilized citizens, and innovative design features. Each of these elements alone will be insufficient to overcome the enormous difficulties of bringing the citizenry into the policy process. Indeed, success almost invariably requires the simultaneous presence of state actors interested in building alliances with civil society, of citizens and civil organizations that display interest in participating in public policies, and of design features that reduce the asymmetric distribution of resources among participants.

To assess the performance of these forums, it is important to look at: a) Who is taking part, in terms of socio-demographic, political, and associative characteristics of the participants; b) The information provided and how the practice of deliberation, bargaining, and confrontation are balanced in the meetings; c) The connections that are being established between these forums and other branches of the political system, the health system, as well as with other public and private organizations.

Looking at these two dynamics — implementation and daily work — helps to understand successful cases and also think about what needs to be improved in less positive experiences.

Edward Andersson, Deputy Director at Involve, on methods of participatory decision making

Edward Andersson, InvolveParticipative methods have a long history in development practice; in fact, many leading engagement practitioners in the UK started their careers working on Participatory Appraisal in development projects.

My organisation Involve works to highlight and promote good examples of participative practice in the UK and beyond. The last decade has seen some very interesting innovations. In certain sectors participation and engagement have gone from being fringe activities to being legally mandated activities. There is more citizen engagement and it is taken more seriously than it was ten years ago; however, the practice often leaves much to be desired. A view shared by many practitioners, including Involve, is that bad citizen engagement is worse than no citizen engagement at all, as it reduces trust.

Common problems relating to engagement include tokenism (engaging citizens primarily to look good, without the real chance for influence), engaging at the last minute, and processes which fail to reach those most in need.

Involve recently researched a publication looking at some of the most innovative citizen engagement from around the world (the report From Fairy Tale to Reality is available on our website). I thought I would highlight some of the key innovations that I hope to see in the future:

Devolving — In Estonia, the 'My Estonia' process of community clean-ups and brainstorming workshops involved tens of thousands of citizens, largely because government let go of control and allowed local groups to self-organise. This may seem like a risky strategy to state actors, used to having full control, but can yield huge dividends.

Deliberating — Experience from organisations like AmericaSpeaks shows that even in the most controversial areas, it is possible to set up spaces where informed deliberation is encouraged. Much citizen engagement never gets beyond the surface level of the debate, and deliberation can help with this.

Diversifying — The question today is not whether to use online or face-to-face methods; rather, some of the best processes combine both. Processes like "Geraldton 2029 and beyond" in Western Australia use dozens of different methods to reach different groups of citizens, with some methods specifically aimed at Aboriginal groups, young people, etc. The quest for a "one size fits all" engagement is a futile one.

It is a very exciting time to be active in citizen engagement — lots of innovation is happening and the boundaries of what is possible are shifting. I’m looking forward to an interesting discussion.

Conversation summary

This Striking Poverty discussion on citizen engagement centered around the question: "How can we leverage the dispersed knowledge of citizens to shape decisions that affect their lives?"

To make citizen engagement more inclusive, citizens, government, businesses and civil society organizations need to move beyond dialogue to taking action. Citizens play different roles including providers of ideas/expertise and representatives of specific interests. In addition to 'distributed' engagement or participation by proxy, where associations represent poor and the 'usual suspects' end up voicing all too familiar views, processes now need to be designed for 'publics' rather than 'the public' — the quest for a "one size fits all" engagement is a futile one.

New, innovative ways of engaging are emerging: devolving (yielding control to local groups), deliberating (moving beyond collecting opinions), and diversifying (combining online and face-to-face methods). Approaches to selecting participants vary (open access, stakeholder/interest based, random selection) and each method has pros and cons. Regardless of the approach, three key elements are necessary for successful citizen engagement: committed public officials, mobilized citizens, and innovative design.

A number of myths and misconceptions inhibit adoption of citizen engagement (e.g. it's too expensive, rarely successful, citizens aren't up to it, only useful for easy problems, people aren't interested, and "it will open a floodgate that may threaten the world as we know it"). Other barriers include: citizens' mistrust of government, public opinion polarization, information asymmetry, and elite capture. Compounding those challenges are poor citizen participation practices: tokenism, engaging at the last minute, failing to reach the vulnerable groups, poorly informing participants, and "fake processes" ("I participate, you participate, they decide!").

Read the full summary.


TiagoPeixoto's picture

Welcome Joanne, Vera and Edward: it is great to have you in this conversation.

In all of your initial posts you mention examples of what seem to be successful initiatives of citizen engagement. But this begs the question: are there some common standards against which we should be evaluating these initiatives? How can we measure the success of citizen engagement initiatives?

Tiago Peixoto
Open Government Practice
The World Bank

EdwardAndersson's picture

Evaluating the success of a citizen engagement process is fraught with difficulty. The first starting point is to identify why engagement was chosen in the first place. Sometimes a key driver is overcoming conflict, in other cases the driver is improving the quality of evidence used to make policy. Success may look very different in these two cases.

Secondly it is important to acknowledge that successful citizen engagement is not the same as participant satisfaction. It is entirely possible for a process to be very enjoyable and achieve very little and vice versa. Success is often simplified into two metrics: 'scores on doors' (participant satisfaction on the day) and 'bums on seats' (number of participants). Both of these are flawed for different reasons. The satisfaction metric is susceptible to factors that have nothing to do with the engagement (venue, temperature, quality of refreshments provided etc) and the number of participants often increases with public dissatisfaction with an issue.

Thirdly, the true impacts of engagement often aren't apparent for some time. Evaluations often need to be completed very soon after the engagement (often due to budget reasons) and may therefore not capture the true impact of engagement. we need more long term evaluations -the key question isn't if participants think their time was well spent the week after a consultation; the key question is if participants feel it was well spent a year or more later.

In terms of standards it is relatively easy to agree some basic things we can (and should) track -I'd list clarity of communication in terms of the purpose, the quality of the information provided to participants, the quality of facilitation and the feedback provided to participants after the event. We know that failing to do this undermines citizen engagement -but we cannot guarantee that simply adhering to these good practices will lead to good results. success often boil down to things that are harder to track (and which are prone to change) such as senior leadership buy in.

I think it is important to track the degree to which decisions are made differently as a result of engagement. Not because each citizen engagement process must change policy in order to be successful but simply because this is often a key driver for citizens to participate and if nothing ever changes as a result of their input we cannot expect them to contribute in the future. Tracking this also enhances our ability to tell the success stories more effectively. I have seen many processes which did have an impact, but where the policy makers were unable to communicate the difference it had made and therefore the participants often felt it had been pointless.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

ricardomatheus's picture

Hello, Edward.

Trying to complement and understanding better your opinion, I would like to know which democratic tools (forum, public consultation, voting, etc.) you talked when you pointed out about tracking degrees of decisions made and the results of engagement. further, if it is only the view that government can change the reality or if civil society can change reality "without" government (here thinking about new kind of governance).

Ricardo Matheus
Bachelor in Public Policy Management - EACH/USP
Master of Sciences in Administration - FEA/USP
University of São Paulo - USP
PhD Candidate - TUDelft
Delft University of Technology, The Netherlands
+55 (11) 9 7227-7521

EdwardAndersson's picture

Hello Ricardo,

You highlight an important point here. Government is by no means the only body with agency; NGOs and citizens themselves can change things. While the dichotomy of top-down and bottom-up engagement is overly simplistic I do think it is worth noting that much engagement is initiated by Government and as such we need to make sure that we track the impact of these processes and hold government to account for the quality of the process and the results.

A process which has been initiated by civil society will need to be evaluated using slightly different criteria, as it has a different background and a different set of objectives. I'm sceptical that it is possible to apply the same success criteria to a government led consultation as we would to a process with far reaching aims and objectives mobilising grass-roots participation.

I hope that makes sense.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

ricardomatheus's picture

Yes, it makes sense and I absolutely agree.

Thanks for the reply.

Ricardo Matheus
Bachelor in Public Policy Management - EACH/USP
Master of Sciences in Administration - FEA/USP
University of São Paulo - USP
PhD Candidate - TUDelft
Delft University of Technology, The Netherlands
+55 (11) 9 7227-7521

tbonnema's picture

Way to kick off the discussion, Edward. Great post!

Founder & CEO
Intellitics, Inc.

I think the quality and number of participants in this process is fundamental and crucial for its success. Unfortunately engaging participants are difficult to come by and those who participator cannot be relied upon for meaningful engagement. Hence again role of govt and politics unavoidable with all its shortcomings like corruption and shortsightedness.

Lucas Cioffi's picture

Yes, I think there are some common standards against which to evaluate citizen engagement initiatives, and also each initiative should have additional customized standards.

The National Coalition for Dialogue and Deliberation (in the US) crowdsourced a document called "Core Principles for Public Engagement" from its 1600 members (pasted below). Each of the seven items in the list could have associated measurements to know if they are being met, although some are harder to measure than others.

Core Principles for Public Engagement
These seven recommendations reflect the common beliefs and understandings of those working in the fields of public engagement, conflict resolution, and collaboration. In practice, people apply these and additional principles in many different ways.

1. Careful Planning and Preparation
Through adequate and inclusive planning, ensure that the design, organization, and convening of the process serve both a clearly defined purpose and the needs of the participants.

2. Inclusion and Demographic Diversity
Equitably incorporate diverse people, voices, ideas, and information to lay the groundwork for quality outcomes and democratic legitimacy.

3. Collaboration and Shared Purpose
Support and encourage participants, government and community institutions, and others to work together to advance the common good.

4. Openness and Learning
Help all involved listen to each other, explore new ideas unconstrained by predetermined outcomes, learn and apply information in ways that generate new options, and rigorously evaluate public engagement activities for effectiveness.

5. Transparency and Trust
Be clear and open about the process, and provide a public record of the organizers, sponsors, outcomes, and range of views and ideas expressed.

6. Impact and Action
Ensure each participatory effort has real potential to make a difference, and that participants are aware of that potential.

7. Sustained Engagement and Participatory Culture
Promote a culture of participation with programs and institutions that support ongoing quality public engagement.

Lucas Cioffi
CEO, AthenaBridge Inc
Board Member, National Coalition for Dialogue and Deliberation

Paolo Spada's picture

I want to derail the discussion a bit to talk about a unintended benefit of increasing the discussion around democratic innovations evaluation. So I am derailing, but I hope I will be forgiven, because I think there is a huge value in what a space of discussion like this one can eventually achieve.

In my experience impact evaluation of real life participatory processes (not academic pilots and experiment) is often an afterthought. Organizing large scale participatory processes is quite complex and resources are usually extremely scarce. A large part of the budget of many participatory processes is directed toward advertising and reaching out campaigns that aim to sell the process as a panacea for all the local problems to attract people to participate. This situation implies that often researchers interested in serious impact evaluation are met with suspicion. In other cases researchers are actually part of the organizing team and have different degrees of conflict of interest or militant involvement that might affect the type of research they conduct. Thus I see two types of impact evaluation that need to be considered somewhat separately:

Type 1: internal impact evaluation
This is impact evaluation aimed to improve and promote the process. This impact evaluation embraces its conflict of interest and it is similar to an R&D unit of a firm. Discovering if it is better to use moderators that intervene a lot in the discussion or moderators that are neutral arbiters, discovering what reaching out method is most effective, discovering how to promote political will and support to the participatory process are typical tasks that could use some help from serious internal impact evaluation.

Type 2: external impact evaluation
This is impact evaluation as we think of it in abstract in academia most of the times. The questions tackled might be identical to the ones tackled by type 1 investigation, but this type of evaluation requires no conflict of interests; it requires the researcher ideological autonomy from the process and the freedom to say terrible things about it. Additionally this type of impact evaluation has the critical role to assess alternatives to the entire participatory process and consider the possibility of abandoning it completely. This is a key question that is often overlooked for example when we claim that PB has impacts on tax collection and we do not ask ourselves if we can achieve the same impact at a much cheaper cost by providing training and motivation to public officials. Most PB processes couple training and motivation of public officials with the actual participatory mechanism. Do we really need the second one to achieve higher tax collection?

Unfortunately there is a great confusion between the two types of evaluation. Such confusion generates fears in the organizers of participatory innovations that researchers might be interested in doing 2 while just pretending to do 1 in order to gain access. This fear reduces the ability of using researchers to help improve processes from within. Most real-life participatory processes have a lot of kinks that need to be ironed in their first years of implementation, protecting them from critique is a rational strategy of the organizers. In an infant participatory process the wrong academic article or the wrong blog-post can be picked up by media adverse to the process and have some significant consequences on public perception. While a few years back impact evaluation researchers could tell worried organizers that their critique would end up in an obscure academic journal and somewhat soothe their fears, now that’s often impossible.

Similarly in academia there is a fear that researchers evaluating these innovations are in reality militants that lack the autonomy to conduct “serious” research. And such misconception has important impact on the ability of publishing and adds the constant necessity to send signals that a researcher of Deliberative Polls or whatever other democratic innovation is not a fanatic with an agenda, but she/he is as serious as a researcher of majoritarian electoral systems, or any other institution.

I hope that I am not misread. I am not trying to create a ranking between the two types of evaluations, claiming the type 1 is bad. Type 1 is a fundamental mechanism to improve participatory innovations. Participatory action research is a fantastic tool that employs type 1 impact evaluation a lot. A ton of great research is conducted by militant-researchers. Many times militant researchers have a much better understanding of the local conditions than more independent researchers.

My point is that confusion between the two types is damaging the field and an impact evaluation standard for type 2 evaluation promoted by the WB or a consortium of researchers could reduce this confusion A LOT. So I really applaud to the idea underlying these discussions, and I push the organizers to think about creating an impact evaluation standard out of them. I hope we can bring as many different voices in as possible to contribute to the dialogue. I would be particularly interested in hearing from expert of impact evaluation of other types of projects, because while democratic innovations have important unique characteristics, still they retain a lot in common with other type of institutions and there is no need to reinvent the wheel.

A standard "package" for impact evaluation could be very useful, not only for its obvious benefits for comparative research, but also because it could help isolate type 2 impact evaluations from type 1 by promoting a sort of clear requirements of lack of conflict of interest and autonomy. I am thinking something similar to the requirement of blind evaluation that is a typical red flag for certain type of content analysis and lab-experiments. Similarly a standard for type 2 impact evaluation promoted by the WB or a consortium of researchers could start with the requirement of a first paragraph in which the researcher states that she/he is not in conflict of interest, a paragraph that describes how he/she was funded, and contains a pledge of ideological autonomy.

Clarifying the distinction between the two types of evaluations would also promote better relationship between academics interested in helping the development of a democratic innovation conducting type 1 evaluations. The discussion surrounding the process of developing such standard could also remove the stigma of using non-disclosure agreements. Currently academics asked to sign non disclosure agreement have often panic attacks thinking that they are crossing some invisible line. Instead non disclosure agreements could be extremely helpful. For example a researcher could enter in an agreement that specifies that she/he can start conducting immediately type 2 evaluation whose outcome could be damaging for the participatory process object of the study as long as anonymity of the process is guaranteed. In many places type two evaluations is delayed because the process is too fragile in its beginning and thus organizers restrict access. This way we have truncated samples that rarely consider the inception of participatory processes. A typical cost of this situation is to compare mature processes that went through years of type 2 evaluation (Kerala, PB in Brazil) with infant ones that allow only type 1 without reflecting on the very different methodologies applied to the two places.

As we currently protect fragile individuals when we conduct research, using anonymity and other devices, having an understanding with organizers that we would protect fragile participatory processes and other similar democratic innovations from unnecessary critique could be extremely helpful. After all for many types of research removing the name of the locations where the research was conducted is not a major deal. This might help access information like the amount of funds invested in PB processes in Brazil that is something that really few cities disclose.

Obviously I see the standard as modular and expandable as modular and expandable are democratic innovations. We need something that works on fuzzy sets of tools so that it does not become a cage, but a stimulus. Also we need a way to expand such set of tools. So I guess the standard would be more a guideline about how to conduct type 2 impact evaluation, how to collect data that is comparable, how to adapt the package to local conditions while retaining the ability to interact with other places, and less about a fix recipe that would be imposed everywhere. In order to do that collecting examples of impact evaluations and distilling some common denominators could be the starting point.

Paolo Spada,

Democracy Fellow

Ash Center for Democratic Governance and Innovation
Harvard Kennedy School of Government


EdwardAndersson's picture


I'm really glad that you raised the distinction between internal and external impact evaluations. The two are very different, as you pointed out. I'd suggest that the reality on the ground is even more confusing. We also have evaluations that are undertaken primarily to justify the resources spent; evaluations which are often under resourced and with biased questions. I take the point that strictily speaking these are not evaluations, but that is how they are presented to the outside world. There are also promising experiments in involving participants in helping to design and carry out evaluations.

You mentioned that evaluation is rarely adequately resourced in participation. I'd like to flag up the UK Sciencewise programme as a good example (please note that I work on the Sciencewise programme so don't take this as a neutral perspective) -where each dialogue project as a matter of course is evaluated, and where usually 10% of the project budget is put towards evaluating the programme. The funded projects and their evaluations can be found here:

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

Veronica Cretu's picture

I think one of the departing points in measuring the success of citizen engagement initiatives can be the degree to which the core 3 levels on “Engaging Citizens in Policy-making” by OECD (2001) are being respected (regardless of who is the implementer the civil society or the Government). Certainly, indicators for each level can be identified and measured.
1. Providing citizens with access and information – here, it is important to assess both the quantity and quality of the tools that were used to deliver clear messages, information, details to different target groups in a clear terminology, language), ways in which citizens understand how certain initiatives or adoption of certain laws/provisions will directly affect their lives/the way of doing business, etc.
2. Consultation – at this stage, it is important to assess the degree to which tools applied to engage citizens were appropriate for them in understanding issues, problems, challenges, proposals, acts, laws, strategies, etc.; the degree to which “representativity” of the target groups was insured, etc.
3. Decision-making – the success of this stage should or could be measured in order to assess the ways in which citizens “voted” for the consulted strategies, laws, acts, norms, recommendations, etc. (and what platforms were used for this purpose: web, mobile, or offline strategies) …
I would also add the 4th level and mainly measuring the engagement of citizens in Monitoring and Evaluation processes at different levels/through different tools.

Veronica Cretu, President, CMB Training Center Moldova. Coordinator of E-Government/Open Government Working Group, National Participation Council.

DrAndyWilliamson's picture

* How can we measure the success of citizen engagement initiatives?
As Edward has already said really well, measurement is difficult and problematic. Really, what you measure depends on what you want to achieve, which really should be defined in advance. There will however but intangibles that you didn’t think of so it’s also important to be reflexive and flexible enough to capture these as well. You’re measuring from your perspective, engagement is a two-way process so also think about 360 degree measures that show success or failure from the perspective of all the stakeholders. You might have reached the target 100,000 people but the message was wrong so nobody took action! I’m also wary of the bureaucratic and political drive to quantify and measure in the short term. It’s vital that we create qualitative measures too and that we also look at the impact over time (there can be delayed reactions to engagement).

* How essential are processes of organisational and institutional change?
This is an interesting question. When the organisation is out-dated, out-of-touch, undemocratic and unable to understand its stakeholders these things are vital! Above all that what is vital is that organisations become responsive and demonstrably reactive. The process of engagement has to change so that it’s seen as business as usual all the way through the life cycle and not a tick-box exercise once the decision’s been made. Organisations need to listen. Easier said than done! They also need to pre-empt by listening so that engagement and decision making can be shaped by prior understanding of stakeholder needs not just statistical data or policy.

*Can political will towards increased participation be stimulated?
The short answer to this is that politician’s interest will be stimulated when they can see there is some benefit from doing better participation (I won’t say more, I’d rather focus on the quality of the process). If there is a clear cost/benefit argument, it will happen. This can sometimes be driven by political cycles but if you can show that the benefit is lower cost decision making (the sooner in the process a problem is discovered the less it costs to fix it!) and better outcome that more people agree with then it’s hard to argue.

*What role does organized civil society play in citizen engagement processes?
Absolutely vital. How can participation happen if one side of the equation isn’t involved? I believe that we have to move from the ‘them and us’ mindset to one where we co-create. This means involving civil society in the process from day 1, involving them in the design of the participation processes and treating them as equal partners in the policy and design process.

*How can we foster inclusiveness and what are the impacts of different methods of participant selection (e.g. open, randomized)?
This comes down to trust and belief in the process. It takes time. One engagement at a time, one person at a time. As people engage, see authenticity in the process, see that the process is clear, transparent and above all genuine, then they will start to trust it. As you build trust, you build social capital. These people become your best ambassadors. But you can’t force this on people or make it happen, you have to trust the network do it… and they only will if you are authentic, so fake it and you lose!

*Can we learn anything from the private sector about listening to external audiences?
Everyone can learn from everyone else. But what’s the question? Yes, I believe public engagement can learn from other industries that engage the public, certainly from brands… building relationships, engaging the customer in a conversation (not just when you need them but over time) and also reacting to customer feedback (commercial brands have to because they lose sales and therefore profits are hit… they have an incentive… maybe we have to incentivise public service through measuring satisfaction with their ‘brand’”

*What is the actual role of technology (if any) in participatory processes?
It’s a tool, it mediates. It makes engagement possible over time and space, more people, different places. It speeds things up (not always good as at creates expectations that can be hard to manage). Digital supports off-line too by being able to aggregate conversations and disseminate data. But it’s just a tool…

I strengthen democracy and transform engagment with a strong dose of digital.

Wow excellent Dr Andy. What I liked most is "building trust". Huge task indeed when we see corruption everywhere.

ricardomatheus's picture

I simply loved all of the views showed about Canada, Brazil, United Kingdom and Estonia. Certainly that I agree with the initiatives found by all of you, mainly which use digital ways for public engagement (consultation, voting, transparency/social control, etc.). Going to the core of the question that Tiago Peixoto done, my view is that we need to recognize the "new governance" emerging with and without government. Sometimes, there are several initiatives that came from civil society and change reality - bottom-up initiatives? - and other that came from governments - top-down initiatives?. We have a lot of initiatives that have its own way of doing the things, and it is not good to how measure that. Sometimes several of them mix the digital processes with face-to-face initiatives as participatory processes in Brazil.
This is not easy to do, specially when we do it across the world. We have a lot of cultural and societal differences that must be considered on the index production, for example. However, if I could do a small framework to check the success, I would think about some dimensions that touch on:
1- Rights (Right to be heard, Right to participate on public management, etc.) If they don't have the right, it woul be difficult to do things;
2- Public Policies (Who is doing? The government or the Civil Society? Does it makes difference?);
3- Data Provision (There is data to be used? Open Data Portal, Transparency POrtals, Static pages, spreadsheets?) without data is hard to do social control, advocacy, benchmark...
4- Usage (Who is using? which kind of usage?) What I see in Brazil is that rich people using data to change only theirs own lives, for example, fix my street, my sidewalk, clean the downtown... while the people that really need change, cannot use data or even have resources (PC, internet) to do that.
5- Impacts (what are they doing with data?) It is good for citizens/government built a repository of initiatives. In Brazil we have the example of the webcitizen ( where they collect initiatives on the same space. from the perspective of governments, the open data portal of United States is a good example of repository of APIs on the same local helping people and citizens.

Ricardo Matheus
Bachelor in Public Policy Management - EACH/USP
Master of Sciences in Administration - FEA/USP
University of São Paulo - USP
PhD Candidate - TUDelft
Delft University of Technology, The Netherlands
+55 (11) 9 7227-7521

jfinlayson's picture

Ricardo, Thanks for sharing 5 success dimensions for citizen engagement. There are some interesting overlaps and differences with Lucas' 7 principals of citizen engagement - Preparation, Inclusion, Purpose, Openness, Trust, Impact, Sustained.

WebCitizen looks like a useful resource.

Quoting the website:
"Webcitizen is an innovative company that stimulates civic engagement and brings citizens closer together and to their governments and the private sector. We focus on the use of digital technology to create channels of participation, bringing more transparency, accessibility and democracy to public and private administration, promoting a collaborative dialogue, a meaningful sense of community and in a final analysis, helping to create a better world."

Can you give an example of a project and outcome of citizen engagement using WebCitizen?

ricardomatheus's picture

Hello, jfnlayson

Webcitizen is online since 2010 and we already have some articles, in portuguese, describing it and some impacts of the website on civil society and governments.

I think you had the idea that webcitizen is a collection of websites and face-to-face actions provided by civil society. So, it is important to highlight that we have several kind of projects towards citizen's rights (social control, access to information, right to participation, etc.). For example, the project "Cidade Democrática" (Democratic City) try, from civil society movement, to do accountability and promote dialogue between civil society and governments by advocacy on internet. The first example was a kid with 14 years old and his post "We don't have bike lanes in our city (Jundiaí)". After that, people who was connected on the website started to ask themselves "why don't we have bike lanes?" and they were to the Government (Executive and Legislative) demanding for more bike lanes.

Another municipality was deeper. They used all the contributions from civil society on the website as the Participatory Budget (Peixoto, 2008 and Matheus et al 2009

Further, another instersting collection of similar projects and actions in the world are on Technology for Transparency ( You could check more here:

Ricardo Matheus
Bachelor in Public Policy Management - EACH/USP
Master of Sciences in Administration - FEA/USP
University of São Paulo - USP
PhD Candidate - TUDelft
Delft University of Technology, The Netherlands
+55 (11) 9 7227-7521

jfinlayson's picture

Thanks for clarifying, Ricardo, and for sharing these examples of initiatives in citizen engagement.

In our work in the US, we are always trying to close the gap between the initiative leaders and the citizens they wish to engage. The first thing to understand is individual citizens' experience of the issue in the moment and at every point of engagement. Leaders too often become impatient, jump ahead and leave folks behind. Once we know where people are in terms of their understanding of and commitment to the issue, then we can begin to apply a principle we call "leaving tracks" ( which helps people learn about the issue, the context, the possible solutions and impacts, etc. so that they can find their own connection to the issue and become meaningfully involved.

JMills's picture

Amen to Edward Anderson's points above regarding measurement. I run the American Institute of Architects' Design Assistance Programs, which pioneered charrette processes (here's a short film on one of the programs - and which have the benefit of over 45 years of practice -- we have projects that are still guiding decisionmaking 3 decades later, in some cases, which places an emphasis on thinking long-term about success. Typically, beyond the immediacy of measuring levels of participation and satisfaction, as well as early actions, we try to view a project on a 2-5yr timeline, knowing that it will continue to evolve after that, particularly since our process is focused on community design and often involves long time horizons. Every community moves at its own pace, but you can see the ingredients for success (civic leadership, cross-sector partnerships, engagement, grassroots action, etc) in all of them early on. Once those ingredients take hold, civic momentum and accomplishments begin to build - one community has even described it as the "snowball effect" ( These impacts can continue to build in a nurturing cycle over time - here are two examples from Austin, Texas (20 years later) and San Angelo, Texas (again, 20 years later):

Of course, in some cases, success is obvious almost immediately. Two recent cases from our work include a rural community of 5,000 in northern Vermont that catalyzed over $250 million in new investment and over 2,000 new jobs in just a couple of years,( and a post-disaster community in Birmingham, AL, that has built a community-wide collaboration and leveraged tens of millions of dollars since 2011:

Joel Mills
Director, Center for Communities By Design
American Institute of Architects
(202) 626-7405

EdwardAndersson's picture


Thank you so much for mentioning this. It is great to see the commitment to tracking the long term impact of engagement processes. It is rare for there to be an organisation with the focus, stability and resources to do this long term, and as a result we often overlook important long term benefits.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

cardososampaio's picture

Very good discussion, folks. Nevertheless I would like to stress some points that are not presented yet and were asked by Tiago.

1) CIVIL SOCIETY’S ROLE: As Brazilian participatory experiments show, civil society’s group can really be helpful for engaging people. Thus these groups could very important for “sustained engagement and participatory culture” as Lucas Cioffi has mentioned, but on the other hand they may not be the best choice to enhance “Demographic Diversity” in certain situations.
Thus instead of saying that they should or not be included, I would follow Mr. Andersson’s defense that depends on the objectives of each participatory venue.

2) TECHNOLOGY: I do not believe that we can think anymore of new participatory venues without technologies. People get informed online; they discuss and share with friends online; they buy their goods online. Why shouldn’t they participate online? Thus is not about “solutions” offered by ICT, rather it’s natural for a lot of people, especially youngster. Of course this addition, especially when dealing with poverty, may complexify the process.

Someone has talked how the best approaches use both offline and online phases. I agree this is probably the best solution in this specific case. Still one needs to design the process in a way that the discussions, proposals, decisions and whatsoever made in each environment (online or offline) don’t compete against each other. For example, in Brazil some face to face participatory budgeting programs have included online phases (e.g. Recife: or online editions (e.g.: Belo Horizonte: A citizen can then vote the works to be done online, but the list had be done in face to face assemblies.

But this is not only about internet, but also cellphones. We also have some good experiences of participatory budgeting that use SMS for voting (e.g. La Plata: In the same way, good apps can be designed to allow people to send inputs to governments. For example, a citizen could send to city hall a proposal for budgeting (or general participation issue) using his or her cell phone with a photo and geo-localising the issue/problem.

In my opinion, if well used, ICT can be inclusive. They can be used to reach a higher number of citizens.

3) PUBLICITY: finally, one important issue that should be more discussed is publicity and I mean both senses here. First, I’m thinking of publicity as advertisement. In my opinion most participatory institutions and program lack more advertisement. In several occasions people don’t participate, because actually they didn’t know about it. Publicity is usually not recognized as part of the planning process, but it should.

Here I am saying about really involving the Communication sector of governments or even NGO’s to spread the word. Before being willing or not to participate, the most important job is to let people know. Both mass and online media should be used in this sense.

MASS MEDIA: here I am saying about both paid publications and ads to reach more people as media coverage. Both are important to effectively let people know and to increment the discussion of all concerned citizens. People in charge of Public engagement programs often don’t interact with journalists or media corporations, because they are afraid of negative and biased coverage. I would rather suggest the opposite. Inviting journalists to the event. Sending them good reports of the main objectives (before), of happenings and discussions (during) and main results (after).

NEW MEDIA: nowadays it’s also important to consider some kind of advertisement and especially engagement through social media. Nevertheless the people in charge generally think that the program only needs a Facebook page and a Twitter account and that is it. Actually it’s necessary to really involve people and letting them feel as part of the process. Obama’s two online campaigns are good proof that people can get involved in politics, but in different levels. Some are really up to do everything they can for the campaign, while others are less willing and are just going to send some emails or to share in their social media website. In both ways, the campaign spreads and can reach more interested citizens. Civic engagement programs rarely have explored this possibility.

Second, I’m talking about publicity as real open processes. Again some technologies could be used to make these processes more ‘public’. For instance, Porto Alegre’s PB is using cellphone apps for broadcasting (live) the assemblies. They are also uses internet to promove a lot of fiscal transparency on their project. One citizen can really find a lot of information on the different participation processes, their works, the people in charge of each project and whatsoever ( And this has been connected to other platforms of online monitoring of city hall. So one can not only participate, but also to monitor the process through ICT. And multivisual tools (e.g. maps and graphs) can be used to facilitate and increase this use (e.g. Porto Alegre:
For opening the process, I am also thinking of making available information to participate, minutes of meeting, videos of the meetings and videos made by citizens and whatsoever.

Rafael Sampaio
PhD Candidate, Federal University of Bahia, Brazil
Associate Researcher of Center of Advanced Studies on Digital Democracy

Rafael Cardoso Sampaio
PhD Candidate in Universidade Federal da Bahia, Brazil

EdwardAndersson's picture


You raised a very important point, which is often overlooked, which is the role of the media. Traditional mass media holds tremendous power around engagement, in terms of making people aware of processes and this driving participants to them, but also in terms of framing how the public perceive it. I've seen cases where a newspaper has had an interest in an issue and has gone out of its way to discredit a citizen engagement process where it disagreed with the results. In other cases certain parts of the media are mainly interested in stories about conflict and do not find engagement processes focussed on consensus building and solutions 'newsworthy'.

Social media also has a key role. Both positively as a route for participation and mobilisation and also potentially a more destructive route. I remember one case in the UK where a prominent political blogger encouraged his readers to go onto a government wiki based consultation site and bombard it with nonsensical information.

My experience is that many practitioners in the field of citizen engagement are not used to dealing with mass media and thus struggle to make the most of the potential in this area.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

Paolo Spada's picture

I agree 100% with both comments.

1) Democratic Innovations that aim to induce unrestricted public participation are basically a massive advertising campaign plus a participatory decision mechanism. Unfortunately really few researchers of Democratic Innovations are advertising experts and thus we know very little of the effects of advertising on participatory decision mechanisms. To make a concrete example current knowledge understands the unusual participation patterns in some Brazilian Participatory Budgetings in which poor people and women participate a lot as a structural effect of the design. More or less the consensus is that the city allows participants to decide how to allocate public funds, the result is that the Brazilian middle and upper classes do not participate because they do not need to, given that they live in better parts of the city and they can access private substitutes to public services. When I started doing my fieldwork in Brazil I quickly realized that 99% of the advertising was all targeted toward poor people independently of the party in power. Note that I am not trying to say that we should not compensate the traditionally low levels of participation of the dis-enfranchised with additional reaching out procedures and that structural forces are not at play, what I am trying to say is that the way advertising is shaped has fundamental effect on participation and has critical evaluation implications. We have reached an incredible attention to the minimal details of the participatory decision mechanism, while instead there is almost no research on the impact of different advertising/reaching out methods on participatory outcomes. We risk to say very problematic things if we do not start to consider advertising part of the design of a democratic innovation.

2) I agree completely with Edward regarding the relationship with media. And again I think that, as we should consider part of the design the reaching out strategy toward potential participants, similarly we should consider part of the design of a democratic innovation its media strategy.

Paolo Spada,

Democracy Fellow

Ash Center for Democratic Governance and Innovation
Harvard Kennedy School of Government


Vineeta's picture

In my view while it is important to assess the impact of participation, a more difficult but essential aspect is sustained engagement not only of the same set of audience but also with geographically disparate audience on the same subject. For example, before the launch of the MCA21 mission Mode project (MMP) under the National e-Governance Plan, over 50 stakeholder consultations were planned and executed across India. The nature and concept of these consultations changed as the project matured.

The second issue in my mind is participation by proxy. Often civil society organisations or business associations may be seen as representatives of their respective constituencies. As a result at various engagements familiar faces or 'usual suspects' can be seen voicing all too familiar views.

The third issue is that of use of web based technology. May be a contrarian view but, while web based interactions may be fairly common place in developed economies, in India where internet penetration and digital literacy is low, extensive reliability on technology enabled interactions may not be a very effective way of sustaining engagement.

Vineeta Dixit
Principal Consultant
National e-Governance Division
Department of Electronics & Information Technology
Ministry of Communications & IT
Government of India

EdwardAndersson's picture


I'm glad you raised the risk of 'participation by proxy'. As a field we really need to become more sophisticated in our analysis of the value added by civil society organisations and business associations. In many cases the insights and evidence they provide is crucial and would either be impossible, or very costly to get through directly engaging with citizens. As Ronak pointed out, adopting a 'distributed' model of engagement, where we rely heavily on proxies, can often be a good idea (in particular where trust in government is low).

However my experience of the proxies is that the degree to which they carry out internal engagement with their members or beneficiaries varies hugely. Some NGOs have very sophisticated models for engaging with and understanding the views of those they work with. Others have a far more paternalistic approach where they make bold statements about what those they work for need without anything to back it up.

Over the past decades pressure on government and the private sector to be more transparent about their decisions; I believe it is inevitable that society will expect civil society organisations to be held to similar levels of accountability.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

JoanneCaddy's picture

Building on Edward’s comments on measuring success in public participation, and the many excellent points which have been made by others, I’d like to offer a few observations based on OECD work in this space.

Back in 1998 – in what now seems the very distant past – a number of OECD member country governments launched a comparative analysis of how citizens could interact with their governments in shaping public policy. Published in 2001, Citizens as Partners described the framework conditions (laws, policies, tools) under which citizens could exercise rights of access to information, take part in government-led consultations and seize the initiative in shaping policy agendas and options through public participation. It also gave rise to a handbook of the same name which offered practical tips for government officials wanting to take concrete action.

Most crucially for this discussion, “evaluation” was one of the 10 guiding principles for successful information, consultation and active participation in policy-making. The report noted that evaluation was needed, “in order to adapt to new requirements and changing conditions for policy-making.” This formulation underscores the use of evaluation not as tool for audit or monitoring, but as a means of improving performance over time and meeting rising expectations.

Having established that there was very real “evaluation gap” in this emerging field of practice, in 2005 we set out to unpack this further in Evaluating Public Participation which drew from initial experience in a range of countries and policy fields.

When we returned to review countries’ progress in implementing the 10 guiding principles a decade after the initial report, we found that this evaluation gap was alive and well. This led to a strong call in Focus on Citizens: Public Engagement for Better Policy and Services for countries to invest more effort and attention to evaluating public participation in order to raise standards and improve practice.

But will they? In an age of fiscal restraint, government officials are certainly under pressure to account for how they spend limited public resources and the value delivered by public engagement. What they need most of all are effective and flexible off-the-shelf tools, the resolve to embed evaluation from the outset and a commitment to improve their practice based on the results.

There is no doubt that this exchange will help move us all towards that goal.

Joanne Caddy
Senior Policy Analyst - OECD Directorate for Education and Skills

e: w:
twitter: joannecaddy linkedin: joannecaddy

rsutaria's picture

One of the greatest participatory and democratic initiative of our times has been the evolution of the "Internet". There is a lot to be learned from how the underlying technologies of the Internet have evolved over the past twenty years and the way it has impacted and enhanced the global participation.

I feel there are two key learning that can be taken from the way the Internet has evolved to enable the engagement of a global audience. The first is the use of open and "interoperable" standards for communicating and engaging with the larger community. The second learning is that for complex tasks/engagement initiatives, it's best to have a "distributed" approach to the initiative/programme execution.

I'll try to explain in brief, the two points I'm trying to make here:

1. A "distributed" approach to citizen engagement essentially means that several independent and parallel citizen engagement initiatives are run by different different civic organizations/NGOs/expert teams to undertake overlapping tasks and results. While this certainly leads to duplication of efforts and redundancy in the work done, this is an extremely effective way to undertake a large citizen engagement initiative such that it provides reliability as well as scalability. The distributed approach to performing the citizen engagement means that even if one or two initiatives end up having biased or faulty results or do not execute correctly, it does not jeopardize the entire citizen engagement initiative. The distributed approach has enough redundancy built into its approach such that the engagement initiative can survive a few failed organizations/executions.

2. An "interoperable" standards based approach to citizen engagement initiatives ensures that when there are large number of organizations working towards a common goal, the (partial or completed) work done by some of these organizations can be built upon or reused by other organizations working in the same development sector.

Taking a cue from the way the Internet has evolved, one of the key reasons for the global participation has been the underlying standardized communication protocols on which the internet has been built. I feel that if a lot of the global citizen initiatives could "inter-operate" with each other, then we could see a lot of citizen engagement initiatives benefiting from each others work.

An example of what I'm trying to explain is as follows: During a recent "Participatory Planning" workshop by BMW Guggenhiem Lab a citizen engagement initiative was launched to get Pedestrian and Traffic Suggestions from the citizens of Mumbai called Karmayog PATH. While the traffic issues faced by each city are unique, I'm certain there are other cities in the world like London, New York which are also facing severe traffic issues.

I have always wondered if it was possible to design these citizen engagement initiatives in such a way that the suggestions and solutions to problems made in certain citizen engagement initiatives could be easily accessible to other engagement initiatives in other parts of the world. For e.g. if someone has identified a way to define and solve a particular traffic problem in London, the solution should be made accessible in an automated way to similar problems all over the world. Since I was the designer and developer for the traffic related citizen engagement initiative in Mumbai, I have always been wondering on how to make this initiative more inter-operable with other such global initiatives.

While I understand that the suggestions I'm making for the citizen engagement initiatives might be more relevant for the "Open Data" discussion that seems to have happened on this site a few months ago (I just came to know of this site today, when I got a link to this specific discussion), but I do feel that the above points are not specific to open data but is more related at the programme level strategy of a citizen engagement initiative.

Ronak Sutaria

Principal Researcher for a Citizen Engagement Initiative - Karmayog PATH

Paolo Spada's picture

Ronak your point is definitively not restricted to ITC participatory/engagement applications. Redundancy is at the core of most democratic innovations when you consider them in their context. After all we already have mechanisms to decide the budget of cities or to make the master plan of the city, thus a new participatory process is redundant almost by definition.

Apart from that basic intrinsic element of redundancy, most designs of face to face participatory mechanisms have redundant procedures from the macro level (in participatory budgeting, PB, you can participate in multiple ways), to the micro-mechanism of deliberation (e.g.; in deliberative polls or America Speaks meeting you have plenaries and small groups, moment of expansion of ideas and then moments of selection of ideas, that repeat multiple times).

BUT it is important to qualify a bit the concept of redundancy. A redundant circuit is a circuit that starts when the primary one fails. (Un)fortunately for institutional designers in democratic innovations we are not talking about electrons but people. Redundancy might generate a variety of problems in democratic innovations. If I offer a participatory face to face process with a voting mechanism and I activate a parallel online voting mechanism the risk is to create a legitimacy crisis when the voting starts and thousands of people vote online without having discussed the issues, but this is just one example. In Porto Alegre when they activated the thematic assemblies they created a pure parallel process, and the result was that nobody used the thematic assemblies to propose more long term projects as the designer was hoping, and instead people first tried to pass their project in their district assembly if that failed they tried to pass it in the thematic ones exacerbating even more the NIMBYness of the process.

Creating parallelism with the idea that the more the better is risky, you might get lucky or not. And please note that I am not at all claiming that you are suggesting that, I am just playing the devil advocate to expand the nuances of the discussion because I think you are right on the money and successful democratic innovations are successful example of well designed redundancy.

Getting back to the risk of redundancy people might chose the path that generates the most rewards at the minimal cost defeating potentially a lot of capacity building goals of the designer. People might simply decide that it is a waste of time to participate and free-ride. Free-riding is a critical issue to consider because it affects inclusion and again legitimacy of decisions of the people that keep participating. There is a fundamental trade-off between the amount of redundancy in a democratic innovation and the possibility of free-ride. We have a myriad of examples of individual level participatotory mechanisms that can at best achieve the exact same result of the existing consultations with community leaders and NGOs and that are not sustainable in the long run due to free-riding. All the popular consultation mechanism built after the 68 (e.g. the italian Assemblea di Quartiere "district assembly") around the world died the slow death of failed participatory processes. I am sure that many of us are familiar with useless meetings were the only ones that keep participating are old people that crave interactions, crazy people that love to hear their own voice, and people that are in just for the snacks. Such venues dis-empower the whole engagement effort and might have perverse effect on other parallel venues that are instead effective by lowering their efficacy and legitimacy by simple association. "I do not participate in PB because the district associations never did anything, and PB is basically the same thing." Change district association with the appropriate local term and this becomes a classic comment that you get all around the world when promoting PB in situation where there are parallel similar processes that are dis-empowered.

Also by creating a new venue you might dis-empower an existing one that is actually working well and that is why it is fundamental to know A LOT about the local context and why transplanting designs and solution is so though. As soon as PB started working well in a Brazilian city that I studied in my fieldwork, the Mayor, that was against PB, introduced a parallel set of meetings conducted the day before each district meeting in which he was promising public work. These meetings killed the participation in the second year of PB and the mayor cancelled the process. This is obviously an extreme case in which there was a boycotting intent, but even if there is no such intent less extreme, but still quite damaging outcome might emerge.

Concluding, the design of redundant (distributed, diversified, differentiated...) venues of participation/engagement is a critical aspect of democratic innovations of any type, but it needs to carefully consider the incentive of participants and non participants to promote overall empowerment, reduce free-riding and limit problems of legitimacy. An approach to the design of democratic innovations that we (me and Allegretti) are tentatively calling empowering redundancy. But we are looking for a better name because empowering is a bit wishy washy. So suggestions or existing definitions that can be adopted are super-welcome.

A proto example in face to face processes is given by the south district of Porto Alegre between 2006 and 2008 that improved the number of participants a lot when it devised a set of rules for which parallel modes of participation were rewarded with different possibilities of influencing the process creating a system of incentive that rewarded participants that spent more effort by giving them more agency in the proposal of projects. The limit of such proto-example was that it generated an oligarchy of participants, and reduced spontaneity. Thus an amended version of such governance procedure would include turn-over mechanism and maybe residual random mechanisms of selection of projects (i.e. out of the 5 projects selected in the district four are voted, 1 is randomly selected among feasible ones) to promote spontaneity and prevent oligarchies to lock the process.

Another example that applies methodologies currently used in many democracy games to hybrid face to face and ICT processes is to introduce the ability to vote online on projects only after you have earned enough "credits" by doing a variety of online or when technology will catch up also face to face activities (e.g. passing an online capacity building exercise, commenting in a forum one of proposals, sharing ideas with x people on facebook, sending x email to invite people to participate, Geo-referencing x-processes...) that guarantee more legitimacy to the online vote and its integration with the face to face one. This way we empower and give legitimacy to the citizens that decide not to come to the face to face meeting and vote online. In the process we are removing the cost of physically coming to the meeting, but we are maintaining the right to vote in the participatory process linked to some costly acts of participation to prevent complete free-riding on information gathering.

Paolo Spada,

Democracy Fellow

Ash Center for Democratic Governance and Innovation
Harvard Kennedy School of Government


EdwardAndersson's picture


I agree that a distributed model has a lot going for it. In particular in cases where the cause of the challenge is distributed as well (for example around Climate Change). I co-authored a pamphlet on this, so rather than repeat my arguments here I'll just link to it:

You also mentioned that many citizen engagement processes provide ideas and information that is valuable to others besides those who organised and ran the process. We need to get better at making sure that the findings of citizen engagement is used more widely - in many cases there is significant duplication of effort where the same citizens are asked similar questions by different people, simply because the work is not coordinated. That said, the challenge of creating a searchable information hub that would allow people to find and make sense of citizen views from across the world is considerable.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

rsutaria's picture

Paolo, Edward,

Thank you for your replies.

Paolo - The point I was making about "distributed" citizen engagement was slightly different than the "redundancy" approach that you have outlined. The analogy that you've given about redundant circuits becoming active when the primary circuit fails is true. But in that context, a distributed approach I mention about is slightly different than having redundancy or a fully-parallel execution or initiative.
To give an technology related analogy (from recent times), a distributed (or to use a more recent buzzword - "Cloud") computing approach basically involves breaking up tasks into multiple parts and then distributing those tasks to multiple clusters or nodes of computation. A technology used by Google, Amazone, etc. everyday (called MapReduce) essentially involves distributing tasks t1, t2, t3, t4, t5, etc. to a cluster of nodes called n1, n2, n3, etc. The distributed approach then assigns tasks t1, t2, t3 to node n1 and tasks t3, t4, t5 to node n2 and so forth.
Basically, task t3 is performed twice and similarly each task is performed more than once, but no two nodes perform exactly the same tasks - and hence there is an overlapping of tasks. Now, if any particular node fails- it will not result in a loss of task completion.

This model has allowed for near infinite scaling of computation and has resulted in what is now known as "cloud" computing. It is a very simple concept but when executed correctly it works very beautifully and can allow us to solve problems which otherwise were not economically possible to solve in a short duration of time.

I'd like to clarify that I'm not at all suggesting that citizen engagement initiatives involving real people is the same thing as how some obscure technology algorithm might work- but I've seen a number of extremely large scale citizen engagement being run in a centralized mode with an apex decision making body. The computing industry has realized that a centralized approach is a recipe for disaster and can never scale effectively. In fact, more often than not they grow so big that they become "too big to fail" and need constant influx of bailouts from citizens taxes.

To give a real world example of what I'm mentioning above is this:

The city of Mumbai, where I live, has a population of 18 million people (this is more than that of countries like Netherlands, Greece, etc.) The number of Vehicles in this city is about 1.8 million. It is needless to say that there are immense traffic and pedestrian related issues, eventually leading to severe accidents and fatalities. My sense is that it is completely impractical to expect that a single organization (the Mumbai Traffic Police) to come up with ALL the solutions for traffic related problems that are afflicting the city. Citizen engagement for these issues is absolutely imperative. My feeling is that we need a few hundred simultaneous citizen engagement initiatives trying to identify the critical problems and then to come up with solutions. There are going to be a large number of experts from different domains (urban planning, infrastructure engineers, technology experts, social sciences experts) all who can work independently as separate autonomous groups working on such civic issues. But what is needed is that at the end, the results of all these independent groups needs to reduce to a common action which can be implemented. In the city of Mumbai, I do see a large number of diverse groups working on related issues (which I call the "Map" phase), but what is sorely missing is the "Reduce" phase of the output , where the results can be actionable. The reason I feel this happens is that organizations are not designed to be autonomous and at the same time inter-operable with each other.

I feel if this can happen, we can scale the citizen engagements in a very organic and yet effective and organized way. This can also allow the individuality and creativity of the various different groups intact and yet, bring their results and output together to solve real problems.

Edward- the "Talking for Change" document that you've linked to seems to be very informative and relevant. In fact, I was involved with an initiative called "Democracy Dialogue" where there were 185 participants from 32 countries, including the Vice President of the newest democracy 'South Sudan'. Subsequently, as an outcome of that 4-day dialogue, an ongoing initiative has been launched by the IofC organization to facilitate trust building in the war torn country of South Sudan.

I will be reading the document that you've shared over the next week or so and I feel that it is going to be useful in some of the trust building and reconciliation initiatives currently underway.

Thank you for all the enlightened sharing!

Ronak Sutaria


Paolo Spada's picture

thanks for the reply.

I hope that my post did not come across like I was criticizing the idea of differentiating venues of participation or integrating them. My main point was simply to be smart at designing the structure that coordinates the different venues fully considering the incentives and limits of the people composing them.

Paolo Spada,

Democracy Fellow

Ash Center for Democratic Governance and Innovation
Harvard Kennedy School of Government


How can we measure the success of citizen engagement initiatives?
By measuring the change in sense of empowerment via surveys before and after the implementation of such initiatives.

How essential are processes of organisational and institutional change?
Adapt or die holds true for everything, not just evolutionary biology.

Can political will towards increased participation be stimulated?
Absolutely. They key is to make participation have an actual effect, unlike the current situation. Take US Congress for example. If lack of voter turnout for a congressional district automatically unseated incumbents...

What role does organized civil society play in citizen engagement processes?
At present, it serves to create the illusion of meaningful participation.

How can we foster inclusiveness and what are the impacts of different methods of participant selection (e.g. open, randomized)?
Fractals. Decrease the ratio of participant to representative, even if the result is a pyramid. Participants should be able to hold individual face-to-face discussions with their representation.

Can we learn anything from the private sector about listening to external audiences?
Not as much as we can learn about manipulating the external audiences perceptions.

What is the actual role of technology (if any) in participatory processes?
At present, a means of manipulation of public perceptions.

Paolo Spada's picture

"How can we measure the success of citizen engagement initiatives?
By measuring the change in sense of empowerment via surveys before and after the implementation of such initiatives."

With regard individual level measures of empowerment, the strategy Grizwald describes is the most common. We have currently a ton of pre-post surveys on individuals' perception, unfortunately they measure customer satisfaction with the process more than anything else. If an individual did not get the result he/she wanted, if he/she did not like the process for any reason he/she would often rate negatively everything.

We also have a ton of pre-post surveys that directly ask about behaviors that are associated with empowerment (like how many time did you contact your representative? Did you vote? Did you participate in a district activity?), but these suffer because people often lie. In these surveys it is clear that the researcher is trying to measure the level of civic participation of the respondent and that in some individuals generates overshooting/undershooting. This is a small problem in certain environments, but it is a bigger problem in development applications and in situations in which people think that funding of projects might be linked even only indirectly to survey responses. There is very little research on this, but there are a lot of intuitions of very smart practitioners and researchers floating around that highlight the difficulties of these surveys in development.

Similarly we have a ton of qualitative studies. These studies are extremely important for theory building, but it is really really difficult to use them for testing convincingly hypothesis. Also it is really difficult to use them to compare different cases if the researchers are different. Finally these studies are filtered through the experiences of an individual researcher and they lack the key element of replicability. Often multiple researchers write very different accounts of the same experience. Often the same interview protocol leads completely different result depending on personal characteristic of the interviewer.

Finally we have very few studies that capture observable behavioral outcomes that are proxies of empowerment, e.g. the number of civic associations that are registered in the city, number of protests, turnout... In this respect ITC applications have a huge advantage because you can more easily track behavior online, and the number of ITC projects that measure online behavior is constantly increasing. The problem here is to understand how these proxies map into our idea of empowerment and if online empowerment maps into political one. Keeping in mind that even if people are empowered in an observable sense (e.g. people that participated in the engagement process have formed associations that have obtained funding for local projects), they still might perceived themselves as dis-empowered. Keeping also in mind that if the engagement process itself requires a behavioral change for participation (i.e. you need to form an association to participate), then we need to make sure that we are not measuring a temporary blip induced by the process (i.e. such associations disappear after the process ends).

Also whatever individual level measure of empowerment we come up with it will suffer from a self-selection problem if we just study participants. People that do not like to participate do not come. So even in the case of randomized invites to participate, a procedure that is often claimed to increase representativeness of the sample we study, we still can't solve the fact that only some people when invited come. If we are just focusing our analysis on those that are willing to come to a participatory process then at best the results will apply to people that are willing to come.

You need a research design that uses a randomized experiment that has a control group to manage the problem of "non compliance" statistically, i.e. the fact that some people do not participate in the innovation. You can randomize encouragements to promote individual participation in an existing process, like we have recently done in Africa with SMS invites to participate in a PB process, and then compare various measures of empowerment in the treatment and control group. This procedure solves the non-compliance problem, but does not solve the fact that we have very rudimentary measures of empowerment. However getting such encouragement design right requires the ability to isolate at least partially the control group from the encouragement, creating a differential uptake in the control and treatment group, something that is often tricky to do.

So as was said initially in better words by Edward, it is a mess to measure empowerment and in general to do any type of impact evaluation of democratic innovations. And why the cheap-talk solution is to invoke a multi-method approach, the problem with multi-method studies is that rarely they are conducted by inter-disciplinary teams and it is extremely rare that a single researcher, or a single discipline team, is fully trained in the multiplicity of methods that are needed.

Paolo Spada,

Democracy Fellow

Ash Center for Democratic Governance and Innovation
Harvard Kennedy School of Government


EdwardAndersson's picture


I agree that measuring subjective empowerment is difficult. I wanted to flag up one British study which looked at this in detail. The previous UK Government used a number of national indicators to track local government effectiveness. One of these measurements was the percentage of people who felt they can influence decisions affecting their local area. However statistical research found that actually the key drivers for whether or not a local authority scored highly or not on feelings of empowerment were the percentage of local residents who belonged to ethnic minorities, practice an organised religion or had higher qualifications. Participatory practices seemed to have limited impact on the scores at a population level. Subjective empowerment is a variable which is driven by many different factors, most of which are outside the sphere of influence for citizen engagement processes.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

Giovanni777's picture

Great topic for discussion, Tiago! From my experience, the successful engagement of citizens that creates innovative solutions and/or leads to better decision-making requires the following elements:

--Prioritizing Problems: Identify and prioritize business/mission problems by size and intensity through ideation and crowd-sourcing.
--Solutions Teaming: Encourage teaming of entrepreneurs, developers, experts, and government entities to develop the solutions to the problem.
--Funding and Prizes: Attract investment community attention to business/mission problems and stimulate development to address the problem.
--Marketing and Showcasing: Draw attention to the business/mission needs and showcase solutions to the problem.

You also need a proven service delivery approach that provides a disciplined, flexible and customizable method for managing dialogues and feedback efforts to engage the public and stakeholders, and improve the policy-making process.

Director, Global Public Sector
Grant Thornton, LLP

Robert L Short's picture

I have not learned of the WB Feb 28 ICT Days summarization as of yet but, did attend the meeting online. In lieu of their final summation, I am pleased with their discussion regarding the strategy for progressing forward. One of the matters discussed was a value system. In addition, other considerations that support a Web 3 Progression were discussed. Also, as part of the ICT Sector progression to Web 3, value theory is essential to value good which in this case is “Public Good”.

Please note that the progression to Web 3 is also the birth of knowledge based intelligent systems and is a turning point in the evolution of the way humans interact with computers. One of those considerations regarding our evolution is dependent upon Crowdsourcing of governance issues and policy. Our evolution has been and will continue to be subsequent to our technological advances. Our economics are subsequent to deployment and capital investment of such technologies.

Why a value theory system for “Public Good” should be at the core of Crowdsourcing? Simple answer, value theory is a science and as a science the results of Crowdsourcing any topic matter does not require human interpretation. On good example is this Citizen Engagement. We as citizens are dependent upon your interpretation as to my input on our needs. Whereas value theory coupled with Crowdsourcing summations would play an important role in the evolution of a true democracy. Crowdsourcing “Public Good” is the first stage of a democracy based upon an individual representation and not allegiances to any political affinity.

In the process, once a Crowdsourcing dialog has been acquired, a value assessment created by a certified professional, would be presented to gauge the results.

Vineeta's picture

As more and more government services move from in-line to online, the challenge of moving services seekers on the same path remains. In a country as diverse as India, where most government services are accessed at district and sub district level (Centre, State, District, Tehsil, Block, Gram Panchayat), further complexities are added due to predominance of rural population, multi-lingual society and limited digital literacy. In such a scenario, the need of the hour is to create an opportunity for hands on experience as well as grassroots level awareness campaign to engage with potential service seekers directly.
Under the National e-Governance Plan (NeGP) over 100,000 Common Services Centres (CSCs) are being established to provide government, private and social services to this target population, residing in over 600,000 villages. However, in a country where people are used to going to government offices to seek In order to create awareness public services, creating trust in these private owned outlets is a difficult proposition. Simultaneously, the trust needs to be built between the government functionaries and village level entrepreneurs to ensure availability and reliability of services.
In order to generate demand for services to ensure the long term sustainability of these centres, a massive grassroots level awareness campaign has been launched across 25 states which will cover nearly 3000 CSCs. This year long campaign is targeted to touch over 300,000 people and is expected to be a critical demand generating mechanism at CSCs.
While such methods of demand generation have been done in private sector, for governments services this is a fairly innovative approach given the fact that providing the ‘touch and feel’ of public services is often a complex and time consuming process. More importantly, it is expected that use of online services over time will also result in higher levels of digital literacy, participation in formal economy resulting from financial inclusion and contribute towards building capacities of the rural population to participate in knowledge driven economy, leading to further demand generation for these and many other types of services thus creating a virtuous circle of learning and demand generation.

Vineeta Dixit
Principal Consultant
National e-Governance Division
Department of Electronics & Information Technology
Ministry of Communications & IT
Government of India

EdwardAndersson's picture


Very useful to have an overview of the efforts that the Indian Government has taken to make sure that ICTs are available to the wider population. I've found that the debate about the right use of ICTs in work with communities affected by poverty is very polarised. On one hand we have e-evangelists and consultants who often oversell what is possible to do and on the others we have some face to face organisers who dismiss any online engagement with the argument that 'poor people don't use the internet'.

In reality access to and use of ICTs is incredibly nuanced and complicated. Just because someone has access to the internet does not mean that they are confident or willing to use the technology to engage. In general I think we overestimate what is possible to do with online engagement in the short term and underestimate the impact in the long term. To give one example I think that the way that translation and transcription software is developing leaps and bounds means that it will soon be possible to engage certain groups who presently are excluded online due to language barriers, low levels of literacy or dyslexia.

Ids be interested to hear what experience others have of online engagement.

Edward Andersson
Deputy Director - Involve

e: w:
twitter: ed_andersson linkedin: Ed_Andersson

I think in India, we do not have dearth of intellectuals and policies. What we lack is governance and national leadership with vision.

Stefaan Verhulst's picture

Re: How can we measure the success of citizen engagement initiatives?

To “do what works”, we need to know what works. Without a deeper understanding whether, when and why an intervention has made an impact, the potential of citizen engagement may remain untapped.

We need to move away from “faith-based” citizen engagements toward “evidence-based” ones.

Four short reflections to potentially deepen the current debate:

- Different horses for different courses. “Citizen engagement” comprises many different “means” to achieve many different “ends”. Depending on the context, citizens can play different roles: as providers of ideas and expertise (think of crowd sourcing, predictive analytics, grand challenges, prize-induced innovation, brainstorming, et al); or as representatives of specific interests (in the context of participatory budgeting, citizen juries and deliberative polling). And the contexts of engagement may differ substantially –from post-conflict zones to gentrified city blocks. The real task at hand is thus to become more sophisticated about what works, with whom, under what conditions, to achieve what objectives.

- Improving people’s lives. Too often, the indicators used to measure citizen engagement are only meant to quantify the level of engagement – such as the amount of people that participated or the volume of comments received. For citizen engagement to be meaningful and relevant, more effort is needed to measure real impact on people’s lives – the ultimate benchmark of success.

- We measure what we value. Indicators (particularly of the statistical kind) are sometimes presented as neutral or scientific tools of measurement. In fact, though, they are inextricably linked with values--i.e., we measure what we care about. Indicators are socio-political constructs. As such, shouldn’t we also consider whether and how to engage citizens in determining how citizen engagement is measured?

- Experimentation in how we measure. While much experimentation in citizen engagement is taking place, experimentation in how we measure citizen participation is limited. Interesting efforts do happen such as the capturing of citizen experiences to measure the value of a particular intervention or the use of big data in trial-and-error experiments. These efforts are starting to compliment the techniques used till date, yet more is needed. To make it a new discipline though, we need to start sharing lessons learned and integrating new methods more widely.

Hope these reflections contribute to the debate on how to become more evidence-based. They underpin some of the work we hope to do at the Governance Laboratory (, a new multi-disciplinary and multi-institutional initiative to change the way we solve public problems.

Thanks much
Stefaan (

gio.allegretti's picture

I would like to answer quickly to one of the questions posed in the launch of this debate, quoting the book “El círculo virtuoso de la democracia: los presupuestos participativos a debate” by Ernesto Ganuza and Francisco Frances. In fact, I think that the authors were able to focus on an important participatory tool (in this case participatory budgeting) showing how results depends from results, in a progressive and incremental perspective of qualitative growth…
Somehow, it seems banal to say, but many results of participatory processes seems strictly dependent from the fact of having pursued those specific goals (or not). A typical example is that of “social inclusion” of weak actors and disadvantaged groups. Many thinkers imagine social inclusion as a natural output of participation, ma is it really so? Or is it a possible outcome only if specific measures and a proper design are shaped in order to make inclusion possible? I see a risk that participation could be a space for elite capture, or at least for reproducing asymmetries and exclusions that affect society “outside the participatory space”. So, the design of rules, and the way in which information and training are offered to social actors can become the only guarantee for disadvantaged social actors to have a real opportunity to access the participatory decision-making spaces with the same rights and conditions as others…
But is not only the “internal design” of a process that is strategic to attract and involve actors. The relationship with other processes of social dialogue and the general governance structure that leads to decision-making is also important. This is the case of some powerful stakeholders, which normally are not interested to take part to participatory arenas, if they can find other spaces of decision-making where is possible to negotiate with and influence political officers. That’s why is very important to reduce the “competition” that can take place between different parallel processes of social dialogue, whose parallel existence can harm one another, stimulating a sort of Darwinian selection of potential participants by the “channel of participation” which grants to them more positive results.
Such reflections suggest to think participation mainly as an “enabling environment” that can affect the governance and the management of a city in the same way that the governance and the management structure of a city can affect a single participatory process and its results. Let’s take the case of a process where the ICT4GOV program of the World-Bank took part, that of Participatory Budgeting in South Kivu Province, in the Democratic Republic of Congo. Here, many of the 8 local authorities that were stimulated to shape a participatory budget in their territory by the Provincial Government acted initially because that superior level of Government promised to distribute funds to them, unlike in the previous years. The Provincial Government had the task and duty to transfer these funding for implementing capital investments, but in the past had not done it, justifying this inertia with a lack of confidence in local government performance. So, in 2011, the guarantee represented by the creation of participatory budgeting processes in some municipalities (with some control granted by the Work Bank Institute and other partners) was considered enough as a warranty of accountability for realizing the due transfers from Provincial government to the 8 local governments. This new inter-institutional relation based on “mutual trust” became the start-up of a “virtuous circle” that bettered the performance of local authorities. In fact, they were “enabled by the transfers” to implement services and public works that fed the trust of their citizens, while giving them the opportunity to show to the Provincial Government that they could properly manage public resources in an effective and transparent way. The most virtuous municipalities (as the three Bukavu municipalities) increased the positive performance, adopting new governing measures that could positively affect the efficacy and efficiency of their policies: for example, two of them did not allow anymore tax-payment in cash, introducing bank-transfers as an important measure to grant accountability through traceability of money transfers… Participatory Budgeting acted as a very positive “enabling environment” for other reforms, that – on their side – retroacted on the participatory process, making it more effective and attractive for citizens In less virtuous local governments (as some of the five guided by traditional customary authorities), the first year outputs of Participatory Budgeting were public works which could create “god precondition” for a better management of the second year process: as – for example – construction of roads and bridges between rural villages, purchase of transportation vehicles for allowing public officials to be present in villages, and the construction of spaces for hosting decentralized branches of local government offices.
This story somehow explains the growing effectiveness of participatory budgeting in South Kivu between 2011 and 2012, and well depicts the possibility of a positive “mutual influence” of structural reforms of government and participatory reforms of governance. So, what Ganuza and Frances call the activation of a “virtuous circle” through a participatory innovation. I would like to use such a reflection to try to answer in a positive way to the question launched by Tiago Peixoto in his opening remarks of this Forum: “Can political will towards increased participation be stimulated?”.
In my vision, the answer is twofold: (1) Yes, scaling-up of participation could be a strong stimulus to political will of local authorities toward the opening of more participatory decision-making (providing funding or other incentives to increase their resources or skills, and so to better their performance); (2) political will of opening decision-making to participation of citizens can be self-fed by the parallel activation of governing reforms which could affect the effectiveness of outputs of participation, so making it more attractive for people and more satisfactory for the political authorities that activated it…
Poland today represents a wonderful field for verifying such a chain of positive effects, being that the Law called “Solecki Funds” (or Village funds) since 2009 made the national government activate almost a thousand of participatory budgeting experiences. They are year-by-year improving their performance, because the government reimburses to rural municipalities a percentage of the money used for implementing the priorities decided through the participatory process, in proportion to the size of that investment (according to a formula contained in the Law), and provided they grant some level of accountability and formal commitment (as indicated by the Law itself).

Giovanni Allegretti
Senior Reseracher at the Center for Social Studies
Coimbra University - Portugal

Olena Ursu's picture

Hello, everyone!

First of all, I would like to thank you for this very professional expert discussion which is timely and useful for me as we in UNDP Ukraine have initiated the project under EU support devoted to "Smart Practices for Oversight by Non-State Actors on Administrative Service Delivery". The project aims to improve the delivery of administrative services by local authorities in Ukrainian municipalities through better public monitoring and effective feedback from citizens. This will be achieved by strengthening the capacities of the civil society organisations to monitor and oversee the provision of administrative services in Ukrainian municipalities.

We really try to make it happen in practice, not just to create the illusion of meaningful citizens' participation wit6h the help of the civil society, as mentioned in one of the critical comments here.

Among our first steps will be the taking stock of the existing smart practices of the citizens’ engagement into the oversight of administrative services provision in Ukraine and worldwide. There are some great thematic local initiatives in Ukraine, e.g. “Secret Client” project in L’viv municipality within which the organised groups of volunteers attend the administrative bodies to get some service and then report to the municipality through the active local CSO with their recommendations for improving the quality of this particular service.

I am sure that you accumulated rich knowledge on the topic, which you could help us to reach. Please tell us about all experiences which you may recommend as a smart practice of public oversight on administrative service provision, be it Web 2.0 based initiatives or real-life exercises accomplished in your countries. We will be sincerely grateful for all thoughts and ideas you share with us.

My e-mail is

Thank you in advance,
Olena Ursu,
Project manager, UNDP Ukraine

Katy Fentress's picture

I would like to echo Vineeta’s point on the challenges of low-levels of digital literacy in developing countries and also underline that there is a difference between countries with high-levels of citizen participation as opposed to ones in which people are not engrained with a culture of political activism/participation. As such, I would like to stress that different approaches to encourage citizen engagement must be adapted to each unique reality on the ground and that there are no hard and fast laws that can be applied across the globe.

As an example of a current citizen engagement program, I would like to highlight how the Participatory Slum Upgrading Program (PSUP), a division of UN-Habitat, is currently test running an e-participatory slum upgrading initiative in the Kenyan coastal town of Mtwapa (
The program is being developed as a partnership between representatives of the Mtwapa community, local CBOs, NGOs, the Kilifi County Council (Mtwapa is in a constituency called Kilifi) and the Ministry for local government, using a framework that has been developed by the organization FUPOL ( - a European policy consortium that is pioneering a new e-governance model that aims to be driven by the demand of citizens and political decision makers.
The Mtwapa slum-upgrading project is focussed on framing e-participation as a tool for deepening democracy and citizen engagement in a country in which citizens often feel disenfranchised from the political process. In order to do so,
PSUP first began by mapping available services, existing stakeholders and ongoing initiatives with the help of local CBOs. This process was followed by a “Project Participatory Review”, which was accompanied by a series of workshops in which participants (most of whom had almost no experience using computers and the internet) were trained on social media tools, primarily Facebook, weblogs and email. With regards to monitoring and evaluation, the FUPOL webpage states that this is an ongoing process in which the community, local and national authorities are being involved.

Although the Mtwapa project is being run in an area in which people speak Swahili much more than English (both languages are official in Kenya but in the coastal areas people are more fluent in Swahili), when we visit the project’s Facebook page ( we discover that it is in fact only in English. Furthermore it is worth noting that although there have been a few comments made by locals on the facebook page, these all appear to be by “youth”. There is no specific information as of yet as to how the older generations in Mtwapa are taking to the initiative and it would be very interesting to know the extent to which they have embraced using computers to voice their needs and concerns.
So an important question at this point might be: how do we involve older citizens in an e-participation process? Is it possible to create a platform that can be used by all notwithstanding language, age and class barriers? Ronak’s point about the “distributed” approach might be applicable here. Are there any case studies where this kind of thing has been successful?

Katy Fentress
Nairobi Bureau Chief - URB.IM

widya anggraini's picture

It has been a very enlighten discussion for myself especially with the work of Involve and many other examples mentioned here. I was working with government in planning agency and dealt with participatory planning for sometimes. I understand that different countries has different kind of planning system and that reflected in how community doing their planning system.

Some countries institutionalized planning system such as in Indonesia where public meeting is a fixed agenda and carried out from the lowest level of government, which is village level. And yes, it consumes HUGH amount of money as we have around 78.000 villages, not to mention public meeting will continue at sub district, district, provincial and national level. It was a lot of work and money. And yet, people don't see it useful because at the end of the meeting, what people consider as priority is not reflected in development planning plan document and it means any project they consider important will not be happening. Therefore, apathy is one big problem in participatory planning system here. However, there are some cases with successful story of participation when it involved good political will from government who willingly and carefully listen what their audience want and make it happen. With such cases, I think it is important to build trust and show good intention to attract people to engage. Attention to local wisdom, dissemination of planning blue print, law enforcement, and officer’s accountability play an important role in building this trust.

Arnstein in her paper introduced the Ladder of participation, which show many gradation of participation. I think I will take Arnstein point about distribution of power that enable citizens involved in the political or economic processes. I think that is the reason why some community apathy as they do not have the power to actually participate in public meeting. They only show up and has no control to whatsoever the end result. Therefore, power distribution is need as guarantee that citizen will be involved at every stages of the planning.

Some of discussant mentioned about technology and I absolutely agree to use it as part of innovation in doing participatory. We can also use art and culture to engage people and many other tools for community designer to create event and methods to actually attract people to participate. There is so many successful and unsuccessful stories, and i am looking forward to find out more in this discussion about it.

Widya Anggraini
Jakarta Community Manager (

If we are shifting focus back to "political will" then we are back to square one!

Paolo Spada's picture

I think this discussion becomes more interesting every day, but I fear that in a few days it will become unmanageable due to the share size of the material.

Can I suggest to add a sub-topic level to this discussion and show just the title of the main posts in the current level, hiding all replies in the sub-topic?

Can it be done?

Paolo Spada,

Democracy Fellow

Ash Center for Democratic Governance and Innovation
Harvard Kennedy School of Government


jfinlayson's picture

Hi Paolo,

Thanks so much for the site suggestion. We appreciate the feedback, even if we may not be able to implement the changes during this conversation. I agree there are many great comments here, and I hope folks will continue to scroll and jump in.

By the way - we are tweeting excerpts from this conversation on Twitter @StrikingPoverty Please follow us! Some of the most retweeted comments so far (Note: each includes a link to the comment it was excerpted from) have been:

"Bad #citizen #engagement is worse than no citizen engagement at all, as it reduces trust." @ed_andersson INVOLVE
3 elements needed to engage citizens committed public officials, mobilized citizens, innovative design @cebrap Vera Schattan Coelho

"Most powerful approaches combine online & face-to-face engagement to mobilise previously silent voices" @joannecaddy

Fostering inclusiveness "takes time. One engagement at a time, one person at a time." "Fake it and you lose!" #edem @andy_williamson

3 mistakes in community engagement: 1. forget how long it took you to understand the issue. 2. assume... Greg Ranstrom @CivilSay

"Many thinkers imagine social inclusion as a natural output of participation, is it really so?" Only w/proper design Giovanni Allegretti

Thanks to everyone who is contributing great thoughts to this conversation!

TiagoPeixoto's picture

Hi Paolo,

I see what you mean and I understand the limitations. Unfortunately it seems that there's very little that we can do at this moment. But to the extent of possible I will try to - every once in a while - nudge the discussion towards a more structured dialogue.

Thanks a lot!

Thanks a lot!


Tiago Peixoto
Open Government Practice
The World Bank

Ivan Tasic's picture

This is very inspiring and useful discussion, thank you for that. I'll try to formulate some of my thoughts about the evaluation of citizens' participation.

I live and work is Serbia, where for the last 5 years different EU, US and UN instruments supported the issue of greater citizen's participation in the decision making process. Milionis of euros are spent but mostly for ad hoc citizens' participation projects. Citizens' participation is a buzzword, those projects are in, but to be honest we don't know how successful we were.

Here, I would try to outline the things I see as an important to be considered while thinking of measuring the success of citizens' participation

As for any other intervention we have to be aware of the purpose of citizens' participation and that should be the starting point of the evaluation. So if the goal of citizens' engagement is better decision making and better public policies, we should measure if the process led to this goal. Of course, it is never that simple, but at the end we should measure if citizens' participation had the impact on decision makers and decisions they made. Participation process per se is not a goal. So if we try to measure its achievements, we should have clear idea why we used it.

However, if we talk about measuring success of citizens' participation in the context of Serbia or Western Balkan (I believe that the situation is similar in most of the ex-socialist transition countries), from my point of vew, evaluators should expand their focus on at least three areas:

1. Environment/context. People in ex-socialist countries generally do not believe to their governments. Socialism used to finger participation but never gave true power to the citizens. In such context it is really hard to evoke citizens to participate. Besides that mechanisms for citizens' participation are not clear. Decision makers can include citizens in the process but they are not obligated to do that. All these and many other things nurture mistrust and undermine participation process. With all this in mind, my recommendations for evaluators would be to examine if participation process influenced on the environment. Are there changes in relations between citizens and decision makers? Did the process build trust among them? Are there now clear, easy-to-use mechanisms for citizens' participation accepted by both citizens and decision makers? I really think that these questions are important, especially if we want to establish participation as common practice in decision making process.

2. Process. Many of you have already pointed some important principals of good participation process. As Edward wrote we could easily agree on communication, information provided, facilitation and feedback as essential for good participation process. Of course, there are many other things we could also consider for evaluation. I would like to add accessibility of the process, its openness and inclusiveness. If the engagement of the citizens in decision making process is the way to empower citizens and give them mechanisms to influence on decision makers, evaluators should examine how and to what extent process included ideas, opinions and considerations of the most vulnerable among us (minorities, people with disabilities, rural women/girls etc.) and empower them to influence on the decision makers?
Some interesting tools and mechanisms were also mentioned. I believe that we all agree that there is no universal participation formula, so we have to be context sensitive. From my point of view, whenever is possible it is good to do prospective evaluation of the suggested method/tool/mechanism.

3. Decision making. I participate, you participate, they decide! Fake processes are the most challenging issue in participation. Good participation process doesn't mean that decision makers will consider citizens' inputs. As I wrote before, success of participation process is equivalent to the extent of its influence on decision makers and decision they made. Evaluators should have that in mind whenever they assess success of citizens' participation in the decision making process.

Finally, I want to ask you a question. Who should be responsible for the evaluation of citizens' participation? In countries like Serbia and other Western Balkan states, there is no country based evaluation system or authority. Civil society can do it, it can offer recommendations for better participation process but authorities are often deaf to those proposals.

twitter: @sociosolidarity

TiagoPeixoto's picture

Thank you all so much for your contributions on this issue of evaluation, which clearly helps shed more light on the matter and provides us with some very actionable steps towards better evaluation processes.

As many of you have indicated, the issue is more complex than it may seem at first. I will try to summarize the outcomes of this conversation in more detail by the end of the exercise, and submit it to the group for discussion and further editing. But in the meantime, I take the liberty of highlighting some issues that have drawn my attention in this first round regarding the evaluation of citizen participation issues:

• A first point to consider is the reasons behind choosing to start an engagement initiative, as different participatory exercises/institutions aim for different objectives/outcomes;
• Equating participant satisfaction or number of participants with the quality of the process is not, per se, an indicator of success;
• Evaluation processes should be flexible enough to take into account positive and negative impacts that may not have been anticipated in the first place;
• There is a need to move beyond short-term assessments, given that impacts may occur (or be better assessed) only in the medium and/or long-terms (I’d say this is particularly important when it comes to bold claims such as “improved service delivery”);
• A clearer differentiation between internal and external impact evaluations is desirable, as well as their respective goals, prospects and limitations.

Another point that caught my attention related to the challenges of actually combining multi-method approaches in a meaningful manner that provides comparable results and encompasses the different effects of participatory processes. I also noticed that the issue of randomized controlled trials as the gold (or only) standard for measuring impact—a rather current issue in the development world—was not directly raised at any point.

The issue of resources for evaluation comes across as a strong point that needs to be addressed. Good evaluation requires resources that go beyond a couple of people running questionnaires or writing superficial accounts of an experience. Allocating specific resources to evaluation at the time interventions are designed seems to emerge as good practice. Nevertheless, finding ways to reduce the material and immaterial costs associated with the assessment of participatory initiatives seems an important issue. The issue of “off-the-shelf” and “flexible” (should we say “modular”?) evaluation frameworks deserves further exploration, although such a perspective faces the challenge of the diversity of participatory initiatives.

This brings to light another point that was raised about the role that technology can play in reducing the costs associated with evaluations—something that may deserve further reflection. I can’t help but think of an example of a recent initiative of ours (Open Development Technology Alliance) during the participatory budgeting process in Rio Grande do Sul: it cost us only a few hundred dollars to collect amazing data from 23,000 participants in the online component of the process (e-voting)—a practice that seems to me quite scalable for supporting the evaluation of a number of (maybe not all) citizen participation initiatives that have an online component in them.

For now, these are my thoughts. But I would appreciate any help (by replying below) in singling out the key issues that arose in this conversation with regard to the assessment of participatory initiatives

Thanks again!

Tiago Peixoto
Open Government Practice
The World Bank