Indicator system to measure participation quality in Decidim.

Introduction

Democratic quality is a core principle of Decidim. To uphold this, we have developed a system of indicators that help evaluate the quality of participatory processes on the platform. This article explains how these indicators are calculated.

How Are Indicators Calculated?

We have defined two methods for collecting quality indicators:

1. Automatic Metrics: These are calculated directly from Decidim platform data.
2. Self-Assessment Questions: These are based on a form filled out by an administrator.

Each indicator is expressed as a percentage (n/total). This percentage is then converted to a score on a 1-to-5 scale:

• 1 = 0–20%
• 2 = 20–40%
• 3 = 40–60%
• 4 = 60–80%
• 5 = 80–100%

Automatic Metrics

These metrics are automatically calculated from data available in Decidim. They are evaluated per participatory process and can be made public by administrators, so all participants can access it.

We have defined four main indicators: Citizen Influence, Hybridization, Responsiveness and Traceability:

Citizen Influence

This indicator measures the level of citizen influence on the accepted proposals and the outcomes of the process.

• Citizen influence on proposals
  • Percentage of proposals accepted (n/total)
  • Percentage of approved citizen proposals over total of citizen proposals (total citizen influence on proposals approved)
  • Percentage of approved citizen proposals over total of approved proposals (relative citizen influence on approved proposal)
• Citizen influence on outputs/results
  • Percentage of results linked to citizen proposals out of the total of results (total citizen influence on outputs)

Hybridization

This indicator measures the level of combination between online and offline spaces during the process.

• Is a meeting component active? (Yes=5, No=0)
• Are there face to face meetings? (Yes=5, No=0)
• Are there online meetings ? (Yes=5, No=0)
• Are there hybrid meetings? (Yes=5, No=0)
• Are there proposals created from meetings? (Yes=5, No=0)

Responsiveness

This indicator measures the level of responsiveness of the organization to participant input. It reflects their engagement through official responses and implemented decisions.

• Percentage of proposals with answers (n/total) – Proposal institutional responsiveness
• Percentage of results completed at 100% (n/total)

Traceability

This indicator measures the transparency and traceability of the participatory process. It reflects how easily participants can follow the evolution of proposals and verify institutional actions through published records.

• Number of Proposals with history (n/total)
• Percentage of results linked to proposals (n/total)
• Percentage of meetings closed with minutes published (n/total)
• Activation of the open data module (dichotomic: 0 or 5)
• Percentage of proposals related to budgets (n/total)

Self-Assessment Metrics

In addition to automatic metrics, we propose the use of a self-evaluation form to assess the qualitative dimensions of each participatory process. This form will be completed by platform administrators at the end of each process, and the responses will be publicly displayed in the indicators content block.

These questions aim to capture aspects of democratic quality that cannot be assessed through quantitative data alone. It is important to note that the responses to this survey are biased by the opinion of the platform administrators.

Self-assessment questions

The questions that make up the self-assessment form answered by the administrators, as well as the values associated with each answer, are set out below.

Inclusiveness

Inclusivity according to cultural origin and functional diversity.

1. Have migrant groups been invited to participate?
   • Yes = 5
   • No = 0
2. How many people from diverse cultural origins have participated out of the invited?
   • 0-20% = 1
   • 20-40% = 2
   • 40-60% = 3
   • 60-80% = 4
   • 80-100% = 5
3. Have groups of people with functional diversity been invited to participate?
   • Yes = 5
   • No = 0
4. How many people with functional diversity have participated out of the invited?
   • 0-20% = 1
   • 20-40% = 2
   • 40-60% = 3
   • 60-80% = 4
   • 80-100% = 5

Relevance

Impact relevance, as the percentage of the organisation's budget, of the organisation's workforce, or of the total population, that will be affected by the process' result, and the time of such impact.

1. Percentage of organisation's budget, workforce or of the total population affected by the process' result.
   • 0-20% = 1
   • 20-40% = 2
   • 40-60% = 3
   • 60-80% = 4
   • 80-100% = 5

Citizens influence

1. Level of influence of the process according to the participation mechanisms enabled, according to the ladder of participation defined by Sherry Arnstein (1969).
   • Informative = 0

     One-way flow of information—from officials to citizens—with no channel provided for feedback and no power for negotiation.

   • Consultation = 1

     A participatory process where citizens are asked for their opinions, typically through surveys or meetings, but without assurance that their input will influence decisions.

   • Placation = 2

     A participatory process where citizens are given limited influence, often in a tokenistic manner, such as inclusion in boards or committees, but without real power, as decision-making remains controlled by the traditional power elite.

   • Partnership = 3

     A participatory process where citizens and organizations share decision-making power, allowing participants to negotiate, allocate resources, and influence planning, with rules that cannot be changed unilaterally. It often emerges from citizen mobilization and collective action.

   • Delegated power = 4

     A participatory process where citizens gain authority, control, or funding to influence and manage an organization.

   • Citizen control = 5

     A highest level of participation where citizens fully govern a program or institution, managing policies, resources, and decision-making without external interference.

2. Percentage of phases with 0-5 citizen decisional intervention.

   Out of all phases in the process (from selecting issues to supervising execution), in how many does the public have the ability to make decisions or take executive action?

   • 0-20% = 1
   • 20-40% = 2
   • 40-60% = 3
   • 60-80% = 4
   • 80-100% = 5

Accessibility

1. In how many languages has the process been communicated?
   • One language = 1
   • Less than half of the languages spoken in the community = 2,5
   • More than half of the languages spoken in the community = 5
2. Were the venues of the face-to-face meetings accessible to people with reduced mobility?
   • No = 0
   • Partially = 2,5
   • Yes = 5