by Juan Carlos Barahona and Carlos Siri, Faculty Research and Development, INCAE Business School, Costa Rica
Public participation is generally defined as activities by which people’s concerns, needs, interests and values are incorporated into decisions and actions on public matters and issues.
Incorporating direct forms of public participation in oversight work can enhance citizen engagement and inform Supreme Audit Institution (SAI) decision-making, as the Comptroller General of the Republic of Costa Rica (CGR) discovered when considering the transfer of $4 billion in health-related purchases to a digital national procurement platform.
A long national debate over public procurement ended in 2015 when public entities were mandated by law to manage public funds and purchasing through a national e-procurement platform. A subsequent debate ensued on which platform to use.
To inform the discussion, CGR leadership sought input from those directly affected by the decision—Costa Rican civil servants—and commissioned a project to actively engage both users and potential users of such a system
As a complement to more traditional methods, such as surveys, where personal opinions may get lost in averages or other metrics, the project employed Q-Methodology to collect and analyze diverse viewpoints.
This method informed final discussions on using the designated e-procurement platform, although the nation’s largest public purchaser, Caja Costarricense de Seguridad Social (CCSS), was among those who rejected platform adoption.
Q-METHODOLOGY: THE PROCESS
Developing the Concourse. The concourse is a set of statements or individual perspectives on a specific topic. These statements are collected through interviews, discussions or other sources.
Developing the Q-Set and Q-Sorting. Domain experts derive a smaller set of statements (Q-Set) from the concourse. Each participant rank-orders this subset using a sorting grid that indicates the degree to which each statement reflects participant opinions. Q-Sorts are then collected, analyzed and inter-correlated to determine possible patterns, which form factors that can be grouped into clusters of opinions.
Q-METHODOLOGY: THE PROCESS FOR CGR
CGR leadership sought to understand the gridlock preventing the transfer of $4 billion in public funds to the e-procurement platform, Sistema de Compras Públicas (SICOP).
The Concourse. Forty civil servants answered guiding (but not limiting) questions via focus groups and open-ended interviews:
- What is your opinion about SICOP?
- What is it about the system that adds the most value?
- What is it about the system that adds the least value?
- When you last used the system—what happened and why?
- Describe your best and/or worst experience with the system.
Answers reflected the full spectrum of views and comprised the 345 concourse statements. From these, neutral experts with syntactic and semantic knowledge of each statement constructed a Q-Set—35 statements that were then Q-sorted (ranked) by civil servants across the nation (124 of 314 civil servants invited ultimately participated in the Q-Sort).
Q-Sort Ranking. Participants first responded to the Q-set by dragging and dropping statements into three separate categories:
- Resembles my opinion the least
- Neutral
- Resembles my opinion the most
Next, participants ranked each statement along a continuum of preference (see Table 1) based on the degree to which the statement “resembles my opinion the least” (-5) through zero (neutral) to “resembles my opinion the most” (+5).
After factoring in response validity, 101 participant rankings (48 users and 53 potential users of SICOP) were used for analysis.
Q-Analysis and Factor Interpretation. Prominent common viewpoints (factors) were extracted, and analysis reduced individual viewpoints to two summarizing factors: (1) users who understand the system well enough to know its flaws and (2) users who believe the system is useful but require specialized attention (see Table 2).
Table 2 also shows factor loading (negatively or positively) per statement, which is also characterized as “distinctive” or “consensus” based on scoring and initial qualitative data.
The statistics suggest “Factor 1” may be primarily composed of responses from a group of “super users” who know the system intricately and are able to pinpoint pitfalls in a way users from “Factor 2” cannot.
Summary
Results show the e-procurement system is widely viewed as useful to users and potential users despite general discontent regarding system management. Information gleaned can be helpful in developing more effective implementation and management strategies, such as focusing on marketing and operations to address negative opinions, establishing courses of action for specific distinguishing factors, and devising improved responses to meet customer needs.
Contribution and Future Direction
Using Q-Methodology, CGR was able to gather stakeholder voices, process these perspectives in a novel way, and use the project results to reframe deliberations on transferring $4 billion in publicly funded health-related purchases to the e-procurement platform (SICOP).
This project allowed stakeholders—those directly affected by the decision—to become part of the discussion and illustrates how SAIs can incorporate public participation into oversight work to better understand and engage stakeholders and positively influence public goods and service delivery.
To learn more about this project and Q-Methodology as a tool to enhance public participation and for a full list of references, contact the authors: juan.barahona@incae.edu and siri.carlos@incae.edu.
