Designing the right monitoring and evaluation system for your NGO

At Hikaya, we build the right system for the job by applying concepts that come from software engineering, especially user experience, to the task of designing Monitoring and Evaluation (M&E) systems that leverage not only our technical background but also our familiarity with the concepts and ways of working that undergird NGO and nonprofit work. As with any concept, there are a number of frameworks that can help organize this work: we use the Desirability vs. Feasibility vs. Viability Matrix to help us understand the tradeoffs between developing a feature. By rating features this way, we can quickly get a sense of what should be prioritized when a feature rates as highly desirable (does it meet the needs of the user?), highly feasible (do we have the resources to build it?), and highly viable (does it align with our strategy?). Using this simple binary scale of high or low desirability, high or low feasibility, and high or low viability, we can quickly evaluate what feature should be prioritized as we plan our development roadmap.

Why is this important for the NGO and nonprofit sector? Well, project implementers in the sector face constant and consistent constraints: financial and human resources and in-house technical capacity being two that especially determine the degree to which a technological solution is desirable, feasible, and viable for any given organization. With this matrix as the foundation for our engagements, we ensure that the product we design is what users want and, more importantly, what they need. Referring back to these three characteristics of a digital product, one must consult field staff as a matter of course. This avoids the frequent pitfall of the design, development, and deployment of digital products in the NGO sector, whereby a solution engineered by staff in head offices fails to meet the needs or work within the constraints of those that must use it at the ground level.

Using this approach, we can work collaboratively with team members at all levels to identify the most pressing priorities while ensuring that any development work remains cost effective, adds the most value for users, and, most importantly, is deployable in the short-term so that we can iterate (see our article on iteration for more on why starting small and gathering real-world feedback from users is so important and can help us design better products at each step of the way).

Desirability

The desirability characteristic is driven by input from the users and the product owner. This will help to determine if a feature or business process has high desirability or low desirability. For example, if a feature is in popular demand by users, it is suggested that a feature should be considered.

A simple but practical example may help to illustrate the concept of desirability. Field staff are often asked to report on – and, necessarily gather data about – global key performance indicators (KPI) in some kind of standard formatting. For example, the head office may need beneficiary data disaggregated by sex and age group. If head office dominates the conversation, the development roadmap may unduly prioritize the integration of features that collect this information. These, though, are head office concerns. Those that are actually administering the project on the ground might have a completely different set of concerns and not urgently need or even be able to use disaggregated sex and age group data. In this case, the feature is desirable from the perspective of head office but not necessarily desirable from the perspective of field staff – while it should remain on the product roadmap, it might make sense to de-prioritize building in this capability in favor of addressing more urgent and fundamental needs.

Feasibility

The feasibility characteristic is driven by examining the current resources and existing system compatibility. This also includes determining if the feature aligns or is compatible with the current operating environment of the system. Feasibility asks: can you do that? At a fundamental level, analyzing this characteristic can guide you to decide whether or not a product or a feature is possible to pursue at all. Desirability tells you whether you will get use out of it and viability helps you decide if you should do it – both are ultimately more subjective judgment calls that depend on a good knowledge of your staff and its resources, your project or program and its procedures, as well as the broader contexts in which your project and organization operate in.

Feasibility, in contrast, is less subject to interpretation and more of a binary – either you do or don’t have the in-house technical capacity to do something, whether that is to build the product or feature in the first place or to maintain it in the long term.

Viability

The viability characteristic is driven by the organization's leadership and its relevant product owners to determine if a feature or business process should be developed for the organization. Since there are tradeoffs involved, there will be cases where an executive decision on which feature or business process will need to be made. Sometimes a proactive, forward-thinking organization might decide to prioritize products or features that serve longer-term needs rather than focusing only on addressing immediate needs. Whereas desirability and viability have a tendency to prioritize the perspectives, needs, and constraints of staff closer to the level of implementation – indeed, taking these characteristics into consideration in a meaningful way may facilitate a dialog between these and higher-level staff that might not otherwise take place – viability focuses on the strategic considerations an organization must account for when deciding to develop a new digital product or add a feature to an existing one.

Taking viability into account will help your organization think through whether what you’re currently developing for, say, Uganda, might also be used in Kenya and Tanzania (a separate post we have that expands on the principles we apply when making digital products talks more about scalability). Assessing the proposed product or feature in terms of its viability will also help answer questions about who will fund it and who will promote it. All of this, ultimately, will help you decide how to strategically shape your product to maximize its impact and get the most longevity out of the investment you will make in developing and maintaining it.

Example: Integration between two existing systems

To close out, it might be helpful to take a look at a real-world example of what this all looks like in practice. We’ll take a look at a data integration between two existing systems.

Taking these three factors into account, you might decide not to go for integration that, at first blush, seems like a great idea. The ultimate decision, though, should be made after carefully analyzing the benefits of a new product or feature against these three characteristics in a process that engages staff at all levels, and especially that takes into consideration the needs, resources, and constraints of end-users. Whether a big project or a small feature, there is always a set of trade offs to analyze. Almost everything is doable in theory – whether it makes sense to do it in practice is another question. When you find the best overlap between all three factors, you get the most value for your money, time, and effort.

At Hikaya, we help NGOs and nonprofits worldwide improve their delivery of goods and services to those most in need while making reporting to donors more transparent.

To learn more about our approach, visit our process page or see some of our past work.