Overview: The question of impact assessment is one that continues to plague microcredit practitioners. Some contend that existing impact assessment studies are meaningless, while others maintain they are absolutely necessary. The authors of this paper advocate for a new paradigm for impact assessment. They outline the key principles for conductingimpact audits that include measurement of transformation among clients. They also review a series of practitioner-oriented impact assessment tools and outline future challenges for practitioners, donors, and academics in advancing the usefulness of impact assessment efforts.

Summary: The debate about impact assessment begs the question of "measuring impact for what purpose?" The authors support the use of impact measurement tools that incorporate financial measures and indicate transformation-that is, deeply rooted change in the lives of individuals or communities-among clients. This paper acknowledges the problems with measuring impact, including issues of academic rigor, cost, and required expertise. Nevertheless, the authors cite the weaknesses of assuming the marketplace is an adequate proxy for impact data and maintain that practitioners should measure impact in order to determine whether microcredit institutions have met their stated purpose-alleviating poverty.

Therefore, the authors call for adoption and implementation of practitioner-oriented impact assessment tools that support a new paradigm for impact measurement, one based on the concept of an impact audit. They highlight the weaknesses of typical impact studies and emphasize that the audit is designed not as an academic study, but as an internal management tool. The purpose is both to determine results and to improve programs.

Using the model of a financial audit, general guidelines for the purpose and scope of an impact audit are given, especially the criteria managers should use in selecting appropriate tools. The criteria include analyses of trends over time, comparison with previous impact data, ease of implementation by existing staff, incorporation into existing management information systems and costs comparable to those spent on financial audits.

As a first step in equipping managers to locate these tools, the authors provide short profiles of some of the best practitioner-oriented tools available. They include a description of each tool, sample questions or data gathered, and a brief summary of the tool’s key strengths and weaknesses. The appendix also contains additional abstracts of impact assessment tools and other resources.

Finally, the authors call on practitioners to reclaim impact assessment as an essential management tool. They also beckon donors to fund the development of replicable tools and reward organizations that use them. They ask consultants to develop expertise in applying these tools. And they point out that clients need to understand that their participation is crucial to the success of all impact assessment efforts.

Measuring Transformation:

Introduction

It's an old joke, but it aptly illustrates the current state of the practice related to impact measurement and microcredit. The microcredit movement exists to alleviate poverty. Yet the only performance indicators used to gauge the effectiveness of microcredit programs measure the profitability of the lending institution and the quality of its portfolio. They don't tell us whether our clients become less poor due to the services we provide.

Why do we solely rely on these financial performance measures if they don't tell us if we're achieving our objective? Because they are much easier to calculate reliably. We spend our time investigating the well-lit areas, while the object we're searching for remains in the dark.

This paper seeks to aim a small band of light on the subject of client impact measurement for microcredit. We will examine the need for good impact measurement tools, describe some of the challenges we face as we try to come up with such, identify elements of an ideal impact assessment, and describe some tools that can help us measure our impact. We'll also try to get a few jabs in at our friends who keep telling us that impact measurement is not important.

We recognize that the subject of impact measurement has been hotly debated and frequently studied by experts, most recently by the USAID Assessing the Impact of Microenterprise Service (AIMS) project and in a series of virtual meetings organized by the World Bank's Consultative Group to Assist the Poorest (CGAP). We commend the work that these organizations are doing and encourage them to keep at it. Meanwhile, in this paper we will focus on what we as practitioners need: tools that will help us design better lending products and assist us in managing institutions.

Right up front we need to admit to two biases. First, as practitioners we need information that can reliably help us make good decisions. We don't have much time or money to spend on making sure the data are 99.44% pure. We appreciate the work that academics and others do to further the theory and practice of impact assessment, but for us, the test will always be whether these tools yield data that help us make better decisions.

Second, as old hands at this work, we still believe that the purpose of doing microfinance work is alleviating poverty. We choose to do this work because we want to help encourage a process of transformation in the lives of our clients and their communities. We readily acknowledge that in order to do this, we need to develop large-scale institutions that operate profitably. However, our first purpose is the transformation of the individual and her community, rather than the transformation of the institution.²

We acknowledge that this business of finding out whether we have achieved our objective can be rather messy, since poverty includes social, spiritual, and political, as well as economic, dimensions. Nevertheless, that’s how we want to measure our success: by whether or not people improve their lives through our programs. We know it’s a murky task, but we would rather spend our time trying to bring light to this subject than searching in the well-lit areas that cannot tell us whether or not we have achieved our goal.

On its face, the case for measuring the impact of microcredit seems obvious. If we seek to alleviate poverty, and especially when we use scarce public funds to do so, surely we need to find some way to determine whether we are achieving our objective. After all, credit is a two-edged tool. To a borrower, credit means debt, and debt can destroy as easily as it can build. If we seek to help people lift themselves out of poverty, we will want to know that they were poor when they started borrowing and they were less poor as a result of borrowing. We don’t want to see them stuck, cycle after cycle, earning low returns. Just knowing that we increased the debt of 100 million people will not tell us that we accomplished what we set out to do, even if we delivered that debt in a financially viable manner.

Yet the case is not so clear-cut, and some of the leading thinkers in the microcredit field have challenged the usefulness of expenditures on impact measurement.

Beth Rhyne writes, "For the most part, evaluations of credit programs are still based on the old-view ideas about causality. They are centered on the presumption of a direct line of causation between receipt of credit by individual borrowers and a particular desired response, for example changed borrower income resulting directly from receipt of a particular loan."³

The basic line of argument against more extensive impact measurement has been:

• 1. Most impact studies do not pass academic muster: Studies that show changes in income, assets, or employment prove that something is different in the lives of the clients, but they do not prove that the lending program caused the change. Without measuring changes in the lives of people who did not receive loans (such as through the use of control groups) you cannot attribute impact to the credit that has been provided. Also, most studies take place at one point in time and rely on the often unreliable memories of clients to determine their status before receiving a loan.
• 2. Proper impact studies cost too much money: Rigorous impact studies with control groups and information baselines are expensive. They require outside experts or well-trained staff members to organize the study, set up the baseline information retrieval system, train the local enumerators to collect the impact data, write the program to collate the data, and then carry out the analysis of the final results.
• 3. It is too expensive to carry out this sort of analysis on a regular basis: Most microcredit institutions struggling to become financially viable cannot afford to pay a team of experts (either outside consultants or full time staff members) to carry out proper impact studies on a regular basis.

• 4. The marketplace provides reasonable proxies for impact data: If the clients of a microcredit organization pay the full cost of the service they receive, and pay back their loans on time, and come back regularly for more loans, then the program must be having a positive impact. (Client payback = client perception of value = client satisfaction = positive impact.) Testing those assumptions does not produce information that justifies the large costs involved.
• 5. Impact measurement, then, becomes the last bastion of those unwilling to face the harsh realities of the marketplace: Impact studies often get used to justify development expenditures for institutions that do not meet marketplace standards of profitability and high portfolio quality. The fuzzy world of impact can easily be used to obfuscate poor financial performance.

While this argument has a certain logical consistency to it, it still fails to answer the question of whether the money spent on microcredit actually achieves the objective of transforming lives out of conditions of poverty. It also does not provide any assistance in assessing different types of microcredit delivery systems. For example, let's say we have two microcredit institutions, both equally profitable and serving the same number of clients. But, the impact of the two institutions differs widely. While the one provides financially sustainable credit, it has marginal and scattered impact on its clients. Another not only provides the credit, but also builds leadership skills and social networks among its clients in a way that brings widespread positive change to their communities. Financial measurements provide us with no way of differentiating between the two institutions. We would need some way of assessing transformational impact to discover the difference.

To us, measures of profitability and portfolio quality are important measures of the institutional health of an organization, but they do not measure client health or well-being. As one of the accounting professors we interviewed for this paper put it, "Using profit and loss to measure the impact of microcredit is like using a speedometer to measure the temperature."

But why would someone try to use a speedometer to measure the temperature? Only if there were no thermometer or the only temperature gauges available were too big, bulky, and costly to be used on a regular basis. Unfortunately, that is the position of the impact measurement field today. It has developed very sophisticated tools for showing where impact happened, but most of us who work in this field find the tools too expensive or time consuming to employ on a regular basis.

If we are going to develop tools we can use, those who manage microcredit organizations need to take a more prominent role in the impact discussion. We feel it’s time for practitioners to get into the game and reclaim impact assessment as an essential tool of management.

For this paper we have reviewed over 100 impact evaluation documents, including over 50 actual tools, looking for tools we could recommend. We came away from this exercise struck by how little there was that we could recommend. This is not because the impact studies were not well done, but because they were not replicable or seemed too costly. In addition, many of the studies shared one or more of the following common weaknesses:

• They often rely on participant memory to produce baseline data, rather than collecting information when a client enters a program.
• They provide information for only one point in time, and cannot tell whether impact is increasing or decreasing over time.
• They rely on too short a time period to allow impacts to manifest themselves.
• They don’t compare the changes in clients and non-clients.
• They require specially skilled people to carry out the research or interpret its results.
• Their data and procedures are so specific to one institution that they cannot be used by other institutions.

Developing an instrument to reliably measure impact on a regular basis will require that we change the paradigm we use when we think about impact measurement. The current paradigm is that of the donor-initiated, academic impact study, whose objective is to prove whether or not the programs funded by the donors did any good. This has produced shelves full of custom-designed, point-in-time studies that cannot provide trend analyses and cannot be compared to any other impact studies. In other words, it produces information whose primary function is to justify donor money that has already been spent or to direct the spending of new donor money.

The fact that the word "studies" most often follows the word "impact" shows how dominant the academic paradigm has become. We don't seem to talk about impact measures, impact monitoring, or impact analysis as often as we talk about impact studies.

We do have some use for these studies. We quote liberally from them (as long as they are in our favor) when we apply for funding. However, we rarely use them when we are doing our planning.

David Hulme talks about a continuum of objectives for impact assessment ranging from proving impacts for the purposes of measuring the results of an investment to improving the practice of a microcredit institution.⁴ Impact measurement tools that focus on improving, rather than proving, may incorporate some of the tools already in use, but will utilize them differently. Impact measurement tools that get used by management on a regular basis will take a different focus.

• They will be incorporated into the regular routine of data collection in an organization.
• Staff members will collect and analyze the data.
• The data collected and analyzed will remain consistent over time and location, allowing for trend analysis and comparisons of different geographic areas.
• They will include a point of comparison with people in the same community who do not receive loans. ⁵

• The results will be reviewed on a regular (at least annual) basis and become a key part of the planning process.
• The analysis will seek to provide input to the question "How can we improve the positive impact and promote transformation?" rather than the question "Was there any impact?"
• Outside experts-ideally local ones-will be brought in to verify the accuracy of the data, make recommendations for improvements in the collection process, and help in developing control group comparisons.

This type of ongoing impact monitoring suggests a different paradigm, an impact audit as an internal tool of management. It most resembles the process of a financial audit, which involves all of the steps listed above. Staff members collect financial data on a regular and consistent basis. They use this data to generate trend analysis and profit-center comparisons. This information gets incorporated into the regular planning process, helping the organization improve its profitability. On an annual basis, the organization calls in outside experts (auditors) to verify the data and suggest ways for improving the accounting and management procedures. When an entire industry holds the same impact assessment standards, organizations are able to compare their performance with similar organizations. An impact audit would work the same way. Staff members would collect baseline information on important financial and social indicators for each client (or a sample group) as a part of the loan application process. This data could be updated on an annual basis or with each new loan application. On a periodic basis (semi-annually or annually) the organization would analyze this data to determine trends amongst its clients and then dig deeper into the issues raised in this analysis through client surveys or focus groups. The organization would then compile this data and reflect on what it means for the design and strategy of their lending activities. On an annual basis the organization could publish a report describing the impact of their program, the levels of transformation desired and achieved, and any changes they will be making in their program as a result of this information. Outside experts could be involved to verify the accuracy obtained, suggest improvements in the information collection process, and assist management to think through the implications of the impact analysis on program activities.

Using the paradigm of the financial audit also helps us think about how much an institution should be willing to spend to measure its impact. Our simple rule of thumb: it should be willing to spend as much to track and audit its impact as it does to track and audit its finances. Or, put another way, measuring whether it is achieving its objective should be at least as valuable to an institution as measuring whether or not it is earning enough money to stay in business. In a microcredit program with up to 10,000 clients, this might cost roughly US$10,000-20,000 for staff time, tracking systems, and outside "auditors." We expect that an investment in this type of impact monitoring should pay off financially as an institution becomes more efficient in delivering the type of impact it desires and creates more benefits for its clients. (Note that this sort of thinking does not apply only to non-profit or socially motivated activities. Increasingly, businesses are turning to the "social audit" and the "balanced scorecard" as a way of determining whether their work has a positive impact on their employees, customers, and communities.⁶ )

One of the members of the Opportunity International Network in India, The Bridge Foundation (TBF) in Bangalore, has shown us the value of this type of impact audit. Initially, TBF gave loans to individuals in urban areas. Work was initiated through churches and community leaders, who helped to identify honest clients. From this point, TBF did their own analysis to determine business expertise and potential. In these initial stages, TBF’s work was geared toward employment generation-creating job opportunities for the target group through "labor intensive projects."

However, experience and an impact audit resulted in a change in the methodology. In 1991, after reviewing the data it had gathered on the impact of its work, TBF decided that employment generation was not creating jobs for their target group, the extremely poor; clients were employing skilled people outside of the target group. As a result, TBF shifted its focus to income generation, and raising levels of family income through "family projects." Further, since 1991, TBF has made a major shift in its program by moving out from the urban to the rural sector and now most of their clients are located in the interior villages of the four southern states of India.

As managers, we want to use impact data to make critical strategic decisions. We want to know such things as which client sectors to work with, which of our lending products are working, what barriers our clients are facing, why repayment rates have dropped, why people are leaving our programs, which clients are receiving more benefits and which ones are receiving less, and how best to expand our programs.⁷ To that end, we want tools that:

• provide analysis of trends over time
• provide results that can be compared with previous impact data
• can be implemented by existing staff with a clearly defined role for outside experts
• become part of the regular information system of an institution
• cost no more than what it costs to track and audit financial information

These criteria tend to blur the lines between impact monitoring and impact assessment. Impact monitoring is the assessment of the performance of a project against its internal targets by gathering information on a regular basis-ongoing or current. It can be useful for such management decisions as whether or not clients can handle larger loans. Impact evaluation is assessment of a project against its main objectives at a particular point-retrospective. It brings intentionality to the question of whether we are fulfilling our mission, and how we might make mid-course corrections. Our view is that these need not be seen as distinct activities, but rather as two necessary components of an ongoing impact auditing system.

In the following section we list some of the most promising impact assessment tools that we found in our review. While no single tool meets all the criteria that we listed above, these could be used in various combinations to develop an impact audit system.

Developer: Helzi Noponen

Implementing Organization: Friends of Women’s World Banking, India, with funding from the Ford Foundation

Description: An internal, decentralized, integrated monitoring, evaluation, management and training system that can be used for decision making at all levels, including clients, field workers, and donors.

Sample Questions:
Have I improved my shelter?
Have I improved my production processes?
Have I improved access to health care?
Am I satisfied with my conditions?
(asked through accompanying pictures)

Key Strengths:
• An integrated system that is incorporated into the ongoing data collection work of clients, loan officers, and managers
• Provides useful information to all decision-makers in the process, including clients
• Based on pictures, so it can be used with illiterate clients

The Integrated Learning System (ILS) is based on a pictorial diary compiled by the clients, who feed the information into a summary diary kept by the microcredit self-help group, then in turn to the area organizer, the program manager, and finally the intermediary provider of funds. At each level, the participants collect data, assess change, analyze the reasons for change, alter their strategies based on their learning, and document progress and share learning.

A test group of illiterate handloom weavers used their pictorial diaries to reflect on their lives, lobby for electricity, and lobby for a clinic for work-related illness. Self-help groups used them, along with related training modules, to monitor and improve their organizational functioning in such areas as leadership and keeping accounts. Field staff used them to ensure good stewardship of resources by directing training to areas of identified need. They also found out the extent of child labor was far greater than they had realized, and therefore created a child labor program. In addition to the impact evaluation data, the system creates a flow of best practices and learning, and in itself can serve as a tool for client and staff empowerment.

The ILS can be adapted by using the Stakeholder Analysis Manual to determine appropriate domains of inquiry and indicators, and then hiring a local artist to adapt the drawings. The implementation cost of the ILS is essentially in the staff training and processing time, transportation costs to travel to training sessions, and the cost of printing or photocopying diaries and manuals. (Three-year client diaries cost about 40¢ each). Because this is an integrated system, the essential requirement is commitment by the NGO to staff involvement at all levels.

Tool: Client Monitoring System

Developer: MSI, Lorraine Blank and Russ Webster

Implementing Organization: Workers Bank, Jamaica

Description: An Access database that provides baseline data on health, education, and financial status of individual lending clients at the time of the first loan and at subsequent points thereafter.

(tool uses data-recording format)
Durable/non-durable household goods in your household.
Current bill payments
Cash resources
Average value of food expenditures

Strengths:
• Low-cost system for tracking key impact indicators over time
• Impact monitoring integrated into loan application system
• Storing information in computer database allows for a wide variety of reports and comparisons between various input factors (loan size, business type and subsequent impact (changes in assets, education, health, etc.))

MSI has worked with the Workers Bank of Jamaica to develop an impact monitoring database that is incorporated into the loan application system. Clients apply for a loan through an interview with a loan officer, with the loan officer entering the data directly into a computer. The data requested in the interview includes information needed to assess the loan as well as information on the client's income, assets, patterns of expenditure, etc. The entire interview process takes about 20 minutes. When clients reapply for loans, the data is collected again. This allows for tracking changes in these indicators over the time that the client borrows from the Workers Bank.

Developers: SEEP, under the direction of then Executive Director Elaine Edgcomb, with Barbara MkNelly, Carter Garber and Nancy Horn (as a part of the USAID AIMS project)

Implementing Organization: ODEF/Honduras, a partner of Katalysis; and Kafo Jiginew/Mali, a partner of Freedom from Hunger

Description: A multi-component impact analysis including a cross-sectional comparison of clients and non-clients; in-depth interviewing of samples of multi-year clients; a survey of ex-clients; and focus group interviews on client satisfaction.

Sample Questions:
In the last four weeks, did you earn income in some way other than working for others?
When you are deciding to undertake a business, what factors do you consider?
During the past two years, were any repairs, improvements, or additions made to your home that cost more than $50?
Compared to the last school year, did your household spending on school and school expenses for this current school year decrease, stay the same, or increase?

Strengths:
• Use of control groups to help in attributing impact
• Integrated mix of tools looking at different aspects of impact

At Kafo Jiginew, the impact survey was administered with three sample groups -- one-year clients, two-year clients and incoming clients. The test was carried out over a three-week period by staff who were assigned to clients they did not know. Kafo’s costs (for salaries, per diem, etc.) totaled US$2,800; Freedom from Hunger’s costs were US$8,700 for salary, expenses, and car rental for the trainer and driver.⁸ A total of 197 person days were used, including planning, pre-testing, data collection, software installation and data entry, and analysis.

The assessment does a good job of proving positive impact at multiple levels, including enterprise, household, individual, and community. In addition, it provided Kafo with feedback, on such issues as clients’ actual use of the loan funds, that will help it better shape the program.

Developer: compiled by Carter Garber for the USAID AIMS Project,⁹ based on the Women’s Opportunity Fund "Trust Bank Client Exit Interviews"

Implementing Organization: AGAPE, Colombia, a member of the Opportunity International Network

Description: A standard interview given to group loan clients when they leave the lending program

Sample Questions:
What are the main reasons that you are leaving or left the program?
What did your loans help you do in your business?
In what ways did the loan help your family?
Could you name the two things you liked best about the program?

While these are very limited in scope, we thought it useful to include a tool that is an easily implemented first step in determining client satisfaction with the program. It consists of 5-15 minute surveys given by loan officers to clients as they leave the program. When AGAPE/Colombia used the tool, they learned about changes in client income, use of the loans, and loan sizes. Based on their findings, they decided to focus on training clients in how to invest their loans and on strengthening initial business assessments.

Using a similar approach, FINCA/Nicaragua produced a report on client targeting, turnover, and program impact, by John Hatch, Todd Manwaring, Meredith Terrell, and Judd Horn, September 1998. However, the survey tools were used among new clients, departing clients, and ex-clients. The results indicated the distinct issues in product satisfaction and client impact for each group.

Developer: Monawar Sultana and Ashok Nigam

Implementing Organization: Family Development Fund, Egypt (UNICEF/Egypt, 1998)

Description: Measures improvements in material well-being, access to basic social services, improved status of women, and institutional sustainability of the program using individual interviews with structured questionnaires, focus group interviews and case studies.

Sample Questions:
How often does your family eat vegetables or fruits?
In the last two years, has any one from your family borrowed money besides from the project loan?
How would you treat your children if they had diarrhea?
Did the relation with your husband (or in-laws) change after getting the loan?

Strengths:
• Good combination of tools to confirm and check data (triangulation)
• Use of control group

This study used a good tiered approach for collecting different types of data on clients. The first tier, shallow and broad, was the interview questionnaire. This went to a random sample of clients and a control group. Loan officers carried out the interviews, and the entire study occurred over a 30-day period. The second tier, the focus group interviews, involved open-ended discussions with 107 women in eight centers. The final tier, the in-depth case studies, involved individual interviews with eight borrowers from the same group.

Among the many interesting findings were that income from the credit activities is mostly spent on children’s education and family subsistence, and that after taking the loan, the number of borrowers whose children did not go to school dropped from 23% to 16%, although most borrowers preferred to educate their sons rather than their daughters. Among the recommendations for improving the program was that more emphasis should be given to targeting women from the younger age group in order to meet UNICEF’s objectives of improving the condition of children.

Mbeya Credit Facility Client Impact Evaluation. By Corey Huntington for Mennonite Economic Development Associates (MEDA) Tanzania. September 1996. Clients participated in group discussions and in-depth, individual interviews about economic and social impacts experienced by individuals and their households, resulting in such positive findings as women’s increased financial independence, and recommendations such as the need for more appropriate client training. Cost was US$3,700.

Group Guaranteed Lending and Savings Program in the West Bank and Gaza. By Khalid Nabris. Save the Children/West Bank and Gaza Strip. September 1997. This is an interesting mix of qualitative and quantitative data including a questionnaire survey with borrowers and one with dropouts; focus group discussions with borrowers, borrowers’ husbands and borrowers’ children; and workshops with field staff.

Measuring Client Success: An Evaluation of ACCION’s Impact on Microenterprises in the United States. By Cristina Himes with Lisa J. Servon. The U.S. Issues Series Document No. 2, April 1998. The loan application form collects baseline data for new and repeat loans, thereby providing longitudinal data on the business as well as personal consumption and income. This keeps costs down, although it also means the system does not track clients who have left the program, and data is kept irregularly. In-depth interviews were also conducted about the gains individuals, families, and communities experience as a result of the business and the microcredit program.

Tools used by ADEMI-BANCOADEMI. As described by Pedro Jimenez. March, 1998. Since its start, ADEMI has collected data related to its clients’ business growth as part of every loan and re-loan application. Indicators include cash, equipment, assets, sales, profits, salary of owner and employees, expenses, savings, and number of employees. ADEMI now has data on over 44,000 enterprises collected since 1983, and has been able to use the data to prove the macroeconomic impact of its lending and to lobby for attention to the microenterprise sector.

A good program of impact measurement and monitoring will not rely solely on one tool, just as business managers utilize a set of measures to help them assess current financial performance and predict future growth. While business people often refer to "the bottom line" as the final measure of their performance, there are, in fact, many measures that businesses use to assess their performance. These include earnings per share, market share, internal rate of return, net present value, and-the most recent business measurement tool-economic value added. Businesses add to these numerical tools a wide range of subjective measures to assess customer satisfaction and behavior, including surveys and focus groups.

The same will be true of a good regimen of impact analysis. It will start with a good base of information that is tracked regularly in a form that can be assessed in many different ways (such as the database used by the Workers Bank of Jamaica). As much as possible the tools for this base of information should be standardized so that the information can be audited on a regular basis. On top of this will come other forms of impact tracking, including surveys and focus groups, that give decision-makers more depth of understanding and a feel for the conditions, values, and experiences of their clients.

For impact measurement to become as common and expected as a financial audit, all of us in the microcredit community will need to take it much more seriously.¹¹

• Practitioners will need to reclaim impact measurement as an essential tool of management. We will need to take the initiative in discovering, developing, testing and refining tools that tell us whether we are achieving our objective. We can no longer wait on donors or professors to do our impact analysis for us.
• Donors will need to apply funds and expertise to this area, as USAID has in the AIMS project. They should work with practitioners to develop replicable tools that can be used for analyzing and improving the performance of microcredit institutions in transforming the lives of their clients, and they should reward institutions that implement the tools.
• Consultants-especially local consultants-will need to develop expertise in applying these tools and auditing their use so that we can come up with standardized tools that produce consistent and reliable data useful for practitioners.
• Academics will need to work with practitioners to find ways that the tools they have already developed can be adapted to fit into an ongoing monitoring system. They can also help to analyze the data that comes in from institutions employing different lending methodologies in different parts of the world.
• Clients should become not just objects of study, but users of the data that is generated. Clients should be able to see the impact their participation has had on their own lives, compare it with the impact on others, and make suggestions for how the microcredit institution can improve its ability to assist in the process of their transformation.

Working together, we all may be able to shine some light on this field of impact measurement and find ways to not only measure impact, but to also improve our ability to transform the lives of the clients we serve with microcredit.

1 For the purpose of this document, the 1997 Microcredit Summit, and the Summit’s nine-year fulfillment campaign, any reference to microcredit should be understood to refer to programs that provide credit for self-employment, and other financial and business services (including savings and technical assistance), to very poor persons.

2 For this paper we want to make a distinction between impact and transformation. For us, impact is any change, positive or negative, that results from an intervention. We define transformation as a deeply rooted change in beliefs, values, attitudes, actions, relationships, and structures manifested in a sustained higher level of existence of an individual and/or community.

3 Rhyne, Beth, "A New View of Finance Program Evaluation," 1992, GEMINI, Development Alternatives Inc., Bethesda, MD.

4 Hulme, David, “Impact Assessment Methodologies for Microfinance: A Review,” May 1997 - AIMS - [Paper prepared in conjunction with the AIMS Project for the Virtual Meeting of the CGAP Working Group on Impact Assessment Methodologies (April 17-19, 1997)]

5 A note on the use of control groups: While control groups may be the most common way of determining causality in impact assessment, they also produce logistical and ethical difficulties for microcredit institutions. On the logistical side, it is costly and time consuming to track data on people who do not receive credit and other financial services. The ethical dilemma arises when you look at using impact assessment as an ongoing management tool. It would be wrong to continuously withhold credit from people who might be able to use it just so that we can prove the value of that credit. Here are some ways that microcredit institutions have tried to get around these difficulties:

• Use data generated by the government statistical office to develop profiles of the average household in a community and how it has changed over time, and compare this to changes among clients.
• Encourage a university to have students carry out ongoing research among people who could serve as a control group.
• Develop comparisons between new applicants and clients who have been with the organization through several loan cycles, using new applicants as a proxy control group.