ORGANIZATION:

Friends of Women’s World Banking, India

DOCUMENT TITLE:

Participatory Internal Learning Systems for Micro-credit and Livelihoods Programmes

TITLE OF PROGRAM EVALUATED:

Kerala Gandhi Smarak Nidhi (KGSN), Handloom Weaver Development Society (HLWDS)

COUNTRY(IES):

India

YEAR COMPLETED:

1998

AUTHORS:

Helzi Noponen

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Participatory rural assessment format. Various pictorial learning diaries for client, group, area, and organization record data over time beginning with a baseline, and are used in conjunction with supplementary account, administrative, or financial sustainability information.

CONCEPTUAL FRAMEWORK:

Each level of the organization including clients, field workers, and donors, can assess and track their own progress and guide future planning using participatory assessments.

HYPOTHESES:

The program increases participation, empowerment, decision making, proper goals and priorities, healthy living and working conditions, and a higher quality of life. The organization promotes strong performance of client groups as well as effective staff performance.

QUESTIONS ADDRESSED:

Quality of life indicators include shelter, living conditions, health, nutrition, education, debt, savings, empowerment, and business practices, goals, leadership, and income levels.

SAMPLING DESIGN:

All participants including clients and staff

TARGET PARTICIPANTS:

Current clients

DATA ANALYSIS:

Done at each level to varying degrees with aggregated data reported up each level

HIGHLIGHTS/SUMMARY:

Participatory, simple, pictorial, decentralized, flexible diaries which all levels of the organization use to gather and analyze their own data; qualitative and quantitative data recorded, and data is aggregated for analysis at each higher level of the organization

EXPERTISE NEEDED:

Loan officer equivalent, clients trained to use diaries, training for all staff participating

EASE/TIME NEEDED FOR IMPLEMENTATION:

Translation, adaptation of instrument to local context; regular analysis at all levels, aggregated data must be passed up to other levels for additional analysis

TIME FRAME:

Recorded over three years

PERSONNEL NEEDED:

Loan officer equivalent, clients, all organizational levels participate

COST:

Time costs of clients, loan officers, and others in organization, to gather and analyze the data, materials adapted to context

PROBLEMS/SHORTCOMINGS:

Integration into existing reporting and assessment unclear; no control group; materials require careful contextualization; requires strong commitment and time at all levels of the organization.

USE AS A MANAGEMENT TOOL:

Information gleaned from diaries can be used to design client training sessions. Analyzing the status of various client groups can influence decisions on whether to begin new groups or focus on capacity building of existing groups. The organization’s diary can highlight areas where staff requires training.

ORGANIZATION:

Workers Bank of Jamaica

DOCUMENT TITLE:

Client Monitoring Systems for Microfinance Institutions: AIMS Assistance to the Workers Bank of Jamaica

TITLE OF PROGRAM EVALUATED:

Workers Bank of Jamaica

COUNTRY(IES):

Jamaica

YEAR COMPLETED:

1998

AUTHORS:

Lorraine Blank and Russ Webster

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Loan application data is entered into an Access database client profiling system which can analyze and produce reports on that data as well as longitudinal studies of changes in various groups.

CONCEPTUAL FRAMEWORK:

Impact can be monitored using an automated system that stores client information from the initial loan cycle and can track changes over time as reported in subsequent loan applications.

HYPOTHESES:

The program has a positive impact on the business operations, inventory/equipment, balance sheet, revenues/expenses, and household assets which increases over time.

QUESTIONS ADDRESSED:

Durable goods, expenditures on food, level of income, value of client savings, value of enterprise sales, value of fixed assets of enterprise, number of employees by gender, number and size of new and repeat loans by gender, number of jobs created by gender

SAMPLING DESIGN:

Every new and repeat client

TARGET PARTICIPANTS:

New and repeat clients

DATA ANALYSIS:

The database system compiles and analyzes reports on the data. Includes client profiles, group profiles and longitudinal analysis.

HIGHLIGHTS/SUMMARY:

Database and client monitoring system provide immediate quantitative reports on welfare of household and business. Monitoring integrated into loan application process; analysis produced by system. Designed for individual lending.

EXPERTISE NEEDED:

Team of experts to design, program and establish the system; loan officer equivalent to enter data; trained system administrator to analyze data, prepare reports, maintain and upgrade system

EASE/TIME NEEDED FOR IMPLEMENTATION:

20 minute interviews with each client during application process

TIME FRAME:

Ongoing

PERSONNEL NEEDED:

Loan officer equivalents, one system administrator

COST:

Time costs as well as laptop computers for all loan officers, training for both loan officers and system administrator

PROBLEMS/SHORTCOMINGS:

Initial costs of equipment and training are high; access to phone lines and power is important for data transfer; concerned with quantifiable indicators only

USE AS A MANAGEMENT TOOL:

Can assist in targeting clients by revealing which types the program is currently attracting and by tracking that over time. It can track sales trends of clients and other constraints which may impact the loan terms, and it can immediately indicate the client’s capacity for re-loan. The organization can track the threats to clients’ businesses which can signal risks to portfolio.

ORGANIZATION:

Non-specific

DOCUMENT TITLE:

Impact Survey – SEEP/AIMS Project (9/98)

TITLE OF PROGRAM EVALUATED:

N/a (piloted in Katalysis in Honduras, and Credit with Education/Food for the Hungry in Mali)

COUNTRY(IES):

Piloted in Honduras and Mali

YEAR COMPLETED:

1997, 1998

AUTHORS:

Barbara MkNelly (SEEP)

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Survey with predetermined responses

CONCEPTUAL FRAMEWORK:

Household benefit model

HYPOTHESES:

Participation in an MFI/MED program will benefit the individual, the family, the community and the business

QUESTIONS ADDRESSED:

Individual, household, education of children, loan use and individual income, enterprise level: income, labor and working capital; enterprise level: improvement and assets; individuals level: savings and business skills; household level: assets, housing improvements, diet and coping with difficult times; repayment and program issues; optional questions on client productivity, hired labor, use of savings, plans for the enterprise, control over resources, solidarity, alternative credit sources

SAMPLING DESIGN:

Between 10 and 12 percent sample of client population with the program one year or more than two years; control group composed of those who have opted to join the lending program but have not yet received orientation

TARGET PARTICIPANTS:

Group and individual lending participants with the program about one year or more than two years.

DATA ANALYSIS:

Use of EPI for data entry and analysis

HIGHLIGHTS/SUMMARY:

A practitioner tool developed by SEEP under the AIMS project; reports on field testing on Honduras and Mali detail the findings produced by the survey (and other tools)

EXPERTISE NEEDED:

Training for two days on interviewing in general and on the use of the instrument; loan officer use.

EASE/TIME NEEDED FOR IMPLEMENTATION:

Between 50 and 60 minutes

TIME FRAME:

Can be implemented at any time desired by a practitioner organization

PERSONNEL NEEDED:

See Honduras and Mali reports

COST:

See Honduras and Mali reports

PROBLEMS/SHORTCOMINGS:

Data entry and analysis of 37+ questions requires skill and organizational time.

Analysis of responses can help an organization redesign its lending products, determine if business services are desired.

ORGANIZATION:

Women’s Opportunity Fund

DOCUMENT TITLE:

Exit Interviews (multiple version)

TITLE OF PROGRAM EVALUATED:

Trust Bank (or Village Banking model)

TYPE OF PROGRAM:

Group lending

COUNTRY(IES):

N/a

YEAR COMPLETED:

N/a

AUTHORS:

Variations

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Questionnaire with predetermined responses

CONCEPTUAL FRAMEWORK:

Survey upon leaving group lending program

HYPOTHESES:

Participation in MED program has positively impacted client individually and client’s business; program has met client’s needs

QUESTIONS ADDRESSED:

Program assessment, business growth, household changes

SAMPLING DESIGN:

--

TARGET PARTICIPANTS:

Clients leaving group lending program

DATA ANALYSIS:

(only conducted by AGAPE-summarized)

HIGHLIGHTS/SUMMARY:

--

EXPERTISE NEEDED:

For loan officer use

EASE/TIME NEEDED FOR IMPLEMENTATION:

Approximately 5-15 minutes

TIME FRAME:

Routinely implemented by loan officer when client leaves program

PERSONNEL NEEDED:

Loan officer

COST:

Minimal

PROBLEMS/SHORTCOMINGS:

Doesn’t assess sustainable impact of program on client

USE AS A MANAGEMENT TOOL:

Useful to examine if program is satisfying client needs, minimal impact assessment

ORGANIZATION:

UNICEF-Egypt

DOCUMENT TITLE:

Impact Assessment Study of the Family Development Fund, Egypt

TITLE OF PROGRAM EVALUATED:

Family Development Fund

COUNTRY(IES):

Egypt

YEAR COMPLETED:

1998

AUTHORS:

Monawar Sultana, Ashok Nigam

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Individual interviews using structured questionnaire, focus group interviews of members, and case studies of individuals

CONCEPTUAL FRAMEWORK:

A three tiered approach assesses quality of life impacts general financial impacts and compares them to a control group.

HYPOTHESES:

Program positively impacts improvements in material well-being, access to basic social services, status of women, and is institutionally sustainable.

QUESTIONS ADDRESSED:

Quality of life indicators such as health, nutrition, sanitation, child labor, education, empowerment of women, and financial indicators such as assets, income, debt level, and consumption

SAMPLING DESIGN:

156 of 1208 (13%) borrowers who received two or more loans in two years; control group of 46 of 1100 (4%) potential group members from same target group; 107 women participated in focus group discussions; 8 individual interviews for case studies

TARGET PARTICIPANTS:

Current clients with potential clients as the control

DATA ANALYSIS:

Tabulation of survey results and comparison with discussions and interviews

HIGHLIGHTS/SUMMARY:

Compares current clients with a control group of similar individuals; uses discussion groups and individual interviews to confirm the validity of the structured questionnaire; provides extensive information on quality of life indicators

EXPERTISE NEEDED:

Loan officer equivalent

EASE/TIME NEEDED FOR IMPLEMENTATION:

Can be completed in one month

TIME FRAME:

One-off evaluation

PERSONNEL NEEDED:

Loan officer equivalent

COST:

Loan officer time to administer surveys, conduct interviews, and analyze results

PROBLEMS/SHORTCOMINGS:

Questionnaire survey with 102 questions is quite long; survey relies on client recollection of changes over time; not clear how this assessment would fit into existing reporting processes; data analysis process not disclosed

USE AS A MANAGEMENT TOOL:

Can identify which level of clients are being reached and key limitations on their business potential which may influence the organization’s client targeting and repayments. Identifies clients’ loan use which can indicate the level of risk for the portfolio. Indicates the level of reliance on other sources of income which can suggest adjustments to lending terms.

ORGANIZATION:

Mennonite Economic Development Associates (MEDA)

DOCUMENT TITLE:

Mbeya Credit Facility Client Impact Evaluation

TITLE OF PROGRAM EVALUATED:

Mbeya Credit Facility (MCF)

TYPE OF PROGRAM:

Solidarity Group for Women

COUNTRY(IES):

Tanzania

YEAR COMPLETED:

1996

AUTHORS:

Corey Huntington (Waterloo, Canada)

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Qualitative/Participatory – semi-structured in-depth individual and focus group interviews, group discussions, personal narratives, structured questionnaire; follow-up interviews/group discussions/feedback seminars

CONCEPTUAL FRAMEWORK:

To develop a clear understanding of the economic and social impacts, both positive and negative, experienced by individual microentrepreneurs and their households as a result of receiving an MCF loan.

HYPOTHESES:

Impacts of the loan are both positive and negative; clients have learned different things from participation; clients have gained from their experience in a solidarity group.

QUESTIONS ADDRESSED:

Interviews: Client perspectives on loan product, skills and training offered by the Mbeya Credit Facility; role of the enterprise within the household. Survey: demographics, the business, loan, solidarity group, household status.

SAMPLING DESIGN:

100+ participants (representing 57 solidarity groups) for interviews; 50 for the survey

TARGET PARTICIPANTS:

Categories: current, late, very late, and graduates

DATA ANALYSIS:

Notes and summary transcriptions, descriptive summaries, all categorized. Quantitative analysis of survey questions.

HIGHLIGHTS/SUMMARY:

The study was not only to determine socio-economic impact on the client, her household, and her business, but also to determine reasons for nonpayment. The study determined that more appropriate client training and promotion of greater client participation in the program itself would increase positive impact.

EXPERTISE NEEDED:

Ability to conduct in-depth, open-ended interviews; ability to deliver a 94-question survey with both pre-determined and open-ended questions.

EASE/TIME NEEDED FOR IMPLEMENTATION:

N/a

TIME FRAME:

Approximately 5 months (April through August)

PERSONNEL NEEDED:

Researcher, translator, survey interviewer

COST:

$3,700 (researcher’s living expenses in the field, local staff salaries, transportation, photocopying, producing of final report, contingency fund); amount includes $200 to cover the cost of 5 persons implementing 20 surveys each.

PROBLEMS/SHORTCOMINGS:

The length of time needed to complete interviews was attributed to the discovery of fraud in the organization and the desire of clients to discuss this as opposed to being respondents in the research. Also, because of the investigation, lending was suspended which prevented the weekly cash-flow monitoring.

USE AS A MANAGEMENT TOOL:

Findings determined how MEDA could assist clients in becoming more involved with increasing repayments.

ORGANIZATION:

Save the Children

DOCUMENT TITLE:

Group Guaranteed Lending and Savings Programs in the West Bank and Gaza

TITLE OF PROGRAM EVALUATED:

Baseline Survey

TYPE OF PROGRAM:

Microfinance

COUNTRY(IES):

Palestinian West Bank & Gaza

YEAR COMPLETED:

1997

AUTHORS:

Khalid Nabris

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Questionnaire survey of 411 active borrowers and 20 focus group discussions with borrowers, their husbands, and male and female children; also, a questionnaire survey of 59 dropouts and three regional workshops with loan promoters were conducted

CONCEPTUAL FRAMEWORK:

In order to develop indicators for routine monitoring and impact evaluation, a baseline study should be undertaken

HYPOTHESES:

The group lending program results in both social and economic impact to borrowers, families and their children

QUESTIONS ADDRESSED:

--

SAMPLING DESIGN:

Survey questionnaires and focus groups; both qualitative and quantitative questions

TARGET PARTICIPANTS:

Representative sample of 411 borrowers, 59 dropouts and 20 focus group discussions with borrowers and families; workshops with field staff and program promoters

DATA ANALYSIS:

Tabulation of survey results

HIGHLIGHTS/SUMMARY:

Results give valuable information on quality of life indicators, particularly improvements in housing, increased expenditures on food and family needs, impacts on free time, etc.; results also unique in that they include input from children and borrowers’ families (including children)

EXPERTISE NEEDED:

Training manual already developed and could be used with existing agency staff

EASE/TIME NEEDED FOR IMPLEMENTATION:

8 weeks for interviews; final documentation 16 weeks after initiation of surveying

TIME FRAME:

One-off evaluation

PERSONNEL NEEDED:

Conducted by two teams, each consisting of three field workers and one coordinator

COST:

Loan officer time to administer surveys and analyze results

PROBLEMS/SHORTCOMINGS:

No clear process for incorporating surveying into routine monitoring and assessment; only a baseline survey; time intensive

USE AS A MANAGEMENT TOOL:

If standardized and incorporated into regular meetings, could be very useful for non-income impacts on clients

ORGANIZATION:

ACCION

DOCUMENT TITLE:

Measuring Client Success: An Evaluation of ACCION’s Impact on Microenterprises in the United States

TITLE OF PROGRAM EVALUATED:

ACCION, Albuquerque, Chicago, El Paso, New York City, San Antonio, San Diego

COUNTRY(IES):

USA

YEAR COMPLETED:

1998

AUTHORS:

Cristina Himes, Lisa Servon

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Quantitative loan application data compiled into a longitudinal database, and a controlled sample of interviews with open ended questions for supplementary qualitative data

CONCEPTUAL FRAMEWORK:

Collecting data at each loan application and interviewing a sample of clients can track the outcomes resulting from participation in the program

HYPOTHESES:

Measurable outcomes in business stability and growth (profits, equity, take-home income) result from the program; initial income levels affect measurable outcomes; clients are fulfilling qualitative goals by participation in the program

QUESTIONS ADDRESSED:

Employees, experience with the program, household finances, and business finances, employment history, personal impact

SAMPLING DESIGN:

72 clients interviewed (4% of entire client population) for qualitative information from all program sites’ types of clients (financial growth clients, job creators, no-growth clients, and one-loan clients); quantitative information on consumption and income entered from loan applications

TARGET PARTICIPANTS:

Current clients

DATA ANALYSIS:

Trend and regression analysis on the quantitative data as well as compilation and comparisons for the qualitative interview data

HIGHLIGHTS/SUMMARY:

Results give detailed longitudinal analysis of clients’ financial and business capacity; extensive financial information captured from each loan application

EXPERTISE NEEDED:

Loan officer equivalent, trained interviewers, data analysts

EASE/TIME NEEDED FOR IMPLEMENTATION:

40 min-1 hour per interview plus time for analysis

TIME FRAME:

Quantitative portion is on-going, qualitative one-off evaluation

PERSONNEL NEEDED:

Non-loan officer interviewers (conducted by an intern and Director of US Operations from within organization)

COST:

Hiring interviewers, capacity for data analysis

PROBLEMS/SHORTCOMINGS:

No control group for comparison; database is entirely numerical; interview with 95 questions is quite long; targeted toward a higher educational level

USE AS A MANAGEMENT TOOL:

Database is useful for longitudinal studies and to track progress toward strategic objectives answering questions such as what type of businesses do our clients have, is their total income increasing over time, are they employing more people, how are they using their loans; decisions on targeting the program, focus of training for clients, and the appropriateness of loan terms; the qualitative study confirms the quantitative results

ORGANIZATION:

FINCA-Nicaragua

DOCUMENT TITLE:

Report on Client Targeting, Turnover & Program Impact

TITLE OF PROGRAM EVALUATED:

Group Lending in Nicaragua

COUNTRY(IES):

Nicaragua

YEAR COMPLETED:

1998

AUTHORS:

John Hatch, Todd Manwaring, Meredith Terrell, & Judd Horn

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Intake questionnaire for new clients (2 pages) and exit interview for withdrawing clients (4 pages)

CONCEPTUAL FRAMEWORK:

Assessing household income and poverty level can be accomplished through use of proxy indicator, such as household expenditures

HYPOTHESES:

To what extent is FINCA-Nicaragua recruiting new clients who are severely poor? Which characteristics other than income that closely correlate with very poor clients?

QUESTIONS ADDRESSED:

Total monthly household expenditures were converted to daily per capita income equivalents; severe, moderate and non-poor poverty levels determined using World Bank definitions for annual and urban poverty lines

SAMPLING DESIGN:

177 clients total interviewed; 90 new, 61 withdrawing and 26 ex-clients

TARGET PARTICIPANTS:

New, withdrawing and ex-clients

DATA ANALYSIS:

Tabulation of survey results

HIGHLIGHTS/SUMMARY:

Results give very detailed and very valuable information on success in client targeting and reaching "poorest of the poor"; also, extensive information on causes of client withdrawal also indicate quality of life impacts (e.g., poor health, sufficient savings, group dynamics, etc.)

EXPERTISE NEEDED:

Loan officer equivalent

EASE/TIME NEEDED FOR IMPLEMENTATION:

6 weeks for interviews, analysis and write-up of findings

TIME FRAME:

One-off evaluation

PERSONNEL NEEDED:

Conducted by three interns

COST:

Loan officer time to administer surveys and analyze results

PROBLEMS/SHORTCOMINGS:

No clear process for incorporating surveying into routine monitoring and assessment; no baseline data or control groups available; concerned primarily with income poverty measurement and indicators, although perceptions of non-income benefits by withdrawing clients also assessed

USE AS A MANAGEMENT TOOL:

Results provide very valuable program information that differentiates between new, withdrawing and ex-clients; can be used to adjust methodology and deal with problem products/services, especially for the group of "withdrawing" clients.

ORGANIZATION:

AIMS

DOCUMENT TITLE:

An Assessment of the Impact of Microfinance Services in Uganda

TITLE OF PROGRAM EVALUATED:

FINCA, PRIDE, FOCCAS

TYPE OF PROGRAM:

Microenterprise Services (USAID funded)

YEAR COMPLETED:

1997, 1998

AUTHORS:

Drs. Carolyn Barnes, Gayle Morris, and Gary Gaile

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Rural and Urban Questionnaire with predetermined and open responses

CONCEPTUAL FRAMEWORK:

Assessment of impact on household resources, household activities, and circular flow of interaction between household resources and household activities, assuming microenterprises are embedded in households.

HYPOTHESES:

Impacts from microenterprise services occur on the household level, the enterprise level, and the individual level.

QUESTIONS ADDRESSED:

Individual level: greater control of resources, increased ability to plan for future.

SAMPLING DESIGN:

Stratified random samplings from selected sites, client and non-client groups

TARGET PARTICIPANTS:

Clients that received loans in August-October 1997

DATA ANALYSIS:

Statistical testing

HIGHLIGHTS/SUMMARY:

A practitioner tool developed by AIMS for USAID/Uganda

EXPERTISE NEEDED:

Training for one week on microfinance, interview techniques, explanation of questionnaires, mock interviews, field administration of questionnaires, related problems

EASE/TIME NEEDED FOR IMPLEMENTATION:

Able to cover three respondents per day; interview for an hour and twenty minutes

TIME FRAME:

Open

PERSONNEL NEEDED:

--

COST:

--

Training needed for use of tool, analysis of tool fairly complicated

USE AS A MANAGEMENT TOOL:

Able to determine impact, actual client group vs. target group, product effectiveness

ORGANIZATION:

BRAC

DOCUMENT TITLE:

Poverty Alleviation and Empowerment (Impact Assess. Study on BRAC’s Rural Development Programme (IAS-II)

TITLE OF PROGRAM EVALUATED:

BRAC’s Rural Development Programme

TYPE OF PROGRAM:

Rural Development

COUNTRY(IES):

Bangladesh

YEAR COMPLETED:

1993-2000

AUTHORS:

Impact Assessment Team, A.M.Muazzam Husain, ed.

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Household survey, qualitative and case studies, village profile

CONCEPTUAL FRAMEWORK:

N/a

HYPOTHESES:

The Rural Development Program impacts the material well-being of the participants, including a reduction in poverty. BRAC’s program impact is sustainable.

QUESTIONS ADDRESSED:

Seasonal economic vulnerability, coping capacity of participants, analyses of women’s empowerment, panel data, membership performance and coverage of the program. Social and economic aspects of well-being covered in household survey; empowerment, member performance and rural development covered in qualitative studies. Inputs: credit, training, technical assistance, amount of land, gender, occupational status of HH, women’s involvement, changes in assets, savings, net worth, expenditure, improvements in housing, health, and family planning.

SAMPLING DESIGN:

1700 households with 1250 BRAC participants and 250 comparison households (selected at random) and 200 "success" households selected intentionally. 25 area offices selected at random, with 10 panel offices and 15 non-panel offices.

TARGET PARTICIPANTS:

Participants in the Rural Development Programme

DATA ANALYSIS:

Multivariate analysis, Fixed Effect Model, SPSS computer software

HIGHLIGHTS/SUMMARY:

Provides: record of achievement in poverty reduction with comparison over time, comparative analysis of households benefiting from BRAC and those not, assessment of impact relating to poverty.

EXPERTISE NEEDED:

Conducted by trained team

EASE/TIME NEEDED FOR IMPLEMENTATION:

Extensive: comparison study conducted over multiple years

TIME FRAME:

--

PERSONNEL NEEDED:

--

COST:

--

PROBLEMS/SHORTCOMINGS:

Extensive survey done incrementally over multiple years; complex data analysis

USE AS A MANAGEMENT TOOL:

Useful in examining if program is meeting its goals, if it is having a sustainable impact on poverty.

ORGANIZATION:

The Bridge Foundation

DOCUMENT TITLE:

Self-Evaluation Guidelines

TITLE OF PROGRAM EVALUATED:

The Bridge Foundation - All Clients

COUNTRY(IES):

India

YEAR COMPLETED:

1990, 1993

AUTHORS:

N/A

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

CONCEPTUAL FRAMEWORK:

Changes in 10-point poverty level scale including the following indicators: total family income, food, clothing, shelter, education, health/medical facilities

HYPOTHESES:

Family income increases due to business financing.

QUESTIONS ADDRESSED:

Client’s non-business income; client’s business income; total family income; actual business income; actual additional business income; number of jobs projected; actual number of jobs created

SAMPLING DESIGN:

All clients quarterly

TARGET PARTICIPANTS:

All clients – individual and group borrowers

DATA ANALYSIS:

Not provided

HIGHLIGHTS/SUMMARY:

--

EXPERTISE NEEDED:

Loan officer equivalent

EASE/TIME NEEDED FOR IMPLEMENTATION:

N/a

TIME FRAME:

Conducted quarterly

PERSONNEL NEEDED:

Loan officer time

COST:

Regular administrative cost as evaluation is built into normal operations

PROBLEMS/SHORTCOMINGS:

Definition of all 10 points on the grid not given; not well formatted; published for internal use only

USE AS A MANAGEMENT TOOL:

Results indicated a need to refocus program to target poorer people, especially women, living outside of Bangalore in more rural areas. Mixed gender groups do not allow women’s empowerment, so group lending targets women only.

ORGANIZATION:

Canadian Co-Operative Association

DOCUMENT TITLE:

Village Center Monitoring Program

TITLE OF PROGRAM EVALUATED:

Canada/China Women’s Income Generating Project

COUNTRY(IES):

China

YEAR COMPLETED:

1997

AUTHORS:

--

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Survey with pre-determined responses; open-ended questions on community development issues; open-ended question on group dynamic issues.

CONCEPTUAL FRAMEWORK:

N/a

HYPOTHESES:

Changes in village economic, social and political situations can favorably or unfavorably influence clients in pursuing the development and expansion of their microenterprises.

QUESTIONS ADDRESSED:

Economic indicators including household income and business performance; household information including education of children and sanitation; attitudes toward the project, gender roles and participation in group meetings.

SAMPLING DESIGN:

Includes collection of baseline data, monthly monitoring and annual evaluation of key indicators.

TARGET PARTICIPANTS:

Center members, segregated by sex (there is a women’s questionnaire and a men’s questionnaire).

DATA ANALYSIS:

Results entered into local computerized information management system.

HIGHLIGHTS/SUMMARY:

This tool includes a broad array of questions that address the community context in influencing the success and dynamic of the credit groups. It also asks extend questions about group participation and gender roles, which complement the financial information and economic indicators.

EXPERTISE NEEDED:

The tool includes a brief outline and instructions for project officers, loan officers or others who might administer the tool

EASE/TIME NEEDED FOR IMPLEMENTATION:

Approximately 3-pages, including open-ended questions.

TIME FRAME:

Done as a baseline first, with monthly updates and an annual evaluation.

PERSONNEL NEEDED:

Loan officer or equivalent.

COST:

N/a

PROBLEMS/SHORTCOMINGS:

The results of the application of the tool were not provided, making assessment of practical usefulness difficult. Also relies on good interviewing/facilitation skills of loan officer in case clients are unclear about open-ended questions. The tool is heavy on community impact, gender and group participation questions, and weaker on financial and economic indicators (for the individual, household or business).

USE AS A MANAGEMENT TOOL:

If program managers are concerned about community impact, gender issues and group participation/leadership dynamics, this tool provides many useful questions. Also, the sex-segregated questionnaires are unique and may be appropriate in some contexts while not in others.

ORGANIZATION:

Grameen Bank

DOCUMENT TITLE:

Client Impact Measures

TITLE OF PROGRAM EVALUATED:

Ten Indicators for Measuring Poverty Reduction

COUNTRY(IES):

Bangladesh

YEAR COMPLETED:

1997 and 1998

AUTHORS:

Dipal Chandra Barua

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

Survey Form

CONCEPTUAL FRAMEWORK:

Efficiency of microcredit in reducing poverty and finding ways to refine the microcredit tools further.

HYPOTHESES:

Microcredit contributes to the economic welfare of poor households.

QUESTIONS ADDRESSED:

Household income, diversification of income sources, loan repayment, three square meals per day, clothing, housing, medical care, children’s education, sanitation and water sources; number of clients who have passed poverty line.

SAMPLING DESIGN:

Grameen Bank members who are associated with Grameen Bank for three years and above.

TARGET PARTICIPANTS:

Grameen Bank Members

DATA ANALYSIS:

Statistical analysis of survey data

HIGHLIGHTS/SUMMARY:

Survey is designed to assess quality of life indicators for Grameen Bank members. The results/findings are preserved in a data base form in the computer for further analysis and measuring the poverty level among the Grameen Bank borrowers. The results are needed to initiate policy options by the management for developing appropriate poverty reduction tools.

EXPERTISE NEEDED:

Loan officer level

EASE/TIME NEEDED FOR IMPLEMENTATION:

13 simple yes/no questions; easy and quick to implement

TIME FRAME:

Can be incorporated into quarterly or annual monitoring efforts.

PERSONNEL NEEDED:

Loan officers or equivalent can administer; computer personnel with basic computer knowledge

COST:

Low-cost

PROBLEMS/SHORTCOMINGS:

Baseline questions are not included. The simple survey format does not allow for client participation, except to answer "yes/no."

USE AS A MANAGEMENT TOOL:

This tool can provide a cursory overview of food, clothing, shelter and education impacts among clients. However, the questions do not clearly demonstrate whether improved quality of life indicators are a result of the loan methodology, group participation, or other factors. The information is used to refine the existing microcredit tools for poverty reduction and identify the deficiency for taking policy actions that would contribute towards improving the socioeconomic conditions of the target clientele.

ORGANIZATION:

CARE-Bangladesh

DOCUMENT TITLE:

Impact Evaluation of Income Generating Projects: A Training Course for NGO Staff on Assessment of the Economic and Social Benefits of Income Generation

TITLE OF PROGRAM EVALUATED:

CARE – Bangladesh Income Project

COUNTRY(IES):

Bangladesh

YEAR COMPLETED:

Manual completed 1995

AUTHORS:

Anne Ritchie, Fahmid K. Bhuiya and Harun-or-Rashid

METHOD(S) OF EVALUATION (INCLUDING DATA COLLECTION TECHNIQUES):

For economic benefits at the individual level, the tools are a simple survey and a PRA technique called the "Chapati Diagram." For those on the household level, a poverty profile and a PRA technique called the "Seasonal Calendar" are used. For social benefits on the individual level, a PRA technique called a "Time Line," a simple survey, and knowledge testing are used as tools. For the group level, a PRA tool called the "Time Line" is used.

CONCEPTUAL FRAMEWORK:

Impact Evaluation at the beneficiary level focuses on economic and social benefits. Economic benefits focus on the individual and household levels; social benefits focus on the individual and the group level (the latter is optional).

HYPOTHESES:

QUESTIONS ADDRESSED:

Indicators under individual economic change: participation in economic activities, labor time, perception of participant’s contribution to household income, savings, and expenditure of IGA income. Indicators under household economic change: living standard, financial dependency, stability of income, income.

SAMPLING DESIGN:

Simple random sampling based on random numbers table provided

TARGET PARTICIPANTS:

Group and individual clients

DATA ANALYSIS:

Instructions for quantitative and qualitative data analysis are provided in the manual, but there is no indication as to how long it actually takes.

HIGHLIGHTS/SUMMARY:

--

EXPERTISE NEEDED:

Mid-level managers are suggested

EASE/TIME NEEDED FOR IMPLEMENTATION:

Good guidelines are given for the implementation of the PRA and the survey tools, and these can be done on an annual basis to determine changes

TIME FRAME:

N/a

PERSONNEL NEEDED:

N/a

COST:

N/a

USE AS A MANAGEMENT TOOL:

N/a

PROBLEMS/SHORTCOMINGS:

The tools and the course manual are very instructive. It would be good to have an actual impact evaluation conducted on the basis of the manual to determine the usefulness of the questions, the cost to implement the study, the time requirement, etc.