Health Systems and Policy Research

Introduction

Focus of public health research has traditionally been on descriptive and analytical studies and there is relative lack of implementation research (IR) particularly in developing countries [1-3]. As opposed to basic research conducted under controlled or ideal conditions with strictly defined inclusion criteria, IR projects are implemented in the resource constrained health systems facing multiple barriers; political, economic and social etc. [2,4]. It is these factors combined with the complex nature of health systems that makes IR challenging. Thus it takes beyond study deigns and study methodologies to answer research questions. Role of external factors greatly influence the outcome of interventions. These range from inadequate financial resources within the national programs and insufficient or less functional human resources to rigid political milieus leading to low acceptability of interventions [5,6].

IR is dependent on factors specific to study settings such as population characteristics, environment and health systems [7]. Every health system is unique in that it caters to a population with specific health needs. This makes IR context specific as uniqueness of each health system leads to specific set of challenges such as combating HIV/AIDS [8] and Malaria [6] in African countries. A key feature of IR is that it seeks to work with all relevant stakeholders including program managers, policy makers and communities from the start of the project. This, although challenging, helps in identification and removal of implementation barriers and facilitates the integration of the intervention into existing health system and/or national programs [2,3,9]. However, literature shows dearth of documentation of health system level challenges while executing implementation research.

Pakistan’s National program for family planning and primary healthcare (lady health workers program [LHW-P]) has been a land mark in the health system’s history for being the first program to integrate community health workers (CHWs) into existing health infrastructure. An important objective of the program was to provide care for children with pneumonia and diarrhea, the two major causes of mortality in children under five. While uptake of family planning and ante-natal care service has improved because of LHWs, the impact of their services on improving coverage and quality of diarrhoea and pneumonia treatment is unremarkable [10-12]. Independent studies including external evaluations of the program shed some light on the factors responsible for this poor outcome. These are poor clinical mentorship and supervisory skills of Lady Health Supervisors (LHS) and dysfunctional supervisory links between LHSs and LHWs [11,13-18]. In order to bridge these supervisory gaps, the Aga Khan University Pakistan in collaboration with health department, government of Sindh introduced an IR project called NIGRAAN. NIGRAAN (Urdu word meaning supervisor), a cluster randomized trial was designed to improve LHWs and community caregivers (CCGs) practices in community case management (CCM) of diarrhea and pneumonia by enhanced supportive supervision through LHSs. However, considering the context specific barriers and the structural gaps in the LHW-P, implementation of NIGRAAN was nevertheless challenging.

This study describes the barriers and challenges encountered while implementing various phases of project NIGRAAN.

Methods

Study setting:

Project NIGRAAN was rolled out in district Badin, province of Sindh. Badin is a coastal district with 84% of its population living in rural areas. In addition to a network of primary and secondary level healthcare facilities [19], there is an operational LHW-P in the district to cater to the maternal and child healthcare needs of the population (Table 1) [17].

Estimated population	1.2 million
Basic health units (BHUs)	35
Rural health centers (RHCs)	9
Taluka head quarter hospitals	4
District head quarter hospital	1
Source: District Disaster Management Authority. Disaster Risk Management Plan District Badin Government of Sindh (18)

Table 1: Estimated population and public sector health infrastructure in District Badin.

LHW program and its infrastructure in Badin: LHWs are the backbone of the outreach component of primary health care in Pakistan. LHW-P in Badin comprises of 1097 LHWs supervised by 36 LHSs whereas 45% of the area remains uncovered [20]. An LHW is a salaried staff with a minimum of eight years of schooling and preferably married. She is a resident of the same area where she works serving approximately 100 – 150 households (1000 population on average) [10]. An LHS is the immediate supervisor of LHWs and is responsible for monitoring and supervision of LHWs according to a pre-defined checklist. Like an LHW, LHS is also a local resident of the area, educated and having experience of working as an LHW or lady health visitor (LHV). Each LHS supervises about 15 – 25 LHWs and is expected to make two supervisory visits each month, guide and address the concerns of LHWs. LHWs submit monthly progress reports to LHSs who in turn report to assistant district coordinator (ADC) based at District Program Implementation Unit (DPIU) [21].

NIGRAAN intervention and study participants: For this study, each LHS served as a cluster. All LHSs working permanently in Badin and 5 randomly assigned LHWs per LHS constituted the study population. Thirty four functional LHSs were allocated to either intervention or control arms in a ratio of 1:1. The intervention consisted of training to build LHS knowledge and skills, clinical mentorship and written feedback to LHWs. Pre- and post-intervention assessments of LHSs, LHWs and community caregivers were conducted via focus group discussions, indepth interviews, knowledge assessment questionnaires, skill assessment scorecards and sample household surveys. For the purposes of monitoring and evaluation through continuous skills assessments and measuring change in LHSs/LHWs performance as a result of intervention, NIGRAAN team established an active case reporting surveillance system. Based on Short Messaging

Service (SMS) through simple mobile phones, the surveillance system aimed to ensure timely case reporting and follow-up visits by LHWs and LHSs respectively for the management of cases of diarrhea and pneumonia. The surveillance system was superimposed on the routine management information system of the LHW-P.

The main study outcome was improvement of CCM for childhood diarrhea and pneumonia. Further details of the study are published elsewhere [17].

Data collection methods to extract information on the implementation challenges: Various methods including a focus group discussion (FGD), six in-depth interviews (IDIs) and a group interview were conducted with research and field team members to extract information on implementation challenges encountered during execution of NIGRAAN. Structured interview guides were developed for FGD and IDIs whereas an open ended (unstructured) interview guide was developed for the group interview (Figure 1). A group interview was an in-depth interview targeted at a group of people (field team in our study) with the intention of gaining information and insights on the field implementation challenges of NIGRAAN, with communication being bi-directional between the interviewer and the interviewees without involving discussion between the group participants [6].

Figure 1: Study participants, data collection methods and sources of information used to document implementation challenges of NIGRAAN.

In addition, documents including minutes of NIGRAAN meetings and the report of NIGRAAN’s year 1 activities were also reviewed to get additional information on the implementation challenges. Figure 1 gives information on study methods, tools and participants.

Data Analysis: All FGDs, IDIs and group interviews were transcribed in English. After reading all the transcripts brief notes were made at the points of interesting and relevant information. Process of data analysis was inspired by Braun and Clark’s thematic analysis method leading to researchers getting familiarized with the data while searching, reviewing and naming themes [22]. Using this method, relevant probes were picked from the transcribed data by manual content analysis and linked to the identified theme. Process of data analysis involved feedback sessions with the study participants for sharing and validating the study findings. All disagreements were discussed with the participants and consensus was reached on the overall analysis. The transcripts were reviewed again in the end to make sure that the necessary information had been captured. Several common themes emerged from categorization as reported in the results section.

Results

Engaging community and the LHW-P stakeholders: Culture of making decisions in silos

To increase ownership of NIGRAAN, policy makers and LHW-P stakeholders including LHSs and LHWs were engaged early on, by formulating an executive district project management team (EDPMT). First meeting of EDPMT was conducted before starting the project work. When interacting with policy makers, it was observed that the culture of ad hoc decision making was very common where the policy makers relied on making phone calls for problem solving and facilitating various operations e.g. given lack of transport and medical supplies, provincial level representative of the LHW-P contacted relevant personnel to make the required resources available. What was missing was an evidence-based approach to address these structural gaps leading to a permanent solution. Progress of the system was rather dependent on individuals instead of an integrated functional system [23].

In the initial phase of the project, LHSs and LHWs were apprehensive of NIGRAAN due to fear of increased workload, however, presence of a LHW-P focal person i.e., ADC proved crucial in successful implementation of NIGRAAN as she facilitated cooperation and involvement of all LHSs and LHWs.

For successfully conducting the various activities, building stronger communication with the community and the district level stakeholders of the LHW-P was important. A challenge was that pre-existing formal community committees were found to be non-functioning in the study clusters requiring research team to find an alternative. Research team identified a resource person from the same community as community mobilizer who facilitated the communication between the team and the community. His facilitation helped in gaining community’s trust and enhanced the cooperation during household surveys conducted by the project.

Issues for consideration in selection and demarcation of study site

As pointed out in the external evaluation of the LHW-P, the program suffers from structural gaps and resource deficiencies leading to suboptimal performance in majority of the districts [11]. Considering these gaps, selection of study site was done by convenience because for NIGRAAN intervention to be implemented, it was important to have a functional LHW-P in the selected district. With 34 out of 36 LHSs along with respective LHWs actively involved in service delivery, Badin was considered as a suitable site. This was despite the fact that the LHW-P in district Badin was facing issues; lack of transportation and medical supplies to name a few. Two other districts namely Hyderabad and Thatta, though convenient, were not considered as there was possibility of parallel major interventions through government and other non-governmental organizations being implemented in these areas with resultant duplication of efforts.

Demarcating the intervention and control clusters in Badin required that research team be provided with a map of the study site. Each LHW is expected to have hand sketched maps of her catchment area which need to be regularly updated. However, such maps were unavailable or outdated emphasizing the need for creation of new ones. Since it was a rural site, there were no clearly demarcated boundaries, house numbers or prominent landmarks to define the catchment areas of LHWs. To resolve this issue, research team conducted meetings with LHSs, LHWs and the district polio teams. This added to the field team’s geographic understanding of the study site leading to accurate demarcation of the catchment area.

Following creation of handmade maps, these were geocoded. Use of geographic information system technology was challenging for the field team as this required multiple trainings. At first, community was concerned about their homes being mapped however after counseling by the community mobilizer this problem was resolved.

Challenges in selection of required health workers as study participants

34 LHSs and 170 LHWs (17 LHS & 85 LHWs in each arm) constituted the required sampling frame. Over time it was realized that many LHSs could not make a supervisory visit to the area of the randomly chosen LHW because of lack of transport. Further, due to scattered population and problematic road infrastructure of the rural terrain, it was difficult for LHWs to visit assigned households hence the coverage was less than expected. The researchers found it challenging to obtain updated lists of all LHWs attached to specific LHSs by assigned households. The lists provided were mostly hand written with illegible handwriting. The process of compiling a correct list of health workers was thus time consuming with back and forth transfer of documents between LHW-P and research team.

Issues in assessment of the knowledge and skills of LHSs and LHWs

Knowledge and skills assessments of LHSs and LHWs were conducted pre-intervention and then every six months after NIGRAAN intervention was launched. Knowledge was assessed using written tests comprising of short answer questions whereas for skills assessments independent evaluators (IE) were hired who observed both LHW and LHS while performing clinical assessment and giving feedback on a reported case of diarrhea and pneumonia, respectively. The periodic knowledge assessments showed that LHSs/LHWs found it difficult to answer scenario based questions in terms of writing even in local languages (Sindhi or Urdu) although they could read well. This indicated their lack of familiarity with written mode of assessment emphasizing the need for careful consideration in selecting the knowledge assessment methodologies because if the person being assessed cannot communicate clearly via selected method of assessment, then evaluators may not capture his/her true comprehension of knowledge.

In order to conduct knowledge and skills assessments, sampling methodology required a minimum of 18 – 25 unique cases of pneumonia and diarrhea to be randomly selected from each of the intervention and control arms. However, fulfilling the required number of assessments was difficult in the initial phase of the project due to low case reporting and the fact that the cases were being reported by the same 10-12 out of 85 LHWs in each arm. This was because of the fact that the culture of timely case reporting was uncommon in the study district before NIGRAAN was introduced. LHWs are required to give only a monthly report to the program office irrespective of the fact as to when the case was identified. ADC assisted in coordinating meetings with LHSs/ LHWs to raise their motivation level leading to a subsequent increase in reporting frequency.

Moreover lack of transport precluded LHSs to make supervisory visits to remote and distant areas. NIGRAAN had to make transport arrangements to ensure timely supervisory visits and skill assessment of health workers.

Another key challenge in doing periodic assessments was unavailability of qualified human resources mainly the IEs. Due to cultural reasons, persons doing skill assessments of LHSs and LHWs i.e., IEs had to be women, however, it was challenging to find literate and qualified ladies. Since in this rural setup, LHS and LHV were amongst the few literate female candidates, research team selected senior LHS (not related to NIGRAAN) and LHVs for this job. An LHV is a cadre of community health workers in Pakistan whose scope of work includes conducting deliveries as skilled birth attendants and provide nutritional and health counseling to mothers and children [24,25].

Challenges in setting up an active SMS based case reporting surveillance system

For the periodic skill assessments of health workers overtime, NIGRAAN established an active case reporting surveillance system. Based on SMS, the surveillance system aimed to ensure timely case reporting and follow-up visits by LHWs and LHSs for the management of cases of diarrhea and pneumonia.

It is important to consider that despite case reporting being an integral part of the LHW-MIS, NIGRAAN’s surveillance found low frequency and quality of reporting. Similarly, supervisory visits by LHSs were uncommon before NIGRAAN was introduced.

Consequently, similar trend of low case supervision continued in the first quarter after NIGRAAN was launched. Due to feasibility, LHWs/LHSs were more comfortable in communicating via SMS as filling up the surveillance forms required effort, time and careful consideration. Issues of mobile connectivity also created some hurdles when coordinating a follow-up visit. Poor network coverage, frequent changes in mobile numbers of care-providers and poor response on the phone were some of the challenges making telephonic follow-up of reported cases difficult.

Compliance with providing written feedback

Despite written feedback by LHS being an integral part of the LHW-P, the practice was rare prior to introducing the NIGRAAN intervention. Due to lack of culture of accountability the introduction of written feedback was difficult to install in the early phases as it enforced accountability within the system. However, by the last quarter of surveillance the distribution of feedback cards rose to 94% with continuous reinforcement by the research team and ADC.

Workload Overburdens LHWs/LHSs

Being an important community health workforce, LHWs and LHSs were frequently overburdened with several other responsibilities outside their major charter of duties including measles and polio supplemental immunization campaigns. Due to the latter being quite frequent, LHWs and LHSs were better recognized in the community as “polio workers” instead of community health workers. Hence, they were left with less time to focus on their primary responsibilities.

Randomized controlled trials involving behavior change

Proposal development was carried out in consultation with donors and extensive discussions were done on the selection of the type of study design for NIGRAAN. When it comes to quality and strength of evidence produced, randomized controlled trials clearly have an edge over other study designs because of characteristics like randomization, blinding and presence of a control group. However, large scale community based trials involving behavioral interventions are always challenging due to the fact that blinding of groups or individuals for behavioral intervention is not always possible. Focus of NIGRAAN intervention was on improving the supervisory mechanisms within the LHW-P which involved change in practices of LHSs and LHWs in the intervention arm. Study clusters were located in the same district with frequent interactions taking place between LHSs and LHWs of both intervention and control arms in the official meetings at LHW-P offices. Knowledge and skills assessments showed equal improvements in both the arms indicating a likely contamination.

Funding for post-intervention assessment

In order to assess retention of skills and knowledge among LHSs and LHWs, research team realized that a post-intervention skills assessment was necessary. This was important in order to determine the sustainability of intervention. However, there was no provision for it in the original grant. So while the intervention was going on, research team submitted a proposal to the granting agency in order to convince them about the importance of conducting a post-intervention assessment. A grant was secured to allow a six months post-intervention knowledge and skills assessment.

Discussion

Doing IR in real life settings is challenging. Engaging community, LHW-P and political leadership; structural issues of the LHW-P along with perplexing geography of the study site; establishing a culture of accountability within the LHW-P; non-availability of qualified human resource and having a randomized controlled trial (RCT) design for a behavior change intervention were some of the key challenges encountered during implementation of NIGRAAN.

In NIGRAAN, engaging policy makers and LHW-P leadership early on proved to be a key to successful implementation. Similar large scale community based RCTs and health system projects have proved that the engagement of communities, local and national political leadership is crucial to effectively roll out the health interventions [6,26]. In a community based cluster RCT in Zambia, researchers reported study participation as excellent due to close collaboration with political, administrative and community stakeholders. A trial conducted in Tanzania to demonstrate the effectiveness of improved treatment services for sexually transmitted infections in preventing HIV infection is an example of how linkages with policy makers help translation of research into policy. Communication ties between researchers and policy makers before trial implementation increased the trust of stakeholders in research findings resulting in a policy shift [8].

Understanding culture and societal norms of the community beforehand is important because it helps identify the community’s research needs, guides design and tools of intervention and helps build bilateral trust between community and the research team. In NIGRAAN, conducting FGDs and IDIs during the formative phase with policy makers and LHSs/LHWs helped understand community’s perceptions about the LHW-P. This in turn facilitated research team in tailoring the intervention and its tools to community needs. Other studies have also demonstrated the role of a formative research phase in providing insights into cultural norms and community acceptability to facilitate study implementation by improving community cooperation [26]. Literature shows that in rural areas, people are unaccustomed to unknown or new visitors and to research; and the usual way to get to them is through their trusted leaders or people from their own community. In a rural area of Manguzi, South Africa the research team involved tribal leaders to get community buy-in [27]. In NIGRAAN, community buy-in was ensured by selecting a community representative as the social mobilizer who facilitated communication with the community.

Studies conducted to evaluate knowledge and skills of CHWs have mostly relied on direct observations in simulated or real life clinical settings instead of written assessments. In Uganda, CHWs were evaluated for their ability to identify childhood pneumonia by skills demonstration in a hospital setting and in the presence of observers [28]. In another study, maternal and child health workers were assessed for knowledge and skills by conducting interviews and demonstrating clinical skills in a simulated environment [29]. In NIGRAAN, direct observations for skills assessment by IEs provided better comprehension of the LHWs’ skill levels compared to written assessments which LHSs/LHWs found difficult to attempt. This highlights the importance of direct observations as a preferred mode to assess CHWs knowledge and skills.

CHW programs throughout the world have been reviewed extensively. A review conducted by world health organization (WHO) has highlighted the structural issues common to most CHW programs i.e., lack of integration with main health systems, shortage and irregular supply of medicines and logistics, lack of recognition by community, inadequate remuneration and weak supervision. Impact of these structural gaps in turn has shown to have a negative influence on health workers’ performance [30,31]. Literature shows that dysfunctional supervisory mechanisms and weak answerability lead to a culture of impunity resulting in weak service performance [32,33]. Due to a weak culture of accountability, initially it was difficult to establish a case reporting system for diarrhea and pneumonia as part of NIGRAAN surveillance system. LHSs/LHWs did not feel accountable to an external project resulting in low frequency of reported cases in the initial quarters of implementation. Instilling a culture of responsibility was gradual and was made possible by including LHW-P program representative (ADC) at district level. Though presence of ADC proved useful in NIGRAAN implementation, it also highlights a system level challenge where dependency on individuals is so strong that their absence can become threatening to the system’s sustainability.

Given the rural nature of the study site, lack of proper road infrastructure and land marks it was difficult to access and geocode the households. Moreover, scattered and sparsely located populations posed a significant challenge as reaching them was resource intensive requiring proper transport. Similar challenges were encountered while conducting research in a rural African community. Flat terrain and lack of road signage (in presence of bushes) made it difficult to locate and re-locate the households despite availability of geo-mapping [27].

Due to lack of educational and career development opportunities in rural areas, finding qualified and trained personnel (like a neutral IE) is very difficult [34]. Studies from Malawi and other developing countries have pointed out that inadequate opportunities exist for continuing education and career progression. This affects staff retention in rural areas [35,36]. NIGRAAN also faced similar challenge while recruiting and retaining IEs.

Compared to an RCT dealing with a drug efficacy or a vaccine trial, RCTs dealing in HSR are faced with several external influences such as program leadership; availability and consistency of resource supply, CHWs characteristics such as experience, morale, workload and team work; issues in access to outreach populations and characteristics of the population served to name a few. For example we observed that because of lack of medicine and supplies the credibility of LHWs in community was generally low. However, (in the context of a curative oriented society) they were recognized better when they approached community with a tangible thing in hand viz “polio vaccines”. A comprehensive review of CHW programs by WHO has highlighted the provision of requisite core supplies to CHWs by the programs as its key finding in order to enable their optimal performance and increased community recognition [30].

In cluster RCTs involving behavior change, blinding is not always possible [37]. Hence, spillover effect of intervention into control arm may be anticipated such as during informal gatherings and professional staff meetings particularly if the intervention is being implemented in the same district under a common administrative infrastructure [38-41]. In NIGRAAN LHSs and LHWs had a shared platform for official activities and meetings (the LHW-P office) which served as a potential source of contamination in the control arm. Therefore, in situations like these quasi-experimental design such as a before – after design could be considered as an alternative where the impact of intervention could be evaluated independently without risk of contamination.

Conclusion

While IR is an important tool to identify and address the gaps in health systems and service delivery, it does come with its own set of challenges. First, working in a new rural community requires a deeper understanding of their culture, norms and social structure in order to increase the acceptability of research. To fulfill this, an exploratory/formative phase is important prior to starting the project work to better inform the health systems researchers about community dynamics and tailor the research methodologies and tools to as per the context specific needs of IR. Second, establishing stronger links with policy makers and including them as key stakeholders is essential to successful implementation of the intervention at grass root level and uptake of the findings by the national programs. Third, ensuring national program infrastructure is in place before implementing an IR project, though a necessity is very challenging. Sometimes, research project resources have to be deviated to conduct IR. In this case, NIGRAAN had to maintain their own transport to ensure that supervisory visits of LHSs take place. Therefore, designing and implementing a pilot project to test the intervention on a small scale could be a viable solution to understand the barriers and structural gaps in IR implementation. Fourth, in a behavior change RCT with both arms in the same district, contamination is difficult to avoid. Therefore having a quasi-experimental i.e., a before-after design where both arms are located in the same district or having two separate districts for intervention and control arms could be some other alternatives. Finally, post-intervention assessments should be an integral part of IR behavioral interventions in order to determine the sustainability and retention of health workers’ skills.

11199

References

1. Sanders D, Haines A (2006) Implementation research is needed to achieve international health goals. PLoS Med 3:e186.
2. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM (2015) An introduction to implementation science for the non-specialist. BMC psychology 3: 1.
3. SJ Hoffman, JA Røttingen, S Bennett, JN Lavis, JS Edge, et al. (2012)Review of conceptual barriers and opportunities facing health systems research to inform a strategy from the World Health Organization.WHO.
4. Martins JoS, Zwi AB, Hobday K, Bonaparte F, Kelly PM (2012) The implementation of a new malaria treatment protocol in Timor-Leste: challenges and constraints. Health policy and planning 27:677-686.
5. Victora CG, Habicht JP, Bryce J (2004) Evidence-based public health: moving beyond randomized trials. Am J Public Health 94:400-405.
6. Philpott A, Maher D, Grosskurth H (2002) Translating HIV/AIDS research findings into policy: lessons from a case study of ‘the Mwanza trial’. Health policy and planning 17:196-201.
7. Peters DH, Adam T,Alonge O, Agyepong IA, Tran N (2014) Implementation research: what it is and how to do it Implementation research is a growing but not well understood field of health research that can contribute to more effective public health and clinical policies and programmes. British journal of sports medicine 48:731-736.
8. Hafeez A, Mohamud BK, Shiekh MR, Shah SAI, Jooma R (2011) Lady health workers programme in Pakistan: challenges, achievements and the way forward. Journal of the Pakistan Medical Association 61:210.
9. (2009) Lady health worker programme: external evaluation of the national programme for family planning and primary health care. Quantitative survey report.
10. Rajaratnam JK, Marcus JR, Flaxman AD, Wang H, Levin-Rector A, et al. (2010)  Neonatal, postneonatal, childhood, and under-5 mortality for 187 countries, 1970–2010: a systematic analysis of progress towards Millennium Development Goal 4. The Lancet 375:1988-2008.
11. Wazir MS, Shaikh BT, Ahmed A (2013) National program for family planning and primary health care Pakistan: a SWOT analysis. Reprod Health 10:4755-4710.
12. Ali A, Naseem M, Khalid SN, Iqbal W (2014) Assessment of services provided by Lady Health Workers' (LHWs) at primary health centers at District Quetta, Balochistan. Pakistan Journal of Public Health 4:4.
13. Jalal S (2011) The lady health worker program in Pakistan: a commentary. The European Journal of Public Health 21:143-144.
14. Omer K, Mhatre S, Ansari N, Laucirica J, Andersson N (2008) Evidence-based training of frontline health workers for door-to-door health promotion: a pilot randomized controlled cluster trial with Lady Health Workers in Sindh Province, Pakistan. Patient education and counseling 72:178-185.
15. Rabbani F, Mukhi AAA, Perveen S, Gul X, Iqbal SP, et al. (2014) Improving community case management of diarrhoea and pneumonia in district Badin, Pakistan through a cluster randomised studythe NIGRAAN trial protocol. Implementation Science 9:186.
16. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, et al. (2009) Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 4: 50.
17. Pakistan MoH (2008) Curriculum for lady health supervisors. National program for family planning and primary health care.
18. Braun V, Clark V (2006) Using thematic analysis in psychology. Qual Res in Psychol3:77-101.
19. Rabbani F, Perveen S, Abbasi IN, Aftab W, Sangrasi K, et al. (2015) Policy makers’ perceptions regarding the performance of the lhw program: is there a know-do gap? Findings from the NIGRAAN project. Pakistan Journal of Public Health 5: 4.
20. Siddiqui S, Shah N, Ayub R, Khan N (2012) Student lady health visitors: their stress profile and willingness to practice. J Ayub Med Coll Abbottabad 24:3-4.
21. Nishtar S (2007) Health indicators of Pakistan: gateway paper II: Heartfile.
22. Hamer DH, Herlihy JM, Musokotwane K, Banda B, Mpamba C, et al. (2015) Engagement of the community, traditional leaders, and public health system in the design and implementation of a large community-based, cluster-randomized trial of umbilical cord care in Zambia. Am J Trop Med Hyg 92:666-672.
23. Casale M, Lane T, Sello L, Kuo C, Cluver L (2013) Conducting health survey research in a deep rural South African community: challenges and adaptive strategies. Health Res Policy Syst11:14.
24. Kllander K, Tomson Gr, Nsabagasani X, Sabiiti JN, Pariyo G (2006) Can community health workers and caretakers recognise pneumonia in children? Experiences from western Uganda. Transactions of the Royal Society of Tropical Medicine and Hygiene 100:956-963.
25. Carlough M, McCall M (2005) Skilled birth attendance: what does it mean and how can it be measured? A clinical skills assessment of maternal and child health workers in Nepal.Int J GynaecolObstet 89:200-208.
26. Global Health Workforce Alliance, World Health Organization (2010) Global Experience of Community Health Workers for Delivery of Health Related Millennium Development Goals: A Systematic Review, Country Case Studies, and Recommendations for Integration into National Health Systems.pp: 1-391.
27. 31.Perry H, Zulliger R (2012) How effective are community health workers? Johns Hopkins Bloomberg School of Public Health 2: 108.
28. Topp SM, Chipukuma JM, Hanefeld J (2015) Understanding the dynamic interactions driving Zambian health centre performance: a case-based health systems analysis. Health policy and planning 30:485-499.
29. Kok MC, Dieleman M, Taegtmeyer M, Broerse JEW, Kane SS, et al. (2014) Which intervention design factors influence performance of community health workers in low-and middle-income countries? A systematic review.Health Policy Plan 30: 1207-1227.
30. Dolea C, Stormont L, Braichet JM (2010) Evaluated strategies to increase attraction and retention of health workers in remote and rural areas. Bulletin of the World Health Organization 88:379-385.
31. Manafa O, McAuliffe E, Maseko F, Bowie C, MacLachlan M, et al. (2009) Retention of health workers in Malawi: perspectives of health workers and district management. Human resources for health 7:65.
32. Willis-Shattuck M, Bidwell P, Thomas S, Wyness L, Blaauw D, et al. (2008) Motivation and retention of health workers in developing countries: a systematic review. BMC health services research 8:247.
33. Puffer S, Torgerson DJ, Watson J (2005) Cluster randomized controlled trials. J EvalClinPract 11:479-483.
34. English M, Schellenberg J, Todd (2011) Assessing health system interventions: key points when considering the value of randomization. Bulletin of the World Health Organization 89:907-912.
35. Sanson-Fisher RW, Bonevski B, Green LW (2007) Limitations of the randomized controlled trial in evaluating population-based health interventions. Am J Prev Med 33: 155-161.
36. Torgerson DJ (2001) Contamination in trials: is cluster randomisation the answer? British Medical Journal 322: 355-357.
37. De Bruin M, McCambridge J, Prins JM (2015) Reducing the risk of bias in health behavior change trials: Improving trial design, reporting or bias assessment criteria? A review and case study. Psychology & health 30:8-34.