Based on a recent systematic review of youth employment programs, David Robalino shares how he would design a program from scratch.
After three and half years of work, we have finally completed our systematic review of youth employment programs. Many thanks to the co-authors who did the heavy lifting (Jose Manuel Romero, Jonathan Stöterau, Felix Weidenkaff and Marc Witte). The paper was presented at our recent Jobs and Development Conference. The team went over 40,000 papers to eventually find 103 that reported on credible impact evaluations of youth employment programs.
These were more-or-less equally focused on high and middle/low income countries. These studies were codified in detail, including programs’ design features so that we could understand why some worked and others did not. Because different studies often look at more than one outcome variable (such as employment and earnings) and focus on more than one population group we ended up with a database with over 3,000 observations. We ran thousands of regressions trying to identify how different interventions and program design features affected the outcomes and the probability of success.
For those of you who are fans of youth employment program the results will be disappointing. Only 30 percent of programs in our database were successful; the majority had no positive effect or the effect was most likely the result of chance. Moreover, among the successful programs the effect size was often small. This is dramatic. Most of the programs operating out there have not even been evaluated. But most likely, only a few are achieving what they were set up to do.
What makes a program successful?
The key for success is to be able to respond to the needs of an often very diverse population of beneficiaries. For that you need flexible programs that offer multiple services ranging from counselling and training to job search assistance and stipends. So, there is no one type of intervention that is systematically better than others. We can’t say, for instance, that training programs are better than employment services, or that entrepreneurship programs are better than wage subsidies. It all depends on the population group and the local context. The programs also need to have systems that allow managers and service providers to understand the constraints that individuals face to access jobs and the resources and incentives to address these constraints.
You can read the paper and draw your own policy conclusions. If I had to design a youth employment program from scratch this is what I would do:
- Have a state-of the art information system to biometrically ID, register, and profile beneficiaries, mapping, broadly, the type and severity of the constraints they face.
- Identify providers (public or private) that can offer multiple services, directly or in partnership with others. This includes administering cash transfers such as wage or transport subsidies, providing counseling, and identifying training needs and training opportunities (including through employers). These providers must be accountable for placing beneficiaries into jobs or internships/apprenticeships or connect them to entrepreneurship programs. In countries or regions where these providers do not exist or are few, competent international organizations and NGOs can be mobilized to build capacity working with local NGOs or with individuals they recruit and organize.
- Set-up contracts and payment systems with these providers that give incentives to respond to both the needs of beneficiaries and employers. For each beneficiary, the provider should receive a budget that depends on the results of the profiling, with larger budgets given to more demanding profiles. There can then be different arrangements to compensate high performance and penalize underperformance. For instance, bonuses when specific targets are met. In all cases, providers should be monitored for performance, and should be ranked according to specific outcome variables such as placement rates and cost-effectiveness. Providers that do not meet minimum targets should be taken out of the system.
- Allocate beneficiaries to these providers making sure that there is no adverse selection. Providers need to manage the pools they were assigned without discriminating. That is why they should receive larger budgets for more complex cases.
- Have a state of the art monitoring and evaluation system to track beneficiaries, evaluate the quality of jobs/internships, and evaluate and pay providers.
But, even if successful, this type of program is likely to have only limited impacts on labor market outcomes. After all, the programs are simply trying to connect young workers to job opportunities that already exist. These programs won’t do much if there arent a lot of opportunities. Thus, these programs need to be connected to interventions that expand job opportunities, for instance, through investments to support SME growth or the development of value chains in specific regions.
A final point to emphasize is that most existing public employment services are not designed in this way. These programs are most likely not making a difference and the resources they use are being wasted. We are working, for instance, with the ANETI in Tunisia that manages a set of dysfunctional active labor market programs. Great, motivated staff but wrong design. I am not suggesting these programs should be shut down because they do have a role to play. But they should be reformed. Once done, they would need to be expanded to be able to reach more vulnerable workers, particularly those living in rural areas.