Leader of Working Group 2 – “Quantitative Methods in Evaluations”
As part of our ongoing “Voices of Profeedback” series, we asked Dr. Prof. Oto Potluka, the leader of Working Group 2 (Quantitative methods in evaluations), to share his thoughts about their work on strengthening the use of quantitative methods in evaluations. Read below for insights into the group’s goals, progress, and the importance of methodological diversity in evaluation practice.
Let’s start by talking about the main goals of your Working Group. Could you share what those goals are and the topics you’re focusing on?
We have three goals: First, to increase the theoretical knowledge of using quantitative evaluation methods. Second, to increase the practical ability to use quantitative evaluation methods. Third, to increase the understanding and ability to collect data through various means. Thus, the overall objective is to provide knowledge and skills to our members. Thus, our training school combined theoretical approaches in quantitative evaluation methods with practical training in various software (MS Excel, STATA). To support growth in the theoretical knowledge of quantitative methods, we shared among us two links to libraries with many papers. The first one is 3IE, which collected dozens of thousands of impact evaluations. The second one is BetterEvaluation, where our members can find both qualitative and quantitative methods and some papers with cases of how these methods were applied.
We also organized a workshop about digitalization in evaluations. Based on this workshop, we succeeded in publishing several papers in a special issue in the journal Evaluation (it should be published in July 2025).
That sounds impressive! Have you been able to achieve those goals so far? And what’s left to be done in the fourth year?
I would say yes. Definitely the first two objectives have been achieved. One of our goals was to prepare some scientific papers for publication. Now, some of them are under review. Still, we plan to organize an online workshop about web scraping as a method of data collection. As new members of the working group came again and again, we could not simply repeat all the activities. Thus, we started preparing an educational project under the auspices of the Erasmus+ program.
That sounds fantastic! What would you consider the best result or accomplishment of the Working Group so far?
I would say the special issue in the journal Evaluation.
On a more personal level, what has your experience been like participating in this Working Group? What have been your main takeaways?
It was networking with various people. An interesting experience was to speak with less experienced evaluators who shared their needs with us. I mean the international audience. Thus, I could see differences in how the evaluation cultures have progressed in various countries (according to what national evaluation societies and universities provide to young emerging evaluators). If I were to start a new project, I would definitely carry over the value of international collaboration.
That’s a great insight! Speaking of new projects, are there any new topics or challenges that have come up for you recently that you think should be addressed in the future?
The complexity of evaluations and the need for evaluation education covering various aspects of evaluations. Second, it is the imbalance in knowledge basis across countries and evaluators. Many evaluators simply apply the same evaluation methods again and again, because they know them and are used to applying them. Nevertheless, evaluators should have a wider portfolio of methods to apply according to what evaluation questions they want to answer, what data is available, and what is the context of the intervention. We want to react to this in our future project.
Sounds like an important challenge to tackle. How did the collaboration work among such a large group of members?
There is a group of about ten to fifteen people engaged. The others are passive members. The first year, I was the only leader in the WG2. Then, Lena Tsipouri joined me, and later, Irene Christoforidi also joined, which helped a lot.
It must have been challenging to manage all those different people. What was the level of activity like during your participation?
We found a balance between keeping in touch and organizing too many meetings. We speak together quarterly.
📌 Stay tuned for our next interview with Jaroslav Dvořák, leader of Working Group 3, in the “Voices of Profeedback” series!