On assessing the efficiency of research and innovative activity of Russian universities

Objective: to develop and test a methodology for evaluating the effectiveness of scientific and innovative activities of Russian universities based on econometric modeling to identify key factors affecting revenues from R&D.
Methods: regression analysis of a sample of 99 leading Russian universities out of the RAEX top 100 in 2023 and 2024.
Results: the paper considers an urgent problem of evaluating the effectiveness of scientific and innovative activities of Russian universities. The analysis of existing approaches to assessment, including government monitoring and scientometric indicators, revealed their limitations and lack of objectivity. The econometric analysis revealed a statistically significant positive impact on the total income from R&D of the following factors: the number of full-time researchers, the number of academic Olympiads winners and prize-winners enrolled to the university, and the number of articles prepared in collaboration with foreign organizations. Labor productivity (income from R&D per a professor) is positively related to the proportion of young employees (under 40 years old), the proportion of experienced employees (over 65 years old), and the proportion of master's students in the total number of students.
Scientific novelty: a comprehensive econometric methodology was developed and empirically verified, that identifies new efficiency factors that previously had not been taken into account in such models (for example, the role of academic Olympiads winners and the personnel age structure). The results quantitatively confirm the hypothesis about the importance of human and social capital, as well as of international cooperation, for the financial results of universities’ scientific work.
Practical significance: the models proposed in the paper have a high explanatory power and can be used as a tool for benchmarking and rating the universities’ effectiveness in gaining income, including from R&D. This opens up new prospects for a more objective and comprehensive assessment of university performance.

1. Astafyeva, M., Zyateva, O., Peshkova, I., & Pitukhin, E. (2015). Russian universities in terms of performance indicators. Journal University Management: Practice and Analysis, 4(98), 4–18. (In Russ.).

2. Berbegal-Mirabent, J., Lafuente, E., & Solé, F. (2013). The pursuit of knowledge transfer activities: An efficiency analysis of Spanish universities. Journal of Business Research, 66(10), 2051–2059. https://doi.org/10.1016/j.jbusres.2013.02.031

3. Cricelli, L., Greco, M., Grimaldi, M., & Llanes Dueñas, L. P. (2018). Intellectual capital and university performance in emerging countries: Evidence from Colombian public universities. Journal of Intellectual Capital, 19(1), 71–95. https://doi.org/10.1108/JIC-02-2017-0037

4. Davydova, T., Barkalov, S., & Poryadina, V. (2019, May). Content and evaluation of innovative components of human potential in the ratings of universities. In Society. Integration. Education: proceedings of the International Scientific Conference (Vol. 6, pp. 117–132). (In Russ.). https://doi.org/10.17770/sie2019vol6.3736

5. Efremova, P. V. (2019). Indicators for assessing the effectiveness of the universities innovative activity development. Russian Journal of Innovation Economics, 9(3), 989–1010. (In Russ.). https://doi.org/10.18334/vinec.9.3.41001

6. Egorshin, A. P., & Gladysheva, I. A. (2014). How to assess the efficiency of universities’ performance? Vysshee obrazovanie segodnya, 11, 29–32. (In Russ.).

7. Fan, H. L., Huang, M. H. & Chen, D. Z. (2019). Do funding sources matter?: The impact of university-industry collaboration funding sources on innovation performance of universities. Technology Analysis & Strategic Management, 31(11), 1368–1380. https://doi.org/10.1080/09537325.2019.1614158

8. Gafforova, E.B., & Karlovskiy, A. V. (2009). About approaches to the effectiveness of higher education institutions assessment. Vestnik Novosibirskogo Gosudarstvennogo Universiteta. Seriya: Soczialno-economicheskie nauki. (In Russ.).

9. Gasanov, E. A., & Zubarev, A. E. (2019). Principles for the integration of universities in the global space of higher education and neo-industrial development. Bulletin of Pacific National University, 3(54), 35–46. (In Russ.).

10. Grigorash, O. V. (2014). About indicators of an assessment of efficiency of activity of higher education institutions. Polythematic Online Scientific Journal of Kuban State Agrarian University, 95, 1237–1262. (In Russ.).

11. Guba, K., & Zavadskaya, M. (2017). It is better to be inefficient than non-state: how Rosobrnadzor punishes universities: Analytical notes on law enforcement. Saint Petersburg. (In Russ.).

12. Gür, U., Oylumlu, I. S., & Kunday, Ö. (2017). Critical assessment of entrepreneurial and innovative universities index of Turkey: Future directions. Technological Forecasting and Social Change, 123, 161–168. https://doi.org/10.1016/j.techfore.2016.09.008

13. Hughes, A. (2006). University-industry linkages and UK science and innovation policy. Cambridge: Centre for Business Research, University of Cambridge.

14. Kuhlmann, S. (2003). Evaluation of research and innovation policies: a discussion of trends with examples from Germany. International Journal of Technology Management, 26(2–4), 131–149. https://doi.org/10.1504/IJTM.2003.003366

15. Lavrinenko, Y. B., Novennikov, R. M., & Tinyakova, V. I. (2023). Evaluation of the impact of university income on the level of their innovative activity. Modern Economics: Problems and Solutions, 105–122. (In Russ.). https://doi.org/10.17308/meps/2078-9017/2023/5/105-122

16. Levashov, E. N. (2016). Criteria for assessing the efficiency of Russian universities’ performance. Symbol of Science: International Scientific Journal, 2-2, 170–173. (In Russ.).

17. Litvinova, N. A. (2024). Development of human capital in universities as one of the directions of innovation policy in the digital economy. Journal of Economics, Entrepreneurship and Law, 14(12), 7927–7944. (In Russ.). https://doi.org/10.18334/epp.14.12.122380

18. Lvovich,Y. E., Kashirina, I. L., & Schwindt, A. N. (2017). Network-based simulation of the results monitoring evaluation of activity of universities. Modeling, Optimization and Information Technology, 4(19), 359–371. (In Russ.).

19. Maksimova, T. G., Nikolaev, A. S., & Byambatsogt, D. Research universities in the national innovation ecosystem. Theory and Practice of Social Development, 126, 81–87. (In Russ.). https://doi.org/10.24158/tipor.2018.8.15

20. Melikyan, A. (2018). Internal factors of education export performance in Russian universities. Educational studies. Moscow, 3, 146–178. (In Russ.). https://doi.org/10.17323/1814-9545-2018-3-146-179

21. Mirolyubova, T. V., & Solomatova, L. O. (2012). Methodical approaches for factor analysis of university’s innovation system development. Fundamentalnye Issledovaniya, 11–4, 1004–1010. (In Russ.).

22. Molchanova, N., Skantsev, V., & Spasennikov V. (2019). Discussion issues of evaluation of the scientific activity’s effectiveness using citation indices (review of domestic and foreign publications). Ergodizayn, 4(6), 186–195. (In Russ.). https://doi.org/10.30987/2619-1512-2019-2019-4-186-195

23. Moskaleva, O. V. (2013). Using of scientometric indices for research assessment. Naukovedcheskie Issledovaniya, 85–109. (In Russ.).

24. Niyazova, M. V. (2003). Problems of assessing the efficiency of business activity of budget-funded universities. Journal University Management: Practice and Analysis, 3, 68–76. (In Russ.).

25. Parakhina, V., Godina, O., Boris, O., & Ushvitsky, L. (2017). Strategic management in universities as a factor of their global competitiveness. International Journal of Educational Management, 31(1), 62–75. https://doi.org/10.1108/IJEM-03-2016-0053

26. Perfileva, O. V. (2016). Review of the modern methods of assessing the efficiency of innovative development. Izvestiya Saratovskogo universiteta. Novaya seriya. Seriya: Economika. Upravlenie. Pravo, 16(2), 166–171. (In Russ.). https://doi.org/10.18500/1994-2540-2016-16-2-166-171

27. Pestov, I. P. (2024). Assessment of factors affecting the income of Russian universities. Journal of Monetary Economics and Management, 5, 185–194. (In Russ.). https://doi.org/10.26118/2782-4586.2024.44.65.026

28. Polikhina, N. A., & Trostyanskaya, I. B. (2018). The effectiveness of the evaluation of the university through scientometric indicators. Education and Science in Russia: the State and Development Potential, 3, 257–281. (In Russ.).

29. Ponomareva, A. M., Zhumanova, M. H., Kolchugina, T. A., Solovieva, I. V., & Puchkova, E. E. (2019, October). Evaluating the Effectiveness of the University as Intellectual and Creative Organization. In Institute of Scientific Communications Conference (pp. 559–566). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-69415-9_64

30. Pushkarev, Yu. V., & Latuha, O. A. (2012). Estimation of management efficiency innovative activity of university as innovative higher school. Novosibirsk State Pedagogical University Bulletin, 5(1), 25–31. (In Russ.).

31. Ramsden, P. (1998). Managing the effective university. Higher Education Research & Development, 17(3), 347–370. https://doi.org/10.1080/0729436980170307

32. Rechinskij, A. V., Chernenkaja, L. V., Mager, V. E., Matveeva, E. V., & Chernenkij, A. V. (2015). Development of the method of calculating the effectiveness of the university and its structural divisions. Modern Management Technology, 6(54), 46–54.

33. Romanov, E. V. (2019). Efficiency assessment of higher education institutions: contradictions and paradoxes. Part I. Education and Science Journal, 21(9), 9–48. (In Russ.). https://doi.org/10.17853/1994-5639-2019-9-9-48

34. Sandler, D. G., & Gladyrev, D. A. (2020). Construction of an economically effective system of target indicators for the development of university research activities considering correlation dependencies. Statistics and Economics, 17(4), 71–84. https://doi.org/10.21686/2500-3925-2020-4-71-84

35. Sandler, D. G., & Gladyrev, D. A. (2021). Review of empirical researches on the factors of economic performance of Russian universities. Vestnik of Samara University. Economics and Мanagement, 4, 37–49. (In Russ.). https://doi.org/10.18287/2542-0461-2021-12-4-37-49

36. Sharshov, I. A. (2014). Assessing the efficiency of R&D activity at a modern university. Psychological-Pedagogical Journal Gaudeamus, 1(23), 38–44. (In Russ.).

37. Sheshukova, T. G., & Sergeeva, N. V. (2012). Forming the system of indices for assessing the efficiency of research activity of national research universities. Economic Analysis: Theory and Practice, 4, 53–63. (In Russ.).

38. Sizov, V. S., & Sizova, E. N. (2013). Problem of an assessment of efficiency of activity of higher education institutions. Economika Obrazovaniya, 2, 91–96. (In Russ.).

39. Slobodnyak, I. A., & Baenkhaeva, A. V. (2015). On methodology for monitoring the efficiency of higher school activities. Economic Analysis: Theory and Practice, 36(435), 50–60. (In Russ.).

40. Sviridova, N. V., & Sazonova, I. V. (2011). The comparative analysis of efficiency and productivity of activity of high schools. Journal University Management: Practice and Analysis, 4, 83–86. (In Russ.).

41. Szücs, F. (2018). Research subsidies, industry-university cooperation and innovation. Research Policy, 47(7), 1256–1266.

42. Tereshchenko, D. S., & Shcherbakov, V. S. (2019). Statistical аnalysis of differentiation between Russian regions by level of publications. ECO, 9(543). (In Russ.). https://doi.org/10.30680/ECO0131-7652-2019-9-132-154

43. Timofeeva, A. Yu. (2018). Evaluation of the impact of profile direction to the indicators of the activities of Russian universities. World of Economics and Management, 18(3). (In Russ.). https://doi.org/10.25205/2542-0429-2018-18-3-81-90

44. Tseng, A. A., & Raudensky, M. (2014). Performance evaluations of technology transfer offices of major US research universities. Journal of Technology Management & Innovation, 9(1), 93–102. https://doi.org/10.4067/S0718-27242014000100008

45. Tsivinskaya, A. O., Guba, K. S., Guba, K., & Zavadskaya, M. (2017). Better Ineffective than Private: How Universities are Penalized by Rosobrnadzor. Saint Petersburg: The Institute for the Rule of Law, European University. (In Russ.).

46. Vinig, T., & Lips, D. (2015). Measuring the performance of university technology transfer using meta data approach: the case of Dutch universities. The Journal of Technology Transfer, 40(6), 1034–1049. https://doi.org/10.1007/s10961-014-9389-0

47. Vinokurov, M. A. (2013). Monitoring the effectiveness of Russian universities: enhancing the methodology. Izvestiya Irkutskoy Gosudarstvennoy Economicheskoy Akademii, 6(92), 5–11. (In Russ.).

48. Vkusov, A. V. (2018). Problems of assessing the effectiveness of universities. Sotsiologicheskie Issledovaniya, 4, 140–145. (In Russ.). https://doi.org/10.7868/S0132162518010154

49. Yelshansky, S. P. (2017). On the assessment of the effectiveness of education in a university. Problemy Sovremennogo Obrazovaniya, 5, 99–108. (In Russ.).

50. Zinchenko, D., & Egorov, A. (2019). Efficiency modeling of Russian universities. Higher School of Economics Economic Journal, 23(1). (In Russ.). https://doi.org/10.17323/1813-8691-2019-23-1-143-172

51. Zhylinska, O., Sitnicki, M., & Vikulova, A. (2019). Systematic assessment of the innovative potential of a research university. Baltic Journal of Economic Studies, 5(2), 38–44. https://doi.org/10.30525/2256-0742/2019-5-2-38-44