The main responsibilities of JGU lie in the quality assurance, monitoring and evaluation activities during the project as well as in the creation of the Nurturing Toolkit.
The quality assurance, monitoring and evaluation tasks consist firstly of the provision of preparatory measures before the project’s implementation, including the evaluation and quality framework and its annual reviews, as well as the development of supporting tools in coordination with all partners, secondly in the implementation of the quality assurance procedures throughout the project (with the contribution of all partners) and thirdly in the preparation of the half-yearly quality monitoring reports and delivery of the final report on the overall monitoring and evaluation results.
In accordance with the overall objective for the work area quality assurance, monitoring and evaluation, R&I LOOP defines the task of scientific monitoring and evaluation first of all as a process-accompanying instrument, i.e. it is designed to provide critical-reflective, constructive, and actively formative support for the project partners to improve their internal and project-specific development processes, which serve to ensure the achievement of the projects’ objectives and the gain of knowledge.
The scientific evaluation within the framework of R&I LOOP pursues two basic goals: (i) formative evaluation of the individual consortium partners and (ii) process-accompanying formative evaluation, the implementation of which is based on two instruments. In preparation for this evaluation procedure, an initial review of the individual partner projects and analyses is necessary. The results provide the basis for the present modeling and evaluation concept, which will be coordinated between the partners in the consortium.
Fig. 1: Dimensions of the underlying quality assurance, monitoring and evaluation concept
(1) Basis: Monitoring and evaluation framework and systematic review
The objective of the systematic review is to analyze the initial conditions of all consortium partners on the basis of a document analysis. Subsequently, the underlying evaluation criteria (“(underlying) objectives”, “structure”, “process”, and “results”) are examined and adapted in terms of their applicability to the present project. A first draft of a monitoring and evaluation concept has been developed, and the further procedure will be coordinated in the consortium.
(2) Formative in-process evaluation (according to the cross-project evaluation criteria)
The process-accompanying formative evaluation takes the form of guideline-based individual or group interviews (in person or digitally/remotely) for each partner project of the consortium. It serves to observe the status and development potential of knowledge acquisition within the projects as objectively as possible. The results provide the basis for the recommendation of possible measures for the further development and optimization of the project work to facilitate the achievement of the objectives.
(3) Formative cross-project evaluation
The formative cross-project evaluation is carried out using online questionnaires. It serves to collect quantifiable data (e.g. planned and realized publications, intellectual output, results etc.), which are aggregated on a cross-project level.
(4) Summarizing cross-project evaluation
Subsequently, the findings of all in-process (2) and cross-project (3) evaluations presented above will be systematically and structurally combined and evaluated across projects.
To fit the evaluation context of R&I LOOP, the model (Fig. 1) was expanded to include an additional dimension, “quality of (underlying) objectives”, as it is to be expected that sound structures and processes can only lead to good results if they align with the underlying objectives.
 |
 |
Fig. 2: Evaluation dimensions according to Parsons and exemplary operationalisation in R&I LOOP
Structuring the underlying monitoring and evaluation questions according to this model enables a systematic approach to quality assurance in this complex consortium. This model-based understanding of quality constitutes the starting point for the selection of appropriate indicators and, therefore, a suitable analysis scheme for all project partners. This, in turn, enables the compatibility of analyses at the project level.
These models are not to be regarded as fixed indicators, but rather serve to provide better orientation and transparency in the evaluation process. Based on these central criteria and following the specified sub-criteria, concrete questions are formulated to guide the process of developing the evaluation instruments and the process of data collection and analysis.
Figure 2 presents these central evaluation criteria and serves as a basis for the further specification of a common monitoring and evaluation concept in the R&I LOOP consortium.
In addition to the described quality assurance, monitoring and evaluation activities, JGU is entrusted with the development of the Nurturing Toolkit. It is supported in this by the UPM. JGU is responsible for the development and final fine-tuning, UPM for the pilot testing of the toolkit.
The toolkit will develop a set of materials to support higher education institutions in their institutional transformation when reshaping the way that they are doing research and innovation, with a focus on how research results are communicated and how students are being prepared to be society-focused researchers and able of promoting social engagement in their future work. The toolkit will include: i) a capacitation training course for professors and research staff on how to communicate research and research results and to enable them to design new lessons for their student on the topics; ii) lessons to be incorporated in existing bachelor’s, master’s and PhD programmes to promote students’ capability of being future society-focused researchers, while also being able to engage society and citizens in their research and better communicate the research results.
In concrete terms, the JGU's tasks therefore consist firstly of developing the training programme for teachers and staff as well as its content, and secondly of developing teaching topics and designs, with all project partners providing support and creative input in both cases. In the course of fine-tuning the toolkit, the partners are given the opportunity to enrich their existing study programmes with innovative teaching units, focusing on i) preparing students to become society-focused researchers, ii) equipping them with the right skillset to consider and represent the interests of citizens and society in their research activities, iii) and transform research results into tangible results for society by improving students’ communication skills. With the support of JGU, each consortium partner will develop five different teaching units, which will then be subjected to a peer review. The result will be a catalogue of lessons that will be translated into all the respective national languages.