TOOLS

Evaluation

The evaluation of the RIS3Andalucia is permanent throughout the entire implementation process, also extending to its consequences. Based on valid, reliable, timely and feasible sources of information, RIS3Andalucía is subject to continuous evaluation through external and independent evaluations at two specific times:

  • Midterm evaluation: mid-term, with a pluralistic and process approach on the design, implementation and consequences of the Strategy.
  • Final evaluation: comprehensive after the end of the execution period of the Strategy, with a participatory approach on the effectiveness, efficiency, impacts and collateral effects, to establish the progress in Andalusia as a result of the RIS3, reach conclusions for subsequent initiatives to promote regional innovation and generating information to improve decision-making by the Strategy actors.

Objectives | Ris3Andalucía

  • Identify the effects generated by the actions taken and understand how and why they have been obtained, taking into account possible unintended results and what their origin has been, isolating the influence of factors external to the execution of the Strategy itself.
  • Assess the progress in achieving the challenges and final objectives of the Strategy.
  • Propose the necessary changes in the design and implementation of the Strategy to improve the results of the actions and address new needs to achieve the challenges and final objectives of the RIS3
  • Generate information that helps to solve the needs of the RIS3 stakeholders and improve decision-making

INTERMEDIATE PROGRESS Evaluation (March-June 2019)

The evaluation of RIS3Andalucia is permanent throughout the entire implementation process, also extending to its consequences. Based on valid, reliable, timely and feasible sources of information , RIS3Andalucía is subject to continuous evaluation through external and independent evaluations at two specific times :

Participatory process | Ris3Andalucía

  • 60

    Interviews

  • 12

    group workshops

  • 155

    regional actors and experts

  • 315

    questionnaires completed in 3 surveys

Main findings

  • 1.

    Disparate progress on final goals. There are still challenges to be completed

  • 2.

    Positive influence on the formulation of regional sector strategies, plans and programs

  • 3.

    Low integration of the RIS3 Andalusia in the decision-making processes of the managing bodies

  • 4.

    There is room for improvement for the Monitoring System

  • 5.

    Low level of knowledge of the Strategy by the agents of the Andalusian R & D & I System

  • 6.

    Little momentum of Governance

Recommendations and improvements

To conclude, the Evaluation Report includes 27 final Recommendations regarding the design, implementation and Governance of the Strategy:

27 recommendations have been made

Design Recommendations

Challenges, objectives and priorities

  • Incorporate two new challenges: social innovation and technology transfer, defining final objectives for them
  • Reformulate (update) specialization priorities
  • Redefine the lines of action of the specialization priorities as areas or themes of specialization, reducing their content with respect to the current one to make them more specific

Programming

  • Set quantified Axis-level objectives for 2023
  • Assign specific funds to RIS3Andalucía
  • Improve the fit between the priorities of specialization and their lines of action with the structure of axes and measures
  • Strengthen the recruitment of R&D projects of high and medium-high technology companies and the location in Andalusia of these companies for these activities

Instruments

  • Greater relevance to the actions of information, awareness and dissemination of knowledge
  • Use financial instruments according to the characteristics of the projects, directing grants to high-risk projects with great impact
  • Apply reimbursable financial instruments in those cases where the existence of a market failure is evidenced, combining them with grants when necessary to make them more effective
  • Intensify Innovative Public Procurement in the Junta de Andalucía
  • Review the classification of instruments provided in the RIS3 Andalusia to adapt it to the reality of the implementation of the measures
Implementation Recommendations

Management

  • Raise the pace of definition and execution of actions
  • Improve aid management processes
  • Establish coordination mechanisms at different levels
  • Eliminate the measures in which no action has been taken and no action is planned
  • Incorporate an integrated gender approach and prioritize projects that include equal opportunities measures
  • Carry out a dissemination program for RIS3Andalucía

Tracking

  • Strengthen the monitoring system to increase its usefulness, solving the deficiencies found: standardization of collection
  • Redefine the Monitoring and Evaluation Plan to adapt it to the reality of the execution of RIS3 Andalusia
  • Quantify, disaggregating by sex, the indicators referring to people who benefit from the actions carried out within the framework of RIS3Andalucía
Governance recommendations
  • Institutionally promote RIS3Andalucía, placing it in a central position on the agenda of those responsible for the Junta de Andalucía.
  • Review the governance model, assessing its institutionalization and incorporating elements that simplify and make it more efficient, reinforcing the commitment and coordination between management centers, with a calendar of meetings of its different bodies
  • Incorporate the CDTI into the Governance of RIS3Andalucía
Recommendations for future specialization strategy
  • Analysis of bottlenecks for the diffusion of innovation, including digitization
  • Implementation plan with the actions that guarantee the feasibility of the follow-up
  • Creation of working groups that promote the participation of the agents of the Andalusian innovation system to update the results of the entrepreneurial discovery process

Detail of the improvement proposals

  • Review of the RIS3 Andalusia

    • Ambit Priority
      Design Very high
    • Responsable No. of recommendations
      Organ of codecision 11

  • Governance Improvement

    • Ambit Priority
      Governance Very high
    • Responsable No. of recommendations
      General Secretary for Business, Innovation and Entrepreneurship 3

  • Entrepreneur Discovery Process Update

    • Ambit Priority
      Governance Half
    • Responsable No. of recommendations
      Technical Secretariat RIS3 Andalusia 1

  • Review of the Monitoring and Evacuation plan

    • Ambit Priority
      Implementation High
    • Responsable No. of recommendations
      RIS3 Andalusia Technical Team 11

  • Implementation plan

    • Ambit Priority
      Implementation Very high
    • Reponsable No. of recommendations
      Management centers of the Junta de Andalucía 7

  • Actions of dissemination and knowledge of RIS3 Andalusia

    • Ambit Priority
      Implementation High
    • Reponsable No. of recommendations
      Technical Secretariat RIS3 Andalusia 2