Passport to Research Futures programme evaluation

1. Background

The Passport to Research Futures (PRF) is a structured development programme for early career researchers designed to focus thinking about career planning, professional development and employability.  It is recognised by the Institute for Leadership and Management (ILM). The programme includes a range of development activities, including:

  • Workshops
  • Question and Answer panel sessions
  • Networking events
  • Epigeum online Professional Skills for Research Leaders courses
  • Kintish networking online resources
  • Access to the Teaching, Research and Academic Mentoring scheme
  • A personal Vitae Researcher Development Framework Planner account.

The PRF programme has been running for 3 years and an evaluation of the programme was carried out in autumn 2017. This report presents the findings of the evaluation.

2. Evaluation method

Two questionnaires were developed, one for graduates of the programme and the other for current participants. The questions were broadly the same with some minor variations. The questionnaires were developed as an online survey and were tested out by a graduate of the programme and a current participant. The time taken to complete the survey was timed so that an estimate of how long the survey would take to complete could be provided. Following feedback and minor revisions the survey link was emailed to the PRF graduates and current participants via the PRF management platform SUMAC. A three week timescale for completion was set and an estimate of 15 minutes was given for completion. A reminder email was sent out at the end of the three week period with an extended closing date.

3. Response rate

3.1     PRF graduates

15 graduates completed the evaluation out of a total of 24 graduates a response rate of 62.5%. However, seven graduates had left the University since completing the programme and there were no forwarding email addresses, so the survey was only sent to 17 graduates making the response rate 88%.

3.2    Current PRF participants

14 current participants completed the evaluation out of a total of 28 people registered as being current participants, giving a response rate of 50%. Two were just due to start and therefore could not complete the evaluation. An additional four people had left the University and one further participant had dropped out of the programme although they remain at the University. This gives a total of 21 people who would have been able to complete the evaluation and a completion rate of 66.7%.

4. Summary of Results

4.1    Participants

The majority of participants are Research Fellows and they have come from 12 out of the University’s 21 schools. They take an average of approximately one year to complete the programme. The most common reason for joining the programme was to provide structure for career development, followed by having interesting courses on offer.

Area for development: Connect directly with individual schools to identify if there are any barriers to participation.

4.2    Courses

a) Face to face workshops

The PRF is divided into nine themes:

  1. Programme orientation
  2. Equality and diversity
  3. Career futures for research staff
  4. Raising your research profile
  5. Public engagement
  6. Entrepreneurship and enterprise
  7. Funding and financing research
  8. Leading the team
  9. Get the job.

The most frequently mentioned theme when participants were asked about usefulness of workshops was “Leading the Team” followed by “Career futures for research staff” and “Get the job”. The most mentioned individual workshop was “Psychometric Masterclass”.

The most frequently mentioned theme when participants were asked about the least useful workshops was “Career futures for research staff” and the most frequently mentioned workshop was “Managing research information: Pure hands on training”.

The most mentioned idea for improving the courses was to reduce the time allocated for some of the workshops.

Areas for development: Ensure workshop time is appropriate for the content and look at content and suitability of the least helpful workshops.

b) Online courses (excluding Epigeum)

PRF includes a small number of online courses and the most frequently mentioned helpful course was “Recruitment and selection”. Having too many web pages to flick through and it not being clear where to click was a criticism of some of online content.

Areas for development: Ensure efficient layout of online content. Use feedback from evaluations to promote PRF.

c) Epigeum on-line courses

One of the components of the PRF is access to the six Epigeum Professional Skills for Research Leaders online courses. There was a low uptake for these Epigeum courses with six graduates and three current participants having completed some of the courses. The most common reason for not taking up this development opportunity was lack of time followed by preferring face to face workshops.

Area for development: Consider setting up an Epigeum workshop where participants can register for the courses and complete the first course in a workshop setting.

4.3 Vitae Researcher Development Framework planner

Five graduates and five of the current participants had used the Vitae Researcher Development Framework planner. The most common uses had been to help structure grant applications and to identify gaps in competencies.

Area for development: Consider setting up a Vitae workshop to help participants to register for the planner and to provide guidance on how to benefit from the Framework.

4.4 Other supporting services

At the orientation meeting for PRF, participants are signposted to the Careers Centre, the Research Business and Development Contracts Team and to the Teaching, Research and Academic Mentoring Scheme. Nine participants had used the Careers Centre and they mentioned CV help, identifying career options and help with job applications as being the main reasons for seeking support. Six participants had used the Research Business and Development Contracts Team to seek advice on funding applications. Ten people had joined the Teaching, Research and Academic Mentoring Scheme and a number of them commented on how their mentor had helped them.

4.5 Main benefits of completing the PRF programme.

The participants commented on how the programme had helped with career development, helped them to meet new people and expanded their personal growth and development.

The PRF has a number of key objectives and participants were asked to what extent they felt the programme contributed towards these objectives:

Objective Rank
Clarifying my career path =3
Formulating a career development plan 5
Assessing my level of competence against my development goals =3
Filling gaps in my confidence 2
Becoming more confident in my chosen career path 6
Becoming more self-aware about my development needs 1

4.6 Improving the programme

Participants were asked how the PRF programme could be improved.

Ideas included:

  • Offer advanced classes in some subjects
  • Better advertising
  • Provide a better way to keep track of progress
  • Providing support for teaching
  • Encourage more social engagement
  • Set requirements of PRF to correspond with the ILM certificate.

Barriers to making progress with the PRF programme included lack of time and when courses were full.

Areas for development: Look at all suggestions for improvement and implement where feasible.

5. Overall conclusions

Participants were asked about some aspects of the organisation of the programme and the results are given below.

Average score (out of 5)
Programme information on Passport to Research Futures’ website 3.9
Programme information received at the orientation meeting 4.3
Overall support from CAPOD during the programme 4.6
Overall quality of the Passport to Research Futures programme 4.4

 

Participants were also asked to compare the programme with similar programmes they had experienced at other institutions.

Rating Number or respondents
Better 10
Much better 6

 

Participants were also asked what they would say to early career researchers thinking about signing up to PRF.

Advice included:

  • It helps focus on a career plan
  • It was a good way to fill the gaps in training
  • There are some great courses on offer
  • Recommend it to all early career researchers.

Overall the Passport to Research Futures evaluates positively and provides a valuable way to structure professional development for early career researchers. Useful areas of development have been identified which will be further analysed by the Research Staff Developers in order to inform changes to the programme in 2018/19.

6. Recommendations

  1. Connect directly with individual schools to identify if there are any barriers to participation.
  2. Ensure workshop time is appropriate for the content.
  3. Look at content and suitability of the “least helpful” workshops.
  4. Ensure efficient layout of online content.
  5. Use feedback from all forms of evaluation to promote Passport to Research Futures.
  6. Consider setting up an Epigeum workshop where participants can register for the courses and complete the first course in a workshop setting.
  7. Look at how to increase the use of the Vitae planner e.g. provide a workshop where participants can register and start to use the planner.
  8. Continue to promote the Careers Centre, Research Business and Development Contracts Team and the Teaching, Research and Academic Mentoring Scheme throughout the PRF programme.
  9. Develop a course for Fellowship applications with attention to finances.
  10. Provide advanced classes in some subjects e.g. research funding, presenting your research, public engagement.
  11. Build in flexibility for new and more experienced post-docs.
  12. Survey Post-Docs to ascertain what further courses they wish to see in the programme.
  13. Look at advertising process and ensure the target audience is reached.
  14. Assess which courses would benefit from being divided into social vs natural sciences.
  15. Look at developing a better way to keep track of progress e.g. online or physical passport.
  16. Look at increasing the amount of targeted and specific advice available.
  17. Provide support for teaching and applications for lectureships.
  18. Support social engagement.
  19. Identify most popular courses and run them more frequently.
  20. Improve programme information on the website.
  21. Look at matching PRF and ILM requirements.
  22. Deliver a higher level project management course.
  23. Hold an event for experienced researchers to share their experiences with new researchers.
  24. Look at teaching-focused route to Higher Education Academy fellowship.
  25. Look at setting up a Passport to Teaching Futures programme.
  26. Provide a course on funding management delivered by Finance Advice and Support team.

 

Authors: Marie Paterson and Diane Munday, Staff Developers (Research Staff)

Jos Finer, Head of Organisational and Staff Development.

February 2018.

This entry was posted in Uncategorized. Bookmark the permalink.