Here is a quick reminder of the pros and cons of the different data collection methods you might consider using when carrying out initial pre-training research.
For more details see our chapter on Practical Research on page 49 in the book.
Surveys (online or other)
- An inexpensive, quick and easy method of gathering data from a large audience
- Brings consistency to the data gathered and is especially useful for collecting quantitative data to get a representative picture over a large group of respondents
- Questions can be either open for subjective views, closed for quantitative data or multiple choices
- Questions require careful thought and it can be difficult to collect qualitative data.
- A good method for gathering qualitative data
- Can be very beneficial as you are able to see the whole communication- body language, voice and tone
- Possible to surface points that may be relevant but were not covered in the research questions as you can choose to explore particular areas in more detail
- Difficult to conduct a large-scale research using this method
- Results can be inhibited as some are less eager to share and others may dominate the discussion.
- Especially good for gathering qualitative data
- Can send questions to a respondent in advance so that they are able to reflect
- The interview can be delivered in many different ways – informal and conversational with no predetermined questions; it may have a strict structure; it may be standardised with open questions.
- A face-to-face interview is effective for gathering qualitative data and the one-to-one format encourages an open and honest response to questions
- A very time consuming method
- More general interviews make it difficult to compare responses
- It is easy to get side-tracked and gather data less important to you research.
Critical incident analysis
- Gives respondents the opportunity to tell stories of previous ‘critical incidents’ to articulate details that can then be explored further by the interviewer
- A particularly flexible method that does not restrict the respondent to respond to a specific framework
- Helps determine issues in the current or previous way of working
- Focuses on recent event and relies upon the respondents memory so does not paint a very objective picture
- Time consuming.
Observation of work
- Provides both quantitative and qualitative data.
- Can be by direct observation or through the use of technology to report on work activities: telephone calls, mystery shoppers, videos
- Accurate and objective data on what is currently happening which can build a picture of the training needs
- Can be time and resource intensive
- Does not surface participant’s views only their performance.
- Can be used individually or combined within an assessment exercise
- Scenarios can be used to incorporate role play, videos and actors to help identify how they would respond
- Texts and questions help identify existing knowledge and gaps
- Case studies identify and test critical thinking
- Psychometrics allow you to measure the knowledge, abilities and attitudes which help you identify strengths, gaps and areas for development.
- Assessments can be constructed to meet specific needs
- A consistent form of assessment can monitor progress
- Requires careful construction to ensure that you are testing and measuring the correct things.
- Recreating excellence by identifying the behaviour patterns, language, strategies and beliefs of exemplars to build a model which can be taught to other
- Observation and questioning of exemplars to make conscious the subtleties of what they do that makes them successful
- Exemplars are not often aware of what makes them good in a particular area
- Questioning what they are thinking about when they carry out a task or skill and noticing their use of verbal and body language can be more relevant than simply focusing on training more general skill
- Helps to identify very specific nuances that makes a difference for exemplars and brings the organisational context into learning
- Requires expertise by the individual conducting the modelling
Which method do you prefer?
- Findings will need to be based on the modelling of a limited number of people.