Article and Author Information
DeAnna Myers wrote this article in December 2012 for the Capstone 3 Research Analysis and Interpretation course. This executive summary assignment is the culmination of a nine-month capstone research project. DeAnna will graduate from the MSLOC program in 2013 and is the Training and Development Manager at Sargent & Lundy.
Engineering firms in the power generation industry, like many knowledge organizations, commonly attempt to sustain their intellectual capital by utilizing in-house experts to train novice staff, but an expert’s ability to predict what is necessary to transfer knowledge to novice learners can be compromised by biases associated with an expert’s superior level of expertise (Hinds, 1999). This study surveyed how one organization’s experts perceive the task of novice-level knowledge transfer, and compared these perceptions to feedback from novices who participated in their classes. Findings from the study suggest that experts misdiagnose novice learning needs and though they usually attempt to adjust content to accommodate the novice learner, those adjustments are less successful than the experts perceive.
The Growing Focus on Knowledge Transfer
By the year 2015, nearly half of all engineers now working in the power generation industry will be eligible to retire, taking with them a significant slice of the industry’s knowledge and expertise (Bogdanowicz, 2010). Part of the answer to this predicament lies within a company’s ability to leverage factors that can promote knowledge transfer while mitigating the factors that inhibit it (Argote, & Miron-Spektor, 2011). Leveraging this resource requires experts to transfer expertise “from those who have it to those who need to know” (Hinds, Patterson, & Pfeffer, 2001, p. 1232). There are, however, obstacles to this process. Biases that are typical to an expert level of expertise can actually make it more difficult to communicate or understand the knowledge being transferred (Hinds, et al., 2001; Nueckles, Winter, Wittwer, Herbert, & Huebner, 2006). Novice feedback from more than 100 classes taught by one organization’s experts speaks of skipped steps, misdiagnosed levels of complexity and an inability to apply class content (McFadden, Myers, & Zavala, 2011).
How an expert perceives the task of knowledge transfer represents a virtual void in current literature. This study focused on experts’ perspectives on what would be necessary to transfer knowledge to a novice audience, and seeks to answer the question,
“How well do experts diagnose what will be necessary to transfer problem-solving knowledge to novice learners in an understandable way?”
This study also asked novice learners to provide feedback regarding how well they learned from courses designed by experts and whether aspects of the instruction such as level of complexity, relevance, topical order were appropriate for their level of expertise. Therefore, the secondary question explores the following:
“Do expert-initiated adjustments affect the novice learner’s ability to learn from the instruction?”
Findings from this study advance the topic of novice learning within knowledge organizations that frequently utilize in-house experts to train novice staff. Results of this study also provide learning and development (L&D) teams, instructional designers and expert staff with a more informed foundation from which to create responsive learning for novice employees. Such advances improve an organization’s ability to preserve its intellectual capital by utilizing the organization’s most knowledgeable, in-house resources more effectively.
Research Methods: A Tale of Two Surveys
In order to answer the research questions, two surveys were designed to collect analogous data; one was tailored to the expert and one to the novice perspective. Both surveys assessed the same variables through custom questions using 5-point Likert scale, multiple option or open-ended question formats.
Key variables in this study were selected after an extensive literature review of empirical studies that examined biases affecting knowledge transfer between expert and novice levels of expertise. Independent variables included:
- Oversimplification or skipped steps resulting in the inability to apply the learning
- Inappropriate level of complexity (either too complex or too simple) for the learner
- Illogical order in the way topics were introduced
- Failure to define technical terms
- Irrelevancy to current work tasks
(Byram, 1997; Hinds, 1999; Langer & Imber, 1979; Kirschner, P. A., Sweller, J., & Clark, R. E. 2006)
The expert survey consisted of 17 questions; three open-response and 14 closed response questions. Seven closed-ended questions addressed the five bias variables listed above; three questions requested demographic and experience data; two closed questions addressed the need and use of adjustments and two closed questions assessed how well the expert believed participants learned from his instruction. The three open-ended questions addressed factors that the expert believed helped or hindered training and best practice adjustments to accommodate the novice learner.
The novice survey consisted of 18 questions; four open-response and 14 closed response questions. Eight closed-ended questions addressed the five bias variables listed above; two questions requested demographic data; two closed questions addressed the need for adjustments and two closed questions assessed how well the novice learned from the instruction. The four open-ended questions addressed factors that the novice believed helped or hindered novice learning and best practice adjustments to accommodate the novice learner.
About the Participants
The sample included expert and novice engineers from one engineering firm. Expert participants (n=42) had at least 10 years of practice in the associated industry and had developed or taught a class designed specifically for in-house novice staff. Novice participants (n=96) had no more than four years of industry experience and had completed an in-house class designed specifically for novice staff.
Qualifying expert responses (n=41) represented 87% of the eligible population. Qualifying novice responses (n=94) represented 39% of the eligible population. The difference in responding percentages was primarily a result of a limited number of courses being offered for various groups of novices during the data collection period. Three responses were rejected for this study as they did not meet the required experience criteria. Respondents represented every business group and engineering discipline within the organization and had participated in novice classes on site (instead of online or through a video broadcast of the class).
Volunteer sampling was used to recruit novice participants from communities of practice and classes designed specifically for novice staff. Experts were personally invited to participate based on company reports of them having taught novice-level classes.
Analyzing the Data
Adjusting Learning to the Novice
The first level of analysis measured whether experts believed they were able to predict novice learning needs and whether they felt it necessary to adjust their instruction for a novice audience. Expert feedback showed that 75% of experts (n=31) surveyed felt predicting novice needs was not difficult. In fact, 73.5% (n=30) of expert respondents managed novice staff indicating that experts had direct contact with novices prior to class design. Additionally, 95% (n=40) of the experts confirmed that they adjusted their instruction to accommodate the novice learner. Exhibit 1.0 summarizes the nature of instructional adjustments made and reported by the experts.
Experts reported that in 55% (n=22) of the cases, they would have applied additional adjustments or attempted to better accommodate the novice learner if they had had more time to develop the course. Experts also reported that they were able to fully apply perceived best practices to their instruction only 20% (n=8) of the time. Experts cited a lack of time/ budget (n=12) or an unawareness of audience demographics (n=7) as the primary reasons for their inability to apply their best practices to their instruction. Finally, 82.5 % (n=33) of experts felt that novices in their courses learned the material and had enough detail to apply it. Appendix A provides a detailed summary of these results.
Learning Needs Through Two Perspectives
The second level of analysis was to assess whether expert perceptions were accurate based on actual learner (novice) feedback. Parallel questions from both surveys were evaluated to ensure that questions could be paired for comparison. Each pair of questions proved acceptable (p≥.05) for comparison using robust tests of homogeneity of variance. ANOVA tests assessed the significance of mean differences in perception between the two groups for the five variables. As shown in Figure 2.0, each category showed a statistically significant difference (p≤ .05) except for whether class complexity was appropriate for a novice learner.
Perceptions on Learning
The most significant differences occurred around how well each group felt the participant learned and their ability to apply the learning to their work environment. Novices rated both variables significantly lower than experts. These results support empirical studies that have found that experts routinely overestimate the learning performance of novice learners (Dane, 2010). These findings support the notion that experts significantly misdiagnose novice learner needs.
Qualitative data were coded to characterize group perspective on instructional elements that helped or hindered novice learning. Qualitative feedback also provided more specific detail of how expert adjustments affected a novice’s ability to learn as a means of addressing the second research question. Finally, qualitative feedback was assessed for frequently used words. Figure 3.0 provides a summary of the word counts conducted.