FUTUREVIEWs
Vol. 2 Issue 2: March 2010. A WTRI Publication.
Dr. Robert
R. Hoffman discuss how companies are dealing (or not dealing) with a
loss of expertise as the baby boomers retire and what can be done about
it
Hoffman is
recognized as one of the world leaders in the field of cognitive systems
engineering and Human-Centered Computing. He is a Fellow of the
Association for Psychological Science and a Fulbright Scholar. His Ph.D.
is in experimental psychology from the University of Cincinnati, where
he received McMicken Scholar, Psi Chi, and Delta Tau Kappa Honors.
Following a Postdoctoral Associateship at the Center for Research on
Human Learning at the University of Minnesota, Hoffman joined the
faculty of the Institute for Advanced Psychological Studies at Adelphi
University. Hoffman has been recognized internationally in disciplines
including psychology, remote sensing, weather forecasting, and
artificial intelligence, for his research on human factors in remote
sensing, his work in the psychology of expertise and the methodology of
cognitive task analysis, and for his work on HCC issues intelligent
systems technology and the design of macrocognitive work systems.
Hoffman is a Co-Editor for the Department on Human-Centered
Computing in
IEEE:
Intelligent Systems. He is Editor
for the book Series, "Expertise: Research and Applications." He is a
co-founder and Track Editor for the Journal of Cognitive Engineering
and Decision Making. Currently,
Hoffman is a Senior Research Scientist at the Institute for Human and
Machine Cognition in Pensacola, FL.
His
major current projects involve evaluating the effectiveness of knowledge
management, and performance measurement for macrocognitive work systems.
A full vita and all of his publications are available for download at
[www.ihmc.us/users/rhoffman/main].
WTRI: Based
on your extensive research and experience with knowledge capture in
organizations are there any lessons you see for companies dealing with
issues of expertise in a changing market?
ROBERT HOFFMAN: One of the big changes right now is
that businesses are discovering the “boiled frog” problem and it’s
freaking them out. The allegory is that apparently you can take a frog
and put it in boiling water and for the moment that it lives it will
protest vigorously, whereas if you put it in room temperature water and
slowly bring it up to a boil, it will languish happily until it’s ready
to be a puree. There is a baby boomer effect of inevitable retirements.
Some believe that this problem has been delayed because people
were postponing their retirement due to the recession, but I haven’t
seen much evidence of that. It is still a massive cohort effect, and
it’s been publicized by huge organizations, such as NASA and the
electric utilities. What’s happening in companies is that they are at
risk of losing expertise and at some point it snowballs and results in a
panic attack. And it can
happen catastrophically. Even when one really key person retires, the
company suddenly realizes it is very much at risk.
The field of “knowledge management” is supposedly
devoted to helping organizations with exactly this kind of problem by
preserving expertise through web seminars and building spreadsheets.
Ultimately, however, all they do is help administrators
administrate. When the question of evaporating expertise was raised, the
typical business person would say, “What’s it going to cost? What are
the costs and risks and benefits of doing knowledge capture?” That was
disappointing. Let’s say a business is knowledge-based and it has a
cadre of senior people 55 and older who are the ones know how to
actually make things work, and will retire in 5 years. If no one’s being
trained up to do what they do, and if the leadership of that business
doesn’t recognize this as a problem, and all they’re worried about is
estimating the cost of knowledge elicitation, which doesn’t fit into
they’re usual budget categories, then they’re in trouble. But some
companies have decided to do something about this.
WTRI: Why do
you think we don’t hear more about what companies are doing to solve the
problem?
HOFFMAN: There’s a lot going on out there that
we’re not really aware of because companies are keeping it close to the
hip, because if they revealed this, they would basically be advertising
their own vulnerability. My
best guess based on what I’ve seen is that projects to do this sort of
thing are massively underfunded. Many businesses don’t have a clue about
how traditional business models have to change in order to accommodate
this issue of what it means to be a knowledge-reliant organization.
WTRI: How
would knowledge-reliant organizations need to change?
HOFFMAN: The key to capturing knowledge is, for
each expert, you have to identify what is their unique knowledge, the
knowledge has to be unique to them, it has to be critical to the
organization, and it’s got to be undocumented.
You don’t want to spend your time capturing knowledge that
doesn’t fit those definitions. You’ve got to be very specific. Companies
call me up and need knowledge capture done on an entire cohort of
experts. They have only
budgeted $40,000 and I have to tell them it’s going to cost $250,000.
They come back ask me what I could do for $40,000. Either they want to
deal with the risk or they don’t, and this is what it takes.
The next thing they’ll say is: “Can we get the experts knowledge
automatically? Is there
software for it?” At this point I realize they have massively
underestimated what it takes to do knowledge capture at this level.
Overall, I think there are a lot of companies who understand the issue
and want to do something about it, but it’s massively underfunded, and
the process can’t be short-circuited. That takes time and effort to
really get clear about concepts, procedures, principles and their
interrelations. Thomas Edison said, “Opportunities are often missed,
because they come wearing overalls and they look like work.”
WTRI: You
guys were on to this 15 years ago, what kinds of methods have been
developed in that time to address these issues? Why haven’t they caught
on?
HOFFMAN: There is what I call a tough nut problem,
which is the following: If I’ve got somebody who’s an expert and really
is essential to keep the organization working, the last thing I want to
do is take them away from their job so they can sit and answer questions
about their knowledge, making concept maps or doing CDM procedures – you
need to keep them on their job.
The option I prefer is to hire them back after they retire and
pay them as a consultant to get them to do knowledge elicitation.
It’s a win-win, because it’s way for the individual to remain ]
active after retirement, a cognitive benefit for aging individuals, and
is something a lot of experts are intrinsically motivated to do (unless
their former employer was toxic). But there must be a value added for
the expert, otherwise they will simply view it as additional work.
WTRI: We are
of course invested heavily in the immersive learning paradigm. We all
agree that knowledge elicitation cannot be a fully automated process,
but in your opinion, how immersive does the knowledge capture technique
have to be? What kinds of
specific methodological features need to be in place?
HOFFMAN: Methods such as practice-based learning
and simulation come in here. Lots of practice, lots of practice, lots of
practice. There are ways of spoon feeding people knowledge from a
knowledge base, generally not a good idea, and there are ways of
"transferring" knowledge well.
This comes up regarding Concept Maps. The best concept map a person can
learn from is one they’ve made themselves.
They read some material, they make their own Cmaps, and then they
look at the experts Concept Maps. But, don’t just give them a multiple
choice test. If any company
wants to do that, fine, but its not the approach I'd prefer.
WTRI: Your
story about how these companies don’t want to pay for these services is
near and dear to us. One of the things we try to do is show a value by
linking our service to a particular outcome.
HOFFMAN: The more business savvy companies would
present their knowledge capture techniques as a value added. In fact,
I’ve got a calculational method for businesses to determine their level
of risk with regard to their loss of expert knowledge. We're going to
make a software tool. Stay
tuned.
WTRI: We’ve
been talking a lot about knowledge capture and retention in terms of
expertise, but what about accelerated learning? Let’s say a company
isn’t able to capture expert knowledge, or they are in a business that
has no experts to handle new problems? How can companies accelerate the
development of expertise?
HOFFMAN: In order to teach somebody something, you
have to have that knowledge on hand. By capturing the knowledge, you’re
creating materials, libraries, and a case corpus that can then feed over
into training. Accelerated learning is something you might be able to do
as an added benefit, but if and only if you’ve done the knowledge
capture part of it. so knowledge capture and accelerated learning are a
coupled thing. Part of the problem is that there’s always more material
to train than there is time, always, and those involved in instructional
design are always under pressure to improve training. How do we define
improvement? Better performance with less training time. That’s how
governments define expert performance, that’s how managers define it.
But it is not necessarily the
best mind-set for growing expertise.
The question is, what are some new methods that
could help trainers do training better and in an accelerated timeframe?
This is where simulations come in. Simulation technology has a long way
to go and even the more advanced simulations can be limited. But that’s
where the research is and that’s why we need to keep pushing the
envelope. But if it takes 10 years to become an expert, can we make
experts faster? As far as I know, the only really substantive idea on
the table is something like time compression. It takes 10 years to make
an expert with on the job learning. Experts are people who by definition
are able to deal with rare and tough cases. But rare and tough cases
are, by definition, rare. If you could make an appropriate case corpus,
time compression is a possibility.
WTRI: We’re
finding that it may be the number of exemplars and failure cycles and
not necessarily the amount of time.
If you can compress the number of failure cycles and exemplars
with simulations do you feel as though learning could be accelerated in
this way?
HOFFMAN: There are cautionary tales here.
One of the ways in which people benefit from experience is that
they work on a difficult case. Maybe they get it right, maybe they get
it right for the right reasons, maybe they get it right for incidental
reasons, maybe they get it wrong. But, eventually they get feedback. In
the real world, there is often time to let the error and the feedback
sink in. One of the problems in some simulation-based training is the
idea that feedback has to be immediate, that is has to be process
feedback and outcome feedback. It’s a fine idea, but if feedback is
really immediate, you’re short circuiting the learner's opportunity to
stew about what they got wrong and to perhaps make some progress in
figuring out on their own why they got it wrong.
Unless the person has skills and experience at successfully
figuring out why they got something wrong, they will not achieve true
expertise. So immediate feedback may preclude learning all the right
skills. Not everything has
to be taught in order to be learned, and some things are better learned
than taught.
There’s another reason to be cautious about time
compression: the tough cases are really tough.
A weather forecaster may encounter, say, three cases a summer
where it’s very hard to tell where the thunderstorms are going to
suddenly arise, maybe five a year.
As an instructional designer, maybe you to knowledge elicitation
and capture the knowledge and now you’ve got 50 cases in your corpus.
Now you’re going to try to have learners go through 50 of the very tough
cases a year and at the end of that year, you’re going to have someone
whose brains are burnt out. So there are issues about time compression,
but there are a lot of significant researchable questions in this area
that should be researched.
The problem there is that we are talking about a
rather major, longitudinal study to track learning and performance
across, say, the apprentice to journeyman to expert stages.
And the study would have to be in at least two (interestingly
different) domains. Likely
only the government could pay for such a project.
It would take the better part of five years of someone's
professional career to convince the government to try it.
The stress-o-meter would be in a
logarithmic scale. But If I
did not have to do the leg work to promote it, I'd sure want to lead it!
For more information,
please contact us at 619.232.8054
or email at info@wtri.com
