Mapping knowledge

Using a questionnaire has proved, in my experience, to be the least effective way to map and understand knowledge gaps, surface issues , to identify worthwhile new practices or opportunities

Why?

Knowledge is a complex subject, most people confuse information & knowledge and need exposure & training to understand and appreciate the distinction

A questionnaire will return what the respondent want you to 'see' rather than their assumptions, true experiences, actual issues and deep desires

There are always problems with 'your' interpretation and scoring no matter how carefully the survey wording is crafted and how mant times it is tested

To appreciate knowledge gaps, you need to understand the personal networks and work context - This is impossible to get via a survey - it requires immersion

Answers are clouded by personal and group assumptions / worldviews which are mostly tacit and unarticulated

Awareness of knowledge gaps comes best through conversation and engagement in practice - This awareness is emergent, never complete and strongly dependent on prior experiences and exposures

Surveys tend to overstimate the problem and the returns are very difficult to aggregate for the resons given

May I suggest some alternatives?

Convene an openspace gathering so participants can expore and reflect on their issues and gaps

Conduct an ethnographic, action research project observing and asking questions insitu to obtain situated recall and to document exceptions

Observe rather than ask - people forget, they remember selectively and with constant bias

Knowledge gaps come in many forms, the most prevalent is a lack of a forum (community) to surface distinctions, share insights, support learning and having trusting colleagues to make you aware

Conducting a knowledge mapping exercise is likely to give a more holistic picture and a far better ROI.

Comments

[Jirka]

Denham, it's very interesting how your premise and your "alternatives" correspond with what Jacob Nielsen suggested in his "First Rule of Usability" (Alertbox, August 5, 2001, http://www.useit.com/alertbox/20010805.html): "To design an easy-to-use interface, pay attention to what users do, not what they say. Self-reported claims are unreliable, as are user speculations about future behavior." ie. "Watch what people actually do." + "Do not believe what people say they do." + "Definitely don't believe what people predict they may do in the future."

[Dina Mehta]

I couldn't agree with you more Denham. And this is where i feel most KM solutions fail - in really understanding what users of the system really need and desire. Often, users themselves maynot even be conscious or aware of their needs - then how can cold questionnaires capture them. Openspace sessions and ethnographic studies and observations seem to be critical precedents in designing KM systems and solutions.