Protocol-Generation Techniques

The aim of these techniques is to produce a protocol, i.e. a record of behaviour, whether in audio, video or electronic media. Audio recording is the usual method, which is then transcribed to produce a transcript.


Interviews

Various types of interviews can be used to produce a transcript. Unstructured interviews have a rough agenda but no pre-defined structure, so that the expert and knowledge engineer are free to explore the domain. This is an inefficient way of gathering detailed knowledge, but can prove useful as an initial interview when little is known of the domain. It also acts as an ice-breaker to establish a rapport between the expert and knowledge engineer. A semi-structured interview combines a highly structured agenda with the flexibility to ask subsequent questions. The questions for a semi-structured interview are ideally constructed some time before the interview and are sent to the expert so he/she can start to prepare responses. For an interview lasting 1 hour, around 10-15 questions might be asked. This allows time in between the set questions for the knowledge engineer to ask supplementary questions to clarify points and ask for more detail where necessary. This is often the preferred style of interview as it helps to focus the expert on the key questions and helps avoid them giving unnecessary information. Another form of interview is the structured interview. This allows no flexibility on the part of the knowledge engineer whose questions are all pre-established. As such, structured interviews often involve filling-in a matrix or other diagrammatic notation.


Commentary

Another family of techniques that produce protocols are think aloud problem-solving or commentary. These techniques generate protocols by having the expert provide a running commentary on a typical task used in the domain. The basic technique here is the self-report, in which the expert provides a running commentary of their thought processes as they solve a problem. Experimental evidence has shown that self-reports can access cognitive processes that cannot be fully recalled without bias and distortion if explained after the task has been completed. A problem with the self-report technique is that of cognitive overload, i.e. the mental effort required by the expert to provide the commentary interrupts and affects their performance of the task. This is especially true in dynamic domains where time is critical. One way around this is to use an off-line reporting technique. Here the expert is shown a protocol of their task behaviour, typically a video, and asked to provide a running commentary on what they were thinking and doing. An advantage of this is that the video can be paused or run at slow speed to allow time for full explanation. Variants of these reporting techniques involve a second expert commenting on another expert’s performance.


Teach Back

In the teach back technique, the knowledge engineer describes part of the knowledge that has been acquired during previous sessions or from other sources. The expert comments on what the knowledge engineer is describing to reveal misunderstandings.


Observation

Observational techniques are another way of generating protocols. Simply observing and making notes as the expert performs their daily activities can be useful, although a time-consuming process. Videotaping their task performance can be useful especially if combined with retrospective reporting techniques. On the whole, though, simple observation techniques are rarely used, as they are an inefficient means of capturing the required knowledge.



Other Knowledge Acquisition Techniques:

Go to: Protocol Analysis techniques
Go to: Laddering techniques
Go to: Diagram-based techniques
Go to: Matrix-Based techniques
Go to: Sorting techniques
Go to: Repertory Grid technique
Go to: Limited-Information and Constrained-Processing tasks


Last modified: 20 November 2003