Susan McRoy, Syed Ali, and Ethan Munson
Electrical Engineering and Computer Science
University of Wisconsin-Milwaukee
Milwaukee, WI 53201
The IMPROVE (Interactive Multimodal Probability Value Explanations.) project aims to improve the delivery of health care by increasing the accessibility of technical medical information to both health professionals and the public at large. The project is investigating novel techniques for explaining medical research results and their implications. These techniques combine a multimodal computer interface (supporting speech, typed text, and direct manipulation of graphics and animations) with intelligent discourse processing that tailors the explanations to the user's level of expertise.
As large volumes of clinical data are becoming publicly available, the ability of health professionals to inform the public has also improved. An informed public can make better decisions about their health choices. One difficulty is the limited resources that health professionals can devote to keeping up with developments in their areas of expertise and to imparting these developments in an understandable way to the public. The IMPROVE project will automate explanation of clinical data for health professionals and the public using artificial intelligence techniques.
Tools for medical decision analysis offer health professionals a systematic way to interpret new diagnostic information or to select the most appropriate diagnostic test. These tools support a doctor's practical experience with quantitative information about how diagnostic test results affect the probability of different diagnoses or outcomes. (The probabilities are based on the results of prior statistical studies.) Such systems can also be a basis for educating the general public about health issues. However, the probability information alone can be difficult to interpret, unless it is embedded in a system that can interact with the user and provide understandable explanations.
The IMPROVE system will address the complexity of explaining probabilistic models by integrating multimodal depictions of the data and by conveying explanations incrementally, to suit users' needs and abilities. The approach facilitates an understanding of the overall model, by allowing users to view graphical representations of the overall model topology and to obtain natural language summaries of important results. Users can also view animated sequences that illustrate the flow of information through the model during inference and use natural language to identify situations or events that occur and make queries about them. The approach facilitates a fine-grained understanding of the model by allowing users to select portions of the model via direct manipulation or natural language description and ask questions about the numeric or symbolic content. Users will receive synchronized graphical and verbal replies, that clarify and reinforce the system's answers. The system will also adapt to users' abilities to assimilate new information, by presenting information in a conversational manner, and tailoring the interaction to the users' concerns and apparent level of understanding. To do this, the system will analyze the the presentation objects that users select and the number and types of questions that they ask, and then adapt the content, pace, and format of the interaction appropriately.
The IMPROVE project also addresses the problem of designing effective methods for communicating an explanation to users of varying levels of expertise, given a uniform representation of the underlying facts. Although the underlying health information is the same, explanations directed at the general public must be different from the explanations aimed at health care professionals. In particular, when dealing with the public, an educator cannot simply present the information as a set of facts and justifications based on clinical utility. Instead, the educator must relate the health information to the particular concerns of the individual. Thus, the educator must first identify what those concerns are and then construct a compelling argument in terms of those concerns. For example, if the educator learns that her audience is a mother who is concerned about her inability to keep up with her young children, the educator can explain how improving her own nutrition will increase her energy level and will also reduce the children's tendency to be hyperactive. IMPROVE will take such special interests into account when providing an explanation.
The IMPROVE system will help make the complex probabilistic relationships
that underly modern decision support systems cognitively accessible to
health professionals and the general public. As a result, people will be able
to use these systems confidently and effectively, without requiring special
expertise in probabilistic modelling.
This research project is significant because: it addresses the pressing
problem of learning about and disseminating important new developments in
health care, despite the descreasing resources available to health care
professionals; and it provides a framework for multimodal explanation of
complex decision-support systems to both technical and non-technical users.