The Patient Education and Activation System (PEAS) project aims to prepare people to take a more active role in their health care decisions. In this paper, we describe our work on the Layman Education and Activation Form (LEAF). LEAF is designed to be an interactive, Internet-based system for collecting a patient’s medical history. It is unique in that it gives patients access to educational information when it is most pertinent, while they are attempting to complete a form. It avoids overwhelming the patient, by only providing the information when it is likely to be relevant. The system avoids asking irrelevant questions or providing irrelevant facts by tailoring the content of the form based on the patient’s interaction with the system. The system also uses the patient’s answers to suggest questions that the patient might ask her doctor and provides online resources that the patient can browse.

Keywords: health education, Internet, medical history form

1. Introduction

The PEAS (Patient Education and Activation System) project aims to prepare people to take a more active role in health care decisions. With the collaboration of the Department of Medicine at the University of Wisconsin Medical School (Milwaukee Area Clinical Campus), the PEAS project group has been developing a coordinated set of computer programs for educating patients. The project is investigating strategies for helping people identify their health care concerns, learn what actions they can take, and verbalize their concerns to health-care professionals. These strategies address the problem of how to present relevant information efficiently and effectively. This is accomplished by a multi-media computer interface with intelligent tutoring and intelligent discourse processing.

 

One goal of our design has been to create an architecture that would alleviate the problem of misunderstanding and miscommunication between the patient and a computer system. Ultimately, we hope this will also alleviate possible miscommunication between the patient and her physician. Such breakdowns in communication can arise because of differences in vocabulary or differences in beliefs about the significance of different states of health. Moreover, errors in understanding can propagate, because people interpret each new fact on the basis of their understanding of old facts. To address these concerns, computer systems (and people!) must not ask patients to make choices without allowing them the opportunity to obtain clarification [1]. In addition, systems must avoid overwhelming their users with too much information at once. As in human conversations, computer systems should allow a dialog with users, where each participant takes a turn and then gives the other party a chance to accept it or to ask for clarification. In this way, misunderstandings can be detected and addressed, before communication breaks down completely [2].

 

This paper discusses an architecture for patient education that we have developed for the Layman Education and Activation Form (LEAF). LEAF extends the normal activity of filling in a medical form by including educational activities that will help patients to understand medical terminology and various health-care related issues. At any point during an interaction with LEAF, patients can access facilities for explaining the terminology of the form (Definitions) and for learning about topics that they might want to discuss with their doctor (Dialogue). (See Figure 1.) Unlike a hypertext-based system, which presents the same information to everyone, LEAF is interactive and tailors its interaction with each patient. It does this by using the information that the patient provides about herself.

 

For example, suppose that a patient has indicated that she is female, childless, and has a family history of heart disease. LEAF will dynamically filter irrelevant parts of the medical history form, such as those related to breast feeding. (This customization is important because irrelevant parts of the form are especially likely to contain terminology that is unfamiliar or confusing.) If asked for advice (via Dialogue), LEAF will offer tailored information about prevention and treatment strategies for heart disease and suggest questions for the patient to ask the doctor. Similarly, if asked for a medical definition, LEAF constructs a customized definition, based on the health history given by the patient.

 



 

Figure 1: LEAF’s Interface

 

A prototype implementation of LEAF has been built using the JAVA programming language, and is publicly available over the World Wide Web. This prototype demonstrates the functionality of LEAF and its ability to tailor the information that it provides. It also demonstrates the portability of LEAF. Along with the form itself, LEAF provides a short questionnaire that allows users to participate in an informal (and anonymous) evaluation of LEAF. This informal information has given us insight into how real patients might react to LEAF and where design improvements should be focused. The objectives of the prototype are limited in scope (to establish the proof-of-concept). For example, the current project does not address the problems of verifying the accuracy of the medical information that is provided. Any medical information provided is for illustrative purposes only, and should not be taken to represent the views of any of the PEAS project participants. The LEAF prototype demonstrates the feasibility and cost-effectiveness of patient-tailored computerized interfaces to medical information.

 

2. Background
1. Motivations for LEAF

LEAF aims to help bridge the communication gap that exists between patients and their doctors. Patients are being asked to take a more active role in their health care. Patients must therefore be prepared for the increasing amount of information that they will receive describing their options and possible courses of action. This preparation includes a better understanding of medical terminology. Although the resources available to health care professionals may be limited, computers can help by giving people access to medical information outside the doctor’s office and by adapting the presentation of information to suit the individual’s interests or expertise.

 

To adapt the presentation of medical (or any type of) information to each user, a system must have information about the user and must be able to reason about which parts of its knowledge apply to a given situation. Ideally, a system will monitor each patient’s interaction with the system and build a profile (called a user model) of the patient. As the system collects this information, it can use it (as well as the current state of the task) to modify the questions that it asks and to customize the educational materials that it presents. For example, it may choose to elaborate the core definition of a concept with specific information relevant to a health condition that the patient has. This customization is facilitated by organizing the knowledge base into small units of text that are indexed by facts about the patient or goals that the patient might have. LEAF’s design provides this type of flexibility.

 

In designing LEAF, we considered previous methods for collecting medical histories from patients and presenting educational materials to them. The results of our investigation supported the need for a system such as LEAF and motivated some additional design constraints that have been incorporated into the prototype. In the remainder of this section, we consider some of the lessons learned from prior work. We then will discuss the architecture that has been developed.

2. Traditional Medical History Forms

For reasons of generality, typical medical history forms contain questions and sections that might be irrelevant to any particular patient. They can also be confusing if they are worded awkwardly or contain medical terminology that is unfamiliar to patients. At the start of this project, we evaluated a number of such forms. We found questions about the health of the patient's spouse, which are irrelevant if the patient is single. We found sections that were intended "for administrative use only". We found potentially ambiguous questions, such as one that asked patients "To what extent are you experiencing difficulty in the area of:" and presented "apathy, lack or interest in things" as an area. Our evaluator found this confusing, because he felt that he could either respond "no difficulty", because he never feels apathetic, or he could respond "no difficulty", because he found it easy to be apathetic. (This confusion is especially easy for non-native speakers of English.) Finally, none of the medical forms that we examined provided explanations of the diseases or medical terms used, relying on the strategy that if a patient has not heard of a disease then she has not had it. However, this is not a good assumption for non-native speakers. Moreover, even native speakers may confuse terms that sound alike, such as "constipation" and "obstipation".

3. Computerized Medical Forms

Computer programs for collecting patients’ medical history offer the possibility of suppressing parts of the form that are unnecessary for patients and for including information that will help patients respond to the questions more accurately. Researchers and clinicians have been experimenting with computerized medical forms since the 1960’s [3-7]. This and subsequent work confirms that medical interviewing by computer can be as reliable as the traditional medical form and that patients will enjoy using it.

For example, HealthQuiz is a computer program, developed by the Clinical Practice Enhancement Project (CPEP), that gathers health-risk data from patients and provides feedback to both the patient and physician concerning guideline-based suggestions [8,9]. These suggestions include possible immunizations or diagnostic tests. In experiments with HealthQuiz, 89% of patients found the computer enjoyable to use and 85% considered HealthQuiz important to their care. Previous studies by CPEP examined the reliability of different methods for gathering various types of health information from patients. They found that consistent preventive care information can be obtained when patients use a computer. In addition, patients were more likely to be candid about risky health behaviors when communicating with a computer than with a human interviewer. As in LEAF, HealthQuiz filters questions on the basis of responses to earlier ones. Unlike LEAF, however, it does not support questions from patients about the terminology that it uses.

4. Customized Medical Explanations

Another potential benefit of computerization of medical history forms and educational materials is the possibility of customizing the content of these materials to individual patients. Several studies have shown that customized information is more effective than generic information. For example, in a study targeting smoking cessation, 30.7% of patients who received tailored health letters reported quitting after six months versus 7.1% in a control group (who received a generic health letter or no information)[10]. In a similar study targeting nutrition, total fat intake in the tailored group decreased 23%, but by only 9% in the non-tailored group [11]. The primary difficulty is in automating the customization process. Here we discuss three systems that have shown that automatic customization is possible and effective.

 

Migraineur is a system that provides customized information to patients suffering from chronic or acute migraine headaches [12]. It consists of an interactive history-taking module and an intelligent explanation module. The history-taking module collects information from patients prior to each visit, builds a user model, and summarizes the patient’s status for their physicians. The intelligent explanation module produces an interactive information sheet containing explanations in everyday language that are tailored to individual patients, and responds intelligently to follow-up questions about topics covered in the information sheet. This information sheet contains generic text that is presented to all patients, customized text that is presented depending on an analysis of the user model, and text that, when selected, will show the patients questions that they might ask about the words in that context. The information presented depends on the discourse history. As a result, it can avoid repeating the same response, even if the patient asks the same question more than once.

 

The most important difference between Migraineur and LEAF is that LEAF will provide customized information while its model of the patient is incomplete, whereas Migraineur can only do so after knowledge acquisition is finished. This requires motivated users who are willing to provide a history prior to being helped. Also, Migraineur’s knowledge acquisition component does not allow the user to ask for medical definitions or for help about the form while the patient is still working on it. Migraineur’s ability to track the discourse history is unique. (This ability is part of the design for LEAF, but it is not part of the current prototype.)

 

HealthDoc uses the patient’s medical information, physical characteristics, and medical diagnosis to create a document with medical advice tailored to the user [13]. HealthDoc differs from LEAF in that it is intended to be used by health-care professionals (rather than patients) and, like Migrainuer, it does not integrate the task of acquiring knowledge about the patient with the task of generating educational materials.

 

The work of Jimison et al., like Migraineur and HealthDoc, uses a model of the patient to generate customized educational materials [14]. These materials explain therapy decisions in terms of the patient’s own condition, medical history, and lifestyle (sedentary vs. active). The system chooses to include or omit therapy explanations depending on the contents of the patient model. It does not support questions about the explanations that it provides.

 

5. Systems to be Used at Home

One goal of LEAF is to prepare patients for subsequent discussions with a health care professional. Ideally such a system should be usable by people over the Internet, without any outside assistance. The feasibility of this idea is demonstrated by CHESS (Comprehensive Health Enhancement Support System), a system that people can access from their home[15,16]. CHESS provides access to a variety of services, including a compilation of answers to commonly asked questions; a dictionary of health-related terms; a database of articles, brochures, and pamphlets; a tutorial to show users what health and social services are available to them; a database of real-life accounts of people coping with health crises; facilities for anonymously contacting health-care experts (Ask an Expert) or members of appropriate support group (Discussion Group); and programs to help users assess their lifestyle, think through hard decisions, and develop a plan of action to implement their decisions.

 

The primary difference between the design of LEAF and the design of CHESS is that, in CHESS, each component is separate from the rest, so that, for example, a person who is using the assessment tool could not look up a word in the dictionary without leaving the assessment program. Also, CHESS does not build any model of its users or their interaction with the system, providing the same information to everyone. Moreover, tools that collect information about the user (e.g. to perform a risk assessment) cannot make this information available to other components. As a result, users of CHESS tend to confine themselves to those components that are personalized and dynamic (Ask an Expert, Discussion Group), but not wholly automated.

3. Design Objectives for LEAF

LEAF combines three tasks: collecting medical information about patients, providing information about the vocabulary and use of the form, and directing patients to additional educational materials that might be of interest. We anticipate that such a system would be used by a person who is waiting to see a physician or who is at home considering a visit to the doctor. Such a person will not always know what they do not know or what they will need to know to understand the choices that a physician might ask them to make.

 

LEAF has been designed to minimize misunderstandings and to help patients reach the most relevant information efficiently. The design supports these goals by allowing separate tasks to share information about what the patient is doing and what facts the patient has already told the system. Wherever possible, the system suppresses irrelevant questions and information. It also allows users to switch between tasks easily without causing the system to "forget" what it has been told. For example, the patient will have access to definitions of medical terminology, links to online resources, and instructions for filling out the form, while the patient is working on the form. Moreover, the information provided by these services is tailored according to which parts of the form the patient has completed or is currently working on.

.

4. System Description
1. The Implementation Platform

This section describes a prototype implementation of the Layman Education and Activation Form (LEAF). This implementation achieves the design that we have discussed. In addition, the implementation has been developed to maximize its portability---we want LEAF to be usable from a variety of locations using a variety of computer hardware. The solution that we have selected is to build a system that could be run using free and widely available programs for accessing the Internet, such as Netscape’s Navigator or Microsoft’s Internet Explorer. Such a program can be used to access files at a remote location on the Internet or can be used on a standalone machine that is running another program called a "server".

 

For reasons of network security, applications run from network browsing programs cannot write files on the local machine. Moreover, for portability reasons, we do not want files to be written on the remote machine either. LEAF addresses both of these concerns by generating new screens dynamically and communicating this information to the interface program directly.

 

The implementation of LEAF is composed of a number of programs written in JAVA, C, and the Hypertext Markup Language (HTML). JAVA is an object-oriented programming language that is well suited to building user interfaces because of its portability and the availability of ready-made code for implementing forms. The core medical history form presented by LEAF is a JAVA program that can be run by any "JAVA-aware" browser. HTML is a language for specifying the format of a document and for creating links between documents. LEAF uses it in the presentation of customized medical definitions and context-sensitive help. C is a general-purpose programming language; it is used to write CGI ("Common Gateway Interface") scripts that can be executed by the browser software. LEAF uses these programs to generate new HTML documents dynamically, on the basis of the user’s interaction with the system.

 

In the remainder of this section, we will consider LEAF’s overall architecture and the implementation of the modules that comprise LEAF.

2. Architecture

The architecture of LEAF has been designed to integrate three tasks: the collection of information about patients; the explanation of how to use the system; and the selection of educational materials that are most likely to address patients’ concerns. Each task is managed by independent modules that share underlying knowledge sources. In addition, because the program is run from a browser, the architecture has been refined to include a separate module that controls the look and feel of LEAF’s interface while providing access to the modules that control its content.

 

Figure 2 is a block diagram that shows the information flow between different modules. Each block represents a different module. In the figure, "User" refers to the person that uses LEAF. The user communicates with LEAF through LEAF’s interface (the Interface Module). Considering the figure from bottom to top, the Context-Sensitive Help Module provides information that explains use of the form, such as how to navigate between screens. The Knowledge Acquisition Module controls the medical history form and updates the User Model with the information provided by the patient. It also uses this model to customize the form. The Dialogue Module generates a list of suggested topics that are likely to interest the patient, given the current state of the user model. The Definitions Module generates a list of customized medical term definitions that have been constructed within the Knowledge Base Module, given knowledge passed from the user model.

 

In the remainder of this section, we will consider the architecture of LEAF in greater detail.

 



Figure 2: General Architecture of LEAF

3. Interacting with LEAF

The interface to LEAF consists of a window with six buttons above it, as shown in Figure 1 (at the beginning of this article). Three buttons help navigate the medical history form: Goto, Previous, and Next. The rest of the buttons: Definitions, Dialogue, and Help are used to access other modules. All output produced by LEAF is displayed using this interface, so that the patient always has access to any of the modules. Following is an explanation of the functionality provided by each of the buttons on the interface:

1. Goto: Displays a list of pages from the medical history form. This list may change depending on the information that the user has provided so far. LEAF will display the page selected by the user. (One section of the medical history form is displayed at a time.)
1. Previous: Shows the previous page of the medical form. If the user is on the first page of the form, pressing this button will show them the last page of the form.
1. Next: Shows the next page of the medical form. If the user is on the last page of the form, pressing this button will show them the first page.
1. Definitions: Provides access to the definitions module (described later). When a patients selects this action, LEAF shows a list of medical terms that it can define. Selecting any of the medical terms from the list causes LEAF to display a window with a definition of the medical term in hypertext. Patients can navigate these definitions by clicking the buttons, Back and Forward, in their Web browser.
1. Dialogue: Displays a list of relevant topics for the patient. Figure 3 shows an example of the topics that would be suggested for a female patient that has indicated that she has one of the risks of breast cancer and has a history of alcoholism and diabetes.
1. Help: Shows context-sensitive help on how to use LEAF.

1. Building a Model of the Patient

The Knowledge Acquisition module collects medical information about the patient and stores it in the user model. The mechanism that is used for collecting this information is a medical history form consisting of three components: a generic form with questions for all patients, a form with questions for females, and a form with questions for males over 40 years of age. The generic form consists of questions on demographics, the patient’s previous illnesses, and the patient’s blood relatives’ previous illnesses. The customized form for females includes a questionnaire on risks for breast cancer. The customized form for males includes a questionnaire for Benign Prostatic Hyperthrophy (BPH).

 

Users of LEAF can choose to fill out the medical form in any order. As they do so, the information that they provide will be used immediately to customize the form. Thus questions and sections may appear (or disappear) as soon as an answer is given. For example, in the absence of any information from the patient, LEAF will suppress questions related to breast cancer; however, as soon as a patient indicates that she is female, the form will be extended to include a questionnaire on the risk factors for breast cancer. Similarly, LEAF will not ask any questions about breast feeding unless the patient has indicated that she has children. If she does, then new questions will be added to the risk-factor form immediately. The information provided by other components (such as Dialog and Definitions) will also improve immediately, as they will have access to any information that has been added to LEAF’s model of the patient.

 

While LEAF is running, information about the patient is stored (within the program itself) as a user model. User models capture the system’s knowledge about the user [17]. This information may be static (based on a stereotype) or dynamic (based on the user’s interaction with the system). LEAF’s user model is constructed dynamically. Presently it takes into account only the information content of patients’ answers to the medical history form. This information includes demographic information like age and sex, reported illnesses and immunizations, and illnesses of family members. Mirroring the form itself, it also includes specialized structures for representing the risks factors of breast cancer of female users and the BPH symptoms of men. Other factors that might be added are the sequence of actions of the user or the user’s apparent level of expertise [18].

 

2. Generating Customized Information

Customized educational materials are provided to the patient in the form of medical definitions and suggested questions and topics. The list of suggestions is presented by the Dialogue Module in HTML format, with links to definitions or other resources. (See Figure 3.) For example, if a female reports at least one risk factor of breast cancer, LEAF will suggest "Do you want to know more about breast cancer?" and provide links to the definition of breast cancer and to sites that provide more information about breast cancer. Also, when a suggestion references multiple definitions, such as how diabetes can cause cataracts and glaucoma, LEAF provides links to information about these topics as well. The links are all active from LEAF, so that users connected to a network can access these sites by clicking on the item.

In future work, the Dialogue Module will be extended to support two-way, English-language dialog between the patient and the system, as in our B2 project [19,20]. The system will ask questions about concerns that the patient might have (beyond those indicated by the medical form) and will suggest assessment exercises (such as risk assessment) that might help clarify the patient’s understanding. It is anticipated that the tool will also link patients to programs that perform these exercises.

Customized definitions are generated when the patient selects Definitions from the main menu. The definitions that are generated take into account the information in the user model. For example, Figure 4 shows two possible definitions of estrogen that a patient might see. This particular customized definition would be given to a female patient who has indicated that she has at least one of the risk factors of breast cancer. Figure 5 shows three possible definitions of heart disease, illustrating how the current context of the interaction can also affect the selection of information. For example, the second definition would be given if the patient was involved in learning about estrogen. In general, definitions may contain text, images, audio, or video, as most appropriate to the concept being defined.

The content of definitions is provided by the Dynamically Customized Knowledge Base.

The knowledge base is organized as a semantic network. A semantic network represents concepts in terms of objects and relationships among them [21]. Each concept is represented as a node in the network.

Figure 6 shows a portion of the information in the nodes of the semantic network and the relationship between these nodes. Links connect nodes that represents generic information to nodes that represent context-sensitive information. In the current implementation, the nodes of this semantic network consist of HTML files and CGI scripts. Each of these nodes can be referenced many times by other nodes and contain general health information, a medical term definition, or a list of questions. Below is a list of the current types of relationships (links) between nodes:

1. Definition: It links a medical term or other term to its definition, e.g. "Estrogen is a hormone."
1. Definition Answer: It links a question asking for a definition to the answer to that question, e.g. "What is estrogen? Estrogen is a hormone."
1. Prevention Answer: It links a question asking how to prevent an illness to its answer, e.g. "How does one prevent osteoporosis? Consume calcium to prevent osteoporosis."
1. Questions: It links a medical term to questions about it, e.g. "Questions about osteoporosis are: 1. What is osteoporosis? 2. How does one prevent osteoporosis?"
1. Relationship: It shows the relationship between two things, e.g. "Estrogen can reduce the risk of heart disease."

In the implementation, this information is organized as a set of files which can be revised or extended:

• HTML Definition files contain core definitions of medical terminology
• CGI Definition scripts build customized definitions of medical terminology
• Context sensitive files contain definitions of medical terminology as they relate to other definitions in context.
• Files with lists of questions that LEAF can suggest to a patient within a particular context.
• Files with answers to the questions that LEAF suggests to patients.
• Files with treatments that are associated with particular medical conditions.

1. Preliminary Evaluations

 

Based on the design discussed above, we have implemented a prototype implementation using JAVA, C, and the Hypertext Markup Language. (We have also implemented an all JAVA version, to investigate how LEAF might be used from a machine not connected to a network.) As mentioned earlier, the current prototype provides much of the intended functionality of LEAF, but contains limited amounts of content information. (This information comes from published sources [22,23].) With this prototype we have been evaluating the design of LEAF according to two sets of criteria: first, we aim to evaluate LEAF with respect to its usability as a system (e.g. its speed and portability); second, we aim to evaluate LEAF with respect to its perceived usefulness by patients. Evaluations are performed online by users after they access LEAF and provide measures of user-satisfaction and directions for future enhancements.

 

1. An evaluation of system performance

 

The prototypes have been tested using a variety of hardware (standalone, connected to a phone line, and connected to the computer network via Ethernet) and software (Mac OS, Microsoft Windows 95/NT, Linux) configurations. The speed of LEAF depends only on the user’s machine’s ability to load and run JAVA; once loaded, interaction delays are negligible and do not depend on the amount of knowledge in LEAF’s knowledge base or user model. In practice, the only difficulties that we have discovered involve users who had difficulty in the configuration of their browser software (or fire-wall restrictions) that prevent them from running JAVA programs or CGI scripts. The all-JAVA versions of our program can address these issues, but at some sacrifice in speed. Overall, the tool is clearly portable and loads and runs quickly. There have been no complaints from users regarding system performance.

 

2. An evaluation of user satisfaction

 

The ultimate usefulness of LEAF will depend on how well real it meets the needs and expectations of real medical patients. However, before significant time and money is spent to collect and validate large amounts of medical information, it is important to verify whether this investment is warranted. Toward this end, as we have been adding more content to LEAF, we have been conducting an ongoing assessment with the cooperation of visitors to the LEAF website (including students who have accessed the site as part of a class exercise.) We have also been conducting an evaluation of the questionnaire itself.

 

The questionnaire (part of which can be seen in Figure 7) asks users to comment on how often they used the different services provided by LEAF and how useful and understandable they found the information that was provided. The questionnaire also asks users to report their satisfaction with the system as a whole. The surveys are completely anonymous and do not include any information from the medical history form itself. The only demographic information that is requested is the user’s gender, because, although LEAF contains some gender-neutral information (such as about eye problems), it contains proportionately more information on women’s health issues.

 

The results of this survey indicate that users responded favorably to LEAF’s design. Overall, out of 35 completed user evaluations, 87% of users prefer to use an online medical questionnaire over a paper one; they cite the ease of filling out the form and the links to medical definitions and Internet sites among their favorite aspects of LEAF. Users also like having medical information tailored to their interests; 71% found that the definitions were somewhat or very helpful and 58% thought the dialogue sections helpful. Users chief complaint, in fact, was that they wanted LEAF to provide more customized information in definitions. Users even suggested adding the customized link information provided by the Dialogue component within the Definitions themselves.

 Figure 7: Dialogue Screen

 

2. Discussion

In this paper, we have considered the design and implementation of Layman Education and Activation Form (LEAF). Upon consideration of existing medical forms, we found that computerization offered the opportunity for flexibility and customization not allowed by traditional methods. Our work has addressed the problem of developing an architecture that would allow patients to move freely between tasks in such a way that the information that they provide, whenever they provide it, can be used to improve the amount of customization that is provided. The design, being browser based, is also highly portable. We have also explored the incremental presentation of definitions, where we allow users to specify when they have received enough information.

 

LEAF allows users to interleave the tasks of filling in a medical form with activities that will help them understand medical terminology. Our studies with potential users have shown that having access to medical information while they are working on the form was helpful. One difficulty that we anticipate is adding all the vocabulary and resources that patients will want to access. Currently, the addition of new information requires the assistance of someone with knowledge of computer programming. Future developments will focus on providing tools and specification languages for allowing users to create medical content that can be included in future versions of the prototype.

 

Acknowledgments:

 

This work has been supported by grants from the National Science Foundation (IRI-9523646 and IRI-9701617) and by a gift from the University of Wisconsin Medical School, Department of Medicine.

 

4. References

1. McRoy SW. Misunderstanding and the Negotiation of Meaning, Knowledge-Based Systems, 1995; 8(2-3): 126-134.

 

2. Schegloff E. Repair After Next Turn: The Last Structurally Provided Defense of Intersubjectivity in Conversation, American Journal of Sociology, 1992; 97(5):129-135.

 

3. Slack WV, Hicks GP, Reed ER ,Van Cura, LJ. A Computer-based Medical History System. New England Journal of Medicine 1966; 247: 194-198.

 

4. Slack WV, Van Cura LJ. Patient Reaction to Computer-based Medical Interviewing. Comput. Biomed Res 1968; 1:527-531.

 

5. Chodoff, P, Helrich M. Construction of an Automated System for the Collection and Processing of Preoperative Data. Anesth. Analg. 1969; 48:870-876.

 

6. Collen MF, Cutler JL, Siegelaub AB, Cella RL. Reliability of Self-Administered Medical Questionnaire. Arch Intern Med 1969; 123:664-681.

 

7. Rockart JF, McLean ER, Hershberg PI and Bell GO. An Automated Medical History System Arch Intern Med 1973; 132:348-358.

 

8. Hayward RSA, Ford DE, Summerell D, Roizen M, Steinberg E. Randomized Clinical Trial of a Patient-Administered Computerized Preventive Care System to Implement Adult Practice Guidelines. Clinical Research 1992; 40: 608A.

 

9. Hayward RSA, Smittner JP, Meyer P, Ford DE, Roizen M, Steinberg E. Computer Versus Interview Administered Preventive Care Questionnaire: Does Survey Medium Affect Patient Response Reliability? (Manuscript).

 

10. Strecher VJ, Kreuter M, Den Beur D, Kobrin S, Hospers HJ, Skinner CS. The Effects of Computer-Tailored Smoking Cessation Messages in Family Practice Settings. Journal of Family Practice, 1994; 39(3):262-268.

 

11. Campbell MK, Devellis BM, Strecher VJ, Ammerman AS, Devellis RF, Sandler RS. Improving Dietary Behavior: The Effectiveness of Tailored Messages in Primary Care Settings. American Journal of Public Health, 1994; 84(5):783-787.

 

12. Buchanan BG, Moore JD, Forsythe DE., Carenini G, Ohlsson S, Banks G. An Intelligent Interactive System for Delivering Individualized Information to Patients. Artificial Intelligence in Medicine 1995; 7 (2): 117-154.

 

13. DiMarco C, Hirst G, Wanner L, Wilkinson J. HealthDoc: Customizing Patient Information and Health Education by Medical Condition and Personal Characteristics. Workshop on Artificial Intelligence in Patient Education. Glasgow, August 1995.

 

14. Jimison HB, Fagan LM, Shachter RD, Shortliffe EH. Patient-specific Explanation in Models of Chronic Disease. Artificial Intelligence in Medicine 1992; 4: 191-205.

 

15. Gustafson DH, Bosworth K, Hawkins RP, Boberg EW and Bricker E. CHESS: A Computer-Based System for Providing Information, Referrals, Decision Support and Social Support to People Facing Medical and Other Health-Related Crises. In Proceedings of the 16th Annual Symposium on Computer Applications in Medical Care. 1993.

 

16. Gustafson DH, Hawkins RP, Boberg EW, Bricker E, Pingree S, Chan, C. The Use and Impact of a Computer-Based Support System for People Living with AIDS and HIV Infection. JAMIA 1994; 604-608.

 

17. Kass R, Finin T. Modeling the User in Natural Language Systems. Computational Linguistics 1988; 14(3):5-22.

 

18. Cawsey A. Explanation and Interaction: The Computer Generation of Explanatory Dialogues. Cambridge, MA: The MIT Press, 1992.

 

19. McRoy S, Haller S, Liu-Perez A, Helwig J. B2: A Tutoring Shell for Bayesian Networks that Supports Natural Language Interaction. In Working Notes, 1996 AAAI Spring Symposium on Artificial Intelligence and Medicine, Stanford, CA 1996.

 

20. McRoy S, Haller S, and Ali S. Uniform Knowledge Representation for Language Processing in the B2 System. Journal of Natural Language Engineering 1997; 3(2).

 

21. Quillian R. Semantic Memory. In Semantic Information Processing Minsky M (ed). Cambridge, MA: The MIT Press, 1968.

 

22. Simons A, Hasselbring B, Castleman M. Before You Call the Doctor: Safe Effective Self-Care for Over 300 Common Medical Problems. Fawcett, Columbine, 1992.

 

23. Wallis C. The Estrogen Dilemma, Time Magazine, June 26, 1995.