From bogus-date-line@gacvx1.gac.edu Mon Jan 1 00:00:00 1980 Date: Fri, 27 Sep 1991 22:11 CST From: Paul Kleeberg Subject: Criticism of Clinical Decision Support Systems I saw this in AI-Medicine@VUSE.Vanderbilt.Ed and thought I'd pass it on to the group. The article starts out referring so someone's algorythm for a decision support system, but knowing the contents of that is not necesary to understand what this writer says. I agree with this author. If you wish to know how to join ai-medicine, send me e-mail (if I had it handy at the moment I'd include it here). If enough people request it, I will post it to this group. Paul ------------------------------------------------------------------------- From: IN%"sean@dsl.pitt.edu" 27-SEP-1991 12:30:19.68 To: ramesh@ISI.EDU CC: ai-medicine@vuse.vanderbilt.edu Subj: RE: Diagnostic Algorithms: results at last! Ramesh has, quite skillfully, responded to the posting regarding the modelling of diagnostic behaviors and I can add litte to this. But I would like to suggest that a more reasonable criticism of systems to facilitate diagnosis in clinical medicine is not are these reasonable, but rather, are they needed at all? In an editorial which was written by Huth and which appeared sometime in 1989 he discussed previously published work dealing with test over- utilization among clinicians and concluded that the most probable reason was excluded from the list of 50 or so namely, plain old uncertainty. In essence what was in question was NOT the diagnosis, but rather, whether everything had been done to exclude alternatives; it is not the diagnosis, but confidence in *my* diagnosis that is is questioned. The importance of this, I believe, cannot be underemphasized. Who is satisfied NOT to get an EMG in a patient with a clear clinical diagnosis of ulnar neuropathy? What do you imagine it will tell you? Who doesn't get a CT scan in a young patient with recent onset of recurrent headache refractory to treatment *IN SPIET OF* the total absence of clinical indicators for a structural cause (there are reasons, but far fewer than one would think). Echocardiogram is, perhaps, one of the most overutilized diagnostic procedures while (as was pointed out in an article which also appeared two years ago) auscultation of the heart is clearly underutilized. Is the diagnosis the issue, here? Rarely? What is in question is the clinician's ability to defend themselves against the charge that they didn't look hard enough. When can you be satisfied that you know enough to say that you are reasonably certain? Against that background are systems based on compiled knowledge very helpful (rule-based systems, for example)? I would argue NO! Not because they aren't, at some point, correct, but because they are not the best sources of knowledge. In a recent publication Henry Krakauer of the Health Care Financing Administration discussed the utility of Medicare discharge data as input to clinical decision support. To back up his claim, he cited the use of Medicare outcomes data to provide a predictive model for the selection of patients who would benefit from coronary artery revascularization following acute myocardial infarction. In demonstrating system he posed the question in this way: My patient is a XX year old white male with a history of ... and the question is: will this patient benefit from coronary revascularization? What is the predicted outcome if I do nothing? What is the predicted outcome if intervene. On the basis of case features a cohort poulation was selected >from current pooled patient data and regression analysis applied to project outcomes. The question was answered not on the basis of an article published last week (meaning written last year), but on the real time analysis of the most current patient data collected on over 35 million Americans over the age of 65. The very same method could be used to answer the question "Will the information that I get from this test have any effect on the patient's outcome sufficient to justify the expense, discomfort, risk?" Note that there are markedly different technologies needed in this model than what most medical-AI types deal with, i.e., an large network through which data can be collected, and a very large and very fast database. We seem to forget, sometimes, that the first researchers in AI that chosen medicine as a problem domain did so, not because of an interest in medicine, but because of an interest in diagnosis as an example of intelligent behavior. Medical diagnosis was one example (perhaps a poor one given that there are much simpler and easier models in other physical systems). Automated diagnosis has rarely interested the medical community, not because of a fear of removing the human element (we've already done that with our reimbursement system) or of replacing humans with machines but, more simply, because diagnosis (as most people view it), is not really the problem. Most clinicians manage some form of diagnosis and most patients are treated appropriately. What is needed is better information on the utility of information and the means to obtain it which least stresses the system. The best source of this information may, in fact, be pooled knowledge of real patients, not compiled knowledge of some particular problem domain. Of course there are probably not 10 people in NIH who have read Krakauer's article so I don't expect to see any sorely needed policy shifts in NIH funding in the next few years. So I guess my two cents are: if you wanna do funded research build systems in Prolog which exploit UMLS using the Arden Syntax to query AIDS literature databases. If you wanna have an influence on patient care, look at distributed architectures, networked databases, high-bandwidth Wide Area Networks that link up health care organizations at minimal costs, the development of good (not ad hoc) standards for patient information inter- change, improving the quality of hospital information systems so that they manage patients not patient's money and removing legal barriers to health care information sharing. Of course, no one will fund you, but you'll be doing something important. Sean McLinden Medical Applications Group Information Technology Center Carnegie Mellon University School of Computer Science -- Paul Kleeberg, M.D. | PGY2, Family Practice, University of Minnesota 217 West Swift St. | Editor, Computers in Family Medicine Newsletter St. Peter, MN 56082 | Owner, Fam-Med@GAC.Edu, Fam-Med@GACVAX1.BITNET Voice: 507-931-6721 | Paul@GAC.EDU Fax: 507-931-6752 | Paul@GACVAX1.BITNET ================ From bogus-date-line@gacvx1.gac.edu Mon Jan 1 00:00:00 1980 Date: Sun, 9 Feb 1992 22:14 CST From: Paul Kleeberg Subject: Post Operative Expert Medical System (POEMS) The following traces a brief discussion regarding a Post Operative Expert Medical System (POEMS) I had with Dr. Sawar about a month ago that I am (finally) passing on to the list. Dr. Sawar is not on our list however I will forward any relevant comments on to him. PK ----------------------------Original Messages Follow------------------------- Date: 16-JAN-1992 15:52:04.67 From: IN%"sawar@cbl.leeds.ac.uk" "M J Sawar" To: PAUL@gacvx1.gac.edu Subj: medical dictionary ... I am interested in the medical dictionary you mention in you email to info-mac. We at Leeds University are working on a medical expert system for post-operative care, and often need access to the medical terminology. We recently got hold of UMLS Meta-thesaurus and related material and are thinking of extracting the medical terminology from it.... M.J.Sawar CBLU Leeds University U.K. --------------- Date: 16-JAN-1992 23:21:19.49 From: GACVX1::PAUL "Paul Kleeberg" To: IN%"sawar@cbl.leeds.ac.uk" Subj: RE: medical dictionary ...I am very interested in hearing more about your system. Could it be utilized in the States or would it require major revision? Some of the Family Physicians on my list might be interested in hearing about your project if you would care to describe it. With your permission, I would post it on my list.... Paul --------------- Date: 20-JAN-1992 17:00:24.85 From: IN%"sawar@cbl.leeds.ac.uk" "M J Sawar" To: PAUL@gacvx1.gac.edu Subj: RE: medical dictionary POEMS is currently being developed as a Post-Operative Expert Medical System, but it utilizes a generic diagnosis engine which can also be applied to other domains of medicine. The basic principle behind POEMS is that the data collected from the patient (past medical history, operative history, present problem history, examination, investigative tests) is entered, and then the user can ask for the diagnosis. POEMS returns a list of diagnoses orderd by their likelyhood, varying from very-likely to possible. The user can ask various explanatory questions, or request for a recomended treatment action. The diagnosis is incremental, so as new data comes along over a period of time the diagnosis gets updated. POEMS has a realistic investigative loop implemented, which keeps tabs on the contra-indication of the treatments given, plus the signs or symptoms which are not yet known, but will help in discriminating the diagnoses allready reached by raising or lowering their likelyhoods. With the help of this loop the system can request information not provided by the user. As for your question "Could it be utilized in the States or would it require major revision?" Yes it can be utilized in States, Infact we have a collegue at Mexico State University, Dr. Stuart Duban who is also involved with the development of this system, and has provided us with a lot of data required for explanation purposes and other diagnosis steps. Of course some of the treatment actions doses will have to be modified, since they have been taken from the British National Formulary (BNF), Plus the Biochemistry and Haematology normal values which vary from hospital to hospital. If you need any more info let me know, I would be more than happy to provide it. M.J.Sawar CBLU Leeds University Leeds LS2 9JT U.K. -- FORWARDED BY: Paul Kleeberg, M.D. | PGY2, Family Practice, University of Minnesota 217 West Swift St. | Editor, Computers in Family Medicine Newsletter St. Peter, MN 56082 | Owner, Fam-Med@GAC.Edu, Fam-Med@GACVAX1.BITNET Voice: 507-931-6721 | Paul@GAC.EDU Fax: 507-931-6752 | Paul@GACVAX1.BITNET ================ From bogus-date-line@gacvx1.gac.edu Mon Jan 1 00:00:00 1980 Date: Sun, 9 Feb 1992 23:41 CST From: Paul Kleeberg Subject: Neural network bests doctors at diagnosis "Neural network bests doctors at diagnosis" Computerworld 12/16/91 MI Diagnosis Scoreboard: Sensitivity Specificity ER Physicians: 78% 85% Neural Net 97% 96% Conclusion: "Someday, emergency room doctors will be able to enter patient symptoms, health history and other data into a handheld calculator attached to a personal computer to double-check their decisions and to help alleviate the pressure of having to instantly diagnose patients, according to Baxt." COMPUTERWORLD [Seems to me it could soon become a standard of care. PK] --------------- The Article: "Use of an Artificial Neural Network for the Diagnosis of Myocardial Infarction," William G. Baxt, M.D., Annals of Internal Medicine. 1991;115:843-848. ABSTRACT: * Objective: To validate prospectively the use of an artificial neural network to identify myocardial infarction in patients presenting to an emergency department with anterior chest pain. * Design: Prospective, blinded testing. * Setting: Tertiary university teaching center. * Patients: A total of 331 consecutive adult patients presenting with anterior chest pain. * Measurements: Diagnostic sensitivity and specificity with regard to the diagnosis of acute myocardial infarction. * Main Results: An artificial neural network was trained on clinical pattern sets retrospectively derived from the cases of 351 patients hospitalized with a high likelihood of having myocardial infarction. It was prospectively tested on 331 consecutive patients presenting to an emergency department with anterior chest pain. The ability of the network to distinguish patients with from those without acute myocardial infarction was compared with that of physicians caring for the same patients. The physicians had a diagnostic sensitivity of 77.7% (95% CI, 77.0% to 82.9%) and a diagnostic specificity of 84.7% (CI, 84.0% to 86.4%). The artificial neural network had a sensitivity of 97.2% (CI, 97.2% to 97.5%; P= 0.033) and a specificity of 96.2% (CI, 96.2% to 96.4%; P<0.001). * Conclusion: An artificial neural network trained to identify myocardial infarction in adult patients presenting to an emergency department may be a valuable aid to the clinical diagnosis of myocardial infarction; however, this possibility must be confirmed through prospective testing on a larger patient sample. -- Paul Kleeberg, M.D. | PGY2, Family Practice, University of Minnesota 217 West Swift St. | Editor, Computers in Family Medicine Newsletter St. Peter, MN 56082 | Owner, Fam-Med@GAC.Edu, Fam-Med@GACVAX1.BITNET Voice: 507-931-6721 | Paul@GAC.EDU Fax: 507-931-6752 | Paul@GACVAX1.BITNET ================ From bogus-date-line@gacvx1.gac.edu Mon Jan 1 00:00:00 1980 Date: Mon, 24 Feb 1992 23:54 CST From: Paul Kleeberg Subject: RE: diagnostic software The following appeared on MedLib-L@UBVM.Bitnet, "Medical Libraries Discussion List:" ----------------------------Original Message Follows------------------------- Date: Mon, 24 Feb 1992 20:36 CST From: IN%"MEDLIB-L@UBVM.BITNET" "Medical Libraries Discussion List" To: "PAUL KLEEBERG, M.D." Subj: RE: diagnostic software [Header truncated. PK] We also have a site license from Applied Informatics in Utah for ILIAD here at the University of Michigan Medical School, in the Learning Resource Center. The software is mounted both on computers in the LRC (six Mac II's) as well as on MacIIs on each of the Medicine floors of the Adult General Hospital. We currently require all M-3s who rotate through Medicine to complete a simulated patient expereince with ILIAD. SO far, student response to our exercise has been *overwhelmingly* positive. We have also found that a number of House Officers have used the package (in consultation mode) over at the hospital. The LRC also has a old version of QMR, which we are planning to update. I would like to offer a medical informatics course here which deals with various issues about "decision support software", and believe that each package has certain strengths and weaknesses. Judith Miller, Manager(and Ass't Director) Learning Resource Center 3960 Taubman Medical Library 1135 East Catherine St Ann Arbor, MI 48109-0726 judith_miller@umich.edu -- FORWARDED BY: Paul Kleeberg, M.D. | PGY2, Family Practice, University of Minnesota 217 West Swift St. | Editor, Computers in Family Medicine Newsletter St. Peter, MN 56082 | Owner, Fam-Med@GAC.Edu, Fam-Med@GACVAX1.BITNET Voice: 507-931-6721 | Paul@GAC.EDU Fax: 507-931-6752 | Paul@GACVAX1.BITNET ================ From bogus-date-line@gacvx1.gac.edu Mon Jan 1 00:00:00 1980 Date: 24 Sep 1992 14:40:05 -0300 (BST) From: A.J.Birchall@compsci.liverpool.ac.uk Subject: Apache - Acute Physiology and Chronic Health Evaluation This article appeared in the UK's Daily Telegraph newspaper recently: "Computing our chances" (Uneasy decisions may be made by technology, say Myles Harris GP). In a letter to The Lancet a year ago, the secretary of the Dutch Physicians League expressed his disquiet about mercy killing in Holland, a country which has liberal rules on euthanasia. He quoted a 1970 editorial from the Journal of the Californian Medical Association, which warned that in the future people might be disposed of whose quality of life fell short of certain medical criteria, and that after birth control could come death control. Society will accept euthanasia, voluntary or compulsory, because: "The new ethics of relative rather than absolute and equal values will ultimately prevail." Many doctors share his feeling that this is what will eventually happen. Because human beings find such decisions on life and death so difficult and upsetting, one wonders if computer developments may take over many of the "rational" and "objective" judgements on moral issues. American scientists have developed an intensive-care computer program called Apache - Acute Physiology and Chronic Health Evaluation. Apache monitors patients conditions and offers, it is claimed, a 95 percent reliable estimate of a patient's chances of dying. Apache III, which plumbs its memory banks of 18,000 previous cases to reach a prediction, is claimed to be as good as the country's leading intensive-care specialists in predicting the odds. Apache is used to guide doctors on how well they are treating their patients; it is not used to decide on mercy killings. But in a world where mercy killing was permitted, how long would it be before similar computerized assessments were allowed to enter into a doctor's decision to terminate somebody's life? Modern health care is hugely expensive and, as the population gets older and unable to produce the wealth needed to keep the state functioning, the temptation to remove the very sick using computer-aided judgements that are seen to be "less biased" would be hard to resist. The Rubicon would be crossed when the state - the main stakeholder in health care - began inserting variables of its own, known to influence survival, such as quality of life outside the hospital measured against such factors as loneliness, reports by social workers on mental state and so forth. All risks, it would be argued, should be considered, to make the prognosis absolutely fair. Once the principle of mercy killing is established, it only takes a series of nibbling amendments to take it well outside the moral law. Medical progress has now poised us at the top of a slippery slope: once we start to descend we may never be able to clamber back up. Legalised euthanasia, however hedged about with counsels about freedom of choice, would mean state-sanctioned killing. In the end, it may lead to state killing. End of article. *********************************************************************** I am researching the use of computers in medical education with the aim of establishing how medical curricula have been changed to take advantage of their capabilities. It would interest me to learn of any medical informatics courses that address the general issue identified in the quoted article -- ie, how computers can aid in medical decisions, and when should they be relied upon. I would also be interested to recieve more information on the Apache system and others of a similar ilk. Please address any responses directly to me at Liverpool University. Alexander J. Birchall Department of Computer Science Liverpool University Liverpool L69 3BX England email: alex@compsci.liverpool.ac.uk ================