Computers versus paper

How should audit be undertaken?

Traditionally we have been told that a simple paper based exercise is the best and most effective way of performing an audit. This is only true if the goals of the audit are limited or a pro forma is created prior to commencing the audit and completed for each patient.

A ‘common’ type of audit has been a review of clinical records. This is a simple method of ensuring that the stan­dard record keeping is adequate from a medical and medicolegal standpoint. A random selection of case records from an identified group is gathered and criteria are defined to determine the adequacy of the notes; the next stage should be to recommend changes and to re-audit. This type of audit does not require any detailed methodology; it requires that a record of the defects and recommendations is kept. This type of audit method has value in educating new housemen about the need to keep adequate records and so should probably be used early on in their employment in the firm.

Another simple audit is to choose a particular disease or treatment (surgical operation, for example). Data about the patients in this group are then used either to review aspects of process (stay length, time to operation, etc.) or to review the actual patient outcome — alive or dead, cured or recurrence, etc. This methodology may offer reasonable insight into the pathological process under consideration. However, a number of possible sources of error of inter­pretation and difficulty must be highlighted. These include the following.

  Patients are identified from an unreliable source resulting in either patients who did not have the condition or, more often, missing varying numbers of patients who did have the condition. Patients may also be lost at follow-up.

       Inappropriate statistical tests may be used or the correct test used wrongly.

  Historical controls may be used for comparison which are not comparable.

If paper-based audits are to be undertaken a few simple rules should be followed.  

1.       Develop a simple and comprehensible pro forma which will record the information upon which the audit is based. Avoid proformas that require a lot of text entry. Wherever possible use a series of ‘tick’ boxes with either single or multiple choices. This type of pro forma can be completed speedily and accurately as the person placing the ticks is presented with all of the answers. The disadvantage of this type of form is that it can look complex and large, especially if there is a lot of choices for each field.

2.       Undertake the audit prospectively. Retrospective data are helpful as a basis of a pilot study in the sense that the poten­tial numbers under study might be predicted from a review of the admissions, etc., over the last months or years. There is usually very little sense in auditing a rare event as it will require many years of data acquisition in order to get a study of sufficient size to make any worthwhile comment. The retrospective data may also give a clue to the types of outcome measures that should be employed.

3. Choose outcome measures that can be readily assessed.

4. Choose a method of assessment that will be independent of bias that might be introduced by the initiator of the audit. Patients will often tell doctors what they wish to hear and therefore either an independent person needs to assess the outcome or a method that is neutral in its presentation needs to be used (e.g. a patient questionnaire).

Computer-based audits

Using modern information technology is very attractive and can be efficient and effective. Alternatively, there may be a tendency to develop such complexity that data will not be entered.

The main advantages of using a computer-based system for audits are as follows.

  Large numbers of entries can be analysed easily.

  In the future electronic patient records will automatically record information concerning a clinical event that will allow us to perform effective audits.

The main disadvantages are as follows.

       Knowledge of the construction and operation of a data­base is required.

       Commercial databases are often very expensive but increasingly online national databases are being created that make use of a central server.

       Appropriate fields for the database need to be created before data entry. If fields are added after data are captured it can be extremely difficult to complete the new fields.

  The level of complexity of the database needs to be deter­mined at the outset and will be dependent upon the uses to which the database will be put.

  Entry of data will need to be checked (validated) against an independent source to ensure accuracy. It is usual to select randomly 10 per cent of cases and compare the computer

records with the original case records held by the hospital.

  Methods of highlighting inappropriate entries into particular fields and the completion of obligatory fields will need to be considered.

Coding

Traditionally all patient diagnoses and treatments have been coded, i.e. given an alphanumeric code for the purposes of storing the diagnosis on computer. The International Classification of Diseases (lCD) coding system has been accepted as the international method of coding diagnoses and treatments. The codes (lCD version 10) are available on CD-ROM and are a fairly comprehensive system. Using this coding system allows easy international comparison of some aspects of data

collection, especially disease incidence, geographic distribu­tion of disease, etc. The Department of Health in the UK has invested in an alternative coding system, originated by Dr James Read. In the latest form of this coding system there is automatic mapping to lCD-b. The development of the Read Codes is attempting to unify an alphanumeric coding system with the use of familiar clinical terms in real language. With modern computers there is a real question as to whether coding is needed at all. Its main role seems to be in defining terms so that what one surgeon defines as an infection is the same as another surgeon. Without this audit data cannot be compared or combined and much of their power is lost.