Computers versus paper
How should
audit be undertaken?
Traditionally we have been told that a simple
paper based exercise is the best and most effective way of performing an audit.
This is only true if the goals of the audit are limited or a pro forma is
created prior to commencing the audit and completed for each patient.
A
‘common’ type of audit has been a review of clinical records. This is a
simple method of ensuring that the standard record keeping is adequate from a
medical and medicolegal standpoint. A random selection of case records from an
identified group is gathered and criteria are defined to determine the adequacy
of the notes; the next stage should be to recommend changes and to re-audit.
This type of audit does not require any detailed methodology; it requires that a
record of the defects and recommendations is kept. This type of audit method has
value in educating new housemen about the need to keep adequate records and so
should probably be used early on in their employment in the firm.
Another
simple audit is to choose a particular disease or treatment (surgical operation,
for example). Data about the patients in this group are then used either to
review aspects of process (stay length, time to operation, etc.) or to review
the actual patient outcome — alive or dead, cured or recurrence, etc. This
methodology may offer reasonable insight into the pathological process under
consideration. However, a number of possible sources of error of interpretation
and difficulty must be highlighted. These include the following.
•
Patients are identified from an unreliable source resulting in either
patients who did not have the condition or, more often, missing varying numbers
of patients who did have the condition. Patients may also be lost at follow-up.
•
Inappropriate statistical tests may be used or the correct test used
wrongly.
•
Historical controls may be used for comparison which are not comparable.
If
paper-based audits are to be undertaken a few simple rules should be followed.
1.
Develop a simple and comprehensible pro forma which will record the
information upon which the audit is based. Avoid proformas that require a lot
of text entry. Wherever possible use a series of ‘tick’ boxes with either
single or multiple choices. This type of pro forma can be completed speedily and
accurately as the person placing the ticks is presented with all of the answers.
The disadvantage of this type of form is that it can look complex and large,
especially if there is a lot of choices for each field.
2.
Undertake the audit prospectively. Retrospective data are helpful as a
basis of a pilot study in the sense that the potential numbers under study
might be predicted from a review of the admissions, etc., over the last months
or years. There is usually very little sense in auditing a rare event as it will
3.
Choose outcome measures that can be readily assessed.
4.
Choose a method of assessment that will be independent of bias that might
be introduced by the initiator of the audit. Patients will often tell doctors
what they wish to hear and therefore either an independent person needs to
assess the outcome or a method that is neutral in its presentation needs to be
used (e.g. a patient questionnaire).
Computer-based
audits
Using modern information technology is very
attractive and can be efficient and effective. Alternatively, there may be a
tendency to develop such complexity that data will not be entered.
The
main advantages of using a computer-based system for audits are as follows.
•
Large numbers of entries can be analysed easily.
•
In the future electronic patient records will automatically record
information concerning a clinical event that will allow us to perform effective
audits.
The
main disadvantages are as follows.
•
Knowledge of the construction and operation of a database is required.
•
Commercial databases are often very expensive but increasingly online
national databases are being created that make use of a central server.
•
Appropriate fields for the database need to be created before data entry.
If fields are added after data are captured it can be extremely difficult to
complete the new fields.
•
The level of complexity of the database needs to be determined at the
outset and will be dependent upon the uses to which the database will be put.
•
Entry of data will need to be checked (validated) against an independent
source to ensure accuracy. It is usual to select randomly 10 per cent of cases
and compare the computer
records
with the original case records held by the hospital.
•
Methods of highlighting inappropriate entries into particular fields and
the completion of obligatory fields will need to be considered.
Coding
Traditionally all patient diagnoses and
treatments have been coded, i.e. given an alphanumeric code for the purposes of
storing the diagnosis on computer. The International Classification of
Diseases (lCD) coding system has been accepted as the international method of
coding diagnoses and treatments. The codes (lCD version 10) are available on
CD-ROM and are a fairly comprehensive system. Using this coding system allows
easy international comparison of some aspects of data
collection, especially disease incidence,
geographic distribution of disease, etc. The Department of Health in the UK
has invested in an alternative coding system, originated by Dr James Read. In
the latest form of this coding system there is automatic mapping to lCD-b. The
development of the Read Codes is attempting to unify an alphanumeric coding
system with the use of familiar clinical terms in real language. With modern
computers there is a real question as to whether coding is needed at all. Its
main role seems to be in defining terms so that what one surgeon defines as an
infection is the same as another surgeon. Without this audit data cannot be
compared or combined and much of their power is lost.