Shah S1, Saini R2, Singh SB3, Aggarwal O4, Goel AK5
1Dr Seema Shah Goel, Assistant Professor, Biochemistry, Vardhman Mahaveer Medical College & Safdarjung Hospital, New Delhi. 2Dr Reetika Saini, Senior Resident, Biochemistry, Vardhman Mahaveer Medical College & Safdarjung Hospital, New Delhi, 3Dr Sangeeta B. Singh, Associate Professor , Biochemistry,Shaheed Hassan Khan Mewati Govt Medical College, Nalhar,4Dr Onam Aggarwal, Junior Resident , Biochemistry, Maulana Azad Medical College, New Delhi,5Dr Anil Kumar Goel, Associate Professor, Paediatrics, Shaheed Hassan Khan Mewati Govt Medical College, Nalhar, India
Address for correspondence: Dr Seema Shah Goel, sshah.goel@gmail.com
Abstract
Quality control in healthcare system is still less understood because of the relative complexity in choosing an appropriate Westgard rule .Six Sigma methodology is a manufacturing strategy first pioneered by Motorola Company in 1980s, with the goal of decreasing the defect rates in production. It has improved the production efficiency of different industries. To achieve the similar high quality and near zero defect rates in healthcare system, six sigma metrics is being used in many clinical laboratories and diagnostic industry. Six sigma metrics is used in combination with total allowable error (CLIA ’88 proficiency testing criteria), method imprecision and bias. The goal is to attain the highest possible sigma scale within the acceptable limits of total allowable error. This article reviews the different Westgard control rules and their implications in pointing towards a particular error, as well as basic principles of Six Sigma methodology & their practical utility in the clinical laboratory.
Keywords: Clinical laboratories; Quality control; Sigma Metrics; Six Sigma; Westgard rules.
“Total quality management is a journey, not a destination.” Thomas Berry [1]
Manuscript received: 03rd Apr 2014, Reviewed: 05th Apr 2014
Author Corrected: 13th Apr 2014, Accepted for Publication: 21st Apr 2014
Introduction
The concept of quality management in healthcare system remains an evergreen discussion. A study by the Institute of Medicine reports annual preventable death of 44,000- 98,000 in USA alone [2].Among healthcare services, clinical laboratory services remain important as around 70% of the patient related decision are based on the clinical laboratory results[3]. The total testing procedure is divided into preanalytical, analytical and postanalytical phase. Estimated error rate in the three phases are 30-75% for preanalytical, 4-30% for analytical, and 9-55% for postanalytical phase [4]. Hence stringent quality control in clinical laboratory will improve patient care.
Principle and concept of quality management
“The main objective of internal quality control is to ensure day to day consistency.” (WHO 1981)
Quality is defined as conformance to the requirements of the end users [5]. It is assessed in terms of accuracy (closeness to the true value), precision (reproducibility of a test result), sensitivity and specificity .The three purposes of quality control are monitoring the accuracy and precision of the analytical processes and detection of immediate error. In simpler terms it is attainment of the intended quality of results. Implementing quality control is a continuous dynamic procedure so that patients test results produced by the laboratory are reliable and contribute to patient care.
The basic workflow in designing an quality control system involves five continuous steps of establishing goals (Quality Planning), planning of laboratory policies (Quality Process), implementation of the plan using specific procedures (Quality Control), assessing the effectiveness of the plan (Quality Assurance), and finally taking appropriate steps to achieve desired outcome (Quality Improvement) as suggested by Westgard et al as five steps of total quality management (TQM) [6].
What is Quality Control?
Quality control in medical laboratory is a statistical process to monitor and evaluate the analytical system. Its results are used to validate whether the system is working within the predefined conditions and to know whether the patients tests results are reliable or not. There are basically two types of schemes – internal quality control (IQC) and external quality control (EQC). IQC ensures a continuous monitoring of the analytical system, so as to check whether the results are reliable enough to be released. Control charts like Levey Jennings chart and Westgard’s rules are applied on daily QC data. External quality control involves analysing and reporting of control samples supplied by an external agency, at a predefined time interval of a fortnight or a month. The external supplier of the QC sample studies the results of all the participating laboratories and then provides feedback to all [7].The participating laboratories are divided in groups according to the analytical method and instruments used. This is followed by calculation of mean and standard deviation for a particular group and is referred to as consensus mean and standard deviation. Individual laboratory’s performance is judged by comparing the mean, standard deviation and CV ( coefficient of variation) with consensus mean ,standard deviation and CV.
QUALITY CONTROL PLANNING: Choosing a specific QC procedure is done with the aim of minimizing the false rejection and maximizing the error detection. CLSI (formerly NCCLS, National Committee For Clinical Laboratory Standards ) has recommended following essential steps for setting up a QC system [8].
1. Defining quality requirement of the test
2. Evaluating tests’ performance as method precision and bias
3. Identifying the possible QC procedures with regard to rules to apply, candidate control material, levels and numbers of control samples, time of run for the control sample and frequency of QC running
4. Prediction of performance of QC procedure
5. Setting up goals based on required quality
6. Selecting an appropriate QC procedure
Defining the quality requirements involves establishing analytical goals as per the total allowable error (error within acceptable limits), which is done by a hierarchy of approaches published as list of models for quality specifications in the Scandinavian Journal of Clinical and Laboratory Investigation [9].When feasible and appropriate , models higher in the list are preferred over those lower in the list [9].
Table/Fig.1: Shows TEa for some of the common biochemical and other investigations as per CLIA recommendation
|
Analyte
or test |
CLIA
criteria for Acceptable performance |
|
Blood
Glucose |
Target
value ±10% or 6mg/dl (greater) |
|
Bilirubin,
Total |
Target
value ±20% or 0.04mg/dl (greater) |
|
ALT |
±20% |
|
AST |
±20% |
|
ALP |
±30% |
|
Total
Protein Serum |
±10% |
|
Albumin |
±10% |
|
Cholesterol
Total |
±10% |
|
Triglyceride |
±25% |
|
Cholesterol,
HDL |
±30% |
|
Calcium
Total |
Target
value ±1.0mg/dl |
|
Sodium |
Target
value ±4 mmol/l |
|
Potassium |
Target
value ±0.5 mmol/l |
|
LDH |
±20% |
|
Amylase |
±30% |
|
Creatine
Kinase |
±30% |
|
Creatine
Kinase , MB |
Target
value ± 3SD or presence/absence |
|
Blood
urea |
Target
value ±9 % or 2 mg% (greater ) |
|
Creatinine |
Target
value ± 15% or ±
0.3 mg/dl/(greater) |
|
Uric
acid |
±17% |
|
TSH |
Target
value ± 3 SD |
|
Thyroxine
Total |
Target
value ± 20% or ±
0.1.0mcg/dL(greater) |
|
Triiodothyronine
Total |
Target
value ± 3 SD |
|
Alcohol,carbamazepine,gentamycine,Phenytoin,
valprotate , theophylline,Quinidine, tobramycin |
Target
value ± 25 |
|
Alpha-1
antitrypsin |
Target
value ± 3 SD |
|
Alpha-fetoprotein |
Target
value ± 3 SD |
|
Anti-Human
Immunodeficiency virus, Hepatitis (HBsAg, anti-HBc, HBeAg) |
Reaction
or nonreactive |
|
Complement
C3 |
Target
value ± 3 SD |
|
Complement
C4 |
Target
value ± 3 SD |
|
IgA |
Target
value ± 3 SD |
|
IgE |
Target
value ± 3 SD |
However generally the criteria lay down by CLIA (Clinical Laboratories Improvement Act) ‘88 proficiency limits [10] or biological variation values as published in Scandinavian Journal [11] are used to set up the maximum acceptable error for individual tests. For example as per CLIA recommendation a TEa (total allowable error) of ±10% or ±6 mg/dl (greater of the two) is accepted for the blood glucose determination whereas for blood pH it is ±0.04 only. Table /Fig.1 enlists TEa for some common biochemical parameters [10]. Table /fig. 2 shows Biological Variation Values for common biochemical investigations [11]. This table provides desirable analytical quality specifications for imprecision, bias and total error based upon biological variation
Table 2: Shows Biological Variation Values for common biochemical investigations
|
Analyte |
Biological
Variation |
Desirable
Specifications |
|||
|
|
CVw |
CVb |
Imp
(%) |
Bias
(%) |
TE
a (%) |
|
Amylase |
8.7 |
28.3 |
4.4 |
7.4 |
14.6 |
|
ALT |
19.40 |
41.6 |
9.7 |
11.48 |
27.48 |
|
Albumin |
3.2 |
4.75 |
1.6 |
1.43 |
4.07 |
|
ALP |
6.45 |
26.1 |
3.23 |
6.72 |
12.04 |
|
AST |
12.3 |
23.1 |
6.15 |
6.54 |
16.69 |
|
Bilirubin
total |
21.8 |
28.4 |
10.90 |
8.95 |
26.94 |
|
Bilirubin
conjugated |
36.8 |
43.2 |
18.4 |
14.2 |
44.5 |
|
Calcium |
2.1 |
2.5 |
1.05 |
0.82 |
2.55 |
|
Cholesterol |
5.95 |
15.3 |
2.98 |
4.1 |
9.01 |
|
Creatine
kinase (CK) |
22.8 |
40.0 |
11.4 |
11.5 |
30.3 |
|
CK MB, % |
6.9 |
48.2 |
3.5 |
10.8 |
16.5 |
|
CK MB, activity |
19.7 |
24.3 |
9.9 |
7.8 |
24.1 |
|
CK
MB, mass |
18.4 |
61.2 |
9.2 |
14.88 |
30.06 |
|
Urea |
12.1 |
18.7 |
6.05 |
5.57 |
15.55 |
|
Creatinine |
5.95 |
14.7 |
2.98 |
3.96 |
8.87 |
|
Urate |
8.6 |
17.5 |
4.3 |
4.87 |
11.97 |
|
Glucose |
5.6 |
7.5 |
2.8 |
2.34 |
6.96 |
|
HDL
cholesterol |
7.3 |
21.2 |
3.65 |
5.61 |
11.63 |
|
Lactate
dehydrogenase (LDH) |
8.6 |
14.7 |
4.3 |
4.3 |
11.4 |
|
LDL
Cholesterol |
7.8 |
20.4 |
3.9 |
5.46 |
11.9 |
|
pCO2 |
4.8 |
5.3 |
2.4 |
1.8 |
5.7 |
|
pH
[H+] |
3.5 |
2.0 |
1.8 |
1.0 |
3.9 |
|
Phosphate |
8.15 |
10.8 |
4.08 |
3.38 |
10.11 |
|
Potassium |
4.6 |
5.6 |
2.3 |
1.81 |
5.61 |
|
Protein |
2.75 |
4.7 |
1.38 |
1.36 |
3.63 |
|
Sodium |
0.6 |
0.7 |
0.3 |
0.23 |
0.73 |
|
TSH |
19.3 |
24.6 |
9.7 |
7.8 |
23.7 |
|
Thyroxine
(T4) |
4.9 |
10.9 |
2.5 |
3.0 |
7.0 |
|
Thyroxine,
free (FT4) |
5.7 |
12.1 |
2.9 |
3.3 |
8.0 |
|
Triglyceride |
19.9 |
32.7 |
9.95 |
9.57 |
25.99 |
|
Triiodothyronine
(T3) |
6.9 |
12.3 |
3.45 |
3.53 |
9.22 |
|
Triiodothyronine,
free (FT3) |
7.9 |
17.6 |
4.0 |
4.8 |
11.3 |
|
Parathyroid
hormone (PTH) |
25.9 |
23.8 |
13.0 |
8.8 |
30.2 |
Evaluating test performance in terms of method precision and bias are done as per the following equations:
Bias is calculated from the EQC data using the following formula:
Bias =(mean of all laboratories using same instrument and method −lab’s mean) /mean of all laboratories using same instrument and method ) X 100 %
CV is calculated from the calculated laboratory mean and calculated standard deviation procured from the internal quality control data over preceding months
CV%= (Standard deviation /Laboratory mean) X 100 %
CV is used to compare precision, to check manufacturer’s claims, peer group QC report and as a part of internal quality control. A test with high standard deviation means poor precision, greater instability and high random error in the laboratory.
Prediction of performance of QC procedure is done using operational process specifications (OPSpecs) charts available in the Westgard website at www.westgard.com. These charts describe operational limits for imprecision and inaccuracy for specific QC procedure. Using the TEa, precision and accuracy of an analyte, optimal Westgard rule can be selected using OPSpecs charts.
The goals of internal QC is to catch all significant error ( probability of error detection) with minimum false rejection( probability of false rejection).
Selecting an appropriate QC procedure is done with sigma metrics. The appropriate IQC procedure is one having a 90% chance of detecting medically important errors (Ped≥0.90) and a 5% chance of false rejection (Pfr≤0.05), preferably 1 % or less.
Candidate control material:
Ideal QC material should have same material matrix as the patients sample, long stability, ready to use with minimum operators handling. There should be little or zero vial to vial variation for a particular lot and it should cover the clinically important range for analyte concentration. It should be tested in the same manner as patient’s sample. It may further be of dependent or independent type. Independent control or third party control provides an independent assessment of the testing method and also detects changes associated with lot variation of reagent or calibrator, which might be missed by a dependent control. A dependent control is manufactured by the same company supplying the analytical instruments or reagents. A control sample by the same manufacturer as that of calibrator, will shift in the same direction as calibrator and fails to detect any error and correct values are obtained for the control sample falsely which fails to detect error in the analytical system. Similarly package QC sample (supplied along with reagent vial) show reagent lot specific target values and hence reagent lot- to- lot variations will be missed. So independent or third party QCs with peer group comparison data should be used to see if other users are also experiencing the similar reagent lot related shift in QC values.
However if commercial third party control is not available, pooled patients sample may be used and result matched with previous result. The number of levels of QC samples , frequency of running the QC and time of run varies from laboratory to laboratory, according to the sample load for a particular analyte, number of analytical run per day, shift change of the laboratory staff and of course as per the laid down criteria of different accreditation bodies(12). As per NABL guidelines [13] frequency and number of QC samples to be run as per the sample load is as follows
< 25 per day - one level QC once a day.
25-75 per day - two level QCs once a day.
>75 per day - two level QCs at least twice a day
analytes.
Identifying candidate IQC procedure:
Several statistical process control rules as proposed by Dr. James O.Westgard in 1981are used to assay quality control performance [14]. Rules are expressed as NL where N represents the number of control observations to be evaluated and L represents the statistical limit for evaluating the control observations. Thus 13s represents a control rule which is violated when one control observation exceeds the ±3s control limits. Six basic rules are used in various combinations as a multirule procedure , where some rules detect random error while others are sensitive to systematic error as presented in table /figure 4. These rules are used with Levey-Jennings (LJ) chart [15]. The LJ charts are prepared by calculating the mean and S.D of the control material by analyzing it for a minimum of 20 times over a 30 day time period. Concentration of the analyte is plotted on theY axis and time as day on X axis. Horizontal lines are drawn at mean, mean ±1 SD , ±2SD and ±3 SD. Each day’s data is plotted on the charts [14].
Table /Figure 3: Levey Jennings Chart showing mean as grey line at 252.32, purple line at mean ±1 SD and orange line at mean ±2 SD
|
Rules |
What
does it mean |
Type
of error |
Alarm
produced |
|
12s |
one level
of control is
beyond ± 2 SD |
Random
or systematic |
Warning
rule |
|
13s |
one level
of control is
beyond ± 3 SD |
Random
or beginning of systematic error |
Rejection
rule |
|
22s |
two levels
of control are
beyond ± 2 SD on same side of the mean |
Systematic
error |
Rejection
rule |
|
41s |
four consecutive
data in one level of control more than 1 SD on the same side of mean |
Systematic
error |
Not
a rejection rule. Indicates the need to perform instrument
maintenance or reagent calibration |
|
R4S |
Two levels of control show a difference of 4 SD There
is at least a 4SD difference
of value between two control levels within
a single run |
Random
error |
Rejection
rule |
|
10X |
rules are violated when there are 10 control
results on the same side of the mean
regardless of the specific standard deviation that they are located in. |
Systematic
error |
Rejection
rule (
this rule has a lower probability of false rejection than 7x, 8x, or 9x
rule) (this
rule can be applied within a control level or across
control levels indicating systematic bias over a particular range or
over broader analytical range respectively) Indicates the need to perform instrument
maintenance or reagent calibration. |
|
7x
, 8x, 9x, 12x |
These rules are violated when there are 7 or 8
or 9 or 12 control
results on the same side of the mean
regardless of the specific standard deviation that they are located in. |
Systematic
error |
Rejection
rule (
12x has a lower probability of false rejection than 7x, 8x, or 9x or
10x rule) (These rules can be applied within a control level or across
control levels indicating systematic bias over a particular range or
over broader analytical range respectively) |
|
7T |
Seven
consecutive readings of a single level of control show either a strict increasing or strict decreasing pattern |
Systematic
error |
Rejection
Rule Strict
increasing or decreasing pattern means each subsequent point shows
incremental increase or decrease from the previous point |
|
2 of 32s |
two of
three levels of control are
beyond ± 2 SD on same side of the mean |
Systematic
error |
Rejection
rule ( applied when testing three or more levels of control in one run ) |
|
12.5s
, 13.5s
, 14s, 15s |
one level
of control is
beyond ± 2.5, 3.5 , 4 or 5 SD respectively |
Random
sometimes systematic
error |
These
rules are modified rules |
However there is no particular set of rules that is right for all the tests and methods as some methods have better precision than others and hence rules should be selected as per the quality required and the observed performance for the test.
Table/Figure 4: Table showing different Westgard rules and their implications
|
Sigma
metrics |
Defects
per million |
|
1
σ |
317,310 |
|
2
σ |
45,500 |
|
3
σ |
2699 |
|
4
σ |
63 |
|
5
σ |
0.573 |
|
6
σ |
0.002 |
If 1 2s rule is taken as rejection rule, it will result in varying percentage of false rejection as per the number of levels of control used [14]
5% of total analytical run with 1 level of control used
9% of total analytical run with 2 levels of control used
14% of total analytical run with 3 levels of control used
This is because the data show a normal Gaussian distribution with following distribution pattern:
68.3% of data are within ± 1 SD( standard deviation) of the mean
95.5% of data are within ± 2SD of the mean
99.7% of data are within ± 3 SD of the mean
As per NABL, the criteria to accept or reject run are as follows [13]:
With one level QC material reject QC if
a. It is outside 3 SD (13s)
b. Two consecutive values obtained are outside 2 SD on the same side but within 3 SD (22s )
c. Ten consecutive values are above or below the mean, but within 2 SD (10x)
With 2 level QC materials reject QC if
a. Either QC value is outside 3 SD (13s)
b. Both QC values are outside 2 SD on the same side, but within 3 SD (22s)
c. Difference between both QC values is >4 SD i.e. one level QC is > 2 SD and other level QC is <2SD (R 4s).
d. Ten consecutive values of the same level QC are >/< the mean, but within 2 SD (10x).
e. Five consecutive values of one level QC and five consecutive values of other level QC are >/< the mean but within 2 SD (10x) .
Time of run of QC samples can be decided by different strategies with run scheduled at either fixed time interval, random time interval or average interval of 8 hours between 2 consecutive QC run (16).
Number of Quality Control to choose:
Application of Six Sigma (σ) principles provides a scientific basis for designing an appropriate QC strategy for improving the quality control process. Evaluation of laboratory’s performance on six sigma scale was first studied by Nevalainen D et al and Westgard JO in the year 2000 and 2001 respectively [17,18].
The concept of six sigma metrics was pioneered by Motorola company in mid 1980s, with the aim of improving their manufacturing process so that virtually no defective product would be produced .Making a process six sigma compliant essentially implies decreasing the variation from the system so that the standard deviation becomes so small that six numbers of standard deviation fits within the tolerance limits. A process which is six sigma compliant will produce only 3.4 defects per million opportunities even with a 1.5 SD shift in mean value.
For calculating a sigma metric, one needs to find out numbers of defects produced per million opportunities. This can be converted to a sigma metric by comparing with normal Gaussian distribution. As per Gaussian distribution 0.682689 % of the distribution falls within one standard deviation, implying 0.317310 % of the data outside the one standard deviation range. Multiplying by 1,000,000(1million) gives a value of 317,310 defects per 1 million opportunities, which corresponds to tolerance limits of one standard deviation, or a 1 σ process [19]. Table 5 shows defects per million opportunity for a perfectly centred process ( no deviation from mean value) different for sigma metrics 1 – 6. Assuming a 1.5 SD shift in mean will produce a defect of only 3.4 per million opportunities for six sigma compliant process as per table / figure 6 and table / figure7 .
Table / figure 5: Number of defects per million in a particular SD as per the normal Gaussian distribution with perfectly centred mean
|
Sigma
metrics |
Defects
per million |
|
1
σ |
317,310 |
|
2
σ |
45,500 |
|
3
σ |
2699 |
|
4
σ |
63 |
|
5
σ |
0.573 |
|
6
σ |
0.002 |
Table /figure 6: Defects per million in different range of SD assuming a 1.5 SD shift in mean
|
Sigma
metrics |
DPMO |
Percent
defects |
Percentage
yield |
|
1 |
691,462 |
69% |
31% |
|
2 |
308,538 |
31% |
69% |
|
3 |
66,807 |
6.7% |
93.3% |
|
4 |
6,210 |
0.62% |
99.38% |
|
5 |
233 |
0.023% |
99.977% |
|
6 |
3.4 |
0.00034% |
99.99966% |
The advantage of adopting a performance goal of six sigma is that small shifts in mean (classically described as a shift of 1.5 SD from the mean)will still be acceptable within the tolerance limits , without increasing the defect rate.
Table/ Figure 7: Diagrammatic presentation of six sigma assay quality with a 1.5 SD shift in the mean
Choosing a specific Westgard Rule:
The sigma value is a good indicator of the performance charactersitic of the test in question, as it considers both bias and imprecision. Bias refers to the systematic error of the assay and imprecision is the analytical S.D of the method. For analytical processes with known total allowable error and for which analytical performance can be estimated in the form of accuracy (bias) and precision (CV%) , sigma value can be calculated from the following equation:
Sigma = (TEa-bias )/CV
TEa as per CLIA 1988
Bias : systematic error of the assay
CV : analytical SD or variation of the assay
Choosing suitable Westgard QC rules for each analyte is done in five steps :
1. Calculation of analytes CV, bias and TEa
CVa (%): Coefficient of variation is calculated by using standard deviation (SD) and mean of the test as per the equation: CV% = ( S.D / Mean) X 100% This information can be derived from results of laboratory’s internal QC program.
Bias (%): It indicates systematic difference between the result obtained by the laboratory’s test method and that obtained from an accepted reference. The reference can be another test method , a standard or a consensus reference like mean of a group using same method , instrument and reagent i.e peer group comparison. Bias is calculated from results of an external QC program. Bias =(mean of all laboratories using same instrument and method −lab’s mean) /mean of all laboratories using same instrument and method ) X 100 %
TEa (%): It indicates allowable difference from the true value. If the difference between the true concentration of an analyte and the reported concentration in a patients sample exceeds TE a , the result is considered unreliable. A list of TEa is given by CLIA 1988 as in table 1.
2. Calculation of the Sigma value of the test using the formula:
Sigmas =(TEa–bias)/CVa.
3. Selection of the optimal Westgard QC (multi)rule is done using the table showing the hypothetical sigma values for different Westgard rules . If the calculated sigma of an assay is more than the sigma for a particular Westgard rule then that particular rule is suitable for monitoring the error of the test . The table shows Westgard rules and their corresponding hypothetical sigma values along with the number of QC materials and repetitions (measurements ) needed. As the Sigma value of the Westgard rules decreases, the chance of missing an error increases(p error detection decreases). Therefore, the best Westgard rule is the one with a Sigma value closest to, but smaller than, the Sigma value of the test.
Example:
a) For Total Cholesterol:
TEa= 10%
If bias of a lab for it is 3% and CV% at 2 %
Sigma=(TEa−Bias )/ CV%
= (10%-3%)/2%= 3.5 S
So Westgard rule corresponding to sigma of 3.4 is chosen to monitor the performance of cholesterol assay i.e, a multirule of 13s / 2 22s / R 4s / 41s is applied with two levels of control .
b) For Creatinine : TEa = 15%
If bias of a lab is 3% and CV% at 2% for creatinine
Sigma = (TEa−Bias )/ CV%
= (15%-3%)/2%= 6S
So a Westgard rule of 13.5s two levels of control is sufficient to monitor its performance.
Table/ Figure 8: Hypothetical sigma values for different Westgard rules [20]
|
Sigma |
Westgard
rules |
Levels
of control |
Measurements |
p
error detection |
P
false rejection |
|
6.0 |
13.5s |
2 |
1 |
0.98 |
0.01 |
|
5.8 |
13.5s |
2 |
1 |
0.98 |
0.00 |
|
5.6 |
13s |
2 |
1 |
0.97 |
0.00 |
|
5.4 |
13s |
2 |
1 |
0.94 |
0.00 |
|
5.2 |
13s |
2 |
1 |
0.91 |
0.00 |
|
5.0 |
12.5s |
2 |
1 |
0.96 |
0.03 |
|
4.8 |
12.5s |
2 |
1 |
0.93 |
0.03 |
|
4.6 |
13s |
2 |
1 |
0.92 |
0.01 |
|
4.4 |
12.5s |
2 |
1 |
0.96 |
0.04 |
|
4.2 |
12.5s |
2 |
1 |
0.92 |
0.04 |
|
4.0 |
13s /
2 22s /
R 4s /
41s |
2 |
2 |
0.91 |
0.03 |
|
3.8 |
13s /
2 22s /
R 4s /
41s |
2 |
2 |
0.86 |
0.03 |
|
3.6 |
13s /
2 22s /
R 4s /
41s |
2 |
2 |
0.79 |
0.03 |
|
3.4 |
13s /
2 22s /
R 4s /
41s |
2 |
2 |
0.65 |
0.03 |
|
3.2 |
13s /
2 22s /
R 4s /
41s |
3 |
2 |
0.48 |
0.03 |
|
3.0 |
13s /
2 22s /
R 4s /
41s |
3 |
2 |
0.36 |
0.02 |
Source: Schoenmakers CHH,Naus AJM ,Xvermeer HJ , Loon DV and Steen5 G , Practical application of Sigma Metrics QC procedures in clinical chemistry . Clin Chem Lab Med 2011;49(11):1837–1843
Table clearly shows that with increased numbers of levels of QC , a particular Westgard rule produces a lower sigma value. Multirule combinations though more powerful than single rule , does not have a very high sigma value.
Simple guidelines for choosing the Westgard rules and levels of QC as proposed by Westgard are as follows [21]:
≥6σ :– 2 levels of QC per day with a 13.5s greater rule
5σ :– 2or 3 levels of QC per day with a 12.5s or 13s rule
4σ :– 3or 4 levels of QC per day with a 13s / 2 22s / R 4s / 4 1 s rule
3.5σ :– 6 of QC per day with a 13s / 2 22s / R 4s / 4 1 s rule
<3.5 σ :– maximum affordable levels of QC per day with a 13s / 2 22s / R 4s / 4 1 s rule
Another guideline as published by Cooper et al , suggests grouping of tests as per sigma performance and QC strategy as follows :[22]
>6σ (excellent tests) –one QC per day (alternating levels between days) and a 13s rule.
4σ–6σ (suited for purpose) –two levels of QC per day and the 12.5s rule.
3σ–4σ (poor performers) –combination of rules with two levels of QC twice per day.
<3σ (problems) – maximum QC, three levels, three times a day. Preferably testing specimens in duplicate.
To infer, for lower sigma values, more QC samples and more powerful QC rules are recommended. In general, for large sigma value processes (≥6σ) simple QC rules with low false rejection rates are adequate. For intermediate sigma value processes (sigma values between 3.5 and 6) quality goals are met with more elaborate QC strategies . For low sigma values (<3.5 sigma) reducing analytical bias and imprecision is a key to improve the quality.
Troubleshooting QC results:
Whenever QC results exceeds defined Westgard rules , corrective actions taken and should be documented before reporting the patients results. It should be done in five steps as follows:
Inspection of Levey Jennings chart or the rules violated to determine the type of error as different Westgard rules are sensitive to systematic or random error . Westgard rules 13s and R 4s are sensitive to random error and 2 22s / 4 1s/10x are sensitive to systematic error.
Relating the type of error to possible causes
Common factors if multiple tests affected
Relating the error to any recent changes
Taking the corrective steps and documenting the remedy
Relating the type of error to possible causes:
Systematic error affects all samples equally in a proportionate manner .Systematic error may result in shift of the mean of the control either in a gradual manner (Trend) or abruptly (Shift). Causes of systematic error include change in lot of reagent or calibrator, improper preparation of reagent, deterioration of reagent or calibrator, wrong calibrator values, improper volumes of reagent or sample because of pipettor misadjustment, problems in the temperature of the reaction chamber or incubation chamber and also deterioration of the light source . Most commonly improper calibration is the cause of systematic error , which produces error for all patients results.
Random error is a sudden unexpected deviation from the expected result. It may not cause a shift in the mean as error produced occur randomly. It may be caused by bubbles in the reagent or sample line, inadequately mixed reagents , fluctuating electricity supply , fluctuating incubation or reaction chamber temperature, a small clot in the pipettor and of course operator to operator variation in following different steps of reaction procedure ( particularly in semiutoanalyzers settings). Common factors if multiple tests affected: When multiple tests are affected , the common factors between all the affected tests are found like tests using same filter , same lamp , same reaction kinetics ( end point vs. rate ) or similar volume of sample.
Relating the error to any recent changes: If the rules violated indicates a systematic error producing a sudden shift in the mean then reagent, calibrator and maintenance records are studied to find out if the sudden shift has followed any reagent or calibrator replacement or any instruments maintenance .
If the error produced shows systematic trend , it indicates slowly deteriorating reagent or calibrator , deterioration of filter or lamp or slowly deteriorating temperature adjustments over a long time interval.
Random error occurs without a defined pattern or frequency. It can be identified by delta checks or precision checks by doing paired runs (duplicate runs) . A delta check identifies a random error by comparing the current results with a previous result from the same patient and monitoring the difference (delta) between the two results. Delta limits take into account analyzer’s imprecision and systematic error as well as physiological variation of the analyte. Precision checks can be done by running patients’ sample in pairs or duplicates at fixed or random sample interval. Finally the documentation of the corrective steps is done.
Conclusion
Clinical laboratories are in a constant search of methods to solve analytical problems and decrease errors to a negligible level . With the advent of consumer protection acts and legal suits citing medical errors , in the healthcare industry, the need of improving the quality of healthcare services to near zero level has become the need of the hour. Clinical labs remain in a constant effort to increase their work load, decrease error , improve the quality and decrease the cost. To achieve these goals more automated and computerized technology has come up for clinical chemistry , immunochemistry and hematology.The automated results produced ,should closely be monitored so as to ensure that no unreliable reports are generated or despatched to a patient. This is done by running the different levels of control materials ,covering the clinical decisive range , for every analyte. Different Westgard rules are used to monitor the results of internal quality control datas and different Westgard rules have varied sensitivity to random or systematic error and thus violation of different rules points towards a particular error.
The performance of an analytical process is judged by its imprecision and bias as compared to a particular reference method or reference values. Selecting an appropriate Westgard rule is done using Six Sigma principle , along with total allowable error , method imprecision and bias for that particular analyte. Six Sigma methodology has greatly improved the process outcome in various production industries, with a goal of 3.4 DPMO. Recently its use in healthcare system , to improve the quality to a six sigma level has come up. It can be used to choose an optimal Westgard rule and thus can improve the reliability of the results of diagnostic tests.
Funding: Nil, Conflict of interest: Nil
Permission from IRB: Yes
References
1. Berry T. Managing the total quality transformation. New York: McGraw-Hill; 1990.
2. Kohn LT, Corrigan JM, Donaldson MS. To err is human, Building a safer health system. National Academy Press : Washington DC; 2000. [PubMed]
3. Forsman RW. Why is the laboratory an afterthought for managed care organizations. Clin Chem. 1996; 42:813-6. [PubMed]
4. Bonini P, Plebani M, Ceriotti F, Rubboli F. Errors in laboratory medicine. Clin Chem. 2002 May; 48(5):691-8. [PubMed]
5. Burtis AC, Ashwood RE, Bruns ED. Tietz Fundamentals of Clinical Chemistry. 6th ed. 2008.
6. Westgard JO, Burnett RW, Bowers GN. Quality Management Science in Clinical Chemistry: A dynamic framework for continuous improvement of quality. Clin Chem. 1990; 36: 1712-6. [PubMed]
7. Lewis SM. The WHO International External Quality Assessment Scheme for Haematology. Bulletin of WHO . 1988; 66: 283-290. [PubMed]
8. Westgard JO. Internal quality control: planning and implementation strategies. Ann Clin Biochem 2003; 40: 593-611. [PubMed]
9. Kenny D, Fraser CG, Petersen PH, Kallner A. Consensus agreement. Scandinavian Journal of Clinical & Laboratory Investigation .1999; 59 (7) : Jan 1, 1999. [PubMed]
10. CLIA proficiency testing criteria for acceptable analytical performance: Federal Register 1992 Feb 28; 57(40):7002-186. [PubMed]
11. Ricos C, Alvarez V, Cava F, Garcia Lario JV, Hernandez A, Jimenez CV, Minchinela J, Perich C, Simon M. Current databases on biologic variation: pros, cons and progress. Scand J Clin Lab Invest. 1999; 59: 491-500. ( updated 2012). [PubMed]
12. Parvin C, Robbins S. Evaluation of the Performance of Randomized versus Fixed Time Schedules for Quality Control Procedures. Clin Chem 2007; 53: 575-580. [PubMed]
13. NABL 112 : Specific Criteria for Accreditation of Medical Laboratories : 2008 1 Feb; 03:28-45. Amend 2012 16 Oct; 03:28-45.
14. Westgard JO, Barry PL, Hunt MR and Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem.1981; 27: 493-501. [PubMed]
15. Levey S, Jennings ER. The use of control charts in the clinical laboratories. Am J Clin Pathol .1950; 20:1059-1066. [PubMed]
16. Parvin CA. Assessing the impact of the frequency of quality control testing on the quality of reported patient results. Clin Chem 2008; 54: 2049-2054. [PubMed]
17. Nevalainen D, Berte L, Kraft C, Leigh E, Picaso L, Morgan T. Evaluating laboratory performance on quality indicators with the Six Sigma Scale. Arch Path Lab Med 2000; 124:516–519. [PubMed]
18. Westgard JO. Six Sigma Quality Design and Control : Washington, DC; AACC Press, 2001.
20. Carl Garber. Six sigma: its role in the clinical laboratory, from basic to advanced series.Cln’s Lab. 2004.
21. Schoenmakers CHH, Naus AJM, Xvermeer HJ, Loon DV and Steen G. Practical application of sigma metrics QC procedures in clinical chemistry. Clin Chem Lab Med 2011; 49(11):1837–1843. [PubMed]
22. Westgard JO. Six sigma quality design & control.2nd ed. Madison WI: 2006.
23. Cooper G, Jonge N D , Ehrmeyer S, Pacheco J Y, Jansen R , Rico C, Plebani M . Collective opinion paper on findings of the 2010 convocation of experts on laboratory quality .Clin Chem Lab Med .2011;49(5):793–802. [PubMed]
How to cite this article?
Shah S, Saini R, Singh SB, Aggarwal O, Goel AK. Six Sigma Metrics and Quality Control in Clinical Laboratory. Int J Med Res Rev 2014;2(2):140-149.doi:10.17511/ijmrr.2014.i02.013