Evaluating Proficiency Testing Results

Previous

Download and Print Forms


Proficiency Testing Action Form


LABORATORY NAME: ____________________________________________________________________________

Section:        __________________________________________________________________________________

Completed by: ___________________________________________________________________________________

Core lab Manager / Department Supervisor: ____________________________________________________________


Problem: _______________________________________________________________________________________

Attach documents as needed


Corrective Action/Preventive Action: _______________________________________________________________

Attach documents as needed


Reviewed by:

Laboratory Manager ____________________________________________________________ Date: ______________


Medical Director _______________________________________________________________ Date: ______________


------------------------------------------------------------------------------------------------------------------------------------------------------------------


PROFICIENCY TEST CORRECTIVE ACTION CHECKLIST FORM


Laboratory Name: ___________________________________________________________         CLlA #: __________________

Testing Event: ______________________________________________________________            Year: ____________________

Proficiency Testing Module: ___________________________________________________        Analyte:  __________________


Date PT Sample Rcvd __/__/___                Test Date: __/__/___                Report Date: __/__/___


Sample #:___________________        Reported Results: ___________________        Expected Range: ___________

Expected Results: ____________        Repeat Analysis Result _______________        (Original or new specimen)


Sample #:___________________        Reported Results: ___________________        Expected Range: ___________

Expected Results: ____________        Repeat Analysis Result _______________        (Original or new specimen)


Sample #:___________________        Reported Results: ___________________        Expected Range: ___________

Expected Results: ____________        Repeat Analysis Result _______________        (Original or new specimen)


Sample #:___________________        Reported Results: ___________________        Expected Range: ___________

Expected Results: ____________        Repeat Analysis Result _______________        (Original or new specimen)


Sample #:___________________        Reported Results: ___________________        Expected Range: ___________

Expected Results: ____________        Repeat Analysis Result _______________        (Original or new specimen)


I.  Does this failure represent unsatisfactory performance for this analyte, specialty, or subspecialty?                Y / N

2. Does this failure represent unsuccessful performance for this analyte, specialty, or subspecialty?                Y / N

       (Unsatisfactory performance for two events in a row or two out of three consecutive testing events:        

------------------------------------------------------------------------------------------------------------------------------------------------------------------


PT Failure Classification:                Submitted Late                Lack of Consensus        Failure to Submit

Clerical Error                Equipment Error                Educational Challenge

Trend / Bias                Other

FINDINGS:_____________________________________________________________________________________________________________________________________________________________________________________________________________


CORRECTIVE ACTION: _______________________________________________________________________________________

___________________________________________________________________________________________________________


COULD THIS ERROR AFFECT PATIENT RESULTS?        Y / N

If yes, state course of action:  ___________________________________________________________________________________


[Review process to be modified by each lab to what is appropriate for that lab]


Investigated by: ______________________________________________________________________ Date: ___/___/____

Technical Consultant/Supervisor: ________________________________________________________ Date: ___/___/____

Laboratory Director: ___________________________________________________________________ Date: ___/___/____

INVESTIGATION CHECKLIST

This form is to be used as a guide to assist in investigating, documenting, and correcting proficiency test failure or unacceptable results. Identify the reasons for failure or unacceptable results in proficiency testing and take appropriate corrective measure. Complete Proficiency Testing Corrective Action Form and attach copies of all records reviewed to this form.


  1. SPECIMEN HANDLING
    1. Were proficiency test specimens checked for acceptability when received? (Review notes made at the time proficiency test was received).                                                                                          Y / N / NA
    2. Were the specimens handled properly? (Review instruction for specimen preparation).                        Y / N / NA


  1. CLERICAL ERRORS
    1. Verify correct value was transcribed from instrument printout to report form, or that the correct response was entered from the list of results.                                                                                        Y / N / NA
    2. Verify that decimal point and units of measure were honored on the report form.                                Y / N / NA
    3. Verify that the correct code from the instrument or reagent list was entered on the report form.        Y / N / NA
    4. Verify that the correct testing method information was provided.                                                        Y / N / NA
    5. Verify that any calculations were performed correctly. (even  if automated calculation)                        Y / N / NA
    6. Check summary report to verify value on report form was honored by the PT service.                        Y / N / NA


  1. QUALITY CONTROL
    1. Were quality control materials within the acceptable range on the date of PT testing?

(Verify the quality control acceptable range in use.)                                                                        Y / N / NA

    1. Any evidence of trends or shifts in the periods just before and just after PT was tested?                        Y / N / NA


  1. CALIBRATION
    1. What was the date of the last calibration?                                ___/___/____
    2. How often is calibration to be performed?                                ___/___/____
    3. When was last calibration verification performed?                ___/___/____
    4. Were any calibration problems noted?                                                                                Y / N / NA


  1. INSTRUMENT
    1. Were instrument parameters entered correctly?                                                                Y / N / NA
    2. Was daily maintenance performed on the date of PT testing?                                                Y / N / NA
    3. Was special maintenance performed just prior to PT?                                                        Y / N / NA
    4. Were instrument problems noted when PT was performed?                                                Y / N / NA


  1. REAGENTS
    1. Check reagent / instrument log for notation of recent problems.                                        Y / N / NA
    2. Check reconstitution instructions in package insert versus procedure -any changes?                Y / N / NA
    3. Verify that open stability of reagent was not exceeded by reviewing procedure with testing personnel.

        Y / N / NA

  1. TESTING PERSONNEL
    1. Date of last competency assessment for testing personnel.        ___/___/____
    2. Review assay procedure and proficiency test sample preparation instructions with testing personnel to ensure that instructions were followed.                                                                                        Y / N / NA
    3. Review with testing personnel how samples were loaded to rule out misidentification or transposition of samples.                                                                                                                Y / N / NA
    4. Was retraining of testing personnel required and if so is this completed?                                Y / N / NA


  1. PROCEDURE
    1. Review procedure versus manufacturer’s most current recommendation for any changes.        Y / N / NA
    2. If retained frozen or refrigerated specimens were retested, were the results the same as those reported?

       Y / N / NA

    1. Call instrument or reagent manufacturer for input if cause is not readily identified.                Y / N / NA


  1. INTERPRETATION ERRORS
    1. Was PT challenge beyond the scope and extent of the testing routinely performed in your lab?                Y / N / NA
    2. Has summary report been reviewed for an explanation of key features of the element presented in the photomicrographs and/or pictures?                                                                                        Y / N / NA
    3. Have textbook references been consulted for additional information?                                                Y / N / NA
    4. (Microbiology) Compare the test characteristics found in your laboratory with the characteristics of the

correct identification. Review your results and procedure for the key to distinguish the correct identification

from the incorrect identification.                                                                                                Y / N / NA



Evaluating Proficiency Testing Results


Suggested Process

The role of proficiency testing (PT) has traditionally been one of an external quality assurance check. However, since successful PT performance has become an assessment tool for determining regulatory compliance, effectively evaluating your PT results is imperative.


This sheet suggests a process to follow when evaluating your PT results. Feel free to incorporate this outline into your policy/procedure manual and use it to evaluate previous PT reports or when you get your next set of PT results. If you aren't already preserving your PT specimens, then consider retaining them for use in this evaluation process.


A Beneficial Process for Evaluating Your PT Results

Once you receive your PT results, review the CMS summary page to determine if all regulated analytes were scored for CLIA (regulatory) purposes. If you performed PT on a regulated analyte and this analyte does not appear on the summary page contact AAB-PTS.


Check to be sure the CLIA number for the lab is included and/or correct on the report, along with the name and identifier of any other regulatory or accrediting body (I.e., CMS/State/COLA) that should receive copies of your PT report. If the CLIA number and/or regulatory information are lacking or incorrect your regulatory or accrediting agency will have problems receiving and monitoring your PT enrollment and scores. Notify AAB-PTS of any corrections to this information.


Review your scores for the individual analytes and review the overall specialty scores. The criterion for satisfactory performance is a minimum score of 80% for all analytes (except a minimum score of 100% for ABO/Rh and compatibility testing). For analytes in the same specialty, the scores are averaged to obtain the overall specialty score.


"Unsatisfactory" PT performance occurs when there is a failure in one event. PT performance is "unsuccessful” if there are two consecutive PT event failures or two out of three PT event failures. If repeated analyte / specialty scores indicate unsuccessful PT performance, then the lab is at risk of losing its ability to continue to test the analyte and/or specialty. After completing this initial review, continue with the more extensive review that follows:


  1. All results were passing; you should:
    1. Review the report.
      1. Are three of five results for an analyte outside +/-1.5 Standard Deviations, if this information is provided?
      2. Are three results for an analyte outside +/-50% of mean?
      3. Do the five results for the analyte range from -50 to +50% of mean?
      4. Did any analytes receive an automatic 100% because they could not be graded?
    2. If conditions 1, 2, or 3 exist, then take corrective action since it identifies gradual long term trends and indicates test instability.
    3. If condition 4 exists, perform a self evaluation, since the score does not reflect actual laboratory performance.
    4. If none of these conditions exist, document this review.

[Stop review here.]


  1. If PT results for any analytes are unsatisfactory:
    1. Check your original documentation for discrepancies. Look for:
      1. Clerical errors (transcription, transposition, method coding).
      2. PT program errors.

B.        Check testing records for technical processing errors. Look for:

1.        Misidentification of specimen.

2.        Misinterpretation of results.

3.        Results mistakenly reported outside the reportable range or when QC was out of range.

C.        If any of the above appear to be the reason for the PT problems:

1.        Document the causes and the corrective action taken to prevent them from happening in the future.

[Stop review here.]


III.        If the reason for the problems is still not apparent, then evaluate the test systems affected.

A.        Expand the scope of the inquiry by asking:

1.        Is the problem affecting more than one test on an instrument?

a. If yes, expect an instrument-related problem.

2.        Is the problem affecting only tests results in a certain range, e.g., only specimens with high values are affected?

a. If yes, this could be due to a linearity/calibration problem.

3.  Are several tests affected from the same PT specimen?

a. If yes, it could be a problem of PT specimen integrity or reconstitution.

B.        Evaluate status of the affected tests at the time of initial testing and determine:

1.   Has maintenance been performed appropriately?

2.   Are controls in range, or starting to trend or shift?

3.   When was the last calibration?

4.   Is temperature a factor?

5.   Are all reagents or controls in date?

C.        Retest PT specimens retained specifically for this purpose. Serum specimens may be frozen; however, check to determine the time period your PT program's hematology specimens will be stable when refrigerated (they cannot be frozen). If the results in question are now in range and:

1.  One test or specimen was affected, it is termed "random analytical error" that may have been due to:

a. Aliquot evaporation.

b. Pipetting error/dilution error

c. Instrument instability/power surge.

2.        Two or more poor results for the same test were biased in the same direction, it is referred to as short-term systematic analytical error" that may have been due to:

a. Improper instrument maintenance.

b. Reagent deterioration.

c. Improper calibration.

3.        If all of the PT problems were explained by the above, then check for possible effects on patient results since the PT specimens were done. If the effect could have been clinically significant, then document appropriate corrective action. Take steps to prevent the problems from recurring.

[Stop review here.]


IV.        If the results of the retest are NOT in range, obtain a new sample of the PT material in question from your PT program and test it Availability of these specimens varies greatly. If they aren't available, then consider performing split specimen testing on several patient specimens instead.

A        If the new specimens are in range, then the problem could have been due to:

1.        Problems with the PT material specimen itself, such as:

a. Bacterial/fungal contamination.

b. Delay or temperature damage in shipment.

c. Hemolysis of specimen.

d. Evaporation of the specimen.

e. Reconstitution error or delay in testing.

B.        Document the cause of the errors and the corrective action taken to prevent future problems.

[Stop review here.]


V.        If the results of these newly obtained specimens are out of range as well, then it's most likely due to a Iong-term systematic error."

A.        Examples of some of these problems and their solutions are:

1.  Miscalibration -recalibrate the instrument.

2.  Repetitive procedural error -reread procedure / retrain staff.

3.  Infrequent performance of the test-retrain staff or consider discontinuing the test.

4.  Major instrument maintenance problem··call for service.

5.  Matrix effect /incompatibility with your method -call AAB-PTS.

B.        If the problems are corrected by any of the above reasons, then check the effect of the problem on patient results since the PT was originally performed. If the effect was clinically significant, then take appropriate corrective action. Document the corrective action taken to prevent them from happening again.


VI.        Perform a scheduled QA follow-up review of the effectiveness of all corrective actions taken to prevent future PT problems. Document this review.

[Stop review here.]


Proficiency Testing is a well-justified laboratory expense. Taking the time to evaluate the results according to the above outline will aid in your efforts to attain successful proficiency testing results, as well as produce quality laboratory test results.