A Prospective Trial of Interrater and Intrarater Reliability of Wound Measurement Using a Smartphone App Versus the Traditional Ruler

Background. Wound measurements are essential to effective wound management and should be recorded every time a wound is assessed. The move towards evidence-based practice to achieve optimal treatment outcomes, cost effectiveness, and reductions in antibiotic use demands more objective, reliable assessment tools and documentation to allow treatments to be compared through an accurate interpretation of results. Today, smart phones are commonplace and their high-quality photography could allow significantly improved accuracy and reliability of wound measurements. Objective. The purpose of this prospective study is to test the reliability of wound surface area measurements using a smartphone application versus the current gold standard (ruler method).  Both interrater and intrarater reliability were tested. Materials and Methods. Twenty-five volunteers (raters) of varying clinical experience measured a set of 12 photographs composed of various sizes and shapes of wounds. Each rater measured the same set of photos twice on day 1: once using a ruler and then using the smartphone app. Each rater then repeated these 2 measurements on the same set of photos 1 week later. Results. Overall, the ruler method produced wound area measurements that were 30% larger than the app method with the difference tending to be greater for smaller wounds. The resulting intrarater intraclass correlation coefficient (ICC) averaged across all wounds was in the excellent range for both the app and the ruler (ICC, 0.99 for the app; ICC, 0.92 for the ruler), indicating that raters using either method had a high degree of agreement. The interrater reliability using the app was high (ICC, 0.77), showing an excellent level of agreement across raters with the app; however, the ruler was poor (ICC, 0.34), demonstrating a low level of agreement between raters with the ruler method. Conclusions. This study found that the smartphone wound assessment app produced sufficiently consistent results and appears to be superior to the current gold standard of linear measurements with a ruler.

Wound measurements are essential to effective wound management and should be recorded every time a wound is assessed.¹ They are an objective way to evaluate therapies while standardizing wound evaluation and communication among medical professionals and patients.² The shift towards evidence-based practice to achieve optimal treatment outcomes, cost effectiveness, and a reduction in antibiotic use demands more objective, reliable assessment tools and documentation to allow treatments to be compared and contrasted through an accurate interpretation of results.³ A decrease in wound size is one of the strongest predictors of wound healing, with successful therapy having 10% reduction rates in 1 week and 50% over the first 4 weeks.⁴ Both the Wound Healing Society guidelines and the American Professional Wound Care Association protocol require treatments be evaluated every 4 weeks, and alternative treatments should be considered if the wound size has failed to reduce by ≥ 40%.⁵ Apart from guiding treatments, wound measurements are required by insurance companies as well as in litigation.⁶ 

The most commonly used technique measures the wounds perpendicularly (length and width) using a ruler.⁷ Surface area is then calculated by multiplying the 2 measurements, using the formula for a rectangle. Because wounds are more commonly oval or irregularly shaped, this only yields an estimate that has been shown to overestimate the wound size. Other studies have concluded that the ruler method has an error rate of 44%⁸ and interrater reliability error rate of up to 50%.⁹ Due to the inaccuracy of the ruler method, an alternative technique is needed that will provide more accurate measurements with enhanced reliability.

Today, smartphones are commonplace, with more than 87% of professionals owning one.¹⁰ A plethora of medical applications (apps) have become available in recent years. Several focus on wound care, but, to the best of the authors’ knowledge, independent testing of the apps is extremely limited. The high-quality photography available on current smartphones could allow significantly improved accuracy and reliability of wound measurements without the need for any special training. If a wound photo uploaded by a home care nurse or patient could automatically yield more accurate and reliable measurements than those currently taken by trained professionals, it may allow for longer times between clinic visits, quicker identification of stalled wounds, and more efficient and cost-effective wound care overall. If these apps can be trusted, they could drastically advance wound care documentation without the need for special equipment.

The purpose of this prospective study is to test the reliability of wound surface area measurements using a smartphone app versus the current gold standard of the ruler method. Reliability and consistency are determined by the similarities or differences between measurements of the same wound by 2 or more individuals (raters). Interrater reliability is indicated by a high degree of agreement between raters using the same measurement technique on the same wound. Intrarater reliability refers to the consistency of individual raters observing different wounds. Both interrater and intrarater reliability were tested.

Smartphone measurement app
The smartphone app, Tissue Analysis (Tissue Analytics, Baltimore, MD), uses machine learning and computer vision algorithms to automatically define wound parameter, segment, classify, and measure wounds. The algorithms also standardize for lighting, distance, and camera angle. Data were collected on a secure, Health Insurance Portability and Accountability Act of 1996-compliant smartphone app and is available for both iOS and Android platforms. 

Methods
Twenty-five volunteers (raters) of varying clinical experience (medical students, residents, attendings, and nurses) measured a set of 12 photos composed of various sizes and shapes of wounds. Each rater measured the same set of photos twice on day 1: first using a ruler and then using the smartphone app. Each rater then repeated these 2 measurements on the same set of photos 1 week later.  All raters used an identical ruler, phone, and the same folder of photographs with instructions and practice photographs on the front cover. Instructions for the ruler method included measuring length by width perpendicularly and recording these 2 measurements for each wound on an included answer sheet. Instructions for the app specified to include both the entire wound and the green calibration sticker needed by the app for accurate size calibration in every photograph. Raters were not required to record any measurements for the app and were given 2 to 3 minutes to practice on the 2 practice photos before starting.

Each rater’s paired measurements were identified with a number and recorded in Excel (Microsoft Corp, Redmond, WA). Surface area was calculated by multiplying the recorded length by width for each wound for the ruler method, and the automatically calculated surface area was recorded by the app method. 

Statistical analysis
SPSS for Windows 18.0 (SPSS Inc, Chicago, IL) was used for statistical analysis. The intrarater reliability of wound surface area measures the agreement between 1 rater’s measurements when measuring the same wound. The interrater reliability measures the agreement between measurements from several raters when assessing the same wound. Intraclass correlation coefficient (ICC) was used for analysis. The 2-way mixed effect model was selected for calculations of the ICC. High ICC values indicate superior rater reliability. Using Fleiss’ Kappa scale, ICC < 0.4 represents poor reliability, 0.4 to 0.75 represents fair to good reliability, and > 0.75 represents excellent reliability.

Overall, the ruler method produced wound area measurements that were 30% larger than the app method with the difference tending to be greater for smaller wounds. The average wound area using the ruler and the app were 42.3 cm² and 33.9 cm², respectively.  The mean difference in wound area between the 2 methods was 8.4 cm². Comparing these means using a paired t test returned a t statistic of 18.01, with P < .001. Ruler measurements were larger to a statistically significant degree (Figures 1, 2).

The results of the ruler and app measurements for all wounds and all raters are presented graphically (Figures 3, 4). Visually, it appears that there is more consistency in the app measurements (Figure 3) compared with the ruler measurements (Figure 4). Testing agreement using the ICC confirms this.

Intrarater reliability was assessed using a 2-way mixed-effects ICC to assess the degree to which raters using the app provided consistency in their ratings of surface area across wound photos. The resulting intrarater ICC averaged across all wounds was in the excellent range for both the app and the ruler (ICC, 0.99 for the app; ICC, 0.92 for the ruler), indicating that raters using either method had a high degree of agreement. This suggests that surface area was rated similarly across a rater’s measurements. The interrater reliability between raters using the app was high (ICC, 0.77), showing an excellent level of agreement across raters with the app. The interrater reliability for the ruler was poor (ICC, 0.34), demonstrating a low level of agreement between raters with the ruler method.

In addition, the volunteers (raters) were asked for feedback on the app, its ease of use, and if, given the option, they would switch from the measurement tools they currently use to a smartphone-based app. Feedback was very positive, with the consensus (92%; 23/25) being that they would switch to the app if the price was reasonable. Many thought the app was faster and easier to use and liked that it did not require any contact with the surface of the wound. Overall, there was very positive feedback and a high level of interest in the app and the other wound documentation features it offers that were not utilized in this study.

The major limitation of this study was the use of wound photos as surrogates for actual wounds. Photos reflect light, which could cause the app’s algorithms to miscalculate the border of the wound. Photos also do not take into account that the app measurements may be taken from different angles based on the position that the phone is held. The app used in this study has built-in algorithms that standardize for lighting, distance, and camera angle, but the investigators were unable to evaluate this in the current study.

For future research, a study utilizing the same app and similar research protocol but with actual wounds or wound models would be beneficial to eliminate the glare from using photos. If using patient’s wounds, the measurement sets should be taken close together to avoid the wound changing in size. The authors have no future studies planned at this time.

Tracking wound dimensions is an essential part of wound management in order to monitor healing and prevent a delay if an alternative therapy is needed. Documentation also is used in determining insurance reimbursement. Because wound care is often managed by a team, consisting of the physician, nurses, physical therapists, and even the patient, it is important that measurements be reliable regardless of who takes them.

This study found that a smartphone wound assessment app produces sufficiently consistent results to be useful in the clinical management of slow-healing wounds and appears to be superior to the current gold standard (linear measurements with a ruler). Measurements taken by 1 person had an excellent level of agreement with both the app and ruler methods, although the app was higher. The reliability between raters was still excellent with the app but poor for the ruler method. This could potentially make the app the superior method if more than 1 person will be recording the wound measurements.