Interrater and intrarater reliability of photoplethysmography for measuring toe blood pressure and toe-brachial index in people with diabetes mellitus

Abstract

Background

A reliable tool to measure arterial flow to the feet in people with diabetes is required given that they are particularly prone to peripheral arterial disease. Traditionally, the ankle brachial index (ABI) has been used to measure arterial circulation, but its application is limited due to calcification of larger arteries. More recently, toe pressure and the toe brachial index (TBI) has been suggested as superior to ABI measurements because they assess smaller digital arteries less prone to arterial calcification. However, reliability studies for the clinical use of photoplethysmography (PPG) in people with diabetes are lacking.

Methods

Sixty people with diabetes mellitus (35 males and 25 females, mean age 59.6 yrs) consented to take part in the study. The majority (92%) had type 2 diabetes and 8% had type 1 diabetes. Forty-three percent were diagnosed as having peripheral neuropathy when tested using a biothesiometer and 15% were current smokers (10 – 40/day). A podiatrist and a diabetes educator measured toe and brachial blood pressure independently and in a random order using PPG. These measurements were repeated again seven days later, and subsequently analysed with intraclass correlation coefficients (ICC), 95% confidence intervals (CI) and standard error of measurement (SEM).

Results

The intrarater reliability of measuring toe pressures was excellent (ICC_{3,1 =}0.78-0.79, SEM 8 mmHg) and interrater reliability was also excellent (ICC_2,2 = 0.93, SEM 4 mmHg). The intrarater reliability for measuring brachial pressures was generally poor (ICC_3,1 = 0.40 – 0.42, SEM 19 mmHg) and interrater reliability was fair-good (ICC_2,2. 0.65, SEM 14 mmHg). The TBI intrarater reliability was fair-good (ICC_3,1 = 0.51-0.72, SEM 0.08), whilst the interrater reliability of TBI was excellent (ICC_2,2 = 0.85, SEM 0.07).

Conclusions

Based on these results, interrater and intrarater reliability of PPG is excellent for measuring toe blood pressure, good for TBI and only fair for brachial pressures in people with diabetes mellitus.

Background

Peripheral arterial disease (PAD) is one of the most serious complications affecting the lower limbs of people diagnosed with diabetes mellitus. PAD is an indicator for widespread atherosclerosis and increased mortality rates [1]. It is imperative that PAD is adequately assessed and diagnosed early in patients with diabetes to offer patients the best possible outcomes, and prevent foot ulceration and subsequent foot or lower limb amputation. Arterial flow to the periphery has traditionally been measured by palpation of dorsalis pedis and posterior tibial pulses or through the calculation of the ankle brachial index (ABI) [2–5]. The ABI is a ratio of the systolic ankle blood pressure and the systolic brachial blood pressure. It is generally accepted that a ratio of 1:1 for the ABI suggests adequate peripheral perfusion. However, both of these tests have their limitations. Palpation of pedal pulses is a subjective measure and pulses become increasingly difficult to palpate as PAD becomes more significant [6–8]. There is also large interrater variability in palpation of pedal pulses with inexperienced clinicians [9, 10]. The ABI is a reliable measure of PAD in patients without diabetes and its sensitivity and specificity are excellent. However, for those patients with long standing, or poorly controlled diabetes the ABI has limited application due to the likelihood of falsely elevated readings. This occurs due to the non compressibility of the larger calcified arteries that is a complication of long standing diabetes [8, 11–13].

Finding a reliable test to measure the perfusion distal to the ankle is required to adequately assess and treat patients with diabetes and PAD. Damage to the smaller branches of the arteries, known as microvascular disease, is common in diabetes. In particular it is important to understand the severity of microvascular disease distal to the ankle in patients who have active foot ulceration, as it provides the clinician with a greater understanding of healing potential and whether there is an opportunity for the vascular team to improve the flow to the extremity via revascularisation techniques such as angioplasty, arterial stenting or bypass surgery. The ABI is unable to adequately assess these microvascular complications as it measures proximal to the ankle joint encompassing the arterial flow of the anterior and posterior tibial arteries and does not identify any occlusion or calcification of vessels distal to this site [13, 14]. The concept of measuring toe blood pressures and calculating the toe brachial index (TBI) is not new [7]. The measurement of toe pressures using a PPG unit such as the Hadeco Smartdop is a relatively simple procedure involving a small digital toe blood pressure cuff and PPG probe to capture red blood cells as they pass through the underlying tissue. To calculate the TBI, the process is the same as calculation of the ABI, with the exception of the ankle pressure being replaced by the toe pressure. One study reported quite high intra-class correlation coefficients using the Hadeco Smartdop 45 PPG to assess systolic toe pressures and TBI in people with diabetes, but with a wide range of error [15]. These reliability concerns may relate to the type of PPG unit evaluated which required manual sphygmomanometer to assess systolic toe pressures and TBI, or may relate to the small sample size of 30 participants. Therefore the aim of this study was to evaluate intrarater and interrater reliability of measuring systolic toe pressures and TBI using the Hadeco Smartdop 30EX PPG with built-in automated cuff inflator in 60 patients with diabetes mellitus.

Methods

Participants

A sample of 60 participants with diabetes mellitus confirmed by an oral glucose tolerance test participated in this study. All participants were recruited through advertisements at the Diabetes Service and High Risk Foot Clinic at Nepean Hospital, and the Foot Wound Clinic at Westmead Hospital. Participants were excluded if they were under 18 years, had an active ulceration on one of their great toes, or had recent history of infection, osteomyelitis or acute charcot neuroarthropathy. The study was approved by the human research and ethics committee of Nepean Hospital (HREC no: 08/045).

Instrument

The device used to obtain the pressure measurements was the Hadeco Smartdop 30EX PPG. This particular unit was chosen as it is widely used in our high risk foot clinics and associated teaching facilities. Two studies have used the Hadeco Smartdop 30EX PPG as an effective screening test and outcome measure of coronary and peripheral arterial disease [16, 17] . This device has the ability to measure and calculate ankle, brachial, toe and segmental leg blood pressures as well as PPG and doppler waveforms. It has a built-in automated cuff inflator and has the ability to change cuff sizes. Blood pressures can be obtained by using either the doppler or PPG probe to measure the blood flow during occlusion. The device is a portable unit that runs off battery or mains power.

Procedure

The participants were measured by two assessors: assessor 1 (CS) was a podiatrist with three years clinical experience and two years experience using the PPG; assessor 2 (DM) was a registered nurse and diabetes educator with over 30 years clinical experience, and only four hours training in the use of the PPG.

All participants were required to lay supine for five minutes prior to any assessment being conducted. After this time assessor 1 conducted a standard neurological and vascular foot assessment to determine an overall picture of the participant’s current foot health. Toe and brachial pressures were assessed on a randomised limb by a randomised first assessor, with results blinded between assessor 1 and 2. Brachial pressures were obtained using the automated blood pressure cuff attached to the Hadeco Smartdop 30EX unit and the 8mhz doppler probe. Toe pressures were obtained using the automated toe blood pressure cuff in conjunction with the PPG probe attached to the Smartdop 30EX unit. Toe pressures and the TBI were automatically calculated by the unit and a printout was obtained. Each participant returned seven days after their first assessment for the final measures to be collected. Participants were required to advise the assessors if anything had changed with their medical history, including changes to current medications, alteration in smoking habits between visits.

Statistical methods

Descriptive statistics were calculated to characterise the study sample in SPSS version 18.0 (Chicago, IL). Normality of data distribution was assessed with the Kolmogorov-Smirnov test with Lilliefors significance correction and the appropriate parametric or non-parametric test was subsequently applied. Intraclass correlation coefficients (ICC) and 95% confidence intervals (CI) were calculated to determine intrarater reliability (ICC_3,1) and interrater reliability (ICC_2,2) [18]. Benchmarks suggested by Fleiss were used to interpret ICC values, where a value of 0.75 or greater indicates excellent reliability; 0.40 to 0.75, fair to good reliability; and 0.40 or less poor reliability [19]. To determine the absolute between-trial variability in scores, the standard error of measurement (SEM) was calculated [18].

Results

Demographics and physical characteristics of all participants are shown in Table 1. Intrarater and interrater reliability results are presented in Tables 2 and 3, and summarised below.

Table 1 Demographic details and physical characteristics of the 60 participants

Table 2 Intrarater reliability of PPG for measuring toe blood pressure and TBI in 60 people with diabetes mellitus

Table 3 Interrater reliability of PPG for measuring toe blood pressure and TBI in 60 people with diabetes mellitus

Intrarater reliability

Toe pressures

The intrarater ICC for assessor 1 (CS) was excellent (ICC_3,1 = 0.78, 95% CI 0.65 - 0.86) and the measurement error was 8.5 mmHg. The intrarater ICC for assessor 2 (DM) was excellent (ICC_3,1 = 0.79, 95% CI 0.67 - 0.87) and the measurement error was 7.6 mmHg.

Brachial pressures

The intrarater ICC for assessor 1 (CS) was fair (ICC_3,1 = 0.42, 95% CI 0.19 - 0.61) and the measurement error was 19 mmHg. The intrarater ICC for assessor 2 (DM) was poor (ICC_3,1 = 0.40, 95% CI 0.17 - 0.59) and the measurement error was 19 mmHg.

TBI

The intrarater ICC for assessor 1 (CS) was fair (ICC_3,1 = 0.51, 95% CI 0.30 - 0.68) and the measurement error was 0.16. The intrarater ICC for assessor 2 (DM) was good (ICC_3,1 = 0.72, 95% CI 0.57 - 0.82) and the measurement error was 0.08.

Interrater reliability

Toe pressure

The interrater ICC was excellent (ICC_2,2 = 0.93, 95% CI = 0.89 - 0.96) and the measurement error was 4 mmHg (Table 3).

Brachial pressure

The interrater ICC was good (ICC_2,2 = 0.65, 95% CI = 0.42 - 0.79) and the measurement error was 14 mmHg.

TBI

The interrater ICC was excellent (ICC_2,2 = 0.85, 95% CI = 0.74 - 0.91) and the measurement error was 0.07.

Discussion

Overall our results showed that the intrarater and interrater reliability of Hadeco Smartdop 30EX PPG was excellent for measuring toe blood pressure, good for TBI and poor for brachial pressures in 60 adults with diabetes mellitus. This finding supports previous research recommending PPG as the most effective and reliable measure of systolic toe pressures in the general community [2, 14, 20, 21].

Systolic toe pressures using PPG give a quantifiable value of the arterial supply to the distal foot. For clinicians working with the high risk foot it is important to determine the healing potential of a patient with diabetes, and systolic toe pressures allow us to do this. They are a simple, yet very effective component of a standard vascular foot assessment. Our results enable better adherence to the suggested threshold of <30 mmHg systolic toe blood pressure, which infers ischaemia, decreased healing potential and increased amputation risk, to ensure more appropriate referral to vascular specialists [12, 21–23]. After obtaining toe pressure results the clinician can then decide on the appropriateness of a referral for additional arterial studies or make evidenced-based decisions on the most appropriate treatment plan for the patient, such as vascular intervention, wound dressings or sharps debridement. The TBI is the ratio between the toe systolic pressure and the brachial systolic pressure, similar to the ABI ratio. When using the TBI as an indicator of peripheral arterial supply the clinician can use the reference ranges that have been reported by multiple authors to determine the level of PAD in patients with and without diabetes. Many authors [12, 21–23] have assigned a TBI <0.64 indicative of arterial insufficiency, whilst others have suggested a TBI < 0.75 is a clinically significant sign of arterial disease [8].

Reliable health-related outcome measures provide the clinician with robust markers of disease severity. As such there are a number of clinical implications from the results of this study. Toe pressures and the TBI can be used as an adjunct to a standard peripheral arterial assessment by general practitioners, podiatrists, vascular surgeons and nurses to obtain quantitative baseline measures or to confirm diagnosis of arterial insufficiency. Our results might assist health professionals working in wound care within a high risk foot clinic to select and monitor the most appropriate treatments based upon probability of healing in those patients with ulceration. These treatments may involve revascularisation, amputation, or conservative management. Health professionals may also use PPG at baseline and at regular intervals during interventions such as exercise programs, smoking cessation or drug therapies to monitor arterial disease status over time. This would have the added benefit of providing the patient with positive reinforcement that the given intervention is advantageous to their health and importantly having a positive effect on collateral circulation. The Hadeco Smartdop 30EX PPG is an easy to use assessment tool that requires minimal training to perform reliably as an adjunct to clinical examination of the diabetic foot.

Our results should be interpreted within the limitations of the study design. First, the assessors were not blinded to the results of the blood pressures they had taken. Although the assessors were blinded to each other’s results, they were able to see the results of the test they had just completed. However, the Hadeco Smartdop 30EX does not allow for the operator to adjust the result as it is completely automated. To reduce the risk of bias, a print-out of the result was obtained from the machine and all results were given to a research assistant after each examination. Finally, this research focused on the intrarater and interrater reliability of toe pressures and TBI and did not take into account the validity of the test. Although some preliminary research has been conducted in this area, further research is required to determine the validity of these tests compared to standard vascular laboratory tests such as arterial duplex ultrasound and angiography.

Conclusions

Based on these results, interrater and intrarater reliability of the Hadeco Smartdop 30EX PPG is excellent for measuring toe blood pressure, good for TBI and only fair for brachial pressures in adults with diabetes mellitus.