|
Comparing Quality of Life Scales in Childhood Epilepsy: What's in the Measures?
Nora Fayed & Elizabeth N. Kerr
Abstract
Appraising the conceptual content of quality of life (QOL) measures relative to a particular study is an important step in content validation. The purpose of our study was to 1) identify childhood epilepsy-specific QOL outcome measures using a systematic review process, and 2) compare those measures based on a broadly defined, international standard of health, the International Classification of Functioning, Health, and Disability (ICF). Usefulness of the ICF for appraising the content of QOL outcome measures was also described.
Method: A review of the literature was conducted to identify QOL measures. Two independent raters assigned an ICF code to each item of each measure using a modified form of published linking rules. ICF codes were then grouped under health components of body functions, body structures, activities and participation, or environmental factors. A comparison of the percentage of ICF components was performed across the measures. Inter-rater reliability of linking was analyzed using Cohen's kappa.
Introduction
Measures of quality of life (QOL) are popular for describing, evaluating, and comparing the experiences of children with epilepsy. They are used by clinicians, researchers, and community-based programs who wish to assess and evaluate the impact of their efforts to improve a child's health and his or her overall life (Ronen, Streiner, & Rosenbaum, 2003). Users of these measures require consistent and valid methods for determining the actual health content of the QOL measures in a manner that is relevant to children and families affected by epilepsy. Quality of life measures in the field of childhood epilepsy seldom report clear definitions of health or quality of life, often stating expert opinion as the basis for content validation. Yet understanding the content of a measure is integral to determining content validity. Norman and Streiner (2008) describe content validity as the compatibility between that which one seeks to measure and the concept of health contained in the QOL tool. Norman and Streiner (2008) also caution that expert opinion may not be sufficient to capture a measure's content validity if the measure is not representative of the intended concept or if theories assumptions or ideas [of health or quality of life] have changed since the development of the measure. Over the last 5-15 years, a shift in the concept of health has impacted upon the way health is defined relative to quality of life (Wilson & Cleary, 1995). Irrespective of a tool's validity or reliability, one must determine the definition of health in tools used to measure quality of life in children with epilepsy prior to selecting the measure for application in hospital or community based settings.
Understanding of a measure's content is important in the field of quality of life where there is ongoing ambiguity on the part of measurement developers and users alike as to what constitutes a quality of life measure versus a health-status measure. Description of the difference between such measurement approaches has been elucidated by various authors (Bradley, 2001; Cieza & Stucki, 2008; Livingston, Rosenbaum, Russel, & Palisano, 2007). The World Health Organization (1998) defines quality of life as an individual's perception of his/her position in life, while Raphael, Brown, Renwick and Rootman, (1996) describe it as the extent to which an individual enjoys the possibility of his or her life. The implications to the measurement process are such that in order to measure quality of life, one must determine a person's perception or ideas about areas of importance in his or her life with only one of these areas of life being health. Health-status measurement on the other hand might measure similar areas as quality of life including psychological, physical and environmental health but determines only the extent to which the person is experiencing good health or bad health according to these areas. Regardless of whether the approach to measurement is based in health-status or quality of life, it is an essential minimum to determine the definition of health contained in the measures using criteria that are sufficiently multi-dimensional to span both concepts and sufficiently standardized to serve as a basis for comparison.
The International Classification of Functioning, Disability and Health (ICF) provides a guide for comparing the health content of different measures if a broad biopsychosocial definition is intended as the purpose to which the QOL tool will apply. The ICF framework is intended to describe health according individuals' functional capacity within a certain environment as opposed to measuring levels of impairment or physiological dysfunction consistent with a medical model (WHO, 2001). Separate from the framework, the ICF classification can be used to delineate health content through a coding process that employs the several ICF categories. Previous studies using the ICF have assisted researchers to select appropriate outcome measures based on content when the psychometric properties of a measure are appraised by other methods (Stucki et al., 2006; Ceiza & Stucki, 2005). The ICF is divided into components called body function, body structures, activities and participation, and environment. Body function classifies physiological and psychological functions of body systems, body structures classifies anatomical parts of the body such as organs, limbs and their components, activities classifies the execution of tasks or actions while participation classifies the task or action for involvement in life situations. Finally environment classifies the physical, social and attitudinal environment in which people live and conduct their lives (WHO, 2001).
To our knowledge, published reviews of quality of life measures specific to childhood epilepsy have focused on the psychometric properties of a measure (Cowan & Baker, 2004) without focusing on the content of the measure or require revision to include new measures (Carpay & Arts, 1996). No studies have appraised the content of epilepsy specific measures for children. A study which systematically assesses the specific constructs within declared quality of life measures will assist potential users to establish their content validity relative to a particular research or clinical purpose.
The purpose of this study was to conduct a systematic evaluation of the content of childhood-epilepsy specific measures that are described as quality of life tools by evaluating the representativeness of their items relative to the ICF classification.
Methods
Article Identification
A systematic review was conducted to identify QOL measures that were specific to childhood or adolescent epilepsy over the last ten years. We employed a strategy for the search of the literature on Ovid Medline using the keywords epilepsy and quality of life to identify papers published between 1997 and February 2007. The search criterion was limited to articles published in English and confined to children and adolescents aged 0 to 18. This search generated a list of 439 articles. The abstract of each article was reviewed to identify any outcome measures employed or developed in each study. If the outcome measure was not clearly identified from the abstract, the article was accessed and the method section reviewed. Outcome measures were included if they were created for the purpose of measuring quality of life or health-related quality of life with child or adolescent epilepsy as the primary focus area. Measures were excluded if they were focused on specific health domains (such as impairment or activity of daily living) as opposed to quality of life or health-related quality of life or if the measure was validated on an adult sample. For instance the Hague Restrictions in Childhood Epilepsy Scales (Carpay et al., 1997) was omitted due to its focus on activity limitations and participation restrictions. The Impact of Pediatric Epilepsy Scales (Camfield, Breau, & Camfield, 2001) was also omitted because it was the previous version of the Impact of Child Neurological Disability (ICND) (Camfield, Breau, & Camfield, 2003) measure that has since been updated and did meet the inclusion criteria. This resulted in eight measures being selected for review.
Linking Outcomes to the ICF Scheme
The ICF classification scheme represents a variety of health and health-related states that can be accessed on line at http://apps.who.int/classifications/icfbrowser/ in at least six languages. There is little information within the ICF document directing researchers on how to link the content from measures to the ICF codes. The Cieza et al. (2005) ICF linking rules provide guidance on linking content from measures to the ICF and were employed with slight modification. We modified the linking process by assigning one code per item on a questionnaire instead of reducing each questionnaire item into the recommended meaningful units. We identified the dominant concept within the item prior to assigning the concept a specific ICF code. The exception to this rule was that open-ended questions within a measure such as: Is there anything else you wish to tell us about your child's health? were not linked to the ICF. The first letter of the code represented the ICF component of body functions (b) or structures (s), activities and participation (d), or environment (e). The following digits of the code were numeric. For example, item 13 from the Impact of Child Illness Scale (ICI) (Hoare, Mann, & Dunn, 2000): "My child may not find a job when he leaves school" was linked to the code d845 acquiring, keeping and terminating a job in the activities and participation domain of the ICF (see Figure 1). Thus, a measure with fifteen items or questions generates an output of fifteen codes representing the extent to which ICF health determinants such as body functions, body structures, activities and participation, and environmental factors embody the content of the QOL measure.
Figure 1: Linking the Impact of Child Illness Questionnaire item 13 using the ICF
Items that were not easily coded using the previous description were also linked to the ICF using the Cieza et al., (2005) criteria. According to these rules, concepts from an item on an outcome measure that were conceptually part of the ICF yet not easily linked to a specific code were given the code not definable (nd). Concepts such as 'general quality of life', and 'overall mental and physical health' were also assigned the code nd. Items representing personal factors were coded pf and items representing concepts not included in the ICF were coded nc.
Following completion of the linking process, the codes were analyzed for their distribution across ICF categories. This provided a determination of which categories of health were emphasized in each measure.
Coding Assessors
According to an iterative process, two assessors, an occupational therapist and a neuropsychologist familiar with the ICF classification scheme met to review Cieza et al., (2005) linking criteria and rules as well as the modification of the rules such that each item within a measure was the unit of analysis for coding. Two sample items were jointly coded from an excluded measure. Interpretation of linking rules was discussed and clarified. Following this step, the assessors coded all the items from each outcome independently. They then met to discuss any discrepancies in the linking process and decide on a code for the purpose of reaching consensus on the final appropriate code for each dominant concept in an item. Since consensus was achieved for each code, a third party was not required to settle disagreements.
The inter-rater reliability from previous linking reports for generic childhood-based functional measures using kappa coefficients has demonstrated that the level of reliability varies depending on the concepts that are coded by the ICF (Stucki et al., 2006; Ogonowski, Kronk, Rice, & Feldman, 2004). In our study, the reliability of the coding process was analysed between assessors using Cohen's kappa. Ultimately, a consensus approach was used to generate the final list of codes presented in our results.
Results
Reliability of Coding
An analysis of coding agreement between assessors on the ICF appears consistent with previously reported reliability performed by investigators who were not developers of the linking rules (Ogonowski et al., 2004). Our between-assessor comparison was moderately reliable (kappa (k) = 0.56). The agreement between each assessor and the final consensus list of codes was high: the kappas were 0.74 for the occupational therapy assessor and the final list, and 0.68 for the neuropsychologist and the final list.
Outcomes Included
Eight epilepsy-specific outcome measures meeting our criteria were identified from the retrieved studies. These measures are summarized in Table I.
Categories Represented: In total, 443 items were linked to the ICF. Overall the body function domain of the ICF represented 44% of the items across measures followed by 25% in the activity and participation, 18% of codes were not defined, and 8% were represented by the environment domain. The body function section of the ICF was the domain of health most heavily represented in five of eight QOL measures included (Table 2). Body function was represented as one of the top two categories represented by the ICF in all the measures included in the study (Table 2). The body structures (s), not covered (nc), and personal factors (pf) codes were not displayed in the tables since they involved a negligible percentage of values within the measures.
Table 2: Percentage of codes within each ICF category
|
|
Percentage of codes within each ICF domain* |
|
Outcome
|
b
|
d
|
e
|
nd
|
|
HRQL in CWE
|
28
|
16
|
40
|
16
|
|
ICI
|
30
|
23
|
20
|
17
|
|
QOLIE - AD - 48
|
13
|
23
|
3
|
9
|
|
QOLCE
|
51
|
25
|
4
|
9
|
|
DISABKIDS
|
59
|
17
|
11
|
11
|
|
ICND
|
40
|
20
|
1
|
28
|
|
ELDQL
|
44
|
36
|
0
|
9
|
|
QOLPES
|
20
|
25
|
25
|
10
|
*Percentages may not sum exactly to 100, nc - not covered codes, s - body structures and pf - personal factors are not displayed due to their negligible impact on the final list of codes
b - represents body functions codes
d - represents activity and participation codes
e - represents environment codes
nd - represents codes not defined
|