Outcome of Comprehensive Psycho-Educational Interventions

Outcome of comprehensive psycho-educational interventions for young children with autism


(Svein Eikeseth, Akershus University College)






Aim of the study:

This study evaluated the outcomes of twenty-five comprehensive-psycho educational research papers on early intervention for children with autism. Of these twenty-five studies, three studies were about behavioral treatment, two studies evaluated TEACCH, and two studies evaluated the Colorado Health Sciences project.


Search Method:

Three search methods were used to identify the relevant outcomes of all the studies. The first method was the use of electronic search engines. The second method inspected recent publications to confirm that the search engines had identified the most recent studies. Finally, researchers known to be involved with the studies were contacted via email and asked to produce references of published and in press articles.


Criteria for assigning scientific merit:

Outcome of these studies were graded according to their scientific value and the magnitude of results documented. Scientific merit was evaluated based on: diagnosis, study design, dependent variables and treatment fidelity. Four levels were used to describe scientific merit:

Level 1:
This represented the highest possible rating.

Diagnosis:
A level 1 was assigned if the participants were diagnosed according to current international standards, which includes the use of the ICD-10 and the DSM-IV. Also, the diagnosis must have been set by clinicians who were independent of the study, or the diagnosis must have been based on well-researched diagnostic instruments including ADI-R

Study Design:
A level 1 was given to the design of the study if a randomized design was employed, that is, if participants have been assigned randomly to two or more study groups.

Dependent Variables:
A level 1 was assigned if intake and outcome measures assessed both intellectual and adaptive functioning. The instruments used to assess intellectual and adaptive functioning must be normalized and standardized. The IQ score must be derived from both language / communication skills as well as visual spatial or performance skills. In addition, to ensure objectiveness of the assessments, blind or independent assessors must have conducted the assessments.

Treatment Fidelity:
A level 1 was assigned to treatment fidelity if it was (a) directly assessed, or (b) treatment was described in treatment manuals.

Level 2:
This represented a moderate scientific merit. Criteria for achieving Level 2 scientific merit was identical to that of Level 1 except that the study design was not random, so each participant did not have an equal chance of being placed in either of the study groups. Group assignment would be based on, for example, participants’ geographical location, parental choice, or availability of treatment personnel. These are examples of non-random group designs.

Level 3:
This represented a low scientific merit.

Diagnosis:
A level 3 was given for diagnosis if the diagnosis (based on the ICD-10 or DSM-IV criteria) was not blind or independent; or the diagnosis was not based on diagnostic instruments, or if the diagnosis was independent or blind but not based on ICD-10 or DSM-IV (or DSM-III for older studies); or if the study failed to specify which diagnostic system was used.

Study Design:
A level 3 was given to retrospective (archival) studies with comparison groups, or singlecase experimental studies where outcome measures were assessed pre and post.

Dependent Variables:
A level 3 was assigned when intake and outcome measures did not assess both intellectual and adaptive functioning, or measures were not normalized and standardized.

Treatment Fidelity:
Level 3 was given to treatment fidelity if insufficient assessment of treatment fidelity, or treatment not based on treatment manuals


Insufficient Scientific Value (ISV):

This was assigned to studies where the evidentiary support was so low that outcome data gave insufficient scientific meaning.


Criteria for deciding magnitude of results:

As with evaluating scientific merit, four levels were provided to evaluate the magnitude of the treatment effects. Once again, Level 1 represented the highest possible rating, and Level 4 represented the lowest rating.

Level 1:
Level 1 was given for the magnitude of results, if significant group differences on IQ and adaptive functioning (deviation or ratio scores) were reported. In addition Level 1 was given, if the assessment included measures of empathy, personality, school performance, friendship, and information regarding diagnostic changes.

Level 2:
Level 2 status was provided for significant group differences on either IQ or adaptive functioning (deviation or ratio scores). For both Level 1 and 2, the IQ measure must be based on language/communication skills in addition to visual spatial or performance skills.

Level 3:
Level 3 status was provided for significant group differences on developmental (or mental) age, or significant group differences on assessment instruments that are not normalized standardized (or significant group differences on improvement).

Level 4:
Level 4 studies reported significant pre-post improvements. In this review, only Levels 1-3 scientific evidence studies are evaluated according to magnitude of treatment effect. Studies classified with insufficient scientific value are excluded because for methodological reasons, they did not allow reliable conclusions regarding outcome to be drawn.


Classification of studies based on scientific merit and magnitude of results:

For information on each approach: TEACCH, The Denver Model and Applied Behavior Analysis, please download the PDF from this website (see link on bottom of this page).

Level 1 Scientific Merit:
Of the three models only one study received a level 1 for scientific merit. This was conducted by Smith, Groen and Wynn (2000) and was designed to evaluate ABA treatment. Results showed that the ABA treatment group scored significantly higher as compared to the parent training control group on intelligence, visual-spatial skills, language and academics, though not adaptive functioning. However, as the study did not show a significant group difference on adaptive functioning, it received a level 2 rating for magnitude of results.

Level 2 Scientific Merit:
Of the twenty-five studies that were evaluated, only four level 2 studies were identified and were all based on ABA treatment (Cohen, Amerine-Dickens, & Smith, 2006; Eikeseth, Smith, Jahr, & Eledevik, 2002, 2007; Howard, Sparkman, Cohen, Green, Stanislaw, 2005; Remington et al., 2007). Three of the studies showed that the participants in the ABA treatment groups scored significantly higher on intelligence, language and adaptive functioning as compared to comparison group children (Cohen et al., 2006; Eikeseth et al., 2002, 2007; Howard et al., 2005). As a result these studies received a level 1 for the magnitude of the results. The Remington et al. study found that children in that ABA treatment group scored significantly higher as compared to children in the comparison group on intelligence, but not on adaptive functioning and language (as measured by standard scores). Therefore, this study received Level 2 fir the magnitude of the results rating. All four studies gained Level 2 scientific merit classification because they lacked a randomized study design: three studies (Cohen et al., Howard et al., Remington et al.) based group assignment on parental preference.

Level 3 Scientific Merit:
Eleven of the twenty-five studies received a level 3 rating. Two studies are based on the TEACCH model; (Mukaddes, Kaynak, Kinali, Besikci, & Issever, 2004; Ozonoff & Cathcart, 1998) and both studies received Level 3 on the magnitude of the results rating. Ozonoff and Cathcart did not specify which diagnostic system the children’s diagnosis was based on, whether or not the diagnosis was set independently, or whether any diagnostic instruments was used. Also, number of one-to-one teaching sessions provided by the parents was unspecified. This study also failed to employ a random assignment. The measures were not performed blind or independently, and did not include adaptive functioning. Children in the treatment group improved significantly more over a period of months than those in the control group. The remaining nine studies evaluated ABA treatments (Andersen, Avery, DiPietro, Edwards, & Christian, 1987; Birnbrauer & Leach, 1993; Eldevik et al., 2006; Lovaas, 1987; McEachin, Smith, & Lovaas, 1993; Magiati, Charman, & Howlin, 2007; Sallows & Graupner, 2005; Sheinkopf & Siegel, 1998; Smith, Buch, & Gamby, 2000; Weiss, 1999). The Lovaas (1987) and McEachin et al. (1993) studies received Level 3 scientific merit because intake measures did not include adaptive functioning.


Insufficient scientific value:

Nine outcome studies were classified as having insufficient scientific value. Six studies evaluated ABA programs (Bibby et al., 2002; Handelman, Harris, Celbiberti, Lilleheht, & Tomchek, 1991; Harris, Handleman, Gordon, Kristoff, & Fuentes, 1991, Harris, Handleman, Kristoff, Bass, & Gordon, 1990; Hoyson, Jamieson, Strain, 1984; Luiselli, Cannon, Ellis, Sisson, 2000), one evaluated TEACCH (Lord & Schopler, 1989), two evaluated the Colorado Health Science Program (Rogers & Dilalla, 1991; Rogers, Herbison, Lewis, Pantone, & Reiss, 1986). All studies used a pre-post design without a single-case control or comparison group.


Discussion:

Only one study received Level 1 scientific merit (the highest possible rating) and four studies received Level 2 scientific merits. All these studies evaluated ABA treatment. Eleven outcome studies received Level 3 rating. Nine of the 11 studies evaluated ABA treatments and 2 studies evaluated TEACCH. Finally, nine outcome studies were classified as having insufficient scientific value. One evaluated TEACCH, two evaluated the Colorado Health Science Program, and six evaluated ABA.

Evaluating magnitude of treatment effects, four ABA studies received Level 1 rating showing that children receiving ABA made significantly more gains than control group children on standardized measures of IQ, language and adaptive functioning (Cohen et al., 2006; Eikeseth et al., 2002, 2007; Howard et al., 2005; Sallows & Graupner, 2005). Several studies also included data on maladaptive behavior, personality, school performance and changes in diagnosis. Three studies received Level 2 rating (Eldevik et al., 2006; Lovaas, 1987; Smith, Groen, & Wynn, 2000), demonstrating that ABA treated children made significantly more gains than the comparison group on one standardized measures of IQ or adaptive functioning. Finally, five ABA studies and two TEACCH studies received Level 3 rating. Based on these guidelines interventions based on ABA will be considered ‘‘well established’’. TEACCH and Colorado Health Science model will be considered neither ‘‘well established’’, nor ‘‘probably efficacious’’.


Future Direction:
  • There is need for additional outcome studies, particularly those whose study designs are of level 1 standard.

  • Effective treatment parameters and mechanisms responsible for change need to be identified and should be priority for ABA researchers.

  • Variables that interact with or have an impact on outcomes should be identified. Treatment for children who respond less favorably needs to be established.

  • Further research evaluating the efficacy of bio-medical treatments combined with psycho-social treatment is required.

  • Research could examine the generalization and transportability of interventions shown to be efficacious in controlled research settings to applied settings.

  • Research could be conducted to examine the efficacy of psycho-educational treatments with older children and adults.

  • Research could develop criteria for discontinuing or changing treatment approach.

  • Research could be conducted to examine the cost-effectiveness and cost-benefits of the interventions.


Conclusion
  • ABA treatment is demonstrated effective in enhancing global functioning in pre- school children with autism when treatment is intensive and carried out by trained therapists.

  • ABA treatment is demonstrated effective in enhancing global functioning in children with PDD-NOS.

  • ABA can be effective for children who are up to 7 years-of-age at intake.



Please note that every effort has been made to condense and provide a broad overview of this research, however, in order not to lose the key information some of the information in this summary has been copied directly from the original article. All credits of the summary whether directly worded or reworded are solely given to the researchers.


To download the original study as PDF, click here

To download the summary as PDF, click here


Great thanks for the permission to post and translate to: Svein Eikeseth, Ph. D. Professor, NOVA Institute for Children with Developmental Disorders, www.novaautism.com, Phone: (+47) 33 61 42 97, (+47) 92 21 09 88


For the summary great thanks to: Miss Georgiana Elizabeth Barzey.