Which method facilitates data collection for both interobserver agreement and treatment integrity?

Module 5: Measurement

Behavior Analyst Certification Board Registered Behavior Technician™ (RBT®) Task List 2nd Ed

A-01 Prepare for data collection

A-02 Implement continuous measurement procedures (e.g., frequency, duration).

A-03 Implement discontinuous measurement procedures (e.g., partial & whole interval, momentary time sampling

A-04 Implement permanent product recording procedures

A-05 Enter data and update graphs

A-06 Describe the behavior and environment in observable and measurable terms

Overview

          In Module 1, ABA was identified as an evidence-based practice, meaning that it is scientifically validated. Evidence-based treatments are validated via experimental analyses inclusive of measurement and data analysis. Measurement and data analysis of behavior are necessary at all levels of ABA. For instance, it is also necessary to evaluate the effectiveness of treatments with the individuals we serve in clinical practice. The process of measuring and analyzing treatment data to make informed decisions about the course of treatment is referred to as data-based decision making. Therapists collect data on specific target behaviors (behaviors targeted for increase and behaviors targeted for decrease). Those target behaviors are then graphed and visually inspected by a BCBA. Clinical decisions before treatment (baseline) and during treatment are based upon these data and graphs.

Measurement

Behavioral Definitions If you recall from module 1, behavior is observable and measurable. Therefore, before a behavior of interest can be measured it must be described in observable terms. An operational definition is a clear and technological description of the behavior of interest (also known as topography). In order for paraprofessionals to collect accurate data they must know the parameters of the target behavior they are collecting data on. Therefore, the operational definition must provide the observable scope of the behavior such that two or more people can agree on whether the behavior occurred or not. This is referred to as interobserver agreement (IOA). A BCBA will conduct IOA probes by collecting data alongside a paraprofessional and then calculating the percentage of agreements between the observers (O’Neill, McDonnell, Billingsley, & Jenson, 2011), to ensure that the operational definition captures the target behavior and that the paraprofessional has been trained adequately in data collection methods. If the data are compared and the IOA is low, the BCBA may alter the definition and/or provide moretraining to the paraprofessional(s). Because these data are used to determine treatment effects, it is imperative that accuracy be high, or a clear determination cannot be made as to whether behavior change is due to the treatment strategy, a change in the measured behavior, or both (O’Neill et al., 2011). To assist with high levels of accuracy when collecting data on a target behavior, the operational definition should meet the following three characteristics (Hawkins & Dobes, 1977): 1. Objective Behavior should be defined using only objective terms regarding observable dimensions of the behavior. For example, identifying terms such as “frustration,” or “feeling like,” are subjective and leave the understanding up to the reader’s own interpretation. In place of those terms, it may be more appropriate to use “furrowed brows” or “vocal protest” to define the behavior of interest. 2. Clear Operational definitions meet these criteria if they are written so that two or more people could accurately collect data on the behavior, regardless of their familiarity with that client. 3. Complete Any potential instances of the behavior in a context should be identified, and if necessary exclusions or non-examples of the behavior should also be identified. A predetermined onset and offset to the behavior should also be identified. Identifying these parameters assist with minimal judgment being necessary to determine whether or not the behavior occurred (O’Neill et al., 2011). Examples: Aggression: Any instance of hitting with an open or closed fist, grabbing, kicking, biting, hair- pulling, and/or scratching another person. Including throwing objects at another person. Example: Slapping another person. Non- Example: Tapping another person on the shoulder repeatedly to get their attention. Inappropriate Vocalizations: Any instance of swearing, or threatening directed at another person. Example: Saying “I’m going to hurt you” or “Screw you” Non-Example: Saying “I am really upset, I hate this” Refusal to wait: Any instance of physically declining to wait. Example: Getting up from waiting area and running around or engaging in other challenging behaviors such as aggression, or inappropriate vocalizations during a time-period in which one is supposed to be waiting. Non- Example: Stating “I don’t want to wait” Waiting: Any instance of appropriately sitting or standing for duration of time for a preferred activity or event in the absence of challenging behavior. Example: Sitting in a chair engaged with a book or toy while another person fills a prescription at the pharmacy before getting up to gohome or to a preferred event. Non-Example: screaming and swearing or running around and knocking things off shelves while another person fills a prescription at the pharmacy. Types of Measurement Once a paraprofessional is familiar with the operational definitions for each of their clients, they must also become familiar with the measurement method for each of the target behaviors for their clients. The Behavior reduction plan and/or treatment plan will outline both the operational definitions and measurement methods to prepare the paraprofessional for data collection. There are several measurement options that can be used to collect data. Some of the most common measurement procedures are described below.            Frequency. Frequency measures each time a behavior occurs or the count, typically by using a tally system or hand-held counter. Frequency data are typically collected for behaviors that have a clear onset and offset (Cooper, Heron, & Heward, 2007). The frequency of a behavior during a specified period of time is referred to as rate (Bailey, & Burch, 2017). For instance, one may collect data throughout an entire session on the frequency count and then convert it to rate by taking the total number of responses and divide that total by a unit of time such as minutes or hours (e.g., 50 responses/2 hours = 25 responses per hour).           Duration. This type of measurement procedure that measures the amount of time a behavior occurs from its start to its finish using an appropriate unit of time (seconds, minutes, etc). Duration data is typically collected for behaviors that occur across time in episodes (Cooper, Heron, & Heward, 2007). There are two ways to collect duration data: 1) Duration per occurrence: Duration of each occurrence of a target behavior from its start to its finish is measured. 2) Total session duration: The sum of the occurrences of the behavior throughout the entire session is measured. Duration per occurrence will result in multiple data points per session, total session duration will result in one data point at the end of the session (Cooper, Heron, & Heward, 2007). Duration data are collected when the goal is to increase of decrease the duration of target behavior (O’Neill et al, 2011).   Figure 5.1 Example of a frequency and duration data sheet           Time Sample Interval Recording. In this procedure, a session or sample of time is divided into intervals. Occurrence of the target behavior is measured within that interval. Use of this procedure is effective for behaviors that are difficult to measure because they would require continuous observation. Below are three procedures for collecting time sampling data.           Whole interval recording. This measurement procedure measures if a behavior does or does not occur throughout the entire predetermined interval. For example, if a session is divided into 1-minute intervals and the behavior occurred for the entire minute, data would be collected by marking a “+” for that interval. If the behavior occurred for any time less than one minute, data be collected by marking a “-“ for that interval (Cooper, Heron, & Heward, 2007).            Partial interval recording. This measurement procedure measures if a behavior does or does not occur at any point during the predetermined interval. For example, if a session is divided into 1-minute intervals and the behavior occurred for 1 second, 10 seconds, or for any duration in between, data are collected by marking a “+” for that interval. Any instance of the behavior during the interval would be measured. If there is no occurrence of the behavior duringthe interval, data would be collected by marking a “-“ for that interval. (Cooper, Heron, & Heward, 2007).  Momentary time sampling. This measurement procedure measures if the behavior is occurring only at the moment the predetermined interval ends. For example, if a session is divided into 1-minute intervals, data would be collected by marking a “+” if the behavior occurs at the moment (there is usually a 2 second observation window) the interval ends. If there is no occurrence of the at the moment the interval ends, data are collected by marking a “-“ for that interval. (Cooper, Hero n, &Heward, 2007). 10-second Interval Data Sheet 10 sec interval DS Figure 5.2 An example of an interval data sheet that could be used for Whole, partial interval recording or momentary time sample data collection    Permanent Product. All previous measurement procedures described measurement of behavior in real time. Permanent product measures the effect the behavior produced on the environment (Cooper, Heron & Heward, 2007). For example, one form of measurement for the behavior of cleaning a bedroom is an observation of the room after it has been cleaned. The behaviors that occur while the client is cleaning the room may not be of any importance to the observer, thus the observer may be able to engage in other activities or work with other clients while awaiting the outcome. Another example of permanent product data collection is a worksheet such as a math worksheet. Once the client has completed the worksheet data can be collected on the correct and incorrect math problems written on the worksheet. Video and audio taping behaviors may also be efficient modes of permanent product measures. By using video and audio taping, a more accurate measurement of behaviors may be possible. All procedures can be used to measure data through these modes. It may also support assessing treatment integrity (Module 4).

Data entry and Graphing

Data entry Once data are collected, there are various different data entry procedures an organization may use to prepare these data to be graphed for the purposes of visual inspection. For instance, some organizations collect data via pen and paper and then graph these data using graph paper. Whereas some organizations use Microsoft® Excel™ or other graphing software to graph data, therefore once data are collected via pen and paper they must then be entered into a spreadsheet. However, some organizations use electronic data collection on a lap top or tablet, making the data entry step and often the graphing step automatic. Each agency will train their paraprofessionals on the specific data entry and graphing procedures they use. Graphing Graphs serve several functions however the main functions are to organize data to provide a detailed picture of the relationship between the treatment and the target behavior(s) to both facilitate decision making and share the outcome with other individuals on the team (e.g., parents, other professionals) (O’Neill et al, 2011).          Line Graph. A line graph is the most common graph used in ABA. A line graph typically has a vertical or y axis and a horizontal or x axis. The y axis depicts the measurement of the target behavior and the x axis depicts the time sequence (e.g., sessions, days). Therefore, each data point on the graph represents a quantifiable measure of the target behavior recorded during sessions and the time or session it was conducted in. Connecting each data point with a line creates a data path (Cooper, Heron, & Heward, 2007). The different conditions or phases of treatment (e.g., baseline and intervention) are separated by vertical lines called phase lines (O’Neill et al, 2011). Comparing points on the graph shows the level, trend, and/or variability of behavior change across conditions or phases (Cooper, Heron, & Heward, 2007). Figure 5.3 Example of a simple line graph          Bar graphs. A bar graph is also a commonly used graph in ABA. It is used to display and compare discrete sets of data when an efficient summary of data is warranted. For example, a summary of preference assessment data (module 6) may be depicted using a bar graph. Like a line graph, the y axis depicts the measurement of the target behavior however sometimes the x axis does not depict a time sequence if it lacks an underlying dimension to be scaled as such and therefore may be presented by a stimulus (e.g., different items assessed for preference) or a stimulus change (e.g., a pretest vs. a posttest). Although, a bar graph shares many of the same features as a line graph, however, because a bar graph does not have distinct data points representing successive response measures through time it also does not allow for analysis of variability & trends in behavior like a line graph does. Therefore, the bar graph sacrifices the presentation of trends in behavior and instead provides a summary of large amounts of data into a simple format (Cooper, Heron, & Heward, 2007). Figure 5.3 Example of a simple bar graph Quiz

References

Bailey, J. S., & Burch, M. R. (2017). Research methods in applied behavior analysis. Routledge. Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied Behavior Analysis (2nd ed.). Columbus, OH: Merrill Prentice Hall. Hawkins, R.P., &; Dobes, R.W., (1977). Behavioral Definitions in Applied Behavior Analysis: Explicit or Implicit? In Etzel, B. C. J.M. LeBlanc, J.M., & Baer, D. M.(Eds). New Developments in Behavioral Research: Theory, Methods and Application (pp. 167-188) Hillsdale. NJ Erlbaum. O’Neill, R. E., McDonnell, J. J., Billingsley, F. F., & Jenson, W. R. (2011). Single case research designs in educational and community settings. Boston: Pearson.