Performance Based Monitoring and Analysis System
The Performance-Based Monitoring System was implemented to move special program monitoring from a cyclical, compliance-based, on-site monitoring system to data-driven results-based, system with targeted interventions in the areas of special education, No Child Left Behind (federal Title programs), Career and Technical Education, and Bilingual Education/English as a Second Language. Some underlying strategies of the PBM system include a shift away from process to results (i.e. program effectiveness and student performance), a strong emphasis on data integrity, a focus on a coordinated approach to agency monitoring, and a more creative application of sanctions and interventions.
The system is designed to improve student performance and program effectiveness, and to address financial and program compliance concerns within the context of improving student performance.
Data Driven –
Comprehensive data collection systems enable TEA to evaluate student performance and program effectiveness at the state level and to conduct on-site visits when necessary and appropriate.
Comprehensive Evaluation with Targeted Interventions –
Every program TEA is authorized to monitor is evaluated every year in every district. Graduated interventions targeted to specific areas of concern engage more districts in improvement activities.
Coordination and Consistency –
Indicators and standards are aligned across evaluation systems with coordinated interventions appropriate to the areas of concern and consistent across programs.
School districts receive annual performance information through the PBMAS, a set of performance and program effectiveness indicators for special programs TEA is required by state or federal statute to monitor.
The following programs comprise PBMAS:
Summary of 2014 PBMAS Changes
The ongoing transition to the State of Texas Assessments of Academic Readiness (STAAR)2 is reflected in the 2014 PBMAS with the continued implementation of certain STAAR 3-8 indicators; the discontinuation of certain end-of-course (EOC) indicators and the revision of others in response to the provisions of House Bill 5, 83rd Texas Legislature, 2013; and the introduction of new EOC social studies and English Language Arts (ELA) Report Only indicators in all four program areas. New standards and cut-points have been implemented for the annual dropout rate indicators in all four program areas, and performance level assignments were reinstated for all STAAR Grades 3-8 social studies indicators. Other changes unique to each PBMAS program area are summarized below.
It is anticipated that ongoing development of the PBMAS will include the addition of new indicators, revision of current indicators, and deletion of indicators that are no longer necessary. Factors independent from the PBMAS itself are also likely to have an impact on the future versions of PBMAS. These factors include the following:
A performance level (PL) is the result that occurs when a standard is applied to a district’s performance on an indicator. The performance levels available for indicators in the 2014 PBMAS include Not Assigned (NA) (including Not Assigned through SA, PJSA, or DI), 0, 0 SA, 0 PJSA, 0 RI, 1, 1 SA, 1 PJSA, 2, 2 SA, 2 PJSA, 3, 3 SA, and 3 PJSA. SA refers to special analysis, while PJSA refers to professional judgment special analysis; both are described in the Minimum Size Requirements and Special Analysis section. RI refers to Required Improvement, which is also described in a separate section. DI refers to data integrity and is used in combination with the NA performance level to address situations where compromised data render any other performance level impossible to determine.
A performance level of 0 is the highest designation for any indicator, meaning the district met the standard for the indicator. A performance level of 3 is the lowest designation, indicating the district performance was farthest from the 0 - Met Standard designation. The performance level thresholds are typically referred to as cut-points.
Reminder on Acronyms
l PL = Performance Level
l RI = Required Improvement
l SA = Special Analysis
l PJSA = Professional Judgment Special Analysis
l LEP = Limited English Proficient
l BE = Bilingual Education offered in LISD grades Pk-5
l ESL = English As a Second Language offered in LISD grades 6-12
The PBMAS, by design, has a built-in improvement component. Because the system includes a range of performance levels, districts that demonstrate improvement from one year to the next can progress from one performance level (PL) to another. For example, a district with 65% of its migrant students earning a diploma under the RHSP or DAP received a PL 1 in the 2013 PBMAS. If that same district is able to improve its migrant RHSP/DAP diploma rate to 70% in 2014, it would receive a PL 0 because its performance meets the 2014 PL 0 standard. In addition to the system’s built-in improvement component, the 2014 PBMAS will again include a “required improvement” component for certain indicators.
The Program Monitoring and Interventions (PMI) Division notifies each district selected for a 2014 PBMAS intervention. Performance-based monitoring interventions determinations are based on PBMAS performance levels and other criteria as well as results of the data validation analyses. Sanctions are applied as necessary and according to state and federal law and Texas Administrative Code (TAC), including, but not limited to, Texas Education Code, Chapter 39.
Additional information related to performance-based monitoring interventions is reflected in 19 TAC §97.1071, Special Program Performance; Intervention Stages, available at http://ritter.tea.state.tx.us/rules/tac/index.html. Also available on this rules link is the text of 19 TAC §97.1055, Accreditation Status, which describes how deficiencies in a district’s programs monitored under the PBMAS may affect a district’s accreditation status.
2014 Longview ISD Performance-Based Monitoring Analysis System (PBMAS)
Program Areas and Stages of Intervention
Bilingual Education / English as a Second Language (ESL)
Stage of Intervention = 2 Progress Monitoring
This component of the PBMAS system has over 37 indicators including specific subject areas within each indicator:
Career and Technology Education Monitoring
Stage of intervention = 2 Progress Monitoring
This component of the PBMAS system has 25 indicators including specific subject areas within each indicator:
In addition to the PBMAS intervention staging of a 2 (Progress Monitoring), Longview ISD has been identified to participate in a Program Access Review (PAR). LEAs that offer or administer CTE programs and receive funds from the U.S. Department of Education must meet certain requirements applicable to a CTE program. The Texas Education Agency has the regulatory responsibility to ensure that CTE programs in Texas public secondary schools are accessible to all students. The TEA is required to adopt a civil rights compliance program to identify, remedy, and/or prevent discrimination in CTE programs operated by LEAs. In accordance with this requirement, Longview ISD has been identified for a Program Access Review (PAR) of the Career and Technology Education Program. As a part of the PAR, a targeted on-site review by the TEA will be conducted to address student performance and program effectiveness concerns related to documented substantial, imminent, and/or ongoing risks as reflected in current and/or longitudinal district data and to ensure that Longview ISD complies with civil rights requirements for students served through CTE programs.
During the visit, Agency staff will conduct a program performance and effectiveness review related to the performance-based monitoring (PBM) system and a program access review (PAR). Through the PBM review, the on-site team will examine the LEA’s development of the most recent targeted improvement plan (IP); follow-up on implementation of the previous IP; review current data development and analysis processes; and evaluate program effectiveness. The PAR will also consist of a facility inspection and a review of the documentation to ensure compliance with program requirements. To conduct the on-site review the TEA team will:
The official findings of the visit will be presented to the LEA in a written report subsequent to the on-site visit. Actions required of LEAs will be tailored to area(s) of concern identified during the visit and will be designed to address performance, program effectiveness, and/or compliance issues and align with improvement planning processes.
Findings of the on-site review may result in continued implementation of the LEA’s current IP, revision of the LEA’s current IP, revision of the LEA’s current IP, development of a Corrective Action Plan (CAP) if applicable, additional LEA intervention activities, escalated agency oversight, and/or sanctions under the provisions of federal law, TEC, or TAC.
Findings related to the PAR will result in the development of an Action Plan for Program Access and oversight by the TEA related to completion of corrective activities. Information resulting from the PAR will be provided to the United States Department of Education, Office for Civil Rights (USDE-OCR).
No Child Left Behind (NCLB)
No stage of intervention.
This component of the PBMAS system has 16 indicators including specific subject areas within each indicator:
Stage of Intervention – Stage 3 Progress Monitoring
This component of the PBMAS system has 26 indicators including specific subject areas within each indicator:
Districts staged for interventions will engage in the TAIS continuous improvement process and develop a targeted improvement plan to address the identified root causes. The targeted improvement plan and quarterly progress reports will be submitted to the agency. The targeted improvement plan will include improvement strategies and interventions for all systems.
Attached are 2014 Performance-Based Monitoring Analysis System unmasked data tables as released by the Texas Education Agency. A copy of the PBMAS 2014 manual can be located on the TEA website using the following link http://www.tea.state.tx.us/pbm/PBMASManuals.html.