Student Performance & Accountability
Performance Based Monitoring and Analysis System

The Performance-Based Monitoring System was implemented to move special program monitoring from a cyclical, compliance-based, on-site monitoring system to data-driven results-based, system with targeted interventions in the areas of special education, No Child Left Behind (federal Title programs), Career and Technical Education, and Bilingual Education/English as a Second Language. Some underlying strategies of the PBM system include a shift away from process to results (i.e. program effectiveness and student performance), a strong emphasis on data integrity, a focus on a coordinated approach to agency monitoring, and a more creative application of sanctions and interventions.
Performance Based –
The system is designed to improve student performance and program effectiveness, and to address financial and program compliance concerns within the context of improving student performance.
 
Data Driven –
Comprehensive data collection systems enable TEA to evaluate student performance and program effectiveness at the state level and to conduct on-site visits when necessary and appropriate.
 
Comprehensive Evaluation with Targeted Interventions –
Every program TEA is authorized to monitor is evaluated every year in every district. Graduated interventions targeted to specific areas of concern engage more districts in improvement activities.
 
Coordination and Consistency –
Indicators and standards are aligned across evaluation systems with coordinated interventions appropriate to the areas of concern and consistent across programs.
 
Performance-Based Monitoring Analysis System (PBMAS)
School districts receive annual performance information through the PBMAS, a set of performance and program effectiveness indicators for special programs TEA is required by state or federal statute to monitor.
 
The following programs comprise PBMAS:
  • Special Education
  • Bilingual Education/English as a Second Language
  • Career and Technical Education
  • No Child Left Behind (NCLB)
                Economically Disadvantaged
                Migrant
                Limited English Proficient
                Safe and Drug-Free Schools.
 
The Division of Performance-Based Monitoring is responsible for developing the Performance-Based Monitoring Analysis System (PBMAS) which is an automated data system that reports annually on the performance of school districts and charter schools in selected program areas including:
  • bilingual education/English as a second language,
  • career and technical education,
  • special education, and
  • certain programs under the No Child Left Behind Act
From the data contained in the PBMAS, the division also designs and maintains the annual Performance-Based Monitoring Analysis System Summary Report. This is a district-level report that includes specific data for each performance indicator in the PBMAS.
 
The Division of Performance-Based Monitoring is also responsible for developing and reporting on a variety of data validation indicators, including indicators to examine leaver/dropout records, discipline data, and student assessment data.
 
  • Data Validation Monitoring - Leavers
  • Data Validation Monitoring - Discipline
  • Data Validation Monitoring - Student Assessment

Summary of 2013 PBMAS Changes


The 2013 PBMAS includes several key changes from the 2011 system. Changes to the PBMAS indicators for 2013 are marked in the manual as “New!” for easy reference. The manual is listed below.

  • The transition to the new State of Texas Assessments of Academic Readiness (STAAR) is reflected in the 2013 PBMAS with the implementation of certain STAAR Grades 3-8 indicators as appropriate. (See Transition to STAAR in Section II of this manual for more detailed information.) Cut-point adjustments have been made for the first time to all Recommended High School Program and Distinguished Achievement Program (RHSP/DAP) indicators2. Cut-point adjustments were also made to all Annual Dropout Rate indicators. In addition, changes have been made within each program area:
Bilingual Education and English as a Second Language (BE/ESL)
  • English and Spanish assessment results will no longer be evaluated and reported separately. The (formerly) four separate indicators that evaluated BE English, BE Spanish, ESL English, and ESL Spanish assessment results have been combined into two separate indicators: one to evaluate BE results and one to evaluate ESL results.
  • The Limited English Proficiency 3 (LEP) Student Participation Rate indicator has been suspended for the 2012 PBMAS and may be redeveloped in the future based on the new testing requirements for English Language Learners (ELLs) adopted under Texas Administrative Code (TAC): 19 TAC Chapter 101, Subchapter AA.
  • The LEP Annual Dropout Rate (Grades 9-12) indicator has been deleted. During its three years as a Report Only indicator, data analyses at the district and state level determined the indicator to be unnecessary, particularly given that the LEP Annual Dropout Rate (Grades 7-12) indicator which evaluates nearly 70,000 more students annually, including all of the students evaluated in the Grades 9-12 indicator, will continue to be implemented in the 2012 PBMAS.
  • The methodology and calculation for BE/ESL #8, which evaluates the extent to which ELLs assessed on the Texas English Language Proficiency Assessment System (TELPAS) reading test score at the Beginning proficiency level for multiple years, has been revised to take into account performance regression. As a result of this change to the indicator, it will become a Report Only indicator for the 2012 PBMAS.
Career and Technical Education (CTE)
  • The CTE Tech Prep Texas Assessment of Knowledge and Skills (TAKS) Passing Rate indicators have been suspended for the 2012 PBMAS but may be reinstated in the future should federal Tech Prep funding be restored.
  • Significantly revised CTE course offerings were effective beginning with the 2010-2011 school year. Therefore, CTE #8 (CTE Nontraditional Course Completion Rate-Males) and CTE #9 (CTE Nontraditional Course Completion Rate-Females) will revert to Report Only indicators for the 2012 PBMAS. The courses used for these indicators in the 2012 PBMAS can be found in Appendix A of this manual.
No Child Left Behind (NCLB)
  • For the 2012 PBMAS, there are no changes specific to the NCLB program area. However, indicators in this program area have been modified based on the overall changes described above that will affect all PBMAS program areas in 2012.
Special Education (SPED)

  • Cut-points have been adjusted for six indicators to reflect more rigorous expectations: the Representation indicator (SPED #12), two of the Less Restrictive Environments (LRE) indicators (SPED #6, #7), the Discretionary Disciplinary Alternative Education Program (DAEP) Placements indicator (SPED #16), the Discretionary Placements to In-School Suspension (ISS) indicator (SPED #17), and the Discretionary Placements to Out-of-School Suspension (OSS) indicator (SPED #18).
  • With two years of comparable data based on the new federal race/ethnicity categories available, the components of required improvement, special analysis, and aggregation across two years to meet minimum size requirements have been added to SPED #13 (African American [Not Hispanic/Latino] Representation) and SPED #14 (Hispanic Representation).

Planning for the Future: 2014 and Beyond


It is anticipated that ongoing development of the PBMAS will include the addition of new indicators, revision of current indicators, and deletion of indicators that are no longer necessary. Factors independent from the PBMAS itself are also likely to have an impact on the future versions of PBMAS. These factors include the following:
  • development and implementation of the new statewide assessment program, STAAR;
  • development and implementation of a new ELL progress measure for STAAR;
  • reauthorizations of federal statutes, including NCLB;
  • results of any USDE reviews, guidance, or regulations;
  • changes to data collection definitions and processes; and
  • legislation from a regular or special legislative session.

Standards and Performance Levels

A performance level (PL) is the result that occurs when a standard is applied to a district’s performance on an indicator. The performance levels available for indicators in the 2011 PBMAS include Not Assigned (NA) (including Not Assigned through SA, PJSA, or DI), 0, 0 SA, 0 PJSA, 0 RI, 1, 1 SA, 1 PJSA, 2, 2 SA, 2 PJSA, 3, 3 SA, or 3 PJSA.
  • A performance level of 0 is the highest designation for any indicator, meaning the district met the standard for the indicator. (The Best Designation)
  • A performance level of 3 is the lowest designation, indicating the district performance was farthest from the 0 - Met Standard designation. (The Worst Designation)
Reminder on Acronyms
  • PL = Performance Level
  • RI = Required Improvement
  • SA = Special Analysis
  • PJSA = Professional Judgment Special Analysis
  • LEP = Limited English Proficient
  • BE = Bilingual Education offered in LISD grades Pk-5
  • ESL = English As a Second Language offered in LISD grades 6-12

Types of Standards

Absolute standards are tied to an absolute requirement or goal that all districts have the possibility of achieving each year. The state accountability system provides absolute standards to which PBMAS standards can be aligned for certain TAKS performance indicators.
 
The standards, by subject, for a rating of Academically Acceptable in the 2011 state accountability system are as follows:
  • Mathematics 70%
  • Reading /ELA 70%
  • Science 65%
  • Social Studies 70%
  • Writing 70%
Standards for PBMAS TAKS performance indicators are aligned with these state accountability standards. A district that achieves the Academically Acceptable performance standard for a TAKS subject-area test receives a PBMAS performance level assignment of 0 – Met Standard for that TAKS subject-area indicator in PBMAS. In standard PBMAS analysis, a district with performance up to 10.0 percentage points below the state accountability standard receives a performance level assignment of 1, and a district with performance 10.1 to 20.0 percentage points below the state accountability standard receives a performance level assignment of 2. Any district with performance 20.1 or more percentage points below the state accountability standard receives a performance level of 3, the lowest designation in the PBMAS.
 

Transition to the State of Texas Assessments of Academic Readiness (STAAR)

As student assessment indicators in the PBMAS are redeveloped from TAKS-based indicators to STAAR-based indicators, it will be necessary for the PBMAS to include several transitional elements that will allow the system to continue to evaluate student performance data while the new STAAR assessments and performance standards are phased in. For 2012, the key transitional elements are:


Use of the Met Standard performance standard for TAKS on the Grades 3-8 STAAR assessments. During the transition to the new STAAR, performance standards will not be available for most STAAR assessments in time to assign PBMAS performance levels in the summer of 2012. Instead, for 2012 only, the Met Standard performance standard for TAKS will be used for the STAAR-based student assessment performance indicators in the BE/ESL, NCLB, and SPED program areas. Students who meet the TAKS Met Standard performance standard on the relevant STAAR assessments will be counted in the indicators’ numerators as performing on the STAAR subject test at the TAKS equivalency performance standard. Detailed information about the process to identify the Met Standard performance on TAKS for the STAAR assessments is available at http://www.tea.state.tx.us/student.assessment/reports/. (For the 2012 PBMAS, student assessment performance indicators in the CTE program area will continue to be based on TAKS data and TAKS performance standards for Grades 10 and 11.)

Monitoring Interventions

The Program Monitoring and Interventions (PMI) Division notifies each district selected for a 2011 PBMAS intervention. Performance-based monitoring interventions determinations are based on PBMAS performance levels and other criteria as well as results of the data validation analyses. Sanctions are applied as necessary and according to state and federal law and Texas Administrative Code (TAC), including, but not limited to, Texas Education Code, Chapter 39.
Additional information related to performance-based monitoring interventions is reflected in 19 TAC §97.1071, Special Program Performance; Intervention Stages, available at http://ritter.tea.state.tx.us/rules/tac/index.html Also available on this rules link is the text of 19 TAC §97.1055, Accreditation Status, which describes how deficiencies in a district’s programs monitored under the PBMAS may affect a district’s accreditation status.
 

2013 Longview ISD Performance-Based Monitoring Analysis System (PBMAS)
Program Areas and Stages of Intervention

Bilingual Education / English as a Second Language (ESL)

Stage of Intervention = 1

The highest stage of intervention, Stage 4, is assigned to LEAs with the lowest student performance levels. The Longview ISD has received a stage of intervention for the BE/ESL program for seven consecutive years, beginning with the 2004-2005 school year, as shown below:

  • 2012-2013 Stage 1:
  • 2011-2012 Not Staged for Interventions:
  • 2010-2011 Stage 4 with on-site Performance Effectiveness Review (PER);
  • 2009-2010 Stage 3;
  • 2008-2009 Stage 3 Year after On-site;
  • 2007-2008 Stage 4 with On-site;
  • 2006-2007 Stage 2;
  • 2005-2006 Stage 1B; and
  • 2004-2005 Stage 2.

This component of the PBMAS system has over 37 indicators including specific subject areas within each indicator:

  1. BE/ESL #1(i-v) — BE STAAR Passing Rate
  2. BE/ESL #2(i-v) — ESL STAAR Passing Rate
  3. BE/ESL #3(i-v) — LEP (Not Served in BE/ESL) STAAR Passing Rate
  4. BE/ESL #4(i-v) — LEP Year-After-Exit (YAE) English STAAR Passing Rate
  5. BE/ESL #5 — LEP Annual Dropout Rate (Grades 7-12)
  6. BE/ESL #6 — LEP Recommended High School Program or Distinguished Achievement Program (RHSP/DAP) Diploma Rate
  7. BE/ESL #7 — LEP Graduation Rate
  8. BE/ESL #8 —TELPAS Reading Beginning Proficiency Level Rate
  9. BE/ESL #9 —TELPAS Composite Rating Levels for Students in U.S. Schools Multiple Years.

The areas identified by this monitoring system were:

BE/ESL #1(i-v) — BE STAAR Passing Rate   

  • Reading - which was identified with a PL of a 1;
  • Science - which was identified with a PL of a 3;
  • Writing - which was identified with a PL of a 2;
BE/ESL #2(i-v) — ESL STAAR Passing Rate

  • Math - which was identified with a PL of a 2;
  • Reading - which was identified with a PL of a 2;
  • Science - which was identified with a PL of a 2;
  • Writing - which was identified with a PL of a 3
BE/ESL #3(i-v) — LEP (Not Served in BE/ESL) STAAR Passing Rate
  • Math - which was identified with a PL of a 1;
  • Reading - which was identified with a PL of a 2;
  • Writing - which was identified with a PL of a 2
BE/ESL #4(i-v) — LEP Year-After-Exit (YAE) English STAAR Passing Rate 
  • Science - which was identified with a PL of a 1;
  • Writing - which was identified with a PL of a 2;
BE/ESL #8 —TELPAS Reading Beginning Proficiency Level Rate - which was identified with a PL of a 1;

BE/ESL #9 —TELPAS Composite Rating Levels for Students in U.S. Schools Multiple Years - which was identified with a PL of a 1;


Career and Technology Education Monitoring
Stage of intervention = 3

 This component of the PBMAS system has 25 indicators including specific subject areas within each indicator:

  1.  CTE #1(i-iv) — CTE STAAR EOC Passing Rate

2.     CTE #2(i-iv) — CTE LEP STAAR EOC Passing Rate

3.     CTE #3(i-iv) — CTE Economically Disadvantaged STAAR EOC Passing Rate

4.     CTE #4(i-iv) — CTE Special Education STAAR EOC Passing Rate

5.     CTE #5— CTE Annual Dropout Rate (Grades 9-12)

6.     CTE #6— CTE RHSP/DAP Diploma Rate

7.     CTE #7— CTE Graduation Rate

8.     CTE #8 — CTE Nontraditional Course Completion Rate—Males

9.     CTE #9 — CTE Nontraditional Course Completion Rate—Females

 The areas identified by this monitoring system were:

·       Indicator #3 CTE Economically Disadvantaged STAAR EOC Passing Rate

o   Social Studies- which was identified with a PL of a 1;

·       Indicator #4 CTE Economically Disadvantaged STAAR EOC Passing Rate

o   Math - which was identified with a PL of a 3;

o   Science - which was identified with a PL assignment of a 1;

o   Social Studies - which was identified with a PL assignment of a 2

·       Indicator # 8 CTE Non Traditional Course Completion rate – Males; which was identified with a PL of a 1;


No Child Left Behind (NCLB)
No stage of intervention.

This component of the PBMAS system has 16 indicators including specific subject areas within each indicator:

  1. NCLB #1(i-v) — Title I, Part A STAAR Passing Rate at TAKS equivalency
  2. NCLB #2 — Title I, Part A Annual Dropout Rate (Grades 7-12)
  3.  NCLB #3— Title I, Part A RHSP/DAP Diploma Rate
  4.  NCLB #4— Title I, Part A Graduation Rate
  5. NCLB #5(i-v) — Migrant STAAR Passing Rate
  6. NCLB #6— Migrant Annual Dropout Rate (Grades 7-12).
  7. NCLB #7—Migrant RHSP/DAP Diploma Rate 
  8.  NCLB #8— Migrant Graduation Rate

All indicators for this component were rated at the lowest PL level assignment of 0.


Special Education

Stage of Intervention – Stage 1

 This component of the PBMAS system has 26 indicators including specific subject areas within each indicator:

1.     SPED #1(i-v) — SPED STAAR Passing Rate at TAKS Equivalency

2.     SPED #2(i-v) — SPED Year-After-Exit (YAE) STAAR Passing Rate at TAKS Equivalency

3.     SPED #3 — SPED STAAR Participation Rate

4.     SPED #4 — SPED STAAR-Modified Participation Rate 

5.     SPED #5 — SPED STAAR-Alternate Participation Rate 

6.     SPED #6 — SPED 3-5 Year-Olds Less Restrictive Environments Placement Rate 

7.     SPED #7 — SPED 6-11 Year-Olds Less Restrictive Environments Placement Rate 

8.     SPED #8 — SPED 12-21 Year-Olds Less Restrictive Environments Placement Rate 

9.     SPED #9 — SPED Annual Dropout Rate (Grades 7-12) 

10.  SPED #10 — SPED RHSP/DAP Diploma Rate 

11.  SPED #11 — SPED Graduation Rate 

12.  SPED #12 — SPED Representation

13.  SPED #13 — SPED African American (Not Hispanic/Latino) Representation 

14.  SPED #14 — SPED Hispanic Representation

15.  SPED #15 — SPED LEP Representation

16.  SPED #16 — SPED Discretionary Disciplinary Alternative Education Program (DAEP) Placements

17.  SPED #17 — SPED Discretionary Placements to In-School Suspension (ISS)

18.  SPED #18 — SPED Discretionary Placements to Out-of-School Suspension (OSS)

The activities required under a Stage 1 (continuing Implementation) intervention are:

LEAs with a Stage 1 intervention for Sped are not required to submit intervention activity forms unless selected for submission by TEA or noncompliance has been identified.

The areas identified by this monitoring system were:

·       Indicator #1 (Math, Reading, & Science); which were each identified with a PL of a 1, (Writing) which was identified with a PL assignment of a 3;

·       Indicator # 3; (Math) which was identified with a PL of a 2, (Science) which was identified with a PL assignment of a 1;

·       Indicator # 4; Which was identified with a PL assignment of a 2;

·       Indicator # 5; Which was identified with a PL assignment of a 2;

·       Indicator # 6; Which was identified with a PL assignment of an AR;

·       Indicator # 7; Which was identified with a PL assignment of a 2;

·       Indicator # 15; Which was identified with a PL assignment of a 1;

·       Indicator # 18; Which was identified with a PL assignment of a 3;

·       Indicator # 23; Which was identified with a PL assignment of a 2;

 

Attached are 2013 Performance-Based Monitoring Analysis System unmasked data tables as released by the Texas Education Agency.  

A copy of the PBMAS 2011 manual can be located on the TEA website using the following link http://www.tea.state.tx.us/pbm/PBMASManuals.html.



Any Questions related to the 2012 PBMAS Report may be directed to
Rebeca Cooper, Director of Research Accountability and Planning
Longview ISD
903.381.2214