Slow Dance With Stakeholders: Setting the Pace in North Carolina, Part Two

A Date with Data

Reach out to us if you want to access Podcast resources, submit questions related to episodes, or share ideas for future topics. We’d love to hear from you!
You can contact us via the Podcast page on the IDC website at https://ideadata.org/.

### Episode Transcript ###

00:00:01.52 >> You're listening to "A Date with Data" with your host, Amy Bitterman.

00:00:07.34 >> Hey, it's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA data every day.

00:00:19.50 >> "A Date with Data" is brought to you by the IDEA Data Center.

00:00:25.00 >> Welcome to part two of "A Date with Data" with North Carolina's Kelley Blas and Lauren Holahan. If you missed part one, please make sure to check it out, and in this episode we're going to continue our conversation on how North Carolina has kept stakeholders meaningfully engaged in their SPP/APR.

00:00:45.40 So through all of this, what are some of the challenges that you've faced in getting stakeholders to attend? It sounds like you had maybe not that much of an issue with that. You had a lot of folks, but keeping them engaged and continuing to have them be part of this work.

00:01:07.60 >> How long have we got for this one?

00:01:14.43 >> Well, I'll just start with the first one, and then again pass it over to Lauren because there is quite a few challenges that we faced, but I would say our main one ... and I think most states would probably agree with this ... is parent involvement. A success in that is that, as Lauren mentioned earlier, we did have multiple meetings, and we scheduled the second meeting for each target setting group in the evenings. So it was a 6 o'clock meeting. That is where we had our most engaged parents, and they did ask lots of different questions that only parents would come up with, and their perspective was totally different than what we would find on our morning meetings, so that was a good thing. We just didn't feel like we had as many as we would have liked and as diverse as we would have liked as far as racially diverse, students with disabilities, the different disability categories. We would have liked to have seen more parents, and we're still working on that with our Parent Advisory Council and our other parent advocacy groups on involving more parents. And, Lauren, what do you think?

00:02:27.48 >> Yeah, yeah. I think another challenge is sustaining the effort and the momentum after we got the targets and the baselines selected, just sustaining the engagement because the conversation with stakeholders changes after that, right? After ...

00:02:46.22 >> Yep, absolutely.

00:02:47.24 >> ... that work is done, we have new SPP. Awesome. We ... But I think just keeping that momentum up especially for those stakeholders who aren't involved in a work group, so we've been talking. Kelley and I and our team have been talking about how to kind of recharge that external stakeholder group and ways to ... for us to kind of plan a little bit further out in terms of talking about progress, talking about ... thinking about some of the targets we potentially need to reset. Kelley, I wonder if you want to talk about just some of what we're experiencing in terms of multiple initiatives and leadership changes and ...

00:03:32.04 >> Oh, my goodness, yes. So along with having this huge task of a changing measurement table and getting this information out to our stakeholders and having them work through that with us, and I'm sure many states are in the same boat, but we also had a lot of initiatives going on within the department. Of course, we decided to change our SIMR as well, and so that was a huge change. We went from focusing on graduation rate to focusing really down on our fourth-grade reading and the opportunity gaps between students with disabilities and non-disabled students and between students with disabilities of color and students with disabilities who are white on our State-Identified Measurable Results. So that change alone is a huge shift in opportunity to engage all of our stakeholders, all of our districts in understanding what that looks like and the engagement that they need to have there. And we also did some major changes with our indicator 4, 9 and 10 process as far as reviewing and revising their policies, practices and procedures and what that looks like. And then there were some other division priorities that the PSUs needed to engage in that were very large compliance-related things that they had to roll out, so there was just so much happening in the past 2 years that there's ... I know that there are times, and they have told us directly that there are times when they just feel overwhelmed by the amount of things that are happening and changing when it comes to data and priorities. We also had significant staffing changes, lots and lots of long-term department folks have retired, and so training new staff and helping them understand what our priorities are and getting everybody onboard has been a huge challenge for us, and I'm sure, like I said, I think many other states are experiencing the staffing changes at least, if they're not experiencing all these other changes which some ... some of which we brought on ourselves, Lauren and I.

00:05:52.24 >> Yeah.

00:05:53.53 >> But you had long-term aspirations in mind. It's all working towards improvement and better outcomes so ...

00:06:01.62 >> I think one of the things that ... especially as we've done some revision to how we think about an approach and address indicators 4, 9 and 10 and significant disproportionality, I think what we've experienced is quite a bit of isolation by our local directors of special education in their context, and those indicators and significant disproportionality, those data, we have a friend in an LEA that says that general education is writing the check the EC has to back, right?

00:06:36.33 >> Mm-hmm, mm-hmm.

00:06:37.13 >> That it's general education policies and practices that are driving those particular indicators and significant disproportionality data, and yet the understanding of those requirements, what those data even mean on the general education side is pretty low, and so I think we've got a significant opportunity to do some capacity building with our gen-ed stakeholders at the local level.

00:07:08.64 >> Yeah, for sure, I think that's a common theme and something that I know, in the special ed world at least, trying to continue to push harder to get general ed involved in all of it but certainly those indicators. They really need to be at the table and understanding and part of all of those conversations and decision-making.

00:07:33.53 >> Kelley, will you just talk about sort of the balancing act, I think the challenge of keeping the SPP/APR the right size?

00:07:42.11 >> Well ...

00:07:42.42 >> For us data people ...

00:07:45.21 >> Right.

00:07:45.94 >> ... there's a cliff that we often run up to and like, "Whoa!"

00:07:49.96 >> Right. So, yeah, I know that for me specifically, I do focus on the end game and the targets and the outcomes, but it is important, and I think we've learned, especially through all this work and part of that challenge of all of the different things that we've been implementing, is that we're learning how each one of those indicators actually feed into the bigger picture of significant disproportionality of the policies, practices and procedures that are happening in districts, and so getting down to the student stories that the data represents. And Rachel Wilkinson did a wonderful job of showing us the journey of the indicators on a student level, and so we are really trying to help our districts understand that all of these play a part together and that the outcome or the percentage that they're at at the end is not as important as ensuring that their meeting the needs of each and every student.

00:09:02.79 >> And through all of those kind of challenges that you talked about, it still sounds like there were a lot of strategies and lessons learned and ways that you dealt with those challenges and really faced them. Can you share a few of those? Are there other strategies that you have found to really help keep stakeholders engaged that you can talk about?

00:09:26.52 >> So other than the work that we've already been doing with our stakeholders around the work groups, we actually have been planning to implement strategies to engage, and one of those is going back to that year-long plan or road map for discussing our indicators. So our intention is that we'll have an indicator highlight at each one of our stakeholder meetings and so that ... Again, so that we're not overwhelming them with, "Here's the progress on 17 indicators." We will highlight one or two indicators and just talk about it well, as talk about considering resetting some targets where we know that, based on the data and the timing of those changes, that OSEP [Indistinct] with the measurement table that we just totally missed the mark as far as us hitting those targets.

00:10:18.50 >> Well, I think ...

00:10:21.01 >> Example: indicator 3D.

00:10:22.91 >> Yeah.

00:10:23.24 >> Right.

00:10:23.36 >> Yes, but it was new and to ... I think probably all states missed the mark, and just you don't know what you don't know. It's going to take a few years, as we've seen, to really understand and get what's happening and where you want to go.

00:10:40.78 >> Yeah, so I guess what we learned in that was, if you don't know and it's a new measurement, choose conservatively.

00:10:52.03 >> Right. Yep.

00:10:54.40 >> Yeah, yeah. I think the other thing ... and we've mentioned it before, but even as we've kind of retooled and revisioned the way we're doing our local self-assessment for indicator 17 around fourth-grade reading outcomes and have sort of chunked up the work of that self-assessment into different phases this time, our LEAs just finished a data analysis, and as they move into root-cause analysis at the local level, supporting them, sharing their data analysis with their local stakeholders, what we're finding is we're just continuing to learn that there really is an art to the dosage of data that you share with any stakeholder group at any one time and just to be strategic about which data and how much and how much space to leave for people to respond to it. I think we're just learning, even ourselves and as we're coaching local implementers that it's ... When you're talking about data, it's super easy just to get on a monologue and just share all kinds of charts and graphs and then run out of time, and all you've done at that point is overwhelmed people.

00:12:15.00 >> Mm-hmm.

00:12:15.35 >> And to truly engage stakeholders, like that dosage thing we're still learning.

00:12:22.76 >> Yeah, definitely an art, and when you figure it out, let us know because it is. It's so true. Just knowing your audience as much as you can, and sometimes you really don't. If you're kind of putting a call out and parents and ... It's a whole range of people that have different levels of understanding and interest. It's definitely a challenge, but it sounds like you're really working hard to figure out what that mix is and what works, so I applaud you for that. You talked a little about upcoming plans already. Anything else that you haven't mentioned that you want to highlight?

00:13:01.85 >> Well, just before we move to that, I do ...

00:13:03.11 >> Yeah?

00:13:03.20 >> There's one other thing I think we're learning and which is also sort of ... informed the answer to this "What's next?" question is I think, as Kelley and I and our team just gets deeper and more familiar with how these collection of 17 indicators ... how they all inform one another ...

00:13:29.14 >> Mm-hmm?

00:13:29.42 >> ... how it really is, for all of sort of the headache and heartache sometimes these indicators cause, there is a wisdom and a brilliance to how they work together, and if we let them guide the work, things are going to get better for students with disabilities in our state. We are in full belief of that, and we're also learning as part of that coherence of these indicators that they can be used as a lever for other strategic priorities.

00:14:03.02 >> Mmm.

00:14:03.68 >> Specifically the work that we're leading across our agency with educational equity, I think letting these indicators and the requirements around them, letting them be a lever and not a hammer is one of the things, I think, that we're learning to do, and I'm really excited about what that's going to mean for sort of the energy and enthusiasm of the work going forward.

00:14:27.12 >> Yeah, that's really exciting and just hearing you ... It gets me happy listening to you saying that because it is so often we get so deep in the weeds of the indicators and not thinking about how they work together. How can we really be using these to our advantage and to really make a difference and not looking at it as just like, "Ugh, this something that we have to do every year," and you just do it for the purpose of doing it because it's required?

00:14:56.94 >> Right. We've even, in our more giddy moments, kind of thought about the SPP as our superpower rather than as our beast of burden, right?

00:15:06.47 >> Yeah, yes. Yes, absolutely.

00:15:09.26 >> As far as our plans to continue engaging stakeholders, again, like we said, we really want to focus on increasing our number in diversity of stakeholders, and we just continue to add to our internal and external stakeholder meetings, and those connections that we're making are becoming more broad-reaching. So just different advocacy groups that we may ... hadn't thought of before have been reaching out, and so that's exciting to us, and we're hoping to just see those meetings grow. And as we already said, we're going to be doing an indicator progress highlight at each of those meetings just to keep those indicators front and center and really for them to understand the progress and how it works together with the other work that we present at those meetings around our SIMR and around our educational equity work. So we'll try to weave that in so that it really does make sense with the work that we're doing. And also one of the things that we have been doing and will continue to do is that we will analyze all of our indicators by subgroups, even the ones where OSEP doesn't require us to break it down by race or by disability or by other subgroups, and we've taken advantage, probably too much advantage, but using tools like Tableau and providing dashboards to our districts, so that they can see their data broken down in a different way and they're not having to create their own charts and things like that of data that we already have. Lauren kind of had the light bulb moment as we were considering giving our districts their data for their State-Identified Measurable Result. "Can't we do this in a dashboard?" And once we did, it was life-changing, not only for us but for our PSUs, and so continuing to provide data in an easily accessible readable format and then making changes when they ask for it has been super helpful.

00:17:09.24 >> Yeah, I think we should have said at the beginning, I think one of our most important and cherished stakeholder groups is our local Directors of Special Education.

00:17:19.99 >> Mm-hmm.

00:17:20.35 >> We do, in addition to our sort of formal external stakeholder group for the SPP, we have other formats for engaging those directors as a stakeholder group, multiple formats, and I think we continue to ... We're continuing to evolve in understanding how to use our indicator data as an indicator of the level of a support a given LEA would need, right?

00:17:49.25 >> Mm-hmm.

00:17:49.49 >> Sort of like what DIBELS is to MTSS, for us the indicators are to the proximity and the intensity of our general supervision, and we still have a long ways to go in doing that in a more systematic and organized kind of tiered level of support, but we're on our way and being able to conceptualize the indicator data in that way, I think, is going to be very organizing. I think the ... Finally, the thing that I think we're learning and are going to spend time talking about and focusing on with stakeholders across all different kinds of our stakeholder groups is really thinking about collective efficacy and our belief systems, and I say that because I think until we have a broad and consistent culture of high expectations across all eds and specifically for students with disabilities, I think we're going to struggle to see these data move in a meaningful and lasting way, and that's a really different kind of work, implementation science or improvement science.

00:19:03.64 >> Mm-hmm.

00:19:04.50 >> We're in the midst of quite a bit of research around, what are the evidence-based practices for building collective efficacy and for addressing belief systems and ... Yeah, we're excited about where that might take us.

00:19:18.18 >> Yeah, it's a whole culture shift.

00:19:20.51 >> Yes.

00:19:20.89 >> Wow! Well, you all have a tremendous amount going on, and I'm so excited about all of it. This has been amazing. Thank you for sharing. I think a lot of other states out there will really learn a lot and be able to take and ... some of the examples and strategies and try applying them themselves, so thank you, both, for being on.

00:19:45.64 >> Aw, thanks so much for letting us blather.

00:19:48.44 >> Yeah, anytime, anytime. Thank you.

00:19:52.33 >> Yeah, I think that any state that's crazy enough to do the kinds of things that our state has done in the past couple of years, we're here to support that.

00:20:02.06 >> Yes.

00:20:02.23 >> Wonderful. Thank you, both, so much for being on and sharing with us your SPP/APR journey and how stakeholders have been a part of the process from the start and have continued and will be continuing into the future.

00:20:18.45 >> Thank you.

00:20:18.90 >> Thanks, Amy.

00:20:21.06 >> To access podcast resources, submit questions related to today's episode, or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content, or connect with us via the podcast page on the IDC website at ideadata.org.