A conversation with Paula Palermo, Data Director of Norwalk ACTS
Q: What is data? What would you like our readers to understand about Data?
A: Data is defined as facts about something that can be used in calculating, reasoning, or planning. It is critically important we all understand how Norwalk ACTS uses data and what my role as Data Director entails. The five messages below are a good place to start:
1. The goal of Norwalk ACTS is to be the connective tissue between our community level outcomes on the cradle to career continuum. In order to do this, we must build a data infrastructure that enables better decision making in support of all of Norwalk’s children.
2. Norwalk ACTS is not a program provider. We are a convener of organizations with the goal of sharing knowledge, data and best practices.
3. Data is used to inform organizations. Student data is used only by programs that actually serve the child and not shared with anyone outside that sphere.
4. The purpose of collecting data is to turn it into information that drives action. The data must be relevant to the decision making process.
5. Data is to be used as a flashlight, not a hammer. Learning about a program’s strengths and weaknesses helps us to lift up what’s working and to move away from what isn’t.
Q: What is the biggest challenge when it comes to turning data into useful information that drives action?
A: There are many challenges. We have so much academic data but unfortunately much of the data sits alone in its own data system. If you are looking for demographic data, you go to one system. If you are looking for English Language Arts data, you go to another, etc. Data from one system is not always matched with data from other systems to drive improvement.
Another challenge is compiling historical data- not just data in a snapshot in time, but data that reaches back to the beginning of a student’s education. It is important that we consider not just where students are now, but what their growth pattern has been, and that we keep velocity with that growth.
A third challenge is collecting data around non-academic indicators. For instance, how do we get quality data around healthy eating, dental screening, and exercise? There is no central database that can help us identify wellness or social/emotional indicators.
Q: So where do we begin in terms of collecting non-academic data?
A: Before we do anything with data, we need to know what data our Norwalk ACTS partners care about, and what data actually speaks to the impact we are trying to create in our community. For example, Norwalk ACTS aims to ensure every Norwalk child is Kindergarten-ready. The real work comes in the process of determining what measurements indicate Kindergarten readiness. This definition is what informs the data collection.
Q: What role does data play in determining the root causes of why a student might be struggling or not performing at grade level?
A: A child might not be reading at grade level because he or she has a learning disability, not because he or she is lazy or unmotivated. But there may be other reasons. Is there a wellness issue? Is it related to social/emotional development? Is there something that is happening at home that might be a factor? While we cannot fix every issue, data can help provide the evidence that will allow us to address these challenges.
Q: What does the data tell us about programs?
A: Some children thrive in one program but may not in another. We use data to learn about what is working and what is getting results, and then use these data-proven best practices to help others do better. Data can help us define the standard that we want all programs to rise to.
Q: How is Norwalk ACTS encouraging its member organizations to use data in their work?
A: First and foremost, it is critical that member organizations have a better way to share student and programmatic data. In a perfect world, child service providers would have access to all of a child’s relevant academic, wellness and social/emotional data in order to better serve them. One of our goals is to help build a data system infrastructure that would enable better communication of relevant data between in-school and out-of-school providers.
Q: How does the process work? What is evidence-based decision making?
A: As the Data Director for Norwalk ACTS, it is important for me to analyze the data and present it to our members in a clear, concise and informative way – not just with a single data point, but also with the story behind the data point. Members will then make decisions about how the data can be used within their own organizations in order to change behaviors and systems to achieve the agreed upon outcomes.The process of evidence-based decision making is:
– Identify the cohort we want to serve
– Collect baseline data for a valid, agreed-upon indicator of achievement
– Implement an Action Plan based on that baseline data – an intentional process for improvement for that cohort
– Collect updated data
– Use the data as evidence of improvement (or not) of the outcome indicator
– Repeat
Q: How do data discussions turn into action plans?
A: A data discussion is a conversation between team members who work in the “sphere of influence” for that outcome. We try to set aside assumptions (correct and or not) about any number of things – from the cause of problems to how to solve those problems. Then we look at sources of available data that provide measurable indicators. We use the data to uncover the barriers to success. The discussion, or Action Plan, is built around the best ways to remove these barriers. It’s not trivial work. It takes time to come to an agreement on what the indicators are and for everyone to agree on them. Often, what we believed in the beginning of the process is not necessarily what we believe now. It also takes time to implement changes that can move the needle on improvement. But we learn something at every step of the process.
Q: Are there examples where the Norwalk early childhood community has used data in this way?
A: Yes. The use of PELI (Preschool Early Literacy Indicators) has provided assessment data for 3- to 5-year-olds that measures alphabet knowledge, vocabulary and oral language, phonemic awareness, and listening comprehension. The analysis of this data gives PreK providers a way to identify how prepared for kindergarten their children are in terms of literacy competency.
Children identified as Well Below Benchmark, or in the “Red Zone,” became the cohort we wanted to provide direct action to support. The Action Plan for Red Zone children includes intervention and progress monitoring to help identify the root causes of their struggles and has driven instruction to help them succeed.
For example, the 2014-15 end of the year PELI assessment showed 187 children entered Kindergarten in the Red Zone. By putting programs and interventions in place that were targeted toward assisting this cohort of children, only 28 of the 187 children left Kindergarten in the Red Zone. This is an example of data driven progress. Moving forward, we want to see fewer children graduate from PreK in the Red Zone.
This information also helps inform the Kindergarten teachers about the literacy needs of their incoming students. Parents also have the opportunity to work with their child’s PreK provider to assist them in better understanding the literacy skills of their child and to acquire tools they can use at home in order to support their child.
Q: What other initiatives are Norwalk ACTS currently focused on?
A: TEAM Summer is a group of Summer Learning Providers and Norwalk Community members who want to ensure that children in Norwalk are receiving the best support possible during the summer months. We are interested in increasing the capacity of our city to provide quality opportunities for all of the city’s children because what children spend time doing during the summer can potentially increase their success during the school year.
Q: What role can data play in preventing “Summer Slide?”
A: Without data, targeted intervention is a shot in the dark. Without knowing who needs help, how can we provide children with the best support possible? There are approximately 14,000 children in Norwalk ages 3 to 18 and only 3,000-3,500 engage in some type of summer programming.
Analyzing assessment data collected before and after a summer learning program, an assessment of the type and length of a program, and a student’s actual attendance will enable us to strengthen the summer offerings with better alignment with district goals in order to keep kids learning. The earlier we intervene and enable all students and programs to succeed, especially at the early stage of the cradle to career continuum, the less intervention that will be needed later on.
Q: How do you protect the privacy of student data?
A: Data privacy is taken very seriously. Norwalk ACTS has a data sharing agreement with Norwalk Public Schools that outlines the boundaries of student data use, adhering closely to FERPA and HIPAA guidelines. Norwalk ACTS does not do business with any outside companies. Data is only shared between members with parental consent. An example of this is a summer provider sharing their class list with the district in order to get academic assessment data and to provide focused instruction for each student. Data analysis is only published in the aggregate form and is not identifiable at the student level.
Q: What else do we, as a community, need to take into account in our work to better the lives of our children?
A: Academic data only tells part of the story. There are so many other factors that contribute to student success. Poverty, housing and food instability, language barriers, lack of support structures, illness, and lack of transportation are just a few obstacles that we hope to identify and address in order to raise the bar for all children, at every step along the way. Our goal is to use all relevant indicators to drive action to make Norwalk the healthiest community for children to grow up in.