Major concerns about Kingsley surveys

My name is Lauren McNamara and I am writing on behalf of the Legion of Data Nerds with concerns on the two surveys sent to the Kingsley community on Friday 11/7. Dr. Turner sent a Boundary Input Survey at 2:00pm and Principal Davis sent a Preparing for Possible School Closure Survey at 4:00 pm. For context, I have 11 years of experience working in survey research at a top nonpartisan research organization, and I have direct experience in working to pull insight out of open-ended responses.

I share the feedback below in the interest of helping the district do better on any upcoming surveys.

I want to first note that sending two separate surveys two hours apart reflects a surprising lack of coordination. Response rates to both surveys will be reduced and feedback from the Kingsley community in this impactful time will be suppressed. This is not the way to collect community feedback.

My primary concerns are with the Kingsley Community Boundary Input survey sent by Dr. Turner.

  1. At the last board meeting on November 3, the group discussed what primary factor the boundary optimization model should be built on. I wonder if this survey was constructed to identify this factor(s) for the unnamed “university partner” building these models. If so, there are likely a limited number of concepts encoded as optimization options into this algorithm and these should be asked directly as ranked choice options instead of through open-ended responses.
  2. The survey introduction simply states, “Please complete this short survey to provide input on potential school boundary changes.” There is no background information provided on what the potential boundary changes under consideration are. There is no notice that current closure scenarios are for two north side schools. This process is being forced through at lightning speed and many families likely are not aware of the latest developments being considered. It is unfair to ask their thoughts without the context of the current plan.
  3. The single open-ended response is problematic for many reasons.
    1. Participants are forced to come up with concerns themselves for a closure process moving at blinding speed. Parents may not be caught up on the latest implications of the latest maps. Displaying a finite list of closed-ended options with additional space given for other responses would reflect that the district has heard community feedback already provided and is weighing the options; this method indicates the district does not know where to start.
    2. It is now up to the district to make sense of coding these open-ended responses into discrete concepts. This is a difficult task which takes time (which we don’t seem to have) and relies on an unbiased researcher. This methodology choice means the district will need to be incredibly transparent on how these responses were coded including sharing the raw responses with the public so that the coding can be verified.

Additionally, for both surveys:

  1. There is no information provided on when the survey closes. Parents should be given a timeframe in which they are asked to complete their response. The SDRP process is highly accelerated and it’s likely parents don’t have a long window to respond, but this is not made clear.
  2. While each survey was only sent directly to the Kingsley community, the link is able to be shared and anyone with the link can complete the survey. These surveys are directed at Kingsley and should be limited to Kingsley respondents. How will you ensure quality control to verify that only eligible survey respondents’ answers are incorporated?
  3. Both surveys do not limit the number of responses per respondent. This makes it impossible to accurately measure the Kingsley community’s opinion.
  4. It is not clear how the data will be used by the school or by the administration. This needs to be articulated to potential respondents to ensure relevant responses that are within scope. Many potential respondents will not have an intuitive sense of how this data might be useful.

The Boundary Input survey appears to be a quick and dirty way to collect feedback. This is no way to define school boundaries. School boundaries should be set based on a thoughtful and considered vision for our district. Instead of helping answer questions with data insights and inform policy, this rushed and opaque survey effort will likely result in more distrust and unanswered questions.

I ask the district to slow down to gather data on financial savings already in progress and not send rushed surveys while forcing school closures.

Thank you,

Lauren McNamara
Lauren McNamara has an MS in Biostatistics from the University of Illinois at Chicago and has 20 years of professional experience working with data, including 11 years in survey research.
laurenmcnamara@gmail.com

Kelly McCabe
Kelly McCabe holds an MSPH in Public Health and Tropical Medicine from Tulane University and has over 20 years of professional experience working with data as an epidemiologist focused on community health research.
mccabekelly322@gmail.com

Esteban J. Quiñones
Esteban J. Quiñones holds a Ph.D. in Agricultural and Applied Economics from the University of Wisconsin-Madison and has 20 years of professional experience working with data in global development for evidence-informed policymaking.
estebanjq@gmail.com

Eric Shore
Eric Shore holds a BSE in Computer Science (University of Pennsylvania) and an MBA (UIUC). With 19 years of professional experience in business analytics and data-driven decision-making, he currently serves as Chief Innovation Officer at a technology company.
emshore@gmail.com

Alexis R. Lauricella
Alexis Lauricella holds a Ph.D. in Developmental Psychology and a Masters of Public Policy from Georgetown University. With more than 20 years of professional experience in qualitative and quantitative research related to children’s learning from technology, she currently serves as a Kids and Families Researcher at a large technology company.
alexislauricella@gmail.com

Jean Clipperton
Jean Clipperton holds a PhD from the University of Michigan and has 19 years of professional experience working with data.
jean.clipperton@gmail.com