Indigo Stelian & Will Nuelle 

For one reason or another, the Independent School Health Survey (ISHS) doesn’t always seem as good of a measure of our population as it should be. Whenever the results are released, students are surprised by how poorly the statistics represent the environment. We wanted to figure out how much can we trust the survey results.

 To start, we had to get our own data on what we thought were the most important statistics on the Health Survey in determining how valid it is. The statistic from the Health Survey that emerged as the most relevant in trying to answer our question was that that 67% of people answered completely truthfully to the Health Survey. Therefore we created a survey that asked these two questions:

1. Did you respond honestly to all the questions on the online Health Survey you took in October of this year? Yes or No.

2. While taking the Health Survey, did you feel that you could be identified through your answers? Yes or No.

 The next question was how to distribute this survey with as little bias in the results as possible. The best way to do this is to stratify by grade because we recognized the survey could elicit very different results based on age. We took a random sample of four advisories, one from each grade, and polled every student in the advisory. Of the 31 people sampled, 28 responded “Yes” on the first question and 8 responded “Yes” on the second question.

 For the first question, there is less than 1% chance that we would get results as extreme as those found if the claim was true. Therefore we can reject the claim that 67% of students answered truthfully on the ISHS. The Health Survey got it wrong: the proportion of people who answered completely honestly is not 67%! How can we trust a survey that doesn’t even properly estimate how honest people were on it? In truth, no one has any idea what percentage of people answered completely honestly on the Independent School Health Survey, making it kind of useless.

 For the second question, we are 90% confident that the true proportion of students who felt like they were anonymous when taking the ISHS is between 0.6104 and 0.8696. That means that somewhere between 13% and 39% of students didn’t feel like they were anonymous when taking the Survey. This means that as many as 40% of students did not feel their identity was safely concealed. On a survey that contains sensitive questions, it is important for students to feel like they can be honest without the possibility of someone finding out their answer.

 But we have to admit defeat! Some of our results are biased, but that’s just the nature of statistics. Statistics, in the end, is a malleable game; a statistician can manipulate data and sampling and inference procedures to achieve the results he or she wishes to achieve. In fact, we didn’t achieve the results we thought we would achieve; we thought we would have a very even split in each grade. The nature of our data, timing, and sampling made it hard to get a true read on how valid the ISHS is. For one, months have passed since we took the ISHS and it’s possible that no one really knows anymore how honestly they answered the survey. Being generally honest could’ve easily morphed into completely honest after the span of 6 months. It also would’ve been impossible to get a truly independent sample. The advisories are decided on friendships and therefore each advisory could be significantly different from others in terms of how honest their responses were to the ISHS. This is the problem with statistics, especially those regarding human populations; there are so many contributing factors to everything that make it hard to pin down what is affecting what and what is true and what is not. Maybe with a large enough sample size humanity really acts like a bunch of machines, but to describe our school with statistics is kind of a crapshoot. That’s why the Independent School Health Survey findings should be taken with a grain of salt.