- 4,360
- 2,353
So, generally the members of the wiki argue for consistency after just two samples with similar results while I personally go by three. Why three? Well, here's the thing. Studies conducted by professional institutions (ex. NASA) and educational facilities (ex. like every university out there) generally go for a lot more than two samples in their studies. Heck, Project Farm, a reliable if generally more amateur entity, wouldn't argue for consistency unless similar results are achieved from three samples.:
Granted the wiki's members sandbag a lot of the time when it comes to this sort of stuff. People making calcs may sandbag by just using the generic 8-69-214 deal when calcing rock destruction as opposed to trying to see what materials the rock really is, and calc group members, myself included, may sandbag by using the wiki's internal Calculations page as opposed to finding a study that would boost the reliability of our stuff. Heck, DontTalkDT may sandbag at times. But here's the thing... Why are we sandbagging the most important reliability quality: sample sizes? We already had a whole staff discussion on inconsistent sizing due to sample size issues, so what's the hold-up?
Really, VBW kinda needs to up it's reliability game. I'm not expecting a Spacebattles deal here, but you bet your butt the guy who made a paper airplane for a calc or went out of his way to put all the data points of a study on latent heat of vaporization for 14 different fatty acids into a single easy-to-use Excel document on Google Drive is gonna have a good eye on where the wiki sandbags.
See, oftentimes two samples aren't going to be enough, especially when samples can be entire orders of magnitude off from one another. A good example of non-sandbagged consistency would be MinecraftHater2011's Pizza Tower calcs, which recently consistently showed Peppino's AP to be in the 0.1-tons range over four samples of him digging tunnels like meerkats from the Lion King 1 1/2.
So yeah, go for three samples instead of two before declaring consistency. Less sandbagging on the sample size front would allow us to have more reliable ratings in the future.
Granted the wiki's members sandbag a lot of the time when it comes to this sort of stuff. People making calcs may sandbag by just using the generic 8-69-214 deal when calcing rock destruction as opposed to trying to see what materials the rock really is, and calc group members, myself included, may sandbag by using the wiki's internal Calculations page as opposed to finding a study that would boost the reliability of our stuff. Heck, DontTalkDT may sandbag at times. But here's the thing... Why are we sandbagging the most important reliability quality: sample sizes? We already had a whole staff discussion on inconsistent sizing due to sample size issues, so what's the hold-up?
Really, VBW kinda needs to up it's reliability game. I'm not expecting a Spacebattles deal here, but you bet your butt the guy who made a paper airplane for a calc or went out of his way to put all the data points of a study on latent heat of vaporization for 14 different fatty acids into a single easy-to-use Excel document on Google Drive is gonna have a good eye on where the wiki sandbags.
See, oftentimes two samples aren't going to be enough, especially when samples can be entire orders of magnitude off from one another. A good example of non-sandbagged consistency would be MinecraftHater2011's Pizza Tower calcs, which recently consistently showed Peppino's AP to be in the 0.1-tons range over four samples of him digging tunnels like meerkats from the Lion King 1 1/2.
So yeah, go for three samples instead of two before declaring consistency. Less sandbagging on the sample size front would allow us to have more reliable ratings in the future.