Sample Size Is a Product Decision

Power calculations get treated as a statistics problem handed to a statistician. They are actually a product question: how much are you willing to pay, in time and traffic, to be sure?

This talk reframes the decision in those terms, with the arithmetic that lets a product manager argue it properly in a room.

It ends with the tests that should never have been run, and what running them cost.