I made a mistake recently. You'd think I'd know better (in fact, I do know better). Actually it's something I preach about so often, I thought I'd better do some penance in the form of a blog post.
A colleague emailed me to tell me that all of her students' were having trouble with an assignment I'd created. For the past few years, I've been trying to use a "get these unit tests to pass" style of programming assignment for early semester students. It takes more work upfront, but reduces my marking time significantly. It also gives a much stronger feedback loop for students, who can treat the problems like a game, where getting things to pass is how you win. I've got an assignment starter repo where I've been refining this technique.
For this particular assignment, I wrote a test where students had to work with an Array of users, convert their birthdate (String) to a Date, and figure out who was the oldest. It worked great, until it didn't. My colleague had her due date later than mine, and one day this particular test started failing for everyone.
The problem is that I used new Date() (i.e., the curent date) as one of the pieces of logic in the test. In other words, I used something that will change with every test run, but assumed it was stable between runs.
Working on code in Mozilla taught me most of what I know about writing tests. Writing a good test is hard, especially when any aspect of it depends on timing. A test that relies on any kind of timing data can pass or fail depending on (you guessed it) the timing of the test run. If you get it wrong, you get random failures; and if people start to think that these failures are ignroable, you quickly lose confidence in your test suite, and it all falls apart: "We can ignore this failure, it often happens."
In my tests, I used a bunch of mock user data generated by Mockaroo, and based on the data I got, I figured out who the oldest user was manually. At this point, I should have pinned my test date to the date I wrote the tests vs. creating a current date. If I had, I would have eliminated the variance and my test would have always worked.
If you want to become a great programmer, work on writing tests. It's often more difficult than writing the original code because it forces you to think about the conditions in which your code is executed, and the environment within which it gets embedded.