2024-12-11: Deterministic simulation testing

Today I learned about de­ter­min­is­tic sim­u­la­tion test­ing, an ap­proach to test dis­trib­uted sys­tems. The core idea is to re­move all non-​deterministic be­hav­ior from the crit­i­cal parts of the ap­pli­ca­tion, cre­at­ing a con­trolled en­vi­ron­ment where the se­quence of events can be ma­nip­u­lated.

This in­volves set­ting fixed seeds for ran­dom num­ber gen­er­a­tors, con­trol­ling clock and time de­pen­den­cies, run­ning tests in a sin­gle thread, and using asyn­chro­nous I/O op­er­a­tions. While these con­straints might seem overly re­stric­tive, they allow us to re­pro­duce bugs found dur­ing sim­u­la­tions.

It’s eas­ier said than done, though. Adding this kind of test to an ex­ist­ing sys­tem can be chal­leng­ing. There’s al­ways going to be some non-​deterministic edges. Also, the test is only as good as your sim­u­la­tion.

Read more in this blog post.