Pitfalls for who?

SISC has a test suite for R⁵RS Pitfalls. Most of these are ordinary compliance tests, that test the implementation's completeness or robustness or up-to-date-ness. But several are tests of perverse implications of the standard, where a reasonable person might say R⁵RS is overspecified or even wrong. In particular, the first two test the interaction of letrec with call/cc — letrec is supposed to evaluate all the init-forms before setting any of the variables, effectively requiring an extra layer of variables. This is not obviously better than the simpler definition which initializes one variable at a time. This is less a test of the implementation's quality than of what lengths it's willing to go to to follow the letter of the standard.

The existence of tests like these — and the number of implementations that fail them — says something about how clean a language is. When expert users are surprised by something in a language, and implementors often get it wrong, this suggests a problem in the spec. In these cases, compliance tests can illuminate what parts of the language are ugliest, and why. For Scheme, it looks like the problems are in details that would ordinarily be invisible, but that are exposed by call/cc or syntax-rules. This is a situation where it's easy for language designers to make mistakes, because they don't even realize what they're specifying.

Sometimes, if a spec is widely agreed to be buggy, everyone will just ignore it. SISC's test suite contains a test for whether let-syntax creates a contour for internal define. The test follows the consensus of implementors in saying that R⁵RS is wrong, and that a good implementation should ignore it on this point. Scheme culture values formality, but apparently there is a limit to how much inconvenience they are willing to put up with for it. Language designers, take note.