What was the problem with internal define in Arc?

According to Paul Graham, an early version of Arc had internal define, but there was a problem:

In a language with implicit local variables and macros, you're always tripping over unexpected lexical contours. You don't want to create new lexical contours without announcing it. But a lot of macros that don't look like blocks in the call expand into blocks. So we provided a second block operator, called justdo, which was like do but didn't create a new lexical contour (i.e. it is Common Lisp progn), and this is what you were supposed to use in macroexpansions.

The trouble was, I kept forgetting and using do instead. And I was thereby writing utilities with the worst sort of bug: the kind that might not show up for years, and only then in someone else's code.

Huh? My own Lisp dialect also has internal define, and I haven't had a problem with unexpected contours. Neither have thousands of Schemers. (They might point out the disagreement over define in let-syntax, and the difficulty of writing macros that expand into multiple defines, but I don't think anyone counts these among the biggest problems with the language.) I haven't even had trouble remembering to use justdo in my Lisp, possibly because it has the more distinctive name splice. What are these problems I'm supposed to be having?

I'm also surprised by how complicated the implementation was said to be:

We wrote a hideously complicated interpreter that allowed local variables to be declared this way. The ugliness of this code worried me: ugly things are generally a bad idea.

I suspect they just implemented it the wrong way. You can implement internal define quite simply by defining begin (including implicit begin!) as a macro which transforms define to letrec*. Most Schemes do it this way, and my own Lisp does too, more consistently: basic special forms like lambda and begin are actually macros over more primitive ones. It feels a little odd at first, but once you stop expecting to write in primitives all the time, it's quite comfortable. You get internal define and other macro-based luxuries without compromising the simplicity of the language kernel.

My experience with internal define has been almost entirely positive. This is so different from Graham and Morris' that I wonder if they were really doing something else. Now, how did they describe it?

In Arc we were planning to let users declare local variables implicitly, just by assigning values to them.

Were they actually creating variables by assignment? This is well known not to work, and for reasons having nothing to do with macros — ask a Python user about global. The Arc designers couldn't be repeating this old mistake. Could they?


  1. Defining variable by assignment works perfectly in Lisp. RLisp does it. In RLisp variables defined by assignment stick to the nearest (fn ...) block. It often makes code massively prettier than Scheme code - here's an example of macros which would be pretty much impossible without this feature, but are straightforward in Rlisp.

  2. taw: That's exactly the error in Python that was mentioned in the post.

    One generally wants assignment to be different from definition. E.g.

    If assignment introduces new variables, the following code:

    (let ((x 1)) (let ((y 2)) (set! x 3)) x)

    returns 1.

  3. > Were they actually creating variables by assignment?

    Yes, this *was* the problem!

  4. BTW, I also had a personal Lisp dialect, and internal def worked like charm!

    I sincerely cannot understand how is it that Paul Graham and Robert Morris couldn't get it right.

  5. Does RLisp's let do assignment too, or is it just define with a larger scope? That is, does it modify existing variables or shadow them?

    The rx-match macro is a nice demonstration of the value of a more flexible define. (There's no need to merge it with assignment.) I wonder if there's a reasonable way to make its scope more predictable (so it doesn't depend on distant enclosing forms) without greatly complicating the implementation or losing its convenience.


It's OK to comment on old posts.