Arc gets I/O right

In any significant program, a great deal of code is spent on two activities that have very little intrinsic interest: I/O and error handling. This makes them good targets for language design. If a language can make these two areas easier, it can remove a significant part of the accidental complexity of programs. Conversely, a language in which either is difficult is a language in which effort is wasted on uninteresting problems.

This is where Arc is good. Many parts of it are hacky or just not there yet, but its I/O library is strikingly convenient. There's a lot there worth seeing and maybe stealing. In particular:

• The whole-file I/O routines, starting with readfile, are a great idea - usually what you want in a file is to read the whole thing. There should be one (file?) for reading the whole file as a string too. (The point here is not that they read the whole file in one piece, but that they handle the opening and closing - all they need is the filename.)
• infile and outfile are nice and simple. Opening files for reading and writing are quite different operations, and there's no reason to do both with the same function. Getting rid of the mode argument also makes the functions easier to compose. This can go further: the remaining binary, text and append arguments should be separate functions too.
• Using tostring is much simpler than having an optional stream argument to every output routine. For instance, (tostring (prall thingies "We have ") (when (no done?) (pr " and more coming!"))). Dynamic variables are a perfectly reasonable way to pass arguments, especially those like streams that tend to be used in dynamic scope - so why don't I feel good about this? Are there any cases it handles badly?
• Printing several values separated by spaces is a common operation (especially in throwaway programs) but it needs a good name. I like prs better than what I was using (prv, for "print values") although it might conflict with other functions like "print to stream".
• prall is another function I often want, although it feels unwieldy to me, possibly because the second argument is printed before the first. The prefix argument also seems out of place.
• prf is (format t ...), plus it does string interpolation in case you prefer that. Where's the non-printing format (well, probably fmt in Arc)? This is definitely common enough to deserve a function, not just (tostring (prf ...)).
• There are some other perlesque string operations - tokens and such. You can never have enough of these (even in Perl).
• I'm not sure if w/bars is useful (for what, debugging?) but it's an interesting kind of output wrapper - it captures write operations, not characters. How about a similar wrapper that indents every line?
• It's not an I/O routine, but ellipsize handles a common problem (truncating long strings and adding an ellipsis) that would often go unhandled without it.

I/O is really a subset of a more general category, glue, which probably accounts for more complexity than everything else put together. And it's all accidental complexity, so it should be a target for libraries. Unfortunately I don't think there are any general ways to reduce it, because there are so many different transformations that have to be done. But simplifying I/O addresses the most common kind of glue. Remarkably, considering its age and importance, this area is not solved. There are still considerable improvements to be found, and Arc is finding some of them.

I wish I could find similar improvements in error handling.

A heap of possibilities

I needed a priority queue a few months ago, and didn't have a predefined one, so I used a sorted list, since n was always small. But that won't work for many applications. Yesterday I wanted a pq for a best-first search with a potentially large n. Obviously a heap was the answer. But the modest trouble of implementing one was enough to discourage me from trying that search. It wasn't in the standard library, so it was too much trouble to bother with.

Heaps are quite useful, and quite stereotyped, so they make good candidates for a standard library. I think they (like most data structures) would be most convenient as a heap type which supports the same operations as other collections, but with the particular performance characteristics of heaps. In addition, it's probably a good idea to expose the two internal reheap operations (adding and removing) so they can be reused for variations like limited-size heaps, or heapifying an existing array as in heapsort or heap-select.

And they should be ordinary destructive heaps, not functional ones. The motivation for heaps is performance, and heaps that require consing can't compete with those built on arrays. The flexibility functional heaps offer isn't worth the performance cost, because heaps are almost always used linearly.

When you have heaps, some fancy algorithms become much easier. The search I wanted yesterday, for example (in pseudo-Lisp):

(defun a* (expand min-cost done? init)
  "Best-first heuristic graph search. Starting from INIT,
find the lowest-cost node that satisfies DONE?, where
NEIGHBORS is a function on a node that returns a list of
its neighbors, and MIN-COST is function from a node to a
lower bound on the cost of any node reachable from it."
  (def pq (heap (o* < min-cost)))
  (def (step x)
    (if (done? x)
      x
      (do (each (expand x) (add! _ pq))
          (aif (pop! pq)
            (step it)
            nil))))
  (step init))

If you really want to make fancy algorithms easy, you could include functions like that one in the standard library. The greatest strength of functional programming (in the high-order sense) is the ability to abstract algorithms; we might as well use it.