Why I haven't looked at Rebol much

I looked at Rebol a few years ago, but lost interest pretty quickly. Recently commenter Edoc recommended it, so I took another look, and remembered why I lost interest last time.

The first problem is that the docs are frustratingly introductory. Everything I can find is aimed at explaining the language to beginners, with lots of repetition and very little precision. Would it be so hard to explain the language kernel in a way someone who knows languages can understand? The closest I can find to a description of the semantics is the overview of evaluation and one entry in the FAQ. There are grandiose statements about how different Rebol is from other languages, and how it's supposed to take over the world, but not much explanation.

Okay, even from the beginner documentation you can learn something. Rebol is intended for making little DSLs - so, like Lisp, it separates the syntax from the language. Programs are defined in terms of trees, not text, and the trees are easily available to interpreters. But Rebol also tries to preserve a familiar appearance of programs - infix, and without too many parentheses. The usual way to do this is to keep the trees as abstract as practical, and have the parser transform complex syntax into a simple underlying representation. Rebol does the opposite. It preserves most of the syntax in its parse trees, so the parser is simple but the interpreter is complicated. In Rebol, 2 * sin :x reads as a four-element sequence: [2 * sin :x]. The advantage is that code closely resembles its source text; the disadvantage is that even parsed code is rather complicated and awkward to operate on.

The parser is not actually all that simple, because Rebol has syntax for a great many data types. In addition to the four kinds of symbols and three kinds of sequences used for programs, there are a variety of other types with syntax. The complete list is:

  • Word (symbol; 4 kinds): foo: bar :baz 'error-out-of-metasyntactic-variables
  • Block (vector): [1 two [three] "IV"]
  • Paren (same as Block, but different eval semantics): (alpha [beta * gamma] delta)
  • Refinement (generalized field accessor): /x
  • Path (field access expression): a/b/c
  • Integer: -55 16'777'216
  • "Decimal" (actually binary floating point; there's nothing decimal about them): 10.0 2,718'281'828'4 6.023E23
  • String: "Hello,^(line)world!" {Hello,
    world!}
  • Binary: #{F00DF00DF00D}
  • Email: user@example.com
  • File (pathname): %/etc/passwd %rebol%20stuff\test.r
  • Issue: #123-456-789 (how is this usefully different from String?)
  • Tag: <a href="http://www.rebol.com">
  • URL: file:///etc/passwd
  • Character: #";" #"字"
  • Date: 2008-07-26 26-Jul-2008 7/26/2008
  • Time: 20:51:33
  • Tuple (of 3 or more small integers): 127.0.0.1
  • Money: EUR$0.02 $0.03
  • Pair (2-D point): 1024x768

There are only a few types that don't have syntax:

  • Logic (boolean): true false yes no on off (these are constants, not syntax)
  • None (nil): none (constant)
  • List
  • Hash (hashtable)
  • Image (bitmap)

Curiously, much of the syntax is defined by special kinds of tokens rather than with special characters. It's not just numbers, but dates, times, and even pairs. This makes it a bit hard to tell how a token will be parsed, and thus whether an unusual symbol needs to be escaped. It seems to me that Rebol goes a bit overboard on this. I'm not sure why dates or email addresses need to be handled by the parser rather than by the few operations that use them. The syntactic support is slightly convenient for some small programs, but I have a hard time believing it helps with larger ones.

Of course a language's semantics are more important than its syntax, and it's tricky to figure them out from a tutorial. The obvious alternative is to read the source - but here's the other reason I stopped looking at Rebol: it's not open-source. “REBOL/Core is free and will always be free”, says the license page, but they only mean gratis. Quite frustrating to someone who might understand it by looking at eval, or to anyone considering using the language. Joe Marshall (who used to work on Rebol) apparently wrote a Rebol-to-Scheme compiler, but the actual code seems to have vanished from the face of the Internet. (Update: Here's a copy. Thanks, anonymous!) Fortunately, bits of a Rebol interpreter survive in Joe's talk on the difficulties of doing tail-call optimization. From these, and from the documentation, I think I understand Rebol's eval now, though not with enough confidence to explain it formally - go read that talk if you want to know.

The odd thing is that it operates on sequences, so it has to parse as it evaluates. The two operations which consume arguments (assignments and function calls) evaluate as many as they need from the rest of the sequence, not knowing where an expression ends until they've evaluated it. (This is why tail-call optimization is hard: you can't tell whether you're in a tail context until the last moment.) After evaluating, if there's nothing left, then you just did a tail call, and its result is the value of the block. If there is something left, and it begins with an infix operator, then what you just evaluated was its first argument, so you evaluate another argument and call the infix operator. (This means all infix operators are left-associative.) This is what it takes to interpret unparenthesised prefix with variable arity, but it's certainly awkward.

Here's something disturbing: Rebol 1.0 had lexical scope, but Rebol 2.0 doesn't. Blocks are just self-evaluating, and it's up to functions (or should I say FEXPRs?) like if to evaluate them in some environment. (However, this article suggests words know something about their environment.) I don't know how assignment handles nested scopes. It is worrying, especially in a poorly specified language with no reference implementation, to see what appears to be a well-known mistake, and the possibility of several others. I don't trust my intuition when evaluating a language, but I'm beginning to get the impression that Rebol has a lot of cute features on a rather shaky foundation.

Trying to reconstruct a language from introductory documentation is a frustrating task, so I'll stop here. Unfortunately I've exhausted my patience before learning much about its library or how it works in practice, so I still don't know what the good details were that Edoc was talking about. Maybe some other time.

16 comments:

  1. I'm a Rebol user, and I think its real usefulness is for non-programmers. It's so easy to get started, and deep enough to be practically usable, right away, by people with absolutely no background writing code. I've written several hundred pages of tutorials, including the one at http://musiclessonz.com/rebol_tutorial.html , which sees quite a bit of traffic, and I've seen many times first hand how natural it is for people to pick up. I think that's Rebol's greatest strength - its super easy and intuitive initial learning curve, that extends all the way towards accomplishing practical computing work, for "normal" people. It's just so friendly and simple to learn, tiny to download, quick to get started with - reminds me a bit of old BASIC interpreters from the 80s. The difference is that you can use it to get all sorts of real work done. Creating small GUI's takes a few lines of code. It's useful for web site scripting. Everything's built in, some very useful things take only a few lines of code, and the language somehow makes a lot of sense to people without any previous experience. I don't see REBOL ever replacing typical professional development tools or other mainstream languages, but for average computer users for whom learning even a single, simple paradigm may be difficult, it covers a whole lot of ground. I've tried to communicate this to the Rebol community, as it's been my experience that beginners just "get" it naturally, and Rebol could potentially find a niche in the lives of "normal" people, there doesn't seem to be much interest. It's a shame because Rebol's really perfectly suited for that use.

    ReplyDelete
  2. I'm not sure why dates or email addresses need to be handled by the parser rather than by the few operations that use them. The syntactic support is slightly convenient for some small programs, but I have a hard time believing it helps with larger ones.

    I build many REBOL dialects and believe me, having all sorts of datatypes available like REBOL does is a great way to describe content without explicitly stating the type of the content. REBOL will parse emails, urls, words, strings, numbers and any other of its 54 datatypes directly (you've only listed a few of them).

    So this "cute feature" is very important in the simplification of dialect expressions.

    ReplyDelete
  3. 54 types? Evidently the page I read was not as complete as I thought.

    Nick's tutorial shows Rebol's strengths much better than the ones I looked at. Based on its one-liners, it looks like Rebol has some seriously convenient GUI and I/O libraries. I suspect they can be adapted to other languages pretty easily. Maybe I should not have given up on Rebol so quickly; there are some useful ideas in it.

    ReplyDelete
  4. It's true (and unfortunate) that the original documentation is weak, which is why I rarely use it, but fortunately the number of 3rd party tutorials is growing and there are a lot of helpful people on mailing lists and in the REBOL based AltME chat system.
    REBOL 3 provides full documentation through a standard wiki, edited by Carl Sassenrath himself and other expert developers, but discussing REBOL 3 may be inappropriate here, as it can still be considered vaporware. You can get a quick count of the number of datatypes available, by typing:

    ? datatypes

    in the console. It varies with the version and type, but mine has 54.

    it looks like Rebol has some seriously convenient GUI and I/O libraries

    The convenience factor is indeed very high and it's a real boon when you want to write a script literally in 5-10 minutes that your boss can use, because it has a GUI. The size of scripts help to visibly reduce development time, simply because you have to type less and scripts are simpler to look at and easier to debug.
    It's part of why REBOL still has no IDE or advanced debugger, although it would be nice to have a real debugger for very large scripts above 100-200k.

    Yes, a 100k REBOL script is considered large. :-)

    ReplyDelete
  5. hmmm, if this rebol view is so good and easy at creating GUI front ends then why hasnt the makers such as this "Carl Sassenrath himself and other expert developers" made the time to make them?

    a simple
    http://www.google.co.uk/search?q=gui+front+end+encode+rebol&btnG=Search&hl=en&sa=2

    brings up lots of talk and why and whatfors, even requests that someone make them, but not one _single_ real working front end for any cli app.

    no FFmpeg,Mplayer/mencoder,transcode, or even the most popular x264 encoding shell app.

    no transcoding, or even again the most popular VLC streaming control from end,or aven a replacment VLC Html (rebols suposed to be good at this right?) remote controlling scripts and intigrated client/server, theres not even a simple LAME MP3 front end made and advertised anywere.

    after all this time that rebol has existed is that really credable if its so easy for even non programmers to use and make these fancy cross platform GUI front ends?

    however, take awy the rebol work in the above search
    http://www.google.co.uk/search?hl=en&q=gui+front+end+encode+&meta=

    and there counless from ends make for every app mentioned....

    ohh ok noone seems to have bothered to make that replacement VLC http client/server as yet, but perhaps one of you rebol coders can make that happen and prove that it can be done as you say it can....

    Nicks http://musiclessonz.com/rebol_tutorial.html shows promise, ill admit, especially the very limited entry he's done for interaction with external librays (that MS webcam working example), shows promise but not really what most people call a fully working app example, but that i assume wasnt his intent so that ok..

    perhaps he can take the VLC core librays and the documented API for interacting with them http://www.google.co.uk/search?hl=en&q=VLC+API&meta= and make a working example and a good practical LAN based client server rebol GUI VLC control app in the process to show how he would do it being a rebol script expert that should eb able to pull something together in less than a day if its so simple and clever as we are being told!, or NOT ;)

    if this "Carl Sassenrath" the creator of this GUI scripting language (didnt he make AmigOS or something?) finally wants people to know and use it after all this time in the market place, why hasnt he and his inhouse team already make these GUI scripts for the masses ?,a simple thing to do we are told, as are the people asking for these things iv read.


    if its so quick and easy,make these GUI front end scripts and show the world, im yet to be convinced its so good, small,cross platform, and self contained as claimed.

    not even what appears to be the largest rebol script collection around, has got one single script
    on this search "gui front end encode", why is that, noone bothered to make these simple everyday things, i find that hard to beleave people dont want these things given all the other already existing front ends in all the other laguages.

    can you long time rebol experts explain this?

    http://www.rebol.org/cgi-bin/cgiwrap/rebol/search.r?find=gui+front+end+encode&form=yes

    ReplyDelete
  6. if rebol is so easy to make GUIs and web based control for even non programers.

    heres a real simple challenge for any rebol script writers reading.

    replace all the Ion ruby etc from this VLC webcam recorder app URL below and fully recreate it in rebolview script, and even make it look and interact with an ordinary user better if you can, and post it whereever (that http://www.rebol.org script site perhaps?)for all to see and evaluate on both windows and linux.

    http://blog.raphinou.com/2007/08/building-webcam-recorder-with-vlc-ion.html

    this should be a simple ready made practical project for any longtime rebol coder should it not?

    ReplyDelete
  7. hmmm, if this rebol view is so good and easy at creating GUI front ends then why hasnt the makers such as this "Carl Sassenrath himself and other expert developers" made the time to make them?

    Because we're quite busy building REBOL 3. :-)

    Seriously, REBOL is just not that well known and the number of developers is low, because a lot of people, like you, are just plain suspicious about its capabilities until they see it demoed. It's nothing new and we are used to that.

    Some reasons:

    Not that many people post scripts to rebol.org yet.

    Almost every REBOL developer I know are very busy with their own REBOL projects meant for private customers and have no time to build frontends for this and that application. I'm actually one of the few who post publicly about REBOL software.

    Perhaps also because VID is excellent at building very customized tools very quickly that only you need and see no need for spreading. We're talking minutes of work. In fact early on REBOL was dubbed the throw-away-script language, because it's easy to build GUI apps that you will only be needing a few times and then throw them away.

    Some limitations of REBOL 2:

    - DLL access was for a long time only available in the pay-only version of REBOL. It has only been made freely available in the latest version of REBOL 2.

    - The graphics engine does not allow intergration with whatever your graphics card can offer, so no native hardware acceleration, for video or OpenGL directly in a REBOL/View window. Video player front ends or fast paced games are therefore not so interesting to do in REBOL 2, so you won't find any.

    REBOL 3 will have no such limitations.

    ReplyDelete
  8. A limitation of the current View is it only renders visual content according to its internal compositing engine and presently this does not include video: this makes it difficult to recreate VLC (and somewhat beside the point).

    To understand the intent of View is to appreciate the goal of the language as a whole: Internet scripting. This explains high-level types such as URLs, Emails and point separated tuples: as well as being functional, they are designed for semantic exchange in a manner that is more efficient than XML and even JSON, yet more reliable and readable. Enter VID, a visual 'dialect' designed not just for creating an interface but exchanging a rich client that is still lighter than your average web page (especially one that has a JS library just to provide wrappers for XMLHTTPRequest) and arguably more capable.

    Not only do you have a lightweight method of creating and exchanging GUIs, but is built from the same elements as every other part of the language.

    There are flaws in this model (other than compositing limitations, which will be rectified at some point), and that imo is deployment.

    ReplyDelete
  9. I wouldn't put it down as a beginners' language either. It is very capable in the domain it is primarily designed for: communication.

    ReplyDelete
  10. "i find that hard to beleave people dont want these things given all the other already existing front ends in all the other laguages.

    can you long time rebol experts explain this? "

    If somebody wants something badly enough, they can probably build it in REBOL. That doesn't mean REBOL is the right choice for everything, as in this case. In order to keep the size down, and portability high, a lot of things are left out. You could build libraries or modules to do it, but REBOLers tend not to want to bind themselves to system specific things without good cause. IME, the REBOL mindset is not one of "just add a library", it's more "Do I really need to do this, and what's the simplest REBOL solution?" REBOL wasn't designed for PitL, and doesn't even have a true module system in v2 (v3 will), which contributes to the direction people lean in the things they build with it.

    That said, I've built a system for commercial use that acquires images from cameras, displays MPJG streams, allows image capture, and editing of streamed MJPG, and uses ImageMagick and FFMPEG to process images and create timelapse movies, including titles and credits. It also has tools to take image archives and timelapses and burn them to disc, including labels. On the client side, there is a REBOL plugin client, a Flash front end, and a REBOL desktop app. The back end is all REBOL, currently managing >8M images and 700GB of data.

    My point being that REBOL can be used for serious work. The fact that there aren't equivalent REBOL apps for all the Python/Ruby/PHP apps isn't news; the REBOL community is small, and it may stay that way. It has enormous strengths, and glaring weaknesses. Both depend on your expectations and needs.

    REBOL is very high level, but it is not a "safe" language, WRT limitations and constraints the language imposes. It will let you do things any way you want, be they brilliant or insane.

    Not everybody likes or "gets" REBOL, and that's perfectly OK.

    ReplyDelete
  11. OK, I glanced at the Ruby code, and here is what a REBOL version might look like. Mind you, I just did this for the *flavor* of a REBOL solution. I'm not taking the time to install and actually test things here. i.e. this code wasn't meant to run.

    #!/usr/bin/rebol

    do %async-call ; Use DocKimbel and Gabriele's async-call module
    do %telnet.r ; Frank Sievertsen's, from REBOL.org, assuming it works and gives us a telnet protocol
    do %format.r ; I would use my own format func if you want the ISO formatted file name.

    DESTINATION_DIR: %tmp/
    MAXIMUM_DURATION: 0:5:0
    UVC_STREAM_COMMAND: {uvc_stream -r 640x480 -f 30 -p 5555}
    VLC_COMMAND: {vlc -IRC --rc-host localhost:4444 --sout '#transcode{vcodec=mp4v,vb=3072,scale=1,acodec=mpga,ab=192,channels=2}:duplicate{dst=display,dst=std{access=file,mux=mp4,dst="/tmp/test.mpg"}}'}

    recorder: context [
    recording?: off
    vlc_started?: no
    uvc_stream_started?: no
    telnet-port: none

    main-layout: layout [
    f-btn: btn "Start Recording" [start_stop]
    f-status: text 300 "Stopped"
    ]

    update_button: does [
    either recording? [stop_recording] [start_recording]
    ]

    start_recording: does [start-stop]

    ; stubs
    vlc-done-cbk: func [] []
    vlc-inp-cbk: func [] []
    uvc-done-cbk: func [] []
    uvc-inp-cbk: func [] []

    start_vlc: does [
    async-call/on-input VLC_COMMAND :vlc-done-cbk :vlc-inp-cbk
    vlc_started?: yes
    ]

    start_uvc_stream: does [
    async-call/on-input UVC_STREAM_COMMAND :uvc-done-cbk :uvc-inp-cbk
    uvc_stream_started?: yes
    ]

    start_recording: does [
    recording?: on
    set-face f-btn "Stop Recording"
    set-face f-status "Recording..."
    insert telnet-port "add http://localhost:5555"
    ]

    stop_recording: has [file] [
    recording?: off
    insert telnet-port "stop"
    set-face f-btn "Start Recording"
    file: join format now "YYMMDDTHH:MM:SS" %.mpg
    ; move-file is not a built-in REBOL func, but there are versions out there.
    move-file %/tmp/test.mpg DESTINATION_DIR/:file
    set-face f-status "Stopped"
    ]

    start_stop: does [update_button]
    terminate: does [insert telnet-port "quit"]


    telnet-port: open telnet://localhost:4444
    start_uvc_stream
    start_vlc

    ]

    view recorder/main-layout

    ;Hmmm, can't use a pre tag here it seems.

    ReplyDelete
  12. There seems to be a copy of Sherman, Joe Marshall's REBOL to Scheme compiler, at http://members.core.com/~bhawley/rebol/Sherman.zip

    ReplyDelete
  13. I'll try again.

    Sherman.zip, the REBOL to Scheme compiler.

    The C2 RebolLanguage page, which has links to other Joe Marshall writings.

    ReplyDelete
  14. It's amazing to me that the site where you found the REBOL-to-Scheme compiler still exists - that account has been inactive since the 90's (I switched ISPs). I believe that is the only site on the Internet where you can still find the Oberon Compiler for DOS as well.

    REBOL scoping is not as bad as you think. The main difference between 1.0 and 2.0 is that the bindings of words are looked up at runtime in 1.0, like Scheme, but in 2.0 the words are bound to contexts directly. This makes value lookup much faster.

    The context of a word is usually set during the creation of a function or object. Nested scopes are emulated by rebinding inner words that relate to nested functions or objects. This is called applicative binding order. There are also some functions that rebind words or code blocks passed to them as parameters - this allows you create functions that behave like the syntax of other languages.

    REBOL 2's binding and execution model is well thought through, and is in some ways similar to Forth. There have been papers written about it (mostly by Ladislav) available online, and various essays and explanations on its behavior and implications (mostly by me) published to the user forums and mailing list archives.

    ReplyDelete
  15. It's no fault in a language that something hasn't been done in it yet - especially something remote from the language's goals, and especially something slightly exotic like video.

    Thanks for the links and hints. Looks like I have some more reading to do.

    ReplyDelete
  16. Popper, is that you? :-) You seem to complain about the lack of VLC or other wrappers for quite some time already. You are not apparently able to code it yourself, but that is OK with me, because I am the same. But - if you can't code it, if you can't start and coordinate such project, or if you can't sponsor such development, then what is the point?

    To give you some news, there is one guy under contract who writes ffmpeg wrapper right now. Dunno how complete we will make it. I also don't know, if we realease it for free - depends on my business partners - remember, we pay for the work.

    But - why can't you see things in perspective? As for me, I am not much interested in R2 (although still using it), but many of us devote their free time to R3 for some 2 years or more already. And we are close to alpha, including GUI. There is so many improvements on so many levels, that it is simply being completly new REBOL.

    btw - this new wrapper will not work with R3 probably, as it did not yet get its library wrapper implemented, but hopefully we will adapt it asap, once R3 allows library calls ...

    cheers,
    -pekr-

    ReplyDelete

It's OK to comment on old posts.