Extensible programming

A couple of years ago I did some extra-curricular work with Greg Wilson and Miles Thibault related to the idea of extensible programming. As a warm up, I built a scheme interpreter and implemented hygenic macros from scratch. Learned heaps. Two of Greg's students (golly, can't remember their names) appear to be taking up the extensible programming idea in earnest as their master's projects. For what it's worth, I'll summarise two of the ideas that came out of the project:

• You can't have an extensible language without an extensible parser. Stating it this way makes it seem pretty obvious, but when we moved from scheme to Java this was a bit of a revelation. In Scheme you can extend the language anyway you want simply with a macro, but that's only because the language has an extremely constrained syntax. For Java you'd either have to have predefined grammatical constructs that extensions would fit into (like the JSE macro syntax), or have your language extensions also extend the parser to support themselves.
  
  
• Fully qualified language keywords. One option I came up with to support grammer extensions was the notion of a fully qualified language keyword. In the same way that Java lets you fully qualify classes (e.g. org.w3c.Document) why not fully qualify keywords to allow the parser to handle ambiguous bits of syntax. For example:
  

  import syntax com.pipitone.try;
  
  ...
  
  java.lang.syntax.try {// standard try-catch
  ...
  }
  catch (FooException e) {...}
  catch (BarException e2) {...}
  
  com.pipitone.try {// custom, multi-exception try-catch
  ...
  }
  catch (FooException e1, BarException e2) { ... }

Interesting stuff, this. It's still exciting. If I wasn't trying to save the world I'd probably being working on this project. ;-)