Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Introduction
The Challenges in DSL design and implementation
Orthogonal expertise
Application domain
Language design and implementation
DSLs vs GPLs
= syntax
= semantics
DSLs and GPLs need to be completely different.
Really ?
2/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Extensibility at a glance I
[van Deursen et al., 2000, Vasudevan and Tratt, 2011]
User-defined data types and (infix) operators
Algol 68 [Denert et al., 1975, Pagan, 1979]
Operator overloading
C++ [McNamara and Smaragdakis, 2000]
Compile-Time Meta-Programming
C++ templates [Prud’homme, 2006]
Template Haskell, Meta OCaml [Czarnecki et al., 2004]
Meta Lua, Converge, Nermerle [Tratt, 2005,
Fleutot and Tratt, 2007, Skalski et al., 2004]. . .
Functional languages (Haskell, ML)
[Kamin, 1998, Elliott, 1999, Hudak, 1998]
6/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Step 1: the basic thing
( setq def aul t−t hem e
( make−instance ’ fa ce : name ’ t o p l e v e l
: background ’ blac k
: subfa ces ( l i s t ( make−instance ’ fa ce : name ’ o p t i o n
: fo re g rou n d ’ whi t e
: subfa ces ( l i s t ( make−instance ’ fa ce : name ’ syn tax
: bold t
: fo re g rou n d ’ cyan )
( make−instance ’ fa ce : name ’ usage
: fo re g rou n d ’ ye l l o w ) ) ) ) ) )
Problems with make-instance:
1 Class name exposed
2 Name argument optional
10/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Step 2: an instantiation wrapper
( setq defaul t−t hem e
( make−face ’ t o p l e v e l
: background ’ blac k
: subfa ces ( l i s t ( make−face ’ op t i o n
: fo re g rou n d ’ whi t e
: subfa ces ( l i s t ( make−face ’ sy nt ax
: bold t
: fo re g rou n d ’ cyan )
( make−face ’ usage
: fo re g rou n d ’ ye l l o w ) ) ) ) ) )
Problems:
1 Explicit toplevel face name
2 Explicit creation of sub-faces list
3 Lots of calls to make-face
4 Lots of quoting
11/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Step 3: a theme creation wrapper
Solution to problem #1
( setq defaul t−t hem e
( make−theme
: background ’ blac k
: subfa ces ( l i s t ( make−face ’ op t i o n
: fo re g rou n d ’ whi t e
: subfa ces ( l i s t ( make−face ’ sy nt ax
: bold t
: fo re g rou n d ’ cyan )
( make−face ’ usage
: fo re g rou n d ’ ye l l o w ) ) ) ) ) )
Problems:
1 Explicit toplevel face name
2 Explicit creation of sub-faces list
3 Lots of calls to make-face
4 Lots of quoting
12/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Step 4: CLOS and the MOP
Solution to problem #2
Generic functions, methods
( defm ethod fu nc ( ( arg1 cl a ss 1 ) arg2 . . . )
body )
Methods are outside the classes (ordinary function calls)
Multiple dispatch (multi-methods)
The CLOS Meta-Object Protocol (MOP)
CLOS itself is object-oriented
The CLOS MOP: a de facto implementation standard
The CLOS components (classes etc.) are
(meta-)objects of some (meta-)classes
initialize-instance is a generic function
13/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Step 4: CLOS and the MOP
Solution to problem #2
( setq defaul t−t hem e
( make−theme
: background ’ blac k
: face ( make−face ’ o p t i o n
: fo re g rou n d ’ whi t e
: face ( make−face ’ syn tax : bold t : fo reg rou n d ’ cyan )
: face ( make−face ’ usage : for e gr o und ’ yel l o w ) ) ) )
Problems:
1 Explicit toplevel face name
2 Explicit creation of sub-faces list
3 Lots of calls to make-face
4 Lots of quoting
14/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Step 5: syntax extension
Solution to problem #3
( setq defaul t−t hem e
( make−theme
: background ’ blac k
: face { ’ op t i o n : for e gr o un d ’ wh i te
: face { ’ synt ax : bol d t : for egr o un d ’ cyan }
: face { ’ usage : fore g ro u nd ’ ye l l ow }
}
) )
Problems:
1 Explicit toplevel face name
2 Explicit creation of sub-faces list
3 Lots of calls to make-face
4 Lots of quoting
16/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Step 6: macros
Solution to problem #4
( setq defaul t−t hem e
( define−theme
: background bla c k
: face { op t i o n : for e gro u nd w h ite
: face { synta x : b ol d t : for e gr o und cyan }
: face { usage : fo r eg r ou n d y e llow }
}
) )
Problems:
1 Explicit toplevel face name
2 Explicit creation of sub-faces list
3 Lots of calls to make-face
4 Lots of quoting
18/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Final result
Looks pretty much like a DSL to me. . .
; ; My pe rs o n al theme wi t h so coo l co l o r s
: background bla c k
: face { o p t i o n : fo r eg r oun d whi t e
: face { sy nt ax : bol d t : for e gr o un d cyan }
: face { usage : for e gr o un d y e l l o w } }
read, eval and possibly compile
19/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Conclusion
Impact of GPL on DSL design and implementation
Key GPL aspect: extensibility
Embedded homogeneous approach
A single language
DSL infrastructure smaller
DSL both internal and external
Common Lisp
Functional, Imperative, Object-Oriented
MOP
CTMP
Syntax extension
read, eval, compile
21/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Internal vs External DSLs
[Kamin, 1998, Czarnecki et al., 2004]
Suboptimal syntax ok but. . .
Not ok:
[Fowler, 2010]: “external DSLs have their own custom
syntax and you write a full parser to process them”
[Kamin, 1998, Czarnecki et al., 2004]: “a prerequisite
for embedding is that the syntax for the new language
be a subset of the syntax for the host language”
BTW, same disagreement at the semantic level (MOP)
Poor error reporting
Research: [Tratt, 2008]
Lisp: ? (but Cf. condition system & restarts)
22/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
Controversial aspects of extensibility
Dynamic typing
pros: end-user friendly
cons: run-time type errors / checking
Research: [Taha and Sheard, 1997]
Hybrid languages (Cf. Racket)
Lazy Evaluation
pros: infinite data structures, new control primitives etc.
cons: pure functional languages only
Lisp: lazyness through macros (not as straightforward),
but side-effects for free, and still functional.
23/32
Extensibility
Didier Verna
DSL Overview
Taxonomy
Extensibility
Example
Internal
External
Conclusion
Discussion
The root of (Lisp) extensibility
Reflexion
Introspection
Intercession
Implementation
By API
Inherent: “homoiconicity” [McIlroy, 1960, Kay, 1969]
Further distinction [Maes, 1987, Smith, 1984]
Structural reflexion (program)
Behavioral reflexion (language)
24/32
Extensibility
Didier Verna
Bibliography I
Ahson, S. I. and Lamba, S. S. (1985).
The use of Forth language in process control.
Computer Languages, 10:179–187.
Czarnecki, K., O Donnell, J., Striegnitz, J., and Taha, W.
(2004).
DSL implementation in MetaOCaml, Template Haskell,
and C++.
In Lengauer, C., Batory, D., Consel, C., and Odersky,
M., editors, Domain-Specific Program Generation,
volume 3016 of Lecture Notes in Computer Science,
pages 51–72. Springer Berlin / Heidelberg.
25/32
Extensibility
Didier Verna
Bibliography II
Denert, E., Ernst, G., and Wetzel, H. (1975).
Graphex68 graphical language features in algol 68.
Computers & Graphics, 1(2-3):195–202.
van Deursen, A., Klint, P., and Visser, J. (2000).
Domain-specific languages: an annotated bibliography.
SIGPLAN Notices, 35:26–36.
Elliott, C. (1999).
An embedded modeling language approach to
interactive 3D and multimedia animation.
IEEE Transactions on Software Engineering,
25:291–308.
26/32
Extensibility
Didier Verna
Bibliography III
Fleutot, F. and Tratt, L. (2007).
Contrasting compile-time meta-programming in Metalua
and Converge.
In Workshop on Dynamic Languages and Applications.
Fowler, M. (2005).
Language workbenches: The killer-app for domain
specific languages?
Fowler, M. (2010).
Domain Specific Languages.
Addison Wesley.
27/32
Extensibility
Didier Verna
Bibliography IV
Hudak, P. (1998).
Modular domain specific languages and tools.
In Proceedings of the 5th International Conference on
Software Reuse, ICSR’98, pages 134–142,
Washington, DC, USA. IEEE Computer Society.
Kamin, S. N. (1998).
Research on domain-specific embedded languages and
program generators.
In Electronic Notes in Theoretical Computer Science,
volume 14. Elsevier.
Kay, A. C. (1969).
The Reactive Engine.
PhD thesis, University of Hamburg.
28/32
Extensibility
Didier Verna
Bibliography V
Maes, P. (1987).
Concepts and experiments in computational reflection.
In OOPSLA. ACM.
McIlroy, M. D. (1960).
Macro instruction extensions of compiler languages.
Commun. ACM, 3:214–220.
McNamara, B. and Smaragdakis, Y. (2000).
Functional programming in C++.
SIGPLAN Not., 35:118–129.
Pagan, F. (1979).
ALGOL 68 as a metalanguage for denotational
semantics.
The Computer Journal, 22(1):63–66.
29/32
Extensibility
Didier Verna
Bibliography VI
Prud’homme, C. (2006).
A domain specific embedded language in c++ for
automatic differentiation, projection, integration and
variational formulations.
Journal of Scientific Programming, 14:81–110.
Rompf, T., Sujeeth, A. K., Lee, H., Brown, K. J., Chafi,
H., Odersky, M., and Olukotun, K. (2011).
Building-blocks for performance oriented DSLs.
In DSL’11: IFIP Working Conference on
Domain-Specific Languages.
Skalski, K., Moskal, M., and Olszta, P. (2004).
Meta-programming in Nemerle.
Technical report.
30/32
Extensibility
Didier Verna
Bibliography VII
Smith, B. C. (1984).
Reflection and semantics in Lisp.
In Symposium on Principles of Programming
Languages, pages 23–35. ACM.
Taha, W. and Sheard, T. (1997).
Multi-stage programming with explicit annotations.
In Proceedings of the 1997 ACM SIGPLAN symposium
on Partial evaluation and semantics-based program
manipulation, PEPM ’97, pages 203–217, New York,
NY, USA. ACM.
31/32
Extensibility
Didier Verna
Bibliography VIII
Tratt, L. (2008).
Domain specific language implementation via
compile-time meta-programming.
ACM Transactions on Programming Languages and
Systems, 30:31:1–31:40.
Tratt, L. (2005).
Compile-time meta-programming in a dynamically typed
OO language.
Vasudevan, N. and Tratt, L. (2011).
Comparative study of DSL tools.
Electronic Notes in Theoretical Computer Science,
264:103–121.
32/32
