Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

X: Common Lisp Objects for XEmacs

Didier Verna

didier@xemacs.org

http://www.xemacs.org

http://www.lrde.epita.fr/˜didier

ELS 2010 – Friday, May 7th

1/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Overview

What is it ?



An implementation of CLOS for XEmacs



Including the Meta-Object Protocol (MOP)

What’s in it ?



A port of Closette to Emacs Lisp



Deeper type/class integration



A comprehensive test suite

2/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Table of contents

1 Introduction

2 Closette in Emacs Lisp

3 Type/class integration

4 Project status

5 Conclusion

4/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Context

The state of XEmacs

About XEmacs



Initially a fork of GNU Emacs v.19-to-be (1991)



Greatly diverged since then



User-level Emacs Lisp compatibility



Internals compatibility

About the internals



Very high level of abstraction



111 opaque Lisp types, 35 user-level



Core in “C+”: data-abstraction and OO infrastructure

6/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

A couple of “C+” examples

So why not use C++ directly ?

plonk

Polymorphism / Class-like abstraction

struct console

{

enum console_variant contype;

void

console_data;

struct console_methods

conmeths;

/ ∗ . . . ∗ /

}

Dynamic method lookup

MAYBE_CONMETH (console, mark_console, ...);

7/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Motivation

Bring the same level of abstraction to the Lisp layer

Why isn’t it the case already ?



GNU Emacs compatibility



Emacs Lisp backward compatibility



Less “OO pressure” at the Lisp level

•

Glue to the C level

•

Not the job of package authors

•

Requires more than the average coding skill

•

Ad-hoc abstraction easier to achieve in Lisp

What would be the beneﬁts ?



C-based features: abstract away the Lisp glue



Lisp features: improve maintainability / extensibility



Also for third-party packages (e.g. Gnus)

8/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Why CLOS?

Arguments in favor of it

1 Emacs Lisp



Close to MacLisp and Common Lisp



Many developers familiar with Common Lisp



CL package dependency: 16% ﬁles, 27% LoC

2 CLOS



One of the most powerful object system around



Well documented



No need to start from scratch

3 Porting Common Lisp libraries to Emacs Lisp

4 Attract more Common Lisp developers

5 Gain expertise in CLOS and its MOP ;-)

9/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Alternatives

Other available object systems

1 EOOPS (Emacs Lisp Object Oriented Programming System)



Message passing (Smalltalk style)



No activity since 1992



No known Emacs Lisp package using it

2 EIEIO (Enhanced Implementation of Emacs Interpreted Objects)



Active



Part of the CEDET package (70,000 LoC)



CLOS-like



Additional features (e.g. debugging support)



Doesn’t aim at being fully compliant

10/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Emacs Lisp vs. Common Lisp

All them dialects, they make my head swim

Fundamental differences



Dynamic vs. lexical scope



No package system, Limited λ -list syntax



Different types, condition system, printing facilities etc.

Less obvious ones



Different function names



Similar functions with different semantics



function is different

(X)Emacs Lisp evolution



Self-evaluating keywords since 1996



#’ syntax since XEmacs 19.8



Primitive character type since XEmacs 19.20



Built-in multiple values since a couple of months

 XEmacs no later than 21.5 beta 29 \today is required

12/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

The CL package

Common Lisp emulation layer

Provide missing standard utility functions or macros

(e.g. loop)

Extend existing but limited ones

(e.g. mapcar

)

Support full λ -list syntax

(defun

, defmacro

etc.)

typep

setf / defsetf (no setf functions)

13/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Dynamic vs. lexical scope

A CL

Xwork Orange

CL’s lexical-let to the rescue

Not necessary in most cases



Local use of let bindings or function parameters



“Downward funargs” situations



“Upward funargs” situations in some cases

 Only 6 actual uses of lexical-let

14/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Downward funargs: example

Look at the required-classes argument

(defun compute-applicable-methods-using-classes

(gf required-classes)

# | . . . |#

(remove-if-not #’(lambda (method)

(every #’subclassp

required-classes

(method-specializers method)))

(generic-function-methods gf))

# | . . . |# )

15/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Upward funargs: example

Look at the next-emfun argument

(defun compute-primary-emfun (methods)

(if (null methods)

nil

; ; Common L is p v e rs io n :

(let ((next-emfun (compute-primary-emfun (cdr methods))))

#’(lambda (args)

(funcall (method-function (car methods))

args

next-emfun)))

; ; P a r t i a l l y ev a lu a te d Emacs Li sp v e rs io n :

(let ((next-emfun (compute-primary-emfun (cdr methods))))

‘(lambda (args)

(funcall (method-function ’,(car methods))

args ’,next-emfun)))

17/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Upward funargs: explanation

Look at the next-emfun argument

; ; P a r t i a l l y eval uat ed Emacs Li s p v er si on :

(let ((next-emfun (compute-primary-emfun (cdr methods))))

‘(lambda (args)

(funcall (method-function ’,(car methods))

args ’,next-emfun)))

lambda is self-quoting

(lambda ...) is a function designator

e.g. (funcall ’(lambda (x) x) 1)

function ⇐⇒ quote + byte-compiler hint

Note: use byte-compile on the resulting form

18/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Full blown λ -lists

Emacs Lisp restricted to &optional and &rest

CL provides the rest (defun

etc.)

What about generic functions and methods ?



CL provides cl-transform-lambda



Use it in our compute-method-function

λ -list transformation example

; ; ( lambda ( a &o p ti o na l ( b ’b ) &key ( key1 ’ key1 ) ) # | . . . | # )

( lambda ( a &r e st −−rest−−39249 )

( l e t ∗ ( ( b ( i f −−rest−−39249 ( pop −−rest−−39249 ) ( quote b ) ) )

( key1 ( car ( cd r ( or (memq : key1 −−rest−−39249 )

( quote ( n i l key1 ) ) ) ) ) ) )

( l e t ((−−keys−−39250 −−rest−−39249 ) )

( wh i l e −−keys−−39250

( cond ( (memq ( car −−keys−−39250 )

( quote ( : key1 : allow−other−keys ) ) )

( se tq −−keys−−39250 ( cdr ( cdr −−keys−−39250 ) ) ) )

( ( car ( cdr (memq : allow−other−keys −−rest−−39249 ) ) )

( se tq −−keys−−39250 n i l ) )

( t

( e r ro r " Keyword argument %s not one of ( : key1 ) "

( ca r −−keys−−39250 ) ) ) ) ) )

# | . . . | # ) )

20/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Types

X objects are vectors

Built-in types



C level: integers, characters and lrecord types



User level: corresponding type predicate Lisp function



30 LoC to ﬁlter them out

type-of doesn’t work on CL

X objects



Better not hide the true nature of Lisp objects



Not required to make CL

X work



Will work eventually (C level)

typep is provided by CL



Need predicate functions:

(typep obj ’my-type) ⇐⇒ (my-type-p obj)

 ensure-class creates them



Should work on class objects, not only names

 CL

X-speciﬁc defavice around it

22/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Generic functions speciﬁc problems

Generic function objects are funcallable in Common Lisp

This is already working

(setq mygf (defgeneric gf # | . . . |# ))

(typep mygf ’some-gf-class)

X generic functions are vectors



Not funcallable



= discriminating function

function ≈ quote



function doesn’t return a functional value



symbol-function does

23/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Solution

Handle all references to generic functions

This is now working

(typep (symbol-function ’gf) ’some-gf-class)

(typep #’gf ’some-gf-class) ; ; o r j u s t ’ gf

This as well

(find-method (symbol-function ’gf) ; ; . . .

(find-method #’gf ; ; . . . or j u s t ’ gf

And this

(typep (symbol-function ’gf) ’function)

(typep #’gf ’function) ; ; o r j u s t ’ gf

24/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

One (major) drawback

Rock around the CL

 Method specialization on functions impossible

25/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Available Features. . .

Closette (see section 1.1 of the AMOP)



Missing: class redeﬁnition, non-standard method

combinations, eql specializers and :class wide slots

And beyond



:method option to defgeneric



Aware of the others defgeneric and slot options



slot-unbound protocol with emulated

unbound-slot-instance and cell-error-name



Extended slot-missing protocol



Topmost class hierarchy (class, built-in-class,

function, generic-function and method)



Almost complete type/class integration

27/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

. . . Compared to EIEIO

What’s missing



No underlying MOP



No type/class integration



No :around methods



No :method option to defgeneric calls



Buggy λ -list support



Syntactic glitches

Additional bells’n whistles



A class browser



Automatic T

Xinfo documentation



edebug support



Abstract classes, static methods, C++-like slot

protection

28/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Testing

GRR #1: it is better to be correct than to be fast

Why early testing ?



No edebug support



Limited printing facility (circular structures)



gensym infestation

Paul Dietz’s ANSI Common Lisp test suite



900 tests related to objects



Ported to Emacs Lisp



Corollary: rt package ported as well

Current status



50% of the tests pass



100% of the tests that pass are ok ;-)



EIEIO: only 12%

29/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Performance

GRR #2: it is better to be fast than to be slow (someday)

Class def. Method def.

Instantiation

Slot Access

Generic call

10s

100s

EIEIO

CLoX (methods interpreted)

CLoX (methods byte-compiled)

30/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

Conclusion

What we have



An implementation of CLOS for Emacs Lisp



A comprehensive test suite



Corollary: an Emacs Lisp port of rt

Next steps



Complete the feature set



Improve performance (C-level integration)



Port to GNU Emacs

Potential uses



Core features (e.g. speciﬁers)



User libraries (e.g. Gnus

Hmpf!

)

32/33

Didier Verna

Introduction

Context

Motivation

Alternatives

Closette

ELisp vs. Lisp

The CL package

Scoping

λ -lists

Type/Class

Types

Generic Functions

Project status

Available Features

Testing

Performance

Conclusion

Thanks !

That’s all FL

X . . .

Any quesλ ions ?

“And with a four-button mouse it gets even better.

I bound something to Ctrl-Meta-Hyper-Super-button4,

just because I could.”

-- Calle Dybbedahl

33/33