Our Goal
We would like to introduce a user-defined construct called
destructuring-case, which efficiently selects a clause to evaluate
designated by a destructuring lambda list depending on run-time value of
a given expression.
There semantics of the macro usage should be intuitive.
There are several cases to consider.
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 2 / 32
Different optional arguments
( destru c t uring-c a s e expression
(( X & optional (Y 1))
(* X Y ))
(( X & key (Y 1))
(* X Y ))
(( X & key (Y 1) (Z 0) & allow-oth e r - keys )
(+ (* X Y) Z )))
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 4 / 32
Types of arguments
( destru c t uring-c a s e expression
(( X Y)
( declare ( type fixnum X Y ))
(* X Y ))
(( X Y)
( declare ( type fixnum X )
( type integer Y))
(* X Y ))
(( X Y)
( declare ( type (or string fixnum ) X)
( type number Y ))
(* (if ( stringp X)
( string-t o - number X)
X)
Y )))
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 5 / 32
Does this sequence:
(a 8 8.0 b "a" "an" "the" c 8 88 888 d 8/3)
follow the pattern: (symbol · (number
+
∨ string
+
))
+
?
We construct a
deterministic
finite
automaton
(DFA).
We want to
support :not
and :and in
our DSL.
0 1
2
3
symbol
number
string
symbol
number
symbol
string
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 8 / 32
Code generated from (symbol · (number
+
∨ string
+
))
+
( tag b ody
0
( u n l e s s se q ( r e t u r n n i l ) )
( t y p e c a s e ( pop s e q )
( symbol ( go 1 ) )
( t ( r e t u r n n i l ) ) )
1
( u n l e s s se q ( r e t u r n n i l ) )
( t y p e c a s e ( pop s e q )
( number ( go 2 ) )
( s t r i n g ( go 3 ) )
( t ( r e t u r n n i l ) ) )
2
( u n l e s s se q ( r e t u r n t ) )
( t y p e c a s e ( pop s e q )
( number ( go 2 ) )
( symbol ( go 1 ) )
( t ( r e t u r n n i l ) ) )
0 1
2
3
symbol
number
string
symbol
number
symbol
string
3
( u n l e s s se q ( r e t u r n t ) )
( t y p e c a s e ( pop s e q )
( s t r i n g ( go 3 ) )
( symbol ( go 1 ) )
( t ( r e t u r n n i l ) ) ) ) )
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 9 / 32
Introducing Regular Type Expression
A Regular Type Expression (RTE) is a surface syntax DSL expressing
regular type patterns in sequences.
symbol · (rational
∗
∨ float
+
)
∧ t · ratio
?
· number
RTE DSL notation:
( : and ( : c a t symbol
( : or ( : ∗ r a t i o n a l )
(:+ f l o a t ) ) )
( : not ( : c a t t ( : ? r a t i o ) number ) ) )
Regular type expressions express components:
required, optional, repeating, and typed.
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 10 / 32
Lambda-lists characterized by regular patterns
A lambda-list in Common Lisp has a fixed part, an optional part, and
a repeating part.
( destru c t uring-b i n d (a b & op tional c & key x y)
DATA
...)
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 12 / 32
Lambda-lists characterized by regular patterns
A lambda-list in Common Lisp has a fixed part, an optional part, and a
repeating part part. Any of the variables may be restricted by
type declarations.
( destru c t uring-b i n d (a b & op tional c & key x y)
DATA
( declare ( type integer a x)
( type string b c y ))
...)
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 12 / 32
Macro: destructuring-case
( destru c t uring-c a s e expression
(( X Y)
( declare ( type fixnum X Y ))
: clause-1 )
(( X Y)
( declare ( type fixnum X )
( type integer Y))
: clause-2 )
(( X Y)
( declare ( type (or string fixnum ) X)
( type number Y ))
: clause-3 ))
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 14 / 32
Expansion of destructuring-case
( rt e − c a s e e x p r e s s i o n
( ( : c a t fix n u m f ixnum )
( d e s t r u c t u r i n g − b i n d (X Y) e x p r e s s i o n
( d e c l a r e ( ty p e f i x num X Y) )
: cl aus e − 1 ) )
( ( : c a t fix n u m i n t e g e r )
( d e s t r u c t u r i n g − b i n d (X Y) e x p r e s s i o n
( d e c l a r e ( ty p e f i x num X)
( ty p e i n t e g e r Y) )
: cl aus e − 2 ) )
( ( : c a t ( o r s t r i n g f i xnum ) number )
( d e s t r u c t u r i n g − b i n d (X Y) e x p r e s s i o n
( d e c l a r e ( ty p e ( or s t r i n g fixnum ) X)
( ty p e number Y) )
: cl aus e − 3 ) ) )
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 15 / 32
Simplified rte-case expansion
( rte-case express i on
((: cat fixnum fixnum )
: clause-1 )
((: cat fixnum integer )
: clause-2 )
((: cat ( or string fixnum ) number )
: clause-3 ))
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 16 / 32
Automata for clauses of rte-case
( rt e − c a s e e x p r e s s i o n
( ( : c a t fix n u m
fixnum )
: cl aus e − 1 )
1.0 1.1
fixnum
1.2
fixnum
clause-1
( ( : c a t fix n u m
i n t e g e r )
: cl aus e − 2 )
2.0 2.1
fixnum
2.2
integer
clause-2
( ( : c a t ( o r s t r i n g
fixnum )
number )
: cl aus e − 3 ) )
3.0 3.1
(or string
fixnum)
3.2
number
clause-3
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 17 / 32
Automata for clauses of rte-case
( r t e − c a s e e x p r e s s i o n
( ( : c a t fi xn u m
fi x n u m )
: c l au s e − 1 )
1.0 1.1
fixnum
1.2
fixnum
clause-1
( ( : c a t fi xn u m
i n t e g e r )
: c l au s e − 2 )
2.0 2.1
fixnum
2.2
integer
clause-2
( ( : c a t ( o r s t r i n g
fi x n u m )
number )
: c l au s e − 3 ) )
3.0 3.1
(or string
fixnum)
3.2
number
clause-3
We could select the appropriate clause by executing the three automata in
turn at run-time.
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 18 / 32
Automata for clauses of rte-case
( r t e − c a s e e x p r e s s i o n
( ( : c a t fi xn u m
fi x n u m )
: c l au s e − 1 )
1.0 1.1
fixnum
1.2
fixnum
clause-1
( ( : c a t fi xn u m
i n t e g e r )
: c l au s e − 2 )
2.0 2.1
fixnum
2.2
integer
clause-2
( ( : c a t ( o r s t r i n g
fi x n u m )
number )
: c l au s e − 3 ) )
3.0 3.1
(or string
fixnum)
3.2
number
clause-3
We can do better.
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 18 / 32
DFAs for disjoined clause-1, clause-2, and clause-3
( rt e − c a s e e x p r e s s i o n
( ( : c a t fix n u m
fixnum )
: cl aus e − 1 )
1.0 1.1
fixnum
1.2
fixnum
clause-1
( ( and ( : c a t fix nu m
i n t e g e r )
( : n o t . . . T1 . . . ) )
: c l au s e − 2 )
2.0 2.1
fixnum
2.2
fixnum
2.3
bignum
clause-2
( ( : and ( : c a t ( o r s t r i n g
fi x n u m )
number )
( : n o t . . . T1 . . . )
( : n o t . . . T2 . . . ) )
: c l au s e − 3 ) )
3.0
3.1
string
3.3
fixnum
3.2
number
clause-3
3.4
integer
3.5
(and (not integer)
number)
clause-3
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 19 / 32
Calculating synchronized cross product
We can merge the three disjoint automata into one single automata.
Worst-case run-time is divided by 3.
1.0 1.1
fixnum
1.2
fixnum
clause-1
2.0 2.1
fixnum
2.2
fixnum
2.3
bignum
clause-2
0
3.0
3.1
string
3.3
fixnum
3.2
number
clause-3
3.4
integer
3.5
(and (not integer)
number)
clause-3
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 20 / 32
CXP: after fixnum
Easy, because fixnum transition is found on each input DFA.
1.0 1.1
FIXNUM
1.2
fixnum
clause-1
2.0 2.1
FIXNUM
2.2
fixnum
2.3
bignum
clause-2
0 1
FIXNUM
3.0
3.1
string
3.3
FIXNUM
3.2
number
clause-3
3.4
integer
3.5
(and (not integer)
number)
clause-3
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 21 / 32
CXP: after string
Easy, because string and fixnum are disjoint transitions of state 3.0.
1.0 1.1
fixnum
1.2
fixnum
clause-1
2.0 2.1
fixnum
2.2
fixnum
2.3
bignum
clause-2
0
1
fixnum
2
STRING
3.0
3.1
STRING
3.3
fixnum
3.2
number
clause-3
3.4
integer
3.5
(and (not integer)
number)
clause-3
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 22 / 32
CXP: after fixnum fixnum
Challenging, because fixnum is not found on DFA 3.
(subtypep fixnum integer) ?
1.0 1.1
fixnum
1.2
FIXNUM
clause-1
2.0 2.1
fixnum
2.2
FIXNUM
2.3
bignum
clause-2
0
1
string
2
fixnum
3
number
4
FIXNUM
3.0
3.1
string
3.3
fixnum
3.2
number
clause-3
3.4
INTEGER
3.5
(and (not integer)
number)
clause-3
Challenging because subtypep might return nil, nil.
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 23 / 32
Short Demo
All the valid defmethod forms which are unaccounted for.
0 1
defmethod
2
good-name
4
!-list
3
list
!-list
7
list
6
doc/declaration
5
!-doc/declaration
missing
missing
t
missing
declaration
!-doc/declaration
8
docstring
missing
declaration
!-declaration
missing
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 28 / 32
Short Demo
All the remaining ways a valid defmethod form can appear, some
accounted for in the destructuring-case and some accounted for.
0
2
!-defmethod
1
defmethod
not-defmethod
t
not-defmethod
bad-name
3
good-name
not-defmethod
5
!-list
4
list
not-defmethod
!-list
9
list
not-defmethod
8
declaration
7
docstring
6
!-doc/declaration
missing
11
t
missing
10
t
missing
t
missing
!-doc/declaration
13
declaration
12
docstring
missing
t
defmethod-1
t
defmethod-2
15
t
missing
!-declaration
14
declaration
missing
16
t
missing
t
defmethod-3
t
defmethod-4
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 29 / 32
Summary
Our implmentation of an N-clause destructuring-case reduces the
number of traversals of the sequence in question from N + 1 to 2, once for
descrimination, and one for binding.
The code is available from quicklisp via package :rte.
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 30 / 32
Perspectives
Lots more to be done: benchmarking, connection to method dispatch...
There are two CloJure libraries seqspec and spec which seem very
related. According to the author of seqspec, seqspec does not optimize
using finite automata because of some annoying limitations of the JVM.
Thanks to Didier Verna for begin my PhD advisor for the past 3 years.
Also thanks to Robert Strandh, Pascal Costanza, and Christophe Rhodes
for serving on my PhD defense committee.
Jim Newton (12th European Lisp Symposium)Finite Automata Theory Based Optimization of Conditional Variable Binding1,2 April 2019 31 / 32
