Introduction Knuth-Plass
Extension Experiments
Conclusion
Similarity Problems in Paragraph Justication:
an Extension to the Knuth-Plass Algorithm
e DocEng 2024 E
Didier Verna
EPITA / LRE
didier@lrde.epita.fr
lrde/~didier
@didierverna
didier.verna
in/didierverna
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Similarities
In olden times when wishing still helped one,
there lived a king whose daughters were all
b eautiful; and the youngest was so beautiful
that the sun itself, which has seen so much,
was astonished whenever it shone in her face.
Close by the king’s castle lay a great dark for-
est, and under an old lime-tree in the forest
was a well, and when the day was very warm,
the king’s child went out into the forest
:::
and
sat down by the side of the cool fountain;
:::
and
when she was bored she took a golden ball,
:::
and
threw it up on high and caught it; and this ball
was her favorite plaything.
I
T
E
X’s rendition
I
Line width: 201pt
I
Font: Latin Modern Roman 10pt
I
Problems (also at beginning of line)
I
Micro-interruption
I
Accidental line skipping
I
Ancient: scribal errors (monks)
I
Terminology
I
“Saut du même au même” (french)
🤢
I
Homeoarchy / homeoteleuton (rhetoric)
I
Character / word ladders ?
🤔
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 2/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
The Knuth-Plass in a Nutshell
In
beautiful
that
much,
was
was
astonished
and
this
ball
was
her
favorite
plaything.
I
“Single Pair Shortest Path” problem
Finding the “best” route from beginning to end of paragraph
I
Dynamic optimization technique
Never construct the full graph entirely
I
Cost function
I
Local demerits: line by line
Badness, hyphenation penalty, etc.
I
Contextual demerits: line against line
Hyphenation ladders, stretching / shrinking disparities, etc.
I
Decision: min
P
(local + contextual demerits)
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 3/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Knuth-Plass Extension
I
Idea: a new kind of contextual demerits (“similar demerits”)
I
Compare the beginning & end of consecutive lines
I
Add similar demerits to the total if identical
I
Note: special case for the nal 2 lines
I
Implementation:
I
remember bol / eol characters in nodes
I
only up to the rst “glue” (elastic space) or “discretionary” (hyphenation point / ligature)
I
discard kerns
I
Rationale:
I
Discretionaries: avoid deconstruction overhead
I
Glue: avoid vertical alignment considerations
I
Kerns: small adjustments, identical in similarities
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 4/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Comparing two short sequences of characters
He who can do less can do more…
Introduction Knuth-Plass
Extension Experiments
Conclusion
Knuth-Plass Extension
I
Idea: a new kind of contextual demerits (“similar demerits”)
I
Compare the beginning & end of consecutive lines
I
Add similar demerits to the total if identical
I
Note: special case for the nal 2 lines
I
Implementation:
I
remember bol / eol characters in nodes
I
only up to the rst “glue” (elastic space) or “discretionary” (hyphenation point / ligature)
I
discard kerns
I
Rationale:
I
Discretionaries: avoid deconstruction overhead
I
Glue: avoid vertical alignment considerations
I
Kerns: small adjustments, identical in similarities
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 4/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Comparing two short sequences of characters
He who can do less can do more…
Introduction Knuth-Plass
Extension Experiments
Conclusion
Demo
In olden times when wishing still helped one,
there lived a king whose daughters were all
beautiful; and the youngest was so beautiful
that the sun itself, which has seen so much,
was astonished whenever it shone in her face.
Close by the king’s castle lay a great dark for-
est, and under an old lime-tree in the forest
was a well, and when the day was very warm,
the king’s child went out into the forest
:::
and
sat down by the side of the cool fountain;
:::
and
when she was bored she took a golden ball,
:::
and
threw it up on high and caught it; and this ball
was her favorite plaything.
Similar demerits = 0
In olden times when wishing still helped one,
there lived a king whose daughters were all
beautiful; and the youngest was so beautiful
that the sun itself, which has seen so much,
was astonished whenever it shone in her face.
Close by the king’s castle lay a great dark for-
est, and under an old lime-tree in the forest
was a well, and when the day was very warm,
the king’s child went out into the forest
:::
and
sat down by the side of the cool fountain;
:::
and
when she was bored she took a golden ball,
and threw it up on high and caught it; and
this ball was her favorite plaything.
Similar demerits = 2800
In olden times when wishing still helped one,
there lived a king whose daughters were all
beautiful; and the youngest was so beautiful
that the sun itself, which has seen so much,
:::
was
astonished whenever it shone in her face. Close
by the king’s castle lay a great dark forest, and
under an old lime-tree in the forest was a well,
and when the day was very warm, the king’s
child went out into the forest and sat down by
the side of the cool fountain; and when she was
bored she took a golden ball, and threw it up
on high and caught it; and this ball was her
favorite plaything.
Similar demerits = 5230
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 5/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Is it worth it?
I
Pertinence: is the problem frequent?
I
Ecacy: does this Knuth-Plass extension solve the problem?
I
Two typesetting experiments:
I
One paragraph at many dierent widths
I
Many paragraphs at a single width
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 6/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Pertinence
Experimental conditions
I
Grimm Brothers “Frog King” §1
I
142pt (≈ 5cm) → 569pt (≈ 20cm)
I
427 runs
Expected results
I
Chaotic shapes
I
Close numbers for narrow paragraphs
0
200
400
600
800
1000
1200
150 200 250 300 350 400 450 500 550 600
Total
With similarities
Number of paragraph breaking solutions
Paragraph width (pt)
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 7/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Pertinence
Experimental conditions
I
Grimm Brothers “Frog King” §1
I
142pt (≈ 5cm) → 569pt (≈ 20cm)
I
427 runs
Findings
I
Similarities are frequent
I
So are similarity-free layouts
0
200
400
600
800
1000
1200
150 200 250 300 350 400 450 500 550 600
Total
With similarities
Number of paragraph breaking solutions
Paragraph width (pt)
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 7/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Pertinence
Experimental conditions
I
Grimm Brothers “Frog King” §1
I
142pt (≈ 5cm) → 569pt (≈ 20cm)
I
427 runs
What about T
E
X?
I
4%: similarities unavoidable
I
21%: with similarities
I
26%: experiment 2
0
200
400
600
800
1000
1200
150 200 250 300 350 400 450 500 550 600
Total
With similarities
Number of paragraph breaking solutions
Paragraph width (pt)
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 7/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Pertinence
Experimental conditions
I
Grimm Brothers “Frog King” §1
I
142pt (≈ 5cm) → 569pt (≈ 20cm)
I
427 runs
Conclusion
I
2 paragraphs out of 10 is a lot
I
The problem is worth addressing
0
200
400
600
800
1000
1200
150 200 250 300 350 400 450 500 550 600
Total
With similarities
Number of paragraph breaking solutions
Paragraph width (pt)
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 7/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Ecacy
Experimental conditions
I
Herman Melville’s Moby Dick
I
1524 paragraphs @ 284pt (≈ 10cm)
I
+ experiment 1 → 1951 cases
I
x3 → 5853 runs
Conclusion
I
Automatic treatment possible
Max. similar demerits + Disregard adjacency pb.
Corrected: 48% / 50% Corrected: 53% / 66%
Improved: 50% / 63% Improved: 57% / 73%
Please note
I
Those are extreme conditions!
I
Similarity-free 6= aesthetically better
I
But we have some leeway…
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 8/9
Copyright © 2024 Didier Verna, CC BY SA 4.0
Introduction Knuth-Plass
Extension Experiments
Conclusion
Conclusion
I
Similarity handling for the Knuth-Plass
I
Simple & lightweight
I
Backward-compatible with regular T
E
X (similar demerits = 0)
I
Implemented in ETAP, could be useful in production engines
I
Experimentation → automatic similarity handling is both worthy and achievable
Perspectives
I
Study trade-o with other aesthetic criteria
I
Similar demerits as f (similarity) rather than just a scalar value
I
LuaMetaT
E
X −→ left/right “twin” demerits
Similarity Problems in Paragraph Justication: an Extension to the Knuth-Plass Algorithm / DocEng 2024 9/9
Copyright © 2024 Didier Verna, CC BY SA 4.0