In order to discover both domestic and relevant foreign definitions,
Declt uses a heuristic which establishes different tradeoffs between
performance and accuracy. Remember that this is controlled by the
:introspection-level option to declt or assess.
This heuristic is based on the assumption that most libraries out there
are implemented within packages of their own.
Consequently, this is what happens by default (that is, at introspection
level 1). After the library has been loaded, Declt scans all live
packages and selects the domestic ones. It then scans all symbols from
these packages to find domestic definitions.
Note that if we were to stop there, we would probably miss many domestic
definitions, namely, those on foreign symbols such as methods on
standard generic functions such as make-instance. Fortunately, a
lot of these will be recovered later on anyway, when Declt computes
cross-references between definitions in the so-called finalization
phase.
Suppose for instance that the library provides a new method on
make-instance. This method is domestic because it is defined in
one of the library’s files, but at introspection level 1, Declt only
scans domestic symbols, so it will miss it (common-lisp, the home
package of the make-instance symbol, is foreign). On the other
hand, it is very likely that this method exists because it specializes
on a domestic class already known to Declt. When Declt finalizes a
class definition, it looks up methods that specialize on it (among other
things). These methods are in fact accessible through the class by
introspection (they are called “direct methods”). Consequently,
Declt will eventually discover the new make-instance
method, figure out that it is a domestic one, and add it to the
documentation.
In the end, it is probable that most domestic definitions end up being discovered, either during the initial scanning phase, or later on, during the finalization phase when cross-references are computed. This is why the default introspection level is probably good enough most of the time.
On the other hand, some (rare?) domestic definitions may still escape
the discovery process at that level. For example, if you define a
foreign global variable in one of your library’s files, chances are that
there won’t be any domestic definition cross-referencing it. This is why
Declt provides a second level of introspection, in which it initially
scans all symbols in the Common Lisp image, rather than just the ones
from domestic packages. The resulting documentation will then be more
complete, although at the expense of a much greater computation
time.
And yes, you have just noticed that I said “more complete” rather than “exhaustive”, and mind you, this is not because my vocabulary is limited. The thing is that even with an initial scan of all symbols, we may still miss some information. To be precise, we won’t miss any domestic definition. It is the cross-referencing information that may remain incomplete, and that is because the finalization phase doesn’t re-scan each and every symbol in the Lisp image again; only the definitions that have been created so far. Let’s take an example (granted, a contrived one; but aren’t they all?). You define a regular domestic function, and it is used as the update function in a foreign setf expander. Even with an initial scan of all symbols in the Lisp image, you’ll get a definition for your domestic function, but that’s it. Later on, during the finalization process, there’s no way to go from the function to the setf expander, so that information remains unknown. In order to fix that, the finalization process would need to re-scan the whole Lisp world again, and that would become introspection level 3. Not sure it’s actually worth it; maybe one day...