Lcl · Core · Packages · Libraries

Doc

Doc – documentation tooling for Lcl.

Extracts ;;; doc comments from source text using the language’s own reader (Lex::commands word spans), renders markdown, and runs the doctests embedded in Examples: sections.

A run of ;;; lines immediately above a proc, macro, namespace, let, or var command documents that definition. An unattached run (a blank line or another command between it and the next definition) is prose: above the first definition of a file or namespace body it is that module’s or namespace’s own documentation; after one, it is a section of text rendered in place between the entries around it. ;;; lines inside braced data are ignored: a line only counts when it is not covered by any word span, so doc comments inside string blobs never leak into the docs. Namespace bodies are recursed into by slicing the body span.

Doc text convention: first line is a one-line summary, then markdown prose, then an optional Examples: section:

  ;;; Add two numbers, slowly.
  ;;;
  ;;; Examples:
  ;;; >> + 1 2
  ;;; 3
  ;;; >> + 1 (2 3)
  ;;; !! expected number

Each >> line is a command; the following non-blank line is the expected repr of its result (!! expects an error whose message contains the given text; no expectation just checks it runs). Consecutive >> lines join into one multi-line command while the accumulated text fails to parse (an open {, ", or [); a line that parses complete closes the command, so single-line examples behave as before. A bare >> is a visual separator. Comparing against repr is deliberate: it keeps examples honest about list-vs-string.

An Examples: section ends at the first non-blank line that follows a blank line and is not a >> command: prose resumes there, and another Examples: may follow later. Long-form docs (a manual written as module prose) can therefore interleave text and executed examples; rendering keeps them in order.

namespace Doc

proc Doc::extract text (base 1)

Extract the doc model from source text.

Returns #{doc <module doc> entries (<entry> ...)}. Entries are #{kind proc|macro name N params P line L doc D}, #{kind let|var name N line L doc D} (only when documented), or #{kind namespace name N line L doc D entries (...)} with the body extracted recursively, or #{kind prose name {} line L doc D} for a section of text between definitions. Names starting with _ are private and skipped. base offsets reported line numbers (used by the recursion; callers can ignore it).

Examples:

>> let m [Doc::extract ";;; doubles\nproc dbl {x} { + \$x \$x }"]
>> get [get [get $m entries] 0] name
"dbl"
>> get [get [get $m entries] 0] doc
"doubles"

proc Doc::extract_file path

Extract the doc model from a file (needs the io package).

proc Doc::parse doc

Parse one doc string into its parts.

Returns #{summary S body B blocks (...) examples (...) example_lines (...)}. blocks is the doc in order, each #{kind prose text T} or #{kind examples lines (...)}; body is all prose joined, example_lines all example lines, and each example is #{cmd C kind none|value|error expect E}.

Examples:

>> get [Doc::parse "sums\n\nExamples:\n>> + 1 2\n3"] summary
"sums"
>> len [get [Doc::parse "sums\n\nExamples:\n>> + 1 2\n3"] examples]
1
>> let d "x\n\nExamples:\n>> proc add {a b} {\n>>   + \$a \$b\n>> }\n>> add 1 2\n3"
>> len [get [Doc::parse $d] examples]
2
>> get [get [get [Doc::parse $d] examples] 1] expect
"3"
>> let two "a\n\nExamples:\n>> + 1 1\n2\n\nMore prose.\n\nExamples:\n>> + 2 2\n4"
>> List::map [get [Doc::parse $two] blocks] [lambda {b} { get $b kind }]
("prose" "examples" "prose" "examples")
>> len [get [Doc::parse $two] examples]
2

proc Doc::anchor qname

The HTML anchor id a rendered entry gets: its qualified name, verbatim. markdown emits <a id="..."></a> before every heading so links survive signature changes (pandoc’s derived heading ids do not), and ?/! are legal in an HTML5 id.

Examples:

>> Doc::anchor "List::push!"
"List::push!"

proc Doc::markdown model (title {}) (link {})

Render a doc model to markdown.

Module doc first, then ## namespace sections with ### entries; proc signatures come from the source’s own word records. Every entry heading is preceded by an <a id> anchor named by anchor. An optional title becomes a leading # heading. link, when given, is a proc mapping a name to a URL (or {}): each [Name] in prose outside code spans that it resolves is rendered as a link, so companion docs can cross-reference by qualified name.

Examples:

>> let m [Doc::extract ";;; See \[M::f\].\nnamespace M { ;;; f\n proc f {} { 1 } }"]
>> >= [String::find [Doc::markdown $m] "<a id=\"M::f\"></a>"] 0
1
>> let md [Doc::markdown $m {} [lambda {n} { "u#$n" }]]
>> >= [String::find $md "See \[`M::f`\](u#M::f)."] 0
1

proc Doc::markdown_file path (title {}) (link {})

Extract a file and render it to markdown (needs the io package). The title defaults to the path; link is passed to markdown.

proc Doc::link_text text link

Link [Name] references in free-form markdown text, the same way markdown does for extracted docs. Lets a site builder run plain .md pages through the same resolver.

Examples:

>> Doc::link_text "See \[f\] and `\[f\]`." [lambda {n} { "#$n" }]
"See \[`f`](#f) and `\[f]`."
>> Doc::link_text "no \[zz\] here" [lambda {n} { {} }]
"no \[zz] here"

proc Doc::doctest model

Run every doctest in a doc model.

Examples run in the doctest’s own scope; namespaces referenced in examples must already be loaded (e.g. require the module under test first). Returns #{pass N fail N failures (#{name cmd expect got} ...)}.

proc Doc::doctest_file path

Extract a file and run its doctests (needs the io package).

proc Doc::report res

Print doctest results; returns the number of failures.

proc Doc::register text (name "<registered>")

Register the docs in source text under a source name.

Indexes the ;;; docs in text so lookup and describe can find them. name is the file the text came from, or any label for source the host owns (an editor buffer, an embedded library). Procs loaded from a file are matched by their Proc::origin line when name is that file; everything else, including companion docs for C procs, is matched by qualified name. Registering a name again replaces its entries. Returns the number of entries indexed.

Examples:

>> Doc::register ";;; Two.\nnamespace Tw { ;;; One.\n proc one {} { 1 } }" "tw"
2
>> get [Doc::lookup "Tw::one"] doc
"One."

proc Doc::register_file path

Register a file’s docs under its path (needs the io package).

proc Doc::lookup p

Find the doc entry for a live proc or a name.

p is a proc value or a name, qualified or not. An Lcl proc is matched by its Proc::origin file and line; the file is read and indexed on first use. A C proc, a name that is not live, or a proc whose file is unavailable falls back to the registered docs by qualified name, then (for an unqualified name) by bare name, which must be unique across the index. A line hit whose entry name differs from the proc’s own name is treated as stale (the source changed since it was loaded) and ignored. Returns the entry, #{kind K qname Q name N params P line L doc D file F}, or #{} when nothing is documented.

Examples:

>> Doc::register ";;; Adds.\nproc lk_add {a b} { + \$a \$b }" "lk"
1
>> get [Doc::lookup lk_add] params
"a b"
>> Doc::lookup lk_nothing
#{}

proc Doc::describe p

Describe a live proc or a name: signature, origin, and doc text.

The entry comes from lookup. When p is a name, that name is the one printed (the index may know the same proc under a shorter one). A live proc with no docs still gets its signature from Proc::params and its Proc::origin. Errors when p is neither live nor documented.

Examples:

>> Doc::register ";;; Adds.\n;;;\n;;; Slowly.\nproc ds_add {a b} { + \$a \$b }" "ds"
1
>> String::split [Doc::describe ds_add] "\n"
("proc ds_add {a b}" "  ds:4" "" "Adds." "" "Slowly.")
>> String::split [Doc::describe +] "\n"
("proc + {}" "" "(undocumented)")
>> Doc::describe ds_nothing
!! nothing known about "ds_nothing"

proc Doc::entries

Every indexed doc entry, as a flat list.

Covers the sources seen so far: registered explicitly, or read on demand by lookup. It is not a walk of the live interpreter; register_file the files you want covered.

Examples:

>> Doc::register ";;; A.\nproc en_a {} { 1 }\n;;; B.\nproc en_b {} { 2 }" "en"
2
>> List::map [Doc::search "en_"] [lambda {e} { get $e qname }]
("en_a" "en_b")

proc Doc::search sub

Search the index: entries whose qualified name or doc text contains sub, case-insensitively.

proc Doc::run_value p

Run the doctests of one documented symbol.

p is anything lookup accepts. Returns #{pass N fail N failures (...)} as doctest does, and errors when nothing is documented for p. Examples run in the doctest’s own scope, so the names they use must be live.

Examples:

>> Doc::register ";;; Adds.\n;;;\n;;; Examples:\n;;; >> + 1 2\n;;; 3\nproc rv_add {a b} { + \$a \$b }" "rv"
1
>> get [Doc::run_value rv_add] pass
1
>> Doc::run_value rv_nothing
!! nothing documented

proc Doc::run_ns ns

Run the doctests of a namespace and everything under it.

ns is a namespace name or value. When it is live, its members are looked up first so file-backed docs are read on demand; then every indexed entry whose qualified name is ns or starts with ns:: runs, the namespace’s own doc included. Returns the merged #{pass N fail N failures (...)}.

Examples:

>> Doc::register "namespace Rn { ;;; Examples:\n ;;; >> + 1 1\n ;;; 2\n proc a {} { 1 }\n ;;; Examples:\n ;;; >> + 1 1\n ;;; 3\n proc b {} { 1 } }" "rn"
3
>> let r [Doc::run_ns Rn]
>> list [get $r pass] [get $r fail] [get [get [get $r failures] 0] name]
(1 1 "Rn::b")
>> list [get [Doc::run_ns Rn_nothing] pass] [get [Doc::run_ns Rn_nothing] fail]
(0 0)