Site References

The Site index, which represents the absolute offset of a Unicode character in the source code text, cannot reliably address a token's absolute offset after source code edits. This is because the token could be shifted left or right, or it could disappear during incremental rescanning, depending on the bounds of the edit.

In contrast, TokenRef::site returns the absolute offset of the beginning of the token's string fragment at the time of the call. In other words, this function returns an updated absolute offset of the token after an edit operation, provided the incremental rescanner did not remove the token during rescanning.

This allows for addressing a token's character bounds relative to changes in the source code.

The SiteRef helper object (backed by the TokenRef under the hood) addresses token bounds. Specifically, this object addresses either the beginning of the token or the end of the source code.

ToSite implements the ToSite trait, so it can be used as a valid bound of a range span.

use lady_deirdre::{
    lexis::{SiteRef, SourceCode, TokenCursor},
    syntax::VoidSyntax,
    units::Document,
};

let mut doc = Document::<VoidSyntax<JsonToken>>::new_mutable("foo [bar] baz");

let brackets_start: SiteRef = doc.cursor(..).site_ref(2);
let brackets_end: SiteRef = doc.cursor(..).site_ref(5);

assert_eq!(doc.substring(brackets_start..brackets_end), "[bar]");

// Rewriting "bar" to "12345".
doc.write(5..8, "12345");

assert_eq!(doc.substring(brackets_start..brackets_end), "[12345]");

Similar to TokenRef, the SiteRef interface has a special nil value and the is_nil test function.