Recent Commits to lark:master

scan: Add LineCounter.advance_to(text, pos) for better performance + minor fixes

Scan: Added small warning (possibly TODO if a need arises)

Add more scan() tests

scan(): search start terminals directly so ignored spans aren't skipped

scan(): carry line/column counts across passes instead of recounting

scan: validate config eagerly, reject custom lexers, tighten errors

- Split ParsingFrontend.scan() into _scan() so configuration errors raise on the call
- Lark.scan() now checks parser='lalr' up front too. Reject custom lexer
- Re-raise ConfigurationError instead of counting it as ValueError
- Check source positions on every replayed token, not just the last
docstring.
- Improves docs
- Adds tests

Export ScanMatch from the top-level lark package

Test that Token end positions survive deepcopy and pickle

Token: preserve end_line/end_column/end_pos in __deepcopy__ and __reduce__

Both methods only copied start_pos/line/column, silently dropping the end
positions.

Add Lark.scan() for finding grammar matches in text, and parsing them.

It finds and parses each non-overlapping match, yielding the longest possible match.

Reimplementation of the long-standing #1429 PR on top of the merged
TextSlice support, with some modifications.

Docs: Added recipe for using scan()

Fix Transformer_NonRecursive crashing with ValueError when root is discarded

When the root tree node's callback returns Discard, the stack is left
empty and the unpacking `result, = stack` raises ValueError. The
recursive Transformer.transform() handles this by returning None;
apply the same guard here.

Fixes #1611

show indenter property docs + add working links

tweaks to address build errors

Merge pull request #1603 from sarathfrancis90/fix/declare-rule-name-crash

Raise GrammarError for non-terminal names in %declare

Raise GrammarError for non-terminal names in %declare

%declare is for terminals, but the grammar accepted any symbol, so a
lowercase (rule) name like `%declare foo` defined an abstract rule with
no tree. The compiler later called iter_subtrees() on that missing tree
and crashed with `AttributeError: 'NoneType' object has no attribute
'iter_subtrees'` instead of reporting the bad grammar.

Reject non-terminal names in %declare with a clear GrammarError.

Merge pull request #1601 from lark-parser/fix_issue1598

Fix empty SPPF node from xearley ignore carry-over (fixes #1598)

Fix empty SPPF node from xearley ignore carry-over (fixes #1598)

When an item in to_scan was carried over past an ignored sequence with
item.node=None (e.g. a prediction at ptr=0 that hasn't matched anything
yet), the carry-over still allocated an empty SymbolNode and stored it
in node_cache. That empty node later became the left child of a
PackedNode when the terminal was finally scanned, and crashed
ForestSumVisitor with "max() iterable argument is empty".

Skip SymbolNode creation when item.node is None — there's nothing to
merge, and new_item.node staying None makes the subsequent PackedNode
have left=None, which is well-handled everywhere else.