[#10] Make `TokenKind` and `ExpressionKind` more generic #12

Merged

zyxw59 merged 9 commits from 10-iterator-adapters into main

2024-02-18 04:53:23 +00:00

zyxw59 commented

2024-02-18 02:58:17 +00:00

(Migrated from github.com)

Closes #10

Replaces the Tokenizer trait with Iterator<Item = (Token<'s>, T)>, where T contains any additional data about the token (e.g. a token kind, specific to that tokenizer)
Adds a type parameter to the ParseContext trait, which corresponds to the T in the tokenizer iterator
Adds three new associated types to the ParseContext trait: Term (replacing the specific type variants on Expression), Precedence (replacing the Precedence enum), and Error (replacing some specific variants on ParseErrorKind)
Refactors the Parser implementation to no longer rely on Null expressions to support mixed-arity operators and possibly-empty delimiters.
Replaces the specific term variants of the Expression enum with a generic Term variant

Closes #10 - Replaces the `Tokenizer` trait with `Iterator<Item = (Token<'s>, T)>`, where `T` contains any additional data about the token (e.g. a token kind, specific to that tokenizer) - Adds a type parameter to the `ParseContext` trait, which corresponds to the `T` in the tokenizer iterator - Adds three new associated types to the `ParseContext` trait: `Term` (replacing the specific type variants on `Expression`), `Precedence` (replacing the `Precedence` enum), and `Error` (replacing some specific variants on `ParseErrorKind`) - Refactors the `Parser` implementation to no longer rely on `Null` expressions to support mixed-arity operators and possibly-empty delimiters. - Replaces the specific term variants of the `Expression` enum with a generic `Term` variant