mle/selkirk
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

Support fallible tokenizers #20

Merged
zyxw59 merged 8 commits from fallible-tokenizer into main 2024-07-05 21:26:44 +00:00
Conversation 0 Commits 8 Files changed 4 +441 -178
zyxw59 commented 2024-07-04 22:47:46 +00:00 (Migrated from github.com)
Copy link

Also make CharSetTokenizer even more generic, including adding a BufRead based source.

Closes #18

Also make `CharSetTokenizer` even more generic, including adding a `BufRead` based source. Closes #18
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
mle/selkirk!20
No description provided.