mle/selkirk
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

Document how to handle fallible tokenizers #18

New issue
Closed
opened 2024-07-03 02:50:55 +00:00 by zyxw59 · 1 comment
zyxw59 commented 2024-07-03 02:50:55 +00:00 (Migrated from github.com)
Copy link

The preferred way to handle a fallible tokenizer is to pass the errors on to the parser, and then have the parser produce an error.

The preferred way to handle a fallible tokenizer is to pass the errors on to the parser, and then have the parser produce an error.
zyxw59 commented 2024-07-04 15:33:03 +00:00 (Migrated from github.com)
Copy link

On further thought, this is kinda silly. Just because the first tokenizer we implemented was nearly infallible, doesn't mean that all tokenizers will be.

On further thought, this is kinda silly. Just because the first tokenizer we implemented was nearly infallible, doesn't mean that all tokenizers will be.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
postfix-delim-precedence
No results found.
Labels
Clear labels   
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
mle/selkirk#18
No description provided.