Documentation
¶
Index ¶
- Variables
- func FragLiteral[T internal.TokenTyper](lexer *ActiveLexer[T], chars []rune) (string, error)
- func FragNewline[T internal.TokenTyper](lexer *ActiveLexer[T]) (string, error)
- func LexAtLeastOne[T internal.TokenTyper](lexer *ActiveLexer[T], lex_func LexFunc[T]) (string, error)
- func LexGroup[T internal.TokenTyper](lexer *ActiveLexer[T], is_func func(c rune) bool) (string, error)
- type ActiveLexer
- func (al ActiveLexer[T]) Error() error
- func (al ActiveLexer[T]) HasError() bool
- func (al *ActiveLexer[T]) NextEvents() []*gr.Token[T]
- func (l *ActiveLexer[T]) NextRune() (rune, bool)
- func (l *ActiveLexer[T]) PeekRune() (rune, bool)
- func (l *ActiveLexer[T]) RefuseRune() bool
- func (l ActiveLexer[T]) Tokens() []*gr.Token[T]
- func (al *ActiveLexer[T]) WalkOne(tk *gr.Token[T]) bool
- type Builder
- type LexFunc
- type LexOnceFunc
- type Lexer
Constants ¶
This section is empty.
Variables ¶
var ( // NotFound is an error that is returned when text cannot be lexed yet it is not due to an error // within the text. Readers must return this error as is and not wrap it as callers check for this // error with ==. NotFound error )
Functions ¶
func FragLiteral ¶
func FragLiteral[T internal.TokenTyper](lexer *ActiveLexer[T], chars []rune) (string, error)
FragLiteral lexes a literal if it is found.
Parameters:
- lexer: The lexer.
- chars: The literal.
Returns:
- string: The literal.
- error: An error if the literal is not found or if the literal is not a literal.
Errors:
- NotFound: If the literal is not found.
- *gcers.ErrInvalidParameter: If the lexer is nil or the chars is empty.
- error: any other error.
func FragNewline ¶
func FragNewline[T internal.TokenTyper](lexer *ActiveLexer[T]) (string, error)
FragNewline lexes a newline if it is found.
Parameters:
- l: The lexer.
Returns:
- string: The newline.
- error: An error if the newline is not found or if the newline is not a newline.
Errors:
- NotFound: If the newline is not found.
- *gcers.ErrInvalidParameter: If the lexer is nil.
- error: any other error.
func LexAtLeastOne ¶
func LexAtLeastOne[T internal.TokenTyper](lexer *ActiveLexer[T], lex_func LexFunc[T]) (string, error)
LexAtLeastOne is a convenience function for lexing text according to the lexing function such that at least one character is found and continues until the end of the text (or the valid input) is reached.
Parameters:
- lexer: The lexer.
- lex_func: The lexing function.
Returns:
- string: Text that was lexed.
- error: An error if any.
Errors:
- NotFound: When the group cannot be found because it doesn't exist in the text.
- errors.ErrInvalidParameter: If lexer or lex_func is nil.
- any other error returned by lex_func.
func LexGroup ¶
func LexGroup[T internal.TokenTyper](lexer *ActiveLexer[T], is_func func(c rune) bool) (string, error)
LexGroup is a helper function for lexing a group of characters that satisfy a given predicate according to the following rule:
group+
Parameters:
- lexer: The lexer.
- is_func: The predicate function.
Returns:
- string: The group of characters.
- error: An error if any.
Errors:
- NotFound: When the group cannot be found because it doesn't exist in the text.
- errors.ErrInvalidParameter: If is_func or l is nil.
- any other error returned by l.NextRune() function.
Types ¶
type ActiveLexer ¶
type ActiveLexer[T internal.TokenTyper] struct { // contains filtered or unexported fields }
ActiveLexer is the lexer of the grammar.
func (ActiveLexer[T]) Error ¶
func (al ActiveLexer[T]) Error() error
Error returns the error of the lexer.
Returns:
- error: The error of the lexer.
func (ActiveLexer[T]) HasError ¶
func (al ActiveLexer[T]) HasError() bool
HasError checks whether the lexer has an error.
Returns:
- bool: True if the lexer has an error, false otherwise.
func (*ActiveLexer[T]) NextEvents ¶
func (al *ActiveLexer[T]) NextEvents() []*gr.Token[T]
NextEvents returns the next events of the lexer.
Returns:
- []*grammar.Token: The next events of the lexer.
func (*ActiveLexer[T]) NextRune ¶
func (l *ActiveLexer[T]) NextRune() (rune, bool)
NextRune returns the next rune in the input stream.
Returns:
- rune: The next rune in the input stream.
- bool: True if the next rune exists. False otherwise.
func (*ActiveLexer[T]) PeekRune ¶
func (l *ActiveLexer[T]) PeekRune() (rune, bool)
PeekRune returns the next rune in the input stream without consuming it.
Returns:
- rune: The next rune in the input stream.
- bool: True if the next rune exists. False otherwise.
func (*ActiveLexer[T]) RefuseRune ¶
func (l *ActiveLexer[T]) RefuseRune() bool
RefuseRune rejects the last read rune in the input stream.
Returns:
- bool: True if the last read rune is rejected. False otherwise.
func (ActiveLexer[T]) Tokens ¶
func (l ActiveLexer[T]) Tokens() []*gr.Token[T]
Tokens returns the tokens of the lexer.
Returns:
- []*grammar.Token: The tokens of the lexer.
type Builder ¶
type Builder[T internal.TokenTyper] struct { // contains filtered or unexported fields }
Builder is the builder of the lexer.
func (*Builder[T]) Register ¶
Register registers a new rule for the given character.
Parameters:
- char: The character to register the rule for.
- type_: The type of the token.
- fn: The function to call when the character is encountered.
If fn is nil, Register does nothing. When multiple rules are registered for the same character, the last rule is used.
func (*Builder[T]) RegisterSkip ¶
RegisterSkip registers a new rule for the given character as a skip rule.
Parameters:
- char: The character to register the rule for.
- fn: The function to call when the character is encountered.
If fn is nil, RegisterSkip does nothing. When multiple rules are registered for the same character, the last rule is used.
func (*Builder[T]) Reset ¶
func (b *Builder[T]) Reset()
Reset resets the lexer builder.
This function resets the table and the default case function of the lexer builder.
func (*Builder[T]) SetDefaultCase ¶
func (b *Builder[T]) SetDefaultCase(fn LexOnceFunc[T])
SetDefaultCase sets the default case of the lexer.
Parameters:
- fn: The default case of the lexer.
If fn is nil, the default case is removed.
type LexFunc ¶
type LexFunc[T internal.TokenTyper] func(lexer *ActiveLexer[T]) (string, error)
LexFunc is the lexing function.
Parameters:
- lexer: The lexer.
Returns:
- string: The group of characters.
- error: An error if any.
Errors:
- NotFound: When the group cannot be found because it doesn't exist in the text.
- errors.ErrInvalidParameter: If lexer is nil.
- any other error returned by lexer.NextRune() function or the text is invalid.
type LexOnceFunc ¶
type LexOnceFunc[T internal.TokenTyper] func(lexer *ActiveLexer[T]) ([]*gr.Token[T], error)
LexOnceFunc is the function that lexes the next token of the lexer.
Parameters:
- lexer: The lexer. Assume that lexer is not nil.
Returns:
- []*grammar.Token: The next tokens of the lexer.
- error: An error if the lexer encounters an error while lexing the next token.
type Lexer ¶
type Lexer[T internal.TokenTyper] struct { // contains filtered or unexported fields }
Lexer is the lexer of the grammar.
func (*Lexer[T]) Lex ¶
func (l *Lexer[T]) Lex() iter.Seq[*ActiveLexer[T]]
Lex lexes tokens in the input stream.
Returns:
- error: An error if the lexer encounters an error.
func (Lexer[T]) RuneAt ¶
RuneAt returns the rune at the given position.
Parameters:
- pos: The position of the rune.
Returns:
- rune: The rune.
- bool: True if the rune exists. False otherwise.
func (*Lexer[T]) SetInputStream ¶
SetInputStream sets the input stream of the lexer.
Parameters:
- data: The input stream of the lexer.
Returns:
- error: An error if any.
Source Files