Header menu logo F# Compiler Guide

FSharpLineTokenizer Type

Object to tokenize a line of F# source code, starting with the given lexState. The lexState should be FSharpTokenizerLexState.Initial for the first line of text. Returns an array of ranges of the text and two enumerations categorizing the tokens and characters covered by that range, i.e. FSharpTokenColorKind and FSharpTokenCharKind. The enumerations are somewhat adhoc but useful enough to give good colorization options to the user in an IDE. A new lexState is also returned. An IDE-plugin should in general cache the lexState values for each line of the edited code.

Instance members

Instance member Description

this.ScanToken

Full Usage: this.ScanToken

Parameters:
Returns: FSharpTokenInfo option * FSharpTokenizerLexState

Scan one token from the line

lexState : FSharpTokenizerLexState
Returns: FSharpTokenInfo option * FSharpTokenizerLexState

Static members

Static member Description

FSharpLineTokenizer.ColorStateOfLexState(arg1)

Full Usage: FSharpLineTokenizer.ColorStateOfLexState(arg1)

Parameters:
Returns: FSharpTokenizerColorState

Get the color state from the lexer state

arg0 : FSharpTokenizerLexState
Returns: FSharpTokenizerColorState

FSharpLineTokenizer.LexStateOfColorState(arg1)

Full Usage: FSharpLineTokenizer.LexStateOfColorState(arg1)

Parameters:
Returns: FSharpTokenizerLexState

Get a default lexer state for a color state. NOTE: This may result in an inaccurate lexer state not taking into account things such as the #if/#endif and string interpolation context within the file

arg0 : FSharpTokenizerColorState
Returns: FSharpTokenizerLexState

Type something to start searching.