F# Compiler Services


Compiler Services: Processing typed expression tree

This tutorial demonstrates how to get the checked, typed expressions tree (TAST) for F# code and how to walk over the tree.

This can be used for creating tools such as source code analyzers and refactoring tools. You can also combine the information with the API available from symbols.

NOTE: The FSharp.Compiler.Service API is subject to change when later versions of the nuget package are published

Getting checked expressions

To access the type-checked, resolved expressions, you need to create an instance of InteractiveChecker.

To use the interactive checker, reference FSharp.Compiler.Service.dll and open the SourceCodeServices namespace:

1: 
2: 
3: 
4: 
#r "FSharp.Compiler.Service.dll"
open System
open System.IO
open Microsoft.FSharp.Compiler.SourceCodeServices

Checking code

We first parse and check some code as in the symbols tutorial. One difference is that we set keepAssemblyContents to true.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
// Create an interactive checker instance 
let checker = FSharpChecker.Create(keepAssemblyContents=true)

let parseAndCheckSingleFile (input) = 
    let file = Path.ChangeExtension(System.IO.Path.GetTempFileName(), "fsx")  
    File.WriteAllText(file, input)
    // Get context representing a stand-alone (script) file
    let projOptions = 
        checker.GetProjectOptionsFromScript(file, input)
        |> Async.RunSynchronously

    checker.ParseAndCheckProject(projOptions) 
    |> Async.RunSynchronously

Getting the expressions

After type checking a file, you can access the declarations and contents of the assembly, including expressions:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
let input2 = 
      """
module MyLibrary 

open System

let foo(x, y) = 
    let msg = String.Concat("Hello", " ", "world")
    if msg.Length > 10 then 
        10 
    else 
        20

type MyClass() = 
    member x.MyMethod() = 1
      """
let checkProjectResults = 
    parseAndCheckSingleFile(input2)

checkProjectResults.Errors // should be empty

Checked assemblies are made up of a series of checked implementation files. The "file" granularity matters in F# because initialization actions are triggered at the granularity of files. In this case there is only one implementation file in the project:

1: 
let checkedFile = checkProjectResults.AssemblyContents.ImplementationFiles.[0]

Checked assemblies are made up of a series of checked implementation files. The "file" granularity matters in F# because initialization actions are triggered at the granularity of files. In this case there is only one implementation file in the project:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
let rec printDecl prefix d = 
    match d with 
    | FSharpImplementationFileDeclaration.Entity (e, subDecls) -> 
        printfn "%sEntity %s was declared and contains %d sub-declarations" prefix e.CompiledName subDecls.Length
        for subDecl in subDecls do 
            printDecl (prefix+"    ") subDecl
    | FSharpImplementationFileDeclaration.MemberOrFunctionOrValue(v, vs, e) -> 
        printfn "%sMember or value %s was declared" prefix  v.CompiledName
    | FSharpImplementationFileDeclaration.InitAction(e) -> 
        printfn "%sA top-level expression was declared" prefix 


for d in checkedFile.Declarations do 
   printDecl "" d

// Entity MyLibrary was declared and contains 4 sub-declarations
//     Member or value foo was declared
//     Entity MyClass was declared and contains 0 sub-declarations
//     Member or value .ctor was declared
//     Member or value MyMethod was declared

As can be seen, the only declaration in the implementation file is that of the module MyLibrary, which contains fours sub-declarations.

As an aside, one peculiarity here is that the member declarations (e.g. the "MyMethod" member) are returned as part of the containing module entity, not as part of their class. Note that the class constructor is returned as a separate declaration. The class type definition has been "split" into a constructor and the other declarations.

1: 
2: 
3: 
4: 
let myLibraryEntity, myLibraryDecls =    
   match checkedFile.Declarations.[0] with 
   | FSharpImplementationFileDeclaration.Entity (e, subDecls) -> (e, subDecls)
   | _ -> failwith "unexpected"

What about the expressions, for example the body of function "foo"? Let's find it:

1: 
2: 
3: 
4: 
let (fooSymbol, fooArgs, fooExpression) = 
    match myLibraryDecls.[0] with 
    | FSharpImplementationFileDeclaration.MemberOrFunctionOrValue(v, vs, e) -> (v, vs, e)
    | _ -> failwith "unexpected"

Here 'fooSymbol' is a symbol associated with the declaration of 'foo', 'fooArgs' represents the formal arguments to the 'foo' function, and 'fooExpression' is an expression for the implementation of the 'foo' function.

Once you have an expression, you can work with it much like an F# quotation. For example, you can find its declaration range and its type:

1: 
2: 
fooExpression.Type  // shows that the return type of the body expression is 'int'
fooExpression.Range  // shows the declaration range of the expression implementing 'foo'

Walking over expressions

Expressions are analyzed using active patterns, much like F# quotations. Here is a generic expression visitor:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
34: 
35: 
36: 
37: 
38: 
39: 
40: 
41: 
42: 
43: 
44: 
45: 
46: 
47: 
48: 
49: 
50: 
51: 
52: 
53: 
54: 
55: 
56: 
57: 
58: 
59: 
60: 
61: 
62: 
63: 
64: 
65: 
66: 
67: 
68: 
69: 
70: 
71: 
72: 
73: 
74: 
75: 
76: 
77: 
78: 
79: 
80: 
81: 
82: 
83: 
84: 
85: 
86: 
87: 
88: 
89: 
90: 
91: 
92: 
93: 
94: 
95: 
96: 
let rec visitExpr f (e:FSharpExpr) = 
    f e
    match e with 
    | BasicPatterns.AddressOf(lvalueExpr) -> 
        visitExpr f lvalueExpr
    | BasicPatterns.AddressSet(lvalueExpr, rvalueExpr) -> 
        visitExpr f lvalueExpr; visitExpr f rvalueExpr
    | BasicPatterns.Application(funcExpr, typeArgs, argExprs) -> 
        visitExpr f funcExpr; visitExprs f argExprs
    | BasicPatterns.Call(objExprOpt, memberOrFunc, typeArgs1, typeArgs2, argExprs) -> 
        visitObjArg f objExprOpt; visitExprs f argExprs
    | BasicPatterns.Coerce(targetType, inpExpr) -> 
        visitExpr f inpExpr
    | BasicPatterns.FastIntegerForLoop(startExpr, limitExpr, consumeExpr, isUp) -> 
        visitExpr f startExpr; visitExpr f limitExpr; visitExpr f consumeExpr
    | BasicPatterns.ILAsm(asmCode, typeArgs, argExprs) -> 
        visitExprs f argExprs
    | BasicPatterns.ILFieldGet (objExprOpt, fieldType, fieldName) -> 
        visitObjArg f objExprOpt
    | BasicPatterns.ILFieldSet (objExprOpt, fieldType, fieldName, valueExpr) -> 
        visitObjArg f objExprOpt
    | BasicPatterns.IfThenElse (guardExpr, thenExpr, elseExpr) -> 
        visitExpr f guardExpr; visitExpr f thenExpr; visitExpr f elseExpr
    | BasicPatterns.Lambda(lambdaVar, bodyExpr) -> 
        visitExpr f bodyExpr
    | BasicPatterns.Let((bindingVar, bindingExpr), bodyExpr) -> 
        visitExpr f bindingExpr; visitExpr f bodyExpr
    | BasicPatterns.LetRec(recursiveBindings, bodyExpr) -> 
        List.iter (snd >> visitExpr f) recursiveBindings; visitExpr f bodyExpr
    | BasicPatterns.NewArray(arrayType, argExprs) -> 
        visitExprs f argExprs
    | BasicPatterns.NewDelegate(delegateType, delegateBodyExpr) -> 
        visitExpr f delegateBodyExpr
    | BasicPatterns.NewObject(objType, typeArgs, argExprs) -> 
        visitExprs f argExprs
    | BasicPatterns.NewRecord(recordType, argExprs) ->  
        visitExprs f argExprs
    | BasicPatterns.NewTuple(tupleType, argExprs) -> 
        visitExprs f argExprs
    | BasicPatterns.NewUnionCase(unionType, unionCase, argExprs) -> 
        visitExprs f argExprs
    | BasicPatterns.Quote(quotedExpr) -> 
        visitExpr f quotedExpr
    | BasicPatterns.FSharpFieldGet(objExprOpt, recordOrClassType, fieldInfo) -> 
        visitObjArg f objExprOpt
    | BasicPatterns.FSharpFieldSet(objExprOpt, recordOrClassType, fieldInfo, argExpr) -> 
        visitObjArg f objExprOpt; visitExpr f argExpr
    | BasicPatterns.Sequential(firstExpr, secondExpr) -> 
        visitExpr f firstExpr; visitExpr f secondExpr
    | BasicPatterns.TryFinally(bodyExpr, finalizeExpr) -> 
        visitExpr f bodyExpr; visitExpr f finalizeExpr
    | BasicPatterns.TryWith(bodyExpr, _, _, catchVar, catchExpr) -> 
        visitExpr f bodyExpr; visitExpr f catchExpr
    | BasicPatterns.TupleGet(tupleType, tupleElemIndex, tupleExpr) -> 
        visitExpr f tupleExpr
    | BasicPatterns.DecisionTree(decisionExpr, decisionTargets) -> 
        visitExpr f decisionExpr; List.iter (snd >> visitExpr f) decisionTargets
    | BasicPatterns.DecisionTreeSuccess (decisionTargetIdx, decisionTargetExprs) -> 
        visitExprs f decisionTargetExprs
    | BasicPatterns.TypeLambda(genericParam, bodyExpr) -> 
        visitExpr f bodyExpr
    | BasicPatterns.TypeTest(ty, inpExpr) -> 
        visitExpr f inpExpr
    | BasicPatterns.UnionCaseSet(unionExpr, unionType, unionCase, unionCaseField, valueExpr) -> 
        visitExpr f unionExpr; visitExpr f valueExpr
    | BasicPatterns.UnionCaseGet(unionExpr, unionType, unionCase, unionCaseField) -> 
        visitExpr f unionExpr
    | BasicPatterns.UnionCaseTest(unionExpr, unionType, unionCase) -> 
        visitExpr f unionExpr
    | BasicPatterns.UnionCaseTag(unionExpr, unionType) -> 
        visitExpr f unionExpr
    | BasicPatterns.ObjectExpr(objType, baseCallExpr, overrides, interfaceImplementations) -> 
        visitExpr f baseCallExpr
        List.iter (visitObjMember f) overrides
        List.iter (snd >> List.iter (visitObjMember f)) interfaceImplementations
    | BasicPatterns.TraitCall(sourceTypes, traitName, typeArgs, typeInstantiation, argTypes, argExprs) -> 
        visitExprs f argExprs
    | BasicPatterns.ValueSet(valToSet, valueExpr) -> 
        visitExpr f valueExpr
    | BasicPatterns.WhileLoop(guardExpr, bodyExpr) -> 
        visitExpr f guardExpr; visitExpr f bodyExpr
    | BasicPatterns.BaseValue baseType -> ()
    | BasicPatterns.DefaultValue defaultType -> ()
    | BasicPatterns.ThisValue thisType -> ()
    | BasicPatterns.Const(constValueObj, constType) -> ()
    | BasicPatterns.Value(valueToGet) -> ()
    | _ -> failwith (sprintf "unrecognized %+A" e)

and visitExprs f exprs = 
    List.iter (visitExpr f) exprs

and visitObjArg f objOpt = 
    Option.iter (visitExpr f) objOpt

and visitObjMember f memb = 
    visitExpr f memb.Body

Let's use this expresssion walker:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
fooExpression |> visitExpr (fun e -> printfn "Visiting %A" e)

// Prints:
//
// Visiting Let...
// Visiting Call...
// Visiting Const ("Hello", ...)
// Visiting Const (" ", ...)
// Visiting Const ("world", ...)
// Visiting IfThenElse...
// Visiting Call...
// Visiting Call...
// Visiting Value ...
// Visiting Const ...
// Visiting Const ...
// Visiting Const ...

Note that

  • The visitExpr function is recursive (for nested expressions).
  • Pattern matching is removed from the tree, into a form called 'decision trees'.

Summary

In this tutorial, we looked at basic of working with checked declarations and expressions.

In practice, it is also useful to combine the information here with some information you can obtain from the symbols tutorial.

namespace System
namespace System.IO
namespace Microsoft
namespace Microsoft.FSharp
namespace Microsoft.FSharp.Compiler
val checker : obj

Full name: Typedtree.checker
val parseAndCheckSingleFile : input:string -> 'a

Full name: Typedtree.parseAndCheckSingleFile
val input : string
val file : string
type Path =
  static val DirectorySeparatorChar : char
  static val AltDirectorySeparatorChar : char
  static val VolumeSeparatorChar : char
  static val InvalidPathChars : char[]
  static val PathSeparator : char
  static member ChangeExtension : path:string * extension:string -> string
  static member Combine : [<ParamArray>] paths:string[] -> string + 3 overloads
  static member GetDirectoryName : path:string -> string
  static member GetExtension : path:string -> string
  static member GetFileName : path:string -> string
  ...

Full name: System.IO.Path
Path.ChangeExtension(path: string, extension: string) : string
Path.GetTempFileName() : string
type File =
  static member AppendAllLines : path:string * contents:IEnumerable<string> -> unit + 1 overload
  static member AppendAllText : path:string * contents:string -> unit + 1 overload
  static member AppendText : path:string -> StreamWriter
  static member Copy : sourceFileName:string * destFileName:string -> unit + 1 overload
  static member Create : path:string -> FileStream + 3 overloads
  static member CreateText : path:string -> StreamWriter
  static member Decrypt : path:string -> unit
  static member Delete : path:string -> unit
  static member Encrypt : path:string -> unit
  static member Exists : path:string -> bool
  ...

Full name: System.IO.File
File.WriteAllText(path: string, contents: string) : unit
File.WriteAllText(path: string, contents: string, encoding: Text.Encoding) : unit
val projOptions : obj
Multiple items
type Async
static member AsBeginEnd : computation:('Arg -> Async<'T>) -> ('Arg * AsyncCallback * obj -> IAsyncResult) * (IAsyncResult -> 'T) * (IAsyncResult -> unit)
static member AwaitEvent : event:IEvent<'Del,'T> * ?cancelAction:(unit -> unit) -> Async<'T> (requires delegate and 'Del :> Delegate)
static member AwaitIAsyncResult : iar:IAsyncResult * ?millisecondsTimeout:int -> Async<bool>
static member AwaitTask : task:Task -> Async<unit>
static member AwaitTask : task:Task<'T> -> Async<'T>
static member AwaitWaitHandle : waitHandle:WaitHandle * ?millisecondsTimeout:int -> Async<bool>
static member CancelDefaultToken : unit -> unit
static member Catch : computation:Async<'T> -> Async<Choice<'T,exn>>
static member Choice : computations:seq<Async<'T option>> -> Async<'T option>
static member FromBeginEnd : beginAction:(AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>
static member FromBeginEnd : arg:'Arg1 * beginAction:('Arg1 * AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>
static member FromBeginEnd : arg1:'Arg1 * arg2:'Arg2 * beginAction:('Arg1 * 'Arg2 * AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>
static member FromBeginEnd : arg1:'Arg1 * arg2:'Arg2 * arg3:'Arg3 * beginAction:('Arg1 * 'Arg2 * 'Arg3 * AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>
static member FromContinuations : callback:(('T -> unit) * (exn -> unit) * (OperationCanceledException -> unit) -> unit) -> Async<'T>
static member Ignore : computation:Async<'T> -> Async<unit>
static member OnCancel : interruption:(unit -> unit) -> Async<IDisposable>
static member Parallel : computations:seq<Async<'T>> -> Async<'T []>
static member RunSynchronously : computation:Async<'T> * ?timeout:int * ?cancellationToken:CancellationToken -> 'T
static member Sleep : millisecondsDueTime:int -> Async<unit>
static member Start : computation:Async<unit> * ?cancellationToken:CancellationToken -> unit
static member StartAsTask : computation:Async<'T> * ?taskCreationOptions:TaskCreationOptions * ?cancellationToken:CancellationToken -> Task<'T>
static member StartChild : computation:Async<'T> * ?millisecondsTimeout:int -> Async<Async<'T>>
static member StartChildAsTask : computation:Async<'T> * ?taskCreationOptions:TaskCreationOptions -> Async<Task<'T>>
static member StartImmediate : computation:Async<unit> * ?cancellationToken:CancellationToken -> unit
static member StartWithContinuations : computation:Async<'T> * continuation:('T -> unit) * exceptionContinuation:(exn -> unit) * cancellationContinuation:(OperationCanceledException -> unit) * ?cancellationToken:CancellationToken -> unit
static member SwitchToContext : syncContext:SynchronizationContext -> Async<unit>
static member SwitchToNewThread : unit -> Async<unit>
static member SwitchToThreadPool : unit -> Async<unit>
static member TryCancelled : computation:Async<'T> * compensation:(OperationCanceledException -> unit) -> Async<'T>
static member CancellationToken : Async<CancellationToken>
static member DefaultCancellationToken : CancellationToken

Full name: Microsoft.FSharp.Control.Async

--------------------
type Async<'T>

Full name: Microsoft.FSharp.Control.Async<_>
static member Async.RunSynchronously : computation:Async<'T> * ?timeout:int * ?cancellationToken:Threading.CancellationToken -> 'T
val input2 : string

Full name: Typedtree.input2
val checkProjectResults : obj

Full name: Typedtree.checkProjectResults
val checkedFile : obj

Full name: Typedtree.checkedFile
val printDecl : prefix:'a -> d:'b -> 'c

Full name: Typedtree.printDecl
val prefix : 'a
val d : 'b
val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn
Multiple items
type CompiledNameAttribute =
  inherit Attribute
  new : compiledName:string -> CompiledNameAttribute
  member CompiledName : string

Full name: Microsoft.FSharp.Core.CompiledNameAttribute

--------------------
new : compiledName:string -> CompiledNameAttribute
val d : obj
val myLibraryEntity : obj

Full name: Typedtree.myLibraryEntity
val myLibraryDecls : obj

Full name: Typedtree.myLibraryDecls
val failwith : message:string -> 'T

Full name: Microsoft.FSharp.Core.Operators.failwith
val fooSymbol : obj

Full name: Typedtree.fooSymbol
val fooArgs : obj

Full name: Typedtree.fooArgs
val fooExpression : obj

Full name: Typedtree.fooExpression
type Type =
  inherit MemberInfo
  member Assembly : Assembly
  member AssemblyQualifiedName : string
  member Attributes : TypeAttributes
  member BaseType : Type
  member ContainsGenericParameters : bool
  member DeclaringMethod : MethodBase
  member DeclaringType : Type
  member Equals : o:obj -> bool + 1 overload
  member FindInterfaces : filter:TypeFilter * filterCriteria:obj -> Type[]
  member FindMembers : memberType:MemberTypes * bindingAttr:BindingFlags * filter:MemberFilter * filterCriteria:obj -> MemberInfo[]
  ...

Full name: System.Type
val visitExpr : f:('a -> unit) -> e:'a -> 'b

Full name: Typedtree.visitExpr
val f : ('a -> unit)
val e : 'a
val visitExprs : f:'a -> exprs:'b -> 'c

Full name: Typedtree.visitExprs
val visitObjArg : f:('a -> unit) -> objOpt:'a option -> unit

Full name: Typedtree.visitObjArg
Multiple items
module List

from Microsoft.FSharp.Collections

--------------------
val snd : tuple:('T1 * 'T2) -> 'T2

Full name: Microsoft.FSharp.Core.Operators.snd
val visitObjMember : f:('a -> unit) -> memb:'b -> 'c

Full name: Typedtree.visitObjMember
Multiple items
type DefaultValueAttribute =
  inherit Attribute
  new : unit -> DefaultValueAttribute
  new : check:bool -> DefaultValueAttribute
  member Check : bool

Full name: Microsoft.FSharp.Core.DefaultValueAttribute

--------------------
new : unit -> DefaultValueAttribute
new : check:bool -> DefaultValueAttribute
val sprintf : format:Printf.StringFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf
val f : 'a
val exprs : 'b
val objOpt : 'a option
module Option

from Microsoft.FSharp.Core
val iter : action:('T -> unit) -> option:'T option -> unit

Full name: Microsoft.FSharp.Core.Option.iter
val memb : 'b
val e : obj
Fork me on GitHub