Colin Bull · Functional .NET, F#, C#

Why I like F#.

Recently Dr. James McCaffery, posted Why he doesn't like the F# language. Quite a few of his points are subjective. People have different preferences and it seems like F# and more generally functional programming takes him outside of this comfort zone. This is fine, and I have absolutly no objections about views like this. I have a similar feeling when I'm in C# or Java. I don't feel safe, or comfortable, again it is just a preference thing.

However, there are a few points raised in the blog post that I don't really agree with. I'll tackle each one seperately not to loose any context.

  1. F# has a tiny user base.

I did a quick search on the day I wrote this post at a job aggregation site and found 109 job listings that mentioned F#. There were over 34,000 job listings that mentioned C#. And at MSDN Magazine, where I'm the Senior Contributing Editor, our F# articles get very few reads. The tiny user base means there is relatively weak community technical support on sites like Stack Overflow, compared to mainstream languages. Additionally, unlike other languages with relatively few users (such as R), there`s no real motivation for me to adopt F# from a career point of view, because of the very limited job opportunities.

While I somewhat agree, that F# adoption in industry has been slow. I think alot of this is to do with the fact that in the early days F# wasn't pushed as a general purpose programming language. This was obviously a marketing decision made in Microsoft, for reasons that are unknown to me. This decision caused an elitist view of F# in the early days with the preception that you need a advanced degree in a mathematical subject to use it, categorising it as only being good for Data Science, Finance, Mathematics and Research. Thus these areas were the early adoptors. In fact a quick browse of the testimonials page on FSharp.org backs this up. With testimonails coming from one of these areas. There are of course some exceptions most notably the design of Louvre in Abu Dhabi and it's use at GameSys.

However this metric is only one dimension and just because there are currently only a few jobs in a language doesn't mean you should not learn it. I'm currently learning langauges like Haskell, Coq and Idris. For the latter I doubt there is a single role in this country (although I'm willing to be proved wrong on this). Why do I do this? I hear you ask. Well I believe by learning different langauges and paradigms pushes me slightly out of my comfort zone and makes me a better programmer, in which-ever language I ultimately end up coding a commercial product in.

With the commercial prospects aside a conslusion is drawn that a small user base => a weak technical community. I don't know about other languages but I can categorically say that with F# this is simply not true. In fact, as I started writing this blog post, I raised an issue in the Paket project on github and within an hour, I had a fix presented too me.

For other sites like Stack Overflow, I can't really comment on the experience as I don't tend to use it much myself. However we can use F# to do some data munging to see how the community it doing. i.e. What is the average time for questions with an accepted answer to have got that answer?

To acheive this we can download the first 10000 questions with the F# tag, and write the result of each request out to a set of files.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
let baseUri = "https://api.stackexchange.com/2.2/"
let [<Literal>] dataPath = __SOURCE_DIRECTORY__ + "/data/stackoverflow/"

let dataDir =
    let path = new DirectoryInfo(dataPath)
    if not(path.Exists)
    then path.Create()
    path
    
let getQuestions(page) =
    let outputPath = new FileInfo(Path.Combine(dataDir.FullName, sprintf "questions_%d.questions" page))
    if(not <| outputPath.Exists)
    then
        let results =

            Http.RequestString(baseUri + sprintf "search?page=%d" page + "&pagesize=100&order=desc&sort=creation&tagged=f%23&site=stackoverflow")
        File.WriteAllText(outputPath.FullName, results)

let writeQuestions() =
    [1 .. 100] |> List.iter getQuestions

Next we can merge all of these questions using the Json type provider into a single list,

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
let [<Literal>] questionPath = dataPath + "questions.json"
type Questions = JsonProvider<questionPath>

let questions =
    [
       for file in  dataDir.EnumerateFiles("*.questions") do
           yield! Questions.Load(file.FullName).Items
    ]

Next up is getting the accepted answers. Firstly we build a map of the accepted answersId against the questions so we can relate them again later, then we use getAcceptedAnswers to chunk the requests and write the results out to a file. Once we have the results we again use the Json type provider to merge the results up into a single list.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
34: 
35: 
36: 
37: 
let questionAnswerMap = 
    questions
    |> Seq.fold (fun state question ->
        match question.AcceptedAnswerId with
        | Some answerId -> (answerId, question) :: state  
        | None -> state
    ) []  
    |> Map.ofSeq
    
let getAcceptedAnswers() =
    let answerIds =
        questionAnswerMap
        |> Map.toSeq
        |> Seq.map (fun (answerId,_) -> answerId.ToString())

    let counter = ref 1
    for answers in chunkBySize 100 answerIds do
        let outputPath = new FileInfo(Path.Combine(dataDir.FullName, sprintf "answers_%d.answers" !counter))
        if (not <| outputPath.Exists)
        then
            let answersStr = String.Join(";", answers)
            let answers =
                Http.RequestString(
                     baseUri + sprintf "answers/%s?order=desc&sort=creation&site=stackoverflow" answersStr
                )
            printfn "Writing answers %s" outputPath.FullName
            File.WriteAllText(outputPath.FullName, answers)
            incr(counter)

let [<Literal>] answersPath = dataPath + "answers.json"
type Answers = JsonProvider<answersPath>

let answers =
    [
       for file in  dataDir.EnumerateFiles("*.answers") do
           yield! Answers.Load(file.FullName).Items
    ]

Next up we pair the questions with the accepted answers.

1: 
2: 
3: 
4: 
5: 
6: 
7: 
let mergeQuestionAnswers =
    [
        for answer in answers do
            match questionAnswerMap.TryFind answer.AnswerId with
            | Some question -> yield question, answer
            | None -> ()            
    ]

And we are now at a point where we can compute some statistics around the questions.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
let getTimeToClose (question : Questions.Item, answer : Answers.Item) =
    (unixToDateTime answer.CreationDate).Subtract(unixToDateTime question.CreationDate).TotalHours

let statsByYear =
    mergeQuestionAnswers
    |> Seq.groupBy (fun (q,a) -> (unixToDateTime q.CreationDate).Year)
    |> Seq.map (fun (year, data) ->
         let timeToClose = 
             data |> Seq.map getTimeToClose
         let average = timeToClose |> Seq.average
         let median = timeToClose |> median
         year, average, median
    )
    |> Seq.sortBy (fun (y, _, _) -> y)
    |> Seq.toArray

This gives the following results.

[|(2008, 2345.916019, 1.900555556); (2009, 333.4175912, 0.4058333333);
  (2010, 96.91721817, 0.5219444444); (2011, 48.4008104, 0.5786111111);
  (2012, 165.2300729, 0.66); (2013, 36.97864167, 0.74);
  (2014, 42.91575397, 0.6572222222); (2015, 25.93529412, 0.6344444444)|]

Which we can then plot using

1: 
2: 
3: 
4: 
(Chart.Combine [
    Chart.Line(statsByYear |> Seq.map (fun (y, a, _) -> y, a), Name = "Average")
    Chart.Line(statsByYear |> Seq.map (fun (y, _, m) -> y, m), Name = "Median")
]).WithLegend(true)

And actually we see that in 2008 when FSharp first appeared it took a long time for questions to get closed. This is the year F# was introduced and I suspect there was only a handful of people outside of Microsoft Research that actually where able to answer these questions. However as time has progressed we see an exponential improvement it the time for questions to get answered, which typically bottoms out with an average of 25 hours and a median of about 30 mins. This is clearly a sign of a responsive community, that is indeed growing. Whats more, I still don't think that an average of 25 hours is actually representative. In my experience I rarely use Stack Overflow for F# questions, instead I direct my questions to the fsharp github repository previously on codeplex, the repository of the project I am using, or finally twitter with the #fsharp tag, and wait for the plethora of responses to come in from the help and very active community members. And in these domains the response time is typically around ~5 minutes.

In fact as I write this I'm wondering whether the comment

few irritatingly vocal people in the F# community implicitly try to guilt non-believers into F# adoption.

has been spurred by the willingness to help in the community. Yes there is a certain amount of advertisment that goes on for features specific to F#, but in general it is just sound fundamental programming advice. I'm fairly sure every single one of those people would offer examples in C#, Haskell or VB if asked. Anyway I digress.

The second comment that stood out for me in the post was,

  1. F# has no compelling technical advantage. Some overly zealous F# fans claim that F# can do things that general purpose languages like Perl and C# can't. This all depends on how you define features of languages. But more reasonable F# proponents usually state, correctly, that F# isn't intended to replace languages like C# and that F# doesn't have any unique, magic capabilities. So, there's no technical reason for me to use F#, and the cost of context switching between the primarily procedural C# and the primarily functional F# is a huge price to pay.

I think you only have to look at type providers, which are used to analyse the Stack Overflow questions above are certainly a nice and almost unique feature. That to my knowledge only one other language has Idris. Sure you can do the analysis I have done above in C# but there will be alot more typing and additionally a lot less safety, since you will ultimately loose the strongly typed data access, that type providers offer. Moreover it is this safety that F# and statically typed functional programming languages in general offers you and makes it worth the context switch.

Since I adopted F# and functional programming it has completely changed the way I think about coding problems in all of the other languages I use, most notable C#. It has made me a better developer.

namespace Microsoft.FSharp.Data
namespace System
namespace System.IO
Multiple items
val seq : sequence:seq<'T> -> seq<'T>

Full name: Microsoft.FSharp.Core.Operators.seq

--------------------
type seq<'T> = System.Collections.Generic.IEnumerable<'T>

Full name: Microsoft.FSharp.Collections.seq<_>
module Array

from Microsoft.FSharp.Collections
val zeroCreate : count:int -> 'T []

Full name: Microsoft.FSharp.Collections.Array.zeroCreate
Multiple items
val ref : value:'T -> 'T ref

Full name: Microsoft.FSharp.Core.Operators.ref

--------------------
type 'T ref = Ref<'T>

Full name: Microsoft.FSharp.Core.ref<_>
val sub : array:'T [] -> startIndex:int -> count:int -> 'T []

Full name: Microsoft.FSharp.Collections.Array.sub
Multiple items
val int : value:'T -> int (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.Operators.int

--------------------
type int = int32

Full name: Microsoft.FSharp.Core.int

--------------------
type int<'Measure> = int

Full name: Microsoft.FSharp.Core.int<_>
type DateTimeKind =
  | Unspecified = 0
  | Utc = 1
  | Local = 2

Full name: System.DateTimeKind
field System.DateTimeKind.Utc = 1
Multiple items
val float : value:'T -> float (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.Operators.float

--------------------
type float = System.Double

Full name: Microsoft.FSharp.Core.float

--------------------
type float<'Measure> = float

Full name: Microsoft.FSharp.Core.float<_>
module Seq

from Microsoft.FSharp.Collections
val toArray : source:seq<'T> -> 'T []

Full name: Microsoft.FSharp.Collections.Seq.toArray
val sort : array:'T [] -> 'T [] (requires comparison)

Full name: Microsoft.FSharp.Collections.Array.sort
val floor : value:'T -> 'T (requires member Floor)

Full name: Microsoft.FSharp.Core.Operators.floor
val ceil : value:'T -> 'T (requires member Ceiling)

Full name: Microsoft.FSharp.Core.Operators.ceil
Multiple items
type LiteralAttribute =
  inherit Attribute
  new : unit -> LiteralAttribute

Full name: Microsoft.FSharp.Core.LiteralAttribute

--------------------
new : unit -> LiteralAttribute
val not : value:bool -> bool

Full name: Microsoft.FSharp.Core.Operators.not
val sprintf : format:Printf.StringFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf
Multiple items
module List

from Microsoft.FSharp.Collections

--------------------
type List<'T> =
  | ( [] )
  | ( :: ) of Head: 'T * Tail: 'T list
  interface IEnumerable
  interface IEnumerable<'T>
  member Head : 'T
  member IsEmpty : bool
  member Item : index:int -> 'T with get
  member Length : int
  member Tail : 'T list
  static member Cons : head:'T * tail:'T list -> 'T list
  static member Empty : 'T list

Full name: Microsoft.FSharp.Collections.List<_>
val iter : action:('T -> unit) -> list:'T list -> unit

Full name: Microsoft.FSharp.Collections.List.iter
val fold : folder:('State -> 'T -> 'State) -> state:'State -> source:seq<'T> -> 'State

Full name: Microsoft.FSharp.Collections.Seq.fold
union case Option.Some: Value: 'T -> Option<'T>
union case Option.None: Option<'T>
Multiple items
module Map

from Microsoft.FSharp.Collections

--------------------
type Map<'Key,'Value (requires comparison)> =
  interface IEnumerable
  interface IComparable
  interface IEnumerable<KeyValuePair<'Key,'Value>>
  interface ICollection<KeyValuePair<'Key,'Value>>
  interface IDictionary<'Key,'Value>
  new : elements:seq<'Key * 'Value> -> Map<'Key,'Value>
  member Add : key:'Key * value:'Value -> Map<'Key,'Value>
  member ContainsKey : key:'Key -> bool
  override Equals : obj -> bool
  member Remove : key:'Key -> Map<'Key,'Value>
  ...

Full name: Microsoft.FSharp.Collections.Map<_,_>

--------------------
new : elements:seq<'Key * 'Value> -> Map<'Key,'Value>
val ofSeq : elements:seq<'Key * 'T> -> Map<'Key,'T> (requires comparison)

Full name: Microsoft.FSharp.Collections.Map.ofSeq
val toSeq : table:Map<'Key,'T> -> seq<'Key * 'T> (requires comparison)

Full name: Microsoft.FSharp.Collections.Map.toSeq
val map : mapping:('T -> 'U) -> source:seq<'T> -> seq<'U>

Full name: Microsoft.FSharp.Collections.Seq.map
module String

from Microsoft.FSharp.Core
val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn
val incr : cell:int ref -> unit

Full name: Microsoft.FSharp.Core.Operators.incr
val groupBy : projection:('T -> 'Key) -> source:seq<'T> -> seq<'Key * seq<'T>> (requires equality)

Full name: Microsoft.FSharp.Collections.Seq.groupBy
val average : source:seq<'T> -> 'T (requires member ( + ) and member DivideByInt and member get_Zero)

Full name: Microsoft.FSharp.Collections.Seq.average
val sortBy : projection:('T -> 'Key) -> source:seq<'T> -> seq<'T> (requires comparison)

Full name: Microsoft.FSharp.Collections.Seq.sortBy

Introducing Cricket (formerly FSharp.Actor)

Over the last few years, I have been quite taken by the actor model of computing. Although not a silver bullet it does tend to make concurrent programming orders of magnitude easier to reason about. If you have never heard of Actors then an actor as defined by wikipedia is as follows

The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.

An actor is a computational entity that, in response to a message it receives, can concurrently:

  • send a finite number of messages to other actors;
  • create a finite number of new actors;
  • designate the behavior to be used for the next message it receives.

I also encourage you to look at Erlang/Elixir, Akka, Orleans and the MailboxProcessor<'a> in FSharp.Core.

Introducing Cricket

Cricket, formerly FSharp.Actor, is yet another actor framework. Built entirely in F#, Cricket is a lightweight alternative to Akka et. al. To this end it is not as feature rich as these out of the box, but all of the core requirements like location transpancy, remoting, supervisors, metrics and tracing. Other things like failure detection and clustering are in the pipeline it is just a question of time.

Some key links for Cricket:

The nuget package, contains a single library Cricket.dll and a reference to FsPickler, which is used for serailization.

Creating a simple actor

The following example, creates a echo actor using cricket.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
#I "../packages"
#r "FsPickler/lib/net45/FsPickler.dll"
#r "Cricket/lib/Cricket.dll"

open Cricket


ActorHost.Start()

let echo =
    actor {
        name "echo"
        body (
            let rec loop() = messageHandler {
                let! msg = Message.receive()
                printfn "%s" msg
                return! loop()
            }
            loop())
    } |> Actor.spawn

A couple of things are happening in the code above. Firstly, we start an ActorHost which sets up an environment within the current process for the actor to live in. Next we define the actor, we give it a name echo and a body. The body is actually the only thing that is required. If the name is omitted then it is assinged as a Guid. All the body of an actor consists of is a recursive function, that describes how to handle the messages posted to the actor. In this case we simply print a message to the console. Once we have defined the actor we then spawn it using Actor.spawn. After an actor has been spawned it is ready to consume messages. We can send messages directly to the actor by using the ActorRef that is returned by Actor.spawn.

1: 
echo <-- "Hello, from Cricket"

Alternatively we can resolve the actor by name and send the message that way.

1: 
"echo" <-- "Hello, from Cricket"

From these basic beginings we can build entire systems using actors. These systems can be spread over multiple machines and as long as the underlying message transport supports it different data-centres. To make our echo actor distributed, we don't have to change the implementation of the actor. All we have to do is enable remoting on the actor host.

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
//Node1 host configuration
ActorHost.Start()
         .SubscribeEvents(fun (evnt:ActorEvent) -> printfn "%A" evnt)
         .EnableRemoting(
                   [new TCPTransport(TcpConfig.Default(IPEndPoint.Create(12002)))],
                   new BinarySerializer(),
                   new TcpActorRegistryTransport(TcpConfig.Default(IPEndPoint.Create(12003))),
                   new UdpActorRegistryDiscovery(UdpConfig.Default(), 1000))

All we have done is enchance the ActorHost with a collection of message transports, a serializer, a registry transport and a way for the actors to discover each other. Similar setif we used the same setup on another node.

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
//Node2 host configuration
ActorHost.Start()
         .SubscribeEvents(fun (evnt:ActorEvent) -> printfn "%A" evnt)
         .EnableRemoting(
                   [new TCPTransport(TcpConfig.Default(IPEndPoint.Create(12004)))],
                   new BinarySerializer(),
                   new TcpActorRegistryTransport(TcpConfig.Default(IPEndPoint.Create(12005))),
                   new UdpActorRegistryDiscovery(UdpConfig.Default(), 1000))

then we can on node 2 resolve any actors on node 1, using the example above. Alternatively if I had 10 nodes but wanted to resolve the echo actor on node 9, I could do something like the following

1: 
"node9@*/echo" <-- "Hello, from Cricket"

This would then resolve the actor on node9. If we had kept the original query which was simply echo then this would resolve any actor named echo all of the nodes participating in the group. For more details on remoting and a link to an example see here

val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn

Blogging with F# on GitHub Pages

Introduction

Recently I decided I wanted to move my blog from Wordpress, to a far lighter-weight platform. The platform I chose to host my new blog on was Github Pages. Basically if you have a github account you get a free pages repository where you can host a blog. You simply create a repository with a name that matches the format {username}.github.io and that's basically it. You can now create a index.htm page place it into your repository and away you go.

Unfortunately having a completely static site isn't massively useful for a blog. At the very least, you are going to want a templating engine of some sort. Fortunately Github pages, comes armed with jekyll which is a blog-aware static site generator. Jekyll relies quite heavily on having the correct folder structure, this had me chasing my tail for a moment then I found the superb poole which generates all of the layout a creates a nice looking minimalist blog. Happy Days!

To add a post you simply create a *.md or *.html and save it to the posts directory, push your changes.

Leveraging FSharp.Formatting

FSharp Formatting is a library that enables a form of literate programming, where you can embed markdown directly into a *.fsx script. Then by running a simple command line tool or a script you can convert the script into a HTML or Latex document. When your chosen out put is html you get tool-tips when you hover over terms in your code, exactly like you would in an IDE. For example,

1: 
2: 
3: 
let add a b = a + b

printfn "%d" (add 5 15)

Since Jekyll does not directly support *.fsx files. We need to extend the structure given to us by poole. The first step I took was to include Paket so I can get hold of nuget packages, that I might require for my scripts. This may seem like an odd requirement at first, but because all of my blog posts will be F# script files which are type checked by FSharp.Formatting I effectively have executable / type safe code samples on my blog :). Once paket was installed I ran

./.paket/paket add nuget FSharp.Formatting.CommandTool

This installed the F# command tool, which is a command line wrapper for the FSharp.Formatting library. Next I created a publish.bat so I have a single command to update changes to my blog

@echo off

call .paket\paket restore
call tools\fsformatting.exe literate --processDirectory --lineNumbers true --inputDirectory  "code" --outputDirectory "_posts"

git add --all .
git commit -a -m %1
git push

The script above takes a single parameter which is a commit message, this can be run like so.

./publish.bat "Added post about F# blogging"

At this point all that is left to-do is write some content in the code folder and then run the publish.bat once you have created your master piece. Well that was nearly the case for me. It turns out that jekyll requires a header at the top of each page which looks something like the following

---
layout: page
title: your post title
---

This presented a little bit of a problem as FSharp.Formatting did not have a way of just emitting raw content. Fortunately for me it does have a concept of commands in the documentation. Basically commands allow you to embed the results of computations or hide certain bits of code you may not want in you documentation (more info on this can be found here). All I have done is extend this mechanism slightly by adding an extra command raw. Which allows you to prefix a block of markup that you do not want the formatter to touch. So at the top of each post I now have something like the following,

(*** raw ***)
---
layout: page
title: your post title
----

As of writing this change is not part of the FSharp.Formatting mainline, but there is a PR. However if you deceide that you like this approach, or just want to play, I have created a github repository FsBlog that is this exact blog (minus content).

One more thing is if you are developing in Visual Studio then I highly recommend the Elucidate Extension, so you can visualize your literate scripts as you work on them.

val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn

Handling constant type unsupported error for type providers

When writing type providers you are required to define code that will run at run-time with a quotation. This in itself is not a problem, however if you try and pass a none native type to the quotation you will receive the following error,

1: 
Unsupported constant type: xxxx

There is a stack-overflow post here which has an example and a good explanation of the reasons why. A typical work around is to use each field from a record and pass it to a function call in the quotation as an array or as individual parameters. Either way this can end up being quite painful.

So how can we work around this. Well, what we need to do is build a new instance of the object we are trying to pass to the quotation within the quotation itself, and then use the variable that holds this new instance as the parameter in the function call in the Quotation. I have probably not explained that the best but the final code looks like this.

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
9: 
let recordInstance = { Name = "Tester"; DateTime = DateTime.UtcNow }

let providedMethod  = 
     ProvidedMethod("MethodName",
               [(* some parameters *)],typeof<SomeType>, 
               InvokeCode = 
                    QuotationHelpers.quoteRecord 
                         recordInstance 
                         (fun args var ->  <@@ ((%%args.[0] : SomeType).SomeMethod(%%var)) @@>))

Where the args are the original arguments passed by provided method invoke code and var is a quotation that represents our record instance to pass to our method. The implementation of QuotationHelpers is as follows.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
34: 
35: 
36: 
37: 
38: 
39: 
40: 
41: 
42: 
43: 
module QuotationHelpers = 

    open Microsoft.FSharp.Quotations
    open Microsoft.FSharp.Reflection

    let rec coerceValues fieldTypeLookup fields = 
        Array.mapi (fun i v ->
                let expr = 
                    if v = null then simpleTypeExpr v
                    elif FSharpType.IsUnion (v.GetType()) then unionExpr v |> snd
                    elif FSharpType.IsRecord (v.GetType()) then recordExpr v |> snd
                    else simpleTypeExpr v
                Expr.Coerce(expr, fieldTypeLookup i)
        ) fields |> List.ofArray
    
    and simpleTypeExpr instance = Expr.Value(instance)

    and unionExpr instance = 
        let caseInfo, fields = FSharpValue.GetUnionFields(instance, instance.GetType())    
        let fieldInfo = caseInfo.GetFields()
        let fieldTypeLookup indx = fieldInfo.[indx].PropertyType
        caseInfo.DeclaringType, Expr.NewUnionCase(caseInfo, coerceValues fieldTypeLookup fields)

    and recordExpr instance = 
        let tpy = instance.GetType()
        let fields = FSharpValue.GetRecordFields(instance)
        let fieldInfo = FSharpType.GetRecordFields(tpy)
        let fieldTypeLookup indx = fieldInfo.[indx].PropertyType
        tpy, Expr.NewRecord(instance.GetType(), coerceValues fieldTypeLookup fields)

    and arrayExpr (instance : 'a array) =
        let typ = typeof<'a>
        let arrayType = instance.GetType()
        let exprs = coerceValues (fun _ -> typ) (instance |> Array.map box)
        arrayType, Expr.NewArray(typ, exprs)

    let createLetExpr varType instance body args = 
        let var = Var("instance", varType)  
        Expr.Let(var, instance, body args (Expr.Var(var)))

    let quoteUnion instance = unionExpr instance ||> createLetExpr
    let quoteRecord instance = recordExpr instance ||> createLetExpr
    let quoteArray instance = arrayExpr instance ||> createLetExpr

And thats it. Hopefully this should remove some pain points in developing type providers.

namespace System
namespace Microsoft
namespace Microsoft.FSharp
namespace Microsoft.FSharp.Quotations
Multiple items
val string : value:'T -> string

Full name: Microsoft.FSharp.Core.Operators.string

--------------------
type string = System.String

Full name: Microsoft.FSharp.Core.string
namespace Microsoft.FSharp.Reflection
module Array

from Microsoft.FSharp.Collections
val mapi : mapping:(int -> 'T -> 'U) -> array:'T [] -> 'U []

Full name: Microsoft.FSharp.Collections.Array.mapi
val snd : tuple:('T1 * 'T2) -> 'T2

Full name: Microsoft.FSharp.Core.Operators.snd
Multiple items
module List

from Microsoft.FSharp.Collections

--------------------
type List<'T> =
  | ( [] )
  | ( :: ) of Head: 'T * Tail: 'T list
  interface IEnumerable
  interface IEnumerable<'T>
  member Head : 'T
  member IsEmpty : bool
  member Item : index:int -> 'T with get
  member Length : int
  member Tail : 'T list
  static member Cons : head:'T * tail:'T list -> 'T list
  static member Empty : 'T list

Full name: Microsoft.FSharp.Collections.List<_>
val ofArray : array:'T [] -> 'T list

Full name: Microsoft.FSharp.Collections.List.ofArray
type 'T array = 'T []

Full name: Microsoft.FSharp.Core.array<_>
val typeof<'T> : System.Type

Full name: Microsoft.FSharp.Core.Operators.typeof
val map : mapping:('T -> 'U) -> array:'T [] -> 'U []

Full name: Microsoft.FSharp.Collections.Array.map
val box : value:'T -> obj

Full name: Microsoft.FSharp.Core.Operators.box

Option Operators

We often have to represent the absence of data within a collection of values. In C# the default type to-go to is an Nullable, F# 3.0 introduced the Microsoft.FSharp.Linq.NullableOperators module to help with comparisons and arithmetic over this type.

The Nullable type can still be somewhat tiresome to use in F# though as it can lead to adding lots of type constraints for example

1: 
2: 
3: 
type NullableSeq<'a when 'a : (new : unit ->'a) 
                    and 'a : struct 
                    and 'a :> ValueType> = seq<Nullable<'a>>

I think a nicer approach is to replace Nullable with Option, to relax some of these type constraints.

1: 
type OptionSeq<'a> = seq<Option<'a>>

however in doing this we have lost the nice operators that are available for the type Nullable<'a>. But this is fairly easy to recreate as the semantics of Nullable and Option are approximately the same. So with a quick copy of the NullableOperators module and some simple find and replace we have the exact same set of operators.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
34: 
35: 
36: 
37: 
38: 
39: 
40: 
41: 
42: 
43: 
44: 
45: 
46: 
47: 
48: 
49: 
50: 
51: 
52: 
53: 
54: 
55: 
56: 
57: 
module OptionOperators =

    let (?>=) (x : Option<'T>) (y: 'T) = x.IsSome && x.Value >= y

    let (?>) (x : Option<'T>) (y: 'T) = x.IsSome && x.Value > y

    let (?<=) (x : Option<'T>) (y: 'T) = x.IsSome && x.Value <= y

    let (?<) (x : Option<'T>) (y: 'T) = x.IsSome && x.Value < y

    let (?=) (x : Option<'T>) (y: 'T) = x.IsSome && x.Value = y

    let (?<>) (x : Option<'T>) (y: 'T) = not (x ?= y)

    let (>=?) (x : 'T) (y: Option<'T>) = y.IsSome && x >= y.Value

    let (>?) (x : 'T) (y: Option<'T>) = y.IsSome && x > y.Value

    let (<=?) (x : 'T) (y: Option<'T>) = y.IsSome && x <= y.Value

    let (<!--?) (x : 'T) (y: Option<'T>) = y.IsSome && x < y.Value

    let (=?) (x : 'T) (y: Option<'T>) = y.IsSome && x = y.Value

    let (<>?) (x : 'T) (y: Option<'T>) = not (x =? y)

    let (?>=?) (x : Option<'T>) (y: Option<'T>) = (x.IsSome && y.IsSome && x.Value >= y.Value)

    let (?>?) (x : Option<'T>) (y: Option<'T>) = (x.IsSome && y.IsSome && x.Value > y.Value)

    let (?<=?) (x : Option<'T>) (y: Option<'T>) = (x.IsSome && y.IsSome && x.Value <= y.Value)

    let (?<!--?) (x : Option<'T>) (y: Option<'T>) = (x.IsSome && y.IsSome && x.Value < y.Value)

    let (?=?) (x : Option<'T>) (y: Option<'T>) = (not x.IsSome && not y.IsSome) || (x.IsSome && y.IsSome && x.Value = y.Value)

    let (?<>?) (x : Option<'T>) (y: Option<'T>) = not (x ?=? y)

    let inline (?+) (x : Option<_>) y = if x.IsSome then Some(x.Value + y) else None
    let inline (+?) x (y: Option<_>) = if y.IsSome then Some(x + y.Value) else None
    let inline (?+?) (x : Option<_>) (y: Option<_>) = if x.IsSome && y.IsSome then Some(x.Value + y.Value) else None

    let inline (?-) (x : Option<_>) y = if x.IsSome then Some(x.Value - y) else None
    let inline (-?) x (y: Option<_>) = if y.IsSome then Some(x - y.Value) else None
    let inline (?-?) (x : Option<_>) (y: Option<_>) = if x.IsSome && y.IsSome then Some(x.Value - y.Value) else None

    let inline ( ?*  ) (x : Option<_>) y = if x.IsSome then Some(x.Value * y) else None
    let inline ( *?  ) x (y: Option<_>) = if y.IsSome then Some(x * y.Value) else None
    let inline ( ?*? ) (x : Option<_>) (y: Option<_>) = if x.IsSome && y.IsSome then Some(x.Value * y.Value) else None

    let inline ( ?%  ) (x : Option<_>) y = if x.IsSome then Some(x.Value % y) else None
    let inline ( %?  ) x (y: Option<_>) = if y.IsSome then Some(x % y.Value) else None
    let inline ( ?%? ) (x : Option<_>) (y: Option<_>) = if x.IsSome && y.IsSome then Some(x.Value % y.Value) else None

    let inline ( ?/  ) (x : Option<_>) y = if x.IsSome then Some(x.Value / y) else None
    let inline ( /?  ) x (y: Option<_>) = if y.IsSome then Some(x / y.Value) else None
    let inline ( ?/? ) (x : Option<_>) (y: Option<_>) = if x.IsSome && y.IsSome then Some(x.Value / y.Value) else None
namespace System
type unit = Unit

Full name: Microsoft.FSharp.Core.unit
Multiple items
val seq : sequence:seq<'T> -> seq<'T>

Full name: Microsoft.FSharp.Core.Operators.seq

--------------------
type seq<'T> = System.Collections.Generic.IEnumerable<'T>

Full name: Microsoft.FSharp.Collections.seq<_>
module Option

from Microsoft.FSharp.Core
val not : value:bool -> bool

Full name: Microsoft.FSharp.Core.Operators.not
union case Option.Some: Value: 'T -> Option<'T>
union case Option.None: Option<'T>