Don’t Get Bit By Lazy Evaluation

I’ve written numerous posts in the past about using “yield return” and delayed execution, but yes I was still stumped for a few minutes today when I was implementing a sample application. I had included the normal “Each” extension method that I have written in the past, since it is so useful when you want to easily iterate over a collection easily:

public static void Each<T>(this IEnumerable<T> list, Action<T> action)
{
    foreach (T item in list)
    {
        action(item);
    }
}

I was using this and I got to the point where I wanted to chain something in the same way that jQuery lets you do:

$('#something')
    .each(function(item) { //do something })
    .each(function(item) { //do something else });

So I modified my “Each” extension method to simply pass the last operated item out, and tried to get cute by using “yield return”:

public static IEnumerable<T> Each<T>(this IEnumerable<T> list, Action<T> action)
{
    foreach (T item in list)
    {
        action(item);
        yield return item;
    }
}

And then I walked away cause I had to do something else. I came back a while later and ran my application and nothing happened. Unfortunately I couldn’t remember what I had changed. So I started going through my code, and eventually I came across the code I had modified and it jumped right out at me! I was calling the above method like this:

myCollection.Each(item => item.Clear());

If you haven’t had to deal very often with delayed execution you may still be staring at that wondering what the problem is. And if you picked up on it right away then I applaud you. The problem is that nothing is “pulling” the values out of the “Each” method. Nothing is iterating over the iterator that is coming out of this method. That is the beauty of “yield return”, if the results are never used, then the code is never executed. Here though, the results weren’t important, but the side effects of the method that is the problem.

Side effects… ugh. Yet another reason why side effects and a functional style of programming will slap you in the face. If you still wondering what the deal is exactly, then take a look at the code generated by the above source:

public static IEnumerable<T> Each<T>(this IEnumerable<T> list, Action<T> action)
{
    <Each>d__0<T> d__ = new <Each>d__0<T>(-2);
    d__.<>3__list = list;
    d__.<>3__action = action;
    return d__;
}

Notice that there is a compiler generated class called <Each>d__0 that is being declared has the list and action assigned to it, then this class is returned. It is just a container for the action to be performed. If you then go and look at what the compiler generated for this class (I’m not going to paste the whole thing here), but it is implementing IEnumerable<T> and IEnumerator<T>. So this class is just being passed back from the method and unless this class is iterated over, then the Action delegate is never even being called.

So how do we fix this? Well, there are two ways we could fix this. One is to simply change the Each method so that it iterates the whole list each time:

public static IEnumerable<T> Each<T>(this IEnumerable<T> list, Action<T> action)
{            
    var result = new List<T>();
    foreach (T item in list)
    {
        action(item);
        result.Add(item);
    }
    return result;
}

When this is called we will iterate over the entire list, and perform an action on each item before the method returns. When the method returns, we can just return the original list since we aren’t changing it in any way.

The second way to fix it, which may or may not be preferable depending on your situation, is to force execution on the iterator. This can be done in a slightly odd looking way by calling ToArray(), ToList(), etc… Linq extension methods. Since these methods turn the iterator into a set list, they obviously have to iterate through the whole list:

myCollection.Each(item => item.Clear()).ToArray();

Since we aren’t using the result of this, it looks a bit odd. We can perform a similar action using a custom extension method called “Apply”. This was originally suggested to me a long time ago by Mono.Rocks author Jonathan Pryor. The Apply extension method simply iterates the list and forces execution:

public static void Apply<T>(this IEnumerable<T> list)
{
    foreach (T item in list)
    {
        //ignore T
    }
}

It still returns an IEnumerable so that it can be chained, in case you want intermediate steps executed in a chain of methods. As I was reminded by Sebastian below, it is very bad form to iterate an IEnumerable more than once. We could again cache the result in a list, so I’ll leave it up to you to implement that version.

So, why would you want to solve it in this way? Well, with the “Each” extension method still being delay executed, it means that we can chain a bunch of them together and a single item will get passed down through the chain. So something like this would work:

GetStreamOfNeverendingObjects()
    .Each(o => o.PerformAction())
    .Each(o => o.PerformOtherAction())
    .Take(5)
    .Apply();

Hmmm. So we get a list of some random objects take the first one, call two methods on it. Take the second, call two methods on it. And you’ll notice that we only do this for the first five items in the list, since we are using the “Take” method. Interesting stuff.

Wrap Up

This is really interesting stuff, and can be very confusing. The powers of delayed execution can amaze and confound. Remember, these tools are sharp, so make sure that you’re careful.