ES6 iterators and generators

With ES6, JavaScript now has much better support for iteration. Before, all we had to work with was the usual for loop, either as a C-style loop or the property-based for...in. Then we got the functional programming tools for arrays: forEach, map, reduce, and so on. Now we have even more options that can save us from needing an error-prone C-style loop or the cumbersome for...in.

The new loop

ES6 adds a new subtype of for loop: for...of. At first glance, it looks almost exactly the same as for...in, but it has one very important difference: for...in works on property names, while for...of loops over property values. Prettified JavaScript variants (like CoffeeScript) have had this for years, but now it comes to the base language, and we get to do things like this:

var vowels = ['a','e','i','o','u','y'];

for (var v of vowels) {
    if (v != 'y') {
        console.log(v);
    } else {
        if (Math.random() < 0.5) {
            console.log("and sometimes " + v);
        }
    }
}

Most developers will, at first, use for...of to iterate through arrays, and it excels at that. Just giving the value in each iteration, instead of the index, will save the sanity of thousands of JavaScript programmers. And it’s a good substitute for Array.forEach() for those of you that don’t like the FP style of coding.

But for...of isn’t just for arrays. It works on other objects, too. For strings, it gives you each character (properly supporting Unicode, thanks to other ES6 updates), and the new Map and Set objects work in the way you’d expect (i.e., each entry, but in no particular order). Even better, you can write your own classes to support the new loop, because it can work with anything that uses the new iterable protocol.

Iterables

Protocols are technically a new addition to ES6. They lurked behind the scenes in ES5, but they were out of sight of programmers. Now, though, they’re front and center, and the iterable protocol is one such example.

If you’ve ever written code in Java, C#, C++, Python, or even TypeScript, then you already have a good idea of what a protocol entails. It’s an interface. An object conforms to a protocol if it (or some other object up its prototype chain) properly implements the protocol’s methods. That’s all there is to it.

The iterable protocol is almost too easy. For a custom iterable object, all you need to do is implement a method called @@iterator that returns an object that meets the iterator protocol.

Okay, I know you’re thinking, “How do I make a @@iterator method? I can’t use at-signs in names!” And you’re right. You can’t, and they don’t even want you to try. @@iterator is a special method name that basically means “a symbol with the name of iterator“.

So now we need to know what a symbol is. In ES6, it’s a new data type that we can use as a property identifier. There’s a lot of info about creating your own symbols out there, but we don’t actually need that for the iterable protocol. Instead, we can use a special symbol that comes built-in: Symbol.iterator. We can use it like this:

var myIterable = {
    [Symbol.iterator]: function() {
        // return an iterator object
    }
}

The square brackets mean we’re using a symbol as the name of the property, and Symbol.iterator internally converts to @@iterator, which is exactly what we need.

Iterators

That gets us halfway to a proper iterable, but now we need to create an object that conforms to the iterator protocol. That’s not that hard. The protocol only requires one method, next(), which must be callable without arguments. It returns another object that has two properties:

  • value: Whatever value the iterator wants to return. This can be a string, number, object, or anything you like. Internally, String returns each character, Array each successive value, and so on.

  • done: A boolean that states whether the iterator has reached the end of its sequence. If it’s true, then value becomes the return value of the whole iterator. Setting it to false is saying that you can keep getting more out of the iterator.

So, by implementing a single method, we can make any kind of sequence, like this:

var evens = function(limit) {
    return {
        [Symbol.iterator]: function() {
            var nextValue = 0;
            return {
                next: function() {
                    nextValue += 2;
                    return { done: nextValue > limit, value: nextValue };
                }
            }
        }
    }
}

for (var e of evens(20)) {
    console.log(e);
} // prints 2, 4, 6..., each on its own line

This is a toy example, but it shows the general layout of an iterable. It’s a great idea, and it’s very reminiscent of Python’s iteration support, but it’s not without its flaws. Mainly, just look at it. We have to go three objects deep to actually get to a return value. With ES6’s shorthand object literals, that’s a bit simplified, but it’s still unnecessary clutter.

Generators

Enter the generator. Another new addition to ES6, generators are special functions that give us most of the power of iterators with much cleaner syntax. To make a function that’s a generator, in fact, we only need to make two changes.

First, generators are defined as function*, not the usual function. The added star indicates a generator definition, and it can be used for function statements and expressions.

Second, generators don’t return like normal functions. Instead, they yield a value. The new yield keyword works just like its Python equivalent, immediately returning a value but “saving” their position. The next time the generator is called, it picks up right where it left off, immediately after the yield that ended it. You can have multiple yield statements, and they will be executed in order, one for each time you call the function:

function* threeYields() {
    yield "foo";
    yield "bar";
    yield "The End";
}

var gen = threeYields();

gen.next().value; // returns "foo"
gen.next().value; // returns "bar"
gen.next().value; // returns "The End"
gen.next().value; // undefined

You can also use a loop in your generators, giving us an easier way of writing our evens function above:

var evens = function(limit) {
    return {
        [Symbol.iterator]: function*() {
            var nextValue = 0;
            while (nextValue < limit) {
                nextValue += 2;
                yield nextValue;
            }
        }
    }
}

It’s still a little too deep, but it’s better than writing it all yourself.

Conclusion

Generators, iterators, and the for...of loop all have a common goal: to make it easier to work with sequences. With these new tools, we can now use a sequence as the sequence itself, getting its values as we need them instead of loading the whole thing into memory at once. This lazy loading is common in FP languages like Haskell, and it’s found its way into others like Python, but it’s new to JavaScript, and it will take some getting used to. But it allows a new way of programming. We can even have infinite sequences, which would have been impossible before now.

Iterators encapsulate state, meaning that generators can replace the old pattern of defining state variables and returning an IIFE that closes over them. (For a game-specific example, think of random number generators. These can now actually be generators.) Coroutines and async programming are two other areas where generators come into play, and a lot of people are already working on this kind of stuff. Looking ahead, there’s a very early ES7 proposal to add comprehensions, and these would be able to use generators, too.

Like most other ES6 features, these aren’t usable by everyone, at least not yet. Firefox and Chrome currently have most of the spec, while the others pretty much have nothing at all. For now, you’ll need to use something like Babel if you need to support all browsers, but it’s almost worth it.

Leave a Reply

Your email address will not be published. Required fields are marked *