Regular functions return only one, single value (or nothing).

Generators can return (“yield”) multiple values, possibly an infinite number of values, one after another, on-demand. They work great with iterables, allowing to create data streams with ease.

Generator functions

To create a generator, we need a special syntax construct: function*, so-called “generator function”.

It looks like this:

function* generateSequence() {
  yield 1;
  yield 2;
  return 3;
}

The term “generator function” is a bit misleading, because when called it does not execute the code. Instead, it returns a special object, called “generator object”.

So it’s kind of a “generator constructor”.

// "generator function" creates "generator object"
let generator = generateSequence();

The generator object is something like an “frozen function call”:

Upon creation, the code execution is paused at the very beginning.

The main method of a generator is next(). When called, it resumes execution till the nearest yield <value> statement. Then the execution pauses, and the value is returned to the outer code.

For instance, here we create the generator and get its first yielded value:

function* generateSequence() {
  yield 1;
  yield 2;
  return 3;
}

let generator = generateSequence();

let one = generator.next();

alert(JSON.stringify(one)); // {value: 1, done: false}

The result of next() is always an object:

  • value: the yielded value.
  • done: false if the code is not finished yet, otherwise true.

As of now, we got the first value only:

Let’s call generator.next() again. It resumes the code execution and returns the next yield:

let two = generator.next();

alert(JSON.stringify(two)); // {value: 2, done: false}

And, if we call it the third time, then the execution reaches return statement that finishes the function:

let three = generator.next();

alert(JSON.stringify(three)); // {value: 3, done: true}

Now the generator is done. We should see it from done:true and process value:3 as the final result.

New calls generator.next() don’t make sense any more. If we make them, they return the same object: {done: true}.

There’s no way to “roll back” a generator. But we can create another one by calling generateSequence().

So far, the most important thing to understand is that generator functions, unlike regular function, do not run the code. They serve as “generator factories”. Running function* returns a generator, and then we ask it for values.

function* f(…) or function *f(…)?

That’s a minor religious question, both syntaxes are correct.

But usually the first syntax is preferred, as the star * denotes that it’s a generator function, it describes the kind, not the name, so it should stick with the function keyword.

Generators are iterable

As you probably already guessed looking at the next() method, generators are iterable.

We can get loop over values by for..of:

function* generateSequence() {
  yield 1;
  yield 2;
  return 3;
}

let generator = generateSequence();

for(let value of generator) {
  alert(value); // 1, then 2
}

That’s a much better-looking way to work with generators than calling .next().value, right?

…But please note: the example above shows 1, then 2, and that’s all. It doesn’t show 3!

It’s because for-of iteration ignores the last value, when done: true. So, if we want all results to be shown by for..of, we must return them with yield:

function* generateSequence() {
  yield 1;
  yield 2;
  yield 3;
}

let generator = generateSequence();

for(let value of generator) {
  alert(value); // 1, then 2, then 3
}

Naturally, as generators are iterable, we can call all related functionality, e.g. the spread operator ...:

function* generateSequence() {
  yield 1;
  yield 2;
  yield 3;
}

let sequence = [0, ...generateSequence()];

alert(sequence); // 0, 1, 2, 3

In the code above, ...generateSequence() turns the iterable into array of items (read more about the spread operator in the chapter Rest parameters and spread operator)

Using generators for iterables

Some time ago, in the chapter Iterables we created an iterable range object that returns values from..to.

Here, let’s remember the code:

let range = {
  from: 1,
  to: 5,

  // for..of range calls this method once in the very beginning
  [Symbol.iterator]() {
    // ...it returns the iterator object:
    // onward, for..of works only with that object, asking it for next values
    return {
      current: this.from,
      last: this.to,

      // next() is called on each iteration by the for..of loop
      next() {
        // it should return the value as an object {done:.., value :...}
        if (this.current <= this.last) {
          return { done: false, value: this.current++ };
        } else {
          return { done: true };
        }
      }
    };
  }
};

alert([...range]); // 1,2,3,4,5

Using a generator to make iterable sequences is simpler and much more elegant:

function* generateSequence(start, end) {
  for (let i = start; i <= end; i++) {
    yield i;
  }
}

let sequence = [...generateSequence(1,5)];

alert(sequence); // 1, 2, 3, 4, 5

Converting Symbol.iterator to generator

We can add generator-style iteration to any custom object by providing a generator as Symbol.iterator.

Here’s the same range, but with a much more compact iterator:

let range = {
  from: 1,
  to: 5,

  *[Symbol.iterator]() { // a shorthand for [Symbol.iterator]: function*()
    for(let value = this.from; value <= this.to; value++) {
      yield value;
    }
  }
};

alert( [...range] ); // 1,2,3,4,5

That works, because range[Symbol.iterator]() now returns a generator, and generator methods are exactly what for..of expects:

  • it has .next() method
  • that returns values in the form {value: ..., done: true/false}

That’s not a coincidence, of course. Generators were added to JavaScript language with iterators in mind, to implement them easier.

The last variant with a generator is much more concise than the original iterable code of range, and keeps the same functionality.

Generators may generate values forever

In the examples above we generated finite sequences, but we can also make a generator that yields values forever. For instance, an unending sequence of pseudo-random numbers.

That surely would require a break (or return) in for..of over such generator, otherwise the loop would repeat forever and hang.

Generator composition

Generator composition is a special feature of generators that allows to transparently “embed” generators in each other.

For instance, we’d like to generate a sequence of:

  • digits 0..9 (character codes 48…57),
  • followed by alphabet letters a..z (character codes 65…90)
  • followed by uppercased letters A..Z (character codes 97…122)

We can use the sequence e.g. to create passwords by selecting characters from it (could add syntax characters as well), but let’s generate it first.

We already have function* generateSequence(start, end). Let’s reuse it to deliver 3 sequences one after another, together they are exactly what we need.

In a regular function, to combine results from multiple other functions, we call them, store the results, and then join at the end.

For generators, we can do better, like this:

function* generateSequence(start, end) {
  for (let i = start; i <= end; i++) yield i;
}

function* generatePasswordCodes() {

  // 0..9
  yield* generateSequence(48, 57);

  // A..Z
  yield* generateSequence(65, 90);

  // a..z
  yield* generateSequence(97, 122);

}

let str = '';

for(let code of generatePasswordCodes()) {
  str += String.fromCharCode(code);
}

alert(str); // 0..9A..Za..z

The special yield* directive in the example is responsible for the composition. It delegates the execution to another generator. Or, to say it simple, yield* gen iterates over the generator gen and transparently forwards its yields outside. As if the values were yielded by the outer generator.

The result is the same as if we inlined the code from nested generators:

function* generateSequence(start, end) {
  for (let i = start; i <= end; i++) yield i;
}

function* generateAlphaNum() {

  // yield* generateSequence(48, 57);
  for (let i = 48; i <= 57; i++) yield i;

  // yield* generateSequence(65, 90);
  for (let i = 65; i <= 90; i++) yield i;

  // yield* generateSequence(97, 122);
  for (let i = 97; i <= 122; i++) yield i;

}

let str = '';

for(let code of generateAlphaNum()) {
  str += String.fromCharCode(code);
}

alert(str); // 0..9A..Za..z

A generator composition is a natural way to insert a flow of one generator into another.

It works even if the flow of values from the nested generator is infinite. It’s simple and doesn’t use extra memory to store intermediate results.

“yield” is a two-way road

Till this moment, generators were like “iterators on steroids”. And that’s how they are often used.

But in fact they are much more powerful and flexible.

That’s because yield is a two-way road: it not only returns the result outside, but also can pass the value inside the generator.

To do so, we should call generator.next(arg), with an argument. That argument becomes the result of yield.

Let’s see an example:

function* gen() {
  // Pass a question to the outer code and wait for an answer
  let result = yield "2 + 2?"; // (*)

  alert(result);
}

let generator = gen();

let question = generator.next().value; // <-- yield returns the value

generator.next(4); // --> pass the result into the generator
  1. The first call generator.next() is always without an argument. It starts the execution and returns the result of the first yield (“2+2?”). At this point the generator pauses the execution (still on that line).
  2. Then, as shown at the picture above, the result of yield gets into the question variable in the calling code.
  3. On generator.next(4), the generator resumes, and 4 gets in as the result: let result = 4.

Please note, the outer code does not have to immediately callnext(4). It may take time to calculate the value. That’s not a problem: the generator will resume when the call is made.

This is also a valid code:

// resume the generator after some time
setTimeout(() => generator.next(4), 1000);

As we can see, unlike regular functions, generators and the calling code can exchange results by passing values in next/yield.

To make things more obvious, here’s another example, with more calls:

function* gen() {
  let ask1 = yield "2 + 2?";

  alert(ask1); // 4

  let ask2 = yield "3 * 3?"

  alert(ask2); // 9
}

let generator = gen();

alert( generator.next().value ); // "2 + 2?"

alert( generator.next(4).value ); // "3 * 3?"

alert( generator.next(9).done ); // true

The execution picture:

  1. The first .next() starts the execution… It reaches the first yield.
  2. The result is returned to the outer code.
  3. The second .next(4) passes 4 back to the generator as the result of the first yield, and resumes the execution.
  4. …It reaches the second yield, that becomes the result of the generator call.
  5. The third next(9) passes 9 into the generator as the result of the second yield and resumes the execution that reaches the end of the function, so done: true.

It’s like a “ping-pong” game. Each next(value) (excluding the first one) passes a value into the generator, that becomes the result of the current yield, and then gets back the result of the next yield.

generator.throw

As we observed in the examples above, the outer code may pass a value into the generator, as the result of yield.

…But it can also initiate (throw) an error there. That’s natural, as an error is a kind of result.

To pass an error into a yield, we should call generator.throw(err). In that case, the err is thrown in the line with that yield.

For instance, here the yield of "2 + 2?" leads to an error:

function* gen() {
  try {
    let result = yield "2 + 2?"; // (1)

    alert("The execution does not reach here, because the exception is thrown above");
  } catch(e) {
    alert(e); // shows the error
  }
}

let generator = gen();

let question = generator.next().value;

generator.throw(new Error("The answer is not found in my database")); // (2)

The error, thrown into the generator at the line (2) leads to an exception in the line (1) with yield. In the example above, try..catch catches it and shows.

If we don’t catch it, then just like any exception, it “falls out” the generator into the calling code.

The current line of the calling code is the line with generator.throw, labelled as (2). So we can catch it here, like this:

function* generate() {
  let result = yield "2 + 2?"; // Error in this line
}

let generator = generate();

let question = generator.next().value;

try {
  generator.throw(new Error("The answer is not found in my database"));
} catch(e) {
  alert(e); // shows the error
}

If we don’t catch the error there, then, as usual, it falls through to the outer calling code (if any) and, if uncaught, kills the script.

Summary

  • Generators are created by generator functions function* f(…) {…}.
  • Inside generators (only) there exists a yield operator.
  • The outer code and the generator may exchange results via next/yield calls.

In modern JavaScript, generators are rarely used. But sometimes they come in handy, because the ability of a function to exchange data with the calling code during the execution is quite unique. And, surely, they are great for making iterable objects.

Also, in the next chapter we’ll learn async generators, which are used to read streams of asynchronously generated data (e.g paginated fetches over a network) in for await ... of loop.

In web-programming we often work with streamed data, so that’s another very important use case.

Tasks

There are many areas where we need random data.

One of them is testing. We may need random data: text, numbers etc, to test things out well.

In JavaScript, we could use Math.random(). But if something goes wrong, we’d like to be able to repeat the test, using exactly the same data.

For that, so called “seeded pseudo-random generators” are used. They take a “seed”, the first value, and then generate next ones using a formula. So that the same seed yields the same sequence, and hence the whole flow is easily reproducible. We only need to remember the seed to repeat it.

An example of such formula, that generates somewhat uniformly distributed values:

next = previous * 16807 % 2147483647

If we use 1 as the seed, the values will be:

  1. 16807
  2. 282475249
  3. 1622650073
  4. …and so on…

The task is to create a generator function pseudoRandom(seed) that takes seed and creates the generator with this formula.

Usage example:

let generator = pseudoRandom(1);

alert(generator.next().value); // 16807
alert(generator.next().value); // 282475249
alert(generator.next().value); // 1622650073

Open a sandbox with tests.

function* pseudoRandom(seed) {
  let value = seed;

  while(true) {
    value = value * 16807 % 2147483647
    yield value;
  }

};

let generator = pseudoRandom(1);

alert(generator.next().value); // 16807
alert(generator.next().value); // 282475249
alert(generator.next().value); // 1622650073

Please note, the same can be done with a regular function, like this:

function pseudoRandom(seed) {
  let value = seed;

  return function() {
    value = value * 16807 % 2147483647;
    return value;
  }
}

let generator = pseudoRandom(1);

alert(generator()); // 16807
alert(generator()); // 282475249
alert(generator()); // 1622650073

That also works. But then we loose ability to iterate with for..of and to use generator composition, that may be useful elsewhere.

Open the solution with tests in a sandbox.

Tutorial map

Comments

read this before commenting…
  • If you have suggestions what to improve - please submit a GitHub issue or a pull request instead of commenting.
  • If you can't understand something in the article – please elaborate.
  • To insert a few words of code, use the <code> tag, for several lines – use <pre>, for more than 10 lines – use a sandbox (plnkr, JSBin, codepen…)