The reducers library (in the clojure.core.reducers namespace) has alternative implementations of map, filter, and other seq functions. These alternative functions are called reducers, and you can apply almost everything you know about seq functions to reducers. Just like seq functions, the purpose of reducers is to transform collections. For example, if you wanted to increment every number in a vector, filter out the even ones, and sum the numbers, the seq and reducer versions look virtually identical. Try this in a REPL:

In both examples, (range 10) returns a seq of the numbers 0 through 9. In the first example, map is a seq function. In the second, r/map is a reducer. In both examples, map has the same purpose: transforming an input collection to an output by applying a function to each element of the input. Reducers are just another means of transforming collections, and for the most part they’re used just like the seq functions you already know and love.

So why bother with them? Reducers are designed to perform better (often dramatically better) than seq functions under some circumstances by performing eager computation, eliminating intermediate collections and allowing parallelism.

In fact, as far as I can tell, parallelization is the main reason Rich Hickey, Lambda Whisperer added reducers to the language. The goal of executing collection transformations in parallel completely determined how reducers are designed, and even explains why we use the term reducers: reduce operations can be trivially parallellized (you’ll learn how by the end), and every collection transformation can be expressed in terms of reduce (there’s a big juicy math paper on that).

Thus, the reducers library is built around using reduce to transform collections, and it’s designed this way to make parallelization possible. Both eager computation and the elimination of intermediate collections are secondary - but useful - consequences of this design. (By the way: "elimination of intermediate collections" is a freaking mouthful, so I’m going to start using EIC).

Let’s look at these three strategies more closely. After that, I’ll share rules for how to ensure that your code makes use of each of them, without going into much detail about why they work as they do. Last, I’ll also give some examples of reducers at work, showing how their performance compares to alternatives.

Understanding the reduce library’s three performance strategies - eager computation, EIC, and parallelism - will give you the rationale behind reducers, and that will help you understand their implementation later on. Plus, these ideas are pretty groovy in their own right. I know you’re eager to learn about eager computation, so let’s go!

You’ve seen how, for the most part, you can use reducers just like seq functions. For example, this produces the same result as the seq counterpart:

There are a couple rules to keep in mind, though. First, if you want to make use of parallelism, you have to use the r/fold function with a foldable collection. r/fold is a parrallel implementation of reduce.

I need to explain parallelism a bit more before explaining what foldable means, but for now the relevant fact is that vectors and maps are the only foldable collections. So if you wrote code like this, you might expect it to run in parallel, but it wouldn’t:

Lists aren’t foldable, so reducers can’t operate on them in parallel. Instead, the code falls back to serial reduce.

The second rule is that reducers don’t produce collections. It’s awesome that r/map and r/filter compose without creating intermediate collections, but the catch is that they behave a little differently from seq functions. Here’s one way you could get tripped up:

This happens because r/map and the other reducers don’t actually return a new collection. Instead, they return a reducible.

A reducible is like a recipe for how to produce a new collection, along with a reference to a source collection. In the above example, the recipe is "produce a new collection by incrementing each element each element of the source collection", and the source collection is the vector [1 2 3].

Another way to put it: a reducible is an elemental function, along with the collection whose elements you want to apply the function to.

And yet another way to put it: It’s as if you were one of Santa’s worker elves (or some other mythical creature of labor), and the foreman handed you a piece of paper with your instructions for the day: "Go to the plastic shed out back, get all the plastic lumps and turn them into toy whatevers to feed to the insatiable, gaping maw of consumerism."

If I then walked up to you and said, "give me the first of that piece of paper", you would look at me in confusion, because the request wouldn’t make any sense. The piece of paper is not a collection that you can take the first of. Similarly, when you tell Clojure (first (r/map inc [1 2 3])) it expresses its confusion with an exception because the request doesn’t make sense.

You might initially think you’re telling Clojure "give me the first element of the collection returned by `r/map`", but you’re actually saying "give me the first element of the reducible returned by `r/map`". A reducible is a recipe for how to transform a given collection, but it’s not a collection itself, so Clojure can’t fulfill your request.

This design decision to have functions like r/map return a reducible instead of a collection is what allows reducers to seamlessly compose, building up a single fused elemental function. I’ll explain exactly how this happens later in the book, and for now rely on the power of metaphor so that you’ll have a strong intuitive understanding of how it works.

But you need to get collections somehow. If you want to use reducer functions to produce another collection, you have to explicitly realize the result collection by calling r/foldcat or clojure.core/into. r/foldcat calls r/fold internally, and into calls reduce. Here are two examples:

You should use into when you want to explicitly specify the collection type. In the example above, you’re realizing the collection as a vector. into is serial.

r/foldcat executes parallelly (is that a word?) and returns a collection that acts pretty much like a vector. The collection is foldable, so you can use it with reducers in further parallel computations. It’s also seqable, like a vector, so you can call functions like first on it.

To summarize the rules:

Soooo now that you’ve spent all this time learning about reducers' performance strategies and how to use them, let’s wrap everything together by comparing reducer performance to seq functions. If I’ve done my job right, you won’t be surprised by the differences. If I haven’t done my job right, then, well…​ sorry?

Reducer performance depends on two variables:

To see the performance impact of these variables, we’re going to look at similar computations performed using r/fold (for parallelism and EIC), r/reduce (for EIC alone), and clojure.core/reduce as a baseline. We’ll also perform the computations on foldable collections (vectors) and on unfoldable collections (lazy seqs).

You can find the code for this section in the code folder of your download under src/ifacepalm/using_reducers.clj (paid version only). After doing the performance measurements for awhile, I wanted to make the process a bit less tedious, so I wrote the macro times and its helper functions pretty-time and runs-and-pauses to abstract the process of running a snippet multiple times, timing each run, averaging those time values, and usefully printing the result. This lets you do something like this:

In this example, you’re performing the same computation using three different methods, and seeing how long each method takes.

The code for times is in the ifacepalm.time-helpers namespace, but I’m not going to go over it here for fear that you would (justifiably) shake your fist at me from behind your monitor for veering off topic.

Here are our first comparisons:

First, we define two collections of numbers from 0 to 99,999,999. snums is an unfoldable seq and snumsv is a foldable vector.

Next, we define three functions, t1, t2, and t3. Each function takes a collection (we’ll use snums and snumsv), transforms it using r/fold, r/reduce, and reduce, and prints out how much time each transformation takes. Enough with the yik yak, let’s actually use them:

When we call t1 with snums, the three reduction functions have roughly the same performance. Put another way: we don’t get any performance benefits from EIC or parallelism. This makes sense because we’re not doing a map, filter, or some other function that semantically returns a new collection. We’re also using the unfoldable snums, and in that case r/fold quietly degrades to r/reduce.

When we call t2 with snumsv, though, we see a significant speedup from r/fold. My computer has four cores, so ideally r/fold would take 25% as much time as the serial versions, but it’s rare to meet that ideal because parallelization has overhead costs.

Let’s see what happens when we have an intermediate transformation (map):

Dayumn! The reducer versions take nearly fifty percent less time! There’s no difference between r/fold and r/reduce, though, becuase we’re not using a foldable collection. Here’s what happens when you change that:

Here, r/fold is five times faster than reduce, thanks to EIC and parallelism. (Interestingly, it seems serial reduce takes a bit longer with vectors.)

What happens if you have two intermediate collections? This happens:

When you call r/fold with a vector, it’s almost seven times faster. Not only that, but look at the times for the r/reduce calls in (t2 snumsv) and (t3 snumsv): 214ms and 207ms respectively. Adding the intermediate transformation had virtually no impact on performance. (It actually took less time in this case, but I chalk that up to random noise.) Compare that to the reduce calls: 374ms and 478ms. Huge performance impact!

This should give you a good idea of the kinds of performance gains that reducers can give you. And the best part is that your code will perform even better if you run it on a machine with more cores, without any extra work. Neat!

And thus concludes the introductory tutorial on reducers. Hopefully I’ve given you everything you need to start using them effectively. Next comes the really fun part: learning about parallelism concepts so that you’ll have a complete understanding of what the reducers library is doing.