11 Iterations and Comprehensions
The for family of syntactic forms support iteration over sequences. Lists, vectors, strings, byte strings, input ports, and hash tables can all be used as sequences, and constructors like in-range offer even more kinds of sequences.
Variants of for accumulate iteration results in different ways, but they all have the same syntactic shape. Simplifying for now, the syntax of for is
(for ([id sequence-expr] )
body )
A for loop iterates through the sequence produced by the sequence-expr. For each element of the sequence, for binds the element to id, and then it evaluates the bodys for side effects.
Examples: | ||
| ||
123 | ||
| ||
a...b...c... |
The for/list variant of for is more Scheme-like. It accumulates body results into a list, instead of evaluating body only for side effects. In more technical terms, for/list implements a list comprehension.
Examples: | ||
| ||
(1 4 9) | ||
| ||
(#\a #\b #\c) |
The full syntax of for accommodates multiple sequences to iterate in parallel, and the for* variant nests the iterations instead of running them in parallel. More variants of for and for* accumulate body results in different ways. In all of these variants, predicates that prune iterations can be included along with bindings.
Before details on the variations of for, though, it’s best to see the kinds of sequence generators that make interesting examples.
11.1 Sequence Constructors
The in-range function generates a sequence of numbers, given an optional starting number (which defaults to 0), a number before which the sequences ends, and an optional step (which defaults to 1).
Examples: | ||
| ||
012 | ||
| ||
123 | ||
| ||
13 | ||
| ||
432 | ||
| ||
1 3/2 2 5/2 3 7/2 |
The in-naturals function is similar, except that the starting number must be an exact non-negative integer (which defaults to 0), the step is always 1, and there is no upper limit. A for loop using just in-naturals will never terminate unless a body expression raises an exception or otherwise escapes.
Examples: | ||||
| ||||
0123456789 | ||||
too much! |
The stop-before and stop-after functions construct a new sequence given a sequence and a predicate. The new sequence is like the given sequence, but truncated either immediately before or immediately after the first element for which the predicate returns true.
Examples: | |||
| |||
abc |
Sequence constructors like in-list, in-vector and in-string simply make explicit the use of a list, vector, or string as a sequence. Since they raise an exception when given the wrong kind of value, and since they otherwise avoid a run-time dispatch to determine the sequence type, they enable more efficient code generation; see Iteration Performance for more information.
Examples: | ||
| ||
abc | ||
| ||
in-string: expected argument of type <string>; given (1 2 3) |
Sequences in Reference: PLT Scheme provides more on sequences.
11.2 for and for*
A more complete syntax of for is
(for (clause )
body )
clause
=
[id sequence-expr]
|
#:when boolean-expr
When multiple [id sequence-expr] clauses are provided in a for form, the corresponding sequences are traversed in parallel:
| |||
|
With parallel sequences, the for expression stops iterating when any sequence ends. This behavior allows in-naturals, which creates an infinite sequence of numbers, to be used for indexing:
| |||
|
The for* form, which has the same syntax as for, nests multiple sequences instead of running them in parallel:
| ||||||
|
Thus, for* is a shorthand for nested fors in the same way that let* is a shorthand for nested lets.
The #:when boolean-expr form of a clause is another shorthand. It allows the bodys to evaluate only when the boolean-expr produces a true value:
| ||||
|
A boolean-expr with #:when can refer to any of the preceding iteration bindings. In a for form, this scoping makes sense only if the test is nested in the iteration of the preceding bindings; thus, bindings separated by #:when are mutually nested, instead of in parallel, even with for.
| ||||||
|
11.3 for/list and for*/list
The for/list form, which has the same syntax as for, evaluates the bodys to obtain values that go into a newly constructed list:
| |||
("1. Intro" "2. Details" "3. Conclusion") |
A #:when clause in a for-list form prunes the result list along with evaluations of the bodys:
| ||||
("Intro" "Conclusion") |
This pruning behavior of #:when is more useful with for/list than for. Whereas a plain when form normally suffices with for, a when expression form in a for/list would cause the result list to contain #<void>s instead of omitting list elements.
The for*/list is like for*, nesting multiple iterations:
| |||
("Guide Intro" "Guide Details" "Ref. Intro" "Ref. Details") |
A for*/list form is not quite the same thing as nested for/list forms. Nested for/lists would produce a list of lists, instead of one flattened list. Much like #:when, then, the nesting of for*/list is more useful than the nesting of for*.
11.4 for/and and for/or
The for/and form combines iteration results with and, stopping as soon as it encounters #f:
| ||
#f |
The for/or form combines iteration results with or, stopping as soon as it encounters a true value:
| ||
#t |
As usual, the for*/and and for*/or forms provide the same facility with nested iterations.
11.5 for/first and for/last
The for/first form returns the result of the first time that the bodys are evaluated, skipping further iterations. This form is most useful with a #:when clause.
| |||
"Details" |
If the bodys are evaluated zero times, then the result is #f.
The for/last form runs all iterations, returning the value of the last iteration (or #f if no iterations are run):
| |||
"Conclusion" |
As usual, the for*/first and for*/last forms provide the same facility with nested iterations:
| ||||
("Guide" "Details") | ||||
| ||||
("Reference" "Conclusion") |
11.6 for/fold and for*/fold
The for/fold form is a very general way to combine iteration results. Its syntax is slightly different than the syntax of for, because accumulation variables must be declared at the beginning:
(clause ...) |
body ...+) |
In the simple case, only one [accum-id init-expr] is provided, and the result of the for/fold is the final value for accum-id, which starts out with the value of init-expr. In the clauses and bodys, accum-id can be referenced to get its current value, and the last body provides the value of accum-id for the next iteration.
Examples: | ||||||
| ||||||
15 | ||||||
| ||||||
| ||||||
"Conclusion" |
When multiple accum-ids are specified, then the last body must produce multiple values, one for each accum-id. The for/fold expression itself produces multiple values for the results.
Examples: | |||||||
| |||||||
| |||||||
"Conclusion" | |||||||
4 |
11.7 Multiple-Valued Sequences
In the same way that a function or expression can produce multiple values, individual iterations of a sequence can produce multiple elements. For example, a hash table as a sequence generates two values for each iteration: a key and a value.
In the same way that let-values binds multiple results to multiple identifiers, for can bind multiple sequence elements to multiple iteration identifiers:
While let must be changed to let-values to bind multiple identifier, for simply allows a parenthesized list of identifiers instead of a single identifier in any clause.
| ||
|
This extension to multiple-value bindings works for all for variants. For example, for*/list nests iterations, builds a list, and also works with multiple-valued sequences:
| |||
("apple" "banana" "banana" "banana") |
11.8 Iteration Performance
Ideally, a for iteration should run as fast as a loop that you write by hand as a recursive-function invocation. A hand-written loop, however, is normally specific to a particular kind of data, such as lists. In that case, the hand-written loop uses selectors like car and cdr directly, instead of handling all forms of sequences and dispatching to an appropriate iterator.
The for forms can provide the performance of hand-written loops when enough information is apparent about the sequences to iterate. Specifically, the clause should have one of the following fast-clause forms:
fast-clause |
| = |
| [id fast-seq] |
|
| | |
| [(id) fast-seq] |
|
| | |
| [(id id) fast-indexed-seq] |
|
| | |
| [(id ...) fast-parallel-seq] |
fast-seq |
| = |
| (in-range expr expr) |
|
| | |
| (in-range expr expr expr) |
|
| | |
| |
|
| | |
| (in-naturals expr) |
|
| | |
| (in-list expr) |
|
| | |
| (in-vector expr) |
|
| | |
| (in-string expr) |
|
| | |
| (in-bytes expr) |
|
| | |
| (stop-before fast-seq predicate-expr) |
|
| | |
| (stop-after fast-seq predicate-expr) |
fast-indexed-seq |
| = |
| (in-indexed fast-seq) |
|
| | |
| (stop-before fast-indexed-seq predicate-expr) |
|
| | |
| (stop-after fast-indexed-seq predicate-expr) |
fast-parallel-seq |
| = |
| (in-parallel fast-seq ...) |
|
| | |
| (stop-before fast-parallel-seq predicate-expr) |
|
| | |
| (stop-after fast-parallel-seq predicate-expr) |
Examples: | ||||
| ||||
cpu time: 5 real time: 5 gc time: 0 | ||||
| ||||
cpu time: 68 real time: 68 gc time: 0 | ||||
| ||||
cpu time: 70 real time: 70 gc time: 0 |
The grammars above are not complete, because the set of syntactic patterns that provide good performance is extensible, just like the set of sequence values. The documentation for a sequence constructor should indicate the performance benefits of using it directly in a for clause.
Iterations and Comprehensions: for, for/list, ... in Reference: PLT Scheme provides more on iterations and comprehensions.