327ShuffleGatherer

Author: Dr Heinz M. KabutzDate: 2025-08-29Java Version: 25Sources on GitHubCategory: Tips and Tricks

Abstract: Instead of shuffling our stream at the end in the collect() method, we can also shuffle it in stages using the new stream gatherers. This allows us to shuffle sections at a time and even support infinite streams.

A hearty welcome to our 327th edition of The Java(tm) Specialists' Newsletter, sent from the magical island of Crete. We recently started informal "office hours" for those who bought courses on my JavaSpecialists Teachable School. We talk about where Java has come from and where it is going, and address specific questions about our courses or Java. It is a great little community. One of the folks who joined us yesterday has been reading our newsletter for over 24 years! To join, just buy any one of our Teachable courses and make sure that emails are turned on, so that we can send you an invite.

After my trip to JavaZone next week, I'm heading off to Germany for a small family reunion, in my grandmother's village. It's beautiful with a forest and a sweeping view over the fields below. Her old house is now an AirBnb, so I will sleep in the exact same room I used when visiting her as a youngster. My Oma was a legend. She had a heart attack in her sixties and then turned her life around. She stopped smoking, started walking every day and treating herself to a daily afternoon nap. She lived in this house until her death at the ripe old age of 98. She was forever wanting to learn new things, and no challenge was too hard for her. Yes, she had a laptop and email, she was a modern granny.

javaspecialists.teachable.com: Please visit our new self-study course catalog to see how you can upskill your Java knowledge.

ShuffleGatherer

In our last newsletter, we showed a way to "distinctify" a stream using the new stream gatherer mechanism. As promised, we now look at how we can shuffle a stream using a Gatherer. We proposed a ShuffleCollector before, which I use on my website to give a random suggestion of related newsletters. Whilst the ShuffleCollector worked, it had a few disadvantages. We worked on the end-point of a stream, in the collect() method. We were thus not able to shuffle just parts of a stream. It would also have failed with an OutOfMemoryError if our stream contained more than Integer.MAX_VALUE - 2 elements.

In this newsletter, we show a new version using the Gatherer. It allows us to specify a shuffle window, thus only mixing up a section of the stream at a time. This would allow us to even support infinite streams.

Here is our ShuffleGatherer. Note that in Java 25, we can import an entire module, rather than individual packages. Also, instead of Random, we now pass in a RandomGenerator, an interface that was introduced in Java 17 to abstract random numbers generators. Once we have reached our windowSize, we shuffle that and send the output to the downstream.

package eu.javaspecialists.tjsn.issue327;

// no more need for individual imports in Java 25 :-)
import module java.base;

public class ShuffleGatherer {
    public static <T> Gatherer<T, List<T>, T> of() {
        return of(ThreadLocalRandom.current());
    }

    public static <T> Gatherer<T, List<T>, T> of(
            int windowSize) {
        return of(ThreadLocalRandom.current(), windowSize);
    }

    public static <T> Gatherer<T, List<T>, T> of(
            RandomGenerator random) {
        return of(random, Integer.MAX_VALUE - 8);
    }

    public static <T> Gatherer<T, List<T>, T> of(
            RandomGenerator random, int windowSize) {
        return Gatherer.ofSequential(
                ArrayList::new,
                (list, element, downstream) -> {
                    list.add(element);
                    if (list.size() == windowSize) {
                        shuffleAndSend(random, list, downstream);
                    }
                    return true;
                },
                (list, downstream) ->
                        shuffleAndSend(random, list, downstream)
        );
    }

    private static <T> void shuffleAndSend(
            RandomGenerator random,
            List<T> list,
            Gatherer.Downstream<? super T> downstream) {
        Collections.shuffle(list, random);
        list.stream()
                .takeWhile(_ -> !downstream.isRejecting())
                .forEach(downstream::push);
        list.clear();
    }
}

Here is an example that shuffles a stream of Integers, similar to what we did in newsletter 258. Note that the main() method and println() have also been simplified since Java 25 (JEP 512).

package eu.javaspecialists.tjsn.issue327;

import module java.base;

public class PrimitiveShuffleCollectorTest {
    private void printRandom(
            int from, int upto, int limit,
            Gatherer<Integer, List<Integer>, Integer> shuffler) {
        var shuffled = IntStream.range(from, upto)
                .boxed()
                .gather(shuffler)
                .limit(limit)
                .toList();
        IO.println(shuffled);
    }

    // Simplified main() method for Java 25
    public void main() {
        // Shuffle integers 0..9, and pick the first 5 elements
        printRandom(0, 10, 5, ShuffleGatherer.of());
        // Unpredictable output - ThreadLocalRandom by default

        // Shuffle integers 0..9, and pick the first 5 elements
        // Use Random(0) to get repeatable results.
        printRandom(0, 10, 5, ShuffleGatherer.of(new Random(0)));
        // [4, 8, 9, 6, 3]

        // Shuffle integers 0..999, and pick the first 3 elements
        printRandom(0, 1000, 3, ShuffleGatherer.of(new Random(0)));
        // [490, 539, 694]

        // Shuffle integers 0..7, with a shuffle window of 3
        printRandom(0, 8, 8, ShuffleGatherer.of(new Random(0), 3));
        // [2, 1, 0, 3, 5, 4, 6, 7]
    }
}

Our output is the following:

[0, 9, 3, 7, 1]
[4, 8, 9, 6, 3]
[490, 539, 694]
[2, 1, 0, 3, 5, 4, 6, 7]

The most interesting result is the last one with a shuffle window of 3. It would thus have the values [0, 1, 2] shuffled, then [3, 4, 5] shuffled, and lastly [6, 7] shuffled.

Kind regards

Heinz

P.S. I wrote part of this newsletter walking / standing on my new Walkolution 2. Fortunately my wife supports me getting anything that will make me more productive :-)

Comments

We are always happy to receive comments from our readers. Feel free to send me a comment via email or discuss the newsletter in our JavaSpecialists Slack Channel (Get an invite here)

Related Articles

Browse the Newsletter Archive