301Gazillion Virtual Threads

Author: Dr Heinz M. KabutzDate: 2022-05-24Java Version: 19-ea+23Category: Concurrency

Abstract: As from this month, Project Loom is part of the mainstream OpenJDK as a preview feature. Expect fantastical results, as bloggers around the world kick the tyres of this fantastic technology. In this newsletter we show how we can have over 36 billion parked virtual threads in just 4 gigabytes of memory!

Welcome to the 301st edition of The Java(tm) Specialists' Newsletter, sent to you from beautiful Crete. Yesterday, we were treated to a wonderful lunch by Dave and Kate Farley, famous for their lectures on Continuous Delivery. Be sure to check out the Continuous Delivery YouTube videos - they have millions of views.

javaspecialists.teachable.com: Please visit our new self-study course catalog to see how you can upskill your Java knowledge.

Gazillion Virtual Threads

Project Loom joined the mainstream OpenJDK 19 as a preview release. As thousands of Java programmers around the world start kicking the tyres, expect some fantastical results. Fantastical, not fantastic. Loom is fantastic, but it is bound by normal hardware and operating system constraints.

A few years ago, I gave a talk about Java parallelism. It had Fork/Join, CompletableFuture, all the good stuff. Afterwards one of the audience walked up to the podium and told me confidently: "This will all be so much faster once we have fibers." My blank stare was a hint that I had no clue what fibers were. Even not knowing anything about the topic, I expressed my doubt that this was so, because I knew where the bottlenecks in my current experiments were.

Three years ago, I began playing around with Project Loom. It has always been a fun project and has huge potential, if used correctly. It is a game changer. You probably know the phrase "eat your own dogfood", so I am currently using some parts of Project Loom to power my JavaSpecialists.eu website.

Project Loom is exciting. However, a word of warning. It is easy to get very strange results, that seem too good to be true. And my father always warned his three boys - "If something sounds too good to be true, it probably is."

Take this example. All we are doing is creating virtual threads, incrementing a LongAdder and then parking them. Here is the body of each virtual thread Runnable:

() -> {
  threads.increment();
  LockSupport.park();
}

Seems easy enough. Obviously if the thread is parked, it will hang around forever, and so we could easily test how many we can start. This is how we can test how many platform threads is our upper limit. We would typically get an OutOfMemoryError after a few thousand platform threads. With virtual threads it is a bit more complicated. We need to ensure that we do not start them too quickly, otherwise the queue in the ForkJoinPool might run out of space. It thus works best to throttle the creation of new virtual threads. In this code, once we are more than 10 million thread creations ahead of their actual execution, we pause for a few seconds to allow the scheduler to catch up:

import java.time.*;
import java.util.*;
import java.util.concurrent.atomic.*;
import java.util.concurrent.locks.*;

public class GazillionVirtualThreads {
  public static void main(String... args)
      throws InterruptedException {
    LongAdder threads = new LongAdder();
    long started = 0;
    while (true) {
      Thread.startVirtualThread(() -> {
        threads.increment();
        LockSupport.park();
      });

      started++;
      if (started % 1_000_000 == 0) {
        long threadsCount = threads.longValue();
        System.out.printf(
            Locale.US, "started %,d\tthreads %,d%n",
            started, threadsCount
        );
        if (started - threadsCount > 10_000_000) {
          System.out.print("Waiting for virtual thread " +
              "scheduling to catch up");
          while (started - threads.longValue() > 1_000_000) {
            Thread.sleep(Duration.ofSeconds(1));
            System.out.print(".");
          }
          System.out.println();
        }
      }
    }
  }
}

Before running the code, let's do a quick calculation. Each virtual thread with a miniscule stack uses about 560 bytes. If we give the JVM four gigabytes of memory, we would expect to never be able to create more than 8 million virtual threads.

Let's try it out. We run the experiment with java -XX:+UseParallelGC -Xmx4g -Xms4g --enable-preview --source 19 GazillionVirtualThreads.java

Note: GazillionVirtualThreads.java uses preview features of Java SE 19.
Note: Recompile with -Xlint:preview for details.
started 1,000,000       threads 370,882
started 2,000,000       threads 718,547
started 3,000,000       threads 952,231
started 4,000,000       threads 1,111,716
started 5,000,000       threads 1,210,785
started 6,000,000       threads 1,425,531
started 7,000,000       threads 1,576,518
started 8,000,000       threads 1,711,458
started 9,000,000       threads 2,036,768
started 10,000,000      threads 2,320,047
started 11,000,000      threads 2,576,230
started 12,000,000      threads 2,836,093
started 13,000,000      threads 3,095,980
started 14,000,000      threads 3,354,914
Waiting for virtual thread scheduling to catch up...
started 15,000,000      threads 14,485,749 <--- Hmmmmm
started 16,000,000      threads 14,735,566
started 17,000,000      threads 14,994,275

  *snip*

started 1,000,000,000   threads 993,215,958
started 1,001,000,000   threads 993,463,449
started 1,002,000,000   threads 993,715,201
started 1,003,000,000   threads 993,962,325
started 1,004,000,000   threads 994,216,297
started 1,005,000,000   threads 994,461,520
Waiting for virtual thread scheduling to catch up......
started 1,006,000,000   threads 1,005,436,761
started 1,007,000,000   threads 1,005,712,093

    *snip*

started 7,999,000,000   threads 7,992,495,312
started 8,000,000,000   threads 7,993,003,679
started 8,001,000,000   threads 7,993,553,029
started 8,002,000,000   threads 7,994,047,351
started 8,003,000,000   threads 7,994,530,007
started 8,004,000,000   threads 7,995,040,188
started 8,005,000,000   threads 7,995,591,552
started 8,006,000,000   threads 7,996,106,365
started 8,007,000,000   threads 7,996,722,771
Waiting for virtual thread scheduling to catch up......
started 8,008,000,000   threads 8,007,245,647
started 8,009,000,000   threads 8,007,468,686
started 8,010,000,000   threads 8,007,722,444

    *snip*

started 36,393,000,000  threads 36,387,763,447
started 36,394,000,000  threads 36,388,011,699
started 36,395,000,000  threads 36,388,276,279
started 36,396,000,000  threads 36,388,531,455
started 36,397,000,000  threads 36,388,786,248

  ???

This is fun. We have far more parked virtual threads than bytes in our virtual machine. No wonder the Java world is going bananas over this new technology!

We know of course that there is a catch. The garbage collector can clean up our virtual threads once they become unreachable and parked. We can see this with jmap -histo pid. Of our 36 billion "parked" virtual threads, only 3 million VirtualThread instances remain in the JVM:

 num #instances    #bytes  class name (module)
-----------------------------------------------
  1:   1753240  490907200  jdk.internal.vm.StackChunk
  2:   3071315  417698840  java.lang.VirtualThread
  3:   3071315  147423120  java.lang.VirtualThread$VThreadContinuation
  4:   3071315   73711560  java.lang.VirtualThread$VThreadContinuation$$Lambda$242
  5:   3071315   73711560  java.util.concurrent.ForkJoinTask$RunnableExecuteAction
  6:        13   67112144  [Ljava.util.concurrent.ForkJoinTask;
  7:   3072970   49167520  java.lang.Object
  8:   3071315   49141040  GazillionVirtualThreads$$Lambda$236
  9:   3071315   49141040  java.lang.VirtualThread$$Lambda$243
 10:     14776   14897344  Ljava.internal.vm.FillerArray

Platform threads are GC roots. But not these virtual threads. Once they are no longer reachable and have parked, there is no way to unpark them. They can thus be garbage collected. This creates the illusion that we can create a gazillion virtual threads.

If we change the park to LockSupport.parkUntil(Long.MAX_VALUE) then the JVM becomes unresponsive at about 5.6 million virtual threads. Virtual threads are cool, but they are not free and they do not run faster than the speed of light.

Heinz

Comments

We are always happy to receive comments from our readers. Feel free to send me a comment via email or discuss the newsletter in our JavaSpecialists Slack Channel (Get an invite here)

Related Articles

Browse the Newsletter Archive