Navigation

Java multithreading 7 min read

Thread Pools & Executors

Creating a brand-new Thread for every task your application needs to run sounds simple, but it doesn’t scale — threads are expensive to create, and spawning thousands of them can crash your JVM. Thread pools solve this by keeping a set of reusable worker threads ready to pick up tasks from a queue, dramatically reducing overhead and giving you fine-grained control over concurrency.

Why Thread Pools?

Every raw new Thread(runnable).start() call involves:

Allocating stack memory (usually 512 KB–1 MB per thread by default)
Registering the thread with the OS scheduler
Tearing it all down when the thread finishes

For a web server handling 10 000 requests per second, that cost becomes enormous. A thread pool creates a fixed set of worker threads upfront. Tasks are submitted to an internal queue; an idle worker picks the next task, runs it, and then waits for the next one — no creation or teardown required.

Tip: Even in modern Java with virtual threads, understanding the classic ExecutorService API is essential — most production codebases use it, and virtual threads are configured through the same interface.

The Executor Framework

Java’s concurrency utilities live in java.util.concurrent. The hierarchy looks like this:

Interface / Class	Role
`Executor`	Single method: `execute(Runnable)`
`ExecutorService`	Adds lifecycle (`shutdown`, `submit`, `invokeAll`)
`ScheduledExecutorService`	Adds scheduling (`schedule`, `scheduleAtFixedRate`)
`ThreadPoolExecutor`	Concrete, fully configurable pool
`Executors`	Factory helper that creates common pool types

You almost always program to the ExecutorService interface, and use Executors to build it.

Creating Thread Pools with `Executors`

Fixed Thread Pool

A fixed pool keeps exactly n threads alive at all times. Extra submitted tasks queue up and wait.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class FixedPoolDemo {
    public static void main(String[] args) throws InterruptedException {
        ExecutorService pool = Executors.newFixedThreadPool(3);

        for (int i = 1; i <= 6; i++) {
            final int taskId = i;
            pool.execute(() -> {
                System.out.println("Task " + taskId + " running on " + Thread.currentThread().getName());
                try { Thread.sleep(500); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
            });
        }

        pool.shutdown(); // signal: no new tasks
        pool.awaitTermination(5, java.util.concurrent.TimeUnit.SECONDS);
        System.out.println("All tasks done.");
    }
}

Output:

Task 1 running on pool-1-thread-1
Task 2 running on pool-1-thread-2
Task 3 running on pool-1-thread-3
Task 4 running on pool-1-thread-1   ← thread reused
Task 5 running on pool-1-thread-2
Task 6 running on pool-1-thread-3
All tasks done.

Notice the same three thread names repeating — no new threads were created for tasks 4–6.

Cached Thread Pool

Creates new threads as needed, but reuses idle ones. Threads that have been idle for 60 seconds are terminated. Good for short-lived, bursty tasks.

ExecutorService cached = Executors.newCachedThreadPool();

Warning: newCachedThreadPool() can spin up an unbounded number of threads if tasks are submitted faster than they complete. Under heavy load this can exhaust system resources. Prefer a fixed or custom pool in production.

Single-Thread Executor

Guarantees tasks run sequentially in submission order on one dedicated thread — useful for ordered background work.

ExecutorService single = Executors.newSingleThreadExecutor();

Scheduled Thread Pool

Runs tasks after a delay or on a repeating schedule.

import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

public class ScheduledDemo {
    public static void main(String[] args) {
        ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);

        // Run once after a 2-second delay
        scheduler.schedule(() -> System.out.println("Delayed task"), 2, TimeUnit.SECONDS);

        // Run every 3 seconds, starting after 1 second
        scheduler.scheduleAtFixedRate(
            () -> System.out.println("Periodic: " + System.currentTimeMillis()),
            1, 3, TimeUnit.SECONDS
        );

        // Let it run for a bit, then shut down
        scheduler.schedule(() -> {
            System.out.println("Shutting down");
            scheduler.shutdown();
        }, 10, TimeUnit.SECONDS);
    }
}

Submitting Tasks: `execute` vs `submit`

Method	Accepts	Returns	Exceptions
`execute(Runnable)`	`Runnable`	`void`	Uncaught exceptions go to the thread’s `UncaughtExceptionHandler`
`submit(Runnable)`	`Runnable`	`Future<?>`	Stored in the `Future`, thrown on `get()`
`submit(Callable<T>)`	`Callable<T>`	`Future<T>`	Stored in the `Future`, thrown on `get()`

Use submit whenever you need a result or want to handle exceptions cleanly. See the Callable & Future page for deep coverage.

import java.util.concurrent.*;

public class SubmitDemo {
    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newFixedThreadPool(2);

        Future<Integer> future = pool.submit(() -> {
            int sum = 0;
            for (int i = 1; i <= 100; i++) sum += i;
            return sum;
        });

        System.out.println("Result: " + future.get()); // blocks until done
        pool.shutdown();
    }
}

Output:

Result: 5050

Shutting Down an ExecutorService

Always shut down a pool when you’re finished with it — otherwise its threads keep the JVM alive.

pool.shutdown();                              // finish queued tasks, reject new ones
boolean finished = pool.awaitTermination(10, TimeUnit.SECONDS);
if (!finished) {
    pool.shutdownNow();                       // interrupt running tasks
}

Method	Behaviour
`shutdown()`	Graceful — completes in-flight and queued tasks
`shutdownNow()`	Interrupts running threads, returns list of queued tasks
`isShutdown()`	Returns `true` after `shutdown()` is called
`isTerminated()`	Returns `true` when all tasks have completed post-shutdown

Custom `ThreadPoolExecutor`

The Executors factory methods are convenient, but for production use you often want explicit control over pool sizing, queue capacity, and rejection policy.

import java.util.concurrent.*;

public class CustomPoolDemo {
    public static void main(String[] args) {
        ThreadPoolExecutor executor = new ThreadPoolExecutor(
            2,                          // corePoolSize
            5,                          // maximumPoolSize
            30, TimeUnit.SECONDS,       // keepAliveTime for idle threads above core
            new ArrayBlockingQueue<>(10), // bounded work queue
            Executors.defaultThreadFactory(),
            new ThreadPoolExecutor.CallerRunsPolicy() // rejection policy
        );

        for (int i = 1; i <= 8; i++) {
            final int id = i;
            executor.execute(() -> System.out.println("Task " + id
                    + " by " + Thread.currentThread().getName()));
        }
        executor.shutdown();
    }
}

Rejection Policies

When the work queue is full and all max threads are busy, submitted tasks are rejected. Java provides four built-in policies:

Policy	What happens
`AbortPolicy` (default)	Throws `RejectedExecutionException`
`CallerRunsPolicy`	The calling thread runs the task itself — acts as back-pressure
`DiscardPolicy`	Silently drops the task
`DiscardOldestPolicy`	Drops the oldest queued task and retries

Tip: CallerRunsPolicy is a natural throttle: the submitter can’t submit new tasks while it’s busy running the rejected one. This is a simple and effective back-pressure mechanism.

`invokeAll` and `invokeAny`

When you have a batch of Callable tasks:

import java.util.*;
import java.util.concurrent.*;

public class InvokeAllDemo {
    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newFixedThreadPool(3);

        List<Callable<String>> tasks = List.of(
            () -> { Thread.sleep(100); return "Result A"; },
            () -> { Thread.sleep(200); return "Result B"; },
            () -> { Thread.sleep(50);  return "Result C"; }
        );

        // invokeAll: waits for ALL to complete
        List<Future<String>> futures = pool.invokeAll(tasks);
        for (Future<String> f : futures) {
            System.out.println(f.get());
        }

        // invokeAny: returns the FIRST successful result
        String first = pool.invokeAny(tasks);
        System.out.println("Fastest: " + first);

        pool.shutdown();
    }
}

Output:

Result A
Result B
Result C
Fastest: Result C

Under the Hood

How `ThreadPoolExecutor` Works

Internally, ThreadPoolExecutor maintains three things: a set of worker threads, a work queue (BlockingQueue<Runnable>), and an atomic integer that packs the pool state and worker count into a single int.

When you call execute(task):

If the number of running threads is below corePoolSize, a new worker thread is created even if others are idle.
If at or above corePoolSize, the task is offered to the BlockingQueue.
If the queue is full and the thread count is below maximumPoolSize, a new thread is created.
If the queue is full and the thread count is at maximum, the rejection policy fires.

Core threads stay alive indefinitely by default. Non-core threads (those spawned above corePoolSize) are terminated after keepAliveTime if idle.

Work-Stealing Pool (Java 7+)

Executors.newWorkStealingPool() creates a ForkJoinPool where each worker thread has its own deque of tasks. An idle thread can steal tasks from the tail of a busy thread’s deque — great for uneven workloads and recursive divide-and-conquer tasks.

ExecutorService workStealing = Executors.newWorkStealingPool();
// Uses Runtime.getRuntime().availableProcessors() threads by default

Thread Pool Sizing Rules of Thumb

Workload type	Recommended pool size
CPU-bound tasks	`N_CPU` (number of cores)
I/O-bound tasks	`N_CPU * (1 + wait_time / compute_time)`
Mixed	Profile and benchmark — no one-size-fits-all

The classic formula for I/O-bound work is from Java Concurrency in Practice by Brian Goetz. If threads spend 90 % of their time waiting on I/O, each core can usefully run ~10 threads.

Note: In Java 21+, virtual threads flip this calculus — you can submit millions of virtual threads and the JVM handles the I/O blocking without tying up OS threads. For new I/O-heavy services, virtual threads are the modern answer; ThreadPoolExecutor remains the right tool for CPU-bound work.

Callable & Future — submit tasks that return a result and handle exceptions cleanly
Virtual Threads (Project Loom) — the Java 21 model that makes a thread-per-task viable at massive scale
Synchronization — prevent data races when multiple pool workers share state
Deadlock — how thread pools can deadlock when tasks submit dependent sub-tasks
Java Memory Model — visibility guarantees that apply between pool threads
Multithreading — the full concurrency overview to see where thread pools fit in

Thread Pools & Executors

Why Thread Pools?

The Executor Framework

Creating Thread Pools with Executors

Fixed Thread Pool

Cached Thread Pool

Single-Thread Executor

Scheduled Thread Pool

Submitting Tasks: execute vs submit

Shutting Down an ExecutorService

Custom ThreadPoolExecutor

Rejection Policies

invokeAll and invokeAny

Under the Hood

How ThreadPoolExecutor Works

Work-Stealing Pool (Java 7+)

Thread Pool Sizing Rules of Thumb

Related Topics

Creating Thread Pools with `Executors`

Submitting Tasks: `execute` vs `submit`

Custom `ThreadPoolExecutor`

`invokeAll` and `invokeAny`

How `ThreadPoolExecutor` Works