How to Scale Spring Boot for High-Traffic Apps

Scaling Spring Boot applications for high traffic boils down to addressing bottlenecks across your system. Here’s a quick breakdown of effective strategies:

• Optimise Code and Database: Move to non-blocking I/O with WebClient, enable virtual threads (Java 21+), and fine-tune database queries. For example, optimise queries with indexes, use @EntityGraph, and avoid N+1 issues.
• Caching: Use tools like Redis or Caffeine to reduce database load by up to 80%. Implement multi-level caching for faster reads and plan invalidation strategies to prevent stale data.
• Thread Pools: Configure thread pools for specific workloads. For I/O-heavy tasks, increase pool sizes; for CPU-bound tasks, limit threads to match core count.
• Reactive Programming: Leverage Spring WebFlux for non-blocking I/O to handle thousands of concurrent requests with fewer resources.
• Horizontal Scaling: Distribute traffic using load balancers like NGINX or HAProxy. Ensure stateless architecture with distributed caching for session management.
• Monitoring and Load Testing: Use Spring Boot Actuator, Prometheus, and Grafana to track metrics like request duration and thread utilisation. Conduct load tests with tools like JMeter to identify bottlenecks.

Improving Code and Database Performance

Handling high-traffic applications means addressing slow I/O and inefficient code first. Threads blocked by slow I/O operations – like database queries or external API calls – can create bottlenecks long before your system hits CPU or memory limits. As Karuna once said, “Never let slow I/O run on request threads.”

Asynchronous Programming and Non-Blocking Operations

The traditional thread-per-request model can run into trouble when threads spend more time waiting for I/O than doing actual work. This often leads to thread exhaustion, even when CPU usage remains low. A smarter approach involves offloading heavy operations to dedicated thread pools, which can be achieved in Spring using the @Async annotation.

For Java 21 and later, enabling virtual threads (spring.threads.virtual.enabled=true) transforms concurrency. With virtual threads, a 4-core machine can handle 50,000–100,000 concurrent requests. These lightweight, JVM-managed fibres offer high concurrency without the complexity of reactive programming. To further optimise throughput during external service calls, replace blocking HTTP clients like RestTemplate with non-blocking ones such as WebClient. Additionally, DeferredResult can be used to return responses after background tasks are completed.

“Virtual threads make the old-school thread-per-request model actually viable again for real systems.”
– Simplified Learning

However, virtual threads don’t magically increase the number of physical database connections. You’ll still need to carefully size your HikariCP connection pool to avoid overwhelming the database with a flood of simultaneous queries.

Database Query Optimisation

Database interactions are often the main culprit when performance issues arise. Problems like the N+1 query issue can cripple performance. To address this, use JOIN FETCH in JPQL or Spring Data JPA’s @EntityGraph to retrieve parent and child entities in a single query.

When you only need specific fields, avoid fetching entire entities. DTO projections – whether interface- or class-based – help reduce memory usage and network overhead by fetching only the necessary data. For bulk operations, set hibernate.jdbc.batch_size to a value between 20 and 50, and enable both order_inserts and order_updates to combine multiple changes into a single database call.

Indexing is another critical step. Ensure columns used in WHERE, JOIN, and ORDER BY clauses are indexed. Use your database’s EXPLAIN ANALYSE command to confirm that queries are using indexes instead of scanning entire tables. For large datasets, implement pagination using Spring’s Pageable interface. For extremely large result sets, keyset pagination (e.g., WHERE id > lastId) often outperforms offset-based pagination.

Another often-overlooked optimisation is disabling Open Session in View (OSIV) by setting spring.jpa.open-in-view to false. This prevents database connections from staying open throughout the entire request-response cycle, which is especially important under heavy loads.

“Your app is only as fast as your slowest query.”
– Lakshu

The results of these efforts can be dramatic. For instance, JPA performance tuning has been shown to reduce query counts by up to 97%. In one case, queries dropped from 847 to just 23 per page load, while P95 response times improved from 1,200ms to 85ms.

With optimised queries and indexing in place, monitoring tools become essential for sustaining these improvements.

Monitoring and Profiling Performance

After implementing code and database optimisations, continuous monitoring is key to maintaining performance. You can’t improve what you don’t measure. Spring Boot Actuator provides built-in endpoints like /actuator/metrics for CPU and memory usage, /actuator/health for application status, and /actuator/heapdump for memory analysis. Enable these endpoints during development (e.g., management.endpoints.web.exposure.include=*) but restrict access in production to ensure security.

For database performance insights, enable Hibernate statistics by setting spring.jpa.properties.hibernate.generate_statistics=true. This tracks query execution times and connection pool metrics via HikariCP. Pair this with Micrometer to integrate Spring Boot with external monitoring systems like Prometheus, and visualise metrics in real time using Grafana dashboards.

Java Flight Recorder (JFR) is another powerful tool, capturing detailed performance data with minimal overhead. You can analyse this data later using JDK Mission Control. For local development, VisualVM offers real-time monitoring of heap usage, thread activity, and garbage collection via JMX.

To take monitoring further, consider using Aspect-Oriented Programming (AOP) to log execution times for specific API endpoints or service methods automatically.

Using Caching for Faster Responses

Once you’ve fine-tuned your code and database queries, caching is the next step to handle high traffic and improve response times. Database queries often take 200–300ms, while cache lookups are typically completed in under 1ms. That time difference becomes critical when your application is handling thousands of requests per second. At Tucanoo Solutions Ltd, our Spring Boot consultancy focuses heavily on implementing effective caching strategies to prevent database bottlenecks and keep applications running smoothly under heavy loads.

Caching doesn’t just speed things up – it also significantly reduces the strain on your database. For read-heavy workloads, caching can slash database load by more than 80%. For example, when users repeatedly request the same product details, user profiles, or configuration settings, caching eliminates redundant queries. This frees up database connections for operations that truly need them.

However, caching needs careful planning. Without proper strategies, you could encounter problems like stale data, memory exhaustion, or cache stampedes caused by simultaneous refreshes. Picking the right caching solution and implementing it with clear invalidation strategies is key to avoiding these issues.

Selecting the Right Caching Solution

The best caching solution depends on your application’s architecture. For single-instance setups, local caches like Caffeine offer lightning-fast ~1ms latency but are limited to the JVM. On the other hand, distributed caches like Redis are ideal for horizontally scaled applications, providing shared state with a slightly higher latency of ~5–10ms. Combining both in a multi-level caching setup – using Caffeine as L1 and Redis as L2 – can deliver the best of both worlds.

Spring Boot’s cache abstraction makes it easy to switch between providers. Add the necessary dependencies – like spring-boot-starter-cache for caching and spring-boot-starter-data-redis for Redis – enable caching with @EnableCaching, and use annotations like @Cacheable, @CachePut, and @CacheEvict to manage caching behaviour.

Best Practices for Implementing Caches

Effective caching involves more than just adding annotations. Here are some crucial practices to follow:

• Set explicit TTLs and size limits: Never cache indefinitely. Adjust TTLs based on data volatility – 15–30 minutes for session data, 1–2 hours for product catalogues, and 12–24 hours for static configurations.
• Avoid cache pollution: Prevent caching null values using unless = "#result == null". This avoids masking missing records even after they’re added to the database. Similarly, don’t cache JPA entities with lazy-loaded associations. Instead, use DTOs or eager fetching to sidestep deserialisation issues.
• Plan invalidation carefully: Use @CacheEvict to clear stale entries or @CachePut for write-through updates. For local caches in multi-instance setups, leverage Redis pub/sub to broadcast invalidation events across servers.

“Cache reads, not writes.” – Naidukr

To handle cache stampedes, where multiple threads try to refresh an expired entry simultaneously, set sync = true in your @Cacheable annotation. This ensures only one thread fetches data from the database while others wait. Also, avoid calling @Cacheable methods from within the same class, as this bypasses Spring’s proxy mechanism and skips the cache entirely.

Finally, monitor your cache’s performance. Use tools like Spring Boot Actuator and Micrometer to track hit/miss ratios. A low hit rate may mean you’re caching the wrong data or using overly aggressive TTLs. For distributed caches, implement a CacheErrorHandler to ensure your application gracefully falls back to the database if Redis becomes unavailable.

“Redis should enhance your application, not break it when unavailable.” – Nawaz Dhandala, OneUptime

Configuring Thread Pools and Asynchronous Processing

After optimising caching, the next step is to configure thread pools to handle high levels of concurrency effectively. Spring Boot automatically sets up a ThreadPoolTaskExecutor for @Async tasks, but its default configuration often needs adjustment for heavy traffic scenarios. If you’re using Java 21 or later with Spring Boot 3.2+, enabling virtual threads can allow a 4-core machine to manage up to 100,000 concurrent requests efficiently.

The key is to align thread pool settings with the type of workload. For CPU-intensive tasks, like encryption or image processing, smaller pools work best – usually the number of CPU cores plus one – to minimise the overhead caused by context switching. On the other hand, I/O-heavy tasks, such as database queries or API calls, benefit from larger pools (typically 2–3× the number of cores), as threads spend much of their time waiting.

Thread Pool Configuration

Our Spring Boot consultancy team finds that Spring Boot simplifies thread pool configuration through the spring.task.execution.pool namespace, so you don’t need to write custom Java classes. Here are the main parameters to adjust:

• Core pool size: The minimum number of threads that remain alive.
• Max pool size: The maximum number of threads available during peak loads.
• Queue capacity: The number of tasks that can be buffered before creating new threads.
• Keep-alive time: How long idle threads (beyond the core size) stay active.

It’s important to define a rejection policy, such as CallerRunsPolicy, to handle situations where both the queue and pool are full. This forces the calling thread to run the task, applying natural backpressure. Additionally, enable graceful shutdown by setting setWaitForTasksToCompleteOnShutdown(true), ensuring background tasks finish before the application stops. Monitoring thread pools is made easier with Spring Boot Actuator’s /actuator/metrics/executor.* endpoint, which provides real-time data on active threads and queue sizes.

Once thread pools are configured to meet the demands of both CPU- and I/O-bound tasks, you can further enhance scalability by adopting non-blocking I/O with Spring WebFlux.

Using Spring WebFlux for Non-Blocking I/O

For applications requiring non-blocking I/O, Spring WebFlux offers a powerful alternative to the traditional MVC model. The shift to simpler, synchronous models supported by virtual threads is gaining traction, reducing the complexity often associated with reactive programming .

“Virtual threads run on a tiny number of OS threads. They don’t consume memory. They don’t block the OS. They park cheaply.” – Lakshika

In a 2025 production example, enabling virtual threads allowed a microservice to handle 8,200 requests per minute – up from 800 on the same hardware. Another implementation saw CPU usage drop by 60%, with throughput increasing 4.2×. Traditional platform threads typically use around 1 MB of stack memory each, whereas virtual threads require only bytes to kilobytes .

When using virtual threads, it’s crucial to size your database connection pool (e.g., HikariCP) to match your database’s capacity, rather than scaling it to the number of virtual threads. For external API calls, replace the deprecated RestTemplate with WebClient, which avoids blocking threads while waiting for responses.

Scaling Horizontally with Load Balancers

Once you’ve fine-tuned local performance, horizontal scaling becomes a must to handle growing traffic demands. By spreading traffic across multiple Spring Boot instances, you can achieve both fault tolerance and high availability. To make this work seamlessly, ensuring your application is stateless is key. This is where distributed caches like Redis or Hazelcast come into play. They allow session data to be shared across instances, enabling any instance to process any request. This eliminates the need for sticky sessions, which can lead to uneven load distribution and reduce resilience.

“Effective load balancing isn’t optional – it’s the cornerstone that determines whether your microservices will crumble under pressure or scale gracefully when demand spikes.” – Daniel Swift, Java Microservices Architect at Springfuse

Implementing Load Balancing with NGINX or HAProxy

NGINX is a popular choice for high-traffic websites, and for good reason. To start, define an upstream block that lists your Spring Boot instances by their IP addresses or hostnames. The default round-robin method works well when all instances have similar capabilities and workloads. However, if your requests vary in complexity, you might want to use the least connections algorithm (least_conn;), which directs traffic to the instance with the fewest active connections.

If your servers differ in capacity, weighted balancing is a smart option. For example, the directive server service1:8080 weight=3; sends more traffic to a more powerful machine, helping your system absorb traffic spikes more effectively. Alternatively, for session persistence in the absence of distributed caching, the IP hash method ensures a client is consistently routed to the same instance based on their IP address.

To ensure reliability, configure health checks using parameters like max_fails and fail_timeout to stop routing traffic to unresponsive instances. Additionally, offloading SSL termination to the load balancer can reduce the CPU load on your Spring Boot instances while centralising certificate management. Don’t forget to use proxy_set_header directives (e.g., X-Forwarded-For, X-Real-IP, X-Forwarded-Proto) to preserve the original client’s IP and protocol information.

Spring Cloud Integration for Scalability

For internal communication between microservices, Spring Cloud LoadBalancer offers client-side load balancing that integrates seamlessly with service discovery tools like Eureka. Unlike server-side load balancers, the calling service selects which instance to contact, reducing unnecessary network hops and simplifying the infrastructure. This approach has replaced the now-deprecated Netflix Ribbon, offering better flexibility and features.

When paired with Eureka, Spring Cloud LoadBalancer automatically detects new instances as they come online or scale down, removing the need for manual updates. This dynamic discovery is especially useful in containerised environments like Docker or Kubernetes, where instances frequently start and stop based on load. For cloud-based deployments, you can integrate with auto-scaling groups to dynamically adjust the number of instances based on metrics like CPU usage or request rates.

These strategies form a strong foundation for scaling your application horizontally. With these tools in place, you’ll be ready to conduct rigorous load testing to ensure your system holds up under pressure.

Monitoring and Conducting Load Testing

After implementing horizontal scaling strategies, the next step is to ensure your application can handle real-world traffic. This involves robust monitoring and load testing to identify and address bottlenecks before they affect users. By closely observing how your application behaves under stress, you can uncover hidden performance issues and optimise accordingly.

Using Monitoring Tools

Spring Boot Actuator, when combined with Micrometer, allows you to export metrics to tools like Prometheus, which can then be visualised using Grafana dashboards. Prometheus typically scrapes metrics every 15 seconds, but during high-traffic periods, this interval can be reduced to 5 seconds for more detailed insights.

Key metrics to monitor include:

• HTTP request duration: Helps identify slow endpoints.
• JVM memory usage: Detects potential memory leaks.
• Thread utilisation: Highlights risks of saturation.

For quick setup, you can import pre-configured Grafana dashboards (e.g., IDs 10280 or 4701). Enabling the percentiles-histogram for http.server.requests provides a clearer view of latencies at the 95th and 99th percentiles, which are more revealing than simple averages. For instance, one Spring Boot application was optimised to handle 1.2 million requests per second with sub-100ms response times.

“If you can’t measure it, you can’t fix it.” – CodeTalks

Proactive alerting in Prometheus can notify you when specific thresholds are breached, allowing you to address issues before they escalate. Additionally, tracking business-critical metrics – such as order completions or payment transactions – ensures that high traffic doesn’t negatively affect revenue. With these monitoring practices in place, the next logical step is to test your system under sustained load.

Conducting Load Testing

Start by establishing performance baselines using tools like Apache JMeter, Gatling, or k6. Record metrics such as maximum throughput, average response time, and P95/P99 latencies to measure the impact of any changes. Sustained load tests, rather than short bursts, are essential for identifying deeper issues like memory leaks, excessive garbage collection, or thread contention.

During these tests, keep an eye on system-level metrics like CPU usage, heap memory, and database connection pool limits. For example, initial CPU utilisation might spike to 85–95% before optimisation, and thread pool exhaustion could occur even with a database connection pool size of 100 if queries aren’t optimised. Implementing caching can significantly reduce database load, which is critical for managing traffic surges.

“Performance optimisation isn’t about finding one magic bullet; it’s about methodically identifying and addressing bottlenecks across your entire system.” – Jonathan Valencia

For JVM tuning, consider using the Z Garbage Collector (ZGC) (-XX:+UseZGC) for ultra-low latency needs or G1GC for a more balanced approach. Configure HikariCP connection pooling with a maximum pool size that aligns with your expected request volume. While enabling all Actuator endpoints during development is helpful, remember to secure and limit their exposure in production to avoid revealing sensitive operational data.

Conclusion

Scaling Spring Boot effectively requires systematically addressing bottlenecks throughout your system. As Agam Kakkar, Lead Software Engineer, explains:

“Performance optimisation isn’t about finding a magic bullet. It’s about methodically uncovering bottlenecks and knowing which levers to pull”.

Key strategies include improving database queries, setting up caching layers, adjusting thread pools, and using reactive programming when it fits your needs. For instance, Kakkar’s team made notable progress by shifting I/O-heavy services to Spring WebFlux, fine-tuning connection pools, and introducing Redis caching.

Before diving into optimisation, always establish a baseline using profiling tools. Since database issues are responsible for around 80% of Spring Boot performance challenges, focus on refining queries and ensuring proper indexing before scaling horizontally with additional servers.

For organisations seeking further refinement, working with a consultancy can elevate your approach to scalability. Tucanoo Solutions Ltd offers tailored Spring Boot consultancy to enhance these techniques – whether it’s tuning connection pools, refining caching mechanisms, or setting up autoscaling. Their expertise ensures your application can handle high demand while staying cost-efficient and maintainable over time.

FAQs

Should I use virtual threads or WebFlux?

When deciding between virtual threads and WebFlux, it all comes down to what your application requires.

Virtual threads (introduced in Java 21) let you write blocking, imperative code while still achieving high concurrency. This means you can build scalable applications without diving into the complexities of reactive programming. For most modern Spring Boot applications that need to handle a lot of traffic, virtual threads offer a straightforward way to scale.

On the other hand, WebFlux is a reactive, non-blocking framework. It’s designed for extreme scalability and provides precise control over system resources. However, this fine-grained control comes with added complexity, which might not be necessary unless your app demands it.

In many cases, virtual threads strike a balance between simplicity and scalability, making them a great fit for high-traffic apps. But if your project has very specific performance needs or you’re already comfortable with reactive programming, WebFlux might be worth exploring.

What should I cache, and for how long?

When using caching in Spring Boot for high-traffic applications, focus on resource-intensive tasks such as database queries, web service calls, or intricate computations. The duration of your cache should align with how often the data changes: for data that rarely updates, set longer durations (ranging from minutes to hours), while for frequently changing data, opt for shorter durations (seconds to minutes). The key is to strike a balance between keeping data fresh and boosting performance by customising cache expiry settings to suit your application’s specific requirements.

How do I pick the right HikariCP pool size?

To figure out the best HikariCP pool size for your Spring Boot application, you need to assess your application’s workload and traffic patterns. Begin with a pool size that aligns with your application’s concurrency requirements and what your database can handle. If you’re working with a high-traffic application, increasing the maximum pool size might help boost throughput. However, it’s crucial to keep an eye on performance metrics to prevent bottlenecks or overloading your resources. Fine-tune settings like maximum-pool-size and minimum-idle to achieve the best performance for your specific needs.