IO Threads
When an application talks to the outside world—an exchange connection, a WebSocket client, a database, or a file—it spends time waiting for data to arrive or for a write to complete. IO threads are the dedicated threads that sit in the middle of that waiting and keep the data moving.
In a market data service, IO threads are often responsible for reading messages from many connections at once, parsing them, and handing them off for further processing. If those threads are too few, they can become a bottleneck even when the CPU is mostly idle.
The right IO threading model depends on the workload. Some systems use a small number of event-loop threads that multiplex many connections, while others use a pool of threads that each handle a subset of sockets.
Why It Matters
If IO threads fall behind, data arrives late, buffers grow, and downstream components see increased latency and jitter—exactly what you want to avoid in real-time market data.
How many IO threads should a market data application use?
There isn’t one perfect number, because it depends on connection count, message rates, and whether parsing happens on the same threads. A common approach is to start with a small number of IO threads and measure queueing delay, then scale until the IO stage is no longer the limiter. On Linux, you also need to consider how epoll/event loops map to threads.
What is the difference between IO threads and worker threads?
IO threads focus on waiting for and moving bytes (network reads/writes, filesystem operations), while worker threads focus on CPU work (parsing, normalization, aggregation, analytics). In high-throughput systems, separating them helps prevent heavy computation from delaying network reads. The handoff is usually done via queues or ring buffers.
Can too many IO threads hurt performance?
Yes—more threads can mean more context switching, more cache misses, and more lock contention. It can also make latency less predictable when the operating system frequently reschedules threads. The goal is to match the IO model to the actual concurrency you need, not to maximize thread count.
Real-World Example
A service maintains 2,000 WebSocket connections to stream quotes. With only one IO thread, bursts of messages cause the read loop to lag and updates arrive late. After moving to a small event-loop group (for example, 4 IO threads) and keeping parsing on separate workers, the read lag drops and updates become more consistent.
IO threads and Market Data API
If you build a client that consumes CoinAPI’s Market Data API in real time (for example, via WebSocket), IO threads are the part of your application that keep up with the incoming message flow. Getting IO concurrency right helps you process quotes, trades, and order book updates with steadier latency.