Working with cryptocurrency data can be a real headache sometimes. Whether you're backtesting a trading strategy, conducting market research, or trying to make sense of historical trends, getting reliable data is often one of the biggest challenges.
That's where CoinAPI Flat Files S3 API comes in.
Flat Files provides bulk historical cryptocurrency market data through S3-compatible access, giving developers, researchers, and quantitative teams a familiar way to retrieve large datasets without maintaining their own historical market-data archive.
Breaking Down the Basics
In simple terms, Flat Files S3 API is a way to access bulk historical crypto market data. If you've ever worked with Amazon S3, you'll feel right at home, as it's designed to work with familiar S3-compatible tools and workflows.
Think of it as a massive historical data library. Want to know how Bitcoin traded last month? Looking for detailed historical order book data? Need years of trades for backtesting?
The data is organized into predictable S3-style paths, making it easier to list, filter, and download the datasets you need.
CoinAPI Flat Files is designed for large-scale historical datasets across hundreds of exchanges and hundreds of thousands of symbols.
What's Available via Flat Files?
Here's what you can access:
- Trades: Historical trade records from supported exchanges and symbols.
- Quotes: Historical bid and ask quote updates from supported venues.
- Order Books: Historical order book datasets, including full-depth updates and snapshot-style datasets where supported.
- OHLCV Data: Historical open, high, low, close, and volume data across supported aggregation periods.
For more detailed order book research, current datasets include limitbook_full, limitbook_snapshot_50, and limitbook_depth_bands.
Files are organized using predictable paths based on factors such as dataset, date, exchange, symbol, and timeframe, depending on the data type.
The Real Cost of Data
Flat Files pricing has changed from the older per-request and flat per-GB model.
Today, pricing depends on the data type, amount of data transferred, Usage Credits, and the plan you use.
Different datasets, including order books, trades, quotes, and OHLCV… have their own usage tiers.
Pay As You Go is available, while Committed plans provide lower Usage Credit pricing for teams with larger or more predictable workloads. Enterprise pricing is available for organizations with additional requirements.
Because pricing can vary significantly depending on what and how much you download, it's better to check the current Flat Files pricing before starting a large historical download.
Pro tip: Before you start downloading everything in sight, think about what you actually need. Start with a specific exchange, trading pair, dataset, and date range. Once you've confirmed the data fits your project, scale from there.
Making Flat Files S3 API Work For You
Here are some real-world scenarios where Flat Files comes in handy.
Flat Files S3 API for Traders
If you're developing trading strategies, historical data is essential for training and backtesting them.
Instead of maintaining your own archive across multiple exchanges, you can retrieve historical data for the specific pairs, venues, and periods you need.
CoinAPI applies normalization and historical quality processing to provide consistent datasets suitable for backtesting and research.
For live trading bots, however, Flat Files isn't the right product. Use CoinAPI's Market Data API for real-time market data through WebSocket, FIX, or REST.
Flat Files S3 API for Researchers
Working on market analysis or a quantitative research project?
The organization of the data makes it easier to retrieve historical datasets across multiple exchanges and analyze them using consistent CoinAPI schemas.
This reduces the amount of exchange-specific data collection and normalization teams need to manage before they can start their actual research.
Flat Files S3 API for Developers
Here's where the S3 compatibility shines.
You can use familiar tools such as the AWS CLI, AWS SDKs, boto3, or other S3-compatible tools and libraries.
For example, you can list available data using the AWS CLI:
aws --endpoint-url https://s3.flatfiles.coinapi.io s3 ls s3://coinapi/
Your CoinAPI Flat Files API key is used as the access key, with the S3-compatible authentication configuration described in the Flat Files documentation.
If you prefer a graphical interface, compatible S3 clients can also be used.
No need to build a completely new data-download system just to start working with historical crypto datasets.
Avoiding Common Pitfalls
Here are some tips to save you headaches while working with the S3 API:
- Start Small: Don't try to download years of data before testing your setup. Start with one dataset, exchange, symbol, and date range.
- Cache Smart: If you're repeatedly accessing the same historical data, store it locally instead of downloading it again.
- Use HTTPS: Use the secure HTTPS endpoint in production.
- Monitor Your Usage: Keep an eye on your Usage Credit consumption and data transfers, especially when working with large order book datasets.
- Avoid Unnecessary LIST Requests: If you already know the exact paths you need, you may be able to download them directly rather than repeatedly listing large directories.
- Plan Large Downloads: Historical order book and trade datasets can become very large. Estimate the amount of data you need before starting a full historical backfill.
How to Get Access to Flat Files
Getting started is straightforward:
- Create a CoinAPI account through the API BRICKS Console.
- Go to Billing → Add Usage Credits and make sure your account has sufficient credits.
- Create a Standard API Key and enable access to the Flat Files API.
- Configure your S3-compatible client or SDK with the CoinAPI Flat Files endpoint and credentials.
- Start listing and downloading the historical files you need.
Usage Credits are consumed based on API activity and data transfer according to the current Flat Files pricing model.
New users can also receive $25 in free credits after payment verification and API key creation.
Start Working With Historical Crypto Data
Flat Files combines normalized historical crypto datasets with familiar S3-compatible tooling and scalable bulk access.
Start with the exchange, symbols, data type, and historical period you actually need. Test the dataset with a smaller download, then scale your pipeline as your requirements grow.
New users can start with $25 in free credits through the API BRICKS Console.
