APIs for Journalists: The Basics

1. What is an API?

An API (Application Programming Interface) is a structured way for one computer to ask another for data. Instead of downloading a whole file or scraping a web page, you send a precise request — “give me the officers of company 01234567” — and get back a clean, structured answer. Many public bodies publish APIs so that developers, researchers and journalists can query their data directly and reliably.

The advantage for journalism is precision and repeatability: the same request returns the same shape of data every time, so you can automate it, re-run it as the data updates, and document exactly where a figure came from. It is often cleaner and more defensible than scraping.

2. Endpoints, requests and responses

An endpoint is a web address representing one type of data or action — for example a search endpoint, or one that returns a single company. You send a request to it (most journalism uses the simple GET request, which just asks for data), and the server sends back a response: the data, plus a status code telling you whether it worked. A 200 means success; a 404 means not found; a 429 means you have hit the rate limit.

The response body is almost always JSON— structured text you can turn into rows and columns.

3. Reading the documentation

The documentation is where every API tells you exactly how to use it, and reading it is the real skill. Look for four things: the base URL and available endpoints; the parameters each endpoint accepts; the authentication method; and the rate limits. Good docs include example requests and a sample response you can copy.

Because endpoint paths and field names differ from one API to the next — and change over time — always take them from the current official docs rather than from a tutorial or from memory. This guide deliberately uses placeholder addresses; the real ones live in each provider's documentation.

4. Query parameters

Parameters refine a request. They come after a question mark in the URL, as name=valuepairs joined by ampersands — a search term, a date range, a page number, a limit on how many results to return. The example below (using a placeholder address) searches for a term and asks for twenty results:

https://api.example.gov.uk/v1/search?q=acme&items_per_page=20

Which parameters exist, and what they are called, is defined by each API's documentation. Special characters in a value (spaces, ampersands) must be URL-encoded — a space becomes %20.

5. Authentication and rate limits

Some APIs are open; others require an API keyyou get by registering a free account. The key is attached to each request — often in a header, sometimes as a parameter — and identifies you so the provider can apply its terms and count your usage. Keep keys secret: never paste one into an article, a screenshot or public code.

Rate limits cap how many requests you may send per minute or per day. Respect them: pace loops with a short pause, cache what you have already fetched, and pull big jobs overnight into local storage rather than repeatedly hitting the endpoint.

6. Making a request: browser, curl, Python

The simplest request is a URL pasted into a browser — the JSON appears on screen. To save results and automate, use curl on the command line, attaching a key where required (placeholder shown):

curl -u YOUR_API_KEY: "https://api.example.gov.uk/v1/search?q=acme"

For anything beyond a one-off, Python's requests library is the workhorse:

import requests
r = requests.get(
    "https://api.example.gov.uk/v1/search",
    params={"q": "acme"},
    auth=("YOUR_API_KEY", ""),
)
data = r.json()

7. Parsing the JSON response

A response is JSON: objects (in braces) holding key-value pairs, and arrays (in square brackets) holding lists. A search typically returns a total count plus a list of results you loop through. A simplified, illustrative response might look like this:

{
  "total_results": 2,
  "items": [
    { "title": "ACME LTD", "status": "active" },
    { "title": "ACME TRADING LTD", "status": "dissolved" }
  ]
}

In Python, data["items"] gives you the list; loop over it to pull the fields you need. To analyse in a spreadsheet, flatten the list of results into rows — libraries like pandas do this in a line. See our pandas guide.

8. Pagination

APIs rarely return everything at once. Instead they paginate— giving you a page of results plus a way to ask for the next. The pattern varies: some use a page number and page size, others an offset (start index), others a token pointing to the next page. The response usually reports the total count so you know how many pages exist.

To collect a full dataset, loop: fetch a page, store the results, increment the page or offset, and stop when you have gathered the total. Add a short pause between requests to stay inside the rate limit, and save as you go so a failure halfway through does not lose everything.

9. Real UK data APIs worth knowing

These public APIs are staples of UK data journalism. Consult each one's official documentation for the current endpoints and fields:

• ›Companies House API — search companies and retrieve profiles, officers and filing history; free key required. Documentation at developer.company-information.service.gov.uk.
• ›TfL Unified API — London transport data such as lines, stops, live arrivals and disruptions; free app key. Base at api.tfl.gov.uk.
• ›Nomis — official labour-market and census statistics from the ONS, with an API for large structured queries; at nomisweb.co.uk.
• ›UK Parliament APIs — data on MPs and members, votes, and written questions, via the UK Parliament developer services at developer.parliament.uk.
• ›police.uk data API — street-level crimes, outcomes and stop-and-search by area and month; no key required, at data.police.uk.

10. Terms of service and ethics

Registering for an API means agreeing to its terms of service. Read them. They may require attribution, restrict bulk redistribution, or limit how the data can be used — and a public API is not a waiver of data-protection law, so personal data pulled through one is still personal data you must handle lawfully.

• ⚠Credit the source and keep a note of what you pulled, with the date, for verification.
• ⚠Stay within the rate limits and do not attempt to bypass them.
• ⚠Never publish your API key, in code, screenshots or article text.
• ⚠Check whether the terms restrict identifying individuals or redistributing the raw data.

11. Reading status codes and errors

Every response carries a status code that tells you what happened before you even look at the body. Learning the common ones saves hours of confusion:

• ›200 OK — success; the data is in the response body.
• ›400 Bad Request — your parameters are malformed; check spelling and encoding.
• ›401 Unauthorized or 403 Forbidden — a missing, wrong or unauthorised API key.
• ›404 Not Found — the endpoint or record does not exist; re-check the path against the docs.
• ›429 Too Many Requests — you have hit the rate limit; slow down and try again later.
• ›500 Server Error — the fault is at their end; wait and retry, and report it if it persists.

In a script, check the status code before parsing, and log any request that fails so you can see exactly which call broke and why.

12. API request checklist

• I read the current official documentation for endpoints, fields and limits.
• I registered for a key where required and kept it out of public code.
• I checked the terms of service and any attribution requirement.
• I handled pagination so I collected the full dataset, not just page one.
• I paced my requests and cached results to respect the rate limit.
• I recorded the request and date so the figure can be verified later.

Frequently asked questions