Caching External API Responses

Why cache external APIs

External API calls are slow (100-500ms network latency), unreliable (the service might be down), and rate-limited (you can only make N calls per minute). Caching reduces all three problems: faster responses, fewer calls, and less risk of hitting rate limits.

The pattern

async function getGoodreadsRating(isbn: string): Promise<number | null> {
  return cacheThrough(`goodreads:${isbn}`, 24 * 60 * 60_000, async () => {
    const response = await fetch(`https://api.goodreads.com/rating?isbn=${isbn}`);
    if (!response.ok) return null;
    const data = await response.json();
    return data.averageRating;
  });
}

The Goodreads rating for a book changes slowly (maybe once a day as new reviews come in). Caching it for 24 hours means one API call per day per book instead of one per request. With 100 requests per day for a popular book, that is 99 fewer API calls.

Stale data as a fallback

When the external API is down, return stale cached data instead of failing:

async function getExternalData(
  key: string,
  fetchFn: () => Promise<any>,
  ttlMs: number,
): Promise<any> {
  const cached = cacheGet<{ value: any; fetchedAt: number }>(key);

  // Fresh cache hit
  if (cached && Date.now() - cached.fetchedAt < ttlMs) {
    return cached.value;
  }

  // Try to fetch fresh data
  try {
    const fresh = await fetchFn();
    cacheSet(key, { value: fresh, fetchedAt: Date.now() }, ttlMs * 10); // Store with extra-long TTL
    return fresh;
  } catch (err) {
    // API is down — return stale cached data if available
    if (cached) {
      console.log(
        `[CACHE] ${key}: API failed, returning stale data (${Math.round((Date.now() - cached.fetchedAt) / 60_000)} min old)`,
      );
      return cached.value;
    }

    // No cache at all — propagate the error
    throw err;
  }
}

The cache stores data with an extra-long TTL (10x the normal TTL). When the API is down, the cache still has the old data and serves it. The user sees slightly stale data instead of an error.

[!NOTE] This is the same cached fallback pattern from the Error Handling course’s Fallbacks and Degradation lesson — now implemented with the caching system from this course. Stale data is better than no data for most read operations.

Rate limit awareness

If the external API has a rate limit (100 calls per hour), cache aggressively to stay under the limit:

const GOODREADS_RATE_LIMIT = 100; // per hour
const CACHE_TTL = Math.ceil(3600 / GOODREADS_RATE_LIMIT) * 1000; // 36 seconds minimum

// With 100 calls/hour limit and 1000 unique books, cache for at least:
// 1000 / 100 = 10 hours to query all books within the rate limit

For APIs with strict rate limits, use longer TTLs and precompute data in background jobs (from the Background Jobs course) instead of caching on first request.

What to cache from external APIs

Cache: Data that changes slowly (ratings, metadata, weather forecasts), reference data (currency exchange rates, country lists), and responses from rate-limited APIs.

Do not cache: Real-time data (stock prices, live scores), authentication tokens from OAuth providers (they expire), and webhook verification responses.

Exercises

Exercise 1: Create a simulated external API (a function with a 500ms delay). Cache its responses. Verify the second call is instant.

Exercise 2: Make the simulated API fail. Verify the stale cached data is returned. Verify the log message indicates stale data.

Exercise 3: Calculate the minimum TTL needed to stay under a rate limit of 60 calls per minute with 200 unique keys.