I built a PDF extraction API for a real project and ended up learning more about PDF internals than I expected. Here’s a practical breakdown.
Open any PDF with tabular data. It looks organized, right? Rows, columns, headers.
Now look at what’s actually in the file:
draw "Product" at position (50, 100)
draw "Price" at position (200, 100)
draw "Widget" at position (50, 120)
draw "$99" at position (200, 120)
There’s no table structure. No rows. No columns. Just text floating at coordinates.
Your job is to reconstruct the logical structure from spatial positions.
For simple text extraction, PyMuPDF (also called fitz) is fast and reliable:
import fitz
def extract_text(pdf_path):
doc = fitz.open(pdf_path)
text = ""
for page in doc:
text += page.get_text()
return text
Pros: Fast, handles most PDFs
Cons: Tables come out as jumbled text
Output from a table:
Product Price Quantity
Widget $99 10
Gadget $149 5
Not useful if you need structured data.
pdfplumber is specifically designed for table extraction:
import pdfplumber
def extract_tables(pdf_path):
tables = []
with pdfplumber.open(pdf_path) as pdf:
for page in pdf.pages:
page_tables = page.extract_tables()
tables.extend(page_tables)
return tables
Pros: Detects table boundaries automatically
Cons: Struggles with complex layouts, merged cells
Output:
[
[['Product', 'Price', 'Quantity'],
['Widget', '$99', '10'],
['Gadget', '$149', '5']]
]
Much better! But still needs post-processing.
The best results come from combining approaches:
import fitz
import pdfplumber
def smart_extract(pdf_path):
# First, check if PDF has selectable text
doc = fitz.open(pdf_path)
first_page_text = doc[0].get_text().strip()
if len(first_page_text) < 50:
# Likely a scanned PDF - needs OCR
return {"error": "Scanned PDF detected, OCR required"}
# Extract tables with pdfplumber
tables = []
with pdfplumber.open(pdf_path) as pdf:
for i, page in enumerate(pdf.pages):
for table in page.extract_tables():
if table and len(table) > 1:
headers = table[0]
rows = table[1:]
tables.append({
"page": i + 1,
"headers": headers,
"rows": rows
})
# Extract remaining text with PyMuPDF
full_text = ""
for page in doc:
full_text += page.get_text()
return {
"tables": tables,
"text": full_text,
"page_count": len(doc)
}
Is this one table or two?
Name | Email
---------|------------------
John | john@example.com
Department | Budget
-----------|--------
Sales | $50,000
Humans see two tables. Algorithms often merge them.
Some invoices put totals at the bottom. Some have headers on the left side. Some have no headers at all.
When a table spans pages, you need to:
“$1,234.56” vs “1.234,56 EUR” vs “JPY 1234”
Different locales, different formats. Don’t assume.
After building all this myself, I packaged it into an API so others don’t have to:
curl -X POST "https://pdfpull-895295000838.europe-west1.run.app/api/v1/extract/tables" \
-H "X-API-Key: sk_demo_123456789" \
-F "file=@invoice.pdf"
Response:
{
"tables": [
{
"page_number": 1,
"headers": ["Product", "Price", "Qty"],
"rows": [
["Widget", "$99", "10"],
["Gadget", "$149", "5"]
]
}
],
"table_count": 1
}
It also has smart parsers for invoices and resumes that extract specific fields:
curl -X POST "https://pdfpull-895295000838.europe-west1.run.app/api/v1/parse/invoice" \
-H "X-API-Key: sk_demo_123456789" \
-F "file=@invoice.pdf"
{
"vendor_name": "ACME Corporation",
"invoice_number": "INV-2024-0042",
"invoice_date": "January 15, 2024",
"total_amount": 1250.00,
"currency": "USD",
"line_items": [
{"description": "Widget", "quantity": 10, "amount": 990.00},
{"description": "Gadget", "quantity": 5, "amount": 260.00}
],
"confidence": 0.91
}
PDF extraction is harder than it looks. If you’re building something that only occasionally needs PDF parsing, use a library. If you’re doing it at scale, consider an API that handles the edge cases for you.
If you need consistent results across lots of PDFs, an API can save you time. For one-off jobs, a library is usually enough.
Building this in public. Follow along on Twitter: @uppnrise
]]>That conversation stuck with me. So here’s my attempt to explain the core ML concepts the way I wish someone had explained them to me when I started.
Imagine you’re back in school, studying for an exam.
Supervised learning is like studying with a really good answer key. You look at a practice problem, try to solve it, then check the answer. Got it wrong? You adjust your thinking. Over time, you start recognizing patterns—”Oh, whenever I see X, the answer is usually Y.”
That’s exactly how supervised learning works. You feed the algorithm examples where you already know the right answer (we call these “labels”). The algorithm learns the patterns and eventually can make predictions on new, unseen data.
Examples you’ve probably used:
Unsupervised learning is more like being dropped into a library with no guidance. Nobody tells you what’s important—you just start noticing things yourself. “Hey, these books all have similar covers. And these ones seem to be about the same topics.”
The algorithm finds patterns and groupings without being told what to look for.
Real-world examples:
Here’s the key difference I tell people: supervised learning is like learning with a teacher, unsupervised learning is like being a detective.
This one is my favorite to explain because once it clicks, you never forget it.
Imagine you’re dropped somewhere on a hilly landscape, blindfolded. Your goal? Get to the lowest point in the valley. You can’t see anything, but you can feel the slope under your feet.
What would you do?
You’d probably:
That’s gradient descent. Seriously, that’s it.
In ML terms:
The learning rate (we’ll talk more about this later) is basically how big your steps are. Too big? You might overshoot the bottom and climb up the other side. Too small? You’ll take forever to get anywhere.
I sometimes picture a marble rolling down a bowl. It naturally finds the bottom—that’s the intuition behind gradient descent.
Think about tuning the temperature in your shower. You start too cold, so you turn it up. Now it’s too hot, so you turn it down a bit. You keep adjusting until it feels just right.
That’s gradient descent in action—you’re making small adjustments based on feedback (too hot/too cold) until you minimize your discomfort. The “gradient” is the feedback telling you which way to turn.
Here’s a concept that confused me for the longest time because nobody explained why we need it.
A loss function (sometimes called objective function or cost function—same thing, different names) answers one simple question: How wrong is my model right now?
Think of it like this. You’re playing darts:
If you hit the bullseye, your loss is zero—perfect prediction. The further away you land, the higher the loss.
Different problems need different loss functions. Missing by 2 inches might matter a lot for brain surgery, but not so much for horseshoes. That’s why we have:
The key insight: gradient descent uses the loss function to figure out which way is “down.” Without a loss function, the algorithm wouldn’t know if it’s getting better or worse.
This one’s subtle but incredibly important.
Imagine a student who memorizes every single word in the textbook for an exam. They can recite any page perfectly. But then the exam asks them to apply the concepts to a new problem they’ve never seen—and they freeze.
That’s overfitting. The model learned the training data too well, including all the noise and random quirks that won’t appear in new data.
Regularization is basically telling your model: “Hey, don’t get too fancy. Keep things simple.”
Here’s my favorite analogy. You know how some people pack for a trip and bring everything? “I might need this umbrella, and these three types of shoes, and this backup phone charger…” Their suitcase weighs 50 kg.
Regularization is like telling them: “Every item has a cost. Only bring what you really need.”
In ML, regularization adds a penalty for complexity. The model can still learn complex patterns, but it has to “pay” for them. This forces it to focus on patterns that genuinely matter.
L1 regularization (Lasso): “Each extra feature costs a flat fee” → tends to eliminate useless features entirely
L2 regularization (Ridge): “Each extra feature costs proportionally to how much you use it” → shrinks feature importance but rarely eliminates
The result? A model that doesn’t memorize—it actually learns.
Here’s something that took me way too long to internalize: We don’t care how well a model performs on data it’s already seen.
Let me say that again because it’s that important.
If I show you 100 photos and ask you to memorize which ones have cats, you could get 100% accuracy on those specific photos. But that doesn’t mean you understand what a cat looks like. Can you recognize a cat in a photo you’ve never seen?
That’s generalization—performing well on new, unseen data.
Everything we do in ML is in service of this goal:
I like to think of it like learning to cook. You don’t want to only make your mom’s exact recipe perfectly. You want to understand cooking well enough to adapt when you’re missing an ingredient or trying a new dish.
A model that can’t generalize is basically useless—no matter how impressive its training accuracy looks.
Okay, this is where it gets practical.
Hyperparameters are settings you choose before training starts. They’re not learned from the data—you have to pick them yourself.
Think of it like baking a cake:
You can have the best ingredients and recipe in the world, but bake at the wrong temperature? Disaster.
Common hyperparameters you’ll encounter:
How do you find good hyperparameters?
Honestly? Trial and error, but systematic.
Here’s what nobody tells beginners: there’s no “correct” answer. Different problems need different settings. It’s genuinely an art as much as a science.
My personal approach: start with defaults from whatever library you’re using, train a baseline model, then tweak one thing at a time. Keep notes on what works. Over time, you develop intuition.
Let me tie everything together with a story.
Imagine you’re teaching someone to play chess:
Supervised learning: You show them famous games with commentary. “This move was good because…” They learn from examples with known outcomes.
Loss function: After each game they play, you give them a score based on their performance.
Gradient descent: They adjust their strategy bit by bit, based on what worked and what didn’t.
Regularization: You tell them not to memorize specific openings, but to understand principles that apply broadly.
Hyperparameters: You decide how many games to play per day, how much feedback to give, how long to think per move—settings that affect learning but aren’t the learning itself.
Generalization: The goal isn’t to replay the famous games perfectly—it’s to beat opponents they’ve never faced before.
That’s machine learning in a nutshell.
I’ve been working with ML for years now, and honestly, these fundamentals still come up every single day. When a model isn’t working, nine times out of ten, it’s one of these basics that’s off.
The fancy stuff—transformers, diffusion models, whatever’s trending on Twitter this week—it’s all built on these foundations. Nail these concepts, and the rest becomes much more approachable.
Got questions? I probably glossed over something that deserves more attention. Drop me a message—I genuinely enjoy these conversations.
This post is part of a series where I try to explain technical concepts without the jargon. If you found this helpful, you might also like my posts on distributed rate limiting.
]]>Here’s what I wish someone had told me back then: these ML models are just pattern-matching techniques inspired by how we naturally solve problems. No magic, just math doing what our brains do every day. Let me show you what I mean.
We all have that one folder on our desktop called “Stuff” or “To Sort”, right? Clustering is basically the algorithm that says “let me handle that mess for you.”
It looks at your 1,000 random songs and groups them without you labeling anything. The upbeat workout tracks end up together, chill piano music forms its own group, rock songs cluster separately. The algorithm just finds patterns in tempo, instruments, and style.
I love this because it’s unsupervised - you don’t have to teach it what “rock” means. It just figures it out. That’s the same tech behind how Netflix groups movies or how stores figure out which products are usually bought together.
First off, terrible name. It sounds like something from a supply chain meeting. But it’s actually super simple: it’s the “Yes or No” machine.
Think about deciding whether to bring an umbrella. You check the clouds (80% coverage), humidity (super high at 95%), temperature, wind speed… Your brain weighs all this and concludes: “Yeah, bring the umbrella.” That’s exactly what logistic regression does, except it gives you a probability like 85%.
Your email spam filter? Logistic regression. Loan approvals? Same thing. It’s everywhere because most real-world problems boil down to binary choices.
Remember those “Choose Your Own Adventure” books from when we were kids? Decision trees are exactly that, just with math.
You’re hungry. Very hungry? If yes, do you have 30 minutes? Yes means cook a meal, no means order takeout. Not very hungry? Want something healthy? And so on. Each answer branches to the next question until you land on a decision.
Hungry?
├─ Starving → Got time?
│ ├─ Yes → Cook
│ └─ No → Order pizza
└─ Meh → Healthy?
├─ Yes → Salad
└─ No → Snacks
Those annoying customer service chatbots use these. So do credit scoring systems. Honestly, they’re kind of like how we naturally make decisions anyway.
This is basically digital peer pressure. You know the “people who bought this also bought that” section on Amazon? That’s this guy.
Here’s the idea: you and I both binge-watched The Office and love Thai food. I just discovered a new podcast and rated it 5 stars. The algorithm figures “hey, we have similar taste, so you’ll probably like this podcast too.”
It’s crowdsourcing recommendations. Spotify’s Discover Weekly? Same concept - finding people with your music taste and suggesting what they’re listening to. Sometimes it’s creepy accurate, sometimes it completely misses (no, Spotify, I don’t want to listen to polka just because I clicked one weird link). But when it works, it works well.
Okay, this is where things get a little sci-fi. Neural networks are loosely inspired by how our brains work, but don’t let that scare you.
Think assembly line. Raw materials go in one end, each station does its thing, finished product comes out the other end. No going backwards.
Input → Hidden layers doing math → Output. That’s it.
These are the OG neural networks. They’re used for basic stuff like reading handwritten zip codes on envelopes or predicting house prices. Nothing fancy, but they work.
Feed-forward networks have the memory of a goldfish—they process one thing and immediately forget it. RNNs are different; they actually remember context.
When you read “Sarah went to the store. She bought eggs. Then she went home,” you know “she” refers to Sarah because you remember the first sentence. RNNs do the same thing - they carry context forward.
This is why your phone’s autocomplete works. It doesn’t just look at the current word, it remembers what you’ve been typing. Voice assistants use these too. The problem? RNNs are kind of forgetful with long sequences. They’re like that friend who remembers the beginning of a story but gets fuzzy on the middle parts.
If you’ve ever wondered how your phone knows which photo has your dog in it, this is the answer.
They look at images in layers, starting small and building up:
It’s honestly similar to how we look at pictures. You don’t immediately see “John” - your brain processes edges, then features, then puts it together. CNNs copy that approach.
Self-driving cars use these to spot pedestrians. Doctors use them to find tumors in X-rays. They’re really good at anything visual.
This is the big one. The reason everyone is freaking out about AI right now? It’s mostly because of these guys.
Transformers are what powers ChatGPT, Google Translate, and basically every modern AI that handles language.
The breakthrough? Attention mechanism. Instead of processing words one-by-one like RNNs, transformers look at everything simultaneously and figure out what’s important.
Imagine you’re at a party with multiple conversations happening. RNNs would eavesdrop on one conversation at a time. Transformers have super-hearing - they catch all conversations at once AND figure out that Person A in the kitchen is finishing Person B’s story from 20 minutes ago.
This “attention” to the whole context is why modern translation is so good. The model sees the entire sentence before deciding how to translate each word. Context matters, and transformers are phenomenal at context.
They’re also why we can have these scary-good AI writing tools now. The compute cost is high, but the results speak for themselves.
It’s easy to get lost in the hype, but at the end of the day, these are just tools. Really smart tools, but tools nonetheless.
Your morning routine probably involves face unlock (CNN), checking if there’s traffic (RNNs predicting patterns), scrolling a personalized news feed (collaborative filtering). During lunch, your spam filter (logistic regression) is working. By evening, Netflix is recommending shows (collaborative filtering + neural networks), and you’re asking Siri something (transformers + RNNs).
The trick is knowing which tool fits which problem. You wouldn’t use a hammer to tighten a screw, right? Same deal here:
I still remember being completely overwhelmed by ML terminology. But once you strip away the jargon, these are just formalized versions of how we already think about problems. The math gets complicated, sure, but the core ideas? They’re intuitive.
If you want to dive deeper, pick one model and build something small with it. Trust me, there’s no better teacher than breaking things and fixing them yourself.
]]>Like many of you, I needed rate limiting across multiple service instances. The typical in-memory solutions don’t work when you have 5 instances behind a load balancer - suddenly your 100 req/min limit becomes 500 req/min.
My first thought: “Just use Redis!” Turns out, that’s where the real complexity begins.
Token bucket is everyone’s first choice, right? Tokens refill at a constant rate, requests consume tokens, simple math.
Here’s where it got interesting: When do you refill?
Naive approach (what I tried first):
// Check on every request - seemed elegant
currentTokens = min(capacity, lastTokens + (now - lastRefill) * refillRate)
Problem: Race conditions everywhere. Two requests at the exact same millisecond? You’re in trouble.
Solution: Lua scripts in Redis for atomic operations. But here’s the catch - Lua scripts have a size limit, and complex refill logic bloats quickly.
-- This runs atomically in Redis
local tokens = redis.call('HGET', KEYS[1], 'tokens')
local last_refill = redis.call('HGET', KEYS[1], 'lastRefill')
-- ... refill calculation ...
redis.call('HSET', KEYS[1], 'tokens', new_tokens)
Question for the community: Is there a better pattern than Lua scripts for distributed atomic operations? I looked at RedLock but it felt too heavy for this use case.
Token bucket has a problem: boundary issues. Fire 100 requests at 11:59:59, then 100 more at 12:00:01, and you’ve “technically” stayed within limits but hammered the system with 200 requests in 2 seconds.
Sliding window fixes this by tracking requests in overlapping time windows. But the memory cost is brutal.
The trade-off I faced:
I went with the hybrid - store counts in 10-second buckets, interpolate between them:
// Simplified concept
double weight = (now - windowStart) / windowSize;
count = pastBucketCount * (1 - weight) + currentBucketCount * weight;
Is this a good compromise? I’m seeing ~5% error margin compared to true sliding window. Worth the 90% memory savings?
Fixed window is the simplest: reset counter every N seconds. Everyone hates it because of boundary issues, but hear me out…
When it’s actually perfect:
I implemented it in ~20 lines of code (vs 200+ for sliding window). The memory usage? Nearly zero - just one counter per key with TTL.
-- That's literally it
local count = redis.call('INCR', KEYS[1])
if count == 1 then
redis.call('EXPIRE', KEYS[1], window_size)
end
return count
Question: Why do we over-engineer rate limiting for internal services? Fixed window + generous limits seems fine for 90% of internal use cases.
Here’s where I realized algorithm choice really matters. We had a service calling a legacy system that could only handle 10 requests/second - not 9, not 11, exactly 10.
Token bucket? Allows bursts. Sliding window? Still has variance. Leaky bucket processes requests at a constant rate, queuing the rest.
The implementation challenge: Simulating a queue in Redis without actual queue semantics.
// Conceptually: when will the bucket leak enough for this request?
nextAvailableTime = max(now, lastLeakTime + (queueSize / leakRate))
if (nextAvailableTime - now) > maxWaitTime {
reject();
}
This is where I’d love opinions: Should a rate limiter handle queuing at all? Or just accept/reject and let the client retry? I implemented both modes, but I’m not sure the queuing mode is worth the complexity.
Real-world scenario that broke my elegant single-algorithm design:
“We need to limit API calls to 1000/hour, BUT also limit bandwidth to 10MB/hour, AND ensure no single user exceeds 100 calls in any 5-minute window for compliance.”
One algorithm can’t handle this. I needed to combine multiple algorithms.
The architecture decision: How do you combine rate limiters?
Option 1: Sequential checks (what I built first)
if (!checkApiCallLimit()) return reject("API calls");
if (!checkBandwidthLimit()) return reject("Bandwidth");
if (!checkComplianceLimit()) return reject("Compliance");
return allow();
Simple, but you’re making 3 Redis calls. Latency stacks.
Option 2: Parallel checks
CompletableFuture<Boolean> apiCheck = checkApiCallLimit();
CompletableFuture<Boolean> bandwidthCheck = checkBandwidthLimit();
CompletableFuture<Boolean> complianceCheck = checkComplianceLimit();
return apiCheck.get() && bandwidthCheck.get() && complianceCheck.get();
Better latency, but now you’re consuming tokens from limits you might not even need to check. If API limit fails, why decrement bandwidth?
Option 3: Smart short-circuit Check cheapest limits first (like fixed window), only proceed if they pass. But which order? Do you hardcode it? Make it configurable?
I went with configurable combination logic:
ALL_MUST_PASS: AND logic, fail-fastWEIGHTED_AVERAGE: Each limit gets a score, combined thresholdHIERARCHICAL: User limits before tenant limits before globalPRIORITY_BASED: High-priority limits checked firstIs this over-engineered? Part of me thinks “just do AND and call it a day.” But the flexibility has been useful in testing.
Let me share the performance journey, because this is where things got interesting.
Initial naive implementation: ~200 req/s per instance After optimizations: 50,000+ req/s per instance
What changed?
First mistake: Creating a new Redis connection per request. Rookie error, but the performance impact was insane.
// Don't do this
Jedis jedis = new Jedis("localhost");
jedis.get(key);
jedis.close(); // Connection overhead killed us
Switched to Lettuce with proper pooling. But then…
The connection pool sizing problem: Too small = bottleneck. Too large = connection overhead. I landed on:
spring.data.redis.lettuce.pool.max-active=32
spring.data.redis.lettuce.pool.max-idle=8
How did you arrive at these numbers? Load testing. But I’m curious how others size their Redis pools.
My first Lua script was 150 lines. It was slow.
Key optimization: Pre-calculate as much as possible in Java, keep Lua minimal.
-- Before: Doing math in Lua
local refill_amount = (now - last_time) * rate / 1000
-- After: Pass pre-calculated value from Java
local refill_amount = ARGV[1] -- Already calculated
Sounds obvious but cut latency by 40%.
Memory leak alert: I wasn’t setting TTLs on rate limit keys. After a week in production, Redis was using 10GB for ~100K active users.
The insight: Most rate limit keys are temporary. User stops hitting your API? That key should expire.
-- Always set expiry
redis.call('EXPIRE', KEYS[1], 3600) -- 1 hour
This one change dropped memory usage by 95%. Sometimes the simple solutions are the best.
When Redis goes down (and it will), what do you do?
Fail-Open (what I chose): Allow all requests
try {
return checkRedisRateLimit(key);
} catch (RedisException e) {
log.error("Redis down, failing open");
return true; // Allow request
}
Reasoning: Rate limiting is protective, not critical. Better to briefly over-serve than to have a Redis outage take down your entire API.
But: This can be abused. If someone knows your Redis is down, they can flood you.
Alternative I considered: In-memory fallback with local limits. Problem: Defeats the “distributed” part. With 5 instances, you 5x your limits again.
What would you do? I’m genuinely torn on this one.
Rate limit checks are in the hot path. Every microsecond matters.
Synchronous (current approach):
if (!rateLimiter.isAllowed(request)) {
return HTTP_429;
}
processRequest(request);
Clean, simple, blocks until decision is made.
Asynchronous alternative:
CompletableFuture<Boolean> limitCheck = rateLimiter.isAllowedAsync(request);
// ... do other work ...
if (!limitCheck.get()) return HTTP_429;
Could overlap with other I/O. But adds complexity.
My take: For rate limiting, clarity > async performance gains. But I’ve seen arguments both ways.
Should clients know their token count?
Current: Server returns remaining tokens
{
"allowed": true,
"tokensRemaining": 42,
"resetTime": "2025-10-23T12:00:00Z"
}
Why: Clients can back off proactively, prevents wasted requests.
Risk: Clients can game the system, optimize their behavior to stay just under limits.
Alternative: Just return true/false. Simpler, but clients have to guess-and-check.
Unit testing a rate limiter is deceptively hard. How do you test time-based logic without Thread.sleep()?
Bad approach:
@Test
public void testRefill() {
rateLimiter.consume(10);
Thread.sleep(1000); // Wait for refill
assertTrue(rateLimiter.hasTokens());
}
Slow tests, flaky on CI, terrible developer experience.
My solution: Inject a clock interface
public class RateLimiter {
private final Clock clock; // Can be mocked
public boolean allow(String key) {
long now = clock.millis();
// ... rate limit logic ...
}
}
// In tests
@Test
public void testRefill() {
MockClock clock = new MockClock();
RateLimiter limiter = new RateLimiter(clock);
limiter.consume(10);
clock.advance(Duration.ofSeconds(1)); // Instant time travel!
assertTrue(limiter.hasTokens());
}
Tests run in milliseconds, fully deterministic.
How do you test distributed behavior without actually running distributed instances?
I ended up using Testcontainers to spin up real Redis instances:
@Testcontainers
public class DistributedRateLimiterTest {
@Container
static GenericContainer redis = new GenericContainer("redis:7-alpine")
.withExposedPorts(6379);
@Test
public void testMultipleInstances() {
// Simulate multiple app instances
RateLimiter instance1 = new RateLimiter(redis.getHost(), redis.getPort());
RateLimiter instance2 = new RateLimiter(redis.getHost(), redis.getPort());
// Both instances should share state
instance1.consume(5);
assertEquals(5, instance2.remainingTokens());
}
}
Slower than unit tests, but caught a bunch of race conditions my mocked tests missed.
Looking back, here’s what I’d change:
I spent weeks optimizing token bucket before realizing 80% of use cases don’t need that precision. Should’ve validated with fixed window first, optimized later.
Added metrics after performance problems appeared. Should’ve had Prometheus integration from the start. You can’t optimize what you don’t measure.
I built ADRs (Architecture Decision Records) halfway through. Should’ve done them upfront. Writing “why we chose X over Y” forces you to think through edge cases.
My config system has three layers: per-key, pattern-based, global defaults. Sounds flexible, but in practice it’s confusing. Simpler might be better.
I’d genuinely love feedback on:
Algorithm choice: Am I overthinking this? Should I have just stuck with token bucket and called it done?
Redis patterns: Are there better patterns than Lua scripts for atomic distributed operations?
Fail-open vs fail-closed: What’s your take? Have you been burned by either approach?
Composite limiting: Over-engineered or actually useful? Do people need multi-dimensional rate limiting?
Client transparency: Should clients know their remaining quota, or is that information they shouldn’t have?
Testing approaches: How do you test distributed systems without the tests becoming a maintenance nightmare?
I open-sourced the whole thing: github.com/uppnrise/distributed-rate-limiter
Tech stack: Java 21, Spring Boot, Redis (Lettuce client), React dashboard for monitoring
Performance: ~50K req/s per instance, P95 < 2ms latency, ~100MB memory for 1M active limits
It’s MIT licensed, so use it however you want. But more importantly, I’d love to discuss the approaches and trade-offs.
Since people usually ask, here’s how you’d use it:
# Simple POST request
curl -X POST http://localhost:8080/api/ratelimit/check \
-H "Content-Type: application/json" \
-d '{"key": "user:123", "tokensRequested": 1}'
# Response
{
"allowed": true,
"tokensRemaining": 42,
"resetTime": "2025-10-23T12:00:00Z"
}
Simple REST API, works from any language. There’s also a React dashboard for visualizing what’s happening in real-time.
This project started as “I need distributed rate limiting” and turned into a deep exploration of distributed systems trade-offs.
The biggest lesson? There’s no “best” rate limiting algorithm. It depends entirely on your use case:
I’d love to hear your experiences with rate limiting:
Drop a comment, and let’s discuss! Or check out the repo and open an issue if you spot something questionable in the implementation.
Built with Java 21, Spring Boot, and a lot of Redis Lua scripts. MIT licensed. Uses Testcontainers for testing because mocking distributed systems is a lie we tell ourselves.
]]>The distributed rate limiter follows a straightforward architecture:
Traditional in-memory rate limiters work well for single-instance applications, but fall short in modern distributed systems. When you scale horizontally with multiple service instances, each instance maintains its own rate limit counters, effectively multiplying your allowed request rate by the number of instances.
Our distributed rate limiter solves this by:
The system implements the token bucket algorithm, which provides:
git clone https://github.com/uppnrise/distributed-rate-limiter.git
cd distributed-rate-limiter
docker-compose up -d
./mvnw spring-boot:run
3. **Verify the installation:**
```bash
curl http://localhost:8080/actuator/health
You should see a response indicating the service is healthy:
{
"status": "UP",
"components": {
"redis": { "status": "UP" },
"rateLimiter": { "status": "UP" }
}
}
The simplest use case is checking if a request should be allowed:
curl -X POST http://localhost:8080/api/ratelimit/check \
-H "Content-Type: application/json" \
-d '{
"key": "user:123",
"tokens": 1
}'
Response:
{
"allowed": true,
"tokensRemaining": 9,
"resetTime": "2025-09-17T10:30:00Z",
"limit": 10
}
The request flow works as follows:
For API services, you can implement per-key rate limiting:
@RestController
public class ApiController {
@Autowired
private RateLimitService rateLimitService;
@GetMapping("/api/data")
public ResponseEntity<?> getData(@RequestHeader("X-API-Key") String apiKey) {
RateLimitRequest request = RateLimitRequest.builder()
.key("api:" + apiKey)
.tokens(1)
.build();
RateLimitResponse response = rateLimitService.checkRateLimit(request);
if (!response.isAllowed()) {
return ResponseEntity.status(429)
.header("X-Rate-Limit-Remaining", "0")
.header("X-Rate-Limit-Reset", response.getResetTime().toString())
.body("Rate limit exceeded");
}
return ResponseEntity.ok()
.header("X-Rate-Limit-Remaining", String.valueOf(response.getTokensRemaining()))
.body(fetchData());
}
}
Implement per-user rate limiting for web applications:
@Component
public class UserRateLimitInterceptor implements HandlerInterceptor {
@Override
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response,
Object handler) {
String userId = getCurrentUserId(request);
RateLimitRequest rateLimitRequest = RateLimitRequest.builder()
.key("user:" + userId)
.tokens(1)
.build();
RateLimitResponse rateLimitResponse = rateLimitService.checkRateLimit(rateLimitRequest);
if (!rateLimitResponse.isAllowed()) {
response.setStatus(429);
response.setHeader("Retry-After", "60");
return false;
}
return true;
}
}
Configure global default limits:
curl -X POST http://localhost:8080/api/ratelimit/config/default \
-H "Content-Type: application/json" \
-d '{
"bucketSize": 100,
"refillRate": 10,
"timeWindowSeconds": 60
}'
Set specific limits for individual keys:
curl -X POST http://localhost:8080/api/ratelimit/config/keys/premium-user:456 \
-H "Content-Type: application/json" \
-d '{
"bucketSize": 1000,
"refillRate": 100,
"timeWindowSeconds": 60
}'
Apply limits based on patterns:
curl -X POST http://localhost:8080/api/ratelimit/config/patterns/api:premium:* \
-H "Content-Type: application/json" \
-d '{
"bucketSize": 500,
"refillRate": 50,
"timeWindowSeconds": 60
}'
This configuration system provides flexible rate limiting:
The system evaluates rate limits in this order:
This hierarchy allows for sophisticated rate limiting strategies while maintaining simple configuration.
Our performance testing shows impressive results:
# Run the built-in benchmark
curl -X POST http://localhost:8080/api/benchmark/run \
-H "Content-Type: application/json" \
-d '{
"duration": 30,
"concurrency": 100,
"requestsPerSecond": 1000
}'
Typical Results:
These numbers demonstrate the system’s efficiency:
The system scales horizontally with these characteristics:
The token bucket algorithm naturally handles traffic bursts:
# Allow burst of 10 requests, then 1 per second
curl -X POST http://localhost:8080/api/ratelimit/config/keys/burst-api \
-H "Content-Type: application/json" \
-d '{
"bucketSize": 10,
"refillRate": 1,
"timeWindowSeconds": 1
}'
Implement multiple limit tiers:
public class HierarchicalRateLimiter {
public boolean checkLimits(String userId, String apiKey) {
// Check user limit
if (!checkUserLimit(userId)) {
return false;
}
// Check API key limit
if (!checkApiKeyLimit(apiKey)) {
return false;
}
// Check global limit
return checkGlobalLimit();
}
}
Implement usage-based pricing with rate limits:
@Service
public class UsageTrackingService {
public void trackUsage(String customerId, int tokens) {
// Record usage for billing
usageRepository.recordUsage(customerId, tokens);
// Apply dynamic rate limit based on plan
CustomerPlan plan = getCustomerPlan(customerId);
applyPlanLimits(customerId, plan);
}
}
The system exposes comprehensive metrics:
# Get system metrics
curl http://localhost:8080/metrics
Key Metrics:
Error rates and types
Essential metrics for production deployment:
Rate Limiting Metrics:
System Performance:
Business Metrics:
The project includes pre-built Grafana dashboards:
# Start monitoring stack
docker-compose -f docker-compose.monitoring.yml up -d
Access Grafana at http://localhost:3000 with the pre-configured dashboards.
Multiple health check endpoints ensure system reliability:
# Application health
curl http://localhost:8080/actuator/health
# Rate limiter specific health
curl http://localhost:8080/api/ratelimit/health
# Redis connectivity
curl http://localhost:8080/actuator/health/redis
Secure your rate limiter with proper authentication:
@Component
public class ApiKeyValidator {
public boolean validateApiKey(String apiKey) {
// Implement your API key validation logic
return apiKeyRepository.isValid(apiKey);
}
}
Implement request signing to prevent abuse:
@Component
public class RequestSigner {
public boolean verifySignature(HttpServletRequest request) {
String signature = request.getHeader("X-Signature");
String payload = getRequestPayload(request);
return hmacSha256(payload, secretKey).equals(signature);
}
}
Follow standard HTTP headers for rate limiting:
public void addRateLimitHeaders(HttpServletResponse response,
RateLimitResponse rateLimitResponse) {
response.setHeader("X-Rate-Limit-Limit", String.valueOf(rateLimitResponse.getLimit()));
response.setHeader("X-Rate-Limit-Remaining", String.valueOf(rateLimitResponse.getTokensRemaining()));
response.setHeader("X-Rate-Limit-Reset", String.valueOf(rateLimitResponse.getResetTime().getEpochSecond()));
}
Use the provided Docker configuration for production:
version: '3.8'
services:
redis:
image: redis:7-alpine
command: redis-server --appendonly yes
volumes:
- redis-data:/data
deploy:
replicas: 3
rate-limiter:
image: distributed-rate-limiter:latest
ports:
- "8080:8080"
environment:
- SPRING_PROFILES_ACTIVE=production
- SPRING_DATA_REDIS_CLUSTER_NODES=redis:6379
deploy:
replicas: 3
Deploy to Kubernetes with the provided manifests:
kubectl apply -f k8s/
The Kubernetes configuration includes:
A typical production deployment includes:
Application Tier:
Data Tier:
Monitoring Tier:
Configure for different environments:
# Production settings
spring.profiles.active=production
spring.data.redis.cluster.nodes=${REDIS_CLUSTER_NODES}
management.endpoint.health.show-details=never
logging.level.dev.bnacar.distributedratelimiter=WARN
# Development settings
spring.profiles.active=development
spring.data.redis.host=localhost
management.endpoint.health.show-details=always
logging.level.dev.bnacar.distributedratelimiter=DEBUG
@Component
public class GracefulRateLimiter {
public boolean checkRateLimit(String key) {
try {
return rateLimitService.isAllowed(key);
} catch (Exception e) {
// Log error and allow request in case of rate limiter failure
log.error("Rate limiter failed, allowing request", e);
return true;
}
}
}
Set TTLs on Redis keys to prevent memory leaks:
-- Lua script with TTL
redis.call('EXPIRE', bucket_key, 3600) -- 1 hour TTL
High Latency
# Check Redis connection
redis-cli ping
# Monitor connection pool
curl http://localhost:8080/actuator/metrics/hikaricp.connections.active
Memory Issues
# Check Redis memory usage
redis-cli info memory
# Monitor application memory
curl http://localhost:8080/actuator/metrics/jvm.memory.used
Configuration Problems
# Validate configuration
curl http://localhost:8080/api/ratelimit/config
# Check logs
docker logs distributed-rate-limiter-app
We welcome contributions! The project follows standard open-source practices:
# Clone and setup
git clone https://github.com/uppnrise/distributed-rate-limiter.git
cd distributed-rate-limiter
# Run tests
./mvnw test
# Run with development profile
./mvnw spring-boot:run -Dspring-boot.run.profiles=development
This distributed rate limiter provides a robust, scalable solution for protecting your APIs and services. With its flexible configuration, comprehensive monitoring, and production-ready design, it’s an excellent choice for modern distributed systems.
Key benefits: