Production Case Study: Amazon Style Notification Platform

Now connect the full system. Imagine an ecommerce platform where a customer buys a product, payment succeeds, warehouse packs it, courier picks it up, and the package is delivered.

Business Journey

Customer
  |
  v
Adds product
  |
  v
Checkout
  |
  v
Payment
  |
  v
Warehouse
  |
  v
Courier
  |
  v
Delivered

Every step can create notifications for different people.

Event Map

Step              Event                   Receivers          Channels
Checkout started  CheckoutStarted         customer           in-app, push
Payment success   PaymentSuccess          customer, finance  email, in-app
Order placed      OrderPlaced             customer, seller   email, push
Pick list ready   WarehousePickRequested  warehouse staff    internal app
Package shipped   PackageShipped          customer           push, SMS optional
Out for delivery  OutForDelivery          customer           push, SMS
Delivered         PackageDelivered        customer, seller   push, email digest

Complete Architecture

Product Services
  |
  v
Outbox Tables
  |
  v
Event Broker
  |
  v
Notification Service
  |
  +--> PostgreSQL
  |      - notifications
  |      - attempts
  |      - preferences
  |      - templates
  |
  +--> Redis Cache
  |      - preferences
  |      - templates
  |      - rate limits
  |
  +--> Queues
         - security-critical
         - transactional-email
         - transactional-push
         - sms-urgent
         - marketing-bulk
         - dlq
             |
             v
        Worker Fleet
             |
    +--------+---------+--------+
    |        |         |        |
    v        v         v        v
  Email     SMS       Push    In-app

Checkout Example

When payment succeeds, the payment service writes the payment row and outbox event in one transaction.

BEGIN;

UPDATE payments
SET status = 'success'
WHERE id = 'pay_123';

INSERT INTO outbox_events (id, event_type, payload, status)
VALUES (
  'evt_payment_123',
  'PaymentSuccess',
  '{"paymentId":"pay_123","orderId":"ord_123","userId":"user_123"}',
  'pending'
);

COMMIT;

The outbox relay publishes PaymentSuccess. Notification Service consumes it, checks preferences, creates notification rows, and enqueues jobs.

Channel Decisions

Payment success:

Email: receipt and invoice
In-app: account activity
Push: optional confirmation
SMS: not needed unless risk or regulation requires

Out for delivery:

Push: fast and cheap
SMS: useful when courier is close or app is not installed
Email: too slow for this moment
In-app: useful for tracking page

Security alert:

Email: durable record
Push: immediate
SMS: fallback for high risk
In-app: account audit trail

Queue Routing

function selectQueue(notification: NotificationJob) {
  if (notification.category === "security") return "security-critical";
  if (notification.channel === "sms") return "sms-urgent";
  if (notification.channel === "email") return "transactional-email";
  if (notification.channel === "push") return "transactional-push";
  return "marketing-bulk";
}

This function protects urgent notifications from bulk traffic.

Retry and DLQ

Failure                    Retry?  Action
Provider timeout           yes     exponential backoff
Provider 429               yes     delayed retry with rate limit
Invalid email              no      mark failed
Missing template variable  no      DLQ and alert
Expired push token         no      disable token

Monitoring

For this case study, alerts should cover:

payment email p95 latency > 60 seconds
out-for-delivery push p95 latency > 10 seconds
sms spend above daily budget
security queue age > 30 seconds
transactional DLQ count > 0
provider failure rate > 10 percent

Production Walkthrough

1. Customer pays for order.
2. Payment service commits payment and outbox event.
3. Event broker receives PaymentSuccess.
4. Notification Service creates email and in-app notifications.
5. Email job enters transactional email queue.
6. Worker renders invoice email template.
7. Provider accepts email and returns message ID.
8. Delivery attempt is marked accepted.
9. Provider webhook later marks delivered or bounced.
10. Dashboard shows status to support.

Common Mistakes

1. One queue for all ecommerce notifications.
2. No outbox around payment events.
3. Treating provider accepted as final delivery.
4. Sending SMS for every step and creating cost waste.
5. No support-facing status trail.

Interview Questions

1. Walk through order confirmation from event to email delivery.
2. Which notifications should use SMS in ecommerce?
3. How would you prevent duplicate invoice emails?
4. What metrics prove the system is healthy?

Exercise

Extend this design for returns and refunds. Define events, receivers, channels, queues, retries, and monitoring alerts.