AWS Unpacked #7: Serverless

Categories: AWS

Building blocks of serverless: AWS services that do the heavy lifting

Blog header image
TL;DR
Serverless doesn’t mean there are no servers — it means you don’t have to think about them. No provisioning, no scaling, no patching. You just upload your code, set a few options, and AWS runs it on demand. This approach is ideal for event-driven applications: real-time data processing, APIs, automation, and backend microservices. Serverless lets you focus on what your application needs to do — not how it’s deployed or scaled. In this post, we’ll explore the core AWS services that make up a serverless architecture — and how they all fit together.

What Is Serverless?

Think of serverless as ordering a ride from Uber instead of buying your own car. You don’t worry about fuel, maintenance, or parking. You just say, “I need a ride,” and it shows up.

In AWS, that’s exactly how serverless works. You pay only for what you use, there’s no infrastructure to maintain, and AWS handles scaling for you.

AWS Lambda

Lambda is the core of AWS serverless. It lets you run code without provisioning or managing servers.

You upload your code (in Python, Node.js, Java, etc.) and AWS runs it in response to events like:

  • HTTP requests (via API Gateway)
  • File uploads to S3
  • Database changes in DynamoDB
  • Scheduled cron jobs
  • Custom app triggers

Limits to Know (per region)

  • Timeout: Max 15 minutes per execution.
  • Memory: 128 MB to 10 GB (in 1 MB increments). Note that increasing RAM also improves CPU and network.
  • Ephemeral storage: 512 MB default, up to 10 GB available.
  • Environment variables: 4kB
  • Payload size: Max 6 MB for synchronous requests via API Gateway.
  • Package size: Up to 250 MB unzipped, including dependencies; compressed: 50 MB

Concurrency and Scaling

Lambda automatically scales based on the number of incoming events. By default, you get 1,000 concurrent executions per region (can be increased). If your function exceeds this, requests are throttled.

You can also reserve concurrency for a function or use provisioned concurrency to avoid cold starts.

Pricing

You pay per request and for compute time. Lambda has a generous free tier of 1,000,000 Lambda request and 400,000 GBs of compute time per month.

SnapStart

For Java, Python and .NET functions, Lambda SnapStart improves startup time by initializing the function, taking a snapshot of the memory state, and reusing it for future executions. Great for latency-sensitive Java apps.

Lambda@Edge vs CloudFront Functions

  • Lambda@Edge Functions: Runs Lambda functions closer to the user, inside CloudFront edge locations. Use it for modifying requests/responses or implementing authentication.
  • CloudFront Functions: Lightweight alternative that runs JavaScript code for high-speed, low-latency functions—like URL rewrites or header manipulation.

    Feature CloudFront Functions Lambda@Edge
    Runtime Support JavaScript (only) Node.js, Python
    # of Requests (Scale) Millions per second Thousands per second
    CloudFront Triggers Viewer Request & Viewer Response All 4: Viewer Request, Viewer Response, Origin Request, Origin Response
    Max Execution Time 1 ms 5 sec (Viewer) / 30 sec (Origin)
    Max Memory ~2MB (fixed, no config) Up to 128MB
    Total Package Size <10 KB (code only) Up to 50MB (compressed, incl. deps)
    Network Access No outbound network access Yes (to public internet)
    File System Access No Yes (limited /tmp)
    Access to Request Body No Yes (with some size limits)
    Pricing Cheaper (per request model) More expensive (based on duration + invocations)
    Use cases Cache key normalization, header manipulation, URL rewrites/redirects, request authentication and authorization Code depending on 3rd party libraries, long execution times, file system access, access to body of HTTP requests

Lambda in VPC

You can attach a Lambda function to a VPC if it needs to access private resources like RDS. This lets it talk to your internal systems, but adds startup latency unless you configure it carefully (e.g., using VPC endpoints).

Amazon RDS and Serverless Integration

While RDS itself isn’t fully serverless (except for RDS Aurora Serverless), you can still use Lambda to interact with your database.

Use cases:

  • A user submits a form, and a Lambda function inserts the data into your RDS database.
  • A scheduled Lambda runs reports every night from RDS and emails them.

You can also set up RDS event notifications (like backup completion or DB instance failover) to trigger Lambda for alerting or automation. However, in this case you don’t have any information about the data itself.

Amazon DynamoDB

Amazon DynamoDB is AWS’s fully managed NoSQL database service. It’s built for applications that need consistent, single-digit millisecond latency at any scale. It’s a key part of the AWS serverless ecosystem—there are no servers to manage, no need to worry about infrastructure, and it scales automatically.

What Is NoSQL?

Unlike relational databases (like MySQL or PostgreSQL), NoSQL databases don’t use tables with rigid rows and columns. Instead, DynamoDB is a key-value and document database, which means each item can have flexible attributes and structures.

Core Concepts

Table Structure

  • Each table contains items (similar to rows).
  • Each item is a JSON-like object made up of attributes (similar to columns).
  • You must define a primary key when creating the table:
    • Partition key only (single attribute)
    • Partition key + sort key (composite key)

Example:

{
  "userId": "1234",
  "gameLevel": 42,
  "lastLogin": "2024-05-25",
  "achievements": ["explorer", "collector"]
}

DynamoDB supports data types like String, Number, Boolean, List, Map, Null, and Binary.

Capacity Modes

DynamoDB supports two read/write capacity modes:

  • On-Demand: Pay-per-request pricing. Best for unpredictable workloads.
  • Provisioned: You define read and write capacity units independently of each other (RCUs and WCUs). Ideal for consistent workloads or tight cost control.

You can switch between the two modes as needed.

Limits

  • Item size: max 400KB
  • Max 25 WCUs or RCUs per transaction (by default)
  • Partition key must be present for every item
  • No complex joins or aggregations—those are done client-side or with tools like Athena or Glue

Advanced Features

DynamoDB Streams

DynamoDB Streams is a change data capture (CDC) feature that captures item-level changes in near real-time. Each change creates a stream record that can be processed by Lambda, Kinesis Data Firehose, or other services.

Difference from Kinesis Data Streams:

  • DynamoDB Streams: Tied to one table, used for reacting to item changes (e.g., trigger a Lambda).
  • Kinesis Data Streams: General-purpose stream for ingesting large-scale time-ordered data from multiple sources.

DAX (DynamoDB Accelerator)

  • Fully managed, in-memory cache for DynamoDB.
  • Greatly improves read performance (up to 10x faster).
  • Transparent to application - no need to change application code.
  • Compared to ElastiCache:
    • DAX is purpose-built for DynamoDB (API-compatible).
    • ElastiCache is general-purpose caching (e.g., Redis, Memcached).
    • Use ElastiCache if you need caching for non-DynamoDB data.

Global Tables

  • Provide multi-region replication.
  • Great for apps with a global user base (e.g., a mobile app with users in both the US and Europe).
  • Replicates writes to all participating regions automatically.

Time to Live (TTL)

  • Automatically deletes expired items.
  • You define an attribute (e.g., expiryTime) as a TTL attribute.
  • Items are purged asynchronously.

Use case: Clear out temporary session data after 24 hours.

Conditional Writes & Optimistic Locking

  • Prevents overwriting data accidentally.
  • Conditional writes let you write only if an attribute meets a condition.
  • Optimistic locking uses a version attribute and fails the update if someone else updated it first.

On-Demand Backup

  • Manual, point-in-time backups.
  • Stored until you delete them.

Continuous Backup (PITR – Point-in-Time Recovery)

  • Automatic, rolling 35-day backup window.
  • Enables point-in-time restore within that window.
  • Must be explicitly enabled on each table.

S3 Integration: Export and Import

  • You can export table data to S3 (e.g., for analytics or archival).
  • Requires PITR to be enabled.
  • Exported data is saved in Parquet format, making it easy to analyze in Athena or Redshift Spectrum.
  • You can also import data from S3 using the Import Table feature.
    • It uses data stored in JSON or DynamoDB JSON format.
    • Useful for migrating or restoring archived data.

Real-Life Example

Let’s say you’re building a serverless to-do list app:

  • You store tasks in DynamoDB (userId as partition key).
  • When a task is marked complete, a DynamoDB Stream triggers a Lambda to update user stats.
  • You use DAX to cache frequent reads for mobile clients.
  • If the app expands to multiple regions, Global Tables replicate data to ensure low latency.

Amazon API Gateway

Amazon API Gateway is a fully managed service that allows you to create, publish, maintain, monitor, and secure APIs at any scale. It’s a key component in serverless apps, where it acts as the “front door” for clients (like web or mobile apps) to communicate with your backend (often Lambda functions).

Why Use API Gateway?

You could, in theory, expose a Lambda function directly using an AWS SDK call from the client—but this comes with limitations:

  • Tight coupling: The client would need AWS credentials and know how to call the Lambda directly.
  • Security concerns: You’d need to manage IAM credentials on the client side.
  • Poor API management: You’d miss out on monitoring, throttling, versioning, and request/response formatting.

API Gateway solves all of this:

  • It exposes your backend logic (usually Lambda) as a standard HTTP endpoint.
  • It acts as a secure layer between clients and your backend.
  • It supports advanced features like caching, rate limiting, custom domains, and authorization.

Types of API Gateway Endpoints

API Gateway supports three types of endpoint configurations:

Type Description Use Case Example
Edge-Optimized Requests are routed through Amazon CloudFront globally Public APIs for global users (e.g. mobile app backend)
Regional Served directly from the AWS region where API is deployed Backend APIs for use within a single region
Private Accessible only from within your VPC using VPC endpoints (no public access) Internal microservices within a corporate network

REST vs HTTP vs WebSocket APIs

API Gateway offers different API types:

  • REST APIs: Full-featured, powerful, supports usage plans, API keys, request validation, transformations, etc. Slightly more expensive.
  • HTTP APIs: Lightweight, cheaper, lower latency. Supports most common use cases like routing to Lambda, ALB, and more.
  • WebSocket APIs: For real-time, two-way communication like chat apps or stock tickers.

In most modern projects, HTTP APIs are preferred due to their simplicity and lower cost.

Key Features

Throttling & Rate Limiting

  • Prevents abuse by setting limits on the number of requests per second (RPS).
  • Example: Preventing one user from hammering your API 1000 times per second.

Caching

  • You can enable response caching to improve performance and reduce backend load.
  • Example: Cache the response of a “get product details” API for 60 seconds.

Request/Response Mapping

  • Allows you to transform the incoming request or outgoing response.
  • Useful for adapting to legacy systems or hiding internal backend details.

OpenAPI/Swagger Support

  • You can define your API using OpenAPI specs, import/export them in API Gateway.
  • Handy for generating SDKs or documentation.

Security Options

API Gateway lets you protect your APIs using multiple strategies:

IAM Authorization

  • Used for internal services or trusted clients with AWS credentials.
  • Example: A Lambda inside your VPC calling another internal API.

Amazon Cognito User Pools

  • Managed user directory (sign-up/sign-in) and JWT-based authentication.
  • Great for apps where users log in via username/password or federated identity providers (Google, Facebook, etc.).

Lambda Authorizers (Custom Authorizers)

  • A Lambda function is invoked to authorize each request.
  • You define your own logic (e.g. check a custom token, role, or tenant).
  • Example: In a multi-tenant SaaS app, you can write logic to ensure a user only accesses their own tenant data.

API Keys & Usage Plans

  • You can distribute API keys to consumers and control rate limits and quotas.
  • Good for public APIs you want to monitor and meter.

HTTPS and Custom Domain Names

  • You can map your API Gateway endpoint to a custom domain (e.g. api.mycompany.com) and serve it over HTTPS using an ACM certificate.
  • Helps with branding and also allows better control over TLS/SSL configuration.

Real-Life Example

Imagine you’re building a ride-hailing app:

  • Mobile users send a POST request to https://api.rideapp.com/book-ride.
  • API Gateway accepts and validates the request, then passes it to a Lambda.
  • The Lambda function processes the booking and writes the data to DynamoDB.
  • Cognito ensures the user is authenticated.
  • Rate limiting prevents spam or abuse.

AWS Step Functions

Step Functions help you coordinate a series of Lambda functions or other AWS services into a defined workflow.

It’s like a flowchart with logic. You can:

  • Retry on failure
  • Wait between steps
  • Run branches in parallel
  • Call external services

Real-life example: After a user uploads a video, a Step Function could:

  1. Extract metadata
  2. Transcode the video
  3. Update a database
  4. Send an email when done

Everything is managed visually, and you only pay for transitions (state changes), not time.

Amazon Cognito

Amazon Cognito is a user identity and access management service for your web and mobile applications. It removes the need to build your own authentication system and makes it easy to add user sign-up, sign-in, multi-factor auth, session management, and access control to your apps.

You get secure, scalable authentication—backed by AWS infrastructure—with support for both custom user accounts and federated identity providers (like Google, Facebook, Apple, SAML, etc.).

Key Features

  • User registration & login (hosted UI or SDK/API-based)
  • Multi-factor authentication (MFA)
  • Password policies and account recovery options
  • Federated identity support (OAuth, SAML, OpenID)
  • JWT token issuance (ID, access, refresh tokens)
  • Integration with API Gateway, Lambda, AppSync, and ALB
  • Fine-grained access control using IAM roles per user
  • Sync user data across devices
  • Built-in hosted UI (no need to build your own login page)

Cognito User Pools vs Identity Pools

Cognito has two separate services, and it’s important to understand the difference:

Feature User Pools Identity Pools (Federated Identities)
Purpose User directory & authentication Temporary AWS credentials & authorization to access AWS resources
Manages Users? Yes (email/password, social login, etc.) No, relies on User Pools or external identity providers
Provides AWS Access? No Yes (via IAM roles and STS)
Issues Tokens? Yes (JWT: ID, access, refresh) Yes (temporary AWS credentials via STS)
Main Use Case Signing in users and managing their attributes Letting authenticated users access AWS resources directly
Example A user signs in to your app That user uploads a file to S3 with limited permissions

Real-world example:

  • User Pool: Your fitness app lets users sign up and log in.
  • Identity Pool: After logging in, you allow them to upload their workout videos to a specific S3 bucket.

Often, you use both together: the user authenticates through a User Pool, and then you use that authentication to get AWS credentials from an Identity Pool to access AWS services.

Cognito vs IAM: When to Use What?

Use Case Use Cognito Use IAM
Web/mobile users accessing app resources ✅ (Cognito is designed for this) ❌ (Don’t embed IAM credentials in apps)
Managing end-users
Internal AWS services needing access to each other ✅ IAM Roles/Policies
Temporary access to AWS resources (e.g., S3 uploads) ✅ (via Identity Pools) ✅ (with STS for internal use)

So, Cognito is for end-user authentication and temporary, restricted access, while IAM is for internal users and services.

Integration with API Gateway and ALB

Cognito integrates directly with API Gateway and Application Load Balancer (ALB) as an authorizer.

API Gateway Flow:

  1. User logs in via Cognito (using hosted UI or SDK).
  2. Cognito returns a JWT token (ID/access).
  3. The frontend app includes this token in the Authorization header of API requests.
  4. API Gateway validates the token using the Cognito User Pool authorizer.
  5. If valid, it forwards the request to the backend (e.g., Lambda).

This lets you control access at the API level using token-based rules, without exposing IAM credentials or writing custom logic.

ALB Integration:

  • ALB can authenticate users via Cognito before forwarding traffic to targets (e.g., an ECS container or EC2 instance).
  • Example: A web dashboard that requires login before showing sensitive reports.

Row-Level Security in DynamoDB Using Cognito

A common challenge is limiting users to only their own data in DynamoDB. Cognito and IAM together can help here.

Here’s how:

  1. After login, the user gets a Cognito identity (with a unique sub claim or user ID).
  2. The Identity Pool assigns a temporary IAM role based on that identity.
  3. The IAM policy for that role can include conditions based on the Cognito identity.

Example IAM Policy:

{
  "Effect": "Allow",
  "Action": "dynamodb:GetItem",
  "Resource": "arn:aws:dynamodb:region:account-id:table/UserProfiles",
  "Condition": {
    "StringEquals": {
      "dynamodb:LeadingKeys": "${cognito-identity.amazonaws.com:sub}"
    }
  }
}

This means the user can only read rows from DynamoDB where the partition key equals their user ID.

This is true row-level security, enforced via IAM.

Real-World Architecture Example

Let’s say you’re building a travel booking app:

  • Cognito User Pool handles sign-up/sign-in (users log in with Google or email/password).
  • Cognito Identity Pool gives users temporary AWS credentials.
  • API Gateway exposes endpoints like POST /bookings, and only allows authenticated users.
  • Lambda functions handle business logic and write to DynamoDB.
  • IAM policies restrict users to only read/write their own bookings.

You now have a fully serverless, secure, multi-user app with built-in authentication, authorization, and access control—all without managing a single server.

About the Author

Dawie Loots is a data scientist with a keen interest in using technology to solve real-world problems. He combines data science expertise with his background as a Chartered Accountant and executive leader to help organizations build scalable, value-driven solutions.

Back to Blog