Notes on AWS's serverless frameworkWed Mar 24 2021
tags: draft programming self study notes public aws webdev imda
(written for work)
On Hui Lim's recommmendation, I'm working through the following courses to familiarise myself with the AWS Serverless framework:
- Get into serverless computing with API Gateway, AWS Lambda and other Amazon Web Services! Zero server config APIs & SPAs, by Maximilian Schwartzmuller
- Learn how to develop reliable and scalable back-end applications effortlessly using Serverless Framework, by Ariel Weinberger
I'm learning a lot from these courses and I don't want to forget this stuff, so I'm writing it down as notes.
Maximilian's course covers the following:
- API Gateway
- AWS Lambda
- AWS Cognito
- AWS Route53
A lot of the course focuses on manual setup of the API Gateway and DynamoDB via the web portal which is of course automatable by AWS's CLI. But going through the UI and doing the manual setup actually helps a lot to understand the "big picture": the structure of the entire AWS ecosystem, the API gateway, and so on.
Ariel's course covers some of the same ground of Maximilian's. I'll be working through it very shortly and writing my notes here too.
API Gateway/AWS Lambda
Alex de Brie's "A Detailed Overview of AWS API Gateway is a great overview, and is the source of the following image:
There are six main steps in the AWS Gateway request-response cycle.
- Method Request: in this step we validate the incoming request to make sure it corresponds to a particular JSON schema. This simplifies the backend/error handling code.
- Integration Request: transforms the client's request into a shape that the server/Lambda function can read.
- Integration: the API gateway sends the request to the backend server/Lambda function, which receives the request and does stuff with it (sends an email, reads/writes a DB, performs a calculation...).
- Integration Response: this replaces the result of the lambda function with a function that the API Gateway can receive.
- Method Response: this returns a response with a proper HTTP response code.
The API Gateway's request-response cycle in detail
Six steps, five if you ignore the actual integration because it's not part of the API Gateway:
- Method Request
- Integration Request
- (actual integration -- lambda function or other backend service)
- Integration Response
- Method Response
The first gatekeeper is authentication: this can use a custom (lambda) authoriser, or use Amazon's Cognito authoriser. More on this later.
The Method Request step validates the incoming request to make sure it adheres to a particular model (has particular fields, etc).
If it does then we move to the Integration Request:
data into a shape we want to use on the action
we're about to trigger.
A transformation can be done using the
Velocity Template Language (VTL)
to add headers, metadata, auth info, and so on.
One good usecase for this is in retrofitting
a new JSON API to an existing older backend.
For example, your server might accept only
while your requests are in
This is where you would translate your incoming
request inoto a format the server can read.
Then of course you move on to the actual integration step, where a backend (or Lambda function of your choice) is run. This Lambda function
Integration response: this is the return of the lambda function, resolved calculation/db look up etc.
Method Response shapes the response received by the client
What does it mean to "configure as proxy resource"?
Means catch-all: catch all requests.
If you catch all incoming requests, you can create your own router
What is "API Gateway CORS"?
CORS: cross origin resource sharing.
Imagine your SPA runs on example.com
and the API runs on api.com.
When the SPA tries to ask for a request from
the browser will prevent this request from being handled,
so api.com needs to send some "preflight headers"
that will convince the browser that it's OK
(but what if api.com is malicious?)
before this cross-domain request will be allowed.
- "Access-Control-Allow-Headers" allow
- "Access-Control-Allow-Origin": all domains may send these requests and therefore we're good to go.
What is "Lambda Proxy Integration"?
use incoming metadata like headers and authentication etc to pass that unfiltered data over to the lambda
This also means that the Lambda function will be responsible for handling the authentication rather than the integration request
How do I see what the integration receives/logs?
Use AWS Cloudwatch.
What are "body mapping templates" and why would we use this?
We can use body mapping templates to transform the result of a lambda function into a more standardised form. Why would we do this? If you have a lambda function that returns a value but your client is expecting a particular response schema, you can use the body mapping template inside the integration response step to transform it.
Similarly in the integration request step we can also use this template language to edit the request body your integration receives. I've given an example above:
What are "variable path parameters"?
Variable path parameters allow you to run the same Lambda function
with different input parameters (path parameters).
For example, GET request to
/all vs GET request to
can be handled with the path parameter on resource
and then in the Integration Request
we can get this path parameter by doing
You can use query parameters too
use them for validation and extract them with templates.
How does AWS Custom authorisation work?
We can create a custom authoriser (also called Lambda authoriser) on the Method Request step.
The custom authoriser is a Lambda function that takes in some information from the incoming request (specifically, an authorisation token). That function will run some code to validate/identify that user, and returns an IAM policy to us + a user ID + optional "context".
If the auth token is missing or invalid the IAM policy will give a rejection via a Gateway Response. A Gateway Response is a "short-circuit" response returned by the API Gateway; it's a response that is returned before the request hits the backend.
If it is a success then this 3-tuple will then be passed it into the next step, the Method Request step. (? Is it the Method Request step ?) "This context will be added to the event object in your backing Lambda function"
API Gateway Lambda authorization workflow
First, the client calls a method on an API Gateway API method, passing a bearer token or request parameters. The API Gateway checks whether a Lambda authorizer is configured for the method. If it is, API Gateway calls the Lambda function.
The Lambda function authenticates the caller by means such as the following:
- Calling out to an OAuth provider to get an OAuth access token.
- Calling out to a SAML provider to get a SAML assertion.
- Generating an IAM policy based on the request parameter values.
- Retrieving credentials from a database.
If the call succeeds, the Lambda function grants access by returning an output object containing at least an IAM policy and a principal identifier.
API Gateway evaluates the policy.
- If access is denied, API Gateway returns a suitable HTTP status code, such as 403 ACCESS_DENIED.
- If access is allowed, API Gateway executes the method. If caching is enabled in the authorizer settings, API Gateway also caches the policy so that the Lambda authorizer function doesn't need to be invoked again.
What do I do if I get an "Unsupported Media Type Error" on the client side?
Make sure to set content-type to
application/json rather than
Things to learn
How do IAM policies work? What happens when the Lambda authoriser returns an IAM policy?
How do CORS headers work?
What's a Swagger definition?
What's the difference between the Integration Response step and the Method Response step?
Integration responses transform the response from the backend lambda function (which can be any value at all) into a response that API Gateway can handle.
Method responses validate the output to the client. You must first create your method repsonses before you can use a given status code in an integration response. (source: Alex de Brie)
You can use a VTL template to transform the response in both the integration response step and the method response step (? is this true?)
What is a Partition Key?
A partition key must be unique.
DynamoDB allows you to use a partition key PLUS a sort key as a primary key, where the combination has to be unique.
Why is it called a partition key? Because these keys are assigned to different partitions. (Are they hashed?)
What is a Global Secondary Index? What's a Local Secondary Index?
What is read and write capacity?
You provision certain read and write capacity units: kilobytes read/written per second. You can also just use on demand.
What's the difference between
scan() returns an array,
getItem returns only one item.
Why do I get an Access Denied Exception?
You haven't set permissions right: you need to give your lambda function the correct DynamoDB policy.
- Create a new
CognitoUserPool. It is this object that will be used behind the scenes.
- Create a new attribute object (standard attribute)
- Create a
- Call the
userPool.signUpmethod with username, password, attributeList, and callback function that gets executed once Cognito is done.
- Cognito should actually send you an email and you should be able to verify.
- Initialise a
AuthenticationDetailsand a callback function.
What is a Cognito User Pool?
Amazon Cognito user pools documentation
A user pool is a user directory in Amazon Cognito.
What's an App Client?
App clients have access to the user pool. Neither user pool ID nor app client ID are secrets.
Should contain username and password
should be initialised with a username and userPool.
CognitoUserSession will give you JWTs: CognitoAccessToken, CognitoIdToken, and CognitoRefreshToken.
What's the difference between CognitoAccessToken, CognitoIdToken, and CognitoRefreshToken?
will actually delete the localstorage tokens
checks if tokens are still valid
Why would you use query string parameters to pass tokens?
Rather than passing it in the body request/headers? It seems like that would be bad because this would allow MITMs to use your token
Why do I receive a empty userId when using Cognito authoriser?
When using Cognito user authoriser,
our existing body mapping template will not get the userId with
Instead, we need to use
to get the userID.
Thanks to Hui Lim for sending me these excellent courses.
- Singh, 2020. Creating an API Gateway Lambda Authorizer
- Alex de Brie's "A Detailed Overview of AWS API Gateway is a great overview, and is the source of the following image:
- de Brie. Lambda custom authorizers.
- Green 2018