Terraform

How to Manage AWS Lambda Functions with Terraform: Tutorial

Managing AWS Lambda with Terraform

Serverless applications are becoming popular among the DevOps community because they do not require application server infrastructure. AWS Lambda is an event-driven offering from AWS that can help you run any application without the need for application servers.

In this blog post, we will discuss AWS Lambda and how to use Terraform to manage it. In the tutorial, we will start with the basics of the AWS Lambda function and then deploy a very simple Python app using it.

What we will cover:

What is the AWS Lambda function in Terraform?

AWS Lambda is a serverless compute service that executes code in response to events without the need for provisioning or managing servers. Although you can create and run Lambda functions from the AWS web console, Terraform offers a great solution for managing Lambda functions through Infrastructure as Code (IaC). 

In Terraform, an AWS Lambda function is defined using the aws_lambda_function resource. This resource specifies key details such as the function’s name, runtime environment (e.g., Python, Node.js), handler method, and source code location (either inline or from an S3 bucket). Additionally, configurations for memory, timeout, environment variables, and permissions can also be set up within this resource.

For example, deploying a Python script to AWS Lambda with Terraform requires the following steps:

  1. Install Terraform
  2. Configure the AWS account
  3. Set up the IAM roles and policies
  4. Write the Python application
  5. Create a ZIP file of the Python application
  6. Add the aws_lambda_function
  7. Run and execute Terraform main.tf
  8. Verify the Lambda function from the AWS console

What is AWS Lambda used for?

AWS Lambda function is very flexible and supports various use cases:

  1. Event-driven applications — AWS Lambda can be triggered from a number of different event sources. Examples include triggering when a database row is updated, and triggering when the message count in a queue is greater than a certain number.
  2. Web and mobile backends — For a Mobile touchscreen, GUI event can be integrated with AWS Lambda using AWS API Gateway to trigger the Lambda function by calling the microservice-http-endpoint. (Learn how to create API Gateway using Terraform)
  3. Machine learning and data processing — As AWS Lambda supports Python, it is well suited for machine learning and data analytics applications. ML and Data Analytics applications rely heavily on Python and its libraries (Numpy, TensorFlow, MatPlotlib, Scipy, etc.).

Terraform Lambda resources

Let’s take a deeper look at Terraform Lambda resources and how they provision Lambda functions and other resources.

  • aws_lambda_function — The aws_lambda_function resource creates an AWS lambda function inside your AWS account. You can provide the code via an S3 bucket or local ZIP file and set configuration details such as the handler, the runtime, and other parameters as well.
resource "aws_lambda_function" "this" {
 filename      = "lambda_function.zip"
 function_name = "example_lambda"
 role          = aws_iam_role.lambda_role.arn
 handler       = "index.handler"
 runtime       = "nodejs14.x"
}
  • aws_lambda_alias By using the aws_lambda_alias resource in Terraform, you can set up an alias for your AWS lambda function. With this alias, you can point to a specific version of your lambda function, making it easier to use different versions of your functions.
resource "aws_lambda_alias" "this" {
 name             = "dev"
 function_name    = aws_lambda_function.this.function_name
 function_version = aws_lambda_function.this.version
}
  • aws_lambda_permission Lambda permissions give other services permissions to invoke your lambda functions.
resource "aws_lambda_permission" "this" {
 statement_id  = "AllowAPIGatewayInvoke"
 action        = "lambda:InvokeFunction"
 function_name = aws_lambda_function.this.function_name
 principal     = "apigateway.amazonaws.com"
}
  • aws_lambda_layer_version A layer version resource creates a lambda layer for your functions which is a zip archive containing shared code or libraries shared by your lambda functions.
resource "aws_lambda_layer_version" "this" {
 filename   = "lambda_layer.zip"
 layer_name = "my_nodejs_layer"

 compatible_runtimes = ["nodejs14.x"]
}
  • aws_api_gateway_rest_api The aws_api_gateway_rest_api creates an API Gateway in AWS. You can define endpoints that invoke Lambda functions, acting as a front door for your applications.
resource "aws_api_gateway_rest_api" "this" {
 name        = "my_api_gateway"
 description = "API Gateway for my nodejs app"
}
  • aws_cloudwatch_log_group A cloud watch log group can be used to monitor, store, and access log files from various AWS services.
resource "aws_cloudwatch_log_group" "lambda_log_group" {
 name              = "/aws/lambda/my_lambda_function"
 retention_in_days = 14
}

Prerequisites

Let’s start with the prerequisites for this managing AWS Lambda function with Terraform tutorial.

  1. You should have Terraform installed onto your machine (see our tutorial here: How to Download and Install Terraform on Windows, Linux & macOS)
  2. You must have an AWS account. 

Before proceeding further, make sure Terraform is installed by running the command – $ terraform -version

$ terraform -version

Now, here’s how to manage AWS Lambda functions:

1. Set up the IAM roles and policies

The first step is to set up an IAM Role for your Lambda function and any policies that it requires. The exact policies required will depend on what your function does and what services it needs to access. Since this article is mainly focused on explaining how to manage Lambda functions via Terraform, we will keep the IAM Roles and Policies very basic.

Create main.tf and add IAM role

Let’s start by creating a main.tf file and inside the main.tf file, creating a role called Spacelift_Test_Lambda_Function_Role.

resource "aws_iam_role" "lambda_role" {
name   = "Spacelift_Test_Lambda_Function_Role"
assume_role_policy = <<EOF
{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Action": "sts:AssumeRole",
     "Principal": {
       "Service": "lambda.amazonaws.com"
     },
     "Effect": "Allow",
     "Sid": ""
   }
 ]
}
EOF
}

Add an IAM policy

After creating the IAM Role, let’s create an IAM Policy to manage the permissions associated with the role. As this is a basic application, we will be assigning the following permissions: 

  1. logs:CreateLogGroup
  2. logs:CreateLogStream
  3. logs:PutLogEvents

Add the following IAM Policy resource block to main.tf:

resource "aws_iam_policy" "iam_policy_for_lambda" {
 
 name         = "aws_iam_policy_for_terraform_aws_lambda_role"
 path         = "/"
 description  = "AWS IAM Policy for managing aws lambda role"
 policy = <<EOF
{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Action": [
       "logs:CreateLogGroup",
       "logs:CreateLogStream",
       "logs:PutLogEvents"
     ],
     "Resource": "arn:aws:logs:*:*:*",
     "Effect": "Allow"
   }
 ]
}
EOF
}

Attach IAM policy to IAM role

Now that we have created an IAM policy and IAM role for the Terraform managed AWS Lambda function, let’s attach both the IAM policy and IAM role to each other:

resource "aws_iam_role_policy_attachment" "attach_iam_policy_to_iam_role" {
 role        = aws_iam_role.lambda_role.name
 policy_arn  = aws_iam_policy.iam_policy_for_lambda.arn
}

2. Write the Python application

Since the IAM Role and IAM policy have now been created, let’s call on the Python application that we will be running on AWS Lambda.

  1. Create a directory python parallel to main.tf.
  2. Inside the directory python, create a file index.py.
  3. Add the following python function to it:
def lambda_handler(event, context):
   message = 'Hello {} !'.format(event['key1'])
   return {
       'message' : message
   }

3. Create a ZIP file of the Python application

As in the previous step, we created a Python file index.py. Now, we need to create a ZIP file because aws_lambda_function needs the code to be stored in a ZIP file before uploading it to AWS.

This ZIP file we are going to upload and submit this to the AWS Lambda function:

data "archive_file" "zip_the_python_code" {
type        = "zip"
source_dir  = "${path.module}/python/"
output_path = "${path.module}/python/hello-python.zip"
}

4. Add the aws_lambda_function function

Alright. Now we have everything (IAM role, IAM policy, Python code) in place. Let’s write down the aws_lambda_function resource block:

resource "aws_lambda_function" "terraform_lambda_func" {
filename                       = "${path.module}/python/hello-python.zip"
function_name                  = "Spacelift_Test_Lambda_Function"
role                           = aws_iam_role.lambda_role.arn
handler                        = "index.lambda_handler"
runtime                        = "python3.8"
depends_on                     = [aws_iam_role_policy_attachment.attach_iam_policy_to_iam_role]
}

Here are a few things you should keep in mind while writing aws_lambda_function resource block:

  • Runtime — You should mention the correction runtime that AWS Lambda will use to run your Lambda function. Currently, AWS Lambda supports Node.js, Python, Java, Ruby, Go, and NET.
  • IAM role — Always mention the correct IAM role that you have created for the Lambda function.
  • Depends_on — Mention the correct IAM policy attachment block where you have attached the IAM role to the IAM policy. This is a sanity check to make sure that the IAM Policy and IAM roles are in place before the lambda function is created.

5. Run and execute Terraform main.tf

Here is the complete main.tf file:

provider "aws" {
  region = "eu-central-1"
}
resource "aws_iam_role" "lambda_role" {
name   = "Spacelift_Test_Lambda_Function_Role"
assume_role_policy = <<EOF
{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Action": "sts:AssumeRole",
     "Principal": {
       "Service": "lambda.amazonaws.com"
     },
     "Effect": "Allow",
     "Sid": ""
   }
 ]
}
EOF
}
resource "aws_iam_policy" "iam_policy_for_lambda" {
 
 name         = "aws_iam_policy_for_terraform_aws_lambda_role"
 path         = "/"
 description  = "AWS IAM Policy for managing aws lambda role"
 policy = <<EOF
{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Action": [
       "logs:CreateLogGroup",
       "logs:CreateLogStream",
       "logs:PutLogEvents"
     ],
     "Resource": "arn:aws:logs:*:*:*",
     "Effect": "Allow"
   }
 ]
}
EOF
}
 
resource "aws_iam_role_policy_attachment" "attach_iam_policy_to_iam_role" {
 role        = aws_iam_role.lambda_role.name
 policy_arn  = aws_iam_policy.iam_policy_for_lambda.arn
}
 
data "archive_file" "zip_the_python_code" {
type        = "zip"
source_dir  = "${path.module}/python/"
output_path = "${path.module}/python/hello-python.zip"
}
 
resource "aws_lambda_function" "terraform_lambda_func" {
filename                       = "${path.module}/python/hello-python.zip"
function_name                  = "Spacelift_Test_Lambda_Function"
role                           = aws_iam_role.lambda_role.arn
handler                        = "index.lambda_handler"
runtime                        = "python3.8"
depends_on                     = [aws_iam_role_policy_attachment.attach_iam_policy_to_iam_role]
}

Let’s run the following commands in sequence:

1. $ terraform init

terraform init

2. $ terraform plan

terraform plan

Here is the remaining output of the terraform plan:

terraform plan output

3. $ terraform apply

terraform apply

Here is the remaining output of the terraform apply:

terraform apply output

6. Verify the Lambda function from the AWS Console

Let’s head over to the AWS Console and do the test run. From the AWS dashboard, you can start by searching Lambda from the search bar:

AWS Console search

Click on Lambda to see the Lambda function we have provisioned using Terraform:

AWS Console Lambda

Click on it, then use the Test button to run the Lambda function:

Testing lambda function

A new window will open where you need to specify the event name. As you can see in the picture below, we have assigned the event name MyCustomMessage and entered a custom message as key1.

MyCustomMessage

Now, click on the Test tab and then on the orange Test Button:

test tab

As you can see, it has returned the message Hello Spacelift, which we have written inside the python file.

test message

Example: Terraform configuration with AWS Lambda function

Here’s another example of how to define a simple AWS Lambda function in Terraform.

For this example, we will use a hello world Lambda function:

def lambda_handler(event, context):
   return {
       'statusCode': 200,
       'body': 'Hello, World!'
   }

I’ve saved this into a file called hello.py.

The first step we will do in our Terraform code is to define our provider and a role for the Lambda function:

provider "aws" {
 region = "eu-west-1"
}

resource "aws_iam_role" "lambda_exec_role" {
 name = "lambda_execution_role"
  assume_role_policy = jsonencode({
   Version = "2012-10-17",
   Statement = [
     {
       Action = "sts:AssumeRole",
       Principal = {
         Service = "lambda.amazonaws.com"
       },
       Effect = "Allow"
     }
   ]
 })
}

resource "aws_iam_role_policy_attachment" "lambda_basic_execution" {
 role       = aws_iam_role.lambda_exec_role.name
 policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

The above role uses an AWSLambdaBasicExecutionRole policy that provides the necessary permissions to execute lambda functions and write logs to AmazonCloudwatch.

Next, we will define an archive_file datasource that creates the lambda archive for us:

data "archive_file" "lambda" {
 type        = "zip"
 source_file = "hello.py"
 output_path = "hello.zip"
}

In the end, we define the lambda function to use the role and the archive we’ve created above and specify the handler of our function in this format: function_name.lambda_handler. In addition, we are creating an aws_cloudwatch_log_group to get the logs of our Lambda execution.

resource "aws_lambda_function" "hello_world_lambda" {
 function_name = "hello_world_lambda"
 role          = aws_iam_role.lambda_exec_role.arn
 handler       = "hello.lambda_handler"
 runtime       = "python3.8"
 filename      = data.archive_file.lambda.output_path
}

resource "aws_cloudwatch_log_group" "lambda_log_group" {
 name              = "/aws/lambda/hello_world_lambda"
 retention_in_days = 14
}

Now, let’s apply the Terraform code:

terraform apply
Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
 OpenTofu will perform the actions described above.
 Only 'yes' will be accepted to approve.

 Enter a value: yes

aws_cloudwatch_log_group.lambda_log_group: Creating...
aws_iam_role.lambda_exec_role: Creating...
aws_cloudwatch_log_group.lambda_log_group: Creation complete after 1s [id=/aws/lambda/hello_world_lambda]
aws_iam_role.lambda_exec_role: Creation complete after 1s [id=lambda_execution_role]
aws_iam_role_policy_attachment.lambda_basic_execution: Creating...
aws_lambda_function.hello_world_lambda: Creating...
aws_iam_role_policy_attachment.lambda_basic_execution: Creation complete after 1s [id=lambda_execution_role-20240909063234551900000001]
aws_lambda_function.hello_world_lambda: Still creating... [10s elapsed]
aws_lambda_function.hello_world_lambda: Creation complete after 15s [id=hello_world_lambda]

We are now ready to invoke the lambda function, so we will do that and save the output in a file called output.txt.

aws lambda invoke --function-name hello_world_lambda output.txt

Let’s check the output:

{"statusCode": 200, "body": "Hello, World!"}

We can also check the log event in CloudWatch:

aws lambda cloudwatch output

AWS Lambda Terraform module

There is an AWS Lambda Terraform module available in the Terraform registry. Let’s use this module to build something similar to what we’ve done in the previous example.

provider "aws" {
 region = "eu-west-1"
}

module "lambda" {
 source        = "terraform-aws-modules/lambda/aws"
 version       = "7.8.1"
 function_name = "hello"
 description   = "My awesome lambda function"
 handler       = "hello.lambda_handler"
 runtime       = "python3.8"

 source_path = "./hello.py"
}

This will create almost the same resources as in our above case. The only difference is that the permissions are slightly different, and the archive is created using a null resource with a local-exec provisioner that runs a Python script.

Let’s run the code and see the output:

Enter a value: yes

module.lambda.local_file.archive_plan[0]: Creating...
module.lambda.local_file.archive_plan[0]: Creation complete after 0s [id=824e417df9f89264292831d39287ca8c0de20fe5]
module.lambda.null_resource.archive[0]: Creating...
module.lambda.null_resource.archive[0]: Provisioning with 'local-exec'...
module.lambda.null_resource.archive[0] (local-exec): Executing: ["python3" ".terraform/modules/lambda/package.py" "build" "--timestamp" "1725865297519310000" "builds/6a45bb0571c326179278cd9a7dafe3a10dc848ba2272c6219165a79678344616.plan.json"]
module.lambda.null_resource.archive[0] (local-exec): zip: creating 'builds/6a45bb0571c326179278cd9a7dafe3a10dc848ba2272c6219165a79678344616.zip' archive
module.lambda.null_resource.archive[0] (local-exec): zip: adding: hello.py
module.lambda.null_resource.archive[0] (local-exec): Created: builds/6a45bb0571c326179278cd9a7dafe3a10dc848ba2272c6219165a79678344616.zip
module.lambda.null_resource.archive[0]: Creation complete after 0s [id=938250342508072512]
module.lambda.aws_cloudwatch_log_group.lambda[0]: Creating...
module.lambda.aws_iam_role.lambda[0]: Creating...
module.lambda.aws_cloudwatch_log_group.lambda[0]: Creation complete after 0s [id=/aws/lambda/hello]
module.lambda.data.aws_iam_policy_document.logs[0]: Reading...
module.lambda.data.aws_iam_policy_document.logs[0]: Read complete after 0s [id=1947533982]
module.lambda.aws_iam_policy.logs[0]: Creating...
module.lambda.aws_iam_policy.logs[0]: Creation complete after 1s [id=arn:aws:iam::094873932114:policy/hello-logs]
module.lambda.aws_iam_role.lambda[0]: Creation complete after 1s [id=hello]
module.lambda.aws_iam_role_policy_attachment.logs[0]: Creating...
module.lambda.aws_iam_role_policy_attachment.logs[0]: Creation complete after 0s [id=hello-20240909070216057400000001]
module.lambda.aws_lambda_function.this[0]: Creating...
module.lambda.aws_lambda_function.this[0]: Still creating... [10s elapsed]
module.lambda.aws_lambda_function.this[0]: Creation complete after 14s [id=hello]

Now we can invoke the function to ensure that everything is working properly:

aws lambda invoke --function-name hello output.txt
{"statusCode": 200, "body": "Hello World!"}

Managing AWS Lambda: Terraform vs CloudFormation

Both Terraform and CloudFormation can manage AWS Lambda functions, but they differ in their approach and flexibility. Terraform being cloud-agnostic, can manage resources across multiple cloud providers, while CloudFormation is specifically designed for AWS. Terraform uses its own HashiCorp Configuration Language (HCL) for defining infrastructure, which many find more flexible and readable than CloudFormation’s JSON or YAML.

 

Terraform is often praised for its simpler syntax and better modularity, while CloudFormation offers native support for AWS services, which can be advantageous for users who require the latest updates and features from AWS. 

Using Spacelift with Terraform and AWS Lambda

For this part, we will create a GitHub repository based on the code we initially built. The example GitHub repository is here.

Now, let’s go to our Spacelift account and create a stack:

terraform lambda example stack details

Add a name to your stack, select a Space, optionally add Labels and a Description, and then click on continue.

terraform lambda example source code

In the Connect to source code, select your VCS provider and the repository containing the code.

terraform lambda example iac tool

In the Choose vendor step you can accept the defaults: use the latest FOSS version of Terraform.

terraform lambda example stack

At this point, the stack is created, but we should also consider how to authenticate to AWS. For that, I will attach a cloud integration. If you don’t know how to create a cloud integration for AWS, check out this guide.

terraform lambda example cloud integration

Add your integration, select both read and write and click on Attach. Then you can click on go to summary and confirm.

Let’s trigger a run on our stack to create the resources:

terraform aws lambda example trigger

After the plan finishes, we can check what will be created in a human-readable format:

terraform aws lambda example preview

Let’s confirm the run and wait for the resources to be created:

terraform lambda example resources

Now, we can invoke the function using a task:

terraform lambda example task

As you can see, the function was executed successfully and the message has been printed.

Key points

This article is meant to familiarize you with using Terraform to manage AWS Lambda functions. Although the article uses Python, you can satisfy your business needs using other programming languages such as GO, Java, Ruby, or .NET. The core concepts of setting AWS Lambda with Terraform remain the same regardless of your language choice.

If you want to elevate your Terraform management, create a free account for Spacelift today or book a demo with one of our engineers.

Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.

Terraform Management Made Easy

Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization and includes many more features.

Start free trial

The Practitioner’s Guide to Scaling Infrastructure as Code

Transform your IaC management to scale

securely, efficiently, and productively

into the future.

ebook global banner
Share your data and download the guide