Alerting is an important part of any successful DevOps culture and in incident response practice. Most teams setup some sort of alerting system usually in Slack, but sometime you may want alerts when you are away from Slack. One way to get alerts is via SMS in Grafana. Grafana is a very popular monitoring tool for visualizing, alerting and exploring metrics. There are many different options to create alerts but there is no straight forward way to integrate SMS to Grafana, but we can use web hooks to trigger SMS notification. To be able to trigger SMS notification using web hook we will have to create a web hook and use a tool that will be able to send SMS. The tools we will be using is AWS SNS that is a notification service, AWS API Gateway to create a url for our AWS Lambda Function that is going to trigger the SNS.
What you'll need:
- AWS Account
- Grafana (either running on a server or locally, you can use the docker script below to setup Grafana locally )
Setup Grafana with Docker:
If you don't have a Grafana server you can easily setup with docker or docker-compose. (Click here for Docker Installation)
You can use the following docker-compose file
docker-compose.yml
version: '3'
services:
grafana:
image: grafana/grafana:latest
ports:
- "3000:3000"
volumes:
- grafana:/var/lib/grafana
restart: always
volumes:
grafana:
Or just with two simple docker commands
docker volume create grafana
docker run -d -p 3000:3000 --name=grafana -v grafana:/var/lib/grafana grafana/grafana
Username and password for Grafana are by default is admin:admin
Create IAM Role for Lambda
Now lets get started with AWS part. We will start by creating AWS IAM Role for our Lambda Function. This Role should be able to use the call SNS service provided by AWS.
- Go to IAM
- Create a New Role by going to Roles and clicking Create role
- Select Lambda as use case and then continue to policies
- Search for SNS, for this demo select SNSFullAccess
- Create a Tag, it up to you but a good AWS Practice is to add some sort of Tag
- Go to Review, and give it a Role Name and create role





Create the Lambda Function
Now we will create the Lambda function that will trigger SNS. First step we will go into AWS Lambda

The then create a function name, select python as runtime, as seen here I am using python 3.7. For the execution role select the IAM role we just created.

Then create the function and add the following code in to the Lambda online text editor and then click save.
import boto3
import json
CLIENT = boto3.client("sns")
def lambda_handler(json_input, context):
json_dump = json.dumps(json_input)
request = json.loads(json_dump)
message = request["message"]
numbers = ["xxxxxxx"] # Change this to your number
for number in numbers:
response = CLIENT.publish(
PhoneNumber=number,
Message=message,
MessageAttributes={
'AWS.SNS.SMS.SenderID':
{
'DataType': 'String',
'StringValue': 'Grafana'
}
}
)
return response
This python script is written to loop an array of numbers, so you can add more than one number to send an SMS to. Remember to add your own phone number you have to give your country code. Do notice that I am importing boto3 without installing boto3 first, that is because boto3 is already provided on AWS Lambda function for us.
Connect Lambda Function with API Gateway
The final step of setting up the AWS part is to setup API Gateway and then connect it with Lambda function. Now we go into API Gateway in AWS, if you are creating API Gateway for the first time you will a screen like below else you will see list of previous api and the create API button should be on the top right

Select Build REST API the one on the left, the one of the right is for private VPC, which we aren't doing right now. When creating a new a API for the first time AWS likes to give an example but we won't be using the example, instead select New API and give it a API name and then Create API

After selecting Create API you are redirected to a page similar to below. Select Actions, while `\` is selected. Click create method.

Then you are give a another drop, select POST for its Method

Now we have to setup POST. To setup POST we simply, give the name of the lambda function we created earlier. This name is based on the function name we had set. While typing the function name it should show as a option. Then make sure to save this setting.

Now select Actions again and select Deploy API.

Give a API a stage, for this example I have create a new stage with name demo. Then select Deploy.

Finally you are redirect to this page and in the top you should see the api url. This is the webhook we will use in Grafana

Setting SMS alert in Grafana
Now we have finally come to the final step of setting the alarm. Now log into Grafana. Then select Notification Channels as shown below.

If you don't already have any channels it should show as below, and select Add Channel. If you already have other channels then you can select Add Channel which should be on top right.

Then give the channel a name, and select the type to be webhook. Put the API webhook url we had just created and set the HTTP Method to be POST. Then if you followed all the steps correctly you can do a Test and you should get a SMS on your phone.

Conclusion
I hope by now you are able to create SMS Grafana Alerts with AWS SNS, or at least you learned something. One the important take away I can say this shows how you can use different AWS service together. If you have any questions or suggestion you can contact me at hi@sidops.dev