Skip to main content
This guide walks through setting up CloudWatch metrics streaming to AppSignal by creating each resource manually in the AWS Console. For an automated alternative, see Stream AWS CloudWatch metrics with CloudFormation. For an overview of CloudWatch metrics in AppSignal, including magic dashboards and metric naming, see AWS CloudWatch metrics. Complete the following steps to send CloudWatch metrics to AppSignal through Amazon Data Firehose:
  1. Set up an S3 bucket for failed deliveries
  2. Set up an IAM role for Data Firehose
  3. Create a Firehose stream
  4. Create a CloudWatch metric stream
Before you start, have the following information ready:
  • Your app’s App-level Push API key. To find it, open the API keys settings page, or navigate to your organization settings and select Dev zone > API keys.
  • The AWS region you want to stream metrics from.
  • AWS permissions to create Firehose streams, CloudWatch metric streams, S3 buckets, and IAM roles.

​
Set up an S3 bucket for failed deliveries

Amazon Data Firehose requires an S3 bucket to store records that fail to deliver. Create an S3 bucket through the AWS Console or CLI before proceeding.

​
Set up an IAM role for Data Firehose

Create an IAM role that allows Data Firehose to write to the S3 bucket. Use the following trust policy:
JSON
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
Assign the following permissions policy to the role as an inline policy.
See Adding IAM identity permissions for instructions on creating inline policies for IAM roles.
JSON
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:AbortMultipartUpload",
        "s3:GetBucketLocation",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::<FIREHOSE_S3_BUCKET>",
        "arn:aws:s3:::<FIREHOSE_S3_BUCKET>/*"
      ]
    }
  ]
}
Replace <FIREHOSE_S3_BUCKET> with the name of the S3 bucket you created in the previous step.

​
Create a Firehose stream

  1. Open the Amazon Data Firehose console and select Create Firehose stream.
  2. Select Direct PUT as the source.
  3. Select HTTP Endpoint as the destination.
  4. Under “Transform records”, leave data transformation turned off. The data must be delivered as-is.
  5. Enter the following URL as the HTTP endpoint URL:
Shell
https://appsignal-endpoint.net/metrics/aws-cloudwatch
  1. Enter your app’s App-level Push API key as the Access key. This is not an AWS access key. It is the key AppSignal uses to identify your app.
  2. Under “Content encoding”, select GZIP.
  3. Under “Buffer hints”, set the buffer size to 1 MB and the buffer interval to 60 seconds for the lowest latency. Higher values reduce cost but increase delivery delay.
  4. Under “Backup settings”, set source record backup to Failed data only and select the S3 bucket you created in the previous steps.
  5. Under “Advanced settings > Service access”, select Choose existing IAM role and choose the IAM role you created in the previous steps.
  6. Select Create Firehose stream.

​
Create a CloudWatch metric stream

  1. In the CloudWatch console, go to Metrics > Streams.
  2. Select Create metric stream.
  3. Select Custom setup with Firehose. Under Select your Amazon Data Firehose stream, choose the Firehose stream you created in the previous step.
  4. Choose which metrics to stream:
    • All metrics streams everything from all AWS services. This is the simplest option, but may increase costs.
    • Selected namespaces lets you include or exclude specific namespaces, such as AWS/EC2, AWS/Lambda, or AWS/RDS. See the magic dashboards page for the services AppSignal supports.
  5. Select OpenTelemetry 1.0 as the output format.
  6. For Service access, let the AWS Console create or update the IAM role for you, or choose an existing role with permission to write to the Firehose stream.
  7. Select Create metric stream.
You can create multiple metric streams if you need to send metrics to different destinations or filter metrics differently for separate use cases.

​
Verify the setup

After completing the setup, allow 5 to 10 minutes for metrics to begin appearing in AppSignal. To verify data is flowing:
  1. Check the Monitoring tab on your Firehose stream in the AWS Console. Look for non-zero Incoming records and HTTP endpoint - Success metrics.
  2. Confirm the metric stream state shows Running in the CloudWatch console under Metrics > Streams.
If no data appears, enable CloudWatch Logs error logging on the Firehose stream and check for delivery errors.
Some AWS metrics, such as S3 daily storage metrics, are reported with timestamps older than two hours and do not appear in metric streams. See the AWS CloudWatch Metric Streams documentation for details on metric reporting frequencies.

​
CloudWatch metrics in AppSignal

After setup, AppSignal receives metrics from any AWS service that emits metrics through CloudWatch. AppSignal automatically creates magic dashboards for supported AWS services when it detects their metrics. These dashboards appear in Dashboards with pre-built charts for the most important metrics of each service. You can also build your own views using custom dashboards. In the Metric field, type aws_ to browse the CloudWatch metrics AppSignal has received. AppSignal normalizes CloudWatch metric names to lowercase with underscores. For example, AWS/Firehose DeliveryToS3.Success becomes aws_firehose_delivery_to_s3_success. It can take a few minutes after setup for the first metrics to appear. If the list is empty, wait a few minutes and refresh. If you need help, contact us.