1. Answers
  2. Analyzing AWS ECS Container Logs with Splunk's Docker Logging Driver

How Do I Analyze AWS ECS Container Logs With Splunk's Docker Logging Driver?

Introduction

This guide aims to help you configure an Amazon ECS service to forward its container logs to Splunk using Docker’s logging driver. By following this guide, you will learn how to create an ECS cluster, define a task, set up a service, and configure Docker logging to send logs to Splunk. This setup is crucial for enhancing observability and debugging capabilities by analyzing container logs in Splunk.

Step-by-Step Setup

  1. Define the ECS Cluster: This is where your container instances will run.
  2. Create the Task Definition: This specifies the Docker containers to run, along with their configurations.
  3. Set Up the ECS Service: This maintains the specified number of task instances.
  4. Configure the Docker Logging Driver: Set the logging parameters to forward logs to Splunk.

Here’s an example of how to set this up:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const example = new aws.ecs.Cluster("example", {name: "example-cluster"});
const exampleTaskDefinition = new aws.ecs.TaskDefinition("example", {
    family: "example-task",
    networkMode: "awsvpc",
    requiresCompatibilities: ["FARGATE"],
    cpu: "256",
    memory: "512",
    containerDefinitions: JSON.stringify([{
        name: "example-app",
        image: "nginx:latest",
        essential: true,
        logConfiguration: {
            logDriver: "splunk",
            options: {
                "splunk-token": "your-splunk-token",
                "splunk-url": "https://splunk-server:8088",
                "splunk-index": "main",
                "splunk-insecureskipverify": "true",
            },
        },
    }]),
});
const exampleVpc = new aws.ec2.Vpc("example", {cidrBlock: "10.0.0.0/16"});
const exampleSubnet = new aws.ec2.Subnet("example", {
    vpcId: exampleVpc.id,
    cidrBlock: "10.0.1.0/24",
});
const exampleSecurityGroup = new aws.ec2.SecurityGroup("example", {
    vpcId: exampleVpc.id,
    ingress: [{
        fromPort: 80,
        toPort: 80,
        protocol: "tcp",
        cidrBlocks: ["0.0.0.0/0"],
    }],
    egress: [{
        fromPort: 0,
        toPort: 0,
        protocol: "-1",
        cidrBlocks: ["0.0.0.0/0"],
    }],
});
const exampleService = new aws.ecs.Service("example", {
    name: "example-service",
    cluster: example.id,
    taskDefinition: exampleTaskDefinition.arn,
    desiredCount: 1,
    launchType: "FARGATE",
    networkConfiguration: {
        subnets: [exampleSubnet.id],
        securityGroups: [exampleSecurityGroup.id],
    },
});
export const clusterName = example.name;
export const serviceName = exampleService.name;

Key Points

  • ECS Cluster: Centralized management for your container instances.
  • Task Definition: Defines the containers to be deployed and their configurations.
  • ECS Service: Ensures the desired number of task instances are running.
  • Docker Logging Driver: Facilitates the forwarding of logs to Splunk for analysis.

Conclusion

In conclusion, this guide demonstrated how to configure AWS ECS to use Docker’s Splunk logging driver. By following these steps, you can efficiently analyze and monitor your container logs in Splunk, thereby improving your application’s observability and debugging processes.

Deploy this code

Want to deploy this code? Sign up for a free Pulumi account to deploy in a few clicks.

Sign up

New to Pulumi?

Want to deploy this code? Sign up with Pulumi to deploy in a few clicks.

Sign up