Fix: AWS CloudWatch Logs Not Appearing

The Error

You expect logs to appear in CloudWatch Logs but the log group is empty or missing:

Log group: /aws/lambda/my-function
Log streams: (none)

Or the log group exists but new log streams aren’t being created:

The log stream '/aws/lambda/my-function/2026/03/18/[$LATEST]abc123' does not exist.

Or your application logs using the CloudWatch agent but nothing appears:

# In CloudWatch console — log group exists but no events
aws logs describe-log-streams --log-group-name /myapp/application
# {
#     "logStreams": []
# }

Or Lambda logs appear for some invocations but not others. Or EC2 instance logs stopped streaming after a restart.

Why This Happens

CloudWatch log delivery fails for several reasons depending on the source:

• Missing IAM permissions — the Lambda execution role or EC2 instance profile lacks logs:CreateLogGroup, logs:CreateLogStream, or logs:PutLogEvents permissions.
• Log group not created — some services create log groups automatically on first use; others require the log group to exist before logs are sent.
• CloudWatch agent not running or misconfigured — on EC2, the CloudWatch agent is responsible for shipping logs. If it’s stopped or has a bad config, logs are silently dropped.
• Log stream errors — each Lambda invocation creates its own log stream. If the IAM role can’t create streams, logs are lost silently.
• VPC endpoint missing — Lambda or EC2 in a private VPC without internet access needs a VPC endpoint for CloudWatch Logs. Without it, log delivery fails silently.
• Log retention deleting old logs — if you set a short retention period, logs older than the retention window are deleted automatically.
• Region mismatch — you’re looking at the wrong region in the CloudWatch console.

Fix 1: Check IAM Permissions

The most common cause is missing IAM permissions on the execution role or instance profile.

Required permissions for Lambda:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    }
  ]
}

Check Lambda function’s execution role:

# Get the execution role
aws lambda get-function-configuration \
  --function-name my-function \
  --query 'Role' \
  --output text
# arn:aws:iam::123456789012:role/my-lambda-role

# List policies attached to the role
aws iam list-attached-role-policies --role-name my-lambda-role

# Check if AWSLambdaBasicExecutionRole is attached
# This managed policy includes the required CloudWatch Logs permissions
aws iam get-policy \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Attach the basic execution role if missing:

aws iam attach-role-policy \
  --role-name my-lambda-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

For EC2 instances running the CloudWatch agent:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData",
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogStreams",
        "logs:DescribeLogGroups"
      ],
      "Resource": "*"
    }
  ]
}

Attach the managed policy for EC2:

# CloudWatchAgentServerPolicy includes all required permissions
aws iam attach-role-policy \
  --role-name my-ec2-instance-role \
  --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

Fix 2: Check the Correct Region

CloudWatch Logs is region-specific. If your Lambda or EC2 is in us-east-1 but you’re looking at eu-west-1 in the console, the log group won’t appear.

# Check which region your Lambda is in
aws lambda get-function-configuration \
  --function-name my-function \
  --query 'FunctionArn' \
  --output text
# arn:aws:lambda:us-east-1:123456789012:function:my-function
#                 ^^^^^^^^^^
# The region is us-east-1

# Check CloudWatch log groups in the correct region
aws logs describe-log-groups \
  --region us-east-1 \
  --log-group-name-prefix /aws/lambda/my-function

In the AWS Console: Always check the region selector (top right) matches where your service runs.

Fix 3: Create the Log Group Manually

Some services require the log group to exist before they can send logs. Create it manually:

# Create the log group
aws logs create-log-group \
  --log-group-name /myapp/application \
  --region us-east-1

# Set a retention policy (optional — default is never expire)
aws logs put-retention-policy \
  --log-group-name /myapp/application \
  --retention-in-days 30

For Lambda — the log group is /aws/lambda/<function-name>:

aws logs create-log-group \
  --log-group-name /aws/lambda/my-function

aws logs put-retention-policy \
  --log-group-name /aws/lambda/my-function \
  --retention-in-days 14

Using Terraform to pre-create log groups:

resource "aws_cloudwatch_log_group" "lambda_logs" {
  name              = "/aws/lambda/${var.function_name}"
  retention_in_days = 14

  tags = {
    Environment = var.environment
  }
}

resource "aws_lambda_function" "my_function" {
  # ... lambda config
  depends_on = [aws_cloudwatch_log_group.lambda_logs]
}

Fix 4: Fix the CloudWatch Agent on EC2

The CloudWatch agent must be installed, configured, and running to ship logs from EC2 instances.

Check agent status:

# Amazon Linux / RHEL / CentOS
sudo systemctl status amazon-cloudwatch-agent

# If stopped, start it
sudo systemctl start amazon-cloudwatch-agent
sudo systemctl enable amazon-cloudwatch-agent

# Check agent logs for errors
sudo tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

Verify the agent configuration:

# Default config location
cat /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

# Or use the wizard to regenerate config
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

Example CloudWatch agent config for application logs:

{
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/myapp/application.log",
            "log_group_name": "/myapp/application",
            "log_stream_name": "{instance_id}",
            "retention_in_days": 30,
            "timezone": "UTC"
          },
          {
            "file_path": "/var/log/nginx/access.log",
            "log_group_name": "/myapp/nginx-access",
            "log_stream_name": "{instance_id}-access",
            "retention_in_days": 7
          }
        ]
      }
    }
  }
}

Apply the new config and restart:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config \
  -m ec2 \
  -s \
  -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

Fix 5: Fix Lambda in a VPC Without Internet Access

Lambda functions in a private VPC cannot reach CloudWatch Logs endpoints over the internet. Add a VPC endpoint for CloudWatch Logs:

# Create a VPC endpoint for CloudWatch Logs
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-abc12345 \
  --service-name com.amazonaws.us-east-1.logs \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-abc12345 subnet-def67890 \
  --security-group-ids sg-abc12345

Using Terraform:

resource "aws_vpc_endpoint" "cloudwatch_logs" {
  vpc_id              = aws_vpc.main.id
  service_name        = "com.amazonaws.${var.region}.logs"
  vpc_endpoint_type   = "Interface"

  subnet_ids         = aws_subnet.private[*].id
  security_group_ids = [aws_security_group.vpc_endpoint.id]

  private_dns_enabled = true
}

Alternatively — attach the Lambda to a subnet with a NAT Gateway that routes traffic to the internet.

Fix 6: Verify Log Delivery with AWS CLI

Use the AWS CLI to check if log events are actually arriving:

# List log streams in a log group
aws logs describe-log-streams \
  --log-group-name /aws/lambda/my-function \
  --order-by LastEventTime \
  --descending \
  --max-items 5

# Get events from the most recent stream
aws logs get-log-events \
  --log-group-name /aws/lambda/my-function \
  --log-stream-name '2026/03/18/[$LATEST]abc123' \
  --limit 50

# Filter log events across all streams (CloudWatch Logs Insights)
aws logs filter-log-events \
  --log-group-name /aws/lambda/my-function \
  --filter-pattern "ERROR" \
  --start-time $(date -d '1 hour ago' +%s000) \
  --end-time $(date +%s000)

CloudWatch Logs Insights query (in the console or CLI):

# Query the last hour of logs
aws logs start-query \
  --log-group-name /aws/lambda/my-function \
  --start-time $(date -d '1 hour ago' +%s) \
  --end-time $(date +%s) \
  --query-string 'fields @timestamp, @message | sort @timestamp desc | limit 20'

Fix 7: Check Log Retention Settings

If logs appeared previously but now seem deleted, check if a short retention period is set:

# Check retention settings for a log group
aws logs describe-log-groups \
  --log-group-name-prefix /aws/lambda/my-function \
  --query 'logGroups[*].{name:logGroupName, retentionDays:retentionInDays}'

# Update retention — 0 means never expire
aws logs put-retention-policy \
  --log-group-name /aws/lambda/my-function \
  --retention-in-days 0   # Never delete

# Or a reasonable retention period
aws logs put-retention-policy \
  --log-group-name /aws/lambda/my-function \
  --retention-in-days 90

Still Not Working?

Test permissions directly using the AWS CLI from the failing instance or role:

# Test if you can write to CloudWatch Logs
aws logs put-log-events \
  --log-group-name /test/permissions \
  --log-stream-name test-stream \
  --log-events timestamp=$(date +%s000),message="Test log entry"
# If this fails, the IAM permissions are the problem

Check CloudTrail for CloudWatch Logs API errors:

# Search CloudTrail for failed PutLogEvents calls
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=PutLogEvents \
  --start-time $(date -d '1 hour ago' --iso-8601=seconds) | \
  python3 -c "import json,sys; events=json.load(sys.stdin)['Events']; [print(e['CloudTrailEvent']) for e in events if json.loads(e['CloudTrailEvent']).get('errorCode')]"

Check for resource-based policies on the log group that might deny access:

aws logs describe-resource-policies --region us-east-1
# Look for policies that might deny PutLogEvents

For ECS tasks — check the task definition’s logConfiguration. The awslogs driver requires awslogs-group, awslogs-region, and optionally awslogs-create-group:

{
  "logConfiguration": {
    "logDriver": "awslogs",
    "options": {
      "awslogs-group": "/ecs/my-task",
      "awslogs-region": "us-east-1",
      "awslogs-stream-prefix": "ecs",
      "awslogs-create-group": "true"
    }
  }
}

For related AWS issues, see Fix: AWS IAM AccessDeniedException and Fix: AWS Lambda Import Module Error.