A Complete Guide to Pushing Next.js Logs to AWS CloudWatch

Learn how to seamlessly push Next.js logs to AWS CloudWatch. This guide covers creating log groups, publishing logs using the AWS SDK, and monitoring them in the AWS Console.

Hi there! This is Mrinal Prakash, and today I’m going to walk you through everything you need to know about logging Next.js applications in AWS CloudWatch.

In this tutorial, we’ll explore setting up log groups, configuring the AWS SDK for logging, and accessing and analyzing your logs directly in the AWS Console.

This guide will be valuable for developers aiming to enhance visibility and monitoring for their Next.js applications on AWS. Let’s get started!

Creating log group in the aws cloudwatch using nodejs library of aws-sdk

Step 1: Install AWS SDK for Node.js

npm install aws-sdk

Step 2: Write the Code to Create a Log Group

The following code snippet will create a new CloudWatch log group. This uses the CloudWatchLogs service provided by AWS SDK.

// Import the AWS SDK
const AWS = require('aws-sdk');

// Configure the AWS SDK with your region
AWS.config.update({ region: 'ap-south-1' }); // Replace with your preferred region
// Initialize the CloudWatch Logs client
const cloudwatchlogs = new AWS.CloudWatchLogs();
// Function to create a log group
async function createLogGroup(logGroupName) {
  const params = {
    logGroupName: logGroupName, // Specify the name of the log group
  };
  try {
    const data = await cloudwatchlogs.createLogGroup(params).promise();
    console.log('Log group created:', data);
  } catch (error) {
    if (error.code === 'ResourceAlreadyExistsException') {
      console.log('Log group already exists:', logGroupName);
    } else {
      console.error('Error creating log group:', error);
    }
  }
}
// Call the function with the desired log group name
createLogGroup('masite-log-group'); // Replace 'my-log-group' with your log group name

Explanation of the Code

1. Initialize SDK and CloudWatch Logs Client: We first configure the AWS region and initialize the CloudWatchLogs client.
2. Define Parameters: Specify the logGroupName in the parameters, which is mandatory for creating a log group.
3. Create Log Group: The createLogGroup method sends a request to create the log group. If it already exists, it will throw a ResourceAlreadyExistsException, which we handle in the catch block.

Step 3: Run the Script

node your-script-name.js

Now when you run this file, you might get to see the following error:

The AccessDeniedException error indicates that the IAM user mrinal lacks the permissions to perform logs:CreateLogGroup. To fix this, you need to add the necessary permissions to the IAM user or role.

Here are the steps to grant the logs:CreateLogGroup permission:

1. Update IAM Policy

1. Go to the IAM section in the AWS Management Console.
2. Navigate to Users and select the user.
3. Under Permissions, either:

• Attach an existing policy that includes logs:CreateLogGroup permission, or
• Create a custom policy if you need more granular permissions.

2. Create or Attach Policy

If you create a custom policy, add the following JSON policy, which grants the necessary CloudWatch Logs permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:<Region-name>:<number>:log-group:masite-log-group:*"
    }
  ]
}

• The above policy also allows CreateLogStream and PutLogEvents, which are useful for adding log events to the group.

3. Save and Retry

After adding the policy, save the changes. Then, try running your script again:

Now you will see that the Log Group has been created:

Step 3: Publishing a Test Message

// Import the AWS SDK
const AWS = require('aws-sdk');

// Configure the AWS SDK with your region
AWS.config.update({ region: 'ap-south-1' }); // Replace with your preferred region
// Initialize the CloudWatch Logs client
const cloudwatchlogs = new AWS.CloudWatchLogs();
// Function to create a log group
async function createLogGroup(logGroupName) {
  const params = {
    logGroupName: logGroupName, // Specify the name of the log group
  };
  try {
    const data = await cloudwatchlogs.createLogGroup(params).promise();
    console.log('Log group created:', data);
  } catch (error) {
    if (error.code === 'ResourceAlreadyExistsException') {
      console.log('Log group already exists:', logGroupName);
    } else {
      console.error('Error creating log group:', error);
    }
  }
}
// Function to create a log stream
async function createLogStream(logGroupName, logStreamName) {
  const params = {
    logGroupName: logGroupName,
    logStreamName: logStreamName,
  };
  try {
    const data = await cloudwatchlogs.createLogStream(params).promise();
    console.log('Log stream created:', data);
  } catch (error) {
    if (error.code === 'ResourceAlreadyExistsException') {
      console.log('Log stream already exists:', logStreamName);
    } else {
      console.error('Error creating log stream:', error);
    }
  }
}
// Function to publish a log to CloudWatch
const logToCloudWatch = async (logGroupName, logStreamName, logMessage) => {
  // Describe log streams to check for sequenceToken
  const logStreamParams = {
    logGroupName: logGroupName,
    logStreamNamePrefix: logStreamName,
  };
  try {
    const logStreams = await cloudwatchlogs.describeLogStreams(logStreamParams).promise();
    const logStream = logStreams.logStreams.find(stream => stream.logStreamName === logStreamName);
    let sequenceToken;
    if (logStream) {
      sequenceToken = logStream.uploadSequenceToken;
    }
    // Log the event
    const logParams = {
      logEvents: [{ message: logMessage, timestamp: Date.now() }],
      logGroupName,
      logStreamName,
      sequenceToken,
    };
    await cloudwatchlogs.putLogEvents(logParams).promise();
    console.log('Log message published successfully');
  } catch (error) {
    console.error('Error publishing log:', error);
  }
};
// Create log group if not exists
createLogGroup('masite-log-group'); 
// Create log stream
createLogStream('masite-log-group', 'masite-stream');
// Publish log message
logToCloudWatch('masite-log-group', 'masite-stream', 'Hi there this is a test message');

Explanation:

1. Create a log group: The function createLogGroup checks if the log group exists and creates it if it doesn't.
2. Create a log stream: The function createLogStream creates a log stream within the log group.
3. Publish log events: The logToCloudWatch function handles publishing logs to the stream. It first checks if a sequence token is needed (if it's an existing stream), then publishes the log event.

After we run the above code, we get to see the following error:

The error message you’re seeing (AccessDeniedException: User: arn:aws:iam::730335650482:user/mrinal is not authorized to perform: logs:DescribeLogStreams) indicates that the IAM user (mrinal) does not have sufficient permissions to perform the logs:DescribeLogStreams action on CloudWatch.

Once more we have to update the permissions like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:DescribeLogStreams",
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:<Region-name>:<number>:log-group:masite-log-group:*"
    }
  ]
}

Now we click on Save Changes.

Step 4: View the logs from the aws console

Go to the CloudWatch Console -> Log Groups -> Select your log group to see the log streams and logs being generated.

Step 5: Publish the logs in that group using the sdk

Now in order to publish our NextJS Logs into that group using SDK, we will create a service file by the name LoggerService.ts and modify the code written in Step 2 and write the following:

import * as AWS from 'aws-sdk';

interface LoggingService {
  log: (text: string, logLevel: 'info' | 'warn' | 'error') => void;
}

class AWSLoggerService implements LoggingService {
  private cloudwatchlogs: AWS.CloudWatchLogs;
  private logGroupName = <LOG_GROUP_NAME>;
  private logStreamName = <LOG_STREAM_NAME>;
  private sequenceToken?: string;

  constructor() {
    const accessKey = process.env.AWS_ACCESS_KEY_ID;
    const secretKey = process.env.AWS_SECRET_ACCESS_KEY;
    const region = process.env.AWS_REGION;

    if (!accessKey || !secretKey) {
      console.error('AWS credentials are missing.');
    }

    AWS.config.update({
      accessKeyId: accessKey,
      secretAccessKey: secretKey,
      region: region,
    });

    this.cloudwatchlogs = new AWS.CloudWatchLogs();
    this.setupLogStream();
  }

  // Ensures the log stream is created or updated correctly
  private async setupLogStream() {
    try {
      await this.cloudwatchlogs.createLogGroup({ logGroupName: this.logGroupName }).promise();
    } catch (err) {
      if (this.isAWSError(err) && err.code !== 'ResourceAlreadyExistsException') {
        console.error('Error creating log group:', err.message);
      }
    }

    try {
      await this.cloudwatchlogs
        .createLogStream({
          logGroupName: this.logGroupName,
          logStreamName: this.logStreamName,
        })
        .promise();
    } catch (err) {
      if (this.isAWSError(err) && err.code !== 'ResourceAlreadyExistsException') {
        console.error('Error creating log stream:', err.message);
      }
    }
  }

  // Log events to CloudWatch with the appropriate sequence token
  private async logToCloudWatch(message: string, logLevel: string) {
    const logEvent: AWS.CloudWatchLogs.InputLogEvent = {
      message: `${new Date().toISOString()} ${logLevel.toUpperCase()}: ${message}`,
      timestamp: Date.now(),
    };

    const params: AWS.CloudWatchLogs.PutLogEventsRequest = {
      logEvents: [logEvent],
      logGroupName: this.logGroupName,
      logStreamName: this.logStreamName,
      sequenceToken: this.sequenceToken,
    };

    try {
      const response = await this.cloudwatchlogs.putLogEvents(params).promise();
      this.sequenceToken = response.nextSequenceToken;  // Update sequence token for next log
    } catch (err) {
      if (this.isAWSError(err)) {
        console.error('Error logging to CloudWatch:', err.message);
      } else {
        console.error('Unknown error logging to CloudWatch:', err);
      }
    }
  }

  // Core logging function
  log = (text: string, logLevel: 'info' | 'warn' | 'error' = 'info') => {
    this.logToCloudWatch(text, logLevel);
    console.log(`${new Date().toISOString()} ${logLevel.toUpperCase()}: ${text}`);
  };

  // Convenience methods for logging at different levels
  info = (text: string) => {
    this.log(text, 'info');
  };

  warn = (text: string) => {
    this.log(text, 'warn');
  };

  error = (text: string) => {
    this.log(text, 'error');
  };

  // Utility function to check for AWS errors
  private isAWSError(error: unknown): error is AWS.AWSError {
    return (
      typeof error === 'object' &&
      error !== null &&
      'code' in error &&
      'message' in error
    );
  }
}

export { AWSLoggerService };

Explanation of Code:

• Setup AWS SDK and Configure AWS Credentials: The constructor initializes AWS SDK with credentials from environment variables. It’s essential to set up AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION in your environment configuration.
• Creating Log Group and Log Stream: The setupLogStream function creates a log group and log stream if they don’t already exist. It handles the ResourceAlreadyExistsException to avoid errors on re-creation.
• Logging Events: The logToCloudWatch function is responsible for sending log events to CloudWatch. AWS requires a sequence token for ordering logs, so the code retrieves and updates this token with each log event.
• Logging Levels: This logger supports info, warn, and error levels. Each log message is prepended with a timestamp, log level, and the actual message content.
• Error Handling: The isAWSError function checks if an error is of type AWSError, allowing specific error handling.

Step 4: Using AWSLoggerService in Your Next.js Application

1. Import the AWSLoggerService in your Next.js codebase.
2. Initialize the logger and call the logging methods for different log levels.

import { AWSLoggerService } from './path-to-AWSLoggerService';

const logger = new AWSLoggerService();
logger.info('This is an info log message');
logger.warn('This is a warning log message');
logger.error('This is an error log message');

3. After you make the above changes, run the code using the following:

npm run dev

Now, after you run this, hover over to your Logs Page, you will get to see the following:

Conclusion

In this guide, we’ve covered everything you need to know about pushing Next.js logs to AWS CloudWatch, from creating a log group and log stream to publishing logs and managing permissions.

CloudWatch’s centralized logging solution makes it easy to view and analyze logs, enabling faster issue resolution and deeper insights into application performance.