AWS : CloudWatch configuration on Windows instance with Systems Manager Run Command

bogotobogo.com site search:

Windows server and CloudWatch

We can configure our Amazon EC2 instances to send Windows Server's logs, events, and performance counters to Amazon CloudWatch Logs and Amazon CloudWatch Events.

Amazon EC2 offers several methods for configuring our instances to export this data. The method we choose will depend, in part, on the version of Windows Server we are running and the version of the configuration agent running on our instance. It will also depend on whether we want to manually configure our instances to use a local configuration file or remotely configure them using Systems Manager Run Command or Systems Manager State Manager.

Amazon EC2 instances use an agent to send log data to CloudWatch. With Windows Server 2008 to Windows Server 2012 R2, the agent is either the EC2Config service or SSM Agent. With Windows Server 2016, the agent is SSM Agent.

The following table describes the methods available to integrate with CloudWatch.

Methods to Send Instance Metrics to CloudWatch

The following table (from Sending Logs, Events, and Performance Counters to Amazon CloudWatch) describes the methods available to integrate with CloudWatch.

Sample Configuration File

The following sample file (AWS.EC2.Windows.CloudWatch.json) should be downloaded on our computer:

{
    "IsEnabled": false,
    "EngineConfiguration": {
        "PollInterval": "00:00:15",
        "Components": [
            {
                "Id": "ApplicationEventLog",
                "FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                    "LogName": "Application",
                    "Levels": "1"
                }
            },
            {
                "Id": "SystemEventLog",
                "FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                    "LogName": "System",
                    "Levels": "7"
                }
            },
            {
                "Id": "SecurityEventLog",
                "FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                "LogName": "Security",
                "Levels": "7"
                }
            },
            {
                "Id": "ETW",
                "FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                    "LogName": "Microsoft-Windows-WinINet/Analytic",
                    "Levels": "7"
                }
            },
            {
                "Id": "IISLogs",
                "FullName": "AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                    "LogDirectoryPath": "C:\\inetpub\\logs\\LogFiles\\W3SVC1",
                    "TimestampFormat": "yyyy-MM-dd HH:mm:ss",
                    "Encoding": "UTF-8",
                    "Filter": "",
                    "CultureName": "en-US",
                    "TimeZoneKind": "UTC",
                    "LineCount": "3"
                }
            },
            {
                "Id": "CustomLogs",
                "FullName": "AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                    "LogDirectoryPath": "C:\\CustomLogs\\",
                    "TimestampFormat": "MM/dd/yyyy HH:mm:ss",
                    "Encoding": "UTF-8",
                    "Filter": "",
                    "CultureName": "en-US",
                    "TimeZoneKind": "Local"
                }
            },
            {
                "Id": "PerformanceCounter",
                "FullName": "AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                    "CategoryName": "Memory",
                    "CounterName": "Available MBytes",
                    "InstanceName": "",
                    "MetricName": "Memory",
                    "Unit": "Megabytes",
                    "DimensionName": "",
                    "DimensionValue": ""
                }
            },
            {
                "Id": "CloudWatchLogs",
                "FullName": "AWS.EC2.Windows.CloudWatch.CloudWatchLogsOutput,AWS.EC2.Windows.CloudWatch",
                "Parameters": {
                    "AccessKey": "",
                    "SecretKey": "",
                    "Region": "us-east-1",
                    "LogGroup": "Default-Log-Group",
                    "LogStream": "{instance_id}"
                }
            },
            {
                "Id": "CloudWatch",
                "FullName": "AWS.EC2.Windows.CloudWatch.CloudWatch.CloudWatchOutputComponent,AWS.EC2.Windows.CloudWatch",
                "Parameters": 
                {
                    "AccessKey": "",
                    "SecretKey": "",
                    "Region": "us-east-1",
                    "NameSpace": "Windows/Default"
                }
            }
        ],
        "Flows": {
            "Flows": 
            [
                "(ApplicationEventLog,SystemEventLog),CloudWatchLogs",
                "PerformanceCounter,CloudWatch"
            ]
        }
    }
}

Configure the JSON File for CloudWatch:

Configure the JSON File for CloudWatch

We determine which logs, events, and performance counters are sent to CloudWatch by specifying our choices in a configuration file. The process of creating this file and specifying our choices can take 30 minutes or more to complete. After we have completed this task once, we can reuse the configuration file on all of our instances.

Here are the steps:

Step 1: Configure Settings for CloudWatch

By specifying credentials, a region, and a metric namespace for CloudWatch, we enable an instance to send performance counter data to CloudWatch. If we don't want to send performance counter data, we can skip this procedure. If we want to send the same performance counter data to different locations, we can add additional sections with unique IDs (for example, "CloudWatch2" and CloudWatch3") and a different region for each ID.

To configure settings to send performance counter data to CloudWatch:

In the JSON file, locate the CloudWatch section near the bottom of the file.

{
    "Id": "CloudWatch",
    "FullName": "AWS.EC2.Windows.CloudWatch.CloudWatch.CloudWatchOutputComponent,AWS.EC2.Windows.CloudWatch",
    "Parameters": {
        "AccessKey": "",
        "SecretKey": "",
        "Region": "us-east-1",
        "NameSpace": "Windows/Default"
    }
},

(Optional) If we are using a local configuration file with local credentials, type our access key ID for AccessKey and our secret access key for SecretKey.

Note: If we use Systems Manager Run Command or State Manager, do not provide credentials in the configuration file, as there is a chance that they could be exposed in log files, including debug log files. We'll configure credentials using an IAM role.
For Region, type the region where we want to send log data (for example us-east-1). Although we can send performance counters to a different region from where we send our log data, setting this parameter to the same region is recommended where our instance is running.
For NameSpace, type the metric namespace where performance counter data will be written.

Configure settings to send log data to CloudWatch Logs

Next, specify credentials, region, log group name, and a log stream namespace. This enables the instance to send log data to CloudWatch Logs. If we want to send the same log data to different locations, we can add additional sections with unique IDs (for example, "CloudWatchLogs2" and CloudWatchLogs3") and a different region for each ID.

{
    "Id": "CloudWatchLogs",
    "FullName": "AWS.EC2.Windows.CloudWatch.CloudWatchLogsOutput,AWS.EC2.Windows.CloudWatch",
    "Parameters": {
        "AccessKey": "",
        "SecretKey": "",
        "Region": "us-east-1",
        "LogGroup": "Default-Log-Group",
        "LogStream": "{instance_id}"
    }
},

(Optional) If we are using a local configuration file with local credentials, type our access key ID for AccessKey and our secret access key for SecretKey.

Note: If we use Systems Manager Run Command or State Manager, do not provide credentials in the configuration file, as there is a chance that they could be exposed in log files, including debug log files. You'll configure credentials using an IAM role.
For Region, type the region where we want to send log data (for example, us-east-1).
For LogGroup, type the name for our log group. This name will appear on the Log Groups screen in the CloudWatch console.
For LogStream, type the destination log stream. This name will appear on the Log Groups > Streams screen in the CloudWatch console.

If we use {instance_id}, the default, the log stream name is the instance ID of this instance.

If we specify a log stream name that doesn't already exist, CloudWatch Logs automatically creates it for you. We can define a log stream name using a literal string, the predefined variables {instance_id}, {hostname}, and {ip_address}, or a combination of these.

Step 2: Configure the Data to Send

We can send performance counters, event log data, Event Tracing for Windows (ETW) data, and other log data to Amazon CloudWatch Logs and Amazon CloudWatch Events.

To configure the performance counters to send to CloudWatch

Locate the PerformanceCounter section.

{
    "Id": "PerformanceCounter",
    "FullName": "AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,AWS.EC2.Windows.CloudWatch",
    "Parameters": {
        "CategoryName": "Memory",
        "CounterName": "Available MBytes",
        "InstanceName": "",
        "MetricName": "AvailableMemory",
        "Unit": "Megabytes",
        "DimensionName": "",
        "DimensionValue": ""
    }
},

We can select any performance counters that are available in Performance Monitor. We can select different categories to upload to CloudWatch as metrics, such as .NET CLR Data, ASP.NET Applications, HTTP Service, Memory, or Process and Processors.

For each performance counter, copy the PerformanceCounter section and change the Id parameter to make it unique (for example, "PerformanceCounter2"). Update the other parameters as necessary.
For CategoryName, type the performance counter category. To find the available categories and counters, do the following:
1. Open Performance Monitor.
2. Choose Monitoring Tools, and then choose Performance Monitor.
3. In the results pane, choose the green + (plus) button. The categories and counters are listed in the Add Counters dialog box.
For CounterName, type the name of the performance counter.
For InstanceName, type values from the Add Counters dialog box in Performance Monitor, which can be one of the following:
1. Blank, if the selected object has no instances.
2. A single instance of the selected object.
3. _Total to use the aggregate of all instances.
Do not use an asterisk (*) to indicate all instances because each performance counter component only supports one metric.
For MetricName, type the CloudWatch metric that you want performance data to appear under.
For Unit, type the appropriate unit of measure for the metric. The possible values are as follows:
Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None.
(Optional) Type a dimension name and value for DimensionName and DimensionValue to specify a dimension for our metric. These parameters provide another view when listing metrics. We can also use the same dimension for multiple metrics so that we can view all metrics belonging to a specific dimension.

To send system event log data to CloudWatch Logs

In the JSON file, locate the SystemEventLog section.

{
    "Id": "SystemEventLog",
    "FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
    "Parameters": {
        "LogName": "System",
        "Levels": "7"
    }
},

For Levels, specify the type of messages to upload. We can specify one of the following values:
1. 1 - Upload only error messages.
2. 2 - Upload only warning messages.
3. 4 - Upload only information messages.
We can combine values to include more than one type of message. For example, a value of 3 uploads error messages (1) and warning messages (2). A value of 7 uploads error messages (1), warning messages (2), and information messages (4).

Step 3: Configure Flow Control

Each data type must have a corresponding destination in the Flows section.

For example, to send a performance counter defined in the PerformanceCounter section to the destination defined in the CloudWatch section, add PerformanceCounter,CloudWatch to the Flows section. Similarly, to send the custom log, ETW log, and system log to CloudWatch Logs, add (CustomLogs,ETW,SystemEventLog),CloudWatchLogs.

Note that we can send the same performance counter or log file to more than one destination. For example, to send the application log to two different destinations that we defined in the CloudWatchLogs section, add ApplicationEventLog,(CloudWatchLogs,CloudWatchLogs2) to the Flows section.

To configure flow control

In the AWS.EC2.Windows.CloudWatch.json file, locate the Flows section.

"Flows": {
    "Flows": [
      "PerformanceCounter,CloudWatch",
      "(PerformanceCounter,PerformanceCounter2), CloudWatch2",
      "(CustomLogs, ETW, SystemEventLog),CloudWatchLogs",
      "CustomLogs, CloudWatchLogs2",
      "ApplicationEventLog,(CloudWatchLogs, CloudWatchLogs2)"
    ]
}