Introduction

Previously, I used SSH and pem keys to access EC2 instances. As the number of servers and users grew, the number of management points kept increasing: creating keys for each server, sharing them, and revoking them.
To solve this problem, I adopted AWS Systems Manager Session Manager.

With AWS Systems Manager Session Manager, you can access EC2 from a browser or AWS CLI without opening the SSH port. Access can be controlled through IAM, and you do not need to share pem keys directly.

How SSM Access Works

To connect to EC2 through Session Manager, two things must be prepared.

1. The EC2 instance must be managed by Systems Manager.
2. The developer or operator must have permission to use Session Manager.

According to the AWS documentation, the EC2 instance must have SSM Agent installed and running, and it needs an IAM Role that includes the AmazonSSMManagedInstanceCore policy. If you want to connect through the CLI, your local machine also needs AWS CLI and the Session Manager plugin.

Prepare EC2

Create an IAM Role

Create an IAM Role in the AWS console.

• Console path: IAM -> Roles -> Create role
• Trusted entity type: AWS service
• Use case: EC2
• Permission policy: AmazonSSMManagedInstanceCore

For example, you can name it something easy to identify, such as EC2-SSM-Role.

Attach the Role to EC2

Attach the Role to the EC2 instance.

• EC2 -> Instances
• Select the instance
• Actions -> Security -> Modify IAM role
• Select the Role you created

Check the Connection Status

After attaching the Role, check the instance in the EC2 console.

• EC2 -> Instances -> Select the instance
• Check whether it is marked as Managed by Systems Manager
• Or check whether the Connect -> Session Manager tab is enabled

If it does not appear immediately, check the following.

• Whether the IAM Role is attached correctly
• Whether SSM Agent is installed and running inside the instance
• Whether the instance can access Systems Manager endpoints through the internet or a NAT Gateway
• If the instance is in a private subnet, whether the required SSM VPC Endpoints are configured
• If you just attached the Role, wait a few minutes and check again

Prepare the Developer Account

The user who connects through Session Manager also needs permissions.

For a quick connectivity test, you can use AmazonSSMFullAccess, but it is too broad to leave in production. Remove it after testing, and create a separate policy that only allows the required actions. At minimum, starting a session requires ssm:StartSession. For CLI sessions to work correctly, you should also consider the session data channel and the permissions to terminate or resume sessions.

Connect from the Web Console

You can connect directly from the EC2 console.

• EC2 -> Instances
• Select the instance
• Connect
• Select the Session Manager tab
• Click Connect

After connecting, you can run commands as you would in a normal SSH session.

bash
whoami

Set Up CLI Access

To connect from the terminal instead of the web console, you need AWS CLI and the Session Manager plugin.

First, install AWS CLI.

brew install awscli

Check the installation:

aws --version

For safety, this guide uses temporary credentials with MFA.

CLI Access Structure

The CLI setup separates the IAM User used for local credentials from the IAM Role that has the actual SSM permissions.

junlog-dev access key
  -> MFA
  -> request temporary credentials with sts:AssumeRole
  -> temporary credentials for JUNLOG_SSM_SESSION_ROLE
  -> ssm:StartSession

junlog-dev-base is the base profile that stores the access key. junlog-ssm uses that base profile, asks for an MFA code, and then uses temporary credentials from JUNLOG_SSM_SESSION_ROLE, the role that has SSM access. The access key alone cannot start an EC2 session.

IAM Resource Roles

The IAM resources in this setup have the following roles.

In short, the junlog-dev access key is the login mechanism, and JUNLOG_ASSUME_SSM_ROLE_ONLY is the policy that allows requesting temporary credentials for SSM access.
The actual EC2 access permission belongs to JUNLOG_SSM_SESSION_ROLE.

Policy Examples

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowAssumeJunlogSsmSessionRole",
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::<account-id>:role/JUNLOG_SSM_SESSION_ROLE"
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<account-id>:user/junlog-dev"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "Bool": {
          "aws:MultiFactorAuthPresent": "true"
        }
      }
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowStartSsmSessions",
      "Effect": "Allow",
      "Action": "ssm:StartSession",
      "Resource": [
        "arn:aws:ec2:*:<account-id>:instance/*",
        "arn:aws:ssm:*::document/AWS-StartPortForwardingSession",
        "arn:aws:ssm:*:<account-id>:document/SSM-SessionManagerRunShell"
      ]
    },
    {
      "Sid": "AllowOpenSessionDataChannel",
      "Effect": "Allow",
      "Action": "ssmmessages:OpenDataChannel",
      "Resource": "arn:aws:ssm:*:<account-id>:session/${aws:userid}-*"
    },
    {
      "Sid": "AllowManageOwnSsmSessions",
      "Effect": "Allow",
      "Action": [
        "ssm:TerminateSession",
        "ssm:ResumeSession"
      ],
      "Resource": "arn:aws:ssm:*:<account-id>:session/${aws:userid}-*"
    },
    {
      "Sid": "AllowDescribeForSessionTargetDiscovery",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ssm:DescribeInstanceInformation",
        "ssm:GetConnectionStatus",
        "ssm:DescribeSessions"
      ],
      "Resource": "*"
    }
  ]
}

AWS CLI Profile Configuration

~/.aws/credentials:

[junlog-dev-base]
aws_access_key_id = <access-key-id>
aws_secret_access_key = <secret-access-key>

~/.aws/config:

[profile junlog-dev-base]
region = ap-northeast-2
output = json

[profile junlog-ssm]
role_arn = arn:aws:iam::<account-id>:role/JUNLOG_SSM_SESSION_ROLE
source_profile = junlog-dev-base
mfa_serial = arn:aws:iam::<account-id>:mfa/junlog-dev
region = ap-northeast-2
output = json

AWS CLI prompts for an MFA code, issues temporary STS credentials, and caches them under ~/.aws/cli/cache for a certain amount of time.

After setting this up, first check whether the profile is recognized as the correct IAM user.

aws sts get-caller-identity --profile junlog-dev-base

Then check whether the junlog-ssm profile can receive temporary credentials for SSM access.

aws sts get-caller-identity --profile junlog-ssm

Enter the MFA code at this point. If the resulting ARN has the form arn:aws:sts::<account-id>:assumed-role/JUNLOG_SSM_SESSION_ROLE/..., it is working correctly.

Install the Session Manager Plugin

For macOS Apple silicon, you can install it as follows.

curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/mac_arm64/session-manager-plugin.pkg" -o "session-manager-plugin.pkg"
sudo installer -pkg session-manager-plugin.pkg -target /
sudo ln -s /usr/local/sessionmanagerplugin/bin/session-manager-plugin /usr/local/bin/session-manager-plugin

Verify the installation:

session-manager-plugin

If the installation succeeded, you should see a message similar to this.

The Session Manager plugin is installed successfully. Use the AWS CLI to start a session.

Connect from the CLI

First, check the instances managed by SSM.

aws ssm describe-instance-information --profile junlog-ssm

After finding the target instance ID, start a session.

aws ssm start-session --target <instance-id> --profile junlog-ssm

If you frequently connect to specific instances, aliases are convenient. Replace the instance IDs with values from your own environment.

echo "
alias ssm-dev='aws ssm start-session --target <dev-instance-id> --profile junlog-ssm'
alias ssm-prod='aws ssm start-session --target <prod-instance-id> --profile junlog-ssm'
" >> ~/.zshrc

source ~/.zshrc

Port Forwarding

When you need to access a port inside EC2 from your local machine, you can use Session Manager port forwarding.

Port forwarding requires an SSM Agent version that supports the feature. If you are using an old AMI, update SSM Agent before relying on port forwarding.

For example, the following command maps port 5432 inside EC2 to local port 15432.

aws ssm start-session \
  --target <instance-id> \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["5432"],"localPortNumber":["15432"]}' \
  --profile junlog-ssm

After the session starts, connect to localhost:15432 from your local machine.

Troubleshooting

Python/expat Error When Running aws --version

After installing awscli with Homebrew, you might see a Python pyexpat related error when running aws --version. In that case, try reinstalling the Homebrew packages first.

brew update
brew upgrade

brew reinstall expat
brew reinstall python
brew reinstall awscli

brew cleanup

aws --version

If that does not solve the issue, install AWS CLI using the official AWS pkg installer instead of the Homebrew package.

brew uninstall awscli

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "/tmp/AWSCLIV2.pkg"
sudo installer -pkg /tmp/AWSCLIV2.pkg -target /

which aws
aws --version

Session Manager Tab Is Disabled

Check the following in order.

• Whether the EC2 instance has an IAM Role with AmazonSSMManagedInstanceCore
• Whether the SSM Agent is running inside the instance
• Whether the instance can access Systems Manager endpoints
• Whether the connecting user has ssm:StartSession permission
• Whether you waited a few minutes after attaching the Role

References

• AWS Systems Manager Session Manager
• Complete Session Manager prerequisites
• Sample IAM policies for Session Manager
• Connect to your Amazon EC2 instance using Session Manager
• Install the Session Manager plugin on macOS
• Start a session