Defender — Objective 4: Identify Credential Theft¶
Finding the Suspicious Event¶
Start from the most obviously suspicious API call — ListBuckets. A web application has no reason to enumerate all S3 buckets:
select() — jq's WHERE clause:
| Part | What it does |
|---|---|
select(condition) |
Only pass through events where the condition evaluates to true |
.eventName=="ListBuckets" |
Condition: eventName must equal the string "ListBuckets" |
This is the jq equivalent of SQL's WHERE eventname = 'ListBuckets'. Without select, every event passes through. With it, only matching events do.
Reading the Event¶
The ListBuckets event:
{
"userIdentity": {
"type": "AssumedRole",
"principalId": "AROAJQMBDNUMIKLZKMF64:d190d14a-2404-45d6-9113-4eda22d7f2c7",
"arn": "arn:aws:sts::653711331788:assumed-role/level3/d190d14a-2404-45d6-9113-4eda22d7f2c7",
"accountId": "653711331788",
"accessKeyId": "<REDACTED_ACCESS_KEY_ID>",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"arn": "arn:aws:iam::653711331788:role/level3"
}
}
},
"eventTime": "2018-11-28T23:09:28Z",
"sourceIPAddress": "104.102.221.250",
"userAgent": "[aws-cli/1.16.19 Python/2.7.10 Darwin/17.7.0 botocore/1.12.9]",
"eventName": "ListBuckets"
}
Three red flags in this event:
1. Role: level3 — this is the ECS task role
userIdentity.arn shows assumed-role/level3/.... The level3 role is the IAM role attached to the ECS container (the web app). ECS tasks use this role to call AWS services from inside the container. It should only ever appear with AWS-owned source IPs.
2. Source IP: 104.102.221.250 — not an AWS IP
ECS containers run inside AWS infrastructure. When the container calls an AWS API, the request comes from an AWS-owned IP (the underlying EC2 host or Fargate cluster). 104.102.221.250 is not in any AWS IP range — it's an external IP (the exercise says it's nsa.gov, but in a real investigation it would be the attacker's machine or a VPN exit node).
3. User agent: aws-cli/1.16.19 Python/2.7.10 Darwin/17.7.0
Darwin is macOS. The ECS container runs Linux. The ECS application's code wouldn't use the aws-cli directly — it would use an SDK like boto3 or the Go AWS SDK. Seeing the macOS aws-cli user agent means a person on a Mac ran aws s3 ls from their terminal using the stolen credentials.
All three together = definitive credential theft. The credentials are legitimate (the role exists, the keys are valid), but they're being used from outside the AWS environment, by a human, on a personal machine.
Checking the Role's Trust Policy¶
Confirm the role was only supposed to be used by ECS:
aws iam get-role — fetches the role's metadata, including its trust policy.
| Part | What it does |
|---|---|
aws iam get-role |
Get full details about an IAM role |
--role-name level3 |
The role to look up (by name, not ARN) |
--profile target_security |
Run this against the Target account via the cross-account role |
Response:
"AssumeRolePolicyDocument": {
"Statement": [{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Effect": "Allow"
}]
}
Trust policy explained:
A trust policy is the policy attached to the role that defines who can assume it. It's the "door" — the role's permission policies define what you can do once inside, but the trust policy defines who's allowed to knock.
Principal: { Service: "ecs-tasks.amazonaws.com" }— only the ECS service can assume this role- No human IAM users, no other accounts, no other services
- Only ECS tasks can assume this role — so any API call from this role that didn't originate from within ECS is by definition unauthorized
Comparison:
| Policy type | Attached to | Controls |
|---|---|---|
| Permission policy | Role/user | What you can do (which AWS APIs you can call) |
| Trust policy | Role only | Who can assume this role (which principals can sts:AssumeRole) |
How Credential Theft Detection Works¶
Normal ECS role behaviour in CloudTrail:
| Field | Expected value |
|---|---|
sourceIPAddress |
An AWS-owned IP (the EC2 host IP or Fargate IP) |
userIdentity.type |
AssumedRole |
userAgent |
The application framework: python-requests, boto3, aws-sdk-go, etc. |
eventName |
Only the specific APIs the app needs (e.g., GetObject if it reads S3, not ListBuckets) |
Stolen credentials being used externally:
| Field | What you see instead |
|---|---|
sourceIPAddress |
Non-AWS IP — residential, VPN, cloud provider outside AWS |
userIdentity.type |
Still AssumedRole — credentials are valid |
userAgent |
aws-cli on Darwin (macOS) or Windows |
eventName |
Broad recon calls the app would never need: ListBuckets, ListUsers, DescribeInstances |
Key insight: CloudTrail cannot flag stolen credentials by itself — it just records API calls. The detection comes from comparing the pattern of use against what's expected. This is what UEBA (User and Entity Behaviour Analytics) tools automate: baselines + anomaly detection.
In manual investigation: look for role identities using APIs from IPs they've never used before, or calling APIs they have no business calling.
The talk to understand this better: Detecting Credential Compromise in AWS — Will Bengston The core idea: establish a baseline of "normal" IPs and user-agents for each role, then alert on deviations.