Security Services: GuardDuty, Security Hub, Config & CloudTrail

By the end of this lesson you will be able to switch on AWS's main security watchers. You will also send serious alerts to the right person automatically. These are skills every cloud or DevOps engineer is expected to have.

Good IAM and network design keep most attackers out. But you also need to notice when something goes wrong. A leaked key might be used from an unknown country. A bucket might be made public by mistake. A server might start talking to a crypto-mining pool. Think of a housing society: the gate and locks are your IAM and network rules, and these services are the CCTV cameras and the guard's register. AWS runs them for you and they watch your accounts all the time.

You will set up a central audit trail and turn on threat detection with GuardDuty. Then you will check settings with AWS Config, collect everything in Security Hub, and route serious findings to people.

The security toolbox at a glance

Many companies use several AWS accounts, as in AWS Organizations, Multi-Account Setup & SCPs. There, you make the security tools account the delegated administrator for GuardDuty, Security Hub, Config aggregation and Inspector. A delegated administrator is an account allowed to manage a service for the whole organisation. The services are then switched on in every member account. The security team sees all findings in one place, without logging in to the management account.

An organisation trail

The CloudTrail event history keeps only 90 days and covers only one account. You saw this in CloudWatch Monitoring & Logs. For real auditing, create one organisation trail. You create it in the management account or a delegated administrator. It records every Region of every account into a bucket in the log archive account:

aws cloudtrail create-trail --name org-trail \
  --s3-bucket-name jaipur-org-cloudtrail-777788889999 \
  --is-multi-region-trail --is-organization-trail \
  --enable-log-file-validation \
  --kms-key-id arn:aws:kms:ap-south-1:777788889999:key/1234abcd-12ab-34cd-56ef-1234567890ab \
  --query "{Name: Name, MultiRegion: IsMultiRegionTrail, Org: IsOrganizationTrail}"

aws cloudtrail start-logging --name org-trail

{
    "Name": "org-trail",
    "MultiRegion": true,
    "Org": true
}

Log file validation writes signed digest files. A digest is a fingerprint of each log file. Later you can prove that nobody edited or deleted the logs, much like a sealed exam answer sheet.

Two permissions are needed. The bucket needs a bucket policy that lets CloudTrail write to it. The KMS key needs a key policy that lets CloudTrail use it. If you build the trail in the console, it creates both for you. Member accounts can see the trail, but they cannot stop or change it.

GuardDuty: threat detection

Amazon GuardDuty watches for attacker-like behaviour. It reads three data sources on its own: CloudTrail management events, VPC Flow Logs and DNS query logs. You do not have to turn those logs on yourself. You can also add optional protection plans. These cover S3 data events, EKS, runtime monitoring, malware scanning, RDS logins and Lambda network activity.

GuardDuty uses threat intelligence and machine learning. Threat intelligence means lists of known bad IP addresses and domains. When something looks wrong, it raises a finding, such as:

• UnauthorizedAccess:EC2/SSHBruteForce: someone is trying many SSH passwords.
• CryptoCurrency:EC2/BitcoinTool.B!DNS: an instance is looking up crypto-mining domains.
• UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS: credentials from an EC2 role are being used outside AWS.

Turn it on in a Region with one command. Then create sample findings to test your alerts without any real attack:

aws guardduty create-detector --enable

aws guardduty create-sample-findings \
  --detector-id 12abc34d567e8fa901bc2d34e56789f0 \
  --finding-types "UnauthorizedAccess:EC2/SSHBruteForce"

aws guardduty list-findings --detector-id 12abc34d567e8fa901bc2d34e56789f0 \
  --query "length(FindingIds)"

{
    "DetectorId": "12abc34d567e8fa901bc2d34e56789f0"
}
1

Sample findings are clearly marked as samples, so nobody mistakes them for a real incident. GuardDuty is a regional service. Enable it in every Region you have enabled, not only Mumbai. Attackers love Regions that nobody watches, just as thieves prefer the unlit lane.

AWS Config: configuration and compliance

AWS Config records the settings of your resources over time. It can answer questions like "what did this security group look like last Tuesday?" It also checks your resources against Config rules. AWS provides hundreds of ready-made rules. Examples are restricted-ssh (no SSH open to the world), s3-bucket-public-read-prohibited and encrypted-volumes. A conformance pack is a bundle of rules for one standard.

Once rules are running, ask which ones are failing:

aws configservice describe-compliance-by-config-rule \
  --compliance-types NON_COMPLIANT \
  --query "ComplianceByConfigRules[].[ConfigRuleName, Compliance.ComplianceType]" \
  --output text

restricted-ssh	NON_COMPLIANT
encrypted-volumes	NON_COMPLIANT

Config can also fix problems by itself with remediation actions. These are Systems Manager Automation documents, for example one that removes a public SSH rule. Start with notifications only. Automate fixes once you trust the rule.

Security Hub, Inspector and friends

AWS Security Hub CSPM stands for cloud security posture management. "Posture" simply means how well your setup follows good security habits. Security Hub CSPM does three jobs:

1. It runs automated checks from standards such as AWS Foundational Security Best Practices and the CIS AWS Foundations Benchmark.
2. It gives each account a security score, like a report card.
3. It collects findings from GuardDuty, Inspector, Macie, Config and partner tools in one standard format.

It uses AWS Config to run most of its checks, so Config must be recording.

A few more services complete the toolbox:

• Amazon Inspector scans EC2 instances, container images in ECR and Lambda functions. It looks for known software vulnerabilities (CVEs) and unexpected network exposure. It rescans automatically when new vulnerabilities are published.
• Amazon Macie finds sensitive data in S3.
• Amazon Detective helps you investigate a finding by linking related activity.
• IAM Access Analyzer reports resources shared outside your organisation. It also reports unused roles, keys and permissions.

Send serious findings to people

Findings that nobody reads are useless. A smoke alarm that rings in an empty room saves nobody. Every one of these services sends findings to Amazon EventBridge, an event router. One EventBridge rule can forward high-severity GuardDuty findings to an SNS topic. That topic then emails or messages the on-call engineer. GuardDuty severity is a number, and 7.0 and above is high:

{
  "source": ["aws.guardduty"],
  "detail-type": ["GuardDuty Finding"],
  "detail": {
    "severity": [{ "numeric": [">=", 7] }]
  }
}

aws events put-rule --name guardduty-high \
  --event-pattern file://guardduty-high.json --query RuleArn --output text

aws events put-targets --rule guardduty-high \
  --targets "Id=ops,Arn=arn:aws:sns:ap-south-1:111122223333:security-alerts"

arn:aws:events:ap-south-1:111122223333:rule/guardduty-high
{
    "FailedEntryCount": 0,
    "FailedEntries": []
}

The SNS topic's access policy must allow events.amazonaws.com to publish. You will learn more about EventBridge patterns in Messaging & Events: SQS, SNS & EventBridge. In India, some serious incidents must also be reported to CERT-In quickly. See Data Residency & Compliance in India.

Common mistakes

• Enabling GuardDuty only in Mumbai, while other enabled Regions stay unwatched.
• Keeping CloudTrail logs in the same account they describe. An attacker with admin rights there can delete them.
• Turning on Security Hub CSPM without AWS Config, so most checks cannot run.
• No routing for findings. Connect high-severity findings to people through EventBridge and SNS.
• Auto-fixing everything on day one and breaking real workloads. Observe first, then automate.

Key points

• CloudTrail answers "who did what". An organisation trail with log file validation should write to a locked-down log archive account.
• GuardDuty detects threats from CloudTrail, VPC flow and DNS data without you managing those logs. Enable it in every enabled Region.
• AWS Config records configuration history and checks rules. Remediation can fix issues automatically.
• Security Hub CSPM runs best-practice checks and gathers findings. The newer Security Hub links them into prioritised risks.
• Inspector finds vulnerabilities, Macie finds sensitive data, and Access Analyzer finds external and unused access.
• Use a delegated administrator account and EventBridge rules so the right people see serious findings.

{
  "source": ["aws.config"],
  "detail-type": ["Config Rules Compliance Change"],
  "detail": {
    "newEvaluationResult": {
      "complianceType": ["NON_COMPLIANT"]
    }
  }
}

aws events put-rule --name config-noncompliant --event-pattern file://config-nc.json