perfware.cloud - Getting Started Guide - AI Monitor 26h2
AI/ML-powered anomaly detection for CloudWatch metrics
· Deploy
· Teardown
· Appendix B: Subscription Fields Reference
· Appendix C: Enabling AWS/Billing in Remote Accounts
· Support
AI Monitor ingests CloudWatch metrics via Metric Streams, learns normal behavior per metric using statistical baselines, and alerts on deviations automatically. It includes:
· Real-time metric ingestion via CloudWatch Metric Streams → Firehose → S3 → OpenSearch
· Per-metric anomaly detection with day-of-week baselines and Z-score scoring
· AI-powered anomaly explanations via Amazon Bedrock (advanced+)
· AI Discovery Wizard with natural language metric suggestions
· Maintenance windows to suppress alerts during deployments
· Log correlation — links anomalies to errors in Log Processor (compact+)
· Automated anomaly reports with cost analysis (advanced+)
· Cross-account metric monitoring (advanced+)
· SNS alerts with message attributes for filter-based routing
· Web-based subscription editor with RBAC (Cognito authentication)
· CloudWatch dashboard with alarm status, EMF metrics, and AI health
· DynamoDB Point-in-Time Recovery for state protection (advanced+)
Deploys as a CloudFormation stack. Requires a deployed Log Processor stack (provides VPC, OpenSearch domain, and networking).
· A deployed Log Processor stack (essential tier or above) — provides the shared VPC and OpenSearch domain
· An AWS account with permissions to create CloudFormation stacks, Lambda, DynamoDB, S3, SNS, Firehose, CloudWatch
· Domain and ACM certificate for HTTPS on the editor ALB (see Appendix A)
· AWS CLI configured (for operational scripts)
• Use the provided (.cmd/.sh) scripts to manage the domain, users, stack deletion, cross-account, and snapshot operations. Scripts download: scripts.zip.
• Use the check-quota script to ensure you have sufficient AWS resources, e.g. current VPC utilization, you may need to request a service quota increase (more…) and Bedrock compatibility (more…).
· For AI features (advanced+): Enable Bedrock model access for Anthropic Claude in the AWS console (Bedrock → Model access → enable Anthropic → submit form, ~15 min activation)
1. Find AI Monitor in AWS Marketplace, select your tier
2. Click Continue to Subscribe and accept terms
3. Click Set up your account → redirected to CloudFormation
4. Fill in parameters (below) → check IAM acknowledgment → Submit
|
Parameter |
Required |
Description |
|
ConfirmStackName |
Yes |
Copy your stack name here (3-18 chars, alphanumeric, start with letter) |
|
LogProcessorStackName |
Yes |
Name of your deployed Log Processor stack |
|
AlertEmail |
Yes |
Email(s) for anomaly alerts and reports (comma-separated, up to 5) |
|
EditorCertificateArn |
Yes |
ACM certificate ARN for HTTPS on the editor ALB |
|
EditorDomain |
Yes |
Custom domain for the editor (e.g. monitor.example.com). Provides stable URL |
|
CrossAccountIds |
No |
Comma-separated remote account IDs for cross-account monitoring (advanced+) |
|
OrganizationId |
No |
AWS Organization ID (e.g. o-abc123xyz). If set, any account in the org can deliver metrics without listing individual account IDs (advanced+) |
The stack takes approximately 5-10 minutes to create.
Step 2-continued:
a. Run the second setup-domain command (see Appendix A)
b.
You will also receive an email from AWS Notifications no-reply@sns.amazonaws.com confirming the SNS subscription.
To access OpenSearch and the Subscription Editor the default root-level user is admin.
Use the provided (.cmd/.sh) script to manage the Cognito User Pool from the CloudFormation stack outputs:
|
If you are using the Advanced tier or above, for other users you need to create 'admin' and 'viewer' groups to access the editor. Assign other users to the 'admin' group for full access or 'viewer' group for read-only permissions. For lower tiers all users have full permission. |
Using the helper script:
Usage: users.cmd <stack-name> create admin@example.com
You will receive an email with a temporary password upon create.
Available commands (refer to the readme.txt for more information):
|
Command |
Description |
|
users.cmd <stack-name> create <email> |
Create a user with a temporary password |
|
users.cmd <stack-name> list |
List all users with email, status, and enabled flag |
|
users.cmd <stack-name> delete <email> |
Delete a user |
|
users.cmd <stack-name> reset <email> |
Reset password (user sets new one on next login) |
|
users.cmd <stack-name> disable <email> |
Disable a user (blocks login) |
|
users.cmd <stack-name> enable <email> |
Re-enable a disabled user |
Link: User Guide (PDF)
For RBAC, create admin and viewer groups and assign users:
groups.cmd <stack-name> create-group admin "Full
access"
groups.cmd <stack-name> create-group viewer
"Read-only"
groups.cmd <stack-name> add-to-group user@example.com
admin
If using a custom domain, create a DNS CNAME/alias to the ALB after deployment:
setup-domain.cmd
<stack-name> <domain> <hosted-zone-id> <region>
Check your email for the AWS Notification - Subscription Confirmation email and confirm it.
Access
the editor at: https://<editor-domain>/editor
Features:
· Add/edit/delete metric subscriptions with validation
· Preview metric data (click Preview; Shift+click = 24h; Ctrl+click = query OpenSearch)
· Maintenance windows to suppress alerts during known events
· Anomaly viewer with detail panel, CloudWatch deep links, and AI explanations
· Baseline display showing learned per-day patterns
· Version history with diff and one-click rollback
· On-demand report generation
· Scheduled report auto-email
Click the ✨ button in the header to open the Discovery panel.
· Shows all active CloudWatch metrics in your account (refreshed every 6h)
· Quick-start chips for common monitoring patterns
· AI Suggest (advanced+): describe what you want to monitor in natural language
· One-click Subscribe button pre-fills the subscription form
The
stack creates a CloudWatch dashboard: <stack-name>-monitoring
Includes widgets for:
· Alarm status (all alarms at a glance)
· Collector: invocations, errors, throttles, duration
· Detector: anomalies detected, subscriptions scored, AI cache hits/misses
· AI Explain: processed, failed, circuit breaker status
· Discovery: namespaces and metrics found
· Editor: requests and latency
· SQS queue depth, DLQ, Firehose freshness
· OpenSearch cluster health, CPU, JVM, storage
· Lambda error/warning log queries
Run
setup-cross-account-metrics.cmd (or .sh) in the remote account:
setup-cross-account-metrics.cmd
<central-account-id> <central-stack-name> <central-region>
This
creates a Metric Stream → local Firehose → central S3 bucket pipeline. The
central stack's CrossAccountIds parameter must include
the remote account ID.
See Appendix C for enabling billing metrics in remote accounts.
1. Go to AWS Marketplace → Manage subscriptions
2. Select AI Monitor, choose the new version/tier
3. Follow prompts to update the CloudFormation stack
4. Configuration and data are preserved across updates
1. Disable all subscriptions in the editor (stops the Metric Stream)
2. Delete the CloudFormation stack
3. Retry CloudFormation delete, retaining resources that cleanup.py can later address:
|
Some AWS resources (S3 bucket names, Cognito domains) may take several minutes to fully release after stack deletion, select retain for subsequent delete attempts, use this for final cleanup: Run cleanup.py, described in readme.txt Usage: python cleanup.py <stack-name> <region> [-force] [-confirm] By default, shows what would be deleted (dry run). Note: Review carefully before specifying -force |
4. If Unsubscribing – go to AWS Marketplace →Manage subscriptions →Cancel monthly renewal subscription
Note: The stack uses the Log Processor's VPC and OpenSearch domain. Deleting AI Monitor does not affect Log Processor. Delete AI Monitor first if removing both.
No anomalies detected
• Verify subscriptions are enabled in the editor
• Check baseline training % (needs ~50% before alerting)
• Verify Metric Stream is active (editor syncs this automatically on save)
• Check the <stack>-monitoring dashboard for Lambda errors
Cannot access editor
• Verify user exists in Cognito (users.cmd <stack> list)
• For advanced+ tiers: verify user is in admin or viewer group
• Verify ACM certificate is in Issued state
• Verify DNS points to the ALB
AI features not working
• Verify Bedrock model access is enabled (Console → Bedrock → Model access)
• Check AI circuit breaker state: support-bundle.cmd <stack>
• Circuit breaker auto-resets after 30 minutes
Support bundle
Collect
diagnostics: support-bundle.cmd
<stack-name>
Same
process as Log Processor. Use the provided setup-domain(.cmd|sh) script:
Step 1: Generate the cert (before deploy)
setup-domain.cmd cert-only <sub-domain> <host-zone-id> <region>
e.g. setup-domain.cmd cert-only monitor.example.com Z013AAAAAB2QDFABCDL8S us-east-1
Step 2: Deploy stack with the domain and cert ARN from step 1
Step 3: After deploy, create the DNS alias to the ALB
setup-domain.cmd <stackname> <sub-domain> <host-zone-id> <region>
e.g. setup-domain.cmd AIProcessor dashboards.example.com Z013AAAAAB2QDFABCDL8S us-east-1
|
Field |
Description |
|
namespace |
CloudWatch namespace (e.g. AWS/Lambda, AWS/EC2) |
|
metricName |
Metric name within the namespace |
|
dimensions |
JSON object. {} = all resources. Use ".*" for wildcard per-resource monitoring. |
|
stat |
Average, Sum, Maximum, Minimum, SampleCount, p99, p95, p90 |
|
period |
Aggregation period: 60, 300, 900, 3600 seconds |
|
anomalyThreshold |
Z-score sensitivity (1.0-10.0). Lower = more sensitive. Default: 3.0 |
|
direction |
high (spikes only), low (drops only), both |
|
baselineDays |
Training period override (0 = tier default) |
|
accountId |
Blank = local only. * = all accounts. Or comma-separated IDs. |
|
notifyEmail |
true/false. Set false for silent recording only. |
|
notifyThreshold |
Only email when score exceeds this (0 = use anomalyThreshold) |
|
runbookUrl |
Optional URL included in alert emails |
|
description |
Human-readable name (required, must be unique) |
AWS/Billing metrics (EstimatedCharges) are not enabled by default. Each account that needs billing monitoring must enable this separately:
1. Log in to the AWS Management Console as the root user or an IAM identity with billing permissions.
2. Navigate to the AWS Billing and Cost Management console.
3. In the left navigation pane, choose Billing preferences.
4. Find the Alert preferences section and choose Edit.
5. Select the checkbox for Receive CloudWatch Billing Alerts.
6. Choose Save preferences (or Update) to apply the changes.
Note: Billing metrics are only published in us-east-1 regardless of where your resources are. If your Metric Stream is in another region, create a separate stream in us-east-1 for AWS/Billing. After enabling, metrics begin publishing within a few hours.
Link: User Guide (PDF)
Email: support@perfware.cloud
Include
the output of support-bundle <stack-name> when reporting issues.