RFC 0006 defines query sources over a vendor's OpenAPI description. AWS publishes no OpenAPI; it publishes Smithy models for every service (aws/api-models-aws, Apache-2.0). This RFC lets an integration bundle Smithy models so organizations can define query sources over AWS operations the same way. The definition, the validation feedback loop, storage, collection, evidence, and the rule preview are RFC 0006's, unchanged; this RFC specifies what differs: naming operations across services, deciding what is a read, endpoints and regions, the wire protocols, and pagination. Open questions are in Section 10.
#1. Summary
integration: dev.adversity.aws
name: open-security-groups
version: 1
operation: ec2.DescribeSecurityGroups # <service>.<Operation>
regions: "*" # every region the connection lists
params:
Filters: '[{"Name": "ip-permission.cidr", "Values": ["0.0.0.0/0"]}]'
fields:
id: GroupId
name: GroupName
vpc: VpcId
rules: IpPermissions
Each item gains region. A rule then reads dev.adversity.aws/open-security-groups@1 like any source.
The library tests this unlocks without per-check code: network boundaries (EC2 security groups, AC-04), security monitoring (GuardDuty detectors, LM-02), encryption at rest (EBS default encryption, RDS instances, KMS key rotation, CR-01), and least privilege (IAM roles and policies, AC-03).
#2. Integration support
An integration declares the services it bundles:
smithy:
services:
iam:
model: models/iam-2010-05-08.json
endpoint: "https://iam.amazonaws.com"
signing_region: us-east-1 # global services sign for one region
ec2:
model: models/ec2-2016-11-15.json
endpoint: "https://ec2.{region}.amazonaws.com"
guardduty:
model: models/guardduty-2017-11-28.json
endpoint: "https://guardduty.{region}.amazonaws.com"
read_operations: [iam.GenerateCredentialReport]
deny_operations: [ec2.GetPasswordData, ec2.GetConsoleOutput, ec2.GetConsoleScreenshot, ec2.GetLaunchTemplateData]
- Services are keyed by a short name, which prefixes operations:
ec2.DescribeSecurityGroups. - Models are Smithy 2.0 JSON AST files from
aws/api-models-aws, pinned to the package version.packtrims each to its read operations and the shapes they reach, and drops documentation. EC2's model shrinks from about 7 MB to a fraction of that. - Endpoints are declared, not derived. An
awsJsonservice whoseX-Amz-Targetpredates its Smithy shape name declarestarget_prefix. An endpoint with{region}is regional; one without is global and signs forsigning_region. Smithy endpoint rulesets (FIPS, dual-stack, other partitions) are not evaluated in this revision. - Authentication is SigV4 with the connection's credentials (
aws_role, RFC 0005 section 5), signing for the model'saws.auth#sigv4name. SigV4 moves from the AWS integration into the SDK.
#3. What is a read
AWS models rarely carry Smithy's @readonly trait: none of IAM's 181 operations, EC2's 830, or S3's 112 do. And a read is not necessarily safe: s3:GetObject returns customer data and ec2:GetPasswordData returns credentials. Query sources read configuration, not content. An operation can be queried only if all of these hold:
- It has
@readonly, or its name starts withList,Describe, orGetfollowed by an upper-case letter, or it is listed inread_operations. - For REST protocols (
restJson1,restXml), its@httpmethod isGET. - It is not in
deny_operations. Integration authors list every read that returns content or secrets; first-party review checks the list per release.
The connection's IAM role is the backstop: the recommended AWS-managed SecurityAudit policy grants configuration reads and none of the content reads above. OGRL checks these rules when a definition is saved and before every collection; the SDK checks them again in the integration process.
#4. Operations, parameters, and fields
Input. params names top-level members of the operation's input structure. A scalar member (string, number, boolean, enum, timestamp) takes its value as a string. A list or structure member, such as EC2 Filters, takes JSON text, validated against the member's shape. Members bound to HTTP headers or payloads cannot be set. Pagination members (inputToken, pageSize) are reserved for the paginator.
Items. For a paginated operation, items defaults to the @paginated trait's items member and may be omitted. Otherwise it names a list member of the output; left empty, the output structure itself is the one item, as for ec2.GetEbsEncryptionByDefault.
Fields. Paths use the model's member names (GroupId, IpPermissions, Encrypted), dotted through structures. Types map as: string, enum, and blob to string; byte, short, integer, long, and intEnum to integer; float, double, and bigDecimal to number; boolean to boolean; timestamp to an RFC 3339 string; list and set to array; structure, map, and document to object. A member is required in the derived schema when @required holds at every level of its path. Two paths are reserved: . is the item itself, for lists of strings or numbers such as GuardDuty's detector IDs, and $region is the item's region, which names the single item of a regional operation:
operation: ec2.GetEbsEncryptionByDefault
regions: "*"
fields: {id: $region, name: $region, enabled: EbsEncryptionByDefault}
Regions. regions is a list of region codes, or "*" for the connection's regions config. A regional operation is collected once per region and items gain a region field; a global operation ignores regions. A parameter whose value is exactly $region takes the region being collected, such as S3's BucketRegion.
Per-item calls. Many AWS settings are read one resource at a time: S3 encryption per bucket, the public access block per bucket. for_each (RFC 0006 section 3) runs a list operation first and calls operation once per item it returns:
integration: dev.adversity.aws
name: bucket-encryption
version: 1
operation: s3.GetBucketEncryption
regions: "*"
for_each:
operation: s3.ListBuckets # same service as operation
params: {BucketRegion: $region} # buckets homed in the region being collected
bind: {Bucket: Name} # operation parameter: path in each listed item
limit: 1000 # at most this many listed items (default 1000, at most 5000)
fields:
id: $parent.BucketArn
name: $parent.Name
rules: ServerSideEncryptionConfiguration.Rules
for_each.operationis queryable by Section 3, in the same service, and paginated or givenitems.bindnames required parameters ofoperation, whichparamsmay not also set.- Each call yields items as an ordinary query would. Fields may read the listed item through
$parent.<path>. - A call that fails with a client error other than authentication, permission, or throttling yields one item with every field empty and
errorholding the AWS error code, such asNoSuchPublicAccessBlockConfiguration. Items of afor_eachquery always carryerror(null on success), so a rule can treat a missing configuration as a failure. Any other failure fails the region, as for a plain query. - More listed items than
limitmakes the collection incomplete.
#5. Protocols
The SDK serializes requests and deserializes responses from the model's shapes. Every protocol shares the same shape-driven reader, so a field path means the same thing whatever the wire format.
| Protocol | Services (examples) | Request | Response |
|---|---|---|---|
awsJson1_0, awsJson1_1 | CloudTrail, KMS, DynamoDB | POST /, X-Amz-Target: <service shape>.<Operation>, JSON body | JSON |
restJson1 | GuardDuty, Lambda | @http method and URI, @httpLabel and @httpQuery members | JSON, @jsonName honored |
awsQuery | IAM, RDS, STS | POST /, form Action, Version, flattened members (Name.member.1) | XML <Result> wrapper, member lists, @xmlName, @xmlFlattened |
ec2Query | EC2 | POST /, form with @ec2QueryName keys (Filter.1.Value.1) | XML without a Result wrapper, item lists |
restXml | S3, CloudFront, Route 53 | @http method and URI | XML, @xmlName, @xmlFlattened |
Pagination follows the @paginated trait, merged with the service's default: the output token is sent back as the input token until it is empty or repeats. Operations without the trait return one page.
Errors. The AWS error code and message are added to the collection error. Throttling codes (Throttling, ThrottlingException, RequestLimitExceeded, and similar) are retried like HTTP 429, and access-denied codes are permission errors whatever their HTTP status.
#6. Validation feedback
RFC 0006 section 6 applies unchanged. Static validation adds Section 3's read rules, regions against regional or global endpoints, and JSON parameter values against their shapes. describe_operation returns the flattened output item shape and the read classification, so an author or the assistant sees why an operation is or is not queryable.
#7. Collection and evidence
Unchanged from RFC 0006 section 7. A regional query's snapshot records the regions it covered; a region that fails (for example, an opt-in region the account has not enabled) makes the snapshot incomplete, with the region and the error as the reason.
#8. API
RFC 0006's routes serve Smithy-backed integrations without change. GET /integrations/{id}/operations lists operations as <service>.<Operation> with their protocol. Query definitions gain the optional regions member.
#9. Delivery
| Step | Scope |
|---|---|
| 1 | SDK: Smithy model loading and trimming, Section 3 classification, shape-driven validation and item schemas, SigV4, and the JSON and query protocols (awsJson1_0, awsJson1_1, restJson1, awsQuery, ec2Query) with pagination and regions. |
| 2 | ogrl-aws bundles IAM, EC2, RDS, KMS, CloudTrail, and GuardDuty, and drops its hand-written SigV4 for the SDK's. |
| 3 | OGRL: the shared API-model interface over OpenAPI and Smithy; regions on definitions. |
| 4 | restXml and S3; for_each per-item calls. |
#10. Open questions
- Endpoint rulesets. Declared endpoints cover the commercial partition. FIPS, dual-stack, GovCloud, and China need either partition-specific declarations or an evaluator for Smithy's endpoint rules.
- Classifying reads. The name-prefix rule plus a deny list is a heuristic. A reviewed allowlist per service is stricter but much longer to maintain; the deny list should grow from review and from AWS's own access-level data in the IAM service authorization reference.
- Model drift. New service model versions can rename members. The upgrade check (RFC 0006 open question 3) applies.