RFC 0007

Smithy Query Sources

Status Draft
Date 2026-11-21
Reading time 7 min
Depends on RFC 0005, RFC 0006
On this page

RFC 0006 defines query sources over a vendor's OpenAPI description. AWS publishes no OpenAPI; it publishes Smithy models for every service (aws/api-models-aws, Apache-2.0). This RFC lets an integration bundle Smithy models so organizations can define query sources over AWS operations the same way. The definition, the validation feedback loop, storage, collection, evidence, and the rule preview are RFC 0006's, unchanged; this RFC specifies what differs: naming operations across services, deciding what is a read, endpoints and regions, the wire protocols, and pagination. Open questions are in Section 10.

#1. Summary

yaml
integration: dev.adversity.aws
name: open-security-groups
version: 1
operation: ec2.DescribeSecurityGroups      # <service>.<Operation>
regions: "*"                               # every region the connection lists
params:
  Filters: '[{"Name": "ip-permission.cidr", "Values": ["0.0.0.0/0"]}]'
fields:
  id: GroupId
  name: GroupName
  vpc: VpcId
  rules: IpPermissions

Each item gains region. A rule then reads dev.adversity.aws/open-security-groups@1 like any source.

The library tests this unlocks without per-check code: network boundaries (EC2 security groups, AC-04), security monitoring (GuardDuty detectors, LM-02), encryption at rest (EBS default encryption, RDS instances, KMS key rotation, CR-01), and least privilege (IAM roles and policies, AC-03).

#2. Integration support

An integration declares the services it bundles:

yaml
smithy:
  services:
    iam:
      model: models/iam-2010-05-08.json
      endpoint: "https://iam.amazonaws.com"
      signing_region: us-east-1               # global services sign for one region
    ec2:
      model: models/ec2-2016-11-15.json
      endpoint: "https://ec2.{region}.amazonaws.com"
    guardduty:
      model: models/guardduty-2017-11-28.json
      endpoint: "https://guardduty.{region}.amazonaws.com"
  read_operations: [iam.GenerateCredentialReport]
  deny_operations: [ec2.GetPasswordData, ec2.GetConsoleOutput, ec2.GetConsoleScreenshot, ec2.GetLaunchTemplateData]
  • Services are keyed by a short name, which prefixes operations: ec2.DescribeSecurityGroups.
  • Models are Smithy 2.0 JSON AST files from aws/api-models-aws, pinned to the package version. pack trims each to its read operations and the shapes they reach, and drops documentation. EC2's model shrinks from about 7 MB to a fraction of that.
  • Endpoints are declared, not derived. An awsJson service whose X-Amz-Target predates its Smithy shape name declares target_prefix. An endpoint with {region} is regional; one without is global and signs for signing_region. Smithy endpoint rulesets (FIPS, dual-stack, other partitions) are not evaluated in this revision.
  • Authentication is SigV4 with the connection's credentials (aws_role, RFC 0005 section 5), signing for the model's aws.auth#sigv4 name. SigV4 moves from the AWS integration into the SDK.

#3. What is a read

AWS models rarely carry Smithy's @readonly trait: none of IAM's 181 operations, EC2's 830, or S3's 112 do. And a read is not necessarily safe: s3:GetObject returns customer data and ec2:GetPasswordData returns credentials. Query sources read configuration, not content. An operation can be queried only if all of these hold:

  1. It has @readonly, or its name starts with List, Describe, or Get followed by an upper-case letter, or it is listed in read_operations.
  2. For REST protocols (restJson1, restXml), its @http method is GET.
  3. It is not in deny_operations. Integration authors list every read that returns content or secrets; first-party review checks the list per release.

The connection's IAM role is the backstop: the recommended AWS-managed SecurityAudit policy grants configuration reads and none of the content reads above. OGRL checks these rules when a definition is saved and before every collection; the SDK checks them again in the integration process.

#4. Operations, parameters, and fields

Input. params names top-level members of the operation's input structure. A scalar member (string, number, boolean, enum, timestamp) takes its value as a string. A list or structure member, such as EC2 Filters, takes JSON text, validated against the member's shape. Members bound to HTTP headers or payloads cannot be set. Pagination members (inputToken, pageSize) are reserved for the paginator.

Items. For a paginated operation, items defaults to the @paginated trait's items member and may be omitted. Otherwise it names a list member of the output; left empty, the output structure itself is the one item, as for ec2.GetEbsEncryptionByDefault.

Fields. Paths use the model's member names (GroupId, IpPermissions, Encrypted), dotted through structures. Types map as: string, enum, and blob to string; byte, short, integer, long, and intEnum to integer; float, double, and bigDecimal to number; boolean to boolean; timestamp to an RFC 3339 string; list and set to array; structure, map, and document to object. A member is required in the derived schema when @required holds at every level of its path. Two paths are reserved: . is the item itself, for lists of strings or numbers such as GuardDuty's detector IDs, and $region is the item's region, which names the single item of a regional operation:

yaml
operation: ec2.GetEbsEncryptionByDefault
regions: "*"
fields: {id: $region, name: $region, enabled: EbsEncryptionByDefault}

Regions. regions is a list of region codes, or "*" for the connection's regions config. A regional operation is collected once per region and items gain a region field; a global operation ignores regions. A parameter whose value is exactly $region takes the region being collected, such as S3's BucketRegion.

Per-item calls. Many AWS settings are read one resource at a time: S3 encryption per bucket, the public access block per bucket. for_each (RFC 0006 section 3) runs a list operation first and calls operation once per item it returns:

yaml
integration: dev.adversity.aws
name: bucket-encryption
version: 1
operation: s3.GetBucketEncryption
regions: "*"
for_each:
  operation: s3.ListBuckets            # same service as operation
  params: {BucketRegion: $region}      # buckets homed in the region being collected
  bind: {Bucket: Name}                 # operation parameter: path in each listed item
  limit: 1000                          # at most this many listed items (default 1000, at most 5000)
fields:
  id: $parent.BucketArn
  name: $parent.Name
  rules: ServerSideEncryptionConfiguration.Rules
  • for_each.operation is queryable by Section 3, in the same service, and paginated or given items. bind names required parameters of operation, which params may not also set.
  • Each call yields items as an ordinary query would. Fields may read the listed item through $parent.<path>.
  • A call that fails with a client error other than authentication, permission, or throttling yields one item with every field empty and error holding the AWS error code, such as NoSuchPublicAccessBlockConfiguration. Items of a for_each query always carry error (null on success), so a rule can treat a missing configuration as a failure. Any other failure fails the region, as for a plain query.
  • More listed items than limit makes the collection incomplete.

#5. Protocols

The SDK serializes requests and deserializes responses from the model's shapes. Every protocol shares the same shape-driven reader, so a field path means the same thing whatever the wire format.

ProtocolServices (examples)RequestResponse
awsJson1_0, awsJson1_1CloudTrail, KMS, DynamoDBPOST /, X-Amz-Target: <service shape>.<Operation>, JSON bodyJSON
restJson1GuardDuty, Lambda@http method and URI, @httpLabel and @httpQuery membersJSON, @jsonName honored
awsQueryIAM, RDS, STSPOST /, form Action, Version, flattened members (Name.member.1)XML <Result> wrapper, member lists, @xmlName, @xmlFlattened
ec2QueryEC2POST /, form with @ec2QueryName keys (Filter.1.Value.1)XML without a Result wrapper, item lists
restXmlS3, CloudFront, Route 53@http method and URIXML, @xmlName, @xmlFlattened

Pagination follows the @paginated trait, merged with the service's default: the output token is sent back as the input token until it is empty or repeats. Operations without the trait return one page.

Errors. The AWS error code and message are added to the collection error. Throttling codes (Throttling, ThrottlingException, RequestLimitExceeded, and similar) are retried like HTTP 429, and access-denied codes are permission errors whatever their HTTP status.

#6. Validation feedback

RFC 0006 section 6 applies unchanged. Static validation adds Section 3's read rules, regions against regional or global endpoints, and JSON parameter values against their shapes. describe_operation returns the flattened output item shape and the read classification, so an author or the assistant sees why an operation is or is not queryable.

#7. Collection and evidence

Unchanged from RFC 0006 section 7. A regional query's snapshot records the regions it covered; a region that fails (for example, an opt-in region the account has not enabled) makes the snapshot incomplete, with the region and the error as the reason.

#8. API

RFC 0006's routes serve Smithy-backed integrations without change. GET /integrations/{id}/operations lists operations as <service>.<Operation> with their protocol. Query definitions gain the optional regions member.

#9. Delivery

StepScope
1SDK: Smithy model loading and trimming, Section 3 classification, shape-driven validation and item schemas, SigV4, and the JSON and query protocols (awsJson1_0, awsJson1_1, restJson1, awsQuery, ec2Query) with pagination and regions.
2ogrl-aws bundles IAM, EC2, RDS, KMS, CloudTrail, and GuardDuty, and drops its hand-written SigV4 for the SDK's.
3OGRL: the shared API-model interface over OpenAPI and Smithy; regions on definitions.
4restXml and S3; for_each per-item calls.

#10. Open questions

  1. Endpoint rulesets. Declared endpoints cover the commercial partition. FIPS, dual-stack, GovCloud, and China need either partition-specific declarations or an evaluator for Smithy's endpoint rules.
  2. Classifying reads. The name-prefix rule plus a deny list is a heuristic. A reviewed allowlist per service is stricter but much longer to maintain; the deny list should grow from review and from AWS's own access-level data in the IAM service authorization reference.
  3. Model drift. New service model versions can rename members. The upgrade check (RFC 0006 open question 3) applies.