x-kinesis ¶
x-kinesis:
stream:
Properties: {} # AWS Kinesis CFN definition
Settings: {}
MacroParameters: {}
Services: {}
Define Kinesis Data Streams, new or existing ones, that you wish your services to consume/produce from/to.
Services ¶
As per the generic Services definition, we have a list of object, name and access, which define how the service can access the stream.
For AWS Kinesis streams, we have the following permissions.
ReturnValues ¶
To access the Ref value, use StreamId
See AWS CFN Kinesis Return Values for available values.
IAM permissions ¶
The following predefined permissions are available (see JSON definition of the IAM policy statement below).
-
Producer
-
Consumer
-
PowerUser
{
"Consumer": {
"Effect": "Allow",
"Action": [
"kinesis:DescribeStream",
"kinesis:GetRecords",
"kinesis:ListShards"
],
"Resource": [
"${ARN}"
]
},
"Producer": {
"Effect": "Allow",
"Action": [
"kinesis:DescribeStream",
"kinesis:DescribeStreamSummary",
"kinesis:DescribeStreamConsumer",
"kinesis:SubscribeToShard",
"kinesis:RegisterStreamConsumer",
"kinesis:PutRecord",
"kinesis:PutRecords",
"kinesis:ListShards"
],
"Resource": [
"${ARN}"
]
},
"PowerUser": {
"Effect": "Allow",
"NotAction": [
"kinesis:CreateStream",
"kinesis:DeleteStream"
],
"Resource": [
"${ARN}"
]
},
"kinesis_firehose": {
"kinesisSource": {
"Effect": "Allow",
"Action": [
"kinesis:DescribeStream",
"kinesis:GetShardIterator",
"kinesis:GetRecords",
"kinesis:ListShards"
],
"Resource": [
"${ARN}"
]
}
}
}
Properties ¶
The Properties are the AWS CFN definition for AWS Kinesis streams .
Hint
If you leave the Properties empty, default values are used and ShardCount is set to 1.
MacroParameters ¶
No specific MacroParameters for Kinesis data streams. Given the AWS definition is very straightforward, just define the properties. The only truly required property is the ShardCount
Examples ¶
services: [serviceA, serviceB]
x-kinesis:
streamA:
Properties:
ShardCount: 2
Services:
serviceA:
Access: Producer
serviceB:
Access: Consumer
JSON Schema ¶
Model ¶
x-kinesis ¶
x-kinesis.spec.json |
|||||||
x-kinesis specification |
|||||||
type |
object |
||||||
properties |
|||||||
|
x-resources.common.spec.json#/definitions/Lookup |
||||||
|
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesis-stream.html |
||||||
type |
object |
||||||
|
x-resources.common.spec.json#/definitions/Settings |
||||||
|
x-resources.common.spec.json#/definitions/Services |
||||||
|
type |
object |
|||||
properties |
|||||||
|
type |
object |
|||||
patternProperties |
|||||||
|
#/definitions/SetupIAMForKCLDef |
||||||
additionalProperties |
False |
||||||
definitions |
|||||||
|
oneOf |
Enables IAM permissions as described https://docs.aws.amazon.com/streams/latest/dev/tutorial-stock-data-kplkcl-iam.html |
|||||
type |
boolean |
||||||
type |
object |
||||||
properties |
|||||||
|
If true, allows PutMetricData to Cloudwatch Metrics |
||||||
type |
boolean |
||||||
|
oneOf |
Grant all recommend IAM permissions for DynamoDB. Very permissive, use at own risks |
|||||
type |
boolean |
||||||
type |
object |
||||||
properties |
|||||||
|
If true, adds IAM permissions to create/delete any tables. |
||||||
type |
boolean |
||||||
|
List of DynamoDB tables to restrict the create/delete to instead of * |
||||||
type |
array |
||||||
items |
type |
string |
|||||
additionalProperties |
False |
||||||
additionalProperties |
False |
Definition ¶
{
"$schema": "http://json-schema.org/draft-07/schema#",
"id": "x-kinesis.spec.json",
"$id": "x-kinesis.spec.json",
"title": "x-kinesis",
"description": "x-kinesis specification",
"type": "object",
"additionalProperties": false,
"properties": {
"Lookup": {
"$ref": "x-resources.common.spec.json#/definitions/Lookup"
},
"Properties": {
"type": "object",
"description": "https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesis-stream.html"
},
"Settings": {
"$ref": "x-resources.common.spec.json#/definitions/Settings"
},
"Services": {
"$ref": "x-resources.common.spec.json#/definitions/Services"
},
"MacroParameters": {
"type": "object",
"properties": {
"SetupIAMForKCL": {
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9-_\\.]+$": {
"$ref": "#/definitions/SetupIAMForKCLDef"
}
}
}
}
}
},
"oneOf": [
{
"required": [
"Lookup"
]
},
{
"required": [
"Properties"
]
}
],
"definitions": {
"SetupIAMForKCLDef": {
"oneOf": [
{
"type": "boolean",
"description": "Enables IAM permissions as described https://docs.aws.amazon.com/streams/latest/dev/tutorial-stock-data-kplkcl-iam.html"
},
{
"type": "object",
"additionalProperties": false,
"properties": {
"CloudWatchPutMetricData": {
"type": "boolean",
"description": "If true, allows PutMetricData to Cloudwatch Metrics"
},
"DynamoDB": {
"oneOf": [
{
"type": "boolean",
"description": "Grant all recommend IAM permissions for DynamoDB. Very permissive, use at own risks"
},
{
"type": "object",
"additionalProperties": false,
"properties": {
"EnableCreateDeleteTables": {
"type": "boolean",
"description": "If true, adds IAM permissions to create/delete any tables."
},
"TableNames": {
"type": "array",
"items": {
"type": "string"
},
"description": "List of DynamoDB tables to restrict the create/delete to instead of *"
}
}
}
]
}
}
}
]
}
}
}
Test files ¶
You can find the test files here to use as reference for your use-case.