Using AWS Application Composer to build a serviceful application (virtual part 2)
In (virtual) part 1 of this blog post (yes it has a different title) I have shown how to use a combination of AWS Step Functions and Amazon EventBridge to modify the behaviour of Amazon ECS and virtually adding a new feature that doesn't exist in the product itself. Ok this may sound a big hyperbolic, and it probably is, but go back to (virtual) part 1 to get more context.
The long story short is that I have a couple of EventBridge rules that target a Step Functions state machine which, in turns, calls a mix of EC2 and Route53 APIs. And I now need to stitch these pieces together. Enter AWS Application Composer.
Application Composer offers a canvas where you can drag and drop the components of a serviceful application you need to build and generates a SAM artifact that you can deploy. Starting from the pieces of configurations I have discussed in part 1, this couldn't have been easier leveraging Application Composer.
Note: if you came here just to grab the template go to the very end of this post and have it. Otherwise, stay the course to see how I generated it with Application Composer
All I needed to do was to open a new project in Application Composer, drag a couple of EventBridge Event rule elements and a Step Functions State Machine element onto the canvas, link them to create the relationship (the state machine is a target of both events) and voilà:
As you can see I have customized the name of these components to reflect their job. The other thing I had to do was to populate the event rules and the state machine definition in the components. The code snippets I needed are those I outlined in part 1.
This is how you'd paste one of the rule in the EventBridge component:
And this is how you'd paste the state machine definition in the Step Functions component:
App Composer will show these snippets as YAML the next time you edit them (which I prefer over json anyway, I think...)
Now it is a good time to switch from the Canvas view to the Template view and see what Application Composer has generated for us:
There are a couple of interesting and useful things to note.
First, if you explore the template, you will notice that all the roles and their policies have been super-scoped following the least-privilege permissions best practice. This applies to all the components and their wiring per the canvas. For example, the IAM Roles to invoke the State Machine can only be assumed by the specific Event Bridge rules in the template.
The second thing is that Application Composer tries hard to produce templates that work across AWS regions and partitions. While I haven't tried it first hand, you should be able to take templates you create with Application Composer in commercial regions and deploy them to GovCloud or the China region. For example, in the IAM Role assume role policy for one of the Event Bridge Rules to invoke the state machine we see:
1 Statement:
2 Effect: Allow
3 Principal:
4 Service: !Sub events.${AWS::URLSuffix}
5 Action: sts:AssumeRole
The URLSuffix substitution is the magic that makes this work, as it makes it events.amazonaws.com.cn for China regions, for example.
There is only one thing left to do before I can proceed with the deployment.
Application Composer does not have the ability to parse the Step Functions state machine definition to infer the policies it needs to allow the workflow to interact with the services it's interacting with (in my case EC2 and Route53). I had to explicitly add manually these policies to the template file.
For this, I had to locate the Policies section for the state machine in the template and add additional entries to the Action's. This was relatively straightforward because the Step Functions AWS SDK Service Integrations is a 1:1 mapping to atomic AWS APIs, and so I can add them easily to my template. The APIs I am using in the state machine are DescribeNetworkInterfaces, ListResourceRecordSets and ChangeResourceRecordSets. This is how the state machine Policies section should look like after adding manually these 3 entries:
1 Policies:
2 - AWSXrayWriteOnlyAccess
3 - Statement:
4 - Effect: Allow
5 Action:
6 - ec2:DescribeNetworkInterfaces
7 - route53:ListResourceRecordSets
8 - route53:ChangeResourceRecordSets
9 - logs:CreateLogDelivery
10 - logs:GetLogDelivery
11 - logs:UpdateLogDelivery
12 - logs:DeleteLogDelivery
13 - logs:ListLogDeliveries
14 - logs:PutResourcePolicy
15 - logs:DescribeResourcePolicies
16 - logs:DescribeLogGroups
17 Resource: '*'
And this is it. The template is ready to be deployed! Note that, at the time of this writing, Application Composer does not offer a workflow to deploy your stack. To do so, you just save your template locally from the Application Composer console, and from there you can either just use the CloudFormation console to create a new stack off of the template.yml you saved, or you can use SAM CLI. I personally like to move in the folder of the template.yml and run:
1sam deploy --stack-name r53recupdates --capabilities CAPABILITY_IAM
Tip: make sure to not use a
--stack-namethat is too long because EventBridge rule names are limited to 64 characters, and it's easy to get too close to 64 characters when Application Composer builds the rule name adding the stack name, logical ID of the rule and more random characters.
If you did not bother to go through the steps to create the complete template.yml file (no offense taken) I am pasting it below in its entirety. The cool thing is that you can work backwards and import the file below into Application Composer to see it in the canvas:
1AWSTemplateFormatVersion: '2010-09-09'
2Transform: AWS::Serverless-2016-10-31
3Resources:
4 ecstaskstopped:
5 Type: AWS::Events::Rule
6 Properties:
7 EventPattern:
8 source:
9 - aws.ecs
10 detail-type:
11 - ECS Task State Change
12 detail:
13 lastStatus:
14 - RUNNING
15 desiredStatus:
16 - STOPPED
17 Targets:
18 - Id: !GetAtt ecstaskroute53update.Name
19 Arn: !Ref ecstaskroute53update
20 RoleArn: !GetAtt ecstaskstoppedToecstaskroute53update.Arn
21 ecstaskrunning:
22 Type: AWS::Events::Rule
23 Properties:
24 EventPattern:
25 source:
26 - aws.ecs
27 detail-type:
28 - ECS Task State Change
29 detail:
30 lastStatus:
31 - RUNNING
32 desiredStatus:
33 - RUNNING
34 Targets:
35 - Id: !GetAtt ecstaskroute53update.Name
36 Arn: !Ref ecstaskroute53update
37 RoleArn: !GetAtt ecstaskrunningToecstaskroute53update.Arn
38 ecstaskroute53update:
39 Type: AWS::Serverless::StateMachine
40 Properties:
41 Definition:
42 Comment: State machine to create/update a Route53 record
43 StartAt: DescribeNetworkInterfaces
44 States:
45 DescribeNetworkInterfaces:
46 Type: Task
47 Parameters:
48 NetworkInterfaceIds.$: $.detail.attachments[0].details[?(@.name==networkInterfaceId)].value
49 Resource: arn:aws:states:::aws-sdk:ec2:describeNetworkInterfaces
50 Next: ListResourceRecordSets
51 ResultPath: $.NetworkInterfaceDescription
52 ListResourceRecordSets:
53 Type: Task
54 Parameters:
55 HostedZoneId.$: States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==HOSTEDZONEID)].Value, 0)
56 Resource: arn:aws:states:::aws-sdk:route53:listResourceRecordSets
57 ResultPath: $.ResourceRecordSetsOutput
58 Next: RunningOrStopped
59 RunningOrStopped:
60 Type: Choice
61 Choices:
62 - Variable: $.detail.desiredStatus
63 StringMatches: RUNNING
64 Next: UpsertAction
65 - Variable: $.detail.desiredStatus
66 StringMatches: STOPPED
67 Next: DeleteAction
68 Default: DeleteAction
69 DeleteAction:
70 Type: Pass
71 Next: ChangeResourceRecordSets
72 Result:
73 recordAction: DELETE
74 ResultPath: $.recordActionOutput
75 UpsertAction:
76 Type: Pass
77 Next: ChangeResourceRecordSets
78 Result:
79 recordAction: UPSERT
80 ResultPath: $.recordActionOutput
81 ChangeResourceRecordSets:
82 Type: Task
83 Parameters:
84 ChangeBatch:
85 Changes:
86 - Action.$: $.recordActionOutput.recordAction
87 ResourceRecordSet:
88 Name.$: States.Format('{}.{}', States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==aws:ecs:serviceName)].Value, 0),States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==PUBLICHOSTEDZONE)].Value, 0))
89 Type: A
90 Ttl: 60
91 ResourceRecords:
92 - Value.$: $.NetworkInterfaceDescription.NetworkInterfaces[0].Association.PublicIp
93 HostedZoneId.$: States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==HOSTEDZONEID)].Value, 0)
94 Resource: arn:aws:states:::aws-sdk:route53:changeResourceRecordSets
95 End: true
96 Logging:
97 Level: ALL
98 IncludeExecutionData: true
99 Destinations:
100 - CloudWatchLogsLogGroup:
101 LogGroupArn: !GetAtt ecstaskroute53updateLogGroup.Arn
102 Policies:
103 - AWSXrayWriteOnlyAccess
104 - Statement:
105 - Effect: Allow
106 Action:
107 - ec2:DescribeNetworkInterfaces
108 - route53:ListResourceRecordSets
109 - route53:ChangeResourceRecordSets
110 - logs:CreateLogDelivery
111 - logs:GetLogDelivery
112 - logs:UpdateLogDelivery
113 - logs:DeleteLogDelivery
114 - logs:ListLogDeliveries
115 - logs:PutResourcePolicy
116 - logs:DescribeResourcePolicies
117 - logs:DescribeLogGroups
118 Resource: '*'
119 Tracing:
120 Enabled: true
121 Type: STANDARD
122 ecstaskroute53updateLogGroup:
123 Type: AWS::Logs::LogGroup
124 Properties:
125 LogGroupName: !Sub
126 - /aws/vendedlogs/states/${AWS::StackName}-${ResourceId}-Logs
127 - ResourceId: ecstaskroute53update
128 ecstaskstoppedToecstaskroute53update:
129 Type: AWS::IAM::Role
130 Properties:
131 AssumeRolePolicyDocument:
132 Version: '2012-10-17'
133 Statement:
134 Effect: Allow
135 Principal:
136 Service: !Sub events.${AWS::URLSuffix}
137 Action: sts:AssumeRole
138 Condition:
139 ArnLike:
140 aws:SourceArn: !Sub
141 - arn:${AWS::Partition}:events:${AWS::Region}:${AWS::AccountId}:rule/${AWS::StackName}-${ResourceId}-*
142 - ResourceId: ecstaskstopped
143 Policies:
144 - PolicyName: StartExecutionPolicy
145 PolicyDocument:
146 Version: '2012-10-17'
147 Statement:
148 - Effect: Allow
149 Action: states:StartExecution
150 Resource: !Ref ecstaskroute53update
151 ecstaskrunningToecstaskroute53update:
152 Type: AWS::IAM::Role
153 Properties:
154 AssumeRolePolicyDocument:
155 Version: '2012-10-17'
156 Statement:
157 Effect: Allow
158 Principal:
159 Service: !Sub events.${AWS::URLSuffix}
160 Action: sts:AssumeRole
161 Condition:
162 ArnLike:
163 aws:SourceArn: !Sub
164 - arn:${AWS::Partition}:events:${AWS::Region}:${AWS::AccountId}:rule/${AWS::StackName}-${ResourceId}-*
165 - ResourceId: ecstaskrunning
166 Policies:
167 - PolicyName: StartExecutionPolicy
168 PolicyDocument:
169 Version: '2012-10-17'
170 Statement:
171 - Effect: Allow
172 Action: states:StartExecution
173 Resource: !Ref ecstaskroute53update
Give Application Composer a try if you need to solve similar problems!
Massimo.