Using AWS Application Composer to build a serviceful application (virtual part 2)

In (virtual) part 1 of this blog post (yes it has a different title) I have shown how to use a combination of AWS Step Functions and Amazon EventBridge to modify the behaviour of Amazon ECS and virtually adding a new feature that doesn't exist in the product itself. Ok this may sound a big hyperbolic, and it probably is, but go back to (virtual) part 1 to get more context.

The long story short is that I have a couple of EventBridge rules that target a Step Functions state machine which, in turns, calls a mix of EC2 and Route53 APIs. And I now need to stitch these pieces together. Enter AWS Application Composer.

Application Composer offers a canvas where you can drag and drop the components of a serviceful application you need to build and generates a SAM artifact that you can deploy. Starting from the pieces of configurations I have discussed in part 1, this couldn't have been easier leveraging Application Composer.

Note: if you came here just to grab the template go to the very end of this post and have it. Otherwise, stay the course to see how I generated it with Application Composer

All I needed to do was to open a new project in Application Composer, drag a couple of EventBridge Event rule elements and a Step Functions State Machine element onto the canvas, link them to create the relationship (the state machine is a target of both events) and voilà:

As you can see I have customized the name of these components to reflect their job. The other thing I had to do was to populate the event rules and the state machine definition in the components. The code snippets I needed are those I outlined in part 1.

This is how you'd paste one of the rule in the EventBridge component:

And this is how you'd paste the state machine definition in the Step Functions component:

App Composer will show these snippets as YAML the next time you edit them (which I prefer over json anyway, I think...)

Now it is a good time to switch from the Canvas view to the Template view and see what Application Composer has generated for us:

There are a couple of interesting and useful things to note.

First, if you explore the template, you will notice that all the roles and their policies have been super-scoped following the least-privilege permissions best practice. This applies to all the components and their wiring per the canvas. For example, the IAM Roles to invoke the State Machine can only be assumed by the specific Event Bridge rules in the template.

The second thing is that Application Composer tries hard to produce templates that work across AWS regions and partitions. While I haven't tried it first hand, you should be able to take templates you create with Application Composer in commercial regions and deploy them to GovCloud or the China region. For example, in the IAM Role assume role policy for one of the Event Bridge Rules to invoke the state machine we see:

1        Statement:
2          Effect: Allow
3          Principal:
4            Service: !Sub events.${AWS::URLSuffix}
5          Action: sts:AssumeRole

The URLSuffix substitution is the magic that makes this work, as it makes it for China regions, for example.

There is only one thing left to do before I can proceed with the deployment.

Application Composer does not have the ability to parse the Step Functions state machine definition to infer the policies it needs to allow the workflow to interact with the services it's interacting with (in my case EC2 and Route53). I had to explicitly add manually these policies to the template file.

For this, I had to locate the Policies section for the state machine in the template and add additional entries to the Action's. This was relatively straightforward because the Step Functions AWS SDK Service Integrations is a 1:1 mapping to atomic AWS APIs, and so I can add them easily to my template. The APIs I am using in the state machine are DescribeNetworkInterfaces, ListResourceRecordSets and ChangeResourceRecordSets. This is how the state machine Policies section should look like after adding manually these 3 entries:

 1      Policies:
 2        - AWSXrayWriteOnlyAccess
 3        - Statement:
 4            - Effect: Allow
 5              Action:
 6                - ec2:DescribeNetworkInterfaces
 7                - route53:ListResourceRecordSets
 8                - route53:ChangeResourceRecordSets
 9                - logs:CreateLogDelivery
10                - logs:GetLogDelivery
11                - logs:UpdateLogDelivery
12                - logs:DeleteLogDelivery
13                - logs:ListLogDeliveries
14                - logs:PutResourcePolicy
15                - logs:DescribeResourcePolicies
16                - logs:DescribeLogGroups
17              Resource: '*'

And this is it. The template is ready to be deployed! Note that, at the time of this writing, Application Composer does not offer a workflow to deploy your stack. To do so, you just save your template locally from the Application Composer console, and from there you can either just use the CloudFormation console to create a new stack off of the template.yml you saved, or you can use SAM CLI. I personally like to move in the folder of the template.yml and run:

1sam deploy --stack-name r53recupdates --capabilities CAPABILITY_IAM

Tip: make sure to not use a --stack-name that is too long because EventBridge rule names are limited to 64 characters, and it's easy to get too close to 64 characters when Application Composer builds the rule name adding the stack name, logical ID of the rule and more random characters.

If you did not bother to go through the steps to create the complete template.yml file (no offense taken) I am pasting it below in its entirety. The cool thing is that you can work backwards and import the file below into Application Composer to see it in the canvas:

  1AWSTemplateFormatVersion: '2010-09-09'
  2Transform: AWS::Serverless-2016-10-31
  4  ecstaskstopped:
  5    Type: AWS::Events::Rule
  6    Properties:
  7      EventPattern:
  8        source:
  9          - aws.ecs
 10        detail-type:
 11          - ECS Task State Change
 12        detail:
 13          lastStatus:
 14            - RUNNING
 15          desiredStatus:
 16            - STOPPED
 17      Targets:
 18        - Id: !GetAtt ecstaskroute53update.Name
 19          Arn: !Ref ecstaskroute53update
 20          RoleArn: !GetAtt ecstaskstoppedToecstaskroute53update.Arn
 21  ecstaskrunning:
 22    Type: AWS::Events::Rule
 23    Properties:
 24      EventPattern:
 25        source:
 26          - aws.ecs
 27        detail-type:
 28          - ECS Task State Change
 29        detail:
 30          lastStatus:
 31            - RUNNING
 32          desiredStatus:
 33            - RUNNING
 34      Targets:
 35        - Id: !GetAtt ecstaskroute53update.Name
 36          Arn: !Ref ecstaskroute53update
 37          RoleArn: !GetAtt ecstaskrunningToecstaskroute53update.Arn
 38  ecstaskroute53update:
 39    Type: AWS::Serverless::StateMachine
 40    Properties:
 41      Definition:
 42        Comment: State machine to create/update a Route53 record
 43        StartAt: DescribeNetworkInterfaces
 44        States:
 45          DescribeNetworkInterfaces:
 46            Type: Task
 47            Parameters:
 48              NetworkInterfaceIds.$: $.detail.attachments[0].details[?(].value
 49            Resource: arn:aws:states:::aws-sdk:ec2:describeNetworkInterfaces
 50            Next: ListResourceRecordSets
 51            ResultPath: $.NetworkInterfaceDescription
 52          ListResourceRecordSets:
 53            Type: Task
 54            Parameters:
 55              HostedZoneId.$: States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==HOSTEDZONEID)].Value, 0)
 56            Resource: arn:aws:states:::aws-sdk:route53:listResourceRecordSets
 57            ResultPath: $.ResourceRecordSetsOutput
 58            Next: RunningOrStopped
 59          RunningOrStopped:
 60            Type: Choice
 61            Choices:
 62              - Variable: $.detail.desiredStatus
 63                StringMatches: RUNNING
 64                Next: UpsertAction
 65              - Variable: $.detail.desiredStatus
 66                StringMatches: STOPPED
 67                Next: DeleteAction
 68            Default: DeleteAction
 69          DeleteAction:
 70            Type: Pass
 71            Next: ChangeResourceRecordSets
 72            Result:
 73              recordAction: DELETE
 74            ResultPath: $.recordActionOutput
 75          UpsertAction:
 76            Type: Pass
 77            Next: ChangeResourceRecordSets
 78            Result:
 79              recordAction: UPSERT
 80            ResultPath: $.recordActionOutput
 81          ChangeResourceRecordSets:
 82            Type: Task
 83            Parameters:
 84              ChangeBatch:
 85                Changes:
 86                  - Action.$: $.recordActionOutput.recordAction
 87                    ResourceRecordSet:
 88                      Name.$: States.Format('{}.{}', States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==aws:ecs:serviceName)].Value, 0),States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==PUBLICHOSTEDZONE)].Value, 0))
 89                      Type: A
 90                      Ttl: 60
 91                      ResourceRecords:
 92                        - Value.$: $.NetworkInterfaceDescription.NetworkInterfaces[0].Association.PublicIp
 93              HostedZoneId.$: States.ArrayGetItem($.NetworkInterfaceDescription.NetworkInterfaces[0].TagSet[?(@.Key==HOSTEDZONEID)].Value, 0)
 94            Resource: arn:aws:states:::aws-sdk:route53:changeResourceRecordSets
 95            End: true
 96      Logging:
 97        Level: ALL
 98        IncludeExecutionData: true
 99        Destinations:
100          - CloudWatchLogsLogGroup:
101              LogGroupArn: !GetAtt ecstaskroute53updateLogGroup.Arn
102      Policies:
103        - AWSXrayWriteOnlyAccess
104        - Statement:
105            - Effect: Allow
106              Action:
107                - ec2:DescribeNetworkInterfaces
108                - route53:ListResourceRecordSets
109                - route53:ChangeResourceRecordSets
110                - logs:CreateLogDelivery
111                - logs:GetLogDelivery
112                - logs:UpdateLogDelivery
113                - logs:DeleteLogDelivery
114                - logs:ListLogDeliveries
115                - logs:PutResourcePolicy
116                - logs:DescribeResourcePolicies
117                - logs:DescribeLogGroups
118              Resource: '*'
119      Tracing:
120        Enabled: true
121      Type: STANDARD
122  ecstaskroute53updateLogGroup:
123    Type: AWS::Logs::LogGroup
124    Properties:
125      LogGroupName: !Sub
126        - /aws/vendedlogs/states/${AWS::StackName}-${ResourceId}-Logs
127        - ResourceId: ecstaskroute53update
128  ecstaskstoppedToecstaskroute53update:
129    Type: AWS::IAM::Role
130    Properties:
131      AssumeRolePolicyDocument:
132        Version: '2012-10-17'
133        Statement:
134          Effect: Allow
135          Principal:
136            Service: !Sub events.${AWS::URLSuffix}
137          Action: sts:AssumeRole
138          Condition:
139            ArnLike:
140              aws:SourceArn: !Sub
141                - arn:${AWS::Partition}:events:${AWS::Region}:${AWS::AccountId}:rule/${AWS::StackName}-${ResourceId}-*
142                - ResourceId: ecstaskstopped
143      Policies:
144        - PolicyName: StartExecutionPolicy
145          PolicyDocument:
146            Version: '2012-10-17'
147            Statement:
148              - Effect: Allow
149                Action: states:StartExecution
150                Resource: !Ref ecstaskroute53update
151  ecstaskrunningToecstaskroute53update:
152    Type: AWS::IAM::Role
153    Properties:
154      AssumeRolePolicyDocument:
155        Version: '2012-10-17'
156        Statement:
157          Effect: Allow
158          Principal:
159            Service: !Sub events.${AWS::URLSuffix}
160          Action: sts:AssumeRole
161          Condition:
162            ArnLike:
163              aws:SourceArn: !Sub
164                - arn:${AWS::Partition}:events:${AWS::Region}:${AWS::AccountId}:rule/${AWS::StackName}-${ResourceId}-*
165                - ResourceId: ecstaskrunning
166      Policies:
167        - PolicyName: StartExecutionPolicy
168          PolicyDocument:
169            Version: '2012-10-17'
170            Statement:
171              - Effect: Allow
172                Action: states:StartExecution
173                Resource: !Ref ecstaskroute53update

Give Application Composer a try if you need to solve similar problems!