Submitting an AWS Batch job from AWS API Gateway

I have lately invested a lot of time keeping an eye on StackOverflow because there is a ton to learn in terms of how AWS customers are using our products. I do have all sort of filters setup and I reguarly spend a good chunk of my day there. A couple of days ago I bumped into this question that intrigued me a lot. First, because I did not even know you could trigger API calls to all those services from API Gateway (I do remember and have used the Lambda integration though) and I wanted to learn more. Second, this whole notion of containers provisioners and orchestrators is near and dear to my heart (I even have a re:Invent breakout session on that topic). But more importantly, I wanted to try to be useful to this customer.

Not being an API GW person I started to look around in the docs I did not find something immediately (which will prove to be my fault, as you will see). My next step was to ask internally and I was immediately pointed to this tutorial that demonstrates how to call Amazon Kinesis from API Gateway. This doc put me on the right track because it has 90% of what you'd need to know to launch a job on AWS Batch from API Gateway if complemented with the specific AWS Batch submitJob action reference. But that doesn't explain the many hours I lost for the remaining 10% so I am documenting it in an effort to save other people's time.

One of my awesome colleagues noted the internal question and called me to discuss (thanks Pawan P!). It turned out that this configuration was all it was required:

1Integration Type: AWS Service
2AWS Region: us-east-2
3AWS Service: Batch
4HTTP method: POST
5Action Type: Use path override
6    Path override: /v1/submitjob

Action Type and Path override is what got me to waste most of the time because the API Gateway tutorial for the Kinesis integration uses the default action name with no path override when calling Kinesis. I assume this is because the AWS Batch submitJob path is composed (but I did not investigate further).

In addition to the generic configuration above, in the Mapping Templates section you need to create Content-Type of application/json with the following template:

1application/json:
2{
3  "jobName": "arbitrary-job-name",
4  "jobQueue": "[job-queue-name]",
5  "jobDefinition": "[job-definition-name]"
6}

The jobName is anything you want to pick but jobQueue and jobDefinition neeeds to match what you configured in AWS Batch.

Also note that I did not have to set any HTTTP Headers for this integration to work(the API Gateway tutorial for the Kinesis integration suggests to set an HTTP header of Content-Type equal to 'application/x-amz-json-1.1'). I did not investigate the specific need for that header further.

While I did not explore it, it may be possible to parametrize both the job queue and job definition as part of the API Gateway endpoint path. In my MVP configuration both these parameters are hard coded in the json template.

This is how the general API Gateway integration request is configured in the console:

And this how the Mapping Templates section is configured:

It goes without saying that the region, the job queue and the job definition parameters are specific to your setup.

I have used a role (BatchFromAPIGWRole) that I have purposely crafted to have a trust relationship with apigateway.amazonaws.com and enough permissions to submit a job to Batch (the API Gateway tutorial for the Kinesis integration has more step-by-step inscructions on how to create it).

Last but not least it looks like the API Gateway resource method does not seem to care (GET or POST). What's imortant is that in the integration request configuration the HTTP method is set to POST because that is what the AWS Batch submitJob call expect.