Recently a customer asked me whether it would be possible to enable their developers to deploy VPC's themselves and automatically provision the Aviatrix components, through the AWS Service Catalog. So I set about exploring and figuring out a way this could be achieved.
In this post, we will explore how we can expand on this aws-sample that uses Terraform for deployment, to include the deployment of Aviatrix components. If you're unfamiliar with this sample, stop reading now and go through that first! Once you have that deployed and mastered, come back to this post and we will build on top of that.
...right, welcome back!
Now that we have a baseline deployment, we can extend with our Aviatrix related Terraform code. One note to take here, is the fact that the aws-sample describes that only the AWS provider is supported. As this is just running terraform in a regular way on an ec2 instance, we can integrate any 3rd party provider just fine though...
So here are the steps that we will have to take to accomplish our goal:
Set up AWS Secret with our controller credentials
One of the first challenges compared to the AWS product examples that were provided with the aws-sample, is the fact that we need to talk to the Aviatrix controller. This requires authentication credentials, whereas the AWS code could deploy using the EC2 role attached to the instance. We don't want to have these credentials in our Terraform code, so in stead we will have Terraform access them on deployment time from AWS Secrets Manager.
First we need to create our secret. Browse to Secrets Manager and add a new secret:
Add 3 key/value pairs with the following details and hit next:
Provide a descriptive name (and remember this) for your secret and set any other settings, like tags to your liking and hit next:
We are not configuring auto rotation, so you can skip that step and click next and after reviewing your settings, hit store. Once it is created, open it and copy the secret's ARN for later use.
Set up IAM permissions to access controller secret from our Terraform Fulfillment server
The fulfillment server will run Terraform using the "TerraformResourceCreationRole" IAM role. In order for Terraform to access our controller credentials we need to create an IAM policy with those permissions.
Go to IAM in the AWS console and create a new policy under policies. Insert the following snippet where you replace the resource ARN with the one we copied in our previous step.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
],
"Resource": "arn:aws:secretsmanager:************************"
}
]
}It should look like this:
Finalize the creation of the policy with any tags if you like and give it a descriptive name in the last step. I'm using "AviatrixControllerSecret".
Once we have created the policy, we need to add it to the "TerraformResourceCreationRole" role. Go to roles, find the role and click it to edit it. Under permissions, click the "Add permissions" button and choose "Attach policies". Find the policy you just created and attach it. Our role should now look like this:
Alright, that should sort our privileges. Be aware if you set this up multi account, you may have to make some adjustments to the IAM structure.
Allow access to our controller from the Fulfillment server
By default, the Aviatrix controller should not be accessible from the outside world. So we have to make sure our Terraform Fulfillment server can talk to the controller. Youi can set this up over private connectivity, e.g. over VPC peering, Aviatrix dataplane, run the fulfillment server in the same VPC as the controller, or over public access. In this example we will use public access, but in either case you need to update the security group applied to the controller.
In the EC2 console, find your Aviatrix controller and go to the security settings. Where you should see the security groups attached to the instance:
Assuming you have Security Group Management enabled, you would see the security groups in the green box. Don't alter these, as they are maintained by the Aviatrix controller for gateway to controller communication.
Click on the entry marked in blue, and add an inbound rule, using the IP address of the NAT Gateway that the Terraform fulfillment server uses to communicate to the outside world as the source. Make sure HTTPS is allowed. Assuming our NAT Gateway has IP 1.1.1.1, here's an example of what the rule should look like:
Once saved, we have our access sorted as well.
Write our Terraform code and CloudFormation wrapper
For easy copy and pasting, check the full code example. This chapter goes through the code section by section.
Let's start with our Terraform code.
We need to define our required providers and configure them:
We set AWS and Aviatrix as the required providers and we can set version constraints.
Next we configure the AWS provider to use the region variable, as the region where to execute. After that a couple of more interesting things are happening. We need to configure the Aviatrix provider with our access credentials. But in order to do so, we first need to pull them from the secrets manager. That's what the two data sources accomplish. As you can see, we provide the name of the secret, as we stored it in the Secrets Manager.
Next we define our input variables that we will need to receive from the CloudFormation wrapper:
And finally we define the resources that we want to create. We're using the Aviatrix spoke module to make our life easy:
I have hardcoded for deployment in AWS only and a single gateway (no ha). You can alter this to your needs. We store the above code snippers in a file called aviatrix-spoke.tf.
Tip: As all of the above code is deployed by the Aviatrix controller, nothing is preventing you from deploying this in other clouds like Azure or GCP from the AWS Service Catalog!
Next we need the CloudFormation wrapper, which allows us to deploy this as a service catalog item. This wrapper needs to gather the required information, and pass it on for Terraform execution. Below I highlight the interesting parts, as this is largely identical to the wrappers used in the aws-sample.
We define our input parameters, which will show up as user input fields when our user will deploy this item from the catalog:
As you can see, we are not asking the user which transit gateway to use, as we want to derive that from the region where the user deploys their VPC. In order to do so, we have a map where we can look up our transit gateway name based on a region:
In order to feed the variables to Terraform, we configure them in the Terraform stack resource:
As you can see, we feed the user input directly into the stack, whereas the transitGateway name gets looked up in the TransitMap based on the selected region. Make sure to use the full code example and store it as aviatrix-spoke.json.
Upload the Terraform code and wrapper to S3
Now we have the files ready, we need to upload them to the s3 bucket that was created as part of the aws-sample. Browse to the S3 console and open the bucket named "terraformarchitecture-single-terraformconfigstore-***************". Drag the aviatrix-spoke.tf and aviatrix-spoke.json files into this bucket. We should now have them visible:
Click on aviatrix-spoke.json and find and copy the object URL. We will need that to add it to our product catalog later.
Add our product to the AWS Service catalog
Now that we have prepared our catalog item, lets add it. Go to the Service Catalog console, and check your portfolio's. We should have the "SC Sample TF Portfolio" that was provided with the aws-sample. Lets open it, and click upload a new product.
We're going to create our catalog item, with the following parameters:
Product name: Aviatrix Spoke VPC
Owner: Your name
Version Details: Use a CloudFormation template and add the URL we just copied as shown below:
Leave other field blank or configure to your preference and save it. We should now see it appear in our portfolio:
All good!
Deploy our VPC
Assuming that you have already shared the portfolio with an account or Organizational Node (Under actions in Portfolio), we can now go ahead and deploy a VPC to see how it all comes together!
Browse to the Service Catalog, Products. We can see the products in the service catalogs that are shared with us:
Select "Aviatrix Spoke VPC" and hit the "Launch Product" button. Now we just have to fill out the required fields, to deploy our new spoke VPC!
Hit "Launch product" at the bottom of the page, and sit back. We can see resources are getting created:
If we look on the back-end, we can see it happening on the Aviatrix controller:
And after a while, once deployment is done, we can see the status reflected in our provisioned product:
There are many points where this example can be enhanced, for example using custom CloudFormation resources to do automatic CIDR allocation in combination with an IPAM solution, or to automatically determine the right access account to deploy against. However, I hope this post provides you with a good starting point for your own deployment!
Full code Example
aviatrix-spoke.tf:
terraform {
required_providers {
aws = {}
aviatrix = {
source = "AviatrixSystems/aviatrix"
version = "2.21.2"
}
}
}
provider "aws" {
region = var.region
}
data "aws_secretsmanager_secret" "controller" {
name = "AviatrixController66"
}
data "aws_secretsmanager_secret_version" "controller" {
secret_id = data.aws_secretsmanager_secret.controller.id
}
provider "aviatrix" {
controller_ip = jsondecode(data.aws_secretsmanager_secret_version.controller.secret_string)["controller_ip"]
username = jsondecode(data.aws_secretsmanager_secret_version.controller.secret_string)["username"]
password = jsondecode(data.aws_secretsmanager_secret_version.controller.secret_string)["password"]
}
variable "name" {
type = string
}
variable "cidr" {
type = string
}
variable "region" {
type = string
}
variable "account" {
type = string
}
variable "transitGateway" {
type = string
}
module "spoke_aws_1" {
source = "terraform-aviatrix-modules/mc-spoke/aviatrix"
version = "1.1.2"
cloud = "AWS"
ha_gw = false
name = var.name
cidr = var.cidr
region = var.region
account = var.account
transit_gw = var.transitGateway
}aviatrix-spoke.json:
{
"Parameters": {
"name": {
"Type": "String",
"Description": "Name of the Spoke VPC and Gateways"
},
"cidr": {
"Type": "String",
"Description": "Cidr for the VPC"
},
"region": {
"Type": "String",
"Description": "AWS Region to deploy in.",
"AllowedValues": [
"eu-central-1",
"us-east-1",
"eu-west-1"
]
},
"account": {
"Type": "String",
"Description": "Access account on the Aviatrix controller",
"AllowedValues": [
"AWS",
"AWS2",
"AWS3"
]
}
},
"Mappings": {
"TransitMap": {
"eu-central-1": {
"transitGateway": "avx-eu-central-1-transit"
},
"eu-west-1": {
"transitGateway": "avx-eu-west-1-transit"
},
"us-east-1": {
"transitGateway": "avx-us-east-1-transit"
}
}
},
"Resources": {
"MyTerraformStack": {
"Type": "Custom::TerraformStack",
"Properties": {
"ServiceToken": {
"Fn::Sub": "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:TerraformLaunchHandler"
},
"TerraformArtifactUrl": {
"Fn::Sub": [
"https://${configurl}/aviatrix-spoke.tf",
{
"configurl": {
"Fn::ImportValue": "TerraformConfigBucket"
}
}
]
},
"LaunchRoleArn": {
"Fn::Sub": "arn:aws:iam::${AWS::AccountId}:role/TerraformResourceCreationRole"
},
"TerraformVariables": {
"name": {
"Ref": "name"
},
"cidr": {
"Ref": "cidr"
},
"region": {
"Ref": "region"
},
"account": {
"Ref": "account"
},
"transitGateway": {
"Fn::FindInMap": [
"TransitMap",
{
"Ref": "region"
},
"transitGateway"
]
}
}
}
}
},
"Outputs": {
"ScriptOutput": {
"Value": {
"Fn::GetAtt": [
"MyTerraformStack",
"TerraformScriptOutputLocation"
]
}
},
"MyOutputVariables": {
"Value": {
"Fn::GetAtt": [
"MyTerraformStack",
"Outputs"
]
}
}
}
}
Comments