So let’s start with the FP9300 chassis and the 3 node ASA cluster I already have up and running.

• “Homer” is the name of the FP9300 in my lab. Yes, it is nice to have fun while one works and have fun names for devices.
• “Krusty” is my N9K on the left that is connected to Homer over port-channel 14.
• “Brockman” is the N9K on the right connected to Homer over port-channel 24.
• “Brockman” is also the switch that I am using for the current CCL link (Homer e3/3 and e3/4 on Brockman e1/5-6)

If you are wondering what the devices are on the far left and far right, those are a Spirent Test Center chassis I have connected to run traffic through. I think I have some blogs about the STC (Spirent Test Center) but I for sure know I have a youtube playlist for Spirent stuff I have done.

The above is the setup I was doing for some tests I was running in the CPOC lab (customer proof of concept). After the testing with the 1 FP9300, 3 SM48 security modules and 3 node Intra-chassis ASA cluster I was supposed to then add an additional FP9300, 3 SM48s and add 3 more ASAs to the cluster.

So what I WANT is to have the environment below.

Overview of Steps

Note: Prior to doing the below I also made the FXOS versions of the 2 FP9300s the same. Also uploaded onto the new FP9300 (Marge) the code version I want to use for the ASAs over on there.

The first 3 steps #1 thru #3 we will do and not cover in here. In this blog we will cover #4 thru #8.

1. Cable up the DATA links on the new FP9300 to the 2 N9Ks as per above for Port Channel 14 and Port Channel 24
2. Cable up the CCL links on the new FP9300 to Brockman E1/6 and E1/7
3. Configure the 2 N9Ks to include these new interfaces into PortChannels 14, 24, and 48.
4. FP9300-2: Setup the interfaces in the Chassis Manager on Marge for 3 port channels .
5. FP9300-1: Get config for the ASA cluster
6. FP9300-2: Add a Cluster, Joining Existing, and Copy and Paste ASA Cluster Config In
7. FP9300-2: Notice The New ASAs are Unhappy
8. Fix It. 🙂

#1 thru #3 above are done as per the diagram. Before we move to #4 let’s take a quick look at FP9300-1 (Homer) and how it is setup.

Screen Shots from FP9300-1 (Homer)

Looking at above we can see we have 3 port-channels in the original FP9300. Two are Data port channels (14 and 24) and the remaining port channel (48) is the Cluster (CCL) port-channel for the cluster.

In the screen capture above from the original FP9300-1 (Homer) we can see the 3 node ASA cluster that is already working.

Time to start playing!

1. Cable up the DATA links on the new FP9300 to the 2 N9Ks as per above for Port Channel 14 and Port Channel 24
2. Cable up the CCL links on the new FP9300 to Brockman E1/6 and E1/7
3. Configure the 2 N9Ks to include these new interfaces into PortChannels 14, 24, and 48.
4. FP9300-2: Setup the interfaces in the Chassis Manager on Marge for 3 port channels .
5. FP9300-1: Get config for the ASA cluster
6. FP9300-2: Add a Cluster, Joining Existing, and Copy and Paste ASA Cluster Config In
7. FP9300-2: Notice The New ASAs are Unhappy
8. Fix It. 🙂

With 1, 2, and 3 already done and an overview of how FP9300-1 is already setup. Let’s move to #4.

#4: FP9300-2: Setup the interfaces in the Chassis Manager on Marge for 3 port channels

Let’s go to the FP9300-2 Chassis manager and get the port-channels setup properly. This one is pretty straightforward. We can see the 3 port channels in FP9300-2 chassis manager properly setup up as below.

I thought the next part (building the 3 ASAs on the new FP9300) would be more time consuming that it actually was. It was so easy! Thank you to my colleague, Per Hagen, for teaching me about this.

I think it went like this
Me: “Yo, Per, I want to add a 2nd FP9300 to my existing 3 node ASA Intra-chassis cluster.”

Per: “Oh Fish, that is easy, do the following two things” –

• #5: FP9300-1: Get config for the ASA cluster
• #6: FP9300-2: Add a Cluster, Joining Existing, and Copy and Paste ASA Cluster Config In

#5: FP9300-1: Get config for the ASA cluster

Go into the original FP9300 and into the logical devices. Over on the right side click the 3 buttons and go to “show configuration”.

A pop up window will open. Do as it says. Just copy everything to clipboard.

#6: FP9300-2: Add a Cluster, Joining Existing, and Copy and Paste ASA Cluster Config In

Now go over into the new FP9300 and go into logical devices.

In the upper right hand corner click “add” and then “cluster”. When the pop up in the middle opens in the “I Want to:” drop down select “Join Existing Cluster” and give it a name. Click okay.

You will now get a pop up to put the ASA config you copied into clipboard into here.

Yes. :). Paste in from the clipboard what you grabbed from FP9300-1 and click “ok”

You are now in something you are likely already familiar with – Provisioning. So you might think to yourself – “but I just put the config in from the other one, why am i here?”

Well if you notice below the 3 port channels are already selected. You don’t have to do that.

Let’s click on the ASA box itself. See? Other stuff is already populated also. Cool huh? The only two thing that are different between the window that popped up (left side) and the right side is that on the right side I made it chassis ID 2, and I also confirmed the cluster key. Then just click “ok“ bottom right. Once I did that on the screen capture on the right hand side, it brought me back to the provisioning page you see above. Click “Save” in the upper right hand corner.

And voila! Our 3 new ASAs we wanted to add to the new FP9300 are now installing. Woot!

So first – thank you again, Per. That is cool.

Next. Are we done? lol. Well not exactly. If you recall correctly I had 8 steps.

• #7: FP9300-2: Notice The New ASAs are Unhappy
• #8: Fix It. 🙂

#7: FP9300-2: Notice The New ASAs are Unhappy

To be honest I kicked off the install at night time and was all ready to play and send traffic the next morning and start the testing.

Imagine my surprise when I went into one of the ASAs on the new FP9300-2 and saw the following –

Huh? What do you mean my cluster control link is down? I majorly verified ALL cables were going EXACTLY where I expected dang it! Hmmmm…. didn’t I?

So yeah, I hit my head against the wall a little on my own verifying a ton of stuff. Verifying the ports physically, verifying again how I had the cluster (CCL) port channel defined in the newly added FP9300, verifying port-channel 48 config on the N9K (Brockman) that was working PERFECTLY fine for the original 3 node cluster on FP9300-1.

#8: Fix It 🙂

I chased all the “usual suspects” and then finally went to another colleague – Keith Brister.

Me: “Keith, I know you are super busy. I just don’t know what I am doing wrong. My interfaces on the switch to FP9300-2 are suspended.”

Keith: “Um, Fish, the data ports are shared across the FP9300s but not the CCL. On the FP9300s they will be configured for port-channel 48. But on your N9K switch you need two different port channels.”

You know the embarrassing part about that? lol. I knew that and completely forgot! I mean, if they were in a port-channel how does a keepalive from FP9300-1 go out, and then to the switch and then go out to FP9300-2 if the switch has has them both on the same PortChannel?

So, back to the N9K switch. Keep E1/5 and E1/6 (cabled to FP9300-1) on that switch as being in Port Channel 48. Put E1/7 and E1/8 (cabled to FP93000-2) into a different PortChannel.

Voila! Sorted.

And there you have it! Our new 6 node ASA cluster across 2 FP9300s and 6 SM48s. And below you can see the traffic I am sending spraying across.