VNF Performance Measurement using Shaker across Openstack Nodes(Blog — 3)

8 min readMay 5, 2021

This is the final blog in the three series blog, where we are extending shaker capabilities to measure the network performance of a VNF launched in the openstack platform. In the earlier blogs, we saw how to measure the interface performance of the VNF using shaker tool.

In this blog, we will move the VNF across compute nodes to see how the performance indices like latency and throughput vary for the VNF. The setup details for this are as below

2 Machines — 20.04 Ubuntu OS
Devstack stable/victoria
One controller+compute and the other only compute node
1 Controller+Compute — 8vcpus, 16GB RAM
1 Compute — 2vcpus, 4GB RAM

Testcase Topology and Details

We would use the same topology that we have been using for Blog — 1 and Blog — 2

In the testcase for the first execution all the 3 VMs will be launched on the same compute node, in our case controller+compute node(harrypotter). And in the second execution we will move the VyOS VNF to the other compute node(hermionegranger). Below are the execution scenarios captured.

Execution scenarios for the shaker testcase

Prerequisite

It is expected that the below configurations are already in place for this execution scenarios

Shaker binary installed
Shaker image and Shaker flavor available in openstack setup
VyOS image uploaded to glance
Network, Router and Port configuration are already done in the openstack setup. This includes creation of mgmt, east and west networks. Router creation with mgmt network as interface and public network as gateway. Finally the east and west port creation for the VyOS VNF with allowed address pair configuration enabled. Please refer to Blog — 1 and Blog — 2 for detailed steps. Below are the commands for reference

Management network creation

# openstack network create mgmt

# openstack subnet create mgmt_subnet --network mgmt --subnet-range 20.0.0.0/24

East network creation

# openstack network create east

# openstack subnet create east_subnet --network east --subnet-range 30.0.0.0/24

West network creation

# openstack network create west

# openstack subnet create west_subnet --network west --subnet-range 40.0.0.0/24

Router configuration

# openstack router create router

# neutron router-gateway-set router public

# openstack router add subnet router mgmt_subnet

Port configuration

# openstack port create east_router --fixed-ip ip-address=30.0.0.10 --network east --allowed-address ip-address=0.0.0.0/0

# openstack port create west_router --fixed-ip ip-address=40.0.0.10 --network west --allowed-address ip-address=0.0.0.0/0

For both the scenarios we will be using the same HOT and shaker configuration files as specified below. Copy of the same can be found here. Update the network configuration details in the HOT template before execution.

# VyOS testcase HOT templateheat_template_version: 2013-05-23description:
  Primary instances and minion instances are connected into different networks. The cloud-init configuration helps connects these instances to VyOS VM.parameters:
  net_id:
    type: string
    description: Network Id of management server
    default: "e12265ed-68a0-45c8-8652-fd41b2594f57"
  image:
    type: string
    description: Name of image to use for servers
  flavor:
    type: string
    description: Flavor to use for servers
  external_net:
    type: string
    description: ID or name of external network for which floating IP addresses will be allocated
  server_endpoint:
    type: string
    description: Server endpoint address
  dns_nameservers:
    type: comma_delimited_list
    description: DNS nameservers for the subnets
  east_private_net:
    type: string
    description: NetId of east network
    default: "4bdc7b4e-e1d9-4a36-9dc6-0dc3260b2313"
  east_private_subnet:
    type: string
    description: SubnetId of east network
    default: "cfa98799-f769-4965-861e-0eec405bc849"
  east_ip_address:
    type: string
    description: Ipaddress for east network
    default: "30.0.0.25"
  west_private_net:
    type: string
    description: NetId of west network
    default: "7f5f0429-7453-49b7-86cf-4f013fddbae4"
  west_private_subnet:
    type: string
    description: SubnetId of west network
    default: "33736de5-ccd9-4903-b6f0-0b272e66eda3"
  west_ip_address:
    type: string
    description: Ipaddress for west network 
    default: "40.0.0.25"resources:
  server_security_group:
    type: OS::Neutron::SecurityGroup
    properties:
      rules: [
        {remote_ip_prefix: 0.0.0.0/0,
        protocol: tcp,
        port_range_min: 1,
        port_range_max: 65535},
        {remote_ip_prefix: 0.0.0.0/0,
        protocol: udp,
        port_range_min: 1,
        port_range_max: 65535},
        {remote_ip_prefix: 0.0.0.0/0,
        protocol: icmp}]east_agent_port:
    type: OS::Neutron::Port
    properties:
      network_id: { get_param: east_private_net }
      fixed_ips:
        - subnet_id: { get_param: east_private_subnet }
          ip_address: { get_param: east_ip_address }
      security_groups: [{ get_resource: server_security_group }]
  west_agent_port:
    type: OS::Neutron::Port
    properties:
      network_id: { get_param: west_private_net }
      fixed_ips:
        - subnet_id: { get_param: west_private_subnet }
          ip_address: { get_param: west_ip_address }
      security_groups: [{ get_resource: server_security_group }]test_primary_0:
    type: OS::Nova::Server
    properties:
      name: test_primary_0
      image: { get_param: image }
      flavor: { get_param: flavor }
      key_name: test
      networks:
        - network: { get_param: net_id }
        - port: { get_resource: east_agent_port }
      user_data_format: RAW
      user_data:
        str_replace:
          template: |
            #!/bin/sh
            sudo dhclient ens4
            sudo ip r d default via 30.0.0.1
            sudo ip r a 40.0.0.0/24 via 30.0.0.10 dev ens4
            screen -dmS shaker-agent-screen shaker-agent --server-endpoint=$SERVER_ENDPOINT --agent-id=$AGENT_ID --agent-socket-recv-timeout 10 --agent-socket-send-timeout 10
          params:
            "$SERVER_ENDPOINT": { get_param: server_endpoint }
            "$AGENT_ID": "test_primary_0"test_minion_0:
    type: OS::Nova::Server
    properties:
      name: test_minion_0
      image: { get_param: image }
      flavor: { get_param: flavor }
      key_name: test
      networks:
        - network: { get_param: net_id }
        - port: { get_resource: west_agent_port }
      user_data_format: RAW
      user_data:
        str_replace:
          template: |
            #!/bin/sh
            sudo dhclient ens4
            sudo ip r d default via 40.0.0.1
            sudo ip r a 30.0.0.0/24 via 40.0.0.10 dev ens4
            screen -dmS shaker-agent-screen shaker-agent --server-endpoint=$SERVER_ENDPOINT --agent-id=$AGENT_ID --agent-socket-recv-timeout 10 --agent-socket-send-timeout 10
          params:
            "$SERVER_ENDPOINT": { get_param: server_endpoint }
            "$AGENT_ID": "test_minion_0"outputs:
  test_primary_0:
    value: { get_attr: [ test_primary_0, instance_name ] }
  test_minion_0:
    value: { get_attr: [ test_minion_0, instance_name ] }
  test_primary_0_ip:
    value: { get_attr: [ test_primary_0, networks, east, 0 ] }
  test_minion_0_ip:
    value: { get_attr: [ test_minion_0, networks, west, 0 ] }

The below file defines the shaker testcase execution parameters

# VyOS testcase execution yamltitle: OpenStack VNF testdescription:
  In this scenario Shaker launches pairs of instances. Instances are
  connected to one of 2 tenant networks, which plugged into the router VNF.
  The traffic goes from one network to the other (L3 east-west).deployment:
  template: vyos_test.hot
  accommodation: [pair, best_effort]execution:
  progression: quadratic
  tests:
  -
    title: Download
    class: flent
    method: tcp_download
  -
    title: Upload
    class: flent
    method: tcp_upload
  -
    title: Bi-directional
    class: flent
    method: tcp_bidirectional

Testcase Scenario 1

For the first scenario, we will create the VNF on the harrypotter node by manually specifying the availability-zone in the openstack cli. And the primary and minion VMs get launched on the harrypotter node itself.

# openstack server create --image vyos --flavor m1.small --nic net-id=<mgmt_net_id> --nic port-id=<east_router_port_id> --nic port-id=<west_router_port_id> --availability-zone nova:harrypotter vyos_router

The arguments used here are:

east_router_port_id — port id of the east_router port created above
west_router_port_id — port id of the west_router port created above
image vyos—VyOS image name or id already uploaded to glance

After the VM boots up successfully, we need to configure the interfaces on the VyOS using the below commands

We would now connect to the VM through the horizon console and configure the interfaces. The default username and password to connect to the VM are vyos/vyos. Once connected we execute the below commands.

$ configure
# set interfaces ethernet eth0 address dhcp
# set interfaces ethernet eth1 address dhcp
# set interfaces ethernet eth2 address dhcp
# commit;save;exit

Now that the VyOS VNF and configuration are in place, we will now execute the shaker testcase with the below command.

# shaker --server-endpoint <host_ip:port> --scenario vyos_test.yaml --report harrypotter_test.html --stack-name test

Replace <host_ip:port> with the host_ip and port of your host.

VM details and placement in openstack setup

All the VMs are launched on the single compute node. The testcase output results are compared below

TestCase Scenario 2

In this scenario, we will launch the VyOS VNF on the hermionegranger node and the minion and primary VMs from shaker will be launched on harrypotter node. When the shaker testcase is executing the traffic would flow from one node to another node as the VyOS VNF is deployed on another node. Hence we expect some latency and throughput differences as compared to the first scenario.

To launch the VyOS VNF

After the VM boots up successfully, perform the configuration steps to bring up the interfaces

$ configure
# set interfaces ethernet eth0 address dhcp
# set interfaces ethernet eth1 address dhcp
# set interfaces ethernet eth2 address dhcp
# commit;save;exit

Once the configuration is complete, we execute the shaker testcase using the below command.

# shaker --server-endpoint <host_ip:port> --scenario vyos_test.yaml --report harrypotter_test.html --stack-name test

Replace <host_ip:port> with the host_ip and port of your host.

Below is the topology and VM placement details in openstack for this scenario. We can observe the VyOS router VM is placed on hermionegranger node and the other VMs on harrypotter node.

Testcases Result Comparison

Below is the table comparing the shaker output results from the above scenarios. Copy of the results are available here.

From the above test results we can observe that when the VyOS VNF is launched on a different node than that of minion and primary VMs there is sharp rise in latency and throughputs(ping and download/upload speeds).

By these test scenarios we can learn how the VNF behaves when launched on different nodes and based on the observations we can select the optimal host which best suits our VNF.

Conclusion

The performance test measure is easy when we use shaker tool, it greatly has made out task easy. We can as well extend this use case with the various features available in shaker like deploying multiple minion and primary VMs. We can as well use the accommodation feature and place the minion and primary VMs appropriately as well. It also gives us flexibility to test using different performance measurement tools like iperf, netperf, iperf3, flent or custom tool using shell.

In many ways using shaker will easy our task to test the VNF performance. From the results observed we would be able to place the VNF optimally and would favor us in getting the best performance according to our requirement.