VNF Performance Measurement using Shaker across Openstack Nodes(Blog — 3)

Blog — 1 and Blog — 2

This is the final blog in the three series blog, where we are extending shaker capabilities to measure the network performance of a VNF launched in the openstack platform. In the earlier blogs, we saw how to measure the interface performance of the VNF using shaker tool.

In this blog, we will move the VNF across compute nodes to see how the performance indices like latency and throughput vary for the VNF. The setup details for this are as below

  • 2 Machines — 20.04 Ubuntu OS
  • Devstack stable/victoria
  • One controller+compute and the other only compute node
  • 1 Controller+Compute — 8vcpus, 16GB RAM
  • 1 Compute — 2vcpus, 4GB RAM

Testcase Topology and Details

We would use the same topology that we have been using for Blog — 1 and Blog — 2

Testcase topology

In the testcase for the first execution all the 3 VMs will be launched on the same compute node, in our case controller+compute node(harrypotter). And in the second execution we will move the VyOS VNF to the other compute node(hermionegranger). Below are the execution scenarios captured.

Execution scenarios for the shaker testcase

Prerequisite

It is expected that the below configurations are already in place for this execution scenarios

  • Shaker binary installed
  • Shaker image and Shaker flavor available in openstack setup
  • VyOS image uploaded to glance
  • Network, Router and Port configuration are already done in the openstack setup. This includes creation of mgmt, east and west networks. Router creation with mgmt network as interface and public network as gateway. Finally the east and west port creation for the VyOS VNF with allowed address pair configuration enabled. Please refer to Blog — 1 and Blog — 2 for detailed steps. Below are the commands for reference

Management network creation

# openstack network create mgmt

# openstack subnet create mgmt_subnet --network mgmt --subnet-range 20.0.0.0/24

East network creation

# openstack network create east

# openstack subnet create east_subnet --network east --subnet-range 30.0.0.0/24

West network creation

# openstack network create west

# openstack subnet create west_subnet --network west --subnet-range 40.0.0.0/24

Router configuration

# openstack router create router

# neutron router-gateway-set router public

# openstack router add subnet router mgmt_subnet

Port configuration

# openstack port create east_router --fixed-ip ip-address=30.0.0.10 --network east --allowed-address ip-address=0.0.0.0/0

# openstack port create west_router --fixed-ip ip-address=40.0.0.10 --network west --allowed-address ip-address=0.0.0.0/0

For both the scenarios we will be using the same HOT and shaker configuration files as specified below. Copy of the same can be found here. Update the network configuration details in the HOT template before execution.

# VyOS testcase HOT templateheat_template_version: 2013-05-23description:
Primary instances and minion instances are connected into different networks. The cloud-init configuration helps connects these instances to VyOS VM.
parameters:
net_id:
type: string
description: Network Id of management server
default: "e12265ed-68a0-45c8-8652-fd41b2594f57"
image:
type: string
description: Name of image to use for servers
flavor:
type: string
description: Flavor to use for servers
external_net:
type: string
description: ID or name of external network for which floating IP addresses will be allocated
server_endpoint:
type: string
description: Server endpoint address
dns_nameservers:
type: comma_delimited_list
description: DNS nameservers for the subnets
east_private_net:
type: string
description: NetId of east network
default: "4bdc7b4e-e1d9-4a36-9dc6-0dc3260b2313"
east_private_subnet:
type: string
description: SubnetId of east network
default: "cfa98799-f769-4965-861e-0eec405bc849"
east_ip_address:
type: string
description: Ipaddress for east network
default: "30.0.0.25"
west_private_net:
type: string
description: NetId of west network
default: "7f5f0429-7453-49b7-86cf-4f013fddbae4"
west_private_subnet:
type: string
description: SubnetId of west network
default: "33736de5-ccd9-4903-b6f0-0b272e66eda3"
west_ip_address:
type: string
description: Ipaddress for west network
default: "40.0.0.25"
resources:
server_security_group:
type: OS::Neutron::SecurityGroup
properties:
rules: [
{remote_ip_prefix: 0.0.0.0/0,
protocol: tcp,
port_range_min: 1,
port_range_max: 65535},
{remote_ip_prefix: 0.0.0.0/0,
protocol: udp,
port_range_min: 1,
port_range_max: 65535},
{remote_ip_prefix: 0.0.0.0/0,
protocol: icmp}]
east_agent_port:
type: OS::Neutron::Port
properties:
network_id: { get_param: east_private_net }
fixed_ips:
- subnet_id: { get_param: east_private_subnet }
ip_address: { get_param: east_ip_address }
security_groups: [{ get_resource: server_security_group }]
west_agent_port:
type: OS::Neutron::Port
properties:
network_id: { get_param: west_private_net }
fixed_ips:
- subnet_id: { get_param: west_private_subnet }
ip_address: { get_param: west_ip_address }
security_groups: [{ get_resource: server_security_group }]
test_primary_0:
type: OS::Nova::Server
properties:
name: test_primary_0
image: { get_param: image }
flavor: { get_param: flavor }
key_name: test
networks:
- network: { get_param: net_id }
- port: { get_resource: east_agent_port }
user_data_format: RAW
user_data:
str_replace:
template: |
#!/bin/sh
sudo dhclient ens4
sudo ip r d default via 30.0.0.1
sudo ip r a 40.0.0.0/24 via 30.0.0.10 dev ens4
screen -dmS shaker-agent-screen shaker-agent --server-endpoint=$SERVER_ENDPOINT --agent-id=$AGENT_ID --agent-socket-recv-timeout 10 --agent-socket-send-timeout 10
params:
"$SERVER_ENDPOINT": { get_param: server_endpoint }
"$AGENT_ID": "test_primary_0"
test_minion_0:
type: OS::Nova::Server
properties:
name: test_minion_0
image: { get_param: image }
flavor: { get_param: flavor }
key_name: test
networks:
- network: { get_param: net_id }
- port: { get_resource: west_agent_port }
user_data_format: RAW
user_data:
str_replace:
template: |
#!/bin/sh
sudo dhclient ens4
sudo ip r d default via 40.0.0.1
sudo ip r a 30.0.0.0/24 via 40.0.0.10 dev ens4
screen -dmS shaker-agent-screen shaker-agent --server-endpoint=$SERVER_ENDPOINT --agent-id=$AGENT_ID --agent-socket-recv-timeout 10 --agent-socket-send-timeout 10
params:
"$SERVER_ENDPOINT": { get_param: server_endpoint }
"$AGENT_ID": "test_minion_0"
outputs:
test_primary_0:
value: { get_attr: [ test_primary_0, instance_name ] }
test_minion_0:
value: { get_attr: [ test_minion_0, instance_name ] }
test_primary_0_ip:
value: { get_attr: [ test_primary_0, networks, east, 0 ] }
test_minion_0_ip:
value: { get_attr: [ test_minion_0, networks, west, 0 ] }

The below file defines the shaker testcase execution parameters

# VyOS testcase execution yamltitle: OpenStack VNF testdescription:
In this scenario Shaker launches pairs of instances. Instances are
connected to one of 2 tenant networks, which plugged into the router VNF.
The traffic goes from one network to the other (L3 east-west).
deployment:
template: vyos_test.hot
accommodation: [pair, best_effort]
execution:
progression: quadratic
tests:
-
title: Download
class: flent
method: tcp_download
-
title: Upload
class: flent
method: tcp_upload
-
title: Bi-directional
class: flent
method: tcp_bidirectional

Testcase Scenario 1

For the first scenario, we will create the VNF on the harrypotter node by manually specifying the availability-zone in the openstack cli. And the primary and minion VMs get launched on the harrypotter node itself.

# openstack server create --image vyos --flavor m1.small --nic net-id=<mgmt_net_id> --nic port-id=<east_router_port_id> --nic port-id=<west_router_port_id> --availability-zone nova:harrypotter vyos_router

The arguments used here are:

  • east_router_port_id — port id of the east_router port created above
  • west_router_port_id — port id of the west_router port created above
  • image vyos—VyOS image name or id already uploaded to glance

After the VM boots up successfully, we need to configure the interfaces on the VyOS using the below commands

We would now connect to the VM through the horizon console and configure the interfaces. The default username and password to connect to the VM are vyos/vyos. Once connected we execute the below commands.

$ configure
# set interfaces ethernet eth0 address dhcp
# set interfaces ethernet eth1 address dhcp
# set interfaces ethernet eth2 address dhcp
# commit;save;exit

Now that the VyOS VNF and configuration are in place, we will now execute the shaker testcase with the below command.

# shaker --server-endpoint <host_ip:port> --scenario vyos_test.yaml --report harrypotter_test.html --stack-name test

Replace <host_ip:port> with the host_ip and port of your host.

Topology of the testcase
VM details and placement in openstack setup

All the VMs are launched on the single compute node. The testcase output results are compared below

TestCase Scenario 2

In this scenario, we will launch the VyOS VNF on the hermionegranger node and the minion and primary VMs from shaker will be launched on harrypotter node. When the shaker testcase is executing the traffic would flow from one node to another node as the VyOS VNF is deployed on another node. Hence we expect some latency and throughput differences as compared to the first scenario.

To launch the VyOS VNF

# openstack server create --image vyos --flavor m1.small --nic net-id=<mgmt_net_id> --nic port-id=<east_router_port_id> --nic port-id=<west_router_port_id> --availability-zone nova:hermionegranger vyos_router

After the VM boots up successfully, perform the configuration steps to bring up the interfaces

$ configure
# set interfaces ethernet eth0 address dhcp
# set interfaces ethernet eth1 address dhcp
# set interfaces ethernet eth2 address dhcp
# commit;save;exit

Once the configuration is complete, we execute the shaker testcase using the below command.

# shaker --server-endpoint <host_ip:port> --scenario vyos_test.yaml --report harrypotter_test.html --stack-name test

Replace <host_ip:port> with the host_ip and port of your host.

Below is the topology and VM placement details in openstack for this scenario. We can observe the VyOS router VM is placed on hermionegranger node and the other VMs on harrypotter node.

Testcase topology
VM placement in openstack setup

Testcases Result Comparison

Below is the table comparing the shaker output results from the above scenarios. Copy of the results are available here.

Test results from shaker output

From the above test results we can observe that when the VyOS VNF is launched on a different node than that of minion and primary VMs there is sharp rise in latency and throughputs(ping and download/upload speeds).

By these test scenarios we can learn how the VNF behaves when launched on different nodes and based on the observations we can select the optimal host which best suits our VNF.

Conclusion

The performance test measure is easy when we use shaker tool, it greatly has made out task easy. We can as well extend this use case with the various features available in shaker like deploying multiple minion and primary VMs. We can as well use the accommodation feature and place the minion and primary VMs appropriately as well. It also gives us flexibility to test using different performance measurement tools like iperf, netperf, iperf3, flent or custom tool using shell.

In many ways using shaker will easy our task to test the VNF performance. From the results observed we would be able to place the VNF optimally and would favor us in getting the best performance according to our requirement.

References

  1. https://vyos.io/
  2. https://docs.vyos.io/en/latest/
  3. https://flent.org/
  4. https://docs.openstack.org/victoria/
  5. https://pyshaker.readthedocs.io/en/latest/index.html
  6. https://docs.openstack.org/performance-docs/latest/test_plans/tenant_networking/shaker.html

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sirishagopigiri

Engineer by profession. Chef by passion (applicable only for some dishes :-P). Trying to become a blogger.