Deploying WASM plugin on Istio

In this current blog, we will explore how we can deploy custom envoy plugin filters into Istio’s echo system. We will be using the wasmecli to build, develop and deploy the plugin filter on Istio’s echo system.

Webassembly(WASM) is a binary code format that can use to run easily and efficiently on any web or server client. WASM compilers would help generate the binary code from different supported programming languages.

In this blog, we will explore wasme cli from solo-io which would help build wasm-based filter plugin images which can be deployed to the envoy. These filters would operate on the traffic data when invoked through the envoy. They are configured in the envoy which runs as a sidecar in the application.

The filter development, building, deployment, and configuration would be handled by the wasmecli itself. They also have a webassembly hub built specifically to host, share the envoy filters. The cli supports deployment of the filter to envoy, gloo, and istio service mesh tools. In this blog, we will explore how the filter can be deployed to the Istio’s service mesh hosted on a Kubernetes cluster.

Below are the expected pre-requisites and expected versions to run this setup

It is expected that we would have a Kubernetes cluster running with cni configured. In this current setup, we have a single node Kubernetes cluster with calico cni installed.

NAMESPACE         NAME                                       READY   STATUS    RESTARTS   AGE
calico-system calico-kube-controllers-5ddcc6fc8f-vf2s8 1/1 Running 0 39m
calico-system calico-node-8h96x 1/1 Running 1 39m
calico-system calico-typha-bdb687b85-75z5j 1/1 Running 0 39m
kube-system coredns-f9fd979d6-42r8c 1/1 Running 0 42m
kube-system coredns-f9fd979d6-c2dxt 1/1 Running 0 42m
kube-system etcd-harrypotter 1/1 Running 0 42m
kube-system kube-apiserver-harrypotter 1/1 Running 0 42m
kube-system kube-controller-manager-harrypotter 1/1 Running 0 42m
kube-system kube-proxy-mdcqm 1/1 Running 0 42m
kube-system kube-scheduler-harrypotter 1/1 Running 0 42m
tigera-operator tigera-operator-76bbbcbc85-rcwjh 1/1 Running 0 40m

Follow the below steps to install istio

$ curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.9.0 TARGET_ARCH=x86_64 sh -
$ cd istio-1.9.0
$ bin/istioctl install --set profile=demo -y
✔ Istio core installed
✔ Istiod installed
✔ Ingress gateways installed
✔ Egress gateways installed
✔ Installation complete

Check pods status

$ kubectl get pods -n istio-system
NAME READY STATUS RESTARTS AGE
istio-egressgateway-789b64c5f4-8j6qd 1/1 Running 0 46s
istio-ingressgateway-768ff847df-lssf2 1/1 Running 0 46s
istiod-6f984b7878-629gc 1/1 Running 0 2m46s

Patch istio’s ingress gateway service to nodeport for accessibility

# patch istio-ingressgateway to nodeport$ kubectl patch svc -n istio-system istio-ingressgateway --type='json' -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}]'

Now we will install wasmeand generate a sample go based plugin. More details are available in this blog.

# Install wasme
$ curl -sL https://run.solo.io/wasme/install | sh
$ export PATH=$HOME/.wasme/bin:$PATH
$ wasme --version
wasme version 0.0.33

Create a sample go filter

$ wasme init ./new-filter --language tinygo
✔ istio:1.7.x, gloo:1.6.x, istio:1.8.x, istio:1.9.x
INFO[0002] extracting 1416 bytes to /home/sirisha/new-filter
$ cd new-filter/
sirisha@harrypotter:~/new-filter$ tree
.
├── go.mod
├── go.sum
├── main.go
└── runtime-config.json
0 directories, 4 files

We can notice that in main.go, there is already a response header being appended to all the HTTP requests.

// Override DefaultHttpContext.
func (ctx *httpHeaders) OnHttpResponseHeaders(numHeaders int, endOfStream bool) types.Action {
if err := proxywasm.SetHttpResponseHeader("hello", "world"); err != nil {
proxywasm.LogCriticalf("failed to set response header: %v", err)
}
return types.ActionContinue
}

Next step is to build this go code to a WASM based image. This image then will be pushed to webassemblyhub.io built specifically to host WASM based envoy images.

# building WASM image
$ wasme build tinygo . -t test
Building with tinygo...go: downloading github.com/tetratelabs/proxy-wasm-go-sdk v0.1.1
INFO[0035] adding image to cache... filter file=/tmp/wasme545571676/filter.wasm tag=test
INFO[0036] tagged image digest="sha256:750d63889653e7117fcbc0831f10f0e1d3f7ec0c82fe5787b71d08a783e3393f" image="docker.io/library/test:latest"

Before pushing the image to webassemblyhub.io we need to re-tag it to the specific user account and as well login to the webassemblyhub.io

# Webassemblyhub.io login
$ wasme login -u sirishagopigiri
Enter password: *********█
INFO[0005] Successfully logged in as sirishagopigiri (Sirisha Gopigiri)
INFO[0005] stored credentials in /home/sirisha/.wasme/credentials.json

Publishing the image

# Re-tag the image with username details
$ wasme list
NAME TAG SIZE SHA UPDATED
docker.io/library/test latest 247.6 kB 750d6388 21 Sep 21 14:53 IST
$ wasme tag docker.io/library/test webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0000] tagged image digest="sha256:750d63889653e7117fcbc0831f10f0e1d3f7ec0c82fe5787b71d08a783e3393f" image="docker.io/library/test:latest"
# Publish image
$ wasme push webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0000] Pushing image webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0004] Pushed webassemblyhub.io/sirishagopigiri/wasmetest:latest
INFO[0004] Digest: sha256:27aecb092318b2f922204ce722a34e5c2866baa168cf2f9f00c303b1982cfa9a

Be cautious and replace sirishagopigiri with your username.

To test the above filter we will now deploy a sample application in Kubernetes in Istio enabled namespace. Then configure the side-car envoy proxy in that application with the above filter and observe if the response headers are being reflected.

We will use the httpbin application from Istio to test the filter here.

# Deploy httpbin
$ kubectl label namespace default istio-injection=enabled
namespace/default labeled
$ cd istio-1.9.0
$ kubectl apply -f samples/httpbin/httpbin.yaml
serviceaccount/httpbin created
service/httpbin created
deployment.apps/httpbin created
# Check pods
$ kubect get pods
NAME READY STATUS RESTARTS AGE
httpbin-74fb669cc6-2954z 2/2 Running 0 2m25s

Check the service response for the httpbin pod.

$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
httpbin ClusterIP 10.110.190.141 <none> 8000/TCP 4m26s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 72m
# Patch httpbin service to Nodeport
$ kubectl patch svc httpbin --type='json' -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}]'
service/httpbin patched
# Check service status
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
httpbin NodePort 10.110.190.141 <none> 8000:30276/TCP 5m44s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 73m
$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30276/headers
Note: Unnecessary use of -X or --request, GET is already inferred.
* Trying 127.0.0.1:30276...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 30276 (#0)
> GET /headers HTTP/1.1
> Host:httpbin.example.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: istio-envoy
< date: Wed, 22 Sep 2021 07:46:24 GMT
< content-type: application/json
< content-length: 264
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 89
< x-envoy-decorator-operation: httpbin.default.svc.cluster.local:8000/*
<
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin.example.com",
"User-Agent": "curl/7.68.0",
"X-B3-Sampled": "1",
"X-B3-Spanid": "8f405c030eca99f0",
"X-B3-Traceid": "1b49532e2a6c9ebc8f405c030eca99f0"
}
}
* Connection #0 to host 127.0.0.1 left intact

Alternatively we can create a virtual service and gateway to access the httpbin service through Istio’s gateway.

$ cat gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: httpbin-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "httpbin.example.com"
$ cat virtualservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- "httpbin.example.com"
gateways:
- httpbin-gateway
http:
- match:
- uri:
prefix: /
route:
- destination:
port:
number: 8000
host: httpbin

Apply the gateway.yaml and virtualservice.yaml files to the cluster to access the httpbin service through istio’s ingress gateway

$ kubectl apply -f gateway.yaml
gateway.networking.istio.io/httpbin-gateway created
$ kubectl apply -f virtualservice.yaml
virtualservice.networking.istio.io/httpbin created
# Check service status
$ kubectl get svc -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-egressgateway ClusterIP 10.110.81.83 <none> 80/TCP,443/TCP,15443/TCP 35m
istio-ingressgateway NodePort 10.106.202.134 <none> 15021:32075/TCP,80:30058/TCP,443:32536/TCP,31400:30117/TCP,15443:31262/TCP 35m
istiod ClusterIP 10.101.63.55 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 36m
$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/headers
Note: Unnecessary use of -X or --request, GET is already inferred.
* Trying 127.0.0.1:30058...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 30058 (#0)
> GET /headers HTTP/1.1
> Host:httpbin.example.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: istio-envoy
< date: Wed, 22 Sep 2021 07:53:59 GMT
< content-type: application/json
< content-length: 627
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 56
<
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin.example.com",
"User-Agent": "curl/7.68.0",
"X-B3-Parentspanid": "7ea5e61a0d49f59d",
"X-B3-Sampled": "1",
"X-B3-Spanid": "9cec5ee3c8e6b159",
"X-B3-Traceid": "37d9eabe501f53107ea5e61a0d49f59d",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/httpbin;Hash=84137ef8c10c564e398bf294ad346840a4fd329e1b8eb598d9c108db9ebe51a5;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
}
}

We will use wasme cli to deploy the filter on the sample httpbin application.

$ wasme deploy istio webassemblyhub.io/sirishagopigiri/wasmetest --id=myfilter --namespace default
INFO[0000] cache namespace already exists cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache configmap already exists cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache service account already exists cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache role updated cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache rolebinding updated cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache daemonset updated cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0005] added image to cache config... cache="{wasme-cache wasme}" image=webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0005] waiting for event with timeout 1m0s
INFO[0025] cleaning up cache events for image webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0025] updated workload sidecar annotations filter="id:\"myfilter\" image:\"webassemblyhub.io/sirishagopigiri/wasmetest\" rootID:\"root_id\" patchContext:\"inbound\" " workload=httpbin
INFO[0025] created Istio EnvoyFilter resource envoy_filter_resource=httpbin-myfilter.default filter="id:\"myfilter\" image:\"webassemblyhub.io/sirishagopigiri/wasmetest\" rootID:\"root_id\" patchContext:\"inbound\" " workload=httpbin

We can observe the httpbin is being configured with the filter, we can also check the pod status. There will be a new pod created with the latest filter configured on the envoy proxy side car of the httpbin service.

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
httpbin-74fb669cc6-2954z 2/2 Running 0 16m
httpbin-78fc7548d8-kprkx 0/2 Init:0/1 0 8s

Once the pod comes to running state, check the service status to see if the header is being added.

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
httpbin-78fc7548d8-kprkx 2/2 Running 0 95s

Service check using curl request

# curl request to check service status$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/headers
Note: Unnecessary use of -X or --request, GET is already inferred.
* Trying 127.0.0.1:30058...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 30058 (#0)
> GET /headers HTTP/1.1
> Host:httpbin.example.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: istio-envoy
< date: Wed, 22 Sep 2021 07:59:26 GMT
< content-type: application/json
< content-length: 627
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 56
< hello: world
<
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin.example.com",
"User-Agent": "curl/7.68.0",
"X-B3-Parentspanid": "56d1fd8ed6bdd7b8",
"X-B3-Sampled": "1",
"X-B3-Spanid": "4153dd05c0273b92",
"X-B3-Traceid": "c07708d714c168c656d1fd8ed6bdd7b8",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/httpbin;Hash=84137ef8c10c564e398bf294ad346840a4fd329e1b8eb598d9c108db9ebe51a5;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
}
}
* Connection #0 to host 127.0.0.1 left intact

We can notice that the “hello: world” header being added to the HTTP response.

We can also verify that the corresponding envoy filter being created in the default namespace with the id defined in the deploy command.

$ kubectl get envoyfilters
NAME AGE
httpbin-myfilter 1m
# check the yaml output of the filter for more info
$ kubectl get envoyfilters httpbin-myfilter -o yaml

The headers are being added to every HTTP response, we can try with the other requests as well.

$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/status/200$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/delay

In this blog we explored how to build, develop, deploy and configure envoy filters using wasme cli. The cli utility from solo-io hosted with webassemblyhub community makes the user job much simpler.

  1. https://webassemblyhub.io/
  2. https://docs.solo.io/
  3. https://github.com/solo-io/wasm
  4. https://istio.io/latest/docs/setup/install/istioctl/
  5. https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sirishagopigiri

Engineer by profession. Chef by passion (applicable only for some dishes :-P). Trying to become a blogger.