Deploying WASM plugin on Istio

8 min readSep 22, 2021

In this current blog, we will explore how we can deploy custom envoy plugin filters into Istio’s echo system. We will be using the wasmecli to build, develop and deploy the plugin filter on Istio’s echo system.

Webassembly(WASM) is a binary code format that can use to run easily and efficiently on any web or server client. WASM compilers would help generate the binary code from different supported programming languages.

In this blog, we will explore wasme cli from solo-io which would help build wasm-based filter plugin images which can be deployed to the envoy. These filters would operate on the traffic data when invoked through the envoy. They are configured in the envoy which runs as a sidecar in the application.

The filter development, building, deployment, and configuration would be handled by the wasmecli itself. They also have a webassembly hub built specifically to host, share the envoy filters. The cli supports deployment of the filter to envoy, gloo, and istio service mesh tools. In this blog, we will explore how the filter can be deployed to the Istio’s service mesh hosted on a Kubernetes cluster.

Pre-requisites

Below are the expected pre-requisites and expected versions to run this setup

Ubuntu 20.04
Kubernetes 1.19
Istio 1.9
wasme 0.33
Account in https://webassemblyhub.io/ to push WASM images

It is expected that we would have a Kubernetes cluster running with cni configured. In this current setup, we have a single node Kubernetes cluster with calico cni installed.

NAMESPACE         NAME                                       READY   STATUS    RESTARTS   AGE
calico-system     calico-kube-controllers-5ddcc6fc8f-vf2s8   1/1     Running   0          39m
calico-system     calico-node-8h96x                          1/1     Running   1          39m
calico-system     calico-typha-bdb687b85-75z5j               1/1     Running   0          39m
kube-system       coredns-f9fd979d6-42r8c                    1/1     Running   0          42m
kube-system       coredns-f9fd979d6-c2dxt                    1/1     Running   0          42m
kube-system       etcd-harrypotter                           1/1     Running   0          42m
kube-system       kube-apiserver-harrypotter                 1/1     Running   0          42m
kube-system       kube-controller-manager-harrypotter        1/1     Running   0          42m
kube-system       kube-proxy-mdcqm                           1/1     Running   0          42m
kube-system       kube-scheduler-harrypotter                 1/1     Running   0          42m
tigera-operator   tigera-operator-76bbbcbc85-rcwjh           1/1     Running   0          40m

Istio setup

Follow the below steps to install istio

$ curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.9.0 TARGET_ARCH=x86_64 sh -
$ cd istio-1.9.0
$ bin/istioctl install --set profile=demo -y
✔ Istio core installed
✔ Istiod installed
✔ Ingress gateways installed
✔ Egress gateways installed
✔ Installation complete

Check pods status

$ kubectl get pods -n istio-system
NAME                                    READY   STATUS    RESTARTS   AGE
istio-egressgateway-789b64c5f4-8j6qd    1/1     Running   0          46s
istio-ingressgateway-768ff847df-lssf2   1/1     Running   0          46s
istiod-6f984b7878-629gc                 1/1     Running   0          2m46s

Patch istio’s ingress gateway service to nodeport for accessibility

# patch istio-ingressgateway to nodeport$ kubectl patch svc -n istio-system istio-ingressgateway --type='json' -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}]'

Generate Wasm filter plugin

Now we will install wasmeand generate a sample go based plugin. More details are available in this blog.

# Install wasme
$ curl -sL https://run.solo.io/wasme/install | sh
$ export PATH=$HOME/.wasme/bin:$PATH
$ wasme --version
wasme version 0.0.33

Create a sample go filter

$ wasme init ./new-filter --language tinygo
✔ istio:1.7.x, gloo:1.6.x, istio:1.8.x, istio:1.9.x
INFO[0002] extracting 1416 bytes to /home/sirisha/new-filter$ cd new-filter/
sirisha@harrypotter:~/new-filter$ tree
.
├── go.mod
├── go.sum
├── main.go
└── runtime-config.json
0 directories, 4 files

We can notice that in main.go, there is already a response header being appended to all the HTTP requests.

// Override DefaultHttpContext.
func (ctx *httpHeaders) OnHttpResponseHeaders(numHeaders int, endOfStream bool) types.Action {
   if err := proxywasm.SetHttpResponseHeader("hello", "world"); err != nil {
       proxywasm.LogCriticalf("failed to set response header: %v", err)
   }
   return types.ActionContinue
}

Building and publishing filter

Next step is to build this go code to a WASM based image. This image then will be pushed to webassemblyhub.io built specifically to host WASM based envoy images.

# building WASM image
$ wasme build tinygo . -t test
Building with tinygo...go: downloading github.com/tetratelabs/proxy-wasm-go-sdk v0.1.1
INFO[0035] adding image to cache...                      filter file=/tmp/wasme545571676/filter.wasm tag=test
INFO[0036] tagged image                                  digest="sha256:750d63889653e7117fcbc0831f10f0e1d3f7ec0c82fe5787b71d08a783e3393f" image="docker.io/library/test:latest"

Before pushing the image to webassemblyhub.io we need to re-tag it to the specific user account and as well login to the webassemblyhub.io

# Webassemblyhub.io login
$ wasme login -u sirishagopigiri
Enter password: *********█
INFO[0005] Successfully logged in as sirishagopigiri (Sirisha Gopigiri)
INFO[0005] stored credentials in /home/sirisha/.wasme/credentials.json

Publishing the image

# Re-tag the image with username details
$ wasme list
NAME                                             TAG    SIZE     SHA      UPDATED
docker.io/library/test                           latest 247.6 kB 750d6388 21 Sep 21 14:53 IST$ wasme tag docker.io/library/test webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0000] tagged image                                  digest="sha256:750d63889653e7117fcbc0831f10f0e1d3f7ec0c82fe5787b71d08a783e3393f" image="docker.io/library/test:latest"# Publish image
$ wasme push webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0000] Pushing image webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0004] Pushed webassemblyhub.io/sirishagopigiri/wasmetest:latest
INFO[0004] Digest: sha256:27aecb092318b2f922204ce722a34e5c2866baa168cf2f9f00c303b1982cfa9a

Be cautious and replace sirishagopigiri with your username.

Deploy sample application

To test the above filter we will now deploy a sample application in Kubernetes in Istio enabled namespace. Then configure the side-car envoy proxy in that application with the above filter and observe if the response headers are being reflected.

We will use the httpbin application from Istio to test the filter here.

# Deploy httpbin
$ kubectl label namespace default istio-injection=enabled
namespace/default labeled$ cd istio-1.9.0
$ kubectl apply -f samples/httpbin/httpbin.yaml
serviceaccount/httpbin created
service/httpbin created
deployment.apps/httpbin created# Check pods
$ kubect get pods
NAME                       READY   STATUS    RESTARTS   AGE
httpbin-74fb669cc6-2954z   2/2     Running   0          2m25s

Check the service response for the httpbin pod.

$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
httpbin      ClusterIP   10.110.190.141   <none>        8000/TCP   4m26s
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP    72m# Patch httpbin service to Nodeport
$ kubectl patch svc httpbin --type='json' -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}]'
service/httpbin patched# Check service status
$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
httpbin      NodePort    10.110.190.141   <none>        8000:30276/TCP   5m44s
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP          73m$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30276/headers
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 127.0.0.1:30276...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 30276 (#0)
> GET /headers HTTP/1.1
> Host:httpbin.example.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: istio-envoy
< date: Wed, 22 Sep 2021 07:46:24 GMT
< content-type: application/json
< content-length: 264
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 89
< x-envoy-decorator-operation: httpbin.default.svc.cluster.local:8000/*
<
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin.example.com",
"User-Agent": "curl/7.68.0",
"X-B3-Sampled": "1",
"X-B3-Spanid": "8f405c030eca99f0",
"X-B3-Traceid": "1b49532e2a6c9ebc8f405c030eca99f0"
}
}
* Connection #0 to host 127.0.0.1 left intact

Alternatively we can create a virtual service and gateway to access the httpbin service through Istio’s gateway.

$ cat gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "httpbin.example.com"$ cat virtualservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.com"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        port:
          number: 8000
        host: httpbin

Apply the gateway.yaml and virtualservice.yaml files to the cluster to access the httpbin service through istio’s ingress gateway

$ kubectl apply -f gateway.yaml
gateway.networking.istio.io/httpbin-gateway created$ kubectl apply -f virtualservice.yaml
virtualservice.networking.istio.io/httpbin created# Check service status
$ kubectl get svc -n istio-system
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                      AGE
istio-egressgateway    ClusterIP   10.110.81.83     <none>        80/TCP,443/TCP,15443/TCP                                                     35m
istio-ingressgateway   NodePort    10.106.202.134   <none>        15021:32075/TCP,80:30058/TCP,443:32536/TCP,31400:30117/TCP,15443:31262/TCP   35m
istiod                 ClusterIP   10.101.63.55     <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP                                        36m$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/headers
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 127.0.0.1:30058...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 30058 (#0)
> GET /headers HTTP/1.1
> Host:httpbin.example.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: istio-envoy
< date: Wed, 22 Sep 2021 07:53:59 GMT
< content-type: application/json
< content-length: 627
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 56
<
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin.example.com",
"User-Agent": "curl/7.68.0",
"X-B3-Parentspanid": "7ea5e61a0d49f59d",
"X-B3-Sampled": "1",
"X-B3-Spanid": "9cec5ee3c8e6b159",
"X-B3-Traceid": "37d9eabe501f53107ea5e61a0d49f59d",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/httpbin;Hash=84137ef8c10c564e398bf294ad346840a4fd329e1b8eb598d9c108db9ebe51a5;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
}
}

Configure application with filter

We will use wasme cli to deploy the filter on the sample httpbin application.

$ wasme deploy istio webassemblyhub.io/sirishagopigiri/wasmetest --id=myfilter --namespace default
INFO[0000] cache namespace already exists                cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache configmap already exists                cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache service account already exists          cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache role updated                            cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache rolebinding updated                     cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0000] cache daemonset updated                       cache=wasme-cache.wasme image="quay.io/solo-io/wasme:0.0.33"
INFO[0005] added image to cache config...                cache="{wasme-cache wasme}" image=webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0005] waiting for event with timeout 1m0s
INFO[0025] cleaning up cache events for image webassemblyhub.io/sirishagopigiri/wasmetest
INFO[0025] updated workload sidecar annotations          filter="id:\"myfilter\" image:\"webassemblyhub.io/sirishagopigiri/wasmetest\" rootID:\"root_id\" patchContext:\"inbound\" " workload=httpbin
INFO[0025] created Istio EnvoyFilter resource            envoy_filter_resource=httpbin-myfilter.default filter="id:\"myfilter\" image:\"webassemblyhub.io/sirishagopigiri/wasmetest\" rootID:\"root_id\" patchContext:\"inbound\" " workload=httpbin

We can observe the httpbin is being configured with the filter, we can also check the pod status. There will be a new pod created with the latest filter configured on the envoy proxy side car of the httpbin service.

$ kubectl get pods
NAME                       READY   STATUS     RESTARTS   AGE
httpbin-74fb669cc6-2954z   2/2     Running    0          16m
httpbin-78fc7548d8-kprkx   0/2     Init:0/1   0          8s

Once the pod comes to running state, check the service status to see if the header is being added.

$ kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
httpbin-78fc7548d8-kprkx   2/2     Running   0          95s

Service check using curl request

# curl request to check service status$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/headers
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 127.0.0.1:30058...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 30058 (#0)
> GET /headers HTTP/1.1
> Host:httpbin.example.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: istio-envoy
< date: Wed, 22 Sep 2021 07:59:26 GMT
< content-type: application/json
< content-length: 627
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 56
< hello: world
<
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin.example.com",
"User-Agent": "curl/7.68.0",
"X-B3-Parentspanid": "56d1fd8ed6bdd7b8",
"X-B3-Sampled": "1",
"X-B3-Spanid": "4153dd05c0273b92",
"X-B3-Traceid": "c07708d714c168c656d1fd8ed6bdd7b8",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/httpbin;Hash=84137ef8c10c564e398bf294ad346840a4fd329e1b8eb598d9c108db9ebe51a5;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
}
}
* Connection #0 to host 127.0.0.1 left intact

We can notice that the “hello: world” header being added to the HTTP response.

We can also verify that the corresponding envoy filter being created in the default namespace with the id defined in the deploy command.

$ kubectl get envoyfilters
NAME               AGE
httpbin-myfilter   1m# check the yaml output of the filter for more info
$ kubectl get envoyfilters httpbin-myfilter -o yaml

The headers are being added to every HTTP response, we can try with the other requests as well.

$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/status/200$ curl -v -X GET -HHost:httpbin.example.com http://127.0.0.1:30058/delay

Conclusion

In this blog we explored how to build, develop, deploy and configure envoy filters using wasme cli. The cli utility from solo-io hosted with webassemblyhub community makes the user job much simpler.