Sage Infrastructure Solutions Group IncSage Infrastructure Solutions Group IncLoading…

Sage Infrastructure Solutions Group Inc

Application tunnels

[!NOTE] Application tunnels need a valid license.

An application tunnel connects NullWard to applications in another network, such as an office, a private cloud or a Kubernetes cluster, without opening any inbound ports there.

How it works

Visitor ──HTTPS──▶ NullWard ══ WireGuard (encrypted) ══▶ Connector ──▶ Your application
                   (public)        UDP 7055                (your network)
                                ◀── connector dials out ──
  1. A small program, the connector, runs in the same network as your applications. It makes an outbound WireGuard connection to NullWard on UDP 7055. Nothing in your network needs to accept inbound connections.
  2. Visitors reach your service's public hostname on NullWard. NullWard applies the WAF, authentication and every other check first.
  3. Allowed requests go through the tunnel to the connector, which forwards them to the application and returns the response.

A connector serves one or more upstream groups: named sets of backend addresses, such as Web App or API Server. The connector reports its groups to NullWard, and each NullWard service picks the tunnel and group to send its traffic to. One tunnel can serve many services.

There are three ways to run a connector:

ConnectorRuns asConfigured with
Linux hostA systemd serviceA YAML file
DockerA containerThe same YAML file, mounted into the container
KubernetesAn operator installed with HelmLabels and annotations on your Kubernetes Services

The first two use a Generic connector tunnel. Kubernetes uses a Kubernetes operator tunnel.

1. Prepare NullWard (once)

  1. Open UDP 7055 to NullWard from wherever your connectors run, and publish it in Docker Compose: "7055:7055/udp".
  2. Settings > Tunnels > Public Endpoint: set the address connectors dial, for example waf.example.com:7055. Connector configuration files include it, so set it before you download any.
  3. For the one-command install: connector hosts also need to reach NullWard's public HTTP port (80). It's normally open already, for HTTPS redirects and certificate validation.

2. Create the tunnel

Go to Services > Tunnels and click Create Tunnel. Enter a Name, then choose the Connector type. You can't change the type later.

  • Generic connector: for a Linux host or Docker.
  • Kubernetes operator: for a Kubernetes cluster.

Creating a tunnel

The new tunnel appears in the list as Disconnected until its connector dials in. Each row has buttons to edit the tunnel, open its Guide (the setup instructions for its connector type), download its configuration, and delete it. A message confirms each download.

Under each tunnel's name, the list shows which services use it (Used by …) or Not used by any service. A tunnel can only be deleted while no service uses it: as a destination, a path-route destination or a TLS passthrough target. Remove it from those services first; the delete button's tooltip names them. This needs NullWard 0.6.100 or later.

[!WARNING] The downloaded configuration (tunnel-<name>.yaml, or tunnel-<name>-values.yaml for Kubernetes) contains the tunnel's private WireGuard key. Keep it secret, store it with restrictive permissions, and don't commit it to source control. Anyone who has it can impersonate the connector.

The connector configuration file (generic connectors)

Download the file with the tunnel's download button. Most of it is filled in for you:

tunnel_id: "..."          # do not change
private_key: "..."        # secret
preshared_key: "..."      # secret
wireguard_ip: "..."       # do not change

upstreams:                # edit this section
  - name: "My Backend"
    protocol: http
    targets:
      - address: "localhost:8080"

waf_peers:                # NullWard instances to connect to; filled in for you
  - endpoint: "waf.example.com:7055"
    public_key: "..."
    network_ip: "..."

Edit upstreams to describe your applications. The default localhost:8080 is only a placeholder.

upstreams:
  # Two web servers, load-balanced, with a health check
  - name: "Web App"
    protocol: http
    targets:
      - address: "web1.internal:80"
      - address: "web2.internal:80"
    health_check:
      path: "/health"
      interval_seconds: 30
      timeout_seconds: 5
      unhealthy_threshold: 3
      healthy_threshold: 2

  # An HTTPS backend with a self-signed certificate
  - name: "Internal HTTPS App"
    protocol: https
    tls_skip_verify: true
    targets:
      - address: "app.internal:8443"
  • name: the group name you'll choose on the NullWard service.

  • protocol: http, https, or h3 (HTTP/3 to the backend, see HTTP/3).

  • tls_skip_verify: skip certificate checks for https and h3 backends. Use it only on trusted internal networks.

  • gRPC works through a tunnel with connector 0.6.93 or later and Allow gRPC on the service (see Services). The connector forwards gRPC calls over HTTP/2: over TLS to https and h3 groups, and as cleartext HTTP/2 (h2c) to http groups, which is what a gRPC server's plain port speaks. Nothing else changes for a group's other traffic.

  • FIPS mode: with FIPS mode on for a tunnel (connector 0.6.97 or later), all of its traffic runs over TLS 1.3 inside WireGuard, with keys verified on both ends.

  • TLS passthrough services can use a group too, with connector 0.6.96 or later (see TLS passthrough). The connector relays the encrypted connection to a target of the group as it is, whatever the group's protocol. If the group has a health check, set protocol: https and a path the target answers, because an http check sends plain HTTP to the TLS port and marks the target down. For example, an SSL VPN gateway:

    - name: "VPN"
      protocol: https
      tls_skip_verify: true
      targets:
        - address: "10.0.0.5:443"
    
  • targets: one or more host:port addresses. The connector load-balances across healthy targets.

If you add NullWard nodes to a cluster later, re-download the file to pick up the new waf_peers. Up-to-date connectors also learn about new nodes automatically; the tunnels list shows peer discovery ready for those.

Run the connector on a Linux host (systemd)

There are two ways to install the connector on a Linux or macOS host: one command, or a manual install.

Option A: install with one command

The one-command install runs on the connector host and needs only outbound access to NullWard's public HTTP port (80) and the tunnel port (UDP 7055). It never contacts the management interface, and nothing secret travels over the network:

  • the installer and the connector program are checked against SHA-256 hashes shown in your dashboard;
  • the tunnel's WireGuard private key is created on the connector host and never leaves it. Only the public key is sent to NullWard, signed with the install key;
  • the install key is never sent at all. You type it at a hidden prompt, so it doesn't appear on the screen or in your shell history.
  1. In the dashboard, go to Services > Tunnels, open the tunnel's Guide and find Method 4: Install with one command.

  2. Check the NullWard address the connector host uses. By default it's the host from Settings > Tunnels > Public Endpoint, on port 80. Change it if your connector hosts reach NullWard by another name, then click Create install command.

  3. Copy the install key. It's shown only in the dashboard and isn't part of the command.

  4. On the connector host, paste and run the command for your platform. It looks like this:

    printf 'NullWard install key: '; read -rs NW_INSTALL_KEY; echo; export NW_INSTALL_KEY
    curl -fsSL 'http://waf.example.com/__nullward/install/<install-id>' -o nw-install.sh \
      && echo '<sha256>  nw-install.sh' | sha256sum -c - \
      && sudo --preserve-env=NW_INSTALL_KEY bash nw-install.sh
    unset NW_INSTALL_KEY
    

    At the NullWard install key: prompt, paste the key and press Enter. Nothing is echoed.

Method 4 in the tunnel's setup instructions: the masked install key and the command

The command works for 15 minutes. After that, create a new one. The installer:

  • refuses to run if the downloaded script or connector doesn't match its hash;
  • installs the connector in /opt/nullward-tunnel/bin/, linked from /usr/local/bin/nullward-tunnel;
  • creates the host's WireGuard key, registers its public key with NullWard, and writes /etc/nullward-tunnel/config.yaml (readable only by root). If a configuration already exists, its upstreams are kept and the old file is saved as config.yaml.bak;
  • on Linux, installs and starts the nullward-tunnel systemd service. On macOS, it prints the launchd steps.

It needs curl and unzip on the host. Then edit the upstreams section of /etc/nullward-tunnel/config.yaml and restart the service:

sudo systemctl restart nullward-tunnel

[!WARNING] Installing replaces the tunnel's key. If a connector is already running for this tunnel, it disconnects, and the dashboard asks you to confirm first. Because the private key exists only on the connector host, Download config for that tunnel afterwards also asks before creating a new key.

Option B: manual install

Do the downloads on your admin machine, where you can reach the dashboard. The connector host only needs outbound access to NullWard's tunnel port (UDP 7055), never to the management interface.

  1. Download the connector for your platform from Services > Tunnels > Connector Downloads: nullward-tunnel-linux-amd64.zip or nullward-tunnel-linux-arm64.zip. Each contains the nullward-tunnel program and the nullward-tunnel.service unit file.

  2. Download the tunnel's configuration and edit its upstreams.

  3. Copy both to the connector host, for example with scp. The configuration contains the tunnel's private key, so copy it only over an encrypted channel and delete any other copies.

  4. On the connector host, unpack the archive and install everything:

    unzip nullward-tunnel-linux-amd64.zip
    sudo install -d /opt/nullward-tunnel/bin /etc/nullward-tunnel
    sudo install -m 755 nullward-tunnel /opt/nullward-tunnel/bin/nullward-tunnel
    sudo ln -sf /opt/nullward-tunnel/bin/nullward-tunnel /usr/local/bin/nullward-tunnel
    sudo install -m 600 tunnel-<name>.yaml /etc/nullward-tunnel/config.yaml
    sudo install -m 644 nullward-tunnel.service /etc/systemd/system/nullward-tunnel.service
    sudo systemctl daemon-reload
    sudo systemctl enable --now nullward-tunnel
    

Check and troubleshoot

systemctl status nullward-tunnel
journalctl -u nullward-tunnel -f

The tunnel turns Connected on the NullWard Tunnels page, usually within a few seconds. If it doesn't:

  • check the host can reach waf.example.com:7055 over UDP;
  • check that Public Endpoint is set and that the configuration's waf_peers has a non-empty endpoint. Re-download the configuration if not.

Updates

Turn on Auto Update for the tunnel in the tunnels list to let the connector install newer versions offered by NullWard. It keeps the previous version and rolls back automatically if the new one fails. Otherwise, replace the program in /opt/nullward-tunnel/bin/ with a newer download and restart the service.

NullWard offers its own version, and only when that version is on the Connector Update Channel (Settings > Tunnels): beta, the default, takes every release; stable takes only releases without a suffix such as -beta or -rc. When a connector is behind, the tunnels list shows update → next to its version. The channel needs NullWard 0.6.89 or later.

Run the connector in Docker

The connector image is harbor.sageisg.com/nullward/tunnel. It needs no extra privileges and no published ports.

  1. On your admin machine, download the tunnel's configuration, edit its upstreams (see below), and copy it to the Docker host.

  2. Start the container with the configuration mounted read-only:

    docker run -d --name nullward-tunnel-<name> \
      --restart unless-stopped \
      -v /path/to/tunnel-<name>.yaml:/etc/nullward-tunnel/config.yaml:ro \
      harbor.sageisg.com/nullward/tunnel:latest
    

    Or with Docker Compose:

    services:
      nullward-tunnel-<name>:
        image: harbor.sageisg.com/nullward/tunnel:latest
        container_name: nullward-tunnel-<name>
        restart: unless-stopped
        volumes:
          - ./tunnel-<name>.yaml:/etc/nullward-tunnel/config.yaml:ro
    
  3. Check it: docker logs -f nullward-tunnel-<name>. The tunnel should show Connected in NullWard.

The setup instructions in the dashboard show these commands with your tunnel's name filled in. To upgrade, pull the newer image and recreate the container (docker compose pull && docker compose up -d). Pin a version tag rather than latest in production.

Reaching your applications from the container

Inside a container, localhost is the container itself, not the host. Point targets at addresses the container can reach:

  • Applications in the same Compose project: add the connector to the project and use the service names, for example address: "web:8080".
  • Applications on other Docker networks: attach the connector to that network (networks: in Compose, or docker network connect), then use the container names.
  • Applications on the Docker host itself: use the host's LAN address. On Linux you can also add extra_hosts: ["host.docker.internal:host-gateway"] and use host.docker.internal:<port>.
  • Other machines on your network: use their hostnames or IP addresses as usual.

Kubernetes connector

The Kubernetes connector is an operator that runs in your cluster. Instead of a configuration file, you choose what to expose by labelling Kubernetes Services. The connector forwards to their pods, load-balances across them, and follows changes as pods come and go.

Create the tunnel with the Kubernetes operator type, then open its Guide.

Step 1: download values.yaml

Click Download values.yaml on the tunnel. It holds the tunnel's WireGuard identity and the NullWard endpoints. Keep it secret.

Step 2: install the chart

Install the version shown in the tunnel's setup instructions: the newest connector on the update channel (see Upgrade, roll back and uninstall). The header of the downloaded values.yaml has the same commands with the version filled in.

kubectl create namespace nullward-system

helm install nullward-k8s-connector \
  oci://harbor.sageisg.com/nullward/charts/nullward-k8s-connector \
  --version <version> \
  --namespace nullward-system \
  -f tunnel-<name>-values.yaml

Always pass --version. The chart's versions are pre-releases such as 0.6.88-beta, and Helm skips pre-releases unless you name one: without --version the command fails with Could not locate a version matching provided version string.

The chart installs the operator Deployment, its RBAC, the cluster-scoped NullwardTunnel custom resource definition, and a NullwardTunnel resource for this tunnel. The pod runs as non-root with a read-only root filesystem.

Useful chart values:

ValueDefaultPurpose
tunnel.watchNamespaces[] (all)Only expose Services in these namespaces
tunnel.defaultLoadBalancerround_robinDefault for Services without a load-balancer annotation
rbac.namespaces(unset)Grant per-namespace Roles instead of a ClusterRole
replicaCount1More than one replica needs leaderElection.enabled=true; only the leader runs the tunnel

Step 3: check the tunnel

kubectl -n nullward-system rollout status deploy/nullward-k8s-connector
kubectl get nullwardtunnel
kubectl describe nullwardtunnel <name>
kubectl -n nullward-system logs -l app.kubernetes.io/name=nullward-k8s-connector --tail=50

When the NullwardTunnel reports Connected=true, the tunnel also shows Connected in NullWard.

Step 4: expose Services

Add the label nullward.io/expose=true to each Service that should be reachable through the tunnel. The value must be exactly true.

kubectl label svc my-app -n my-namespace nullward.io/expose=true

Each exposed Service becomes an upstream group, named after the Service unless you set nullward.io/upstream-group. Optional annotations:

AnnotationValuesDefault
nullward.io/upstream-groupGroup name as NullWard shows itThe Service name
nullward.io/portPort name or numberThe first port
nullward.io/load-balancerround_robin, least_conn, p2c, randomThe tunnel's default (round_robin)
nullward.io/protocolhttp, https, h3http
nullward.io/tls-skip-verifytrue, falsefalse (applies to https and h3)

gRPC through the Kubernetes connector follows the same nullward.io/protocol value: http sends gRPC as cleartext HTTP/2 (h2c), https over TLS. It needs connector 0.6.93 or later and Allow gRPC on the service.

With kubectl, values need no quotes:

kubectl annotate svc my-app -n my-namespace nullward.io/protocol=https
kubectl annotate svc my-app -n my-namespace nullward.io/tls-skip-verify=true
# Change a value that's already set:
kubectl annotate svc my-app -n my-namespace nullward.io/load-balancer=p2c --overwrite
# Remove an annotation:
kubectl annotate svc my-app -n my-namespace nullward.io/load-balancer-

In a YAML manifest, quote every value. Labels and annotations must be strings, and YAML reads unquoted true or 8443 as a boolean or number, which kubectl apply rejects:

apiVersion: v1
kind: Service
metadata:
  name: my-app
  namespace: my-namespace
  labels:
    nullward.io/expose: "true"
  annotations:
    nullward.io/port: "8443"
    nullward.io/protocol: "https"
    nullward.io/tls-skip-verify: "true"
spec:
  selector:
    app: my-app
  ports:
    - name: https
      port: 8443
      targetPort: 8443

For h3, the Service port must also accept UDP.

The connector picks up label and annotation changes straight away. It reports its upstream groups to NullWard with its next heartbeat, which runs every 30 seconds, so a new group appears in the tunnel's Upstreams column within about a minute, with its ready and total pods. Then send a NullWard service to it: see Route a service through the tunnel. In the tunnels list, a Kubernetes tunnel's + Expose a Service link opens these steps in its Guide.

To check from the cluster which Services the connector publishes:

kubectl describe nullwardtunnel <tunnel-name>   # see "Exposed Services"

If a Service doesn't show up:

  • The label value must be exactly true (quoted, "true", in a YAML manifest).
  • Its namespace must be watched: tunnel.watchNamespaces in the values file (empty means all namespaces). If you installed the chart with rbac.namespaces, list the namespace there too, then apply the values file (see the configuration change below).
  • It shows with 0 ready pods: the Service has no ready endpoints. Check its selector and that its pods are Ready, for example with kubectl get endpointslices -n my-namespace -l kubernetes.io/service-name=my-app.
  • Upstream group names must be unique: two Services with the same name in different namespaces need different nullward.io/upstream-group annotations.
  • The tunnel must show Connected in NullWard; until then its upstreams show as Unknown.
  • The operator's log: kubectl -n nullward-system logs -l app.kubernetes.io/name=nullward-k8s-connector --tail=100.

Kubernetes setup: exposing a Service

Upgrade, roll back and uninstall

The Kubernetes connector never updates itself: you upgrade it with Helm. NullWard checks which connector versions are published on harbor.sageisg.com and shows update → next to the connector's version in the tunnels list when a newer one is available on the Connector Update Channel (Settings > Tunnels: beta, the default, or stable, which skips versions with a suffix such as -beta). Click it to open the tunnel's upgrade commands with the versions filled in. If NullWard can't reach harbor.sageisg.com, it shows its own version as the target instead. This needs NullWard 0.6.89 or later. From 0.6.103, the tunnel's Guide also starts with an update available notice and the steps, and the Auto Update column shows via Helm for Kubernetes tunnels, linking to the same commands.

Run the commands with the tunnel's values file. If you no longer have it, download it again (Download values.yaml in the Guide): the connector keeps the same identity.

# Configuration change (edited or re-downloaded values.yaml): keep the version
# that is running. Applied in place, no restart.
helm upgrade nullward-k8s-connector \
  oci://harbor.sageisg.com/nullward/charts/nullward-k8s-connector \
  --version <running version> \
  --namespace nullward-system \
  -f tunnel-<name>-values.yaml

# New connector version: apply the chart's custom resource definition first
# (helm upgrade doesn't update it), then upgrade.
helm show crds oci://harbor.sageisg.com/nullward/charts/nullward-k8s-connector \
  --version <new version> | kubectl apply -f -

helm upgrade nullward-k8s-connector \
  oci://harbor.sageisg.com/nullward/charts/nullward-k8s-connector \
  --version <new version> \
  --namespace nullward-system \
  -f tunnel-<name>-values.yaml

# Roll back
helm history nullward-k8s-connector -n nullward-system
helm rollback nullward-k8s-connector <revision> -n nullward-system

# Uninstall
helm uninstall nullward-k8s-connector -n nullward-system

A version upgrade restarts the pod, so the tunnel is briefly disconnected (about 5–15 seconds) while it reconnects. The first time you apply the definition, kubectl warns that it lacks the last-applied-configuration annotation and adds it; that's expected. You don't need to uninstall and reinstall to upgrade.

Keeping a visitor on one pod

By default the connector spreads requests across a Service's pods (or a group's targets) one request at a time. Some applications need each visitor to stay on one pod, because they keep sessions in memory or issue a session cookie per pod. For those, turn on Sticky Sessions on the NullWard service (under General > Load Balancing & Health). NullWard then gives each visitor an affinity cookie and passes it through the tunnel, and the connector keeps that visitor on the same pod. A visitor only moves if their pod goes away. This needs connectors 0.6.85 or later, and works for both the generic and the Kubernetes connector.

3. Route a service through the tunnel

Once the connector is connected and reporting its groups:

  1. Open the service (or create one with Upstream Type: Application Tunnel).
  2. Under General > Destinations, set the destination Type to Application Tunnel.
  3. Choose the Tunnel and the Upstream Group. Default (first group) uses the connector's first group.
  4. Click Save Changes.

You can mix tunnel and direct destinations on one service, and use several tunnels for failover or load balancing. See Services.

The tunnels list shows each tunnel's status, when it was last seen, its connector version, and every upstream group with its health and protocol.