Notes Renovation

Kubernetes Network (1)

Introduction to Core Concepts and Layout

Cheedge Lee's photo
Cheedge Lee
·

7 min read

Table of contents

In this article, we want to give some simple examples to illustrate the basic knowledges of the networking in kubernetes, so that we can have a clear understanding for the network of kubernetes.

1. Basic Concepts

Kubernetes network has used the virtual networking technology from linux, so there are some basic concepts similar as in linux:

  1. virtual ethernet pair

    • A veth pair consists of two interfaces: one end is inside the pod's network namespace and the other end is inside the node's network namespace (i.e., the root network namespace of the node).

  2. bridge

  3. namespace

    • “A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource.” (man page for namespace)

All the detailed explaination of these concepts can be find in Linux man pages or any Linux related articles or wiki.

In order to have a better undersanding in following articles, here we make two simple examples to simulate the process.

1.1 Veth Pair

What is a veth pair?

A veth pair is like a virtual network “cable” connecting two network interfaces, typically used to connect network namespaces in Linux. Each veth pair consists of two virtual interfaces and traffic sent on one interface, will appear on the other. And later we will see the real functionility for networking in k8s.

Therefore, in the first exmpale, let’s create a veth pair to present the connections via veth pair.

Let's set up two network namespaces, ns1 and ns2, each with a veth interface veth1 and veth2. We'll then demonstrate how they can communicate through the veth pair. The structure as following figure.

Fig. 1

1. Create Network Namespaces ns1 and ns2

  • Create the first namespace, ns1 and ns2:
# create namespaces
sudo ip netns add ns1
sudo ip netns add ns2
# check created namespace
ip netns list

2. Create a veth Pair and Assign Interfaces to Namespaces

  • Create a veth pair named veth1 and veth2:
# create veth
sudo ip link add veth1 type veth peer name veth2
  • Assign veth1 to ns1, and veth2 to ns2
# assign veth to namespaces
sudo ip link set veth1 netns ns1
sudo ip link set veth2 netns ns2

3. Configure IP Addresses for the veth Interfaces

  • Assign an IP address to veth1 in ns1 and veth2 in ns2:
# assign ip to veth
sudo ip netns exec ns1 ip addr add 192.168.1.101/24 dev veth1
sudo ip netns exec ns2 ip addr add 192.168.1.102/24 dev veth2
  • Bring up the interfaces and loopback within their respective namespaces:
# bring up interfcae
sudo ip netns exec ns1 ip link set veth1 up
sudo ip netns exec ns2 ip link set veth2 up
# bring up loopback
sudo ip netns exec ns1 ip link set lo up
sudo ip netns exec ns2 ip link set lo up

4. Test Connectivity Between ns1 and ns2

  • Ping ns2 from ns1 (or ns1 from ns2):
sudo ip netns exec ns1 ping -c 3 192.168.1.102
# or 
sudo ip netns exec ns2 ping -c 3 192.168.1.101
# feel free to use other tools eg. nsenter
# enter the space then check ping ... same

Here we can see the traffic can flow in both directions between ns1 and ns2 via veth pair.

lee@srv01:~$ sudo ip netns exec ns1 ping -c 3 192.168.1.102
PING 192.168.1.102 (192.168.1.102) 56(84) bytes of data.
64 bytes from 192.168.1.102: icmp_seq=1 ttl=64 time=0.117 ms
64 bytes from 192.168.1.102: icmp_seq=2 ttl=64 time=0.071 ms
64 bytes from 192.168.1.102: icmp_seq=3 ttl=64 time=0.083 ms

--- 192.168.1.102 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2043ms
rtt min/avg/max/mdev = 0.071/0.090/0.117/0.019 ms

5. (Extral) Proof of Isolation Without a veth Pair

Above we see the traffic can flow between veth1 and veth2, but we also need to prove if there is no veth pair, the connection will cut down. To show that ns1 and ns2 cannot communicate without the veth pair, we’ll delete the veth pair and test connectivity again.

  • Delete the veth pair:
sudo ip link delete veth1
  • Try to ping ns2 from ns1, vice versa
sudo ip netns exec ns1 ping -c 3 192.168.1.102
sudo ip netns exec ns2 ping -c 3 192.168.1.101

The lossing package result proving the namespaces are isolated without the veth connection.

1.2 veth pair with Bridge

Above we have seen the veth pair to connect two network namespaces. But in k8s we have another important concept as we list in the beginning, the bridge. Now we will create two veth pairs (veth3/veth33 and veth4/veth44) that link each namespace to the root namespace (where the bridge br0 resides).

1. Create Network Namespaces and veth Pairs as above

Create two namespaces, ns1 and ns2, and create the veth pairs to link ns1 ns2 to the namespaces

# create ns
sudo ip netns add ns1
sudo ip netns add ns2
# create veth pair
sudo ip link add veth1 type veth peer name veth11
sudo ip link add veth2 type veth peer name veth22

2. Move Interfaces into Their Respective Namespaces

  • Move veth1 into ns1:
sudo ip link set veth1 netns ns1
sudo ip link set veth2 netns ns2
  • veth11 and veth22 remain in the root namespace, which we’ll connect to the bridge.

3. Create a Bridge and Add Interfaces to It

  • Create a bridge named br0 and bring it up:
sudo ip link add name br0 type bridge
sudo ip link set br0 up
  • Add veth11 and veth22 to br0:
sudo ip link set veth11 master br0
sudo ip link set veth22 master br0
  • Bring up veth11 and veth22 in the root namespace:
sudo ip link set veth11 up
sudo ip link set veth22 up

4. Assign IP Addresses and Bring Up Interfaces in Each Namespace

  • Assign IPs to veth1 and veth2 in their respective namespaces:
sudo ip netns exec ns1 ip addr add 192.168.1.103/24 dev veth1
sudo ip netns exec ns2 ip addr add 192.168.1.104/24 dev veth2
  • Bring up the veth interfaces and loopback interfaces in each namespace:
sudo ip netns exec ns1 ip link set veth1 up
sudo ip netns exec ns1 ip link set lo up
sudo ip netns exec ns2 ip link set veth2 up
sudo ip netns exec ns2 ip link set lo up

5. Test Connectivity

Enter ns1 and ping ns2:

sudo ip netns exec ns1 ping -c 3 192.168.1.104
# or
sudo ip netns exec ns2 ping -c 3 192.168.1.103

As ns1 and ns2 are now connected through the bridge br0 in the root namespace, we can see the ping sucessfully.

lee@srv01:~$ sudo ip netns exec ns1 ping -c 3 192.168.1.104
PING 192.168.1.104 (192.168.1.104) 56(84) bytes of data.
From 192.168.1.101 icmp_seq=1 Destination Host Unreachable
From 192.168.1.101 icmp_seq=2 Destination Host Unreachable
From 192.168.1.101 icmp_seq=3 Destination Host Unreachable

--- 192.168.1.104 ping statistics ---
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2085ms
pipe 3

Later we can see this approach is similar with what the networking part in k8s structure.

Note: ip command is more powerful, absolutly you can also use other tools to realise above checking, like the nsenter to make operations insdie the namespace (may need a PID before enter), or use brctl for the brdige settings.

Additionally, as to the reason why we need to set up loopback interface in each namespace, I am not so sure, but as I understand the sent/recieve networking traffic all depend on lo, and loopback is the basics network for every linux system, as we can always see lo 127.0.0.1/8 there. If interested, there’re more explainations on internet.

Above we have understand how the basci virtrual ethernet pair and the bridge work on connection. There are some other basic internet concept we may meet in k8s. Here only gives the points, as to the details, it’s easy to find on Internet.

  1. network model

    1. Layer 2 (Data Layer)

    2. Layer 3 (Network Layer)

    3. Layer 4 (Transport Layer)

  2. TCP/IP & ICMP

Next

In next article, we will give a detailed present for different scenarios of the network traffic flow.

Kubernetesnetworking