Published on
Exercise 4

Network issue

Authors
  • Name
    Twitter

One of your colleague loves playing Wordle during his lunch break, but the corporate firewall is blocking it. He managed to containerise the game and deploy it to the OpenShift cluster, but something is wrong. He thinks it's related to the network, and ask for your help. You love Wordle too, so you decide to give it a go and try to fix it. Important: your colleague told you he created all the required resources, and you only need to edit one or some of them. Hint: that's correct.

A few things to be aware of:

  • There will be more than one problem to troubleshoot and fix.
  • You will have edit permission only on a few OpenShift resources. That's because there are various way to fix the problem, and for the training purpose, we will only let you fix it in some specific places.
  • Your colleague created all the required resource (pod, service, route). You won't need to create / delete any (in fact, you probably can't due to RBAC permission on this project), you just need to fix the existing one.
  • Network resources are visible under the Administrator perspective, rather than the Developer perspective.

Good luck!

Table of Contents

What's going on?

You head straight to the exercise04-userXX namespace, pick the "Developer perspective", check the topology view, and click on the deployment, only to find out that there is no route or service attached to this deployment. No wonder it's not working. But how to fix that?

topology

Poke around, click, explore, and fix it! If no luck after a few minutes, then check the hints below as needed.

Hints

Hint 1

How to check the network resources created?

You have been told network resources were created. Let's have a look. Networking is part of the "Administrator" perspective. Change perspective, pick "Networking", and check Services, Routes, Ingresses, and NetworkPolicies. What do you see? Does the Routes link work at all?

hint1

It looks like the "Wordle" route is using the "Wordle" service, which is good. Let's check the Service next?

Hint 2

How to check the Service is correctly load balancing to pod(s)?

OpenShift Services is an abstraction that expose a set of Pods over the network. In this case, the Wordle Service should expose the the wordle pod. You can check if that's the case by heading over the pod section of the service, see below.

hint2

Something is not correct. The Wordle pod is not displayed. You need to fix that. The following link should help you.

https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service

Once you fix it, the Wordle pod will appear in the Service's pod section, as shown below. If you can't get it to work, you will need to head over the solution section. In addition, the link for the route next to the pod will appear (check the topology view)

hint2

Hint 3

How to check which port an app is listening to?

At this stage, it's probably a good idea to check which port wordle is using. Sometime, this information is available in the application documentation, sometime the port is configurable (via a config file, or an environment variable), sometime the Containerfile (Dockerfile) provides some hints with the EXPOSE directive. Quite often, the application log is displaying which network interface / which port is used. Let's start by checking this:

hint3

Sometime, the application log display nothing. Either because the log level is too low, or because the way the container image has been build does not expose the log correctly. In those rare case, it might be necessary to connect to a running container, and inspect via some command lines tool to understand if the app is listening to some port. That's what we are doing next.

hint3

At this stage, we know the app is running, is listening on all interfaces (0.0.0.0), using port 3000.

Hint 4

How to check service/app port mapping?

We know the app is listening on port 3000. How about the wordle service? If we open a terminal session on any pod in your namespace, we should be able to query wordle:3000. We can even do that from the wordle pod. Let's give it a try.

hint4

Something is not right right. Is the wordle service even listening on port 3000?

Let's check the service

hint4

Looks like the service is listening on port 5000 (which is ok), but targeting port 4000 on the pod, which is not. Let's head over the YAML view and update that.

hint4

We can now test, from the terminal pod, that we can reach the container application via the wordle service on port 5000

hint4

It's good progress, you should check the route to see if it's working now.

Solution

Solution part 1

The first thing to do is to fix the service selector. You probably saw from Hint 2 that the service is not selecting ant pod. There is a typo, and currently the wordle service is looking for pod with the label app: poodle. If you check the wordle pod list of labels, you will notice that it is using app: wordle.

solution1

You could decide to update this label into the pod, (or much better, into the deployment's template). But you can only edit the services in this namespace, so go ahead and fix the service.

solution2

Now, you should see that the service is finally load balancing across a set of one pod. But that's not enough for this exercise. Moving on to the next part of the solution....

Solution part 2

The second thing to do is to fix the port mapping between the application (which listen port 3000), and the service (which target port 4000)

solution3

Hopefully, your route is now working, and you can have a well deserved game of wordle.

solution4

Did you know?

Contextual schema information

OpenShift provides contextual help for the Resource Definitions displayed on the console. Click on it when you need a bit more information.

duk1

OpenShift Administrator/Developer perspectives

Out of the box, you have an Administrator, and a Developer perspectives. You can customise those perspectives, or create new perspectives as you see fit!

https://docs.openshift.com/container-platform/4.15/web_console/customizing-the-web-console.html#odc-customizing-user-perspectives_customizing-web-console

Take away

There are multiple way to gather Networking information. The most experienced of you may have fixed everything within a couple of minutes, yet it's worthwhile looking at the various hints to understand how to gather information:

  • Looking at the Route definition
  • Looking at the Service definition
  • Checking the pods selected by a Service
  • Look at the container logs
  • Opening a terminal session from the pod to inspect network, and test connectivity