How to kill a stuck kubernetes namespace

If you are an hands on person dealing with cloud related technologies, such as container and kubernetes clusters, the following script could avoid you same headaches.

While playing with kuberentes, if it has not alredy happend to you, soon or later, you will stumble in a kubernetes namespace that remains in the “terminating” state for ever.

Usually this happens when a certain application is uninstalled in the wrong way or some weird bug led to this awkward situation.

If you don’t want, or you can’t, rebuild the entire cluster from scratch, forcing the deletion of the namespace can be a good option.

To force the deletion of a stuck namespace, you can try to run the following script by passing the problematic namespace. The only requirement is to have the kubectl command with the kube context properly configured against the target kubernetes cluster.



die() { echo "$*" 1>&2 ; exit 1; }

need() {
which "$1" &>/dev/null || die "Binary '$1' is missing but required"

need "kubectl"

test -n "$STUCK_NS" || die "Missing namespace: kill-ns <namespace>"

kubectl get namespace $STUCK_NS -o json | tr -d "\n" | sed "s/\"finalizers\": \[[^]]\+\]/\"finalizers\": []/"  | kubectl replace --raw /api/v1/namespaces/$STUCK_NS/finalize -f -

Pivotal Cloud Foundry at Home

This post is to share my experience in deploying Pivotal Cloud Foundry on my own server.

Cloud Foundry is one of the many PaaS (Platform As a Service) option available on the market and it comes in 3 different flavors:

  1. Cloud Foundry is open source and community based. It is possible to install on premises to offer a PaaS to the organization.
  2. Pivotal Cloud Foundry is a Pivotal version of Cloud Foundry. Starting from the base product, a number of features have been customized and enhanced with a commercial support available. Like the first option, it can be installed on premises.
  3. Pivotal Web Services is the Pivotal Cloud Foundry offered as a managed service. For those who wants to try it out or don’t want to maintain the platform, it is definitely the fastest and probably the cheapest option. In this case, there is nothing to install since the platform is maintained by Pivotal Web Services.

In my case, I have already an account with Pivotal Web Services and I already know how it works from a developer/user point of view.

Only for education purpose, I wanted to experience the deployment of Pivotal Cloud Foundry as well on my home network although I was fully aware that my hardware did not meet the requirements (not enough RAM on my server). I liked the idea to have a kind of Cloud at home (even for a short time).

By reading the official documentation of the Pivotal Cloud Foundry on vsphere environment, I’ve realized that I had to re-organize my current setup in order to have a chance of success.

In particular the issues were as follow:

  1. VMWare vCenter is not licensed as free product and its license cost would not justify a Home Lab setup like mine.
  2. Minimum 128 GB RAM while my ESXi has only 32 GB.

I resolved the first point by upgrading my ESXi 5.5 to the latest ESXi 6.5. By doing so, I received a full trial license for two months that I can use to connect a trial version of vCenter to my ESXi.

In addition, to isolate and optimize the performance of network traffic between Cloud Foundry VM, I’ve created a VLAN (new Port Group) connected to my existing virtual switch. I’ve created a dedicated subnet network ( for the Cloud Foundry Virtual Machines. Finally, I created a gateway VM to allow routing between the networks and (DNS server is on as well as all my home devices)

Following picture shows the Virtual Switch along with the new Port Group (VM-

To resolve the issue of the low RAM:

  • After installed vCenter appliance ony top of my  ESXi, I’ve moved it to a desktop PC running VMWare Player. In this way I saved 10 GB of RAM.
  • Downsize the resource configuration of VM within Pivotal Ops Manager as well as Pivotal Elastic Runtime
  • Configure external MySQL database running ony my QNAP instead of Pivotal built-in. Unfortunately it did not work for me. Probably due to a wrong version of MySQL. The issue was with the execution of a SQL flywaydb script.
  • Disable all errands

As shown by the following images, I managed to deploy it with still some memory available on my server

The Pivotal Cloud Foundry Virtual Machines visible under pcf_vms folder

The Virtual Machines have been configured with the following resources: