https://blog.lwolf.org/categories/kubernetes/ Recent content in kubernetes on lwolfs blog Hugo -- gohugo.io en-us Thu, 30 Jan 2020 00:00:00 +0000 https://blog.lwolf.org/post/agola-one-ci-to-run-them-all/ Thu, 30 Jan 2020 00:00:00 +0000 https://blog.lwolf.org/post/agola-one-ci-to-run-them-all/ I’ve been looking for a good Continues Integration(CI) system for quite some time now. System, that could work with all the major repository providers and run on my hardware. The need for multi-provider support comes from the fact that my code is spread across: Github Gitlab, cloud and self-hosted Gitea - I’m currently replacing my self-hosted Gitlab with it. I find it much simpler and faster. This means I can neither have a single view of all my tests nor have a unified way of testing. https://blog.lwolf.org/post/switching_to_istio_as_the_primary_ingress/ Mon, 08 Apr 2019 00:00:00 +0000 https://blog.lwolf.org/post/switching_to_istio_as_the_primary_ingress/ I’ve been following the news about istio since it’s first alpha release in 2017. I think this project has a great future, because it solves a lot of pain points in the microservice based architecture, like auth, observability, fault-injection, etc. But the fact is, I never actually tried it. Prior to v1.0 there were too many bugs and limitations to put it into production. So, when they finally released “production-ready” version I got quite exciting. https://blog.lwolf.org/post/how-to-deploy-multi-arch-kubernetes-cluster-using-kubespray/ Sat, 02 Feb 2019 00:00:00 +0000 https://blog.lwolf.org/post/how-to-deploy-multi-arch-kubernetes-cluster-using-kubespray/ How to deploy multi-arch Kubernetes cluster using Kubespray I recently bought 3 ODROID-HC1 devices to add a dedicated storage cluster to my home Kubernetes. I thought that it’s a good excuse to spend some time redeploying the cluster. Usually, I would’ve gone with CoreOS, since I’m a big fan of their immutable OS. Unfortunately, that is not an option if you have ARM nodes. So I had to choose between manual provisioning and Ansible. https://blog.lwolf.org/post/home-lab-infrastructure-overview/ Wed, 23 Jan 2019 00:00:00 +0000 https://blog.lwolf.org/post/home-lab-infrastructure-overview/ Home Lab Infrastructure Overview Every software or technology I blog about usually goes through my home lab first. A lot of people usually got surprised when they first hear that I have a multi-node Kubernetes cluster at home. It usually takes some time to tell them about all the machines and networking. Of course, not accounting for the time spent answering the question “why do you need it”. I added a few new devices and reconfigured everything from scratch recently. https://blog.lwolf.org/post/going-open-source-in-monitoring-part-v-collecting-errors-from-production-using-sentry/ Wed, 06 Sep 2017 00:00:00 +0000 https://blog.lwolf.org/post/going-open-source-in-monitoring-part-v-collecting-errors-from-production-using-sentry/ In this part of the series, I’m going to add Sentry to the monitoring stack. For those who do not know, Sentry is an open-source cross-platform crash reporting and aggregation platform written in Python. I know about it for several years now, since the time it supported only a few languages. A lot has changed since my last use of it: new domain, new languages, new integrations. Today it has integrations with most of the modern languages and frameworks like Elixir, React, React-Native, Go. https://blog.lwolf.org/post/making-long-running-sitemap-generation-kubernetes-friendly-using-k8s-api-and-minio/ Sun, 03 Sep 2017 00:00:00 +0000 https://blog.lwolf.org/post/making-long-running-sitemap-generation-kubernetes-friendly-using-k8s-api-and-minio/ Generation of sitemaps is usually not a problem, at least not until you have several millions of items in the database to iterate through. Currently, I have slightly more than 4M documents, so it takes time to regenerate. Previously all my sitemaps were stored on the disk and were served using simple Nginx container. It worked like this: Sitemap-related containers have nodeSelector to assign to particular node with hostPath volume sitemaps are served from /data/sitemaps folder periodic task generates new sitemaps to /data/sitemaps-temp on completion, folders is atomically switched It was fine in my previous VM-based cluster, which was on a single physical machine. https://blog.lwolf.org/post/going-open-source-in-monitoring-part-iii-10-most-useful-grafana-dashboards-to-monitor-kubernetes-and-services/ Fri, 21 Jul 2017 00:00:00 +0000 https://blog.lwolf.org/post/going-open-source-in-monitoring-part-iii-10-most-useful-grafana-dashboards-to-monitor-kubernetes-and-services/ This post is one of a series of posts about monitoring of infrastructure and services. Other posts in the series: Intro Deploying Prometheus and Grafana to Kubernetes Creating the first dashboard in Grafana 10 most useful Grafana dashboards to monitor Kubernetes and services (this article) Configuring alerts in Prometheus and Grafana Collecting errors from production using Sentry Making sense of logs with ELK stack Replacing commercial APM monitoring SLA, SLO, SLI and other useful abstractions There are dozens of ready dashboards available on grafana. https://blog.lwolf.org/post/going-open-source-in-monitoring-part-ii-creating-the-first-dashboard-in-grafana/ Mon, 19 Jun 2017 00:00:00 +0000 https://blog.lwolf.org/post/going-open-source-in-monitoring-part-ii-creating-the-first-dashboard-in-grafana/ This post is one of a series of posts about monitoring of infrastructure and services. Other posts in the series: Intro Deploying Prometheus and Grafana to Kubernetes Creating the first dashboard in Grafana (this article) 10 most useful Grafana dashboards to monitor Kubernetes and services Configuring alerts in Prometheus and Grafana Collecting errors from production using Sentry Making sense of logs with ELK stack Replacing commercial APM monitoring SLA, SLO, SLI and other useful abstractions Having grafana. https://blog.lwolf.org/post/going-open-source-in-monitoring-part-i-deploying-prometheus-and-grafana-to-kubernetes/ Mon, 29 May 2017 00:00:00 +0000 https://blog.lwolf.org/post/going-open-source-in-monitoring-part-i-deploying-prometheus-and-grafana-to-kubernetes/ This post is one of a series of posts about monitoring of infrastructure and services. Other posts in the series: Intro Deploying Prometheus and Grafana to Kubernetes (this article) Creating the first dashboard in Grafana 10 most useful Grafana dashboards to monitor Kubernetes and services Configuring alerts in Prometheus and Grafana Collecting errors from production using Sentry Making sense of logs with ELK stack Replacing commercial APM monitoring SLA, SLO, SLI and other useful abstractions I’ve been keeping my eye on Prometheus for some time. https://blog.lwolf.org/post/going-open-source-in-monitoring-part-0-intro/ Fri, 12 May 2017 00:00:00 +0000 https://blog.lwolf.org/post/going-open-source-in-monitoring-part-0-intro/ Intro (this article) Deploy and basic configuration of Prometheus Creating the first dashboard in Grafana 10 most useful Grafana dashboards to monitor Kubernetes and services Configuring alerts in Prometheus and Grafana Collecting errors from production using Sentry Making sense of logs with ELK stack Replacing commercial APM monitoring SLA, SLO, SLI and other useful abstractions Monitoring of the infrastructure is an essential part of any product. But it’s not uncommon for companies to postpone monitoring for the later period. https://blog.lwolf.org/post/how-to-recover-data-from-broken-glusterfs-cluster/ Fri, 31 Mar 2017 00:00:00 +0000 https://blog.lwolf.org/post/how-to-recover-data-from-broken-glusterfs-cluster/ A few days ago when I tried to install helm chart in my Kubernetes cluster I noticed that all new pods that required storage were in pending state. After a quick check of the logs, I found out that pods were unable to get PVC from GlusterFS. I recently wrote about my experience deploying GlusterFS cluster. This time I will go through recovering data from the broken GlusterFS cluster, and some problems I faced deploying new cluster. https://blog.lwolf.org/post/how-to-create-ci-cd-pipeline-with-autodeploy-k8s-gitlab-helm/ Thu, 30 Mar 2017 00:00:00 +0000 https://blog.lwolf.org/post/how-to-create-ci-cd-pipeline-with-autodeploy-k8s-gitlab-helm/ Since my previous posts[1][2] about CI/CD, a lot have changed. I started using Helm for packaging applications, stopped using docker-in-docker in gitlab-runner. Recently, I started working on a few Golang microservices. I decided to try gitlab’s caching and split the job into multiple steps for better feedback in UI. Few of the main changes to my .gitlab-ci.yaml file since my previous post: no docker-in-docker using cache for packages instead of a prebuilt image with dependencies splitting everything into multiple steps. https://blog.lwolf.org/post/how-to-deploy-ha-postgressql-cluster-on-kubernetes/ Mon, 06 Mar 2017 00:00:00 +0000 https://blog.lwolf.org/post/how-to-deploy-ha-postgressql-cluster-on-kubernetes/ Creating a high available PostgreSQL cluster always was a tricky task. Doing it in the cloud environment is especially difficult. I found at least 3 projects trying to provide HA PostgreSQL solutions for Kubernetes. Patroni Patroni is a template for you to create your own customized, high-availability solution using Python and - for maximum accessibility - a distributed configuration store like ZooKeeper, etcd or Consul. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in the datacenter - or anywhere else - will hopefully find it useful. https://blog.lwolf.org/post/fully-automated-gitlab-installation-on-kubernetes-including-runner-and-registry/ Mon, 27 Feb 2017 00:00:00 +0000 https://blog.lwolf.org/post/fully-automated-gitlab-installation-on-kubernetes-including-runner-and-registry/ After my previous post - How to easily deploy GitLab to Kubernetes - I’ve got a lot of responses. I helped several people to get GitLab up and running on their clusters. With that manifests it became much easier to deploy GitLab to Kubernetes. It was still few things that were usually misunderstood and misconfigured. You can find out these things only by talking with people who are actually trying to use your work. https://blog.lwolf.org/post/how-i-deployed-glusterfs-cluster-to-kubernetes/ Thu, 09 Feb 2017 00:00:00 +0000 https://blog.lwolf.org/post/how-i-deployed-glusterfs-cluster-to-kubernetes/ I spent last two days installing GlusterFS storage on top of my Kubernetes. It took much more time and effort than it should. I faced all kinds of problems, some if which were not obvious and took a lot of googling. So I decided to write this post. Hopefully it will save some time for somebody. I was playing with helm. Trying to assemble a complex application with several dependencies from official chart repository. https://blog.lwolf.org/post/how-to-run-functions-in-your-kubernetes-cluster/ Mon, 16 Jan 2017 00:00:00 +0000 https://blog.lwolf.org/post/how-to-run-functions-in-your-kubernetes-cluster/ Serverless or Function as a Service What is Serverless/FaaS? Serverless is a new paradigm in computing that enables simplicity, efficiency and scalability for both developers and operators. For a long time, there was only one FaaS implementation available for Kubernetes - Funktion. Which I found a bit complicated and tightly coupled with the fabric8 platform. They claim to support functions in any language, but I found only java and nodejs examples in the repo. https://blog.lwolf.org/post/how-to-speed-up-builds-and-save-time-and-money/ Thu, 01 Dec 2016 00:00:00 +0000 https://blog.lwolf.org/post/how-to-speed-up-builds-and-save-time-and-money/ In one of the previous posts, I described the way I’m using GitLab to build and test images. Despite the fact that it’s pretty simple configuration and actual tests are run for less than a minute, complete pipeline takes around 20 minutes. Which seems very unreasonable. I spent some time digging out the problem and was able to reduce build time to around 3 minutes. Here are few things, that should be considered to optimize pipeline. https://blog.lwolf.org/post/how-to-get-free-dyndns-with-digitalocean-and-kubernetes-scheduled-jobs/ Fri, 04 Nov 2016 00:00:00 +0000 https://blog.lwolf.org/post/how-to-get-free-dyndns-with-digitalocean-and-kubernetes-scheduled-jobs/ DynDNS is annoying/expensive I’m using different DynDNS services at home for the last few years. It’s easy to use, comes preinstalled on most of the routers. The only annoyance is the need to confirm each of your hosts every 30 days And it’s fine because you do not pay anything. Recently, I wanted to attach proper domain to my home server to be able to access my Kubernetes from the outside. https://blog.lwolf.org/post/continuous-deployment-to-kubernetes-from-gitlab-ci/ Mon, 10 Oct 2016 00:00:00 +0000 https://blog.lwolf.org/post/continuous-deployment-to-kubernetes-from-gitlab-ci/ In my previous posts I described how to deploy GitLab to Kubernetes and configure GitLab-CI to build and test docker containers. In this one, I’m going to write about continues deployments. I assume that you already have Kubernetes cluster and application running, and you have some manual way of deploying your application to it. So I will not touch the basics of writing Kubernetes manifests. But I will briefly describe my own scripts and show how I configured GitLab-CI to automate deployment. https://blog.lwolf.org/post/how-to-easily-deploy-gitlab-on-kubernetes/ Thu, 08 Sep 2016 00:00:00 +0000 https://blog.lwolf.org/post/how-to-easily-deploy-gitlab-on-kubernetes/ I spent some time recently researching how to build CI/CD pipeline to automate testing and deploying. Since I’m developing everything in containers and use Kubernetes to manage it GitLab with its CI runners and recent integrations with k8s seemed like a good option. In this post, I will describe the steps needed to get GitLab with GitLab Runner, up and running on top of Kubernetes. All manifests used in this post could be found here https://blog.lwolf.org/post/bootstrapping-coreos-cluster-with-k8s-in-20-min-using-bootkube/ Mon, 22 Aug 2016 00:00:00 +0000 https://blog.lwolf.org/post/bootstrapping-coreos-cluster-with-k8s-in-20-min-using-bootkube/ It’s 4 months now since I migrated my infrastructure to CoreOS and Kubernetes. Now I can say that this was the right decision. Or, even, the best thing that happened to my servers :). Of course, there were some problems during this period, but mostly because of some misconfiguration from my side. Also, there was no single downtime because of infrastructure. My original post was very long and compicated. Actually, even I don’t want to reproduce all of this stuff ever again. https://blog.lwolf.org/post/migrate-infrastructure-to-kubernetes-part2-basic-services/ Mon, 20 Jun 2016 00:00:00 +0000 https://blog.lwolf.org/post/migrate-infrastructure-to-kubernetes-part2-basic-services/ In previous post I finished description of installation of kubernetes cluster on bare-metal hardware. At this point we should be able to communicate with it using kubectl In this post I will go through installation of basic services to use and monitor cluster. For example DNS, heapster and different dashboards. Deploying addon services Kubernetes comes with several very useful addons, available on its github, either in kubernetes or in contrib. https://blog.lwolf.org/post/hosting-own-docker-registry/ Wed, 08 Jun 2016 00:00:00 +0000 https://blog.lwolf.org/post/hosting-own-docker-registry/ First thing you need if you’re using Kubernetes - Docker registry. Because its all about containers. So in this post I will show how to deploy your own registry inside Kubernetes cluster, with UI and tls, with basic http authentication. I’m going to use cluster I deployed in previous post. As short recap - we have Kubernetes cluster with few nodes, and external loadbalancer (ubuntu based machine with nginx) Get ssl certificates from Let’s Encrypt To have proper registry opened to the web, we need to get ssl certificates. https://blog.lwolf.org/post/migrate-infrastructure-to-kubernetes-building-baremetal-cluster/ Wed, 08 Jun 2016 00:00:00 +0000 https://blog.lwolf.org/post/migrate-infrastructure-to-kubernetes-building-baremetal-cluster/ I started trying to switch to Docker about a year ago, but all tools were kind of not-production-ready. With docker-compose it was unreal to scale containers without restart. Deis looked like a black box with a lot of magic. Then Tutum appeared and it was awesome, really, it was the first working solution. So I switched to it. It was fine most of the time, yes it had problems with networking, yes it was annoying to copy-paste all environment variables into each container, but it was in beta, and it was free.