Recently I worked in a project where I had to migrate a Mesos infrastructure to Kubernetes and one of the key requirements was to replace ELK (elasticsearch - logstash - kibana) with EFK (elasticsearch - fluentd - kibana). You can find a lot of information of how to use EFK in Kubernetes when your applications logs in the standard output putting a Fluentd agent in every node, but that did not happen with all the applications in the Mesos infrastructure.
Kubernetes is becoming the standard container orchestrator because of many reasons: Open Source Google Huge community Developed in Go and so on Today we can work in very complex projects which involved too many languages and technologies. Containers (like Docker) and orchestration tools help a lot in these cases. Elixir and Erlang are two languages that I work with, especially when I have to deal with problems of concurrency and low latency.
I like to read about everything but I like too much (yes, too much) to read about tech. Nowadays we are able to find a lot of kinds of books which are a good introductory step when you are going to begin with a new technology, however most of them are only a kind of Hello World. Although you have read a book, you probably know nothing about the technology. And when we talk about a non-mainstream technology the problem of finding a book which covers your needs grows.
New Web, let’s start again I have been offline for a lot of months, but it does not mean that I have been doing nothing (just the opposite). One of the things that I wanted to do is to move my blog from Weebly. It was not bad, but the writing experience was a bit unpleasure. After being experimenting with Hugo and writing in Markdown I decided to do the same with my blog.
Data Scientist is considered the sexiest job around the world, at least in the tech world, and it is true that sometimes the results that you can get applying Machine Learning techniques seem to be sorcery. Therefore a lot of people are trying to become data scientist. A lot of them use Python and almost all the on-line courses use also Python to teach the concepts. Python is a lovely programming language.
Two months ago I published in hex the first version of Galena , a Topic producer-consumer library built on top of GenStage for Elixir . It was initially designed to create some flows of producers , producer-consumers and consumers where the message has to be delivered as soon as possible. Each consumer or producer-consumer can receive messages from several producers and/or producer-consumers. Besides, consumers and producer-consumers can select the messages that want to receive thanks to a topic approach.
Life is a succession of asynchronous events which could depend on each other. Some weeks ago my friends and I wanted to play a football match, therefore we booked a football pitch of a sport centre for the next Saturday at 17pm. On that Saturday, I arrived to the sport centre at 16:44 and other guys were already there and they had arrived between 16:31 and 16:42.
This week I have released my two first contributions to the elixir open source community in hex : Galena and Conejo . Galena is a library based on GenStage that will help you to create your topic producer-consumer data pipelines. Conejo is an OTP application/library that facilitate the creation of publishers and consumers of RabbitMQ or AMQP servers.
Currently I am working in a project where I have to integrate many real-time data sources and show the results to the users as soon as possible. As you can imagine Elixir + PhoenixFramework are my chosen tools. This kind of projects always imply a lot of challenges that we have to solved (check my fork of Floki ). But the tedious one has not been directly related with the particular case of this project, it has been dealing with Brunch .
I like a lot RabbitMQ . It is a really good system when you have to work with events and build your systems with asynchrony in mind. Although silver bullets do not exist in the SW world, when you require a complex routing, scalability and reliability, RabbitMQ is your system. I usually work in a polyglot environment, but when one of the systems has to deal with a lot of connections (i.