Architecture Weekly #116

Architecture Weekly Issue #116. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🤟 means hardcore, 👷‍♂️ is technically applicable right away,  🍼 - is an introduction to the topic or an overview. Now in telegram and Substack as well.

Highlights

Migrating Kafka Cluster to Kubernetes 👷‍♂️

At some point Zendesk decided to move the Kafka clusters from VMs to Kubernetes. However you can't just spin up the new cluster: you need to switch the clients and migrate the data. So Zendesk engineers created an elegant solution for seamless migration without changes at client sides. Get to know how!

Seamless Transition: Migrating Kafka Cluster to Kubernetes
A zero-downtime process with minimal changes to clients, and minimal manual intervention that maintains data integrity and consistency

#kafka #casestudyx

The most Common Domain-Driven Design Mistake 🍼

DDD is a powerful tool for designing microservices architecture and modular modoliths. Still it's easy to incorrectly identify the domains and introduce tons of problems in scope and teams. This post shows how to fight the problem where one domain is a single responsibility for multiple others.

The Most Common Domain-Driven Design Mistake
Mistaking Central Concepts for Bounded Contexts

#ddd

Latency numbers everybody should know 👷‍♂️

Satisfying performance requirements require handling latency issues. The basic foundation for it is understanding the latency numbers of basic things: time to trasnfer a MB of data to a nearby city over a 5G network, time to parse 1 MB of HTML and some more. Find them in this article!

Latency numbers every frontend developer should know – Vercel
Latency numbers every web developer should know

#performance

Follow-Up

Simple Design Doc explained in 4 minutes 🍼

People tend not to write software design documents because they don't know how to do that and think it takes a lot of time. I am showing in unders 4 minutes that the structure is straightfoward and explaining how to write a good oc.

#video #documentation

Beating the CAP Theorem Checklist 🤟

CAP Theorem can be fun... when you're creating a solution to beat it. This checklists is probably having the most reasons why the new idea proving the theorem wrong will fail. If you have one - definitely check it!

Beating the CAP Theorem Checklist

#theory #distributedsystems

Understanding Delta Lake's consistency model 🤟

Delta Lake is an open-source storage framework that enables building a format agnostic Lakehouse architecture. It is an evolution of the views on data warehouse and data lake architecture intended to provide ACID guarantees for data warehouses. Get this explanation how they can be achieved using layered architecture.  

Understanding Delta Lake’s consistency model — Jack Vanlightly
A few days ago I released my analysis of Apache Hudi’s consistency model , with the help of a TLA+ specification . This post will do the same for Delta Lake. Just like the Hudi post, I will not comment on subjects such as performance, efficiency or how use cases such as batch and streaming are sup

#dwh #dataengineering

Implemeting idempotent AWS Lamda Functions with Powertools for AWS Lambda 👷‍♂️

AWS Lambda was built arount the retries. It means you need to be careful considering what happens if your lambda is called with the same payload twice. Likely, there is a library to add idempotency to Lambda. Find out how

Implementing idempotent AWS Lambda functions with Powertools for AWS Lambda (TypeScript) | Amazon Web Services
This post is written by Alexander Schüren, Sr Specialist SA, Powertools. One of the design principles of AWS Lambda is to “develop for retries and failures”. If your function fails, the Lambda service will retry and invoke your function again with the same event payload. Therefore, when your functio…

#aws #serverless #lambda

How LiveWell leverages serverless architecture 👷‍♂️

I don't frequently see purely serverless architecture used in big products, and LiveWell by Zurich insurance company is an example. Find an elaborative post in AWS on how AWS Lambda, DynamoDB and many other AWS tools are used there.

How LiveWell by Zurich leverages serverless architecture to build scalable and cost-effective digital health platforms | Amazon Web Services
Introduction Zurich is a leading multi-line insurer that serves its customers in global and local markets. With about 55,000 employees, it provides a wide range of property and casualty, and life insurance products and services in more than 215 countries and territories. Zurich’s customers include i…

#serverless

How Discord Moved Engineering to Cloud Development Environments 👷‍♂️

Cloud9 appeared 10 years and I was thinking to switch to it, as it provided the possibility to write code anywhere with the internet access. Discord though did the same now migrated the developer experience from local machines to Cloud Development Environments with VScode. More details inside!

How Discord Moved Engineering to Cloud Development Environments
Join us as we tell the story of how we transitioned all of Discord’s backend and infrastructure development to a Linux-based Cloud Development Environment with the help of our friends at Coder.

#casestudy

WARNING 🇺🇦

The brutal and unjustified war against Ukraine continues already 2 years. If you want to help Ukraine directly visit this fund.

Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August, Roman, Egor, Roman, Evgeniy, Nadia, Daria, Dzmitry, Mikhail, Nikita, Dmytro, Denis and Mikhail for supporting the newsletter. They receive early access to the articles, influence the content and participate in the closed group where we discuss the architecture problems. They also see my daily updates on all the things I am working on. Join them at Patreon or Boosty!