Architecture Weekly Issue #46. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🤟 means hardcore, 👷‍♂️ is technically applicable right away,  🍼 - is an introduction to the topic or an overview. Now in telegram as well.

WARNING 🇺🇦

It's already been 284 days since Russia's crazy, brutal and unjustified war against Ukraine. We condemn this war and want it to stop ASAP. We continue this newsletter so you can advance your skill and help the millions of Ukrainian people in any way possible. If you want to help directly, visit this fund.

Video

Highlights

The biggest event of this week is definitely Amazon re:Invent and so far the most exciting announcement is the Amazon Lambda cold start improvement by 90%. This was the major showstopper for many services, and now no more. Another item worth noting is the start of the Advent of Code - everyday programming tasks for the whole of December.  

Synthesized.io and Testcontainers integration

We all love integration tests, and Testcontainers is an excellent tool to set up them. However, in order to have valuable tests we want to run them against the close-to-production data. This is where a Synthesized solution comes in handy. And the good news you can now use both products combined. Check out more details in the video.

GitHub - synthesized-io/tdk-tc: TDK-Testcontainers integration
TDK-Testcontainers integration. Contribute to synthesized-io/tdk-tc development by creating an account on GitHub.
GitHubsynthesized-io

Improving Kafka client performance by 40% by cleaning up unused Topic Metadata By LinkedIn 👷‍♂️

You might wonder how empty topics can slow down a Kafka Cluster, but they actually do: metadata is still stored in ZooKeeper, the partitions polls it etc. At scale it can be significant. LinkedIn Engineering Team came up with a TopicGC service, which will monitor for empty topics and drop them. Get more details inside.

TopicGC: How LinkedIn cleans up unused metadata for its Kafka clusters
Introduction Apache Kafka is an open-sourced event streaming platform where users can create Kafka topics as data transmission units, and then publish or subscribe to the topic with producers and consumers. While most of the Kafka topics are actively used, some are not needed anymore because busines…
LinkedInHao Geng

#kafka

Preparing for the Systems Design Interview 🍼

System Design interview is a common step in the hiring process in many companies.  Gergely Orosz made a cool write-up half a year ago on the books, courses, prep materials and mock interview resources which can help you to prepare. I can personally help with them too :)

Preparing for the Systems Design and Coding Interview
At Big Tech and high-growth startups, coding and systems design interviews are common - and fairly standard. A lot of people have asked me for preparation advice for these. Here is what I used when getting ready for an E5/E6 Facebook interview, and the one at Uber - where
The Pragmatic EngineerGergely Orosz

#systemdesign #course

Follow-Up

How do One-Time passwords work? 🤟

We all use the OTP for 2FA with Google, Github and other services. But how do they work? And who to implement the same scheme for your service? Read a short note below.

How do One-Time passwords work?
Many of us use one-time passwords (OTP) regularly to log into different services. Most probably rely on Google Authenticator and similar tools. But what about building one by ourselves?
ZSerge Zaitsev

#security #encryption

Create Your Distributed Database on Kubernetes with Existing Monolithic Databases  👷‍♂️

Kubernetes gained huge success in container orchestration. But using databases in with it is still not a completely solved problem. The article describes how you can upgrade your database to be distributed and pair it with your K8s workloads through Apache ShardingSphere that we have already spoke(in #34) before.  

Create Your Distributed Database on Kubernetes with Existing Monolithic Databases
The next challenge for databases is to run them on Kubernetes to become cloud neutral. However, they are more difficult to manage than the application layer, since Kubernetes is designed for stateless applications. Apache ShardingSphere is the ecosystem to transform any database into a distributed d…
InfoQTrista Pan

Discovering performance issues at Netflix 🤟

Netflix managed to triple it's RPS per VM Node. However in order to do that, they had to go way down to the JDK implementation and patch the two problems: "False Sharing" and immediately after that "True Sharing", which happened for super-class cache. A complex, but fascinating read about JVM optimization. If it was a bit hard to understand, there is an explanatory article here.

Seeing through hardware counters: a journey to threefold performance increase
By Vadim Filanovsky and Harshad Sane
Netflix TechBlogNetflix Technology Blog

#performance #jvm #vm

Event-Driven Architecture Patterns by Wix 🍼

Wix is a Saas, which allows you to create and host a website in minutes. Natan who is leading their data platform shares 6 receipts for different use cases like separating reads and writes for multiple services, providing real-time updates to a web-page and others - and all using Kafka!  

6 Event-Driven Architecture Patterns — Part 1
Event-driven Patterns that are used at Wix Backend that comprises 1400 microservices
Wix EngineeringNatan Silnitsky

Short ML Course on GANs 🤟

Machine Learning is everywhere. I am sharing a short course from Google, which explains how GAN - Generative adversarial networks - works. Check out the relations between a generator, discriminator and loss functions there!

Introduction | Machine Learning | Google Developers
Google Developers

#ml #course

In search of a robust model

The only constant in IT we have - is that everything changes. Technologies come and go, architecture approaches transform, the way we work change as well. Humans need some basis to rely on; Uwe suggests that a robust change model can save our psyche. He says, let's embrace the constant change and have a process of adopting new things. The full thought journey below.

In search of a robust change model
Coping better with continuous change
Uwe FriedrichsenUwe Friedrichsen

#vision

Folks! I am taking a short vacation next week, so see you in your inboxes 18th of December! Have a rest yourself or go through some previous issues.

Like the newsletter? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel, Robert and Roman for already supporting the newsletter.