Architecture Weekly #152
Architecture Weekly Issue #152. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🤟 means hardcore, 👷♂️ is technically applicable right away, 🍼 - is an introduction to the topic or an overview. Now in telegram and Substack as well.
System Design Course
Looking to advance your system design skills further? I've got a Business Oriented System Design Course to help you! The Cohort #4 starts January, 29th. 10 slots left. Follow this page: https://vvsevolodovich.dev/business-oriented-system-design-course/
Highlights
Modern Hardware Numbers 👷♂️
For a realistic system design you need to calculate stuff: bandwith, storage capacity etc. The hardware does not stop improving, so it is crucial to understand the limits. Find the post with the numbers, however it does not mention the cost unfortunately.
#systemdesign
Getting Storage Engines Ready for Fast Storage Devices 🤟
As mentioned above, hardware improves so everything should catch up, including database engines. MongoDB showed how using memory mapped files and some file space allocations allowed a 60% performance improvement in the third of the benchmarks. Follow the read.
#db #performance
Infrastructure Vendors Are in a Tough Spot 🍼
Chris Riccomini wrote a long read about the WarpStream and Confluent relationship - especially why Confluent could not built WarpStream inside due to cloud era, AI productivity boost for developers, legacy burden and other reasons.
#infrastructure #kafka
Follow-Up
Why aren't we all serverless yet? 🍼
Serverless approach brings 2 valuable fruits: paying only for the consumed resources and skip worrying about setting up infrastructure in the first place.
#serverless
So You Want To Build Your Own Data Center 👷♂️
Typical software engineer work lies in codes and maximum yaml configs. Even SREs are mostly working with networks and resources in a cloud or in an existing data center. What if you want to move to a your own setup like Railway wanted to? That's an entire universe of things to be concerned about, so find this articles about server racks, power, ISPs and cooling.
#hardware
Architecture is a game of constraint satisfaction 🍼
To continue on the topic of both data centeres and modern hardware numbers. Gregor Hohpe argues that you can actually get 4 nines of availability and stupid amount of compute and storage with a single beefy server nowadays, which brings the question: what technology constraint the cloud actually removes? But if you can't answer the question you don't get an edge for your business.
#cloud
Configuration Management: Lessons Learned from managing 1000+ microservices 👷♂️
Bolt has 4 business lines with different load profiles and availability targets, which made us into microservises. But provisioning and supporting them is a task itself, and at Bolt we love be efficient. Ivan Nikulin from Foundation team shares how we came up with an efficient, scalable system for configuring and provisioning services.
#video #sre
Kafka Transactions explained... twice 🤟
Imagine you're processing a topic and count the number of messages per minute and then write it down to another topic. In the situation of a process crash you can either double count them or miss them all. However, Kafka supports transactions, so you can commit the offset on the first topic and make the messages on the second one visible as an atomic operation. Read how it actually works.
#kafka
Big thanks to Nikita, Constantin, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August, Roman, Egor, Roman, Evgeniy, Nadia, Daria, Dzmitry, Mikhail, Nikita, Dmytro, Denis and Mikhail for supporting the newsletter on Patreon! If you like the newsletter, feel free to support it there - with one-time support for example!