Architecture Weekly #170

Architecture Weekly Issue #170. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🤟 means hardcore, 👷‍♂️ is technically applicable right away, 🍼 - is an introduction to the topic or an overview. Now in telegram and Substack as well.

Highlights

1000 Models at Instagram 👷‍♂️

Instagram definitely runs an ML model to provide you with recommendations, but there is a thousand more cases when models are involved like rating comments, suggesting tagged people and others. Running this number of models gets you challenged from the quality, speed and underlying infrastructure perspective. Luckily, instagram engineers share their journey.

#ml #performance #scalability

Is AI code sabotage its own ROI case? 🍼

Charity Majors, CTO at Honeycomb and a guest of our interview on Observability, shares the insights of how AI accelerated the code creation step, but shifting bottleneck into reviewing and shipping. This idea follows the Theory of Constraints - where once the bottleneck is removed, it emerges in other places.

#ai #devops

Welcome to the age of $10/month Lakehouses 👷‍♂️

Tobias Müller makes a great breakdown of the modern tech which you can use to store significant amount of data and run analytical SQL queries agains it for pennies. DuckDB + Cloudflare R2 and containers will make it a slick and efficient solution.

#dataengineering

Business Oriented System Design Course Cohort #6 is officially open!

Looking for a way to advance your career? Felt you overgrew the mere feature development, but lack skills to design complete systems? This course got you covered. 10 hours of content packed lectures, engaging practice and the final work you will be proud to showcase as well as Credly(by Pearson)-based digital certificate proving your experience. More than 70 engineers already passed the course with amazing feedback and advanced their careers. New cohort starts on 23rd of July. Details, Feedbacks and Enrollment into the course is here.

Follow-Up

Kafka: The End of the Beginning

This article argues that while Kafka won the adoption war, its protocol now throttles progress. The streaming ecosystem needs Hadoop-style reinvention—new runtimes, richer time semantics, and cloud-native designs—before it drifts into irrelevance.

#dataengineering

CRDT: Do Not Read! 🤟

CRDT is a great idea for collaborative editing. However applying it to other systems, like e-commerce, might turn into a tragedy of you paying for something you didn't actually ordered in the end. Great explanation in the post!

#distributedsystems

How Pendulum achieves 6x faster processing with S3 tables 👷‍♂️

S3 Tables are basically managed Iceberg files in S3 storage. Switching to it allowed to ditch Glue jobs improving both data processing time and cost simultaneously. Learn how it became possible.

#dataengineering #aws #serverless

The CTO's Relationship Calculus 🍼

Never looked at the relationships at work from a pure value-based perspective, but Etienne de Bruin offers not only approach, but the whole formula how you can evaluate your work relationships with the peers and make the adjustments to your interactions.

#stakeholdermanagement

Don't solve problems you don't have 🍼

I was guilty of that at my early years as a developer. Unnecessary refactorings, unimpactful improvements, discussions about non-existent problems. Unfortunately, engineers continue to do that because those problems seem to be more fun than something really useful to the business. Don't do that.

Don’t solve problems you don’t have. You’re literally creating problems.

Don’t make things harder than they need to be.

ohhfishal.netohhfishal

#philosophy

Big thanks to Nikita, Constantin, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August, Roman, Egor, Roman, Evgeniy, Nadia, Daria, Dzmitry, Mikhail, Nikita, Dmytro, Denis and Mikhail for supporting the newsletter on Patreon!