Architecture Weekly #74

Architecture Weekly Issue #74. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: ๐ŸคŸ means hardcore, ๐Ÿ‘ทโ€โ™‚๏ธ is technically applicable right away, ย ๐Ÿผ - is an introduction to the topic or an overview. Now in telegram as well.

WARNING ๐Ÿ‡บ๐Ÿ‡ฆ

It's already been a year since Russia's crazy, brutal and unjustified war against Ukraine. We condemn this war and want it to stop ASAP. We continue this newsletter so you can advance your skill and help the millions of Ukrainian people in any way possible. If you want to help directly, visit this fund.

Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August, Roman, Egor, Roman, Evgeniy, Nadia and Daria for supporting the newsletter. They receive early access to the articles, influence the content and participate in the closed group where we discuss the architecture problems. They also see my daily updates on all the things I am working on. Join them at Patreon or Boosty! ย 

Highlights

Frugal Software Architecture. Cost Optimization ย ๐Ÿผ

Next chapter of Frugal Software Architecture inbound. This time we consider Cost Optimization and figure out cost drivers in software architecture, explore strategies for cost optimization, discuss the balance between cost optimization and performance, provide practical steps to implement cost optimization and illustrate these concepts with real-world case studies.

Frugal Software Architecture. Part 3: Cost Optimization
The next principle of frugal software architecture is cost optimization. Find out what exactly it means.

#frugality

MongoDB vs PostgreSQL vs ScyllaDB ๐Ÿ‘ทโ€โ™‚๏ธ

This brief yet insightful piece uncovers three key lessons from our journey comparing and migrating NoSQL databases. Tractian engineers found that each database is tailored for specific use cases, and thorough testing reveals the best fit. Adaptability and an openness to change are vital in this process, often leading to major, but necessary, engineering shifts. The ultimate goal is to select a system that supports your product's bright future.

MongoDB vs. PostgreSQL vs. ScyllaDB: Tractianโ€™s Experience
How the industrial monitoring systems vendor selected the best database for its real-time machine learning environment.

#casestudy #db #postgre

Hard Stuff Nobody Talks about when Building Products ๐Ÿ‘ทโ€โ™‚๏ธ

LLM-based startups are blooming like cryptocurrency ones several years ago. This time the technology is really useful but does not mean it's coming without problems. In the blog of Honeycomb.io they tell the story of using LLM to provide natural language for their observability platform and how hard it is from the performance, correctness and relevancy perspective. Exciting read!

All the Hard Stuff Nobody Talks About when Building Products with LLMs
In this post, Phillip talks through the challenges & pitfalls of LLMs we faced when building our Query Assistant - and that you too may face.

#db #vectordb #database

Follow Up

What is a Vector Database ๐ŸคŸ

In the context of AI and machine learning, embeddings generated by models like Large Language Models pose significant management challenges due to their numerous attributes. Specialized vector databases, such as Pinecone, cater to these needs with optimized storage and querying capabilities that traditional scalar-based databases and standalone vector indexes lack. Their specific design for handling complex, large-scale data allows for superior performance, scalability, and flexibility, enabling insights extraction and real-time analysis. A vector database enhances AI capabilities with advanced features, such as semantic information retrieval and long-term memory, underlining its critical role in data-intensive applications.

What is a Vector Database? | Pinecone
The nature of vector embeddings require new methods of storage and retrieval. We need a new kind of database.

EventStorming Tips ๐Ÿผ

EventStorming is a powerful technique to identify the domains boundaries and design the solution based on the business domain. However, it can bring less value if done incorrectly. Find an article with the tips on how to make the most from your EventStorming Sessions.

EventStorming Modelling Tips to Facilitate Microservice Design
EventStorming is a hugely popular technique, allowing us to explore, analyse, and model business domains more effectively than traditionalโ€ฆ

#microservices #eventstorming #event

Detecting AI-Generated Profile Photos ๐Ÿ‘ทโ€โ™‚๏ธ

It became increasingly hard for humans to distinguish the AI-generated profile picture from the real one. This problem is especially annoying for social networks like LinkedIn. They decided to conduct a research and discovered a way to see whether the image is generated or more likely a real one. Follow the research below. ย 

New Approaches For Detecting AI-Generated Profile Photos
Co-authors: Shivansh Mundra, Gonzalo Aniano Porcile, Smit Marvaniya, Hany Farid

#ai

Modular Architecture for Development Teams ๐Ÿผ

Find another story of an overgrown monolith which slows down the feature development. Once you added significantly high amount of features to your application, adding more is getting more time. Not coming as a surprise, but applying the platform approach and Domain-Driven Design can really help. Follow the story in Martin Fowler's blog.

Linking Modular Architecture to Development Teams
How to improve software delivery with a modular architecture

#casestudy #ddd

Migrating Netflix to GraphQL Sarfely ๐Ÿ‘ทโ€โ™‚๏ธ

Netflix migrated to GraphQL with their whole mobile fleet last year. Now they are telling the story of the successful migration: introducing the facade first, fighting the problems with security and performance, and then do underlying changes. Checkout the post in their technical blog!

Migrating Netflix to GraphQL Safely
By Jennifer Shin, Tejas Shikhare, Will Emmanuel

#graphql #migration

Building distributed RocksDB with OmniPaxos in 8 minutes ๐ŸคŸ

The post will demonstrate how to use OmniPaxos to build a simple distributed database. It will bring the system from a single-server database service to a distributed setup with multiple servers replicating the database. OmniPaxos will act as the replicated changelog for the distributed database that provides a single execution order for all replicas so they remain consistent.

Building distributed RocksDB with OmniPaxos in 8 minutes
In this post, we will demonstrate how to use OmniPaxos to build a simple distributed database. As seen in the Figure below, we will go from a single-server database service to a distributed setup with multiple servers replicating the database. OmniPaxos will act as the replicated changelog for the dโ€ฆ

#db #distributedsystems