Thank you for a quality rebuttal, and for taking the time to read and analyse my original post. I'm not going to comment on the individual points, as I agree with most of them — taken in isolation. I particularly appreciate the point on securing database views; I may incorporate it into the post if you don't mind. And top points for sticking to the original Top Gear theme and incorporating The Stig.

My comment is going to relate to the overall piece.

I think the crux of the original post was misunderstood, or quite possibly, I may have failed…

[Image via Unsplash]

But for goodness’ sake, don’t

Engineers are constantly on the lookout for ways of doing more with less. This post a gentle reminder that less, contrary to the cliché, is not always more.

I am cheerfully reminded of an old Top Gear episode, in which the protagonists get £10,000 each to buy a used, mid-engined Italian supercar for a drive from Bristol to a Gentleman’s club in Slough. In a classical Top Gear fashion, carnage ensues: Hammond goes for a rust bucket Ferrari 308 GT4, Clarkson buys an “unstoppable” Maserati Merak (with brakes that had seen better days), and May brings in a Lamborghini Urraco…

On the similarities and differences between two industry heavyweights

My software architecture consulting gigs take me places ranging from startups to more established organisations, having one thing in common: they are looking to solve a problem using a distributed systems approach. Organisations are quick to embrace the benefits of microservices and event-driven architecture—scalability, supporting business change, polyglot culture, deployment flexibility—values that organisations are increasingly looking to adopt.

These architectural paradigms aren’t without their drawbacks too, however. Distributed systems are trickier to debug. Consistency and distributed state don’t play well. Concurrency is a lot more difficult to muster. …

Making Sense of Big Data

The road beyond event sourcing

The topics “event-driven architecture” “event stream processing” and “event sourcing” have enjoyed quite a buzz as of late. While the concepts are not new, it would seem that only now the software engineering community is beginning to appreciate the power and flexibility of building autonomous, loosely coupled systems that intelligently react to events, rather than being told what to do.

Now that our industry has gotten on board the event-driven bandwagon, some of us are starting to look beyond our usual building blocks — Kafka, Pulsar, and NATS Streaming — to address the concerns of long-term persistence and intelligent retrieval…

Getting the best of both worlds

Having been involved in several large-scale Kafka projects for different clients across a broad range of industries, I have heard my fair share of questions on Apache Kafka — ranging from the fundamental to the esoteric. One question that never seems to go out of fashion is: How can you maintain strict order, yet still process records in parallel?

And it’s a fair question. Strict order assumes linearizability, the very notion of which seems to contradict with the objectives of parallelism.

TL;DR Kafka is an Event Streaming Platform, while NATS is a closer to a conventional Message Queue. Kafka is optimised around the unique needs of emerging Event-Driven Architectures, which enrich the traditional pub-sub model with strong ordering and persistence semantics. Conversely, NATS is highly optimised around pub-sub topologies, and is an excellent platform for decoupling systems where message order and reliable delivery is a non-issue.

I’ll preface this post by pointing out that there is another product — NATS Streaming — which is a different beast and is closer to Kafka. …

Architecture, Use Cases, and a Getting Started guide — rolled into one

So, you’ve heard of this Kafka thing that’s popping up all over the shop. From green-sprout startups to greedy multi-nationals, in all sorts of weird and wonderful contexts. It appears that someone has figured out a Swiss Army knife piece of über-tech that works for microservices, event-streaming, CQRS, and just about anything else that makes the headlines.

As someone who’s been working with Apache Kafka since 2015, helping several clients build complex event-driven and microservices-style applications, I’ve heard my fair share of questions. First, it was I who was doing most of the asking. Then I found myself on the…

A journey of revival

TL;DR Kafdrop 3 is a reboot of the original Kafdrop that fills a gaping void in the observability tooling of the world’s most popular event streaming platform. It solves problems that the Kafka community has been crying out for, bringing Event Sourcing and Big Data into closer reach of many developers.

Prologue

The story told so many times: A tech company graced with a few highly-motivated individuals, in an attempt to boost its credibility and one-up its industry rivals, releases an open-source project. Something of genuine value; a truthful act of generosity as it were. …

Get a handle on the concept of perimeter defence and navigate the myriad of firewall types

With the phenomenal level of interconnectedness prevalent in the modern world, opportunities arise not just in legitimate commercial enterprises, but also in the more clandestine establishments — individuals or organisations that seek to take advantage of unprotected systems and networks, with an overarching motive to threaten their targets for the purpose of extortion, cause immediate and lasting harm, or directly profit from illegal activities. Of course, the topic of conversation is cybercrime. But the choice of wording is intentional: it is more instructive to view cybercrime as a business, albeit an illegitimate one.

While just about any target is theoretically…

Discover the deliberate design decisions that have made Kafka the performance powerhouse it is today.

The last few years have brought about immense changes in the software architecture landscape. The notion of a single monolithic application or even several coarse-grained services sharing a common data store has been all but erased from the hearts and minds of software practitioners world-wide. Autonomous microservices, event-driven architecture, and CQRS are the dominant tools in the construction of contemporary business-centric applications. To top it off, the proliferation of device connectivity — IoT, mobile, wearables — is creating an upward pressure on the number of events a system must handle in near-real-time.

Let’s start by acknowledging that the term ‘fast’…

Emil Koutanov

Software architect, an engineer, and a dad. Also an avid storyteller, maintainer of Kafdrop, and author of Effective Kafka.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store