• DDD Weekly: Issue #60 - Patterns for Decoupling in Distributed Systems [twitter/blog] Mathias Verraes. Consumer-driven Coupling: Patterns and Anti-patterns [blog] Nick Tune. Designing microservices can feel quite easy at times. And that’s because it is at first. However we slice our system and our teams, it usually works to begin with. The cost of poor design doesn’t begin to manifest until the system starts to scale. The feedback loop on sociotechnical design choices is long. [DDD Weekly]
• Messaging Patterns: Natural Language Message Names - Problem Our industry has a tradition of unnatural, compact, lossy naming. This may save on typing, but costs in understandability. Solution Naming is hard, because we name things. If we understand the domain and have discussed with domain experts, we don’t have to invent names, we can use the names from that domain. And no domain experts says Payment Event or Invoice Paid or OnAfterPayment, they say The invoice was paid. Use descriptive, natural language names. [Mathias Verraes]
• Eventsourcing Patterns: Migration Events in a Ghost Context - Explicitly conform to the legacy component’s model in an eventsourcing migration. Problem We are replacing a Legacy component with a new Eventsourced component. Besides being fully Eventsourced, the new component has a clear domain model with a well-defined Ubiquitous Language. The domain events are granular, and named in this language, and track the entire history of the component. The new component needs to retain and use data produced by the old component. The old component stores state, but no history. [Mathias Verraes]
• Eventsourcing Patterns: Decision Tracking - Decision Tracking Store the outcome of a decision to guard against rule changes. Problem In an eventsourced system, a consumer listens to a stream of events. IT has some business logic that makes decisions, and acts upon those, for example in the form of a side effect or a new event. Because we’ve already stored the input events in the event store, we can recalculate that decision. Should our event store persist an event that represents that decision? Solution Often, the reasoning is that the event that represents the decision, is redundant. We could store it, but we can recalculate it, so we don’t have to store it. [Mathias Verraes]

Permalink | From 27 May 2019 to 02 June 2019 | Last updated on: Tue, 7 Sep 2021 15:26:25 GMT

• Messaging Patterns: Throttling - Throttling Only keep one event per time unit from a high frequency stream. Problem A stream of Ephemeral Events is too much to handle for a consumer. Solution An intermediary consumer is placed in front of the original consumer. It drops all events, except once per time unit, where it passes the event to the original consumer. Example This pattern is very common in user interfaces. An input device, such as a mouse, emits it’s position at 500 Hz. [Mathias Verraes]
• Messaging Patterns: Ephemeral Events - Ephemeral Events An event’s lifetime lasts until the next event. Problem Some producers, such as sensors, emit realtime or near-realtime measurements, expressed as events. A new measurement makes the previous measurement obsolete. These events can be sent at regular or unpredictable intervals, but they are often high frequency. Consumers aren’t capable of keeping up with all events, or aren’t interested in processing all of them. Solution Mark the events as being Ephemeral Events. [Mathias Verraes]
• Messaging Patterns: Change Detection Events - Change Detection Events Listen to a stream of events and produce a new event when a value changes. Problem A consumer is interested in an event stream, but the consumer only wants to know when a particular value changes, and doesn’t care about the events where that value didn’t change. Optionally, the consumer only cares when the value changes more than a given interval. Solution A second intermediary consumer is placed in front of the original consumer. It acts as a filter, emitting Change Detection Events when the observed value is different from the previous event. It may also ignore changes that are too small for the original consumer. [Mathias Verraes]
• DDD and Messaging Architectures - My aim is to identify, name, and document patterns, that I’ve used or encountered in my consulting practice. All patterns are anti-patterns when used in the wrong context, so I try to add heuristics, trade-offs, and caveats. You can help: tell me your experiences, point me to other people’s writings on patterns, distill your own heuristics, … Meta Eventsourcing: State from Events or Events as State? Patterns Are Not Defined by Their Implementation Patterns for Decoupling in Distributed Systems Domain Queries Summary Event Completeness Guarantee Passage of Time Event Fat Event Explicit Public Events Segregated Event Layers Eventsourcing Patterns Forgettable Payloads Crypto-Shredding Decision Tracking Migration Events in a Ghost Context Multi-temporal Events General Messaging Patterns Ephemeral Events Throttling Change Detection Events Natural Language Message Names Testing EventSourcing Testing Patterns Note: I teach workshops about DDD for Messaging Architectures. [Mathias Verraes]

Permalink | From 20 May 2019 to 26 May 2019 | Last updated on: Sun, 28 May 2023 22:06:33 GMT

• Reconciling New Design Approaches with What You Already Know - Last week at the deliver:Agile conference in Nashville I attended a talk by Autumn Crossman explaining the benefits of functional programming to us old timey object-oriented designers. I also attended the session led by Declan Whelan and Sean Button, on … Continue reading → [The Responsible Designer]
• Big Ball of mud - Reading Time: 4 minutes A Big Ball of Mud is a haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the... The post Big Ball of mud appeared first on DDD. [DDD]
• Eventsourcing Patterns: Crypto-Shredding - Crypto-Shredding Encrypt sensitive information in an event and delete the key. Problem The problem is the same as for Forgettable Payloads: some attributes of an event should not be read by all consumers, and we should be able to delete them, without touching the event store. Solution Encrypt the sensitive attributes, with a different encryption key for each resource (such as a customer). Only give the key to consumers that require it. When the sensitive information needs to be erased, delete the encryption key instead, to ensure the information can never be accessed again. This effectively makes all copies and backups of the sensitive data unusable. [Mathias Verraes]
• Eventsourcing Patterns: Forgettable Payloads - Forgettable Payloads Store the sensitive payload of an event in a separate store to control access and removal. Problem An Event Store is an append-only chronological database of Domain Events, so ideally we never remove any event or event data. This poses a problem when data needs to be deleted, for example after a customer invokes their right to be forgotten under the GDPR. A different problem (with a similar solution) exists when the events in the Event Store are emitted to outside consumers, but some events contain some information that must remain private to the producer. Solution (Note: the second problem can also be solved using Explicit Public Events or Segregated Event Layers. ) Remove the sensitive information from the definition of the event. [Mathias Verraes]
• Patterns for Decoupling in Distributed Systems: Segregated Event Layers - Segregated Event Layers Explicitly segregate a Bounded Context’s events in visibility layers, with their own language. Problem The problem is the same as described in Explicit Public Events. Sometimes marking some events as public is not enough. You really need to be able to evolve the internal events separately, while designing the public events differently to suit the clients. The clients may need Summary Events or Fat Events, but you don’t want to muddy your Bounded Context with all these consumers’ needs. There’s also the problem that if too many consumers impose their needs on a producer, and all depend on the same events, the consumers are implicitly logically coupled to each other. [Mathias Verraes]
• Patterns for Decoupling in Distributed Systems: Fat Event - Fat Event Add redundant information to a Domain Event to reduce complexity in the consumer. Problem A consumer is interested in one event type from a producer, to react to it or report information to a user. The producer’s events are designed with a Completeness Guarantee. The event only contains the attributes that have changed, and none of the other state of the resources that the consumer is interested in. So the consumer has to listen to a number of other events as well, that each contain some of the state changes of the resource. Most of the work the consumer is now doing, is building state based on events, and doing lookups to associate things with each other. [Mathias Verraes]
• Patterns for Decoupling in Distributed Systems: Explicit Public Events - Explicit Public Events Mark a small subset of events as public, keep the rest private by default. Problem Domain Events are not only useful for communicating to other Bounded Contexts, but also for organising and decoupling code inside the Bounded Context. In CQRS/Eventsourcing architectures, this is even the primary way of organising the code. It’s very convenient to publish all these events using a single bus, effectively making them all public. This is a problem when some of these events contain sensitive data. More importantly, the outside API is tightly coupled to the internal structure of the Bounded Context. [Mathias Verraes]
• Patterns for Decoupling in Distributed Systems: Passage of Time Event - Passage of Time Event Replace cron jobs and scheduled commands, with an agnostic event to indicate the passage of time. Problem Many business processes involve some action or job or workload that needs to be performed at a future date. It can be a one-off action or a repeated action, it can be scheduled for a specific date (eg Christmas), a recurring date (the last weekday of the month) or a timeout (thirty days from now). We want to keep the domain logic for the entire process nicely isolated in a single (micro)service, so that’s also where the logic for this action goes. But there is a small amount of logic that is not inside this service: the fact itself that the action needs to happen, and when it needs to happen. Cron is the most troublesome: it only works for repeated actions, and offers little control to outside tools. [Mathias Verraes]
• Patterns for Decoupling in Distributed Systems: Summary Event - Summary Event Instead of emitting a stream of Domain Events, emit a single Summary. Problem A business process involves a number of steps that each produce domain events. A consumer depends on information in those events, and listens to all of them to make meaningful decisions. This in itself is perfectly fine, especially when it’s only a small number of event types. When the number of event types is large, we can get concerned about the amount of knowledge the consumer now has of the producer’s business process. The consumer is now almost like a micromanager, looking over the shoulder of the producer and getting immediate reports of everything the producer does. [Mathias Verraes]
• Patterns for Decoupling in Distributed Systems: Domain Query - Domain Query Replace Free Queries with Domain Queries to decouple from knowledge of the server’s internals. Problem The word query is usually associated with database queries. There are however other ways we can query a system that we don’t perceive as a database. REST and GraphQL come to mind. I propose the term Free Query for every type of message that knows about the structure, schema, endpoints, … and has a great freedom in accessing and combining data from that system, typically using a rich query language. In a very simple setup with a single client and server, owned by a single team, this isn’t a great problem, as changes to the server and client can be applied simultaneously. [Mathias Verraes]
• Patterns for Decoupling in Distributed Systems: Completeness Guarantee - Completeness Guarantee Design the set of Domain Events from a producer so that they can be used to rebuild the producer’s state. Problem Often, the events emitted by a producer, are designed haphazardly. New event types are added whenever a new feature requires them. A consumer needs to be aware of an event, so we come up with an event in the producer, give it a name, and add some attributes the consumer needs. Then perhaps later another consumer is added, and it reuses some existing events, but it needs just a little bit of additional information. This is added as new attributes in an existing event. [Mathias Verraes]

Permalink | From 06 May 2019 to 12 May 2019 | Last updated on: Mon, 18 Apr 2022 18:23:14 GMT