Architecting Digital Products: Fencing the Ecosystem

Challenges from the Agile Mythology

Done well agile development lowers the price of terrible communication between the customer, the developer, and the operators. Iterative approaches and regular engagement allow all sides to figure out what each other is saying by inexpensively building what no-one wants.

That's it. You are going to get a lot wrong. So fail cheap.

As an enterprise architect, the corrosive myth is a "pure bottom-up" product team collaborating with a "customer". The devs and users collaborate. Then iterate. Then magic happens.

This myth almost works in the simplistic standalone start-up, where the user is the customer. Even here the failure rate when the customer is asked to pay the bill is staggering.

Inside a complex enterprise, this myth is a recipe for grief. In-house users are not customers. Ever. They are shopping with someone else's credit card. Their deliveries come with boxes and boxes of complexity and sustainment cost.

Our job is bringing the most powerful start-up feature—market discipline. We keep the bill payer—the stakeholder who is accountable for value, capital cost, operating cost, compliance, risk, and the P&L—front and center.

Even so, I desperately want it all. I want that bottom-up creative freedom and the hard boundaries of what the bill payer wants.

This doesn't mean taking the worst possible approach and doing a big design in front of the sprints. Enterprise architects have no business there.

This doesn't mean do the other worst possible path and jump into the sprint. The purpose of enterprise architecture is to guide effective change, not detail design and argue approach to what should be built.

Instead we need the path where hard boundaries are not day-to-day friction points. Instead, where our creative geniuses have a large boundary to work within. I want to provide my teams the same open-ended power and freedom Apple provides iOS developers, with the same absolute controls that protect enterprise interests.

Boundary Creative Genius

For years Parks Canada provided wild Bison from Elk Island National Park to international re-introduction programs. But they had held off re-introducing them to Banff National Park.

Historically Bison roamed the North American plains. Banff is a huge mostly wild area with significant recreational activity. To the east are the farm and ranch lands of Alberta. We could intend that the re-introduced Bison were for Banff. We could intend that they didn't move themselves to the less remote parts of the park. Or, wander into the ranch lands east of the park.

However, Bison have no interest in lines on a map.

Nine years ago, in 2017, 16 Plains Bison were re-introduced to the mountain meadows of the more remote northeast side of Banff park. When they were last counted in '24 there were more than 130.

Parks controls the herd through simple mechanisms and a hard boundary. Short barrier fences make downstream travel to the ranch lands less easy. Further on, the Rangers will use herding techniques to get the herd to go back to where they'll be left alone. For a few adventurous Bison, tranquilizers and relocation are used to move them back to the deep wilderness.

Day-to-day, no one tries to tell the Bison where to go, or what to eat. Or even protects them from wolves or grizzly bears.

The bison have creative freedom.

They have hard boundaries.

Let's do the same thing with our digital products.

That is our job.

The Critical Interaction of Bounded Freedom

Banff's wild Bison are free to wander in 1,200 square kilometers. Your product teams are free to work within your enterprise. This is a combination of internal and external limits, goals and boundaries.

I want you to think about the fundamental hard limit to modern digital products your data architecture provides—data terms of use. Our digital products inevitably capture data belonging to someone else. Just think about Apple and your digital photos. Or ServiceNow and the customer's work flow and authorizations. Or Gmail and your contacts.

You can never address the hard boundary that comes from data terms of use by waiting until someone starts driving on the wrong side of the road.

We use two specific tools to establish this boundary, Logical Document Model and Architecture Specifications—particularity architecture principles, architecture patterns, and standards.

The Logical Document Model

Logical Documents let you see the digital product data inside other business systems without losing sight that all of the data comprising the logical document has hard restrictions. Not some, not sometimes, all the data all the time.

By using a logical document we get to use a business understanding not a data-structure understanding. I'm talking about calling it a Photo. That includes the jpg or hiec, plus meta data, plus everything.

We start with a few properties:

• Document Type to highlight documents with special retention and content requirements.

  • Record: A documented item required to meet obligations defined by law or contract; its content and retention are externally mandated.

  • Business Document: An internally-defined document supporting business processes; its content and retention are governed by organizational policy to ensure consistency, auditability, and operational efficiency.

  • Transitory Document: A document created by and used by an individual, department, or team. Content is aligned with the creator’s needs, and while not subject to formal retention schedules, retention may be restricted by internal policy to mitigate risk.

• Data Access Property explains required protection of information or its sensitivity. The attributes of this property are often changed. This list starts your thinking

  • Country-only – Information must be constrained to in-country use and residency. Will limit facility, security design, integration, and offline options.

  • Process / Product Only – Data may not be used outside the product or business process. Will limit security design and integration.

  • Custodial Data – Data belongs to another organization (customer). Will limit security design, application design and integration.

• Information Data Protection Property explains the required protection of information. You are seeking Absolute, Enhanced and Relaxed levels. Invariably this property requires an understanding of the data. Standard protection of Custodial Data may be more stringent than Enhanced protection of corporate data.

  • Relaxed – Information has lower than normal data protection requirements. This specification requires a clear definition of the requirement.

  • Standard – Information has industry or organizational standard data protection. This specification should reference an open standard.

  • Enhanced – Information has higher than normal data protection requirements. This specification requires a clear definition of the requirement.

  • Absolute – Information has unique data protection requirements. This specification requires a clear definition of the requirement.

What I'm looking for is an unambiguous indication that I have a constraint limiting my degrees of freedom.

Power of Architecture Specification

The first time I run into a data constraint I need to explore the constraint.

Start with the source of the constraint. Is it product specific tied to term of use (contract), regulation, or policy. This sits outside digital product decisions. This tells you the boundary of the constraint and who owns the decision on whether the architecture and implementation comply.

Yes, we start with who owns the decision on whether the architecture meets the constraint. Between us, some constraints start to collapse a simplistic model of stakeholders own the architecture. Contract and regulation simply exist. Policy often simply exists. We rarely have the time or energy to chase through changes to contract or policy.

I want to get to a re-useable architecture specification—architecture principle, architecture pattern, and standard. When I have a re-usable architecture specification I don't have to re-do the analysis.

My sweet spots are an architecture pattern or architecture building block. Both provide a common approach to a predictable problem. Patterns:

• Shift our work away from reinventing the wheel

• Improve confidence that the architecture completely covers the constraint, and successfully answers the difficulties

• Allow you collapse recurrent architecture trade-off conversations

• Cascade your organization's preferred answers and approach

• Simplify solution assessment during implementation governance

Most of all, architecture patterns simplify architecture governance. We will have already done the math and come to a well considered answer for our architecture.

Good patterns will span every domain—business architecture, data architecture, application, technology, and security architecture.

Great patterns get embedded in platforms and shared services, so the agile team doesn't even have to think about it.

When you have great patterns embedded in your platforms and tooling, implementation governance becomes so much easier. You stop asking how did you deal with the super secret custodial data? Instead you ask the testing team to ensure that they are validating 100% of the accesses of the photo data only happens in user space.

Voila, we mechanically prevent the agile team from violating our customer contracts.

Ok, I know it is never that easy; but I do get to outsource a lot of the investigation to people in the product management, development, and testing chain. We embedded the required architecture specifications into the complete system—starting with Superior Architecture and ending with acceptance criteria.

Every product and project is more likely to be guided and constrained by those prior architectural decisions.

Concluding Fencing the Ecosystem

Great enterprise architecture isn't about micromanaging. It is about using guidance and constraint to make it easier for everyone to do the basics with less thought.

Banff's rangers do not know how many Bison are wandering the park. A few wear satellite trackers so the rangers can follow the herd. Barrier fences make some paths they don't want the Bison following harder. Despite everything, it only took a couple of years for a wayward bull to wander east towards a ranging town—they picked him up walking down the road.

I hope your architecture non-compliance resolutions don't require a tranquilizer dart, winch, and helicopter. But, do not be afraid to intervene to protect the value of the digital product, the product family, and your enterprise's ecosystem.

Take a moment and look at your own digital products. Do you have clear patterns to simplify the key constraints in your architecture? Are those patterns embedded in platforms and tooling? Or are you relying on high-cost intervention after a project has done all the work to do the wrong thing?

Do the math. Guide the change.

Next week, we'll start a journey exploring architecture specifications—architecture principles, architecture patterns, and standards.

Have a great day!

Regards, Dave Dave Hornford Conexiam

PS. No one really wants a herd of Bison basking in the sun on Highway 1. However much I love seeing wild Bison, when I'm on Highway 1 I'm going somewhere. Same thing for the other 60,000 cars that would be backed up.