Alexander’s earlier work Notes on the Synthesis of Form was also a big hit with computer scientists in the 1960s, even though it, too, was primarily aimed at designers and architects. (And if you still have any empty shelves on your bookcase, Alexander’s 2002-2004 four-volume work The Nature of Order: An Essay on the Art of Building and the Nature of the Universeis apparently considered his magum opus; and he also published The Battle for the Life and Beauty of the Earth: A Struggle between Two World-Systems, about how his team designed & constructed a Japanese school, in 2012.)
Will: There's another book with a possibly contrasting view: Semantic Software Design. It says the main work in software is coming up with the concepts/paradigm/metaphor which explains what the software is doing. And also we shouldn't think of developers as manufacturers, builders or architects. But rather creative artists. Mainly because we never produce the same piece of work twice.
Customer Journey Map
Why are they using this and how do they interact with it?
Pains
What about their current process is problematic? Where are they thwarted?
Gains
What new innovations can you offer them that might help them take a leap forward in productivity, delight, options, opportunities?
Outcomes
What is the change that would make a difference to your customers and other
users? What reward do they want at the end of the journey?
Metrics
How will your customers measure the difference that was made for them? How
will they know they got great value from your system? Note that these should be
rough and general at this stage.
Use cases
Overview Provide a one-paragraph description of this use case.
Actors
Identify primary and supporting actors.
Relationship to Other Use Cases
This use case is related to the following use cases:
Extends Use Case 1 (link)
Includes Use Case 2 (link)
Preconditions
The following must be true for this use case to be started:
Condition 1
Postconditions
Describe what will be true when this use case completes:
Condition 1
Primary Flow
Describe the main flow of this use case, excluding error conditions:
Step 1
Step 2
Alternate Flows
Describe any desired alternative flows or error conditions. Identify step numbers
where the alternative paths begin.
Special Requirements
Identify performance, scalability, availability, and internationalization requirements
associated with this use case.
Components
...
Approaches
Approaches are short versions of Design Definition Documents (see below). These are perhaps 3 to 10 pages long and cover a specific
engagement.
For example, we had a customer who wanted to integrate with our APIs in a certain
unusual way because of constraints of their legacy system. So, the architect wrote an
Approach document describing how the integration should be performed, the protocols to use, where to get certain data, the security implementation, and so forth.
Design Definition Document (D3) Template
Program Name Design Definition
Executive Summary
State the purpose of the program and what this document covers.
About this Document
This section contains metadata about the design document itself.
Authors
Names of people who did most of the writing.
Contributors
Names of people who contributed some writing or key ideas.
Reviewers
Names of people who did some quality control on this document.
Document State
Is this a draft, in review, or published?
Use of IETF Keywords
This document employs a subset of the Internet Engineering Task Force key‐
words found in RFC 2119. These words are MUST, SHOULD, MAY, and their
counterparts MUST NOT, SHOULD NOT, MAY NOT. They are capitalized
throughout the document to draw attention to their special status as keywords
used to indicate requirement levels.
Business Design
Describe the business objectives, drivers, and expected benefits for undertaking this
program.
Capabilities
Describe what capabilities the solution will have. What new benefits are provided
in functional and nonfunctional terms? What new scalability, reliability, global
distribution, performance, resilience, extensibility, manageability, portability,
security, or other benefits are expected? Don’t advertise or market to your audi‐
ence; be concrete and concise.
Strategic Fit
What element of the business and technology strategies does this help realize?
How?
Business Drivers
Why are we doing this? What priority is it among known others?
Assumptions
What is expected to be in place for this solution approach to be successful? Refer
to funding, availability of key resources, operations support, contracts, key pro‐
cesses, procurement, Global Network Operations Center (GNOC), regulations,
stated standards and guidance or technology patterns from the leadership team,
and so on.
Constraints
Applicable laws
Does the Americans with Disabilities Act (ADA) apply?
Applicable regulations
Reference need to maintain compliance with Payment Card Industry (PCI),
Personally Identifiable Information (PII), General Data Protection Regula‐
tion (GDPR), and System and Organization Controls (SOC). Do not merely
state that these laws exist, state how your design specifically accounts for
supporting them so that the developers can be sure to implement it
accordingly.
Risks
List business risks in doing the project as envisioned, risks to the customer or
existing business prospects or processes, how it can be maintained and operated
properly, availability of staffing resources, securing funding, countries/markets,
the inherent risks in trade-offs made, and so forth.
Impacts
What will this project or this architecture create in terms of organizational, train‐
ing, and process needs that affect the business? How will the business need to
change to best support this? Are there new processes that are expected to be
adopted internally within product development to support a new technology or
skillset? Should any roles and responsibilities change?
Stakeholders
List the organization, roles, and named individuals who stand to win or lose by a
good or bad outcome on this program. Who must change to accommodate it?
This will eventually feed your Responsible, Accountable, Consulted, Informed
(RACI) document.
Governance
What is the framework you’ll use to govern this project? Must you invent a new
committee? Are there design review processes or cloud governance or financial
governance processes to follow?
Application Design
Application overview and general strategy description.
Applicable Standards and Policies
List of links to published guidelines and conventions for development teams to
follow. For example, any internal policies, industry-specific standards, PCI guide‐
lines, ADA guidelines, and so forth that you expect teams to follow.
Guidelines and Conventions
Links to published guidelines and conventions for Dev teams to follow, such as
published internal standards, Google Java coding guidelines, JavaScript conven‐
tions, code quality guidelines, and so forth.
Patterns
Diagrams, links, and descriptions of patterns for Dev teams to follow in the
implementation. Reference any design patterns for designing and developing
services as applicable.
Services
List services to be created or existing services to be reused, along with the owners
of those services.
Security
Security requirements and design: how data will be secured, encrypted, author‐
ized, authenticated at rest, in transmission, or in processing. Use of Open Web
Application Security Project (OWASP) Top Ten and how those are addressed.
What security groups are required? Highlight security requirements for develop‐
ment such as bastion hosts. List transport or Transport Layer Security (TLS)/
Secure Sockets Layer (SSL) requirements.
Availability
Target SLA in terms of 9’s uptime and how specifically the architecture will sup‐
port such numbers. How recoverability, disaster recovery, and the like is being
supported. Consider these questions:
What are the compensating actions taken in a failure?
Will a circuit breaker be used?
What redundancy is there?
How is caching used to support certain failure events?
Health-check page?
Multideployments?
How are specific key components particularly designed for resilience and
high availability? Are you using multiple cloud zones?
What is your service replacement and versioning strategy to support zero-
downtime releases?
Scalability and Performance
What number of transactions per second at this latency and CPU utilization
must the solution support? What is the unit of scale (container, virtual machine,
cluster)? Use actual numbers and calculate the impact on server footprint that
will have. What are the ways in which the application and services can scale
through statelessness, Auto Scaling Groups (ASGs)? State the ASG threshold so
developers aren’t left to guess.
Extensibility
Will you use certain patterns in your services such as Strategy or Specification?
How will APIs specifically support the ways that the application affords future
change, how will the application support customizing per customer, and how are
configurations afforded?
Testability
How will this be tested, what tools will be used, and what specific automation and
targets will be in place? Include functional testing, regression testing, integration
testing, and chaos/resilience testing. Load testing plan? How will this be automa‐
ted? What specific toolset used in what process?
Maintainability
What software guidance for developers will help make the code base easier,
cleaner, simple to maintain in the long term? Code repository needs or project
needs? What is the maintenance schedule anticipated or downtime for upgrades
strategy?
Monitorability and Metrics
What tools and dashboards are required, logging requirements, how the software
itself must support event publishing to increase visibility. What are the specific
metrics that will indicate system uptime, health, and proper performance? How
will alerts be triggered at what threshold? Consider CPU, memory, drive/filesys‐
tem volumes, process monitoring, logs, event logs, and required procedures.
Data Design
Database Strategy
Overview the general data management strategy for this application. What spe‐
cial customer requirements should be called out for careful handling? What are
the risks involved with this strategy? Expected benefits?
Standards and Guidelines
What is the applicable set of data technology standards, guidelines, and tools?
Technologies
What database technologies will be used for what purposes, what specific serv‐
ices? Are there any new database technologies being used, or new versions?
Where can developers learn more? What guidance exists for them?
What instance types are being used? Are the available networking modes the
proper type for that database? Have you used the cost calculator to size and price
instances?
Import/Export
How will customers get data into your system?
If you’re porting data, what are the source and target maps?
How will customers get their data back out of your system for their own use?
Replication, Backup, and Recovery
How is data replication supported? How will you distribute data globally?
What is the backup/recovery strategy? Will you store full or partial backups? For
how long? On what systems, for what services?
What is the bulk data replication strategy? What is the multiregion strategy?
Data Versioning
How will your database be versioned? What about the data itself?
Database Automation
How will the database farm be automated and updated? What tools, pipelines,
and processes are in place or should be developed?
Database Performance Considerations
What performance thresholds for transactions within the database are expected?
Do you have a known level you must support?
Data Warehousing, Storage, and Management Requirements
What data volumes must be supported? Data movement policies and require‐
ments. Reference how logs will be stored, managed, or forwarded.
Data Maintenance
How data will be maintained, data retention policies, scripting to offload, data
restoration. How will data be populated for different environments for this appli‐
cation? Will data be truncated? At what interval? How will data be encrypted?
Are there GDPR or PII/PCI requirements to be stated for development teams or
infrastructure admins?
Data Migration
How will data get into the system? Is connecting to a legacy system required? Is
Golden Gate or Kafka or Extract, Transform, and Load (ETL) or another tool in
use? What time period is anticipated for this? Will data need to be synchronized
over a certain period of time?
Data Volume
What is the expected starting volume of data? What size database is anticipated?
Will there be multiple data stores? How many rows are anticipated to be added or
removed daily for the key services?
Logging
What are your log rotation policies? Are there requirements for Splunk or
another monitoring tool?
Reporting
How will we do reporting for customer use? For internal use?
Auditing
How must the system audit user changes to track and report in case of a security
breach or for compliance?
Security
How is the database farm itself specifically secured? Will data be encrypted at
rest, and how will that be managed? How will access to different environments be
managed, including at the development level?
Analytics
What data must be exposed by the application to support business analytics. How
must that data be exposed to support analytics tools? Will you have a data lake?
How will that be maintained, accessed, and used properly?
Caching Strategy
Requirements for caching and the locations and technology to support caching.
How distributed is it? Any guidance for application developers as applicable.
Guidelines on caching for implementers and for developers subsequently using
it.
Machine Learning
What specific processes must be called out for machine learning? How will data
pipelines be supported? What is that entire subsystem design? Feature engineer‐
ing considerations.
Infrastructure Design
Infrastructure overview statement.
Infrastructure Strategy
What are the patterns you employ? Are there deployment diagrams, component
diagrams, other UML? Cloud infrastructure design? Do you need to include load
balancers, DNS, application servers, networks, CIDR allowances, security, data
servers, firewalls, storage, queues, and so on?
Latency and Performance
How does your infrastructure support the specific application and customer-level
SLAs described earlier?
Infrastructure Security
How will the environments be secured?
Maintenance
How will teams perform patching (hopefully no-patching)? Related policies such
as operating system (OS) upgrades, replacements.
Standards, Guidelines, Conventions
References or links to existing guidelines that you want the teams to follow.
Infrastructure as Code (IaC)
References to templates, best practices, samples.
Environment Guidelines
When/how to use reserve instances, autoscaling groups, VPN access to different
environments by different teams, and so on.
Global Distribution
How do we enable and maintain the ability to distribute our applications glob‐
ally? Are there any particular concerns, such as regarding the Chinese firewall?
Concerns associated with performance and security in globally distributing?
Server replication processes and automation?
Immutable Infrastructure
Describe how your infrastructure is immutable, including process automation.
Hybrid
Are there any necessary strategies for hybrid architectures (solutions that span
on-premise and cloud)?
SLAs
State SLAs that you must meet. Put them in mathematically testable terms, from
a customer perspective. Include the panoply of customers, not only the end user.
Writing a hundred pages up front is not the answer. What is the answer is this:
Creating a concept of the whole picture as well as it’s understood and committing
that to an external, formal format beyond the designer’s internal thoughts and
conversations that others can see, read, understand, and use.
Performing a thorough and ongoing analysis and deconstruction of the concept
based on its semantics. You don’t need to complete the entire document up front.
We just must make sure we do visit all of these concerns, understanding that they
will evolve.
Making a crisp, declarative, unambiguous, directive, testable set of design deci‐
sions that are clearly derived from the concept and record them formally in a
public recording for the many diverse stakeholders.
Position papers
...
RAID
Risks, Assumptions, Issues, Dependencies.
The business aspect
Proposition 0
By definition, any purposive compound of objects and their relations is called a
system. (Examples can include a software application, a datacenter, a business
organization, a business process, a chemical compound, a written document, a
play, a music composition, and so forth.)
Proposition 0a These compounded elements and their relations are not innate, but are proposed,
socially constructed, captured, augmented, determined, and filtered by the design‐
ers of that system.
Proposition 0b
Any system is either designed explicitly (purposively), or implicitly. If the design
is implicit, its design is regarded and comprehended only after the fact, after the
system is in place, as a result of a series of accidents, which is likely non-optimal.
Proposition 1 Certain principles apply to well-designed systems, and these same principles can
be employed across the design of any system, though seemingly disparate.
Proposition 1a
The attributes of any well-designed system include, at a minimum:
Fitness to purpose
It must serve what it purports to serve, to help users achieve their goals
efficiently.
Felicity
It must afford that purpose in a way that minimizes friction and noise, mak‐
ing it easy and delightful to use, consume, and participate in.
Flexibility
Given that the system operates in a world of frequent change, it should be
designed in a way to allow modifications, updates, and extension according
to future needs.
Proposition 1b
An additional set of attributes contemplated for a well-designed system (software
or otherwise) include the following:
Maintainability
It should be easy to correct faults, improve performance, and adapt to a
changed environment.
Manageability
You should be able to keep the system safe, secure, and operating smoothly.
Monitorability
You should be able to see into the system, to measure and understand how it
is working.
Performance
It must excel at its purposes.
Portability
It should be able to operate in a variety of contexts.
Scalability
It should be able to operate at the same level, even under increased load.
Proposition 2
The software system you design will operate within a business context, and there‐
fore, to be optimally designed, the software system must be designed to support
and operate within this business context or a new business context the software,
in its innovation, potentially requires the creation of.
Proposition 3
The business is a system of systems (these are business elements that are also sys‐
tems: your service-oriented development organization, the sales delivery process,
the architecture review board, the strategic funding process, local executive steer‐
ing committee meetings, the joint venture strategy, the project execution plan,
and so forth).
Proposition 4
The business therefore can be designed as systems; it operates according to these
same principles.
Proposition 5 Because the semantic designer (creative architect) is foremost a designer of sys‐
tems, the purview of the role includes the proper design of the software as well as
the design of the business systems themselves.
Conclusion The business is a system just like the application is, so you as the creative director
must help design the business itself as a cohesive and coherent system according
to these principles, to achieve a better overall business outcome. The resulting
business, as a context in which software is developed, will help improve the soft‐
ware itself, and help you make it on time and on budget and according to user
needs. They inform and help (or hurt) each other.
So there are two points here:
You might not have historically considered it part of your job, but to be especially
effective, consider your purview to include the design of the organization itself
and its processes according to received architecture and design principles.
When you design especially effective software, you not only consider the applica‐
tion frameworks and software attributes, but consider the impact the business
will have on your system, and the impact your system will have on the business.
Therefore, now we turn our attention to the business itself, to ask specifically:
How can you see your organization and processes as systems in themselves to be
understood and purposively designed?
After you begin to see your organization through the lens of systems, how can
you optimize the organization and processes toward maximum effectiveness?
How can you determine the impact your burgeoning system might have on the
business?
• How aligned is the business with the system you are creating? As you bring it to
life, can it be properly supported?
By the end of this chapter, you will be able to answer these questions with the practi‐
cal tools we’ll introduce.
Notes on "The Principles of Clean Architecture" by Uncle Bob Martin
Shows Rails app structure on slide. Why do I know this is a Rails app? Why doesn't it tell me what the application does? What's important about the app is hidden by the structure of the framework. Why is the web (merely a delivery mechanism) central to the application structure. Back in the 90's we thought the web changed everything, but it didn't. We were duped into thinking that the web was an architecturally significant part of an application. Shows blueprints of library and church. The use is clear from the structure.
Suggests that Architecture is about Intent. It's not about tools or libraries, those are details that should be hidden.
Recalls Ivar Jacobsen's classic book Object-Oriented Software Engineering: A Use Case Driven Approach. Use Cases are like User Stories. Best kept simple and high level.
Application specific rules (use cases) may be represented with "interactor" objects.
Cross application rules are represented in "entities".
Interface/Boundary objects/classes (act like java interfaces) provide a way for external systems to present requests.
Request model is passed thru boundary into interactor. Interactors process request and control entities and gather results and emit response object thru interface to delivery system.
Looks like MVC but is NOT because MVC is misunderstood.
MVC is oldest pattern (predates patterns). MVC is not an architecture.
Defines MVC as originally described in the 70s. Model object hides a very small business rule. There is a controller that is connected to an input device like a keyboard or mouse. Controller sends input to model. The view registers with the model, which sends state updates to the view, to trigger updates. MVC was done in the *small*. You had an MVC for *a button*.
Now we have MVC for whole pages etc, because: What happens to any kind of good idea in software is that people know that it's a good idea and they also know that their ideas are good. And so they figure that their ideas are that idea...and they completely lose the original idea.
Problem in app scale: Poor boundaries. Not clear what is view, controller or model logic. Unless you keep extremely careful discipline the concerns will get mixed.
Talks about Plug In pattern.
Dependencies flow only one direction between a plug in and the application.
Changes in a plug in can not impact the core aplication.
Protect you business rules/core application by treating UI and data storage as plug ins.
Things that change frequently should be plugins.
Talk about model View Presenter. Presenter gets data from app across plugin boundary. Feeds data to view model. View renders view model and updates presenter with UX events.
What about the Database? We put it central to a lot of our architecture. Because Oracle fooled us all.
The DB is a storage detail and should be abstracted away.
Use a boundary class that handles every query you might want to make. The DB code should all live across the plug in boundary. All SQL, etc generated in the plug in.
This architecture helps with testing, because you can easily remove the DB.
Case study: Deferring a feature out of existence.
Worked on project called Fitnesse (a wiki, which shows test results). They were going to build it based on MySQL, but they decided to wait and worked on Wiki text to HTML and used a mock DB. Next they needed to do links and navigation, and so thought they were going to do DB storage. Instead they just used a mock class that stored pages in RAM. Then they needed to do start up and shut down, and so they were going to add DB support. Instead they made their mock object serialize the hash to the file system. That worked well enough, so they eventually shipped the product without database support.
Eventually they added MySQL support for a customer, but even they gave up on the DB.
Lessons:
*A good architecture allows major decisions to be deferred.* *A good architecture maximizes the number of decisions NOT made.* *Achieve that by using a plugin model.*
Use cases form the center of the application.
UI, Datastorage, Frameworks, etc are all external to the use cases.
Remember: Framework authors will screw you. You make a tremendous commitment to a framework when you adopt it. The framework author makes NO commitment to you.
Use frameworks because they are powerful and can make you productive.
But don't follow the examples, because the framework author is going to be comfortable with too much coupling to their framework.
(3) use javascript if anything needs to appear/disappear based on user interaction.
But I usually start with a framework layout like Bootstrap, add pre-made Javascript libraries for widgets and only do the above steps for any little tweaks needed.
The current state of the art is the "CSS Grid" layout (for 2D grids).
Endpoints are all nouns, use the HTTP actions as verbs.
All endpoint nouns must always be singular, to match with database tables. Or plural (explanation). The point is they should remain consistent with other APIs from the same team, organisation or whatever.
(for ease of development) Provide a special undocumented "override" URL path for humans, that sets the header appropriately:
i.e. /api/v2/nodes --> redirects to --> /api/nodes and automatically sets header: api-version: 2
or /api/nodes?version=2 --> redirects the same as above
Caution: Don't overload the content-type header.
(for ease of development) Provide a similar parameter for the Accept headers. The reason for this is to make it easier during development to send a URL to someone non-technical, or who does have the right dev environment set up, but the URL will still works without any special software like curl or browser plugins. Example: /api/nodes?accept=application/json
Return 2xx status code to indicate the success of the HTTP request
Return a status field in the content body to indicate the progress of the business domain request
Use a "status" field, not a "state" field. Status refers to a progression.
APIs should conform to previously created APIs where that doesn't contradict the principles above.
Where legacy APIs don't follow the principles above, they should be updated to conform as a pre-requisite to any changes.
Minimal JSONAPI examples
Success response:
(the data array is optional, it could be ommitted if the response is always a single object).
Failure response:
Questions
The response should be valid JSON API. But must the request be JSON API too?
References
If the examples above seem unnecessarily verbose (even though they have been cut down the most they can be), try JSend instead, it's simpler.
How to validate JSON API: Take the schema and your JSON, and input them at JSON schema lint
An api should be easy to learn, easy to use, hard to misuse
Continue to write to the API early and often
Example programs should be exemplary - this code will end up being copied everywhere
When in doubt, leave it out
Be consistent - same word means the same thing across the api
Documentation matters - reuse is something is easier to say than to do - doing it requires both good design and good documentation
Do what is customary - obey standard naming conventions - it should feel like one of the core APIs, know the common pitfalls for the language and avoid them
Pencil and paper: Extremely easy to use, but no templates, and not online
GraphViz: Good for automatically generating a diagram from code, but you need to implement a script to do it. Bad (impossible) for free-form manual drawing.
Asciiflow: Good for quickly knocking up a rough drawing of boxes and lines. Bad for editing that drawing.
OpenOffice/LibreOffice Draw: Seems to be good for diagrams. Can export to PDF. How do you make the canvas bigger?
LucidChart: Free. Online only, but can export to PDF.
Dia: Free, but clunky. Dia by name, dire by nature. Exported images are messed up.
Visio: Microsoft. You must pay.
Omnigraffle: Mac only, but nice.
www.draw.io: Free, but not as intuitive as others.
Posts
