For over 10 years, our small band of technology architects (many of whom have
now turned into business strategists) has been meeting twice a year. This is
my “by invitation only” group known as Patty’s Pioneers.
Each year, I learn an incredible amount from the rich sharing of problems and
approaches these guys and gals are using to design and evolve their businesses’ technology
architectures. The same patterns re-emerge over and over again, across industries
as diverse as financial services, ocean shipping, energy management, retail,
information services, manufacturing, supply chain logistics, and solving the
world’s largest problems.
Each year, the patterns of our conversations form a semi-coherent picture
in my head. I thought you might be interested in my non-Geek’s view of
the way I see that these brilliant and persistent “big thinkers” think
about the things they work on. (Pioneers: I have kept the quotes anonymous,
but I am happy to attribute them! My apologies to any of you who may feel I’ve
missed the point or over-simplified! Please chime in!)
What’s Going On in the Economy. Many of the Pioneers have deep financial
backgrounds. So, of course, we spent some time talking about the state of the
economy. I think the consensus was summed up best this way: “There was
a fraudulent set of activities that started the butterfly effect of mistrust.
Each leg of trust got knocked out. …Trust will come back when there are
profits to be made. Everyone’s going to wake up and want to make money.” We
had a really interesting and informative discussion about economic trends in
different parts of the world, which included the predictions that European
businesses are in a lot more trouble than most people think, that most small
companies that are importers and exporters are going to go out of business
because they can’t get letters of credit, and that the stock market will “bottom
out” at 5,000 to 6,000.
What we can do about it? Pioneers felt that what they can do to help is to
provide more transparency. All of the systems and architectures they design
have transparency built in. Everything is visible and understandable. Proprietary
stuff may be hidden behind APIs, but it’s not impenetrable. They also
pointed out that the companies that will do best are those in which the top
executives understand the means of production in their industries. Executives
who are good at “managing” are no longer able to cope. Executives
who have a visceral, emotional feel for the means of production in their respective
industries will survive.
Why We’re Entering a Golden Age for IT. Pioneers are convinced that
tight times call for innovative uses of technology. They see a golden age of
IT when those who have already invested in smart architectures will vault ahead
of those who have under-invested and/or invested in the wrong approaches. “Our
board recognizes that they’ll need more information that’s processed
in ways that are digestible to them, ways that let them better synthesize that
information.”
What’s the Right Way to Think about Technology Architecture?
Model the Business. Most pioneers create an abstract model of their business
using business objects with attributes and/or metadata. This is a living model.
It evolves organically to reflect the dynamic relationships among entities
and the attributes that become important over time. “Information is the
proxy for reality. If you don’t have a proxy for reality, you’re
doomed!” The brilliance in the systems that Pioneers create is that they
enable the people in their businesses to understand how their businesses actually
work!
Monitor Events and Exceptions. While traditional business process consultants
and IT professionals think in terms of transforming, streamlining, and automating
workflows, Pioneers view the world as “one big exception engine.” Stuff
happens. Automated processes are only as good as the exception handling they
provide. Pioneers assume that exceptions are the rule. So everything they design
assumes that events will be delayed, that people will change their minds, that
interruptions will take place.
Combine Objects and Methods Dynamically Each Time that a Service Is
Requested. Pioneers favor near real-time binding of behaviors or methods to objects or
entities. For example, a “check credit” service may be using different
third-party credit rating services each time that request is made, depending
on the context — Whose credit are we checking? For what purpose? Which
ratings services are most appropriate right now? How many do we need to check?
In which order?
Since change happens and improvisations are the norm, it makes the most sense
to instantiate the state of the virtual view of the real world as close to
the time that something (transaction, state change, event trigger) happens
as possible. That way you can ensure that you always have the right context
for the situation at hand.
Maintain the Current Version of the Truth in Cache/Memory. In order to ensure
that their dynamically binding applications perform well, Pioneers also make
lavish use of memory and disk cache to store the current state of everything.
That way, the only time there’s a performance hit is for those few things
whose state may have changed since the last time they were called into play.
Analytics and Computation. Pioneers design and use many algorithms that are
computationally-intensive and which require aggressive technology calculations.
Like all good architects, Pioneers try to pre-compute anything that isn’t
likely to change on the fly and store it so that results can be retrieved and
acted upon quickly. Believe it or not, mainframes still have their place. But
the Pioneers’ tendency is to distribute computationally-intense loads
across parallel computing clusters and/or to distribute the load across multiple
systems and/or to perform the processing as close to the end-consumer as possible
(in a rich client-side application).
Data Structures. Pioneers model objects, yet store data in hierarchical data
bases (e.g., mainframe IMS), in object databases (e.g., Gemstone), in relational
databases (e.g., Oracle and, more recently, MySQL). When possible, they also
use large flat files (e.g., XML tagged files) to store data records for fast
search and retrieval. Data structures that are of interest to Pioneers today
include HP’s NeoView, GreenPlum’s massively parallel shared-nothing
database, or Hadoop. In five years, distributed file systems will be de rigueur.
Today, most of the applications they’re using don’t require processing
of massive amounts of data. Instead, they are typically read-intensive applications
that let end-users access and manipulate information locally.
Rich Internet Applications. Today’s Rich Internet Application platforms
are of interest to Pioneers. They are trying out Ajax, Flex, Adobe AIR runtime,
Silverstream, and/or Google WebTop (GWT) applications. Right now, based on
our small sample, Google seems to have the most appeal and success.
Virtual Computing/Cloud Computing. Pioneers have been early adopters of virtualization.
They like the idea of using virtual machines, both in their data centers and
in the “cloud.” Virtual computing everywhere fits the Pioneers’ view
of how the world works. You want to have a lot of virtual machines that are
optimized for doing different kinds of things. These will probably become more
and more specialized and you’ll have many of them optimized for different
services. “I hand you a workload and you run it.”
It doesn’t matter where Virtual Machines are as long as they’re
redundant, manageable, and secure. If one fails, there’s another ready
to take its place. You can monitor and manage, observe and optimize a lot of
distributed computing in the cloud and/or in your car, on your PC, or in your
watch. Pioneers predict that the use of virtual machines will explode over
the next few years. It’s simply the logical next step in the continuing
evolution up the ladder of abstraction.
However, one of the things that Pioneers always bear in mind is that no matter
how well you abstract the virtual from the physical world and/or model the
events in the real world, there’s always an unbreakable connection. If
you begin operating as if the virtual world IS the physical world, you’re
in trouble. Physics matter. You can’t escape the fact that even in the
world of virtual computing, you need to be able to start up and shut down physical
as well as virtual machines.
Retail Commodity Computing Sets the Standard. You might think that these Pioneers
would favor exotic systems. But it’s actually just the opposite. They
find commodity PCs, Macs, servers, discs, phones, and gaming devices both powerful
enough and open enough to use for many applications. They favor the use of
low-cost consumer-oriented Web services and applications. Don’t spend
money on expensive and exotic gear when your child’s PlayStation will
do the job just fine! In fact, cloud computing fits this commodity retail model.
With cloud computing you get access to networking, computing and storage available
on a pay-per-use basis. “It moves at the speed of light, has an API,
and takes a credit card.”
Biggest Constraint? One of the biggest issues on Pioneers’ radar this
year is energy consumption. Computing uses energy. As you add more computing
power, your energy consumption increases. Even with green data centers and
more energy efficient computing and power management, they still predict that
we’re going to run out of the energy needed to power the millions of
machines (virtual and physical) that we use to understand and live in the world.
They predict that cloud computing suppliers will lead the charge in moving
to cheap energy. This will happen at the same time that it becomes easier and
easier to spawn and manage many computational threads in parallel (using lots
of virtual servers). “Right now, we have a phenomenal number of watts
running a small number of threads.”
Make Complexity Manageable. Pioneers spend a lot of time trying to make it
easier for people to do what people are good at. They think about information
architecture: what are the objects and entities that people care about and
what do we need to know about them? They provide simple yet accurate models
of what the business is doing.
They scope problems into bite-size chunks so that human beings can wrap their
minds around them and pay attention to the important things, without being
blind-sided by the interdependencies of one problem space with others. The
Pioneers’ habit of always thinking in terms of well-defined interfaces
between interacting entities helps them keep these boundaries logical.
They design applications that let human beings focus on managing exceptions
while passively monitoring what’s going on in their domain space. They
describe it this way: “We remove the human middleware. We take the tedious,
repetitive work away from people. We focus on awareness, rather than on doing
things. We take work away and build a passive awareness of what’s happening.
We simplify things for people, but don’t try to compete with the human
mind.”
Are You a Pioneer Too? Does this sound like the way you view the world? If
so, and if you’d value the opportunity to commune with other like-minded
souls, please contact me (pseybold@customers.com) to find out more about my
Pioneers group.