Proper representation of Enterprise Architecture has always been quite a challenge. Many times the sheer scope of the canvas required can be the problem, but most of the times it is the question of a proper style and standard of representation that poses issues to the IT architects. While Enterprise Reference Models (TRM or ARM) are normally pulled together as simple “box of boxes”, the lower level representations are vulnerable to subjectivity unless a firm modeling framework has been implemented and governed by the architecture board that controls the IT architecture of the enterprise.

And even if you have the standards and guidelines in place, there are still chances of unwished-for scenarios like a powerful senior leader of the enterprise caring to see things in a way he would actually understand it. Important though it is, chances are high that such short-circuiting of standards would be carried forward or be reflected in subsequent architecture artifacts. Why senior leader, – any ‘consumer’ of the artifact can challenge representation for better understandability. The preparation of models that convey the right meaning to right people without compromising across-the-board consistency and uniformity can be tough.

Challenges in Architecture Representations

Depending on the size of the organization or the authority enjoyed by the architecture board in the enterprise, the tools being used to capture and manage architecture models in an enterprise can be anything from MS PowerPoint and MS Visio to feature-laden, metadata-driven tools like Rational Suit and Telelogic System Architect. However, even the representations produced by or controlled within  very powerful, industry leading tools don’t guarantee ‘universal acceptance’ across the enterprise-wide stakeholders for reasons  ranging from complexity of the model ,  lack of features supporting a certain requisite degree of abstraction or more simply, the comfort-level of in-house managerial, architectural and development staff.

Then comes the question of standards. Most MDA (Model-Driven Architecture) complied standards like UML, MOF, XMI are really meta-frameworks meant to drive software design as in SDLC, and not meant for enterprise architecture. They are obviously no substitutes for a representation standard with enough flexibility and abstractive possibilities that EA demands.

There are also standards for visual representation of certain architecture dimensions, like BPMN and IDEF1X .But they, again, are specialized at handling respective information domains like process or data. The methods that IDEF proposes are quite comprehensive in that respect – that covers everything from standards for User interface (IDEF8) to Implementation Architecture (IDEF10) and Data(IDEF1X ) to Process (IDEF3). But they are not still really enough for Enterprise Architecture. Here is why: We simply need an Integrated View of architecture. It is easier said than done, as enterprise IT architecture addresses a different problem at a different level of abstraction than what the above models attempt. But then, when needed there has be a flexibility to go further level down in terms of details covered in the model

Let us take a quick-and-dirty stock of the requirement for EA modeling standard:

·          It should provide a comprehensive modeling framework

·          It should support an integrated (end-to-end) architecture view

·          It should be easily scalable and maintainable.

·          It should be precise and accurate

·          It should be tool-independent and conceptually portable.

·          It should be simple enough to develop and maintain.

·          It should provide representation framework for different types of enterprise architecture domains, and should support inter-domain relationships

·          It should use standard symbols for concepts and relationships, but should also communicate the underlying idea nearly intuitively to stakeholders

·          It should be capable to support major architecture life- cycle frameworks (such as TOGAF, Zachman)

Is there no silver bullet that satisfies all of these ? As usual, we tempt to say ‘nay – there can’t be‘. But there is something shining out there – let’s have a look: oh – is it not ArchiMate ?!

What is ArchiMate?

ArchiMate is a visual language to represent end-to-end enterprise architecture in terms of business processes, applications and infrastructure(technology) . It provides a consistent framework for designing multiple architecture domains and relationships among them. An integrated representation approach, ArchiMate equips the IT architects with a powerful modeling standard for representing, communicating and analyzing enterprise architecture. Like any effective modeling language should ideally be doing, ArchiMate helps evaluate the impact of changes within multiple architecture  domains and to communicate them effectively and with ease.

If you are interested in a little history, ArchiMate was a 30 months project undertaken and managed by Telematica Institute, which is essentially an international consortium of companies and several social / knowledge organizations. The project took 35 man years, and its approximate cost of 4 million euros was collectively funded by Dutch government and business partners like ABN AMRO and CWI. With due focus on the relationship between business and IT architectures, the project managed to come up with a comprehensive language for describing architecture models with precision to enable IT architecture designing solutions to standardize their techniques and offerings for effective visualization and analysis.

It is as much important to know what ArchiMate is not as to understand what it really is. ArchiMate is not software development meta language like UML, and it does not support representations in that level granularity. It is also not, like Zachman or TOGAF, an all-encompassing collection of architecture methods that can serve as a framework for the enterprise architecture to function within – its role is limited in enabling the visualization and analytical problem(s) associated with standard architecture frameworks.

In no man’s land: Positioning ArchiMate

Now, the beauty of ArchiMate is that, though it is not UML and not TOGAF, it can seamlessly correspond and relate to either of this as and when required.

Take UML as an example. ArchiMate modeling framework can accommodate most of the UML view-points. More specifically, the three composite domains (or layers)  that ArchiMate supports – business, application and technology – have a direct correspondence with three major UML diagrams – Class Diagram, Component Diagram and Deployment Diagram respectively. UML’s versions of information modeling are meant to address much finer levels of details, while ArchiMate can act as an abstract modeling framework for UML. It can also be used to meaningfully combine multiple UML views into one, thereby making use of ArchiMate as a more intuitive and self- explanatory representation working on top of the concepts captured and depicted using UML, so as to be able to present them before the stakeholders who are less familiar to Unified Modeling standards. Telematica Institute provides a detailed profiling of UML for ArchiMate in their website.

Though the nature of parallelism varies significantly, a similar statement can be made in case of TOGAF as well. While UML can be abstracted using ArchiMate, TOGAF can be typified with it. There is very high degree of similarity between TOGAF views and ArchiMate domain viewpoints in certain areas. TOGAF’s “Business Architecture” component can be visualized with ArchiMate’s business viewpoint concepts, “Information System Architecture” with application concepts and “Technology architecture” with the concepts of infrastructure concepts in ArchiMate.

There are definite ‘impendence mismatches’ between some of the TOGAF viewpoints and those of ArchiMate  due to the fundamental disparities in the roles each is supposed to play in Enterprise Architecture. TOGAF is meta-strategic blueprint for EA and ArchiMate is rather concerned about more “grounded” implementation aspects.

Dissecting the Dissection Box

 A complete coverage of ArchiMate’s taxonomical /ontological aspects is out of scope of this write-up. But let’s have a very quick overview to see as to what ArchiMate typically offers to an enterprise architect:

The composite view prepared using ArchiMate has 3 Layers : Business, Application and Technology. Each layer is self- contained despite being a component of the integrated model. Each layer caters to one or more architecture domains.

The Business Layer talks about how the business processes, services, functions and events related to each other and with the associated individuals and business units. This layer is defined to be consisting of Information, Product, Process and Organization domains.

In the Application Layer , the software applications that support the components in the business Layer, along with the information processed by these applications are described. This layer is defined to be consisting of Application and data domains.

The Technology Layer deals with the hardware and communication infrastructure needed to support applications specified in the Application Layer. This layer is defined to be consisting of  Technical Infrastructure domain.

Each layer contains multiple component entities called Structural Aspects or simply “structures”. Structures are key to interpreting the layer as well the integrated model at length. The answers to the question of “what do the structures do? “ are embodied by another set of entities called  Behavior Aspects, and those of the question “with/by using what? “ , by a third type of entities called Information Aspects ( ArchiMate calls all these entities as simply “concepts”).

For More details, visit http://www.archimate.org/

Illustration

Here is a composite (integrated) architecture of the IT-setup in a fictional hospital - created using Archimate concepts. The topmost layer is an overview of the way the hospital functions. The one beneath that is the application layer that talks of the major application components that enables the business , along with their inter-relationships. The bottom-most one is the technology/infrastructure framework that hosts the applications of the hospital. Please bear in mind that more than the accuracy, the focus of this representation is on showcasing the potentiality of ArchiMate as an EA tool.

Click on the graphic and zoom it to view in full-size

Wrap-up Comments

The Open Group  (the gods of TOGAF)  has taken over the maintenance and control of ArchiMate couple of months back, and this is a very happy event for genuine lovers of the ArchiMate way of doing things (like me). I always used to wonder why the acceptance (even the awareness, for that matter) of ArchiMate is so flimsy, which should have hardly been the case with a standard of this flexibility, simplicity, comprehensibility and ubiquity (well – you can use virtually any drawing tool to pull together an ArchiMate diagram).  I earnestly hope it will soon be possible for IT architects to communicate their models developed in ArchiMate without using even a legend, to almost everyone within and outside their enterprises. ArchiMate does have that potential.

Integration Methodologies: Melting Frontiers Or Age of Divergence ? – Part II

In the first part of this discussion, we have seen quite a hatful of interesting jargons and acronyms ‘basking’ under both EAI and EDI, the niche of each is apparently unique and delimited. There are overlaps – pardonable ones. But our core discussion is not on these subsections anyway; we are here to know how appropriate it is to address data and application integration problems separately, where they converge – if at all- , and where down the road the industry and integration solution providers want to see each of them.

In IT architecture, the greatest rule of thumb says that if a problem baffles you, state it in business terms before addressing it. Our problem thus reads as to (1) what the users want to see after integration and (2) how far data and application integration methods can work together or in isolation to bring about what they want to see. So, yet again, it all boils down to the obvious center of enterprise universe – requirements.

Here too, the classical classification of requirements is invariably applicable – ‘business’ and ‘technical’. (I would always choose to call them more broadly and sensibly as operational requirements and enabling requirements – but that’s another discussion, anyway). Assuming that the business requirements heavily depend on the pertinent domain, and that it is not all that worthwhile to delineate some across-the-board pattern in business requirements that drive the integration, we’d better focus on the technical ones. But it will be even better if we combine the requirement in a need-and-fulfillment model. Let’s do that.

Hitting The Object(ives)

So, what is it, in simplest terms, a typical enterprise ‘E’ expects out of enterprise integration methodologies in general and how these methodologies meet the expectations?

·          To integrate business processes by facilitating enhanced interaction among ‘relevant’ enterprise components in IT spectrum (data, departments (business units, sub- organizations), actors( human resources), applications, interfaces, locations, services, sub-processes, functions, standards, rules  and infrastructure).

·          To broaden the business perspective across the enterprise by collating/ aggregating/re-organizing/re-structuring/re- representing relevant enterprise data segments.

·          To widen the ambit of enterprise’s functional standards, business policies, operational rules and governance models by unifying/centralizing/federating the hosting and management of business rules.

·          To assist effective and seamless co-ordination/mergers/amalgamation processes among business-units by improving connectivity, inter-operability and communication across disparate platforms and technologies.

·          To improve the enterprise’s adaptability to changes in line with business goals and objectives by establishing enough  ‘channels’  in terms of configurable mappings between business logic and technology for efficient change- propagation.

·          To optimize the efforts involved in altering(adding/removing/updating) an ‘enterprise component’ by optimizing degree of coupling among components

·          To reduce the cost of maintaining and scaling the enterprise IT framework by standardizing and reusing enterprise-wide interfaces or other middle-ware infrastructure and by reducing the number of “point-to-point” connections.

·          To ensure functional and informational consistency in the enterprise by unifying/standardizing transferring protocols in tandem with an increased usage of business semantics while defining interfaces.

Now that we have seen the expectations of integration, we are in a very good position to sum up the “KSAs( Key Solution Areas)” that could be used to appraise the role and effectiveness of integration methodologies to position them accurately :

·          Process Orchestration  : Components’ Coordination (broadly : Functional Co- ordination)

·          Enterprise Perspective : Data Organization

·          Governance : Business Rules Management

·          Business Scalability and Technology Portability : Platform Independence

·          Agility/Change Management -: Change-Propagation

·          Agility/Flexibility  : Coupling Optimization

·          Cost Control: Interface and Components Reusability

·          Consistency :Message Standardization

EAI to Bear The Torch

The distribution of applicability of EAI and EDI methods (that we have seen in Part-1 of this discussion) across each of these KSAs would look somewhat like this:

·          Components’ Coordination  – EAI , EDI

·          Data Organization – EDI

·          Business Rules Management -EAI , EDI

·          Platform Independence EAI , EDI

·          Change-Propagation-  EAI

·          Coupling Optimization – EAI

·          Interface Reusability – EAI,EDI

·          Message Standardization  – EAI , EDI

This is as much an interesting result as an important one. EAI by its definitions and for all its accepted forms/types can be found to be ‘wider and deeper’ in its scope. In other words, EAI turns out to be a definitely broader area, or at least to be more significance to the enterprise in terms of meeting its expectations out of  consolidating its  IT ‘ingredients’.

EAI also has its firm footprints on many layers of integration where data integration methodologies lack obvious and explicit applicability – be it Presentation Layer and Network Layer ( No, not OSI reference – just the implementation layers). If someone wants to adulterate the scopes and definitions to argue otherwise, well – s/he should be an awfully loyal EDI fan. Recommended 3-step algorithm for him/her  is –  start with that assumption , use Reductio ad absurdum and get disappointed.

Shared Pains

 Sarcasm apart,  there are a lot of common integration problems where there is enough room for solid give-and-take between these methodologies. Some of them are,

·          Need for Independence/transparency of platform heterogeneity

·          Data Exchange Requirements and formalized Data Formats.

·          Need for message integration and workflow management

·          Need for Control and coupling

·          Usage of common business integration middleware.

·          Repository-driven Meta Data management requirements

·          Need of Federated Models (in some scenarios)

·          Common security challenges.

·          Distributed data manipulation and concurrency control.

·          Usage of integration assistants like APIs, XML,RMI Service Interface standards such as WSDL and HTTP/SOAP.

·          Desired common traits of integrated framework include Scalability, reliability, consistency and availability.

Role Convergence  

To hone our discussion, we need to have a look at two more distributions – this time to affirm the activity-overlaps across the integration methodologies :

(1) Distribution of EDI presence across EAI categories : The amount of Data Integration activities typically performed as part of the majot EAI types we have seen in Part-1 of this discussion (Functional Integration , User Interface Integration and Data-level integration)

(2)  Distribution of EAI presence across EDI categories : The amount of Application Integration activities typically performed as part of the major EDI types we have seen in Part-1 of this discussion (( EDR,ETL,ECM,EII,MDM,CDI,BPI and EAI itself))

Conclusion

Presence of several monolithic integration approaches may slim down over the years as evolving paradigms would either supersede them or make them completely redundant. Still at lower strata of activities, these changes would prove to be nothing more than surges of ‘terminological feats’ , for, from an Enterprise Architecture perspective, both Application and Data Integration methodologies will forever be considered as key enabling techniques for all broader enterprise technical programs such as mergers and acquisitions, strategic architecture reorientations, SOA implementations, service clustering, business process integration and enterprise activity monitoring.

A good example is what the enterprise integration arena has seen with the advent of ESB or SOA – many schools of thought preferred to see them in mutual exclusion with EAI.  EAI, despite its terminological advantage in terms of ‘interpretive flexibility’ that should have qualified it to be considered as a functional superset of even SOA, was demeaned to deal with sheer point-to-point integration (hub-and- spoke, at best). Fact of the matter is that SOA as a conceptual framework and ESB as a middleware infrastructure ( or as a common information model) could all be a part of EAI union that should help a grand classification of like-minded models. But then,  the proponents of each new model want to see them in isolation, properly insulated from the ‘adulterating  influence’ of their less sophisticated technological ancestors. So things never so happened that SOA had been added to EAI concepts, but rather, SOA has been ‘introduced in place of’ EAI concepts. Of course, only then it’s convenient to present the world with a roadmap so strikingly titled as ‘From EAI-to-SOA’!

My feeling is that EAI is the only term with enough historical and semantic potential to accommodate ESB-based approaches, most of data integration methods, Composite/Federated Applications, Saas, Service Oriented approaches as well as those “inter-silo mashups” ( for which EAI is unfairly synonymous with). I am hardly a name junkie, but I do believe that consistent categorization and evolutionary traceability are important – very important.

Part1 : The Integration Big League

Integration is pretty much what IT architecture is all about. Once we get rid of the ‘fat’  from this area and get to the core of affairs, all that matters is the way components ( data,application,technological ) are fastened together. No wonder Enterprise IT Architecture and Systems Integration were used interchangeably up until a close past.

But then there are a huge lot of methods out there, the purpose of which would rather seem to confuse than to assist. But I am sure there can be no argument as to which of them are most significant – Enterprise Application Integration (EAI) and Enterprise Data Integration (EDI). As a term, EAI has proven to be monumentally more significant than EDI, but when it comes to the activities that either of them deals with, the disparity vaporizes.

In an Enterprise Architecture evolution context, where would they stand tomorrow as two approaches of integration? In a time and age when ‘components’ are made with integration in mind, how relevant (and possible) is it to retain their distinct identity? Are they heading for convergence? If so, which of the two would potentially supercede another (if at all)? Which other integration methods would get encapsulated in either of these two?

Loads of great questions, and let’s not wait to hash them out.

Egg is not the mother of Chicken, or is it?

All serious Enterprise Architecture practicians might’ve noticed how EAI and EDI are often used in affiliation with one another, if not in place of one another. But if we try to ignore this phenomenon as yet another industry misnomer, we need to be careful. I have come across many research papers and books making contradicting remarks when EDI and EAI are referred in context with each other. Some say EDI activities and tools are a part of EAI framework; others speak something quite the opposite. Still others argue that there is not much difference anyway.

And they can all be right, too!

For the time, though, let me jot down my understanding of the most accepted distinctionsbetween EAI and EDI, before dissecting each of them too much:

·          EDI focuses on unifying enterprise data views; EAI enables unifying business processes by blending applications’ capabilities.

·          EDI  is data-centric; EAI is ‘message’ centric.

·          EDI deals with high-volume data movements; data movements in EAI is normally limited to real- time/near real-time data ‘propagation’

·          The technologies used for typical EDI operations are different from those of EAI

·          EAI is normally defined to be integration of data with application or application with application; EDI is to do with data to data integration.

·          In general, data extraction is a ‘pull’ activity in EDI and in EAI; it is a ‘push’ one.

·          Data Integration may not radically alter the systems architecture of an enterprise, but Applications Integration does.

·          At one level down, the frequently talked about sub- types of EDI (EII,MDM,ETL etc) are significantly different from typical activities surrounding EAI

·          EAI can start right from the component development (by ensuring compatibility), whereas EDI is almost always an implementation/interfacing activity.

·          There are more standards, guidelines and protocols to be followed to realize EAI than in the case of EDI. Many EDI efforts can be ad-hoc and tactical.

Sigh! There are quite a few, right?

But as I posited, these ‘differences’ haven’t seemed to restrain many experts from bracketing either of them under the other. Some believe EAI is a part of EDI just like EII,ETL and Data Federation. There are other schools of though that sees no reason why EDI is not a part of EAI framework, which according to them, is immense and flexible enough to accommodate that and much more.

These chickens and eggs!

Let the argument go on; we shall go back to tackle some more important questions by taking a nigher look at each of them.

Implicitly implicit

Interestingly (and quite thankfully), there’s been a clear pattern in the way EAI technologies have evolved overtime. It is fair to say that the definition of EAI has also evolved with them. But then, arguments in favor of placing EDI under the EAI umbralla have been there in all elementary definitions of EAI. Here is how Gartner group defines EAI:

            unrestricted sharing of data and business processes among any connected application or data sources in the enterprise.” (  a clear indication that we can regard           EDI as a subset of operations and framework that EAI offers.)

But the moment we see something like “business process” in its definition, EAI gets much more than a set of application or data components technically welded together. Linking business processes demands a well- orchestrated, business-centric approach.In other words, EAI is the technical consolidation of application silos in accordance with a set of business rules,to satisfy a set of business requirements.

An interesting aspect of EAI is that it can give rise to a set of new functionalities that the component applications wouldn’t provide in isolation. But mind you – these ‘generated functionalities’ are not‘technical accidents’ of EAI, but a result of well thought-out and estimated efforts.But both the expectations as well as the methods around this area have been varying over the years.

Wing and Venky (Enterprise Architecture and Integration,2007) give an interesting picture of how the focus of applications integration efforts have shifted over time – from one of technical to that of business.

They accurately points out that

            “The perception of integration has therefore shifted over time from tactical, low-business-value technical plumbing to a strategic and high-value enabler for     achieving business competitiveness.”

And the roadmap they present has a picture of EAI transmuting from “technical plumbing” to business-enabling frameworks generation.The focus areas making their appearances in the chronological order are systems integration, real-time integration, ERP Integration, Web Integration, Supply chain management, e-business integration,CRM, B2B Commerce and finally – collaborative commerce.

So, the future of integration is decided by business trends and paradigms rather than technological advances in data/message propagation. What does it mean to the question of addressing EAI and EDI as distinct integration streams? Answer seems quite obvious at first glance, now that we have seen the diminishing importance of integration methodologies in a rather business-centric integration environment. But is that truly the case?

To analyse that further, we must have to take a look at the types of integrations falling under each of EAI and EDI.

Let me encapsulate major EDI and EAI categories in two tables :

 *black box integration approach hides internal complexities of the application or database. White box approach does it the other way around.

 Next : Part2 : Convergence and Convenience

Agility and ‘Range of Change’

The greatest benefit of keeping a set of ‘hypothetical’ requirements is to be able to measure what can be called “Range of Change” of the architecture. This is a positive offshoot of being proactive to changes, and this helps define the floor and ceiling of the IT architecture evolution in an organization for a specific period of time (the period during which the current architecture is valid). Range Of Change can not only drive agility, it could well be used as a crystal ball for forecasting cost and resource demands in co- ordination with company’s IT roadmap. This keeps off many ’surprises’ and would be a great tool for CIOs/CTOs/PMO as well as Enterprise Architects.

This translates into a closer alignment of IT Roadmap with economic projections of the company. The closer the alignment, the more pragmatically agile the architecture will be. This is a refreshing answer to an interesting question: “OK – we have an architecture that is agile. Architecture that is responsive to change. But how can we convince the stakeholders that our response to the change is the best response to the change in economic terms? We believe that the interfaces defined, coupling of components, and many other agility factors in our solution are optimal and proper, but how to make a strong case of it?”.

Knowing the Range Of Change is the answer. Once you have the scenarios cut out to present to the concerned parties, you can speak with clarity and with figures for your solution. The stakeholders can anticipate how high and low the cost can really vary with a change, and they can’t ever be ‘unpleasantly surprised’ when they hear about ‘cascaded investments’ that follows a change.

Let us now see how we can put Agile Requirements into practice.

Life of Agility Requirements

In the earlier part of this discussion we have seen how requirements are derived. Enterprise Architecture Agility Requirements are nothing but a collection of expected /projected/desired changes for a specific period of time in Enterprise Business- Application-Technology domains captured at right level of abstraction in the form of requirement statements.

 An Agility requirement can therefore be of one of these types: Business Agility Requirement, Application Agility Requirement, Data Agility Requirement and Technology Agility Requirement. The importance and role of these requirements are decided by the architecture domain since the applicability and the ‘point of activation’ of requirements corresponding to each domain could vary. A Business Agility Requirement would influence Application and Technology Architectures but a Technology Agility Requirement may not call for a change in the Business view.

Alternative Architectures: Building Practicable Scenarios for Agility

Let’s now see how Agility Requirements actually get translated into a rather practicable model. An Agility requirement, as we have seen, is a high level ‘potential’ requirement. So, if we prepare ’scenarios’ applying these requirements to the actual Enterprise Architecture, we end up having multiple ‘Alternative Architectures’ – i.e., multiple hypothetical versions of Enterprise Architecture.

Here is an outline as to how the process of translating agility requirements to alternative architectures look like:

·          Pick each of the Agility Requirements in turn and subject them to a rigorous review process to evaluate the chances of the activation of the requirement (occurrence of the change) and thereby influencing in the architecture during a certain period of time in future.

·          If chances of occurrence are found to be sufficiently high, we mark it as a ‘Potential Major Change Event’(PMCE).

·          Identify in which architecture domain each requirement falls in. Identify which all architecture views would get affected by the PMCE. For example, if a PMCE is Business in nature, it can in turn alter the Application, Data and Technology views of the Enterprise architecture. Mark the views that may potentially be altered by the PMCE.

·          Each Potential Major Change Event can trigger a unique stream of”Alternative” Business- Application-Technology architecture development process. Apply each PMCE to the architecture views/representations/high-level collaterals to come up with one modified set of them corresponding to each PMCE.

What we now have is a set of “Practicable Scenarios”, which are nothing but different versions of hypothetical high-level Architecture views ( a combination of Business/Data/Technology views).This can now tell us about how the Enterprise Architecture would look like in the event of occurrence of any of PMCE. They are much similar to multiple architecture views that one would have used for brainstorming and reviews, usually at the scoping and planning phase of Architecture development process ( Don’t throw away serious Business Scenarios such as those as specified in Part-4 of TOGAF for requirement gathering/scoping etc. They can come very handy in defining Alternative Architectures).The major difference in this case, though, is that the views we now have are much more formal and in alignment with the standard architecture frameworks that have been used to represent the Enterprise Architecture.

This process is described in the graphic below:

Benefits

Now, an obvious question here would be about the cost effectiveness/ROI of this process. Developing and maintaining multiple versions of architectures may be perceived to be a significant overhead.  Of course this is more work, but it is also of lot of value.

Here is how the Enterprise eventually benefits from developing Practicable Scenarios along with Enterprise Architecture and maintaining them:

·          Team Mobilization: If we analyze potential changes and make alternative architectures at the outset, the biggest benefit is the presence of a team that is mobilized and engaged explicitly with the enterprise architecture development work. So if a significant and radical Enterprise Architecture initiative (e.g. Adopting TOGAF) is underway, the team can work on Agility Requirements and Practicable Scenarios development. (In cases of frameworks like TOGAF, Business Scenarios development is a part of development cycle. This will give additional leverage to agility efforts.

·          Context Prevalence: When an EA initiative is in full swing, the context, awareness and ‘mood’ stands to be just right for the documenting and filing alternative architectures as inputs for future changes. If this is to be done in response to a change, the whole context of the architecture has to ‘loaded’ on to the Enterprise ‘memory’ (which is largely volatile!). So do it when the sun is still shining.

·          Finer Focus: As we have seen above, the management, team, stakeholders, vendors and other parties would all be most alert and focused when a major strategic formalization of Enterprise Architecture happens rather than responding to a major change in the enterprise.

·          ‘Swiftest’ Agility: The earlier we have a blue print in hand as to what to do when a major change happens, the less will be the risk involved, and faster the response.

·          Fastest and Effective Change Communication: With predefined alternative architectures, All the stakeholders can be notified about the changes and its repercussions along with an approved and governed strategic response in a standard, presentable to the change in least possible time.

Change-Managing Alternative Architectures

The change management of architecture prepared for agility purposes is a little tricky, and is entirely upto each architecture board and organizational culture. In brief, though, we can say with certainty that changes made to Enterprise Architecture have to get reflected on the Alternative Architectures and need to be cascaded across each set. But this can be done periodically rather than event-based, to save the overhead of having the change reflected in the alternative architectures each time there are updates to EA. The justifying assumption here is that a robust EA won’t change so frequently as to make the delay between one of its changes and the update of alternative architecture can pose a serious problem. As we have seen, the trade-off is between agility and initial efforts.

Conclusion

Developing and maintaining multiple Practicable Scenarios in the form of alternative Enterprise Architecture enhance proactive nature of the architecture management in general and agility in particular. These views help the Enterprise respond to a change with swiftest and most efficient possible agility. It also helps in communicating impacts of major business and technology changes to all concerned parties with ease as and when the change gets implemented, with minimum time spent in waiting for approvals and buy-ins.

Agility, Architecture and Agile Architecture

What a good word Agility is!  You are agile only when you are alert and adaptable, both. Hats off to the wise soul – whoever it was – that discovered its utility in ever-wavering frames of Information Technology practices. In any case, it was just a matter of time since its possible coinage before we started seeing its decisive presence in a gamut of IT methodologies and standards – Agile Programming, Agile Design, Agile Application Development and of course Agile architecture.

Why should architecture be agile? Is it advisable to see something that is to be rightly regarded as an authentic reference/blueprint of action to be as a variable in itself? Is it just about architecture change management?

Or to ask more directly- how important is the concept of agility in architecture?

One plain (and proper) definition of agility is as the ability to react to upcoming changes effectively and efficiently (Ambrose, Morello, 2004). In this sense, each architecture effort, if it is not done as a part of a fresh full life-cycle project, is meant to deliver exactly this purpose. It is so important that one can even argue we do architecture for agility rather than with agility.

Implicit and Explicit Agility

We need to know what we have and how it all add up in order to be able to assess the impact of a change so much as to know what is going to change, when, where, how and why it changes. We also need to know the architecture of our systems to know what each change will cost us in terms of money, time and effort. So to have a good architecture is the undoubtedly the first step of being agile. In other words, the greatest objective of Enterprise Architecture is to assist agility. Going by this logic, one may tend to think that Agile Enterprise Architecture is badly phrased; if not that it is an absolute oxymoron. And believe me – this is not entirely wrong. Agility is implicit in IT architecture.

But intention of this being regarded as a major EA topic is good. We normally say Agile Architecture to stress the importance of an architecture that is optimized for changes. So, while capturing the “As-Is” architecture is implicitly and arguably agility-driven, defining the ‘To-Be’ state is much more than that. In the latter case, we strive to devise an architecture that is to respond to changes ‘effectively and efficiently’, rather than to pull the current architecture together to fine-tune our understanding of the system(s) to handle potential changes ’effectively and efficiently’. We should therefore rather focus on this future-state modeling when we talk on Agile Architecture.

Focus On Agility: Agility Requirements – Foresee & Be Agile

Agility is the conceptual parent of many best-loved architecture features like dynamism, vibrancy and some most popular paradigms like SOA. They all deal with agility in their own ways. Agility is everywhere; though it is important to know how much is present where.

We have just seen how architecture can be implicitly and/or explicitly agile. But what do we normally do for making architecture agile? A good enterprise future-state architect definitely keeps the big agility factor in mind, but how often do we address agility factor upfront? Many architecture frameworks overlook considering agility in scoping and planning phases. We need to aim directly at the bull’s eye to increase the probability of striking it – we need to treat agility as the foremost success-criteria upfront for an architecture undertaking to make the implemented system agile. The first and most important step in this direction is to understand the organization’s ‘agility requirements’. When we gather architecture requirements, we normally tend to neglect the need of capturing a set of requirements under the head ‘agility requirements’. But the challenging question is how we can identify our ‘agility requirements’. How can we put down something that is uncertain and fluid? And most important of all, who is going to provide agility requirements?

We will see this in detail in later sections soon, and for the time, we only need to understand that agility as an architectural characteristic that is worth getting prime attention – expressly and systematically.

If we can actually expect changes, we would respond to them more effectively. And this simple logic makes a powerful case for gathering, organizing and representing agility requirements as one separate part of requirements collection phase of the architecture effort. Not only that it saves time, money and efforts, it also places the change in a politically and strategically favorable zone now that the change was an anticipated one. But is it about predicting changes, really? And if so it is, is it practical?

Science and Methods of Agility Requirements

Deductive Method

One way of identifying agility requirements is to understand the actual pattern of your business and technology changes. In most enterprises, there could be a significant degree of commonality across ‘System Change Events’ (SCE, if you like) happening over a period of time, due to business, ‘political’, geographic, economic and other reasons. It is possible to get considerable insights on the nature of ‘expectable’ changes by analysing changes that have been happening for a certain period of time. Presenting a full- fledged methodology for identifying the SCEs and analysing them is beyond the scope of our present discussion, though I would wish to rake up the applicability of some of the classic methods of pattern identification in this area – such as statistical approaches and data mining.

It is quite possible for a good enterprise to keep a history of their key business, application, data and technology changes, which I prefer to classify as System Changes (yes – I understand ‘system’ is a subjective term). In some cases though, the changes would be captured at a very granular level and without any proper plan to place the changes in context with each other to make it architecturally significant. The changes in Network topologies , changes in CI configurations and resources, applications patching and upgrades, adding new types of users to an application or configuration items are all System Changes but not with much use until they are rolled up to a proper higher level. Either way, with a focused and scientific approach, it is possible not only to conform these changes to a certain desired level of abstraction, but also to ‘make sense’ of this data.

So the ‘deductive method’ of agile requirement capture include mainly 2 steps :

      (1) To Prepare: To collect, organize, level-up the System Change Events in order to dress them       up for analysis.

      (2) To Analyze: To apply mathematical-statistical data mining techniques to draw    inferences that are direct inputs to Agile Requirements capture process.

What we look out for in an organized set of historical change records is a pattern of occurrence of various categories of changes, so as to be able to make sensible prediction and projections..

Let’s see some examples: 

Result of SCE analysis:

      There is a pattern in which a particular technology component ‘x’  is rolled out in a particular  geographical entity ‘y’.

The agility requirement corresponding to this finding would be something like:

      ‘To anticipate the implementation of upcoming releases or versions of ‘x’ in ‘y’.

Another One:

      ‘There are cases of recurring failures of implementing a certain strategic directive /standard ’s’ in     certain client account ‘y’ ‘.

This translates into an Agility requirement statement that goes like:

      ‘To anticipate implementing an alternative to ’s’ in account ‘y’ ; to keep an alternate blueprint of       implementation just for this account.’

In some cases, the ROI can be found significantly higher with specific changes, which would make way to repeat similar changes in future. Cases like this enable us to make fair, rather direct predictions.

Analysis and consultation

The most obvious method of gathering agility requirements is to do that in a requirement gather session from business representatives and stake-holders as ‘parking lot’ buy-ins. It is not recommended to get very formal with the process, though – as it may sound to be a rather speculative business. There should be enough transparency while asking for expectations. The ‘donor’ of a strategic tip-off may not be ready to own it or vouch for it , and rightly so. But it may prove to be valuable when you do agility requirements analysis.

Another important option is to seek internal or external professional assistance to do keep a tab on industry trends and developments. Agility of the enterprise is proportional to its foresight substantiated by quality trend-analysis, and the architecture responsive to the industry developments always identifies and assimilates trend data in consultation with business experts and analysts who can make quantified predictions as to which business and technology components can undergo changes in a certain period of time.

Next: Agility Requirements: The Groundwork of Agile Architecture Part-2 : Real Actions, Range of Change, Cost , resources and risk

“Getting Meticulous About It…..”

EA and EA Projects – Federated Scoping

Sorry, I can’t help harping on (I tried, though) the great ‘problem of rendition’ in EA. Right from the definition through the details, there is enough room for ‘convenient interpretation’ of many terms and ideas we find in countless Enterprise Architecture standards, frameworks and guidelines. Granted – the end-to-end roadmaps like the ones offered by TOGAF ADM are jobs very well done, but confusion lingers on in some less cheerful corners of every framework. A good example would be the difference in approaches to EA and to what can be termed as EA projects.

Wait a minute – did I say EA project? What can be termed as an EA Project? Well – everything you actually do as a part of ‘conducting’ Enterprise Architecture (You heard it right – “Conducting Enterprise Architecture” are the exact words used in the definition of MODAF. Not Conducting EA project, nor Conducting EA Activity – just conducting EA. They say MODAF is a “standardized way of conducting Enterprise Architecture and…”). So an EA Project can be any activity with enough significance to alter the EA status-quo as much as much as  being any component activity of building up Enterprise Architecture or being one that is formally aligned with Enterprise Architecture governance mode,.

The last statement, however, adds an improper degree of flexibility and thus sets in motion the wheels of convenient interpretation.

Out of Governance = Out of Scope?

Can we really take into account something that is not governed by Architecture Board as a part of Enterprise Architecture? Practically – yes , because practically in all organizations, there could be activities significant enough to disturb the EA status-quo without getting adequately controlled by governing council. The process of ‘adoption’ to EA framework would only be done at a later stage as a separate activity in such cases – mostly due to pressing deadlines, cash-flow reasons or simply sheer holes in the framework.

That leads us to having a federated approach to the scoping activity too – no matter at which rung of evolution we are at in terms of EA Framework compliance. What it means to us is this: apart from having a firm traction on governed EA Activities, there is a need for considering all substantial undertakings with definite inputs and outputs happening within the enterprise to be an EA Project; the substantiality can ideally be evaluated by governing body with minimal upfront involvement.

An EA Project can thus have a management and control outside Architecture Board. As shown in the graphic, Adoption and integration of the project to EA Framework are controlled by a pre-defined adoption strategy.

But how are all these important in defining scope? They are very important, as, for one, we just saw that a project can be conceived, planned, scoped, executed and managed independently and can still be considered as an EA effort. Setting up right boundaries for such an isolated effort would be the key to all right things to follow.

The long and short of it: Reductionism in Scoping

Think of a sprint event. Which one would you pick as of more consequence- the distance to be covered in the race or each step that is to take you till the finishing line? Like most of you, I too would say : both. Every step is as much important as the distance to be covered. Though done inadvertently and being mastered by practice, every breath taken in, every muscle movement and every hop has its role in deciding the winner.

No holistic framework can be sound enough when it is frail in parts. Assembly may be fantastic but a week link in the chain can spoil the show, and that is where the importance of federation lies. In both the phases of scoping and planning an EA Project, the architect (an individual or a body) should be conscious of the Enterprise Significance of its outcome.

This gives a whole new importance to the scoping activity. Here we are not only concerned with scoping of the implementation of EA as a framework, but also of individual project scope in context with EA. So, the scope of the project should be defined as the scope of the business, application and technology areas of the enterprise as a whole that will be impacted by the project as well as those that constitute the project in isolation. This activity can prove to be providing great inputs to planning phase.

There are more to “TOGAF dimensions”

Back to the question of ‘meta’ scoping. What should we expect out of scoping exercise? TOGAF talks about 4 different dimensions ( details to be captured/ defined as a part of scoping exercise) : Scope of the Enterprise, Scope of Domains, Scope of Details, Scope of Time (not in the exact words used by The Open Group- refer TOGAF 8.1 for details ).

But are they enough?

Most of us who have ever even tried our hands on an Enterprise-Wide projects would not probably defer when I venture to say they are not all. There are some 4 more dimensions that I can think of:

Scope of Inputs:  A finite line should be drawn to mark the boundaries of information (required + available) concerning the effort. Identifying how much information would be open and/or available as inputs to the architecture effort is vital for successful execution of an EA Project . There can be numerous constraints for the Architecture Body in gathering information related to the project – like security, availability, time considerations and accessibility. On top of it, this effort should address the need of distinguish between available information and required information.

Scope of Compliance: Many Enterprise Architecture efforts are “cross-standard”. For example, while the actual development process of the architecture is being carried out in alignment with TOGAF, the security aspects of the enterprise may need to comply with SOX even as the implementation is to be carried out in an environment that follows ITIL-way of functioning. A ‘matrix of accord’ needs to be in place at the scoping phase of the project in order to meet the standards set by a multitude of standards that may or may not have requirements at variance with one another ( there should not be any, ideally).  

Scope of Usage: This is a highly overlooked factor, especially in Federated Architecture environments. As I mentioned in the previous part of my Thoughts, the effort here is to systematically capture answers to the questions like who is going to take up the architecture from the point where you leave, what will they be doing with the deliverable and whether it is development-facing or the business-facing or both.

Scope of Approval: Not everything is to be approved by everybody. Review and approval of the Enterprise Business Model should be limited to the business reps, whereas Application and Technology Models should follow a different/other review stream. I don’t very much buy in to the idea that this should be a part of defining roles and responsibilities (under planning) and not scoping, for this activity is more to do with defining boundaries of ‘jurisdiction’ rather than assigning work.

Scoping Problem in EA

In EA , Enterprise comes before Architecture

We know our problem: not many of us get to try our hand on an across- the- board Enterprise Architecture (EA) . But then EA can also be what you like to call EA. The nature and size of architectural collateral could also vary significantly - may be it is just that cute little end-to-end view that your put together to sell your new idea to your economic patrons. Or may be it is that dreadfully large documentation that does a bible for the implementation team. EA, we should say, is one of the most debatable (flexible, for the positive you) terms of the industry. After all, you know how TOGAF’s introduction defines Enterprise:

A good definition of “enterprise” in [...] context is any collection of organizations that has a common set of goals and/or a single bottom line. In that sense, an enterprise can be a government agency, a whole corporation, a division of a corporation, a single department, or a chain of geographically distant organizations linked together by common ownership.

And Architecture:

“the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution”.

So a simple fusion of above statements should be good enough a definition of EA.  But frankly, I am not one too impressed. An enterprise is an enterprise. The intuitive image of something holistic that pops up in your mind should be boundlessly more satisfactory than any definition. – the big picture, that is , if not the whole picture. Architecture is big picture. EA is a big picture of a big thing. So, let me try to make a contingent statement on my own right here.

Enterprise architecture of a business/ technological entity/domain “x” is the representation at a pre-defined degree of detail of the way “x” has been/is being/will be rolled out in the smallest possible technologically sovereign organizational entity that can be termed ‘enterprise’ in the purview of the project or program or scheme or system for which the architecture of x is deemed relevant.

Good – though I qualify it to be contingent, it should not be that bad a definition. Well, those who really want to hear a definition for Enterprise Architecture bigger than ‘just enterprise-wide IT architecture’ wouldn’t possibly settle for anything less than something with logical precision such as that one. After all, people do like it when the fence around fluidity is put up as firmly as possible.

But that opens up more problems. Among many, one stands out as the most fundamental question to be answered: How to scope anything that is enterprise- wide? Or at the very least, how to approach the scoping problem?

Who scopes the scope?

An important yet often overlooked sphere of scoping exercise is the “meta-scoping” or scope-of-scoping part. At best, we would get too busy in answering the Zachman interrogative model of What-How-Where-Who-When-Why religiously, only to find ourselves overwhelmed even before the gun goes off. In fact, what we have to essentially do at this point is to be very precise: the scope elements should be captured in words rather than sentences, tables rather than documents or “boxes-of-abstraction” rather than full-blown models. We will get plenty of time before we elaborate on any piece of elementary scoping information, as the key activity in the scoping phase is not done by the architect but the stakeholders and approvers. And this key activity is review and approval! Anyone who thinks otherwise is dealing with a dangerous recipe that has enough potential to perpetuate the burden of redundancy or irrelevance of one or more scope element throughout the life-cycle of an enterprise project.

An architect’s role in the scoping phase for the very first iteration is more of a scribe than a technologist, and rightly so. This is the most objective of all architectural activities and any drive to push in a lot of innovation at this part of affairs would only result in widening the mismatches between the expected and the delivered. And this, I believe, is pretty universal a rule for all scoping exercises.

Even while getting iterative with the scope at hand at a later stage, you essentially keep sharpening the scope, rather than elaborating or expanding it. Shrinking or expanding of it means business and money, and there are people out there who are far more concerned about them than you – in your capacity as a solution designer – are. Even the fine-tuning of the scope, which normally includes activities like defining / describing the scope elements should undergo rigorous reviews with business groups in order to get rid of all traces of subjectivity ( it is slightly different from discretion, though – believe me).

Having said all that, we should not fool ourselves by downplaying the need of systematic scoping in any Enterprise Architecture undertaking. Though there are plenty of guidelines available for scoping activity of EA within TOGAF and elsewhere, a practical approach would cut across frameworks as we are about to see.

Rapid Scoping

You, the architect, are there to deliver an optimum architecture in terms of comprehensibility, usability, efficiency and ROI. The fate of each of these “desirables’” would be influenced by the decisions you make in the planning and scoping phases (I like to see them as separate activities; we will talk on that in just a bit). If you are one among those fortunate or unfortunate ones who should be the master of your own actions by being able to make decision on entire  the architectural activities , the best way is to do the following upfront, before getting too much philosophical, metaphysical or even romantic with the work at hand.

Let us see how best we can start with the scoping problem. Open a text editor or your notebook and scribble (quite literally) the answers to these queries right-away as soon as you, the architect, come out of the project (of enterprise visibility) kick-off meeting:

1. What does the term “enterprise” mean to you? What do you think it means to stakeholders? Did you see a difference?

2. What domains (data/business/process/technology/organization/location/people) would your deliverables consist of?

3. Who is going to take up the architecture from the point where you leave?  

4. What will he/she/they be doing with it – will they go present it to the board-of-the-powerful or just put on the developer spikes and get going with it? That is, who is your deliverable going to face – the development crew or the business world or both?

5. Do they plan to manage /change control / govern the architecture?( only yes or no for the time – details later , just to see if what you plan to do is a proverbial “use-and-throw” architecture )

6. If there is no architectural governance in your side of the world yet, how long will the deliverable be valid? What event(s) would make the architecture invalid or needy for changes?

7. How much detailed should the deliverables be (just in terms of sheer paper space, for now) – can you afford a good 100 pages or 3 presentation slides? 

Finally-

9. What is the verification mechanism by which you can make sure what you have captured as scope, valid?

After this quick session, the answers you would end up having in front of you may not be enough to spring much surprises in your mind, but will definitely have the potential to avert undesirable future surprises.

Now it is time for systematic scoping.

The Classic Interrogatives – power of simplicity

To ensure comprehensiveness, the best next foot forward is to go by Zachman Framework’s first horizontal. What-How-Where-Who-When-Why (We may term this as Rigorous Interrogation Method)

is a time-tested, robust way to pursue truth about almost everything imaginable – so there can not simply be a better way for upfront scoping.

This time, though, we don’t go as fast as we did in Rapid Scoping phase – we do it really meticulously. But before getting into potential challenges in the activity, let’s see what questions we need to actually ask in place of just What or How or .

What:    List of Business Objects,

Question: Which entities constitute the business?

How:     List of Business Processes:

Question: What does the business call various collections of activities that transform the enterprise inputs into enterprise services?

Where:  List of Business Locations

Question: Where ( which geographical entities ) is business rolled out ?

Who:     List of Organizations

Question: In which organizations does the responsibility for execution of processes and services rest?

When:   List of Events

Question: What are the events to which the enterprise responds ( upon their occurrence or relative to time) ?

Why:     List of Business Goals

Question: Where can we find pointers or references to business goals, objectives, strategies, or critical success factors significant to the enterprise relative to motivation?