Enterprise Architecture Demystified

Sethuraj Nair’s Enterprise Architecture Thoughts

Posts Tagged ‘EA’

Integration Methodologies: Melting Frontiers Or Age of Divergence ?

Posted by snair007 on July 18, 2008

Part1 : The Integration Big League

Integration is pretty much what IT architecture is all about. Once we get rid of the ‘fat’  from this area and get to the core of affairs, all that matters is the way components ( data,application,technological ) are fastened together. No wonder Enterprise IT Architecture and Systems Integration were used interchangeably up until a close past.

But then there are a huge lot of methods out there, the purpose of which would rather seem to confuse than to assist. But I am sure there can be no argument as to which of them are most significant – Enterprise Application Integration (EAI) and Enterprise Data Integration (EDI). As a term, EAI has proven to be monumentally more significant than EDI, but when it comes to the activities that either of them deals with, the disparity vaporizes.

In an Enterprise Architecture evolution context, where would they stand tomorrow as two approaches of integration? In a time and age when ‘components’ are made with integration in mind, how relevant (and possible) is it to retain their distinct identity? Are they heading for convergence? If so, which of the two would potentially supercede another (if at all)? Which other integration methods would get encapsulated in either of these two?

Loads of great questions, and let’s not wait to hash them out.

Egg is not the mother of Chicken, or is it?

All serious Enterprise Architecture practicians might’ve noticed how EAI and EDI are often used in affiliation with one another, if not in place of one another. But if we try to ignore this phenomenon as yet another industry misnomer, we need to be careful. I have come across many research papers and books making contradicting remarks when EDI and EAI are referred in context with each other. Some say EDI activities and tools are a part of EAI framework; others speak something quite the opposite. Still others argue that there is not much difference anyway.

And they can all be right, too!

For the time, though, let me jot down my understanding of the most accepted distinctionsbetween EAI and EDI, before dissecting each of them too much:

·          EDI focuses on unifying enterprise data views; EAI enables unifying business processes by blending applications’ capabilities.

·          EDI  is data-centric; EAI is ‘message’ centric.

·          EDI deals with high-volume data movements; data movements in EAI is normally limited to real- time/near real-time data ‘propagation’

·          The technologies used for typical EDI operations are different from those of EAI

·          EAI is normally defined to be integration of data with application or application with application; EDI is to do with data to data integration.

·          In general, data extraction is a ‘pull’ activity in EDI and in EAI; it is a ‘push’ one.

·          Data Integration may not radically alter the systems architecture of an enterprise, but Applications Integration does.

·          At one level down, the frequently talked about sub- types of EDI (EII,MDM,ETL etc) are significantly different from typical activities surrounding EAI

·          EAI can start right from the component development (by ensuring compatibility), whereas EDI is almost always an implementation/interfacing activity.

·          There are more standards, guidelines and protocols to be followed to realize EAI than in the case of EDI. Many EDI efforts can be ad-hoc and tactical.

Sigh! There are quite a few, right?

But as I posited, these ‘differences’ haven’t seemed to restrain many experts from bracketing either of them under the other. Some believe EAI is a part of EDI just like EII,ETL and Data Federation. There are other schools of though that sees no reason why EDI is not a part of EAI framework, which according to them, is immense and flexible enough to accommodate that and much more.

These chickens and eggs!

Let the argument go on; we shall go back to tackle some more important questions by taking a nigher look at each of them.

Implicitly implicit

Interestingly (and quite thankfully), there’s been a clear pattern in the way EAI technologies have evolved overtime. It is fair to say that the definition of EAI has also evolved with them. But then, arguments in favor of placing EDI under the EAI umbralla have been there in all elementary definitions of EAI. Here is how Gartner group defines EAI:

            unrestricted sharing of data and business processes among any connected application or data sources in the enterprise.” (  a clear indication that we can regard           EDI as a subset of operations and framework that EAI offers.)

But the moment we see something like “business process” in its definition, EAI gets much more than a set of application or data components technically welded together. Linking business processes demands a well- orchestrated, business-centric approach.In other words, EAI is the technical consolidation of application silos in accordance with a set of business rules,to satisfy a set of business requirements.

An interesting aspect of EAI is that it can give rise to a set of new functionalities that the component applications wouldn’t provide in isolation. But mind you – these ‘generated functionalities’ are not‘technical accidents’ of EAI, but a result of well thought-out and estimated efforts.But both the expectations as well as the methods around this area have been varying over the years.

Wing and Venky (Enterprise Architecture and Integration,2007) give an interesting picture of how the focus of applications integration efforts have shifted over time – from one of technical to that of business.

They accurately points out that

            “The perception of integration has therefore shifted over time from tactical, low-business-value technical plumbing to a strategic and high-value enabler for     achieving business competitiveness.”

And the roadmap they present has a picture of EAI transmuting from “technical plumbing” to business-enabling frameworks generation.The focus areas making their appearances in the chronological order are systems integration, real-time integration, ERP Integration, Web Integration, Supply chain management, e-business integration,CRM, B2B Commerce and finally – collaborative commerce.

So, the future of integration is decided by business trends and paradigms rather than technological advances in data/message propagation. What does it mean to the question of addressing EAI and EDI as distinct integration streams? Answer seems quite obvious at first glance, now that we have seen the diminishing importance of integration methodologies in a rather business-centric integration environment. But is that truly the case?

To analyse that further, we must have to take a look at the types of integrations falling under each of EAI and EDI.

Let me encapsulate major EDI and EAI categories in two tables :

           

Table-1 : EDI Categories
  EDI Type  What is it ?
 
1 EII

Enterprise Information Integration

  • Multiple DBs, single virtual view
  • Fedarated approach
  • Holistic  Business View
  • Need-driven query redirection
  • “Divide-and-conquer” approach to execute queries. 

 

 

2 MDM

Master Data Management    

  • Multiple business components, single virtual ‘view’.
  • Focus on conformance with “Reference Data”
  • Focus on business consistency
  • Similar to CDI in many ways
  • Depends on a predefined / accepted “business semantics”  
  •  

3 ETL

Extract, Transform and Load

  • Synonymous with Data Warehouses
  • Normally batch- oriented
  • Differed data transfer
  • Normally based on “Pull” extracts
  • Normally uses an intermediate “staging area” for preparation and transformation
  • asynchronous
  • centralized hosting
4 RT-ETL

Real Time ETL

  • ETL process done in real-time or near real time
  • Data Propagation rather than bulk- transfer; similar to EAI
  • Normally based on “push” methods
  • “semi-synchronous”
  • centralized hosting
5 EDR 

Enterprise Data Replication

  • Replication of partial or full database
  • Rarely used as an integration solution; normally used for back-up and duplication
  • Normally use DB- triggers / transaction logs
  • CDC and Data propagation are used   
  •  

6 ECM

Enterprise Content Management

  •  Best suited for integrating data “as-is” / unstructured
  • Used for integration/consolidation of documents, web contents and rich media.
  •  ECM Products normally use a Content Management engine on top of  Data Store
7 EAI (We will see this in detail below)
8 CDI Customer Data Integration
9 CDC

Change Data Capture

  • Not so much of a type of EDI as a technique.
  • Publish-Subscribe mechanism
  •  Normally used for near real-time propagation purposes  

     

  

Table-2 : EAI Categories
  EAI Type  What is it ?
 
1 EDI

(We have seen this in detail)

  •  Integration at data level
  • Whitebox or Blackbox approach.
  • All EDI methods come under this category ( as seen above )
2

Functional Integration (FI)

-AND-

Process Integration (BPI+)

  • Integration at processes and data manipulation / interpretation rules level.
  • “Tight” application-to-application integration
  • Approach can be White / Blackbox
  • Used when presentation integration is inadequate
  • Normally wses any of the following technologies :
Message Oriented Middleware (MOM): integration through messages exchanges     

 Configurations : message queuing and message passing.

Examples : IBM’s MQSeries and Talarian’s Smart Sockets.

Distributed object technology:  Integration through application of object- oriented concepts t o middleware.

Configuration : Software components are made into objects which is then can be accessed by other applications across a network through the object interfaces.

Example : CORBA, MS COM+, J2EE,DCOM,RMI

 Transaction processing monitors (TPMs):   Integration through preserving integrity of distributed information resources (databases, files, MQs).

Configuration :To impliment distributed transaction processing support concepts like two-phase commit.  

 Example : BEA Tuxedo,web services 

 

 

3

Presentation Integration (PI)

-OR-

IUI (User Interface Integration)

  • Integration at applications’ prsentation /UI level
  • Whitebox appoach
  • Improved , Integrated access
  • Integrated user interface management(e.g validation, error handling)
4

Application Interfacing

  • Out-of-the-box Integration
  • Implemented with specializedAPIs, connectors or other forms of interfaces to access application components.
5 BPI

 Improved Functional Integration

  • Blackbox Appoach
  • Normally uses advanced middleware solutions like message brokers
  • Standardized and controlled the flow of information with a bus / hub-and- spoke framework.  

 *black box integration approach hides internal complexities of the application or database. White box approach does it the other way around.

 Next : Part2 : Convergence and Convenience

 

Posted in Data Integration, Enterprise Application Integration, Enterprise Architecture, IT Architecture | Tagged: , , , , , , , , , , , , , , , , | 1 Comment »

Enterprise Architecture : How Not To Miss Anything ? -Part I

Posted by snair007 on June 17, 2008

Scoping Problem in EA

In EA , Enterprise comes before Architecture

We know our problem: not many of us get to try our hand on an across- the- board Enterprise Architecture (EA) . But then EA can also be what you like to call EA. The nature and size of architectural collateral could also vary significantly – may be it is just that cute little end-to-end view that your put together to sell your new idea to your economic patrons. Or may be it is that dreadfully large documentation that does a bible for the implementation team. EA, we should say, is one of the most debatable (flexible, for the positive you) terms of the industry. After all, you know how TOGAF’s introduction defines Enterprise:

A good definition of “enterprise” in […] context is any collection of organizations that has a common set of goals and/or a single bottom line. In that sense, an enterprise can be a government agency, a whole corporation, a division of a corporation, a single department, or a chain of geographically distant organizations linked together by common ownership.

And Architecture:

“the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution”.

So a simple fusion of above statements should be good enough a definition of EA.  But frankly, I am not one too impressed. An enterprise is an enterprise. The intuitive image of something holistic that pops up in your mind should be boundlessly more satisfactory than any definition. – the big picture, that is , if not the whole picture. Architecture is big picture. EA is a big picture of a big thing. So, let me try to make a contingent statement on my own right here.

Enterprise architecture of a business/ technological entity/domain “x” is the representation at a pre-defined degree of detail of the way “x” has been/is being/will be rolled out in the smallest possible technologically sovereign organizational entity that can be termed ‘enterprise’ in the purview of the project or program or scheme or system for which the architecture of x is deemed relevant.

Good – though I qualify it to be contingent, it should not be that bad a definition. Well, those who really want to hear a definition for Enterprise Architecture bigger than ‘just enterprise-wide IT architecture’ wouldn’t possibly settle for anything less than something with logical precision such as that one. After all, people do like it when the fence around fluidity is put up as firmly as possible.

But that opens up more problems. Among many, one stands out as the most fundamental question to be answered: How to scope anything that is enterprise- wide? Or at the very least, how to approach the scoping problem?

Who scopes the scope?

An important yet often overlooked sphere of scoping exercise is the “meta-scoping” or scope-of-scoping part. At best, we would get too busy in answering the Zachman interrogative model of What-How-Where-Who-When-Why religiously, only to find ourselves overwhelmed even before the gun goes off. In fact, what we have to essentially do at this point is to be very precise: the scope elements should be captured in words rather than sentences, tables rather than documents or “boxes-of-abstraction” rather than full-blown models. We will get plenty of time before we elaborate on any piece of elementary scoping information, as the key activity in the scoping phase is not done by the architect but the stakeholders and approvers. And this key activity is review and approval! Anyone who thinks otherwise is dealing with a dangerous recipe that has enough potential to perpetuate the burden of redundancy or irrelevance of one or more scope element throughout the life-cycle of an enterprise project.

 

An architect’s role in the scoping phase for the very first iteration is more of a scribe than a technologist, and rightly so. This is the most objective of all architectural activities and any drive to push in a lot of innovation at this part of affairs would only result in widening the mismatches between the expected and the delivered. And this, I believe, is pretty universal a rule for all scoping exercises.

 

Even while getting iterative with the scope at hand at a later stage, you essentially keep sharpening the scope, rather than elaborating or expanding it. Shrinking or expanding of it means business and money, and there are people out there who are far more concerned about them than you – in your capacity as a solution designer – are. Even the fine-tuning of the scope, which normally includes activities like defining / describing the scope elements should undergo rigorous reviews with business groups in order to get rid of all traces of subjectivity ( it is slightly different from discretion, though – believe me).

 

Having said all that, we should not fool ourselves by downplaying the need of systematic scoping in any Enterprise Architecture undertaking. Though there are plenty of guidelines available for scoping activity of EA within TOGAF and elsewhere, a practical approach would cut across frameworks as we are about to see.

 

Rapid Scoping

You, the architect, are there to deliver an optimum architecture in terms of comprehensibility, usability, efficiency and ROI. The fate of each of these “desirables’” would be influenced by the decisions you make in the planning and scoping phases (I like to see them as separate activities; we will talk on that in just a bit). If you are one among those fortunate or unfortunate ones who should be the master of your own actions by being able to make decision on entire  the architectural activities , the best way is to do the following upfront, before getting too much philosophical, metaphysical or even romantic with the work at hand.

Let us see how best we can start with the scoping problem. Open a text editor or your notebook and scribble (quite literally) the answers to these queries right-away as soon as you, the architect, come out of the project (of enterprise visibility) kick-off meeting:

1. What does the term “enterprise” mean to you? What do you think it means to stakeholders? Did you see a difference?

2. What domains (data/business/process/technology/organization/location/people) would your deliverables consist of?

3. Who is going to take up the architecture from the point where you leave?  

4. What will he/she/they be doing with it – will they go present it to the board-of-the-powerful or just put on the developer spikes and get going with it? That is, who is your deliverable going to face – the development crew or the business world or both?

5. Do they plan to manage /change control / govern the architecture?( only yes or no for the time – details later , just to see if what you plan to do is a proverbial “use-and-throw” architecture )

6. If there is no architectural governance in your side of the world yet, how long will the deliverable be valid? What event(s) would make the architecture invalid or needy for changes?

7. How much detailed should the deliverables be (just in terms of sheer paper space, for now) – can you afford a good 100 pages or 3 presentation slides? 

Finally-

9. What is the verification mechanism by which you can make sure what you have captured as scope, valid?

After this quick session, the answers you would end up having in front of you may not be enough to spring much surprises in your mind, but will definitely have the potential to avert undesirable future surprises.

Now it is time for systematic scoping.

 

The Classic Interrogatives – power of simplicity

To ensure comprehensiveness, the best next foot forward is to go by Zachman Framework’s first horizontal. What-How-Where-Who-When-Why (We may term this as Rigorous Interrogation Method)

is a time-tested, robust way to pursue truth about almost everything imaginable – so there can not simply be a better way for upfront scoping.

 

This time, though, we don’t go as fast as we did in Rapid Scoping phase – we do it really meticulously. But before getting into potential challenges in the activity, let’s see what questions we need to actually ask in place of just What or How or .

 

What:    List of Business Objects,

Question: Which entities constitute the business?

How:     List of Business Processes:

Question: What does the business call various collections of activities that transform the enterprise inputs into enterprise services?

Where:  List of Business Locations

Question: Where ( which geographical entities ) is business rolled out ?

Who:     List of Organizations

Question: In which organizations does the responsibility for execution of processes and services rest?

When:   List of Events

Question: What are the events to which the enterprise responds ( upon their occurrence or relative to time) ?

Why:     List of Business Goals

Question: Where can we find pointers or references to business goals, objectives, strategies, or critical success factors significant to the enterprise relative to motivation?

Posted in IT Architecture | Tagged: , , , , , | Leave a Comment »