Integration Methodologies: Melting Frontiers Or Age of Divergence ?
Posted by snair007 on July 18, 2008
Part1 : The Integration Big League
Integration is pretty much what IT architecture is all about. Once we get rid of the ‘fat’ from this area and get to the core of affairs, all that matters is the way components ( data,application,technological ) are fastened together. No wonder Enterprise IT Architecture and Systems Integration were used interchangeably up until a close past.
But then there are a huge lot of methods out there, the purpose of which would rather seem to confuse than to assist. But I am sure there can be no argument as to which of them are most significant – Enterprise Application Integration (EAI) and Enterprise Data Integration (EDI). As a term, EAI has proven to be monumentally more significant than EDI, but when it comes to the activities that either of them deals with, the disparity vaporizes.
In an Enterprise Architecture evolution context, where would they stand tomorrow as two approaches of integration? In a time and age when ‘components’ are made with integration in mind, how relevant (and possible) is it to retain their distinct identity? Are they heading for convergence? If so, which of the two would potentially supercede another (if at all)? Which other integration methods would get encapsulated in either of these two?
Loads of great questions, and let’s not wait to hash them out.
Egg is not the mother of Chicken, or is it?
All serious Enterprise Architecture practicians might’ve noticed how EAI and EDI are often used in affiliation with one another, if not in place of one another. But if we try to ignore this phenomenon as yet another industry misnomer, we need to be careful. I have come across many research papers and books making contradicting remarks when EDI and EAI are referred in context with each other. Some say EDI activities and tools are a part of EAI framework; others speak something quite the opposite. Still others argue that there is not much difference anyway.
And they can all be right, too!
For the time, though, let me jot down my understanding of the most accepted distinctionsbetween EAI and EDI, before dissecting each of them too much:
· EDI focuses on unifying enterprise data views; EAI enables unifying business processes by blending applications’ capabilities.
· EDI is data-centric; EAI is ‘message’ centric.
· EDI deals with high-volume data movements; data movements in EAI is normally limited to real- time/near real-time data ‘propagation’
· The technologies used for typical EDI operations are different from those of EAI
· EAI is normally defined to be integration of data with application or application with application; EDI is to do with data to data integration.
· In general, data extraction is a ‘pull’ activity in EDI and in EAI; it is a ‘push’ one.
· Data Integration may not radically alter the systems architecture of an enterprise, but Applications Integration does.
· At one level down, the frequently talked about sub- types of EDI (EII,MDM,ETL etc) are significantly different from typical activities surrounding EAI
· EAI can start right from the component development (by ensuring compatibility), whereas EDI is almost always an implementation/interfacing activity.
· There are more standards, guidelines and protocols to be followed to realize EAI than in the case of EDI. Many EDI efforts can be ad-hoc and tactical.
Sigh! There are quite a few, right?
But as I posited, these ‘differences’ haven’t seemed to restrain many experts from bracketing either of them under the other. Some believe EAI is a part of EDI just like EII,ETL and Data Federation. There are other schools of though that sees no reason why EDI is not a part of EAI framework, which according to them, is immense and flexible enough to accommodate that and much more.
These chickens and eggs!
Let the argument go on; we shall go back to tackle some more important questions by taking a nigher look at each of them.
Interestingly (and quite thankfully), there’s been a clear pattern in the way EAI technologies have evolved overtime. It is fair to say that the definition of EAI has also evolved with them. But then, arguments in favor of placing EDI under the EAI umbralla have been there in all elementary definitions of EAI. Here is how Gartner group defines EAI:
unrestricted sharing of data and business processes among any connected application or data sources in the enterprise.” ( a clear indication that we can regard EDI as a subset of operations and framework that EAI offers.)
But the moment we see something like “business process” in its definition, EAI gets much more than a set of application or data components technically welded together. Linking business processes demands a well- orchestrated, business-centric approach.In other words, EAI is the technical consolidation of application silos in accordance with a set of business rules,to satisfy a set of business requirements.
An interesting aspect of EAI is that it can give rise to a set of new functionalities that the component applications wouldn’t provide in isolation. But mind you – these ‘generated functionalities’ are not‘technical accidents’ of EAI, but a result of well thought-out and estimated efforts.But both the expectations as well as the methods around this area have been varying over the years.
Wing and Venky (Enterprise Architecture and Integration,2007) give an interesting picture of how the focus of applications integration efforts have shifted over time – from one of technical to that of business.
They accurately points out that
“The perception of integration has therefore shifted over time from tactical, low-business-value technical plumbing to a strategic and high-value enabler for achieving business competitiveness.”
And the roadmap they present has a picture of EAI transmuting from “technical plumbing” to business-enabling frameworks generation.The focus areas making their appearances in the chronological order are systems integration, real-time integration, ERP Integration, Web Integration, Supply chain management, e-business integration,CRM, B2B Commerce and finally – collaborative commerce.
So, the future of integration is decided by business trends and paradigms rather than technological advances in data/message propagation. What does it mean to the question of addressing EAI and EDI as distinct integration streams? Answer seems quite obvious at first glance, now that we have seen the diminishing importance of integration methodologies in a rather business-centric integration environment. But is that truly the case?
To analyse that further, we must have to take a look at the types of integrations falling under each of EAI and EDI.
Let me encapsulate major EDI and EAI categories in two tables :
|EDI Type||What is it ?|
Enterprise Information Integration
Master Data Management
Extract, Transform and Load
Real Time ETL
Enterprise Data Replication
Enterprise Content Management
|7||EAI||(We will see this in detail below)|
|8||CDI||Customer Data Integration|
Change Data Capture
|EAI Type||What is it ?|
(We have seen this in detail)
Functional Integration (FI)
Process Integration (BPI+)
Message Oriented Middleware (MOM): integration through messages exchanges
Configurations : message queuing and message passing.
Examples : IBM’s MQSeries and Talarian’s Smart Sockets.
Distributed object technology: Integration through application of object- oriented concepts t o middleware.
Configuration : Software components are made into objects which is then can be accessed by other applications across a network through the object interfaces.
Example : CORBA, MS COM+, J2EE,DCOM,RMI
Transaction processing monitors (TPMs): Integration through preserving integrity of distributed information resources (databases, files, MQs).
Configuration :To impliment distributed transaction processing support concepts like two-phase commit.
Example : BEA Tuxedo,web services
Presentation Integration (PI)
IUI (User Interface Integration)
Improved Functional Integration
*black box integration approach hides internal complexities of the application or database. White box approach does it the other way around.
Next : Part2 : Convergence and Convenience