Data integration involves taking data from two or more databases, sometimes within the same company, and producing meaningful reports from the merged data. Data integration can be thought of as one of the holy grails of IT, and at the same time a major headache for most IT professionals.
The vast majority of databases used in corporations are proprietary in nature, designed by the supplier to suit the needs of their querying software, not for integration with other data systems, leading to difficulties creating a report that combines two sets of data, for example sales data from a spreadsheet or customer relationship database, with banking reconciliation data from the accounts software.
On the face of things the example above sounds simple enough that manually comparing the two disparate reports would give the complete picture, but the example is too simplistic, there could be many other reports that are too complicated for a quick scan of the separate results, and this is where data integration becomes the solution.
IT specialists with database programming skills are required, who write specific applications that allow for querying multiple source data sets, often this will be a patch to existing software, but in many cases could be a completely separate piece of software tailor written for the end user.
More often these days IT departments are requiring compatibility between databases before implementing new software across the corporation, adding to the license expenses and requiring a long term commitment to maintaining the custom software thru version changes for all the source databases.
In the online world data integration is being driven by the open source community who have developed OpenID allowing members of social networks to login to many proprietary networks using a single login and password, and carry their personal data to that new network.
Computer software that does use data integration to produce new and unique reports from two or more databases are the type used in keyword analysis in the search engine optimization market where specialist software or websites query databases from the major online advertising agencies.
The corporate world is no different, some of the biggest Fortune 500 companies were created thru mergers and acquisitions of companies using different software and databases, and data integration is essential to producing accurate and reliable financial reports.
One of the most popular methods of making date integration happen with large databases is to create a new database from running queries on the original databases, and then writing specialist front-end software for the new database which queries only the new database to produce the reports needed.
This technique is known as data warehousing and can be a simpler and more cost effective approach because regular exports from the original database can be scheduled, and then imported into the warehouse. With proprietary software this approach reduces licensing issues and possible corruption of databases.