What is Enterprise Information Integration

Enterprise Information Integration, otherwise known by the acronym EII, refers to the ability of software to access data from multiple and perhaps incompatible sources using a single interface, in most cases custom written for the organizational needs, and which may include more display methods than a simple database query allows, for example graphical representation of data.

The goal of enterprise information integration is to present the data homogeneously such that the user is not aware of the actual sources of the information, and thru presenting the data in this manner make for more efficient use of reports instead of wasted time whilst the user manipulates the data for the task at hand.

By extracting information from multiple sources and bringing it all together into a single interface it is possible to offer more information than a standard report, for example extracting sales data from accounts and merging this with data from the customer relationship manager and field testing from marketing to produce accurate sales by demographic.

What is Enterprise Information Integration Enterprise Information Integration

Ever since private enterprise came into being with the first human settlements, business people and their modern enterprise counterparts have been desperate for more information, a situation that computing power has only partially satisfied, although modern search result algorithms are proving that information integration isn’t impossible.

Problems arising from the quest for enterprise information integration mostly stem from older or proprietary software that is by nature not intended to be used outside of it’s original interface. Older software in particular may not even provide methods of accessing the date from an API resulting in interface rewrites or custom interfaces that understand the original database structure.

Recent attempts to create an open framework for information integration on the world wide web have created the XML protocol, a vastly improved form over the original HTML standard, and which is now finding it’s way into the enterprise thru recent software that uses XML formatted documents to store data from spreadsheets, databases, and documents allowing them to be accessible regardless of operating system or platform.

Most efforts at enterprise information integration in major corporates have to date been conducted in piecemeal style, most often instituted at local or departmental level according to a specific need, and without regard for standards that would allow the wider userbase to benefit. Sadly this has the effect of making it even more of a challenge when EII is implemented corporation wide because additional data sources have to be supported.

A further impediment to EII comes from the lack of compatibility between structured (database/spreadsheet) and unstructured (document/slideshow) data, and the inability of staff to find the information anyway once it has been brought together. Corporate intranets go someway to addressing enterprise information integration, and modern search algorithms improve this, nevertheless, getting structured data into an unstructured document such as marketing brief or business plan is still only possible with human intervention or the creation of templates.

Until enterprise information integration strategies and software improve, many corporations will continue to rely on the services of data resellers such as market research firms, but the day is quickly approaching when this may no longer be necessary, at least that’s the promise of EII.

What is an OLAP Database?

OLAP stands for online analytical processing and is the process of easily acquiring selected data from a database.  It also functions by displaying the information in different perspective depending on the user.  If, for example, a user wishes to see a certain product and where it can be bought, OLAP will then generate different reports showing exactly what the user is looking for.  This is due to the multidimensional capability of the OLAP, which, unlike the relational database, uses only two-dimensional interface.  The beauty of a multidimensional approach is that it sees one criterion as a separate entity, therefore, producing separate reports.  OLAP has the capability of finding each item and display them in separate forms.

Functions of OLAP Database

OLAP can be used for searching hidden relationship between several data, also known as data mining.  The database for OLAP isn’t quite large as the storage for databases since only some of the transactions done in the database can be considered for analysis.  Data can be brought from another type of database such as the relational database to build a database that is multidimensional for OLAP.

OLAP is commonly used in generating business reports for sales, marketing and management reports, budget, and financial forecast and so on.  Databases that support OLAP use a multidimensional approach that authorizes difficult queries executed in a fast rate.  The multidimensional database has aspects that are taken from the navigational database and hierarchical database, making the database faster than the relational database in executing an action or query.

OLAP Cube

The center of any OLAP system is the OLAP cube, which represents facts in a numeric representation called measures, which are then categorized as dimensions.  This type of schema was inspired by the snowflake schema, which is commonly used in relational database.  The measures came from the recorded data in tables holding facts, while the dimensions came from the tables for dimensions.  Database users can think of measures having their own labels, while the dimension is the one that defines the measure.

What is an OLAP Database? OLAP Cube

OLAP Aggregations

OLAP is known to provide a result in less than a second, and this is due to the use of aggregations.  The aggregation stemmed from the table containing the facts wherein alteration in some specified dimension granularity can aggregate the data in an upward motion.  The aggregations can be numerous, depending on the number of possible dimension granularities.  Due to the number of possible combination of dimension granulation, there will be a remainder of these combinations ready to be used depending on the demand of the user.  This will also result to a view selection problem.  In order to minimize this type of problem, the view selection can be curbed by determining the size of the aggregates and the time it takes before it can be updated.  Either you can choose one or use them both to constrain your view selection.  The function of the view selection is to simply lessen the time it takes to get a result from OLAP.

There are different types of OLAP in the market today.  These are the multidimensional, relational, and hybrid.

What is an Informix Database?

There are many database management systems out there today.  Different companies offer different versions of their products to provide different solutions.  One of the leading products today in database systems is Informix.  It is a group of relational database management system products by IBM.

Informix Database

Informix is IBM’s flagship product for integrated solutions and online transaction processing.  At first, Informix stood on its own, with its own company with the same name, but later on, it was acquired by IBM.  Its major competitor is the Oracle database, and they go head to head in the database market nowadays.  Although Informix proved to have technical success, failures in management and marketing led it to be taken over by IBM.

What is an Informix Database? Informix Database

Brief History

The original developers of the Informix database were Roger Sippl and Laura King.  They founded the Relational Database Systems in 1980.  When they shifted their attention to relational database systems, they released their own product in 1981 called Informix (INFORMation on unIX).  It included their Informer language, featured an ACE report writer, and PERFORM screen tool.  ACE had the ability to extract data for easy reading, while PERFROM enabled the user to query and edit it in the database.  In 1985, they introduced a new version with an SQL-based query engine called Informix-SQL version 1.10.  It had variants of both ACE and PERFORM.  The database access code was separated from client into an engine process which set the stage for client server computing.  Through the 1980s, Informix was a small player in the face of other big competitors such as the Oracle database.  But when SQL and Unix became popular in the middle of the 1980s, RDS became big enough for an IPO and changed its name to Informix Software.  Other releases of the flagship product followed with developments such as multi-user performance, transaction support, triggers, backups, and stored procedures.  In 1988, the company acquired Innovative Software which made WingZ, a spreadsheet program for Apple Macintosh.  Unfortunately, due to lack of development, WingZ was sold in 1995.  The company then created Dynamic Scalable Architecture with Sequent Computer Systems in 1994, which supported both horizontal ad vertical parallelism.  This feature was included in version 6, then the next version was known as Informix Dynamic Server.  In 1995, the company acquired Illustra and focused on object relational database.  Version 8 and 9 offered a built-in O-R support.  The company suffered problems from 1996 to 1997 due to marketing and management failures.  Product releases were delayed, and revenues fell short of the company’s expectations.  The following years, the company shifted their focus on acquiring other companies.  IBM took over Informix in 2000 and bought its database technology, brand and future plans.

Currently IBM is concentrating on IBM and DB2 which shared technologies with each other.  The version 10 is named Informix Dynamic Server.  Version 11.5 has enhanced application development features and high availability.  Although Informix is now acquired by IBM, it still goes head to head with Oracle as one of the database management system in the market.

What is Data Integration?

Data integration involves taking data from two or more databases, sometimes within the same company, and producing meaningful reports from the merged data. Data integration can be thought of as one of the holy grails of IT, and at the same time a major headache for most IT professionals.

The vast majority of databases used in corporations are proprietary in nature, designed by the supplier to suit the needs of their querying software, not for integration with other data systems, leading to difficulties creating a report that combines two sets of data, for example sales data from a spreadsheet or customer relationship database, with banking reconciliation data from the accounts software.

On the face of things the example above sounds simple enough that manually comparing the two disparate reports would give the complete picture, but the example is too simplistic, there could be many other reports that are too complicated for a quick scan of the separate results, and this is where data integration becomes the solution.

What is Data Integration? digital globe

IT specialists with database programming skills are required, who write specific applications that allow for querying multiple source data sets, often this will be a patch to existing software, but in many cases could be a completely separate piece of software tailor written for the end user.

More often these days IT departments are requiring compatibility between databases before implementing new software across the corporation, adding to the license expenses and requiring a long term commitment to maintaining the custom software thru version changes for all the source databases.

In the online world data integration is being driven by the open source community who have developed OpenID allowing members of social networks to login to many proprietary networks using a single login and password, and carry their personal data to that new network.

Computer software that does use data integration to produce new and unique reports from two or more databases are the type used in keyword analysis in the search engine optimization market where specialist software or websites query databases from the major online advertising agencies.

The corporate world is no different, some of the biggest Fortune 500 companies were created thru mergers and acquisitions of companies using different software and databases, and data integration is essential to producing accurate and reliable financial reports.

One of the most popular methods of making date integration happen with large databases is to create a new database from running queries on the original databases, and then writing specialist front-end software for the new database which queries only the new database to produce the reports needed.

This technique is known as data warehousing and can be a simpler and more cost effective approach because regular exports from the original database can be scheduled, and then imported into the warehouse. With proprietary software this approach reduces licensing issues and possible corruption of databases.

What is a Stored Procedure?

Stored procedure refers to the routine available when using a relational database system.  Stored procedure is also called Proc, Sproc, SP or StoPro.  They are usually available within a data dictionary.

Most stored procedures are used for data validation.  Data validation is integrated within a database.  Stored procedure is also used to unite as well as integrate logic in implementing applications.  For more complex procedures that require SQL statements, the use of stored procedures will definitely be an advantage.  All applications under stored procedure will just need to call the formula only.  Stored procedure is compared to the User-defined function.  Unlike the User-defined function, which can be used by using other expression in the SQL statement, stored procedure only uses the CALL or the EXECUTE statement.

Stored procedure has several unique features.  It can return result sets as well as receive and modify variables, but it depends on how the variable is affirmed.  Other use of stored procedures includes transaction management.  Users can manipulate transaction within a database.  Stored procedure can be derived from a condition handler or a database trigger.  For instance, a user can use a stored procedure to modify the data in a specific table.  Thus, the code in a procedure will be performed.  With the use of stored procedures, users can keep track of significant data and prevent errors or failures.

What is a Stored Procedure? Stored Procedure Test

How to Implement a Stored Procedure

 

Stored procedure can be implemented in different circumstances.  Using stored procedure varies from one system to another.  Depending on the system used, stored procedure can be implemented in several programming languages.  Users can use the SQL statement, C++, C and Java.  On the other hand, stored procedures not using SQL programming languages will probably not execute SQL statements.  Because of stored procedures, SQL statements became a necessity, especially for SQL applications such as Microsoft SQL server that allows transact-SQL as a form of stored procedure and Oracle which uses PL/SQL and PostgreSQL.

Advantages of Stored Procedures

 

Many users would usually ask about the benefits of stored procedures.  Here are the main reasons for using stored procedures in certain applications.

1.         Since stored procedures are managed by SQL programming languages, there is a precompiled execution, which means that the SQL server compiles the data and allows the user to execute the plan of the stored procedure repeatedly.  This allows higher performance on the part of the application used.

2.         If you’re experiencing server or client traffic, then using stored procedures is the best way to control traffic.  Stored procedures reduce the incidence of long SQL queries, thus, keeping all server and clients organized and intact.

3.         Stored procedures can be utilized by several users and servers.  If a user utilized a stored procedure in an organized manner, the user will definitely have less time manipulating and storing data.

With the use of a stored procedure, a user can ensure that data are stored properly and securely.  Each stored procedure has a secure code which allows only registered users to access on significant data.

What is a Data Mart?

Do you want to organize your own business?  To make your business up-to-date and successful, you need to gather significant tools to improve your data storage as well as database organization.  The most brilliant tool you can use when establishing your business is the data mart.  In this article, we will know more about data mart and its significance to businesses.

All about Data Mart

 

Data mart is a division of the organizational data store, which usually is designed for a specific purpose.  A data mart is usually distributed to maintain business needs.  Data mart is generally an analytical data store that is designed to meet the needs of specific business communities.  Most data marts are derivatives of a data house ware.  But oftentimes, data house ware is developed from the combination of several data marts.

What is a Data Mart? data mart

Oftentimes, writers would relate data mart and data house ware, but these two terms have different uses.  Data mart generates from the notion of the user’s needs, while data house ware generates from the analysis of the available data as well as how it can be collected.  In a more constructed manner, data mart is defined as a storehouse of gathered data from various data sources.  A data mart is designed to serve a particular community.  On the other hand, a data house ware is a system of gathered data, which is distributed to other data stores.  In short, a data house ware seems to become fragment of ideas, while a data mart connects to an immediate purpose.  Data mart gathers its information from enterprises (wide database or data warehouse).  By using the data mart, users will have the opportunity to meet their specific demands in terms of presentation, up-to-date amenities as well as content and analysis.

In a certain corporation, several data marts are available to strictly follow its purpose or design.  For instance, a single data mart can become dependent or related to other data marts.  In a particular department, there can only be one data source and one owner of the data mart, including its hardware and software.  In this manner, this certain department can independently manipulate and store its data without affecting the other data marts.

Why Create a Data Mart?

 

In general, data mart is designed to aid in business.  Creating a data mart will help in the easy access of frequently needed information from the database.  When a certain member of a community requires essential information for a project proposal, the member can easily dig up the files with the use of a data mart.  Creating a data mart can also make a clear view of the important data for a group of users.  Another reason for creating a data mart is its accessibility to both apprentices and experts.  A data mart can improve the response of end-users.  Moreover, a data mart is cost-friendly as well as user-friendly.  Potential users of a data mart will be defined appropriately.

With the use of a data mart, every business will definitely be in order.  Thus, flow of data from every department will be properly disseminated.

Data Integration Tools

Data integration is the process by which data and information from disparate and often incompatible  sources is brought together by transforming the data into a single unified view, usually within a new application that offers additional reporting designed for users outside of the usual user base of the source data. Data integration also goes under the name Enterprise Information Integration (EII).

The process of data integration is complex and over time has evolved into very sophisticated tools available as standalone software from third party vendors, or as modules within programming languages that natively query the original database. Two broad methods of achieving data integration have been developed, data warehousing and mediated schema, each with their own advantages and disadvantages that should be considered when evaluating available tools.

In data warehousing, the data from multiple sources is extracted, transformed, and loaded (ETL) into a single new database which can be queried by the end user and reporting configured to their specific need. Date warehousing has the advantage that query time is reduced, but suffers from the disadvantage that data in the warehouse may be out of date, and in some applications even a few seconds might render the query irrelevant.

Data Integration Tools Data Integration

Mediate Schema is a more modern approach where the original source data is accessed for every query, and no single data warehouse ever exists. Data integration tools that use this approach setup multiple wrappers around the source data which are queried in real time and the results fed to a virtual database (the EDI client) where data is transformed for one time use and presented to the user. In some cases, caching may be employed to reduce query times for non time-sensitive data.

Which date integration tool to use and rollout across the organization is unfortunately not a simple decision, with each vendor having their own strengths and expertise, and at the same time numerous weaknesses. A common evaluation method involves matching source data complexity with vendor expertise, and ranking vendors according to relative ease and cost effectiveness of customizing unsupported formats and platforms.

In selecting a data integration vendor, IT administrators should be aware that the industry is undergoing a series of mergers and takeovers as vendors try to position themselves for maximum gain heading in the future. Several of the larger and more established players regularly hunt for smaller niche vendors to takeover which could result in unplanned licensing issues for organizations as well as increased costs of ownership.

The data integration tools market is being driven by customer demand more than ever before as the benefits of EII become more accepted outside of IT. Significantly, cost control is being seen as one of the reasons for the increasing acceptability of data integration tools, notably because organizations are no longer willing to pay for staff time in extracting, merging, and reporting data from multiple sources.

Tools for data integration continue to develop and the next decade promises  to see the traditional market for niche products and custom solutions replaced with general tools capable of interfacing with almost every data format.

Data Integration Software Options

As business becomes increasingly dependent on electronic data the demand to integrate multiplesources of data into a single application or report has driven market growth for software that can natively work with proprietary and open source data. Known as data integration, this software accesses data stored in incompatible formats to produce new data and reports.Combining data from across the corporation, some of which can be decades old, with other data contained in databases, emails, documents, spreadsheets, or applications requires specialized solutions capable of interfacing with the data preferably natively so that unintended alterations aren’t made to the date during conversion.

The bewildering number of data formats means writing a custom translator is limited to corporations with large budgets and in-house developed databases, although even in these situations it is often still economic to license third party data integration translator software. The best third party integration software understands major formats such as SAP, DB2, Oracle etc.

Due to the complexity of data integration products most are proprietary in nature with distinct benefits and their own disadvantages. Typically, each vendor will provide a basic system for mapping data, and each supported format requiring the installation of extensions or plugins. Whilst many corporate databases and other information are also proprietary, the decision to license further proprietary software is contentious.

Data Integration Software Options Data Integration Software Option

Several of the more popular formats for communicating data are open source such as XML, and almost all data integration software will export using this format if required, thus allowing applications such as word processors to access and manipulate the data, a major use of data integration at senior management level.

On road salespeople, financial advisors, delivery personnel, and merchandisers using portable data assistants (PDA) and laptop computers are not always capable to connecting to the corporate servers resulting in data that needs to be synchronized once back in the office. Data integration software is an ideal solution allowing for existing software installed on the laptop or PDA to be used instead of paying for multiple licenses of new software.

Wifi and mobile Internet may allow the use of a web browser to access corporate data, which also allows the data to be accessed by customers, and is another use for data integration software.

Further options for data integration software include the growing range of open source products that can be downloaded and assessed without charge, offering the promise of obtaining software that can be customized in-house yet also offering community assistance that is not limited to the original vendor.

Choosing the correct data integration software might be an involved process, and decisions shouldn’t be taken lightly if any form of data cleansing is to be undertaken and the source data consigned to permanent backup such as when older software are being retired.

Retiring older software without retiring or corrupting the data is possible, and of course may give the information a new lease of life allowing for more detailed and longer term customer profiling. Cleansing is the process of converting old data into new formats that also clears redundancies, and reconciles the information against other sources of data with common indexes.

Data Integration Solutions

When data is scattered across multiple databases with different file formats, most IT administrators will immediately think of data integration solutions to solve corporate data mining operations. Data integration is the process of bringing together all of the organizations data into a single view that allows new manipulations and reporting previously not available.

Combinations of flat file or relational databases needing to be merged with data stored in spreadsheets, word processing files, letters or emails, and calendars never used to be possible without expending significant person time manually collating the information, sometimes many times over if the data aged and needed to be recompiled for monthly and quarterly reports.

Data integration software natively understands multiple data sources and can transform data using standard database tools, sometimes by directly accessing the source file, at other times thru the use of wrapper software that queries the source and then converts the data for use in other applications. Most data integration solutions will require extensive customizing owing to complexity of the task.

Data Integration Solutions Data Integration1

Most corporate data from databases and other software applications are stored in proprietary file formats that are rarely compatible with standards based formats such as XML. Lacking interoperability they are difficult to deploy, and create significant challenges for the organization in their efforts to integrate data and move towards a standards-based future.

Investing a great deal of time and cost in switching enterprise software when in fact existing applications already contain all of the data needed can be better achieved by adopting a data integration solution. Solutions that offer ease of querying thru simpler user interfaces and require less training are gaining in popularity.

Currently the most popular data integration products require teams of programmers, costly server  installations, and often a great deal of ongoing maintenance. Enterprise information can be very complex and require a lot of storage, resulting in huge queries. Integration products therefore need to scale to the enterprise level consistently and with relative ease.

As the market for data integration software matures visual tools are being developed by many of the smaller specialist vendors at prices far below those of the established market leaders. Visual integration tools allow new solutions without the overhead of a team of experts. Once the software has been installed a small group of trained personnel are able to drag and drop data fields and create new relationships with ease.

The advantages to the corporation are many, costs may be reduced, training can be simplified, and new reports designed on as needed basis allowing greater use of the raw data across an ever larger range of departments. Previously the preserve of the strategic planning and marketing departments, data integration solutions are now finding greater application within manufacturing, logistics, legal, and many other departments not normally associated with accounting and CRM data.

With the move to standards-based file formats such as XML and the significant use of Linux powered servers and database applications open source solutions for data integration are entering the market, offering attractive new possibilities.