Problem Definition
When engaging in an integration effort of any kind, one of the key problems we face is determining the impact of changes and figuring out where the right data is coming from. Impact analysis when done manually is a difficult, time consuming, and painful task. This usually involves trying to figure out the meaning of data, from systems that were previously developed by people that may or may not be available to answer questions about the original design.
More often than not we are faced with undocumented processes, data points, and calculations. In many cases the situation was compounded by efforts to simply build a new bolt on to existing systems. This causes redundancy in effort and over time adds to the evolving problem. The most common solution at some point is to pronounce the old system a throw away and start from scratch.
Point Solutions
Metadata lineage is available within many ETL tools, I call these point solutions because you need to be using said tool across the board. In reality, rarely do we see companies using only one solution or methodology for integrating all their data. Many use 3rd party schedulers, scripting languages, custom processes, and even a combination of different commercial tools.
ETL Specific Challenges
When dealing with ETL processes, we encounter real time consuming manual intervention challenges such as:
- Stitching of metadata between tool-sets.
- Connecting the dots where parameterized processes exist.
- Language specific rules processing.
One of the most common reasons I see people give up on metadata management is that they cannot find the “Silver Bullet” tool that will reverse engineer every tool known; including language interpretation. What about the metadata or knowledge that is in peoples heads?
InfoLibrarian’s Unique Solution
At InfoLibrarian, not only do we provide tools to capture metadata from databases, processes, code, and ETL tools; we have developed powerful search algorithms to perform impact analysis from the metadata that exists in your company today to help you find the answers you need. It’s the combination of lineage (where available) and impact analysis that provides the full picture.
We provide our solutions from the stand alone desktop requirement all the way up to enterprise implementations, allowing you to get started today.
InfoLibrarian is the worlds most powerful metadata impact analysis portal!
