Metadata Management Framework

Metadata Management Software Framework

Now that we covered the basics to establishing a program within your organization, let’s have a look at software tools to help drive our initiatives forward.

Remember Metadata Management is a Dependency of:

• Data Quality.
• Data Governance.
• Master Data.
• Big Data.
• Data management best practices.

The human brain metadata sucker device is not terribly practical or realistic. Reverse engineering 3rd party source tools will always require brute force, and there is no magic bullet when it comes to getting our arms around all our IT systems. In fact ETL is part of the problem not the solution.

The key here is what are you trying to do, what do you need to do it. Technologically, what we need to succeed already exists. The trick is identifying the starting point and the problem to be solved so that we can begin to do the work of managing our metadata.

Here we will look at a typical metadata framework technology required for any metadata management program.

• We need to put it somewhere ( Metadata Repository(s)).
• We need to make it searchable (Search engine).
• Provides a way to manage business semantics, such as business glossaries, synonyms, business terms, rules, custom types,  and taxonomies.
• Make is useable to integration and consume, share, and collaborate on its changes (Metadata Portal and Web Services components).
• We must be able to manage it ( Stewardship and Governance management tools).

Software Frameworks

In computer programming, a software framework is an abstraction in which common code providing generic functionality can be selectively overridden or specialized by user code; thus providing specific functionality.

Frameworks are a special case of Software Libraries in that they are reusable abstractions of code wrapped in a well-defined API, yet they contain some key distinguishing features that separate them from normal libraries.

Software frameworks have these distinguishing features that separate them from libraries or normal user applications:

• Inversion Control- In a framework, unlike in libraries or normal user applications. The overall program’s flow control is not dictated by the caller, but by the framework.
• Default behavior – A framework has a default behavior. This default behavior must actually be some useful behavior.
• Extensibility – A framework can be extended by the user usually by selective overriding, or specialized by user code providing specific functionality.

Metadata Management Framework:

• A metadata management framework provides the building blocks to help you integrate and manage your metadata.
• A metadata management framework is a set of software tools that captures, manages. and publishes rich metadata. Especially business metadata.
• A framework provides all that is needed to plug into the technical metadata contained in your environment and make it searchable.
• A framework also typically provides a repository and portal which is similar to Google and Wiki for searching, data entry, impact analysis, and collaboration.
• Existing applications can query the metadata to enrich functionality via the framework.
• A good framework can eliminate the need to develop a complete custom software application.
• A framework is the glue that connects the dots within your environment.

Metadata Management Framework Essential Components

• A repository to store the metadata. Not all of it, just the important stuff.
• Powerful tools to discover as much metadata as possible automatically.
• Flexibility of adapters to accommodate changes over time.
• Tools to map metadata across sources.
• Scalable options to accommodate phased efforts.
• Robust search engine technology bundled.
• Ability to perform impact analysis.
• Simple automated maintenance and scalability.
• Should allow you to add custom properties and types.
• Provide a historical perspective.
• Provide rich business metadata, terms, rules, and taxonomy capabilities.
• Business glossaries and semantics capabilities.
• Should not retain your metadata in an overly complex difficult to query super meta-model, but make it available to query easily.

Buy vs. Build

Build your own Metadata Framework

•   Cost in terms of labor, resources, and time to market testing and debugging.
•   Do we have the resources to develop a solution?
•   Do we have skill set?
•   Most homegrown efforts fail!
•   Developing a framework in-house is a very difficult task.

Buy a COTS (commercial off the shelf solution)

•  Cost – Can we get budget?
•  What is our budget?
•  Not many vendors to choose from, evaluation process should be easy!
•  Do we need customization. (Do we have a finely defined scope, which will allow us to use a product off the shelf).
•  Can save a lot of time and money.
•  A Software Framework provides the modules + adaptability and customizability.

Summary

Your success is dependent on identifying a business problem, finding a solution to that problem, and selling that solution to management. Followed up by a well defined and managed project leveraging a metadata integration framework.