Open IGC: Uploading New Assets!

This entry is one of many in a series that describes the InfoSphere Open IGC API, which allows you to define your own objects for information governance using InfoSphere Information Server and the Information Governance Catalog.

Previous post in this series:
Open IGC: Defining a new bundle!

Original post in this series:
Open IGC is here!

At this point you have your bundle defined, you can see your objects and their icons in the “browse assets” page, and the detailed properties of your new objects are visible within the Query Tool. Congratulations! Now you are ready to start loading new assets, or new “instances” of the objects that you have modeled with your bundle design.

New assets are added using XML and another REST call, this time a special POST for the upload of new assets. The documentation (see URL in the prior blog entry) includes example xml documents and the xsd, but let’s look more closely at one here.

Our Messaging bundle describes a simple hierarchy that has Queue Managers at the highest level, and then Queues. Queue Managers also have “Listeners”. These are the major objects in the bundle. Initially I am just defining new Queue Managers and their Queues. To keep my sanity while initially learning and playing with the API, I am creating a single xml document for each Queue Manager. This is not a requirement, but keeping the documents small, and focused on one higher level object in the hierarchy will help you understand the structure of the xml and speed up your learning curve. Each whole “xml document” or “xml string” is what you will be passing in a single http POST when performing the actual upload.

Here is a list of an initial set of these xml documents.

xmlDocsForPublishing

To stay organized, I keep them in a folder structure, per bundle type, that has a subdirectory for bundle details (see prior blog entry), a subdirectory for publishing new assets, and a subdirectory for publishing “flows” (for lineage…a future post). Ultimately, many of these xml documents will be built “on the fly” in your programs that craft the interface between Information Governance Catalog and whatever you are modeling with your bundle. However, the simplest way to learn the Open IGC is by using static xml documents. Depending on your use case, some of you may only have a few objects to govern, and might always use this file based approach.

highLevelBundleFolderStructure

Let’s take a closer look at one of the publishing xml documents:

publishXML

The elements and attributes above are well documented in the examples, so I won’t go into excruciating detail, but want to point out several items.

1. Your custom properties (light green box). Note how their names each begin with a dollar sign. This uniquely identifies them as “yours”. Every object gets name, short_description, and long_description. Think of these as “free” in your bundle. You didn’t need to define them in the bundle — they are just “there”. As such, they don’t require the dollar sign prefix.

2. The value of the repr attribute in the object header, and the string used for the “value” attribute for “name” immediately below it (purple box) must be identical! This is for internal reasons. It is a requirement of the API. You will get a nice error if they are not identical, so am pointing it out here to save you the trouble.

3. The ID value (red boxes) is a unique identifier for the asset within this xml document. It is just an internal reference that is used throughout this particular xml document (it doesn’t have any overall system significance). It is critical for establishing the hierarchy of your objects and will be even more important when you learn about the “flow” xml for lineage.

4. The “reference” element (blue box) is what helps establish the hierarchy, identifying the parent asset (if applicable). Note the use of “ID”.

Another very important part of the publishing xml is the “importAction” element at the bottom of your xml document. This is an important property that controls the behavior of the API when managing a complex hierarchy. This can be a difficult concept to understand, but I will do my best to explain it here.

partialAssetID

The element importAction has two attributes, partialAssetIDs and completeAssetIDs. These attributes contain a set of comma delimited IDs from up above in the xml document. They describe whether a particular asset, in this xml document, is being uploaded with ALL of its children, or only “some”. If the parent ID is listed in “completeAssetIDs”, then the parent and its collection of child objects is considered complete; any pre-existing child instances “not mentioned” in this new xml document will be blown away. Mentioned child instances, if pre-existing, will have current properties edited (if desired) and retain all governance references (stewardship, term assignments, etc.). If you want to preserve the pre-existing children for a particular parent, place that parent ID in “partialAssetIDs”.

Once you have built your xml document, and have checked it for well-formed-ness (at the very least, make sure you can open it in your browser as a well-formed and recognized xml document), you are ready to upload it to IGC. Go to the igc-rest-explorer page for the Open IGC API and find the bundle “POST” invocation for publishing assets:

publishAssets

Open your xml document in a regular editor and copy/paste the entire xml string into the available property (red box in the screen shot above) and then click “Try it out!”

If there are any errors, you will receive them here directly, and if all is “ok”, you will receive a clean 200 response code, and your assets will have been loaded.

successReturnCode

At this point, you can immediately return to the Information Governance Catalog and view your new assets!

NewAssets

Browse them by returning to the main “Information Assets…Browse All” pull down where you found the icons for your bundle, and then look around….see if your child assets are also loaded, and how they are displayed “within” the parent! Try doing a Query. Edit one of your new assets and make adjustments to one of the properties!

Your assets are now being governed…they can be assigned Terms and Labels, belong to Collections, become the responsibility of a Steward — just about everything that you can do within Information Governance Catalog is now available for your new objects! In the next post we will look at how you can apply your own custom flow definitions for data lineage that includes your new object instances.

Next post in this series:
Defining Lineage Flows (Part 1)

–ernie

Advertisements

4 Responses to “Open IGC: Uploading New Assets!”

  1. Wallace Wong Says:

    I got some quiet time to create my own bundle and assets.

    What we’re currently investigating is defining quasi-technical assets in our lineage. We have some ‘End User Computing’ type of processes such as, importing data to a Spreadsheet then running a macro. Although not automated, this process itself is may need to be documented in the end to end data flow had has as it’s own meta-data and governance attributes. So Open IGC seems to be an ideal solution for this.

    I was successful in creating this in IGC. I created a top level parent called ‘End User Computing’ with a child called Activity which has custom attributes for activity types, e.g. Run Macro, Execute script, etc… I was able to create and import the bundle in to IGC via REST.

    The only thing that got me is when creating the actual asset after the classes were defined, I missed the ‘Reference’ xml tag that links back to the parent class. I kept going back and forth for an hour in my text editor wondering what the problem was. Easy fix. I blame it on trying to do this while listening to the Jays and Texas Rangers game!

    Cheers

    • dsrealtime Says:

      Hi Wallace. Thank you for sharing. That is an excellent use case for governance and illustration. Also a challenging one, I’m sure, but useful for documenting such activities where they are known and accepted, as well as a symbolic reference for enterprises that want to generically document that such activities “exist”. Thanks again!

      –ernie

      PS. I have been burnt by accidentally leaving out certain properties myself. Usually an NHL distraction, in my case. ; )

  2. kumar Says:

    Hi Ernie-
    after creating the asset bundle there is question mark for every attribute (ex- Internal Facing?). When i put the cursor on the question markit,it displays internal facing-I want this to display the definition. How is that possible. Please advise


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: