New Horizons for Lineage!

Hi Everyone,

It has been awhile since I’ve written any observations about lineage and our world of data integration and metadata. I’ve been spending time these past four years listening and learning. During that time I have been able to gather more insights about lineage, data governance and management, the evolution of open-source (for lineage and metadata), and hot new trends like “Observability” and “Active Metadata.”

With that in mind, it’s time to take this blog to places it hasn’t been before — to new technologies and methods for defining, qualifying, and establishing lineage. But also to new use cases and new ways of thinking about meeting the challenges being faced while understanding your data pipelines.

I have gained a deeper appreciation for tracking lineage (not only for Classic DataStage as covered for many years in this blog) but also for a myriad of other Extract-Transform-Load (ETL) solutions, known and unknown relational databases, a vast array of long outdated on-premises tools, sparkling new cloud-based data shaping solutions, as well as the next generation of DataStage!

Stay tuned! It’s been a fun journey and fulfilling learning experience for me, and I look forward to sharing it with you, starting with a series of posts that look closely at the need for something called Descriptive Lineage.

In the meantime, have some fun with this recent experiment in presenting lineage in Virtual Reality!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: