— this is a work in progress at the moment — ugh. there has to be an easier way to do this! I thought I was through writing hand cooked html a long time ago
—
Here is a consolidated list of all the posts, by major subject and link only. I was having trouble finding old entries myself! The Categories in the side-bar are useful, but bring up entire posts, not just a summary of the urls within that category. Let’s try this.
Metadata Management
Data Lineage and Web Services
Cool New Business Glossary Widget!
Anyone else Fried by Nulls?
General RealTime Discussions
ETL vs EAI Re-visited
RSS for your Data Transformations
What is Real-Time ETL Anyway?
Command Line Tools for Event Based Real-Time
Almost Real-Time Jobs
Simulating end-of-file for Real-Time ETL Transformations (what is “end of wave”)
XML
Handling Nulls in XML Sources
Reading XML Content as a Source
another good link on using XML and Web Services
Java
Incorporating Java Classes into your DataStage Jobs
DataStage and TIBCO
Calling formal Web Services (DataStage as a SOAP client)
Tips for using Web Services with DataStage…Getting Started
Tips for using Wb Services Pack with DataStage Part II – Invoking your first Service
Tips for WSPack Part III – Best Practices and Guidelines
How to Invoke Complex Web Services
Calling Web Services that use Arrays
Deploying Services/Web Services (DataStage as a Service Provider)
Launching Jobs via Web Service
Data Quality and Transfomation as a Service
MQ
MQ Series – Ensuring Message Delivery from Queue to Target


August 17, 2009 at 2:53 pm
Can we do screen scraping using datastage.We want to download some data from web by passing parmeters to web screen and then saving desired output in csv.
August 18, 2009 at 7:02 am
I haven’t tried it, but I’ve seen it done via JavaPack, where a java class using HTTP (and used within the Java Client Stage) retrieved items from a remote url, and passed them to the output link as an initial source for the Job. I recall someone trying this with Perl years ago also, but the Perl Stage is no longer generally available on the canvas. Java would likely be the best choice.