Yes indeed!
This post was something I starting writing last December, and at that time only had vague memories of sites connecting DataStage with TIBCO queues. And those memories were of C++ code used in a custom plugin to communicate with TIBCO Rendevous. I wanted to see if anyone out there had tried it in one fashion or another with TIBCO and JMS (Java Message Service) Queues. Fast forward 9 months and I can tell you that it’s been done several times — by at least one of my colleages in IBM Lab Services (more than once), and most recently by myself for a DataStage evaluation.
I’m happy to report that it was a perfectly normal and standard JMS implementation, and JavaPack handled it well, yielding control to the Java Class that connected to the TIBCO JMS queue. Having long since proven that I could connect DataStage to BEA, JBoss, and Sun JMS, once I had the appropriate TIBCO libraries (jars) and connection properties, the class compiled and the Job just “worked.” Amazing how rewarding it is to get some “new” connectivity working. Makes you feel like a kid in a candy store.
Anyway, the process is very straightforward, assuming you are comfortable with Java and JMS (see my other Java entry for example class that works with DataStage). The “initialization()” method does all the start-up connectivity work, and the core of the JMS reading and writing is placed inside of the “process()” method. When the “process()” method is invoked by the DataStage engine, the class determines if the invocation is for an output link or an input link, and the JMS code communicates with TIBCO JMS and either writes to, or reads from, a selected Queue. Data to be written to a target queue is first read from the DataStage Input Link, and payloads read by the java class are “written” to the DataStage Output Link….
If you need to do this with DataStage, find a Java developer who knows JMS, and knows the particular provider…each provider has unique properties for things like “InitialContextFactory” and “ProviderURL” and values for the jndi names of the queues and their Connection Factories (lots of interesting Java stuff — too much to post here). JMS queues can have pure strings in “Text Messages” or have other types of messages with different formatting. There are headers to deal with here also. Not a place to jump into lightly if you are only immersed in DataStage. Instead, find a Java expert, and then work together with him/her to find the synergy you can get with JavaPack integrated inside the DS Engine.
Ernie
Tracing Enterprise Data Footsteps! ......celebrating the journey of data! 
September 11, 2008 at 4:02 pm
Hello Ernie, I tried to connect with datastage to IBM’s “common event infrastructure”. The frustrating experience was, that there seems to be nobody in germany within IBM who can give advise about this topic. On the contrary the “java IBM Consultans” do not seem to like this type of use of datastage. Could you advise me what setup I should try to enable a connection to CEI. Thank you in advance Jaap
September 11, 2008 at 8:25 pm
Hi Jaap…
I’m not deeply versed on the CEI, but after reviewing its architecture, it appears that when an event on some other J2EE application or server occurs, a Common Base Event can be created and sent to the Common Event Infrastructure Server — who then acts upon that event and can make it available for anyone else who might be interested. That “interested” party, from what I can tell, could certainly be DataStage. It appears that the CEI Server can forward events into JMS Queues or Topics, or perhaps (depending on how much you can code on your own within it) issue EJB calls to other Servers or pure Web Services (SOAP over HTTP invocations). For the JMS Queues you could easily use the JavaPack and do things as I’ve suggested in my entries here. For the SOAP or EJB connection you could also use DataStage deployed as a Service via RTI (7.5) or ISD (8.x). Not sure why anyone wouldn’t advise using DataStage in this fashion — while there are good reasons to use DataStage or WebSphere TX, or other tools for these things, the protocol alone is not the determining factor.
Ernie
June 26, 2009 at 10:31 am
Hello Ernie
Thanks first for the Post. I tried to do this exercise to integrate with TIBCO . I was able to integrate the same. But there seems to be a confusion
The sequence of API’s initialize , process and terminate API’s of stage are callbacks and cannot be called manually . In the initialize i start the JMS listener and that is a blocking call.
So although control comes to onMessage method , it is never able to call process , writeRow and send processed data forward .
Did you use process spawning or threading? Any sample code base will be useful as a reference 🙂
Cheers
Ram
June 26, 2009 at 2:12 pm
The DataStage engine calls “you” if you include those particular methods and identify your compiled class inside the JavaTransformer or JavaClient….. You can include them, or not. Initialize() is called when the Job Starts and process() is called “if” and “when” the DataStage engine decides that it needs a row from you for the current link, or if targeting, when the link is ready to “give” you a row.
Ernie
June 26, 2009 at 11:24 pm
Hello Ernie
As mentioned i have the JMS Code in the process() method. Correct my understanding here , since the JOB is a real time job , it will be running state always , hence the JMS Listener inside the process() method is always running ( it is a blocking call) hence the overridden process method will never end ?
Am i missing something ?
Ram
June 28, 2009 at 10:51 am
Bingo. The blocking call for receiving a message will wait (theoretically forever) until a message arrives. When the message arrives, parse it as necessary in your code, and then do a writeRow() to an output link before returning output_status_ready and returning control to the DataStage engine. Of course — the wisest thing is to support a timeout value so that things will not sit there “forever.” …and when the timeout occurs return OUTPUT_STATUS_END_OF_DATA …. I like also to have a logical end point that is based upon some particular payload value.