InfoSphere Advanced DataStage - Parallel Framework v11.5 - KM404G

InfoSphere Advanced DataStage - Parallel Framework v11.5 - KM404G


Experienced DataStage developers seeking training in more advanced DataStage job techniques and who seek an understanding of the parallel framework architecture.


IBM InfoSphere DataStage Essentials course or equivalent and at least one year of experience developing parallel jobs using DataStage.


3 days.

Course Objectives

This course is designed to introduce advanced parallel job development techniques in DataStage v11.5. In this course you will develop a deeper understanding of the DataStage architecture, including a deeper understanding of the DataStage development and runtime environments. This will enable you to design parallel jobs that are robust, less subject to errors, reusable, and optimized for better performance.

Course Content

Introduction to the parallel framework architecture
Describe the parallel processing architecture
Describe pipeline and partition parallelism
Describe the role of the configuration file
Design a job that creates robust test data

Compiling and executing jobs
Describe the main parts of the configuration file
Describe the compile process and the OSH that the compilation process generates
Describe the role and the main parts of the Score
Describe the job execution process

Partitioning and collecting data
Understand how partitioning works in the Framework
Viewing partitioners in the Score
Selecting partitioning algorithms
Generate sequences of numbers (surrogate keys) in a partitioned, parallel environment

Sorting data
Sort data in the parallel framework
Find inserted sorts in the Score
Reduce the number of inserted sorts
Optimize Fork-Join jobs
Use Sort stages to determine the last row in a group
Describe sort key and partitioner key logic in the parallel framework

Buffering in parallel jobs
Describe how buffering works in parallel jobs
Tune buffers in parallel jobs
Avoid buffer contentions

Parallel framework data types
Describe virtual data sets
Describe schemas
Describe data type mappings and conversions
Describe how external data is processed
Handle nulls
Work with complex data

Reusable components
Create a schema file
Read a sequential file using a schema
Describe Runtime Column Propagation (RCP)
Enable and disable RCP
Create and use shared containers

Balanced Optimization
Enable Balanced Optimization functionality in Designer
Describe the Balanced Optimization workflow
List the different Balanced Optimization options
Push stage processing to a data source
Push stage processing to a data target
Optimize a job accessing Hadoop HDFS file system
Understand the limitations of Balanced Optimizations

Virtual Courses

ALL of our courses can be delivered virtually. And our Bath public schedule of courses are now available as live virtual sessions, using the popular Zoom Virtual Classroom and remote labs. Delegates can test their access at:

On-Site Courses

Can't attend one of our public classes? Booking for multiple people?

All our courses are available on your site! Delivered for your staff, at your premises.

Contact us to find out more...