A multitude of processes have to be executed in data warehouse environments on a daily basis. Although an enterprise scheduling system is to be found in many companies, it is frequently not used for controlling the individual processes in data warehouse operation. In the majority of cases, larger and more complex batch processes are triggered by the enterprise scheduling system, but the workflow control within…
Process decomposition involves breaking down complex programs or scripts into smaller programs with a clearly differentiated functionality. Complex, long-running processes Large and complex programs, usually with long runtimes, are used in many IT environments to perform nighttime or background processing tasks. Once they have been started, these programs run completely under their own control. Monitoring the program progress and pinpointing the causes of errors that occur usually involves…
This article is a critical examination of the use of the UNIX shell (sh, ksh, bash, . . .) in the context of process flow control. The occurring and the problems described within this document are of conceptual nature and also apply to the use of other scripting languages (perl, python, . . .). In no way is it our intention to criticize these languages as…
The processing of large amounts of data is typical for data warehouse environments. Depending on the available hardware resources, sooner or later the point is reached where a job cannot be processed on a single processor resp. cannot be represented by a single process anymore. The reasons for that are: Time requirements demand the use of multiple processors Systems resources (memory, disk space, temporary tablespace, rollback…