The XING story: Effortless growth process with automated data warehouse management

Since it was founded in 2003, Xing has enjoyed steady and rapid growth, and today has more than 11 million members. According to its 2016 annual report, XING is the largest and, with 60 million visits per month, the most frequently used social network for professional contacts in the German-speaking area. And no matter which growth figures one looks at – revenue, user numbers or employees – the development curve is always an upward one. At a social network company such as XING, which deals exclu...
More

Relieving backlog situations after maintenance work and system outages

Following maintenance work on the server and system outages, backlogs with resource bottlenecks inevitably occur in the workload automation that need to be resolved as quickly as possible. In a backlog situation, the operator team has to take the necessary measures to maintain production operations, and to be able to return to normal operation as quickly as possible. In doing so, key or critical batches and jobs have to be given a higher priority. Unimportant or non-critical jobs have to be eith...
More

Data warehouse automation

A multitude of processes have to be executed in data warehouse environments on a daily basis. Although an enterprise scheduling system is to be found in many companies, it is frequently not used for controlling the individual processes in data warehouse operation. In the majority of cases, larger and more complex batch processes are triggered by the enterprise scheduling system, but the workflow control within these high level batch processes takes place using alternative solutions outside the e...
More

Process Decomposition

Process decomposition involves breaking down complex programs or scripts into smaller programs with a clearly differentiated functionality. Complex, long-running processes Large and complex programs, usually with long runtimes, are used in many IT environments to perform nighttime or background processing tasks. Once they have been started, these programs run completely under their own control. Monitoring the program progress and pinpointing the causes of errors that occur usually involves con...
More

Process Flow Control With Shell Scripts

This article is a critical examination of the use of the UNIX shell (sh, ksh, bash, . . .) in the context of process flow control. The occurring and the problems described within this document are of conceptual nature and also apply to the use of other scripting languages (perl, python, . . .). In no way is it our intention to criticize these languages as a whole. Each of the named languages has its advantages and the author has developed many successful projects with the use of such languages w...
More

Parallelization of processes in data warehouses

The processing of large amounts of data is typical for data warehouse environments. Depending on the available hardware resources, sooner or later the point is reached where a job cannot be processed on a single processor resp. cannot be represented by a single process anymore. The reasons for that are: Time requirements demand the use of multiple processors Systems resources (memory, disk space, temporary tablespace, rollback segments, . . .) are limited. Recurrent errors require the...
More