BICsuite and schedulix offer powerful functions that enable scripts to be saved and executed in the scheduling server. The linchpin is the Run Program, which we would like to introduce to you in more detail in this series of articles. The following also applies for the Rerun Program and the Kill Program.
The field run program contains the definition of the command line to be executed. This field is parsed by the scheduling server and split into an array of arguments. This array of arguments is then used to invoke the execv() system call (or any of the other system calls from the same family, most notably the execvp()call).
In order to visualize what’s happening under the hood, we use a small program that does nothing more than
to print its number of arguments and their values to stdout.
#include <stdlib.h>
#include <stdio.h>
int main (int argc, char *argv[])
{
printf ("Number of arguments: %d\n", argc);
for (int i = 0; i < argc ; ++i) {
printf ("arg [%d] = -> %s <-\n", i, argv [i]);
}
return 0;
}The program is called printargs and when called (from the shell) it behaves as expected:
ronald@oncilla:~/$ ./printargs a b c d e
Number of arguments: 6
arg[0] = -> ./printargs <-
arg[1] = -> a <-
arg[2] = -> b <-
arg[3] = -> c <-
arg[4] = -> d <-
arg[5] = -> e <-The values of the arguments are enclosed in -> and <- in order to be able to show the quoting that
is left after the run program has been parsed by the scheduling server.
When editing the run program in the GUI, the system takes care of the outer level of quoting. When writing a command, in a script or similar, that can be sent to the scheduling server, the user will have to take care of the correct quoting himself.
Unless stated otherwise, we omit the outer level of quoting and act as if the values of the run program are entered into the GUI.
Simple usage
First of all the printargs utility is called with a simple fixed parameter. Basically it looks like
run program = printargs hello world
As stated in the introduction, we’ve omitted the outer level of quoting. If we create a Job Definition with this run program and execute it, we won’t be surprised if this yields:
Number of arguments: 3
arg[0] = -> printargs <-
arg[1] = -> hello <-
arg[2] = -> world <-Although invaluable in practice, this example isn’t really exiting. It gets more interesting if we start using
parameters. These parameters can be addressed by prepending the name of the parameter with a dollar-sign
($). An example looks like:
run program = printargs $JOBID
The (predefined) parameter JOBID is used here. Its value is the ID of the Job:
Number of arguments: 2
arg[0] = -> printargs <-
arg[1] = -> 2305268 <-The predefined standard parameters aren’t that interesting for our purposes here. We’ll use a parameter called TESTPARM which we’ve defined within the Job Definition. The parameter type is parameter, which enables us to specify its value at submit time. (In fact, because no default value has been specified, we must provide a value for TESTPARM at submit time). The run program is amended to:
run program = printargs $TESTPARM
If we specify:
TESTPARM = Hello World
and submit the job, the result will look like this:
Number of arguments: 3
arg[0] = -> printargs <-
arg[1] = -> Hello <-
arg[2] = -> World <-If we want TESTPARM to be treated as a single argument we have to quote it in the run program like this:
run program = printargs "$TESTPARM"
and as a result we now get:
Number of arguments: 2
arg[0] = -> printargs <-
arg[1] = -> Hello World <-So far so good, everything is easy and relaxed. But what if we would like to have an output that looks like this:
Number of arguments: 2
arg[0] = -> printargs <-
arg[1] = -> $TESTPARM = Hello World <-The first challenge here is to convince the system not to interpret the dollar sign. There are two ways to
achieve this, and which one is best depends on the situation. But let’s try something that doesn’t work first. At submit time, we specify:
TESTPARM = $TESTPARM = Hello World
and try to submit the Job Definition. Here we run into a problem; the scheduling server responds with an
error message:
Run into a loop while trying to resolve variable TESTPARM
So what happened here? The scheduling server resolves parameters recursively. If it finds a substring within
the value of a parameter that looks like a dollar sign followed by a valid identifier, it’ll try to resolve that as
a parameter. But if the identifier happens to be the name of the parameter itself, this will cause a loop (the
system needs the value of TESTPARM in order to determine the value of TESTPARM).
Obviously we’ll have to convince the system, that $TESTPARM within the parameter value shouldn’t be resolved.
This is done by escaping the dollar sign with a backslash. Hence instead of the previous we specify:
TESTPARM = \$TESTPARM = Hello World
This time the Job Definition can be submitted and shows the desired output:
Number of arguments: 2
arg[0] = -> printargs <-
arg[1] = -> $TESTPARM = Hello World <-The dollar sign (and the following identifier) isn’t removed and replaced by a parameter value. Instead it stays in place and the system only removes the escape character.
The same can be applied to the run program, e.g.:
run program = printargs \$TESTPARM = $TESTPARM
But if we run this, a next surprise awaits us:
Number of arguments: 5
arg[0] = -> printargs <-
arg[1] = -> $TESTPARM <-
arg[2] = -> = <-
arg[3] = -> Hello <-
arg[4] = -> World <-All of a sudden we end up with 5 parameters instead of only one. Still the basic idea of escaping the dollar
sign seems to work.
In the shell (sh, bash, ksh, …) double quotes are used to keep parts of arguments together that are
separated by white space. Single quotes can be used as well, with the potential downside (or advantage)
that variables aren’t resolved. Something like $PID will be treated as a string starting with a dollar sign,
followed by PID, and won’t be replaced by a process ID.
Let’s try and see what happens. We’ll start with adding double quotes to the run program:
run program = printargs "\$TESTPARM = $TESTPARM"
And indeed, this does exactly what we’d like to achieve:
Number of arguments: 2
arg[0] = -> printargs <-
arg[1] = -> $TESTPARM = Hello World <-It seems that the behaviour of the scheduling server and the behaviour of a Bourne-like shell are similar. If
we try to confirm this suspicion and use single quotes instead of double quotes in the run program, we yield:
Number of arguments: 2
arg[0] = -> printargs <-
arg[1] = -> \$TESTPARM = $TESTPARM <- This looks very good. It seems to match exactly what we’ve expected. But in fact the system generously interpreted the backslash as a simple backslash, and not as an escape character that changes the meaning of the character that follows it. This is possible because the character that follows, the dollar sign, has no special meaning within a single quoted string. The unambiguous specification of this run program looks like:
run program = printargs ’\\$TESTPARM = $TESTPARM’
with the same result as above. In this case the first backslash defines that the following backslash has to be regarded as an ordinary character, not a special character. This generous behavior plays a role as soon as the character following the backslash has a special meaning, such as a single quote itself. Hence a specification like
run program = printargs ’\’
will fail, because the single quoted string doesn’t have the terminating single quote; it is escaped by the backslash. Depending on what has to be achieved, the correct run program is either
run program = printargs ’\\’
(which corresponds to a backslash as an argument of printargs), or
run program = printargs ’\’’
(which corresponds to a single quote an argument of printargs).
Since we’ve found out that the interpretation of the run program follows a lot of the rules of the Bourne(-like) shell, another way to reach our goal is to use a mix of single and double quotes:
run program = printargs ’$TESTPARM = ’"$TESTPARM"
Since there’s no white space between the ending single quote and the starting double quote, the argument isn’t split into multiple arguments at this point. Indeed, if we try this alternative, the output looks like
Number of arguments: 2 arg[0] = -> printargs <- arg[1] = -> $TESTPARM = Hello World <-
Part 1: Introduction and simple usage
Part 2: Advanced usage
Part 3: Backticks
Part 4: Other interpreters
Part 5: Circumventing limitations