Part 7: The Promise

As promised, here is part seven of the series about Rash, the Raku Shell.

See also: The Introduction | Part 1: The Path | Part 2: The Loop | Part 3: The Execution | Part 4: The Interrupt | Part 5: The Dynamic | Part 6: The Process.

Exercise 5: Blank Lines

No one noticed and reported the error I had placed in Rash (or missed myself; take your pick).

Pressing return at the Raku prompt gives a nasty surprise (that fortunately doesn't terminate the program):

rash: Enter «exit» to exit
> 
Use of uninitialized value $cmd of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at rash line 28
Use of uninitialized value $cmd of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at rash line 28
Use of uninitialized value $cmd of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at rash line 45
Use of uninitialized value $cmd of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at rash line 45
Unknown command: "" (use "help" for a list of commands)

Fix this.

See The solution.

Promises and Proc::Async

In natural language, a promise is a pledge to do something. Breaking a promise is generally frowned upon, and can get you into legal an/or moral trouble.

In Raku a promise is much the same thing; a pledge to do something, and it is possible to break the promise.

In Part 6: The Process we looked at the run and shell functions for executing external programs. They are quite powerful, but are limited to executing «one program at a time». So when the external program is running, the Raku programs is waiting.

Proc::Async is much more flexible, as it allows parallel execution. We'll get back to that. First the basics:

> my $proc = Proc::Async.new('emacs', 'foo.txt', 'bar.txt');

It is dormant, until we turn the key (so to speak).

We can take a look at the Proc::Async object, before fiddling with the key:

> say $proc;
Proc::Async.new(
  path         => "emacs",
  args         => ["foo.txt", "bar.txt"],
  command      => ("emacs", "foo,txt", "bar.txt"),
  w            => Any,
  enc          => "utf8",
  translate-nl => Bool::True,
  started      => Bool::False
)

The last value tells us that we haven't started the process yet. It is shown with the Bool:: prefix as Boolean values are set up as an Enum(eration), and not as their own type.

We can use the enums method to get the values for an Enum(erated) type. E.g.:

> say Bool.enums;
Map.new((False => 0, True => 1))

See docs.raku.org/language/typesystem#enum for more information about the enum type.

See docs.raku.org/routine/enums for more information about the enums method.

Map is essentially a read only version of a hash. See docs.raku.org/type/Map for more information about the Map class.

> say $proc.started;  # -> False

That is pretty much al lwe can do, until we start the process (turn the key). We do that by invoking .start on it:

> my $promise = $proc.start;

This fires off the extrenal program, and returns a Promise object without waiting for the program to finish.

You should now have an emacs window on your screen.

We can take a look at the Promise object:

> say $promise;
Promise.new(
  scheduler          => ThreadPoolScheduler.new(
    initial_threads  => 0,
    max_threads      => 64,
    uncaught_handler => -> $exception { #`(Block|94540249746184) ... }
  ),
  status => PromiseStatus::Planned
)

The last line is the interesting one.

Now, close the emacs window, and inspect the Promise object again:

> say $promise;
Promise.new(
  scheduler           => ThreadPoolScheduler.new(
    initial_threads  => 0,
    max_threads      => 64,
    uncaught_handler => -> $exception { #`(Block|94540249746184) ... }
  ),
  status => PromiseStatus::Kept
)

A Promise is not Kept until the program has finished. This is like sending a registered letter. The job hasn't finished successfully until the recipient has receieved and signed for it.

You may have guessed that PromiseStatus is an enumeration. Let us have a look at the values:

> say PromiseStatus.enums;
Map.new((Broken => 2, Kept => 1, Planned => 0))

A broken promise is something that happens if we try to execute a non-existing program:

> my $proc = Proc::Async.new('emacs-foo', 'foo.txt', 'bar.txt');
> say $proc.started;  # -> False

> my $promise = $proc.start; say $promise.status;  # -> Planned
> say $promise.status;                             # -> Broken

You get «Planned« the first time, as Raku hasn't figured out that the program doesn't exist yet.

You may want to get the exit code from the program, and we can use the result method on the Promise object. The exit code is not known until the program has finished, so a side effect of calling result is that Raku will wait for the program to finish before doing anything else.

Doing this on a broken promise causes an exception:

> my $result = $promise.result;
Tried to get the result of a broken Promise

Original exception:
    Failed to spawn process emacs-foo: no such file or directory
    (error code -2)

Doing it on a running program gives a Proc object, which is what we get if we use run or shell:

> my $proc = Proc::Async.new('emacs', 'foo.txt', 'bar.txt');
> my $promise = $proc.start;
> say $promise.result;   # [1]
Proc.new(in       => IO::Pipe,
         out      => IO::Pipe,
         err      => IO::Pipe,
         exitcode => 0,
         signal   => 0,
         pid      => Nil,
         command  => ("emacs", "foo.txt", "bar.txt"))

[1] This one does nothing (i.e. hangs), until we close the emacs window.

Note that the arguments to Proc::Async.new must be specified as scalar values, and not as a list. This is identical to run and shell.

We can wrap up what we have seen so far in a program:

sub MAIN(*@cmd)                                                     # [1]
{
  my $process = Proc::Async.new(|@cmd);                             # [2]
  my $promise = $process.start;                                     # [3]
  my $result  = $promise.result;                                    # [4]
  
  say "Command exit code { $result.exitcode }" if $result.exitcode; # [5]
} 

[1] A Slurpy argument, so that we get all the arguments in the array.

[2] We flatten the arguments, as Proc::Async accepts single arguments and not a list.

[3] $proc.start returns a Promise.

[4] and we get the result.

[5] Display the status code, if non zero (as that is an error message).

We can test it, as we did in Part 3: The Execution:

$ raku proc-run-err pwd
/home/arne/raku/kurs/text/code

$ raku proc-run-err true

$ raku proc-run-err false
Command exit code:1

$ raku proc-run-err falseX
Tried to get the result of a broken Promise

Original exception:
    no such file or directory

And it works, except for non-existing programs where the Exception terminates the program.

The exception happen when we access $promise.result, as the Promise was Broken.

We can add a CATCH block (or handler) to intercept (or catch) the exception:

sub MAIN(*@cmd)
{
  my $process = Proc::Async.new(|@cmd);
  my $promise = $process.start;
  my $result  = $promise.result;            # [1]
  
  CATCH                                     # [3]
  {
    say 'Unknown command: "{ @cmd[0] }".';  # [4]
    exit 1;                                 # [5]
  }
  
  say "Command exit code { $result.exitcode }" if $result.exitcode;  # [2]
} 

[1] Accessing the result of the Promise fails if the program couldn't be executed. But it is a Soft Failure.

[2] So the actual Failure happens when we access the .exitcode field.

[3] and CATCH prevents the program from crashing.

[4] Display an error message.

[5] Exit with error code 1.

See docs.raku.org/language/exceptions#index-entry-CATCH-CATCH for more information about CATCH.

Note that we don't need the pid to decide if the program was executed, as we did with run.

Besides that, if this is all you'd want to do, use run instead and let Raku handle the complexity.

Let us take a step back, and use $promise.status so that we can avoid the Exception:

sub MAIN(*@cmd)
{
  my $proc    = Proc::Async.new(|@cmd);
  my $promise = $proc.start;
  my $status  = $promise.status;

  say $status;

  if $status ~~ Broken
  {
    say "Unknown command: \" @cmd[0] \".";
    exit 1;
  }
  
  my $result  = $promise.result;

  say "Command exit code:{ $result.exitcode }" if $result.exitcode;
} 

This fails for non-existing programs:

$ raku proc-run-status trueX
Planned
Tried to get the result of a broken Promise

Original exception:
    no such file or directory

The problem is that we don't wait for the program to finish, as the $promise.result call did. So the program runs along, and checks the status of the process before it even started (and the status is Planned). When we did this in REPL mode, typing the command gave Raku time enough to figure it out. But running a script is too fast.

We can give Raku time with the sleep call, which does absolutely nothing for the specified number of seconds. E.g.

> say "Now"; sleep 10; say "Ten seconds later"; sleep 0.5; say "Half a second later";    
Now
Ten seconds later
Half a second later

See docs.raku.org/routine/sleep for more information about sleep.

Applied to the program:

sub MAIN(*@cmd)
{
  my $process = Proc::Async.new(|@cmd);
  my $promise = $process.start;

  say $promise.status;

  sleep 1;

  say $promise.status;

  if $promise.status ~~ Broken
  {
    say "Unknown command: \" @cmd[0] \".";
    exit 1;
  }
  
  my $result = $promise.result;

  say "Command exit code:{ $result.exitcode }" if $result.exitcode;
} 

$ raku proc-run-status2 false
Planned
Kept
Command exit code:1

$ raku proc-run-status2 falseX
Planned
Broken
Unknown command: " falseX ".

$ raku proc-run-status2 true
Planned
Kept

We have added a 1 second delay, that is unneccesary (and irritating) in most situations. And the delay may even be too short if the load on the computer is extremely high.

await

Inspecting $promise.status can be likened to open the front door to see if there are anybody there. It is better to install a bell, so that a visitor can inform you of their presence.

We did this with $promise.result above, but we can also use the more general await on the Promise:

sub MAIN(*@cmd)
{
  my $process = Proc::Async.new(|@cmd);
  my $promise = $process.start;

  say $promise.status;

  await $promise;                     # [1]

  say $promise.status;

  if $promise.status ~~ Broken
  {
    say "Unknown command: \" @cmd[0] \".";
    exit 1;
  }
  
  my $result = $promise.result;

  say "Command exit code:{ $result.exitcode }" if $result.exitcode;
}

[1] It is important to place it before accessing the result.

$ raku proc-run-await true
Planned
Kept

And that works out beautifully. As long as the program exists:

$ raku proc-run-await trueX
Planned
An operation first awaited:
  in sub MAIN at proc-run-await line 10

Died with the exception:
      Failed to spawn process trueX: no such file or directory (error code -2)

And the program must return «0» (to indicate that all went well):

$ raku proc-run-await false
Planned
The spawned command 'false' exited unsuccessfully (exit code: 1, signal: 0)
  in sub MAIN at proc-run-await line 10

The problem is that await on a Broken promise will get the exception from the promise.

And it regards a non-zero exit code from the program as an error, so running «proc-run-await false» gives an exception as well.

We can wrap it in try, like this:

try await $promise;

But that doesn't help, as we disregard the return value of await, and doing so (called sunk value in Raku) causes an Exception.

We can avoid that by getting rid of the value ourself (sinking it):

try sink await $promise;

Some examples:

> "12".say
12

> "12".sink.say
Nil

The example above is pretty useless, as we could (and should) have dropped the say. But we can use sink to get rid of objects that cause Exeptions if we ignore them.

See docs.raku.org/routine/await for more information about await.

See docs.raku.org/routine/sink for more information about sink.

The problem was that we disregarded the value. These lines are equivalent:

try sink await $promise;
my $dummy = try await $promise;
$ = try await $promise;

The modified program (without printing the status of the promise):

sub MAIN(*@cmd)
{
  my $process = Proc::Async.new(|@cmd);
  my $promise = $process.start;

  try sink await $promise;

  if $promise.status ~~ Broken
  {
    say "Unknown command: \" @cmd[0] \".";
    exit 1;
  }
  
  my $result = $promise.result;

  say "Command exit code:{ $result.exitcode }" if $result.exitcode;
} 

Try it with the four use cases; «true», «false», «trueX», «emacs»

Running Programs with a Timeout

We can add support for timeout of running programs:

sub MAIN(*@cmd)
{
  my $timeout = 10;                       # [1]
  my $process = Proc::Async.new(|@cmd);
  my $promise = $process.start;

We skip the CATCH, as we'll handle it another way.

  my $waitfor = Promise.anyof(Promise.in($timeout), $promise); # [1] [2]
  await $waitfor;                                              # [3]

[1] We set up a promise 10 seconds into the future (with Promise.in).

[2] and set up a third promise that waits for any one of the first and second promises to succeed (with Promise.anyof).

[3] Then we have to wait for this third promise, and this is done with await.

Note that the Exception problem in the previous section does not apply here, as the outer promise (the third one) doesn't allow the Exception from the inner promise to be thrown.

The next part is handling this third Promise:

  if $promise.status ~~ Kept                    # [4]
  {
    my $exit = $promise.result.exitcode;
    say "Command exit code: $exit" if $exit;
    exit $exit;
  }

[4] We check if the first promise («I promise to run this program») was kept. If it has, we get the exit code and print it if non zero.

Then we check for failure:

  elsif $promise.status ~~ Broken
  {
    say "Unknown command: @cmd[0].";
    exit 1;
  }

Note that we don't access promise.result as we did in «proc-run» so we don't trigger an Exception. (And we don't check for this Exception to tell us of the failure to run the program.)

If the promise isn't Kept or Broken, it is Planned - and that means that the program is still running:

  else
  {
    $process.kill;                                               # [5]
    say "Program @cmd[] did not finish after $timeout seconds";  # [6]
    exit 2;                                                      # [7]
  }
} 

[5] Kill the program.

[6] Display an error message.

[7] Exit with a suitable error code.

See docs.raku.org/routine/kill for more information about the kill method.

Testing it:

$ raku proc-run-timeout true

$ raku proc-run-timeout false
Command exit code: 1

$ raku proc-run-timeout falseX
Unknown command: "falseX".

$ raku proc-run-timeout emacs 1 2 3 
Program emacs 1 2 3 did not finish after 10 seconds

It is easy to modify the program so that we can specify the timeout on the command line:

sub MAIN(*@cmd, :$timeout = 10)
{
  # my $timeout = 10;
  ...

Then some testing:

$ raku proc-run-timeout emacs 1 2 3 
Program emacs 1 2 3 did not finish after 10 seconds

$ raku proc-run-timeout --timeout=2 emacs 1 2 3 
Program emacs 1 2 3 did not finish after 2 seconds

Note that we have to use a named argument for the timeout parameter, so that the slurpy array doesn't eat it.

The whole program:

sub MAIN(*@cmd, :$timeout = 10)
{
  my $process = Proc::Async.new(|@cmd);
  my $promise = $process.start;

  my $waitfor = Promise.anyof(Promise.in($timeout), $promise);
  await $waitfor;

  if $promise.status ~~ Kept
  {
    my $exit = $promise.result.exitcode;
    say "Command exit code: $exit" if $exit;
    exit $exit;
  }
  
  elsif $promise.status ~~ Broken
  {
    say "Unknown command: @cmd[0].";
    exit 1;
  }

  else
  {
    $process.kill;
    say "Program @cmd[] did not finish after $timeout seconds";
    exit 2;
  }
} 

We have managed to do this without a single Exception handler (try or CATCH). But one can argue that the code is complicated.

Rash with Timeouts

And now we can add timeout support to Rash (the Raku Shell).

We could call our command «timeout6» (so as not to collide with the external «timeout» program, which does the same) and use it like this:

> timeout6 10 top

Where the number is the time limit in seconds.

But Rash has command completion for the first argument only, so this approach would annoy the users (as the completion wouldn't work when we come to the actual command).

So we'll add the time limit at the end of the command line instead, with a prefixed colon to show that it isn't part of the argument list to the program:

> top :10

Adding the procedure handling this is easy, given the «proc-run-timeout» program:

sub do-run-timeout ($cmd, @args, :$timeout)
{
  my $process = Proc::Async.new($cmd, |@args);
  my $promise = $process.start;

  my $waitfor = Promise.anyof(Promise.in($timeout), $promise);
  await $waitfor;

  if $promise.status ~~ Kept
  {
    my $exit = $promise.result.exitcode;
    say "Command exit code: $exit" if $exit;
    return;
  }
  
  elsif $promise.status ~~ Broken
  {
    say "Unknown command: $cmd.";
    return;
  }

  else
  {
    $process.kill;
    say "Program $cmd did not finish after $timeout seconds";
    return;
  }
}

This should look very familiar by now.

Note that we'll keep the old way to run programs without timeouts, so we don't have to handle a no timeout situation here.

I could have used multi subs, but it has no benefits (and is slightly slower due to the lookup).

The default blocks must be updated:

    default
    {
      my ($cmd, @args) = $line.words;

      if %programs{$cmd}
      {
        my $timeout = False;
	
        if @args.elems
       	{
          if @args[*-1] ~~ /^\:\d+$/
          {
            my $last = @args.pop;
            $last ~~ /^\:(\d+)$/;
            $timeout = $0.Int if $0;
          }
        }

        $timeout
          ?? do-run-timeout %programs{$cmd}, @args, :$timeout
          !! do-run %programs{$cmd}, @args;
      }
      else
      {
        say "Unknown command: \"$_\" (use \"help\" for a list of commands)";
      }
    }

Running it:

$ rash-timeout
> top :10
...
Program /usr/bin/top did not finish after 10 seconds

The number of ways this can be used to piss off Rash users is almost limtless...

Exercise 6: Configurable Timeouts

Add support for the «timeout6» command (and support «timeout» as well, if you feel like it) in the configuration file.

When the user runs a program, use the value on the command line, if given. If not, use the one set up with «timeout6» (on the command line or in the configuration file). If the user specifies «:0», that means no limit (ans thus ignore any value already set).

See The solution.

Will There Be a Part 8?

Probably. I have some ideas:

• Respond to feedback (on Reddit)
• IO redirection, with at least ">" and "|" (and possibly more)
• Command lists, with ";"
• Conditional execution, with "&&"
• Background execution, with "&", Control-Z, and the "fg", "bg", "jobs" and "kill" commands
• The "type" command
• Shell script support, with loops («for»/«while») and conditions («if»/«elsif»/«else»)

And that's it for now.