Asynchronous server-side scripting

Sometimes, server-side scripts can take a long time for execution (e.g., crawling many files, writing many entities, communicating with external services) so that naively executing them, e.g., from the web interface may result in connection timeouts. In such cases, we might want to execute these processes asynchronously so that the script simply runs in the background for as long as it takes.

While LinkAhead doesn’t support asynchronous server-side scripts per se (yet), in this tutorial, we will use LinkAhead’s server-side scripting API to effectively achieve the same thing. It is meant for users who are already familiar with the server-side scripting infrastructure and know how to write and execute (Python) server-side scripts.

For simplicity, let’s consider the following: We have a very simple script that takes a name and a description and inserts a new RecordType if it doesn’t exist already. To emulate long execution times, we may make the script sleep for a few seconds. In total, this script may look like the following.

#!/usr/bin/env python3

from time import sleep

import linkahead as db
from caosadvancedtools.serverside import helper


def main():
    parser = helper.get_argument_parser()
    parser.add_argument("name", help="Name of the new RT")
    parser.add_argument("description", help="Description of the new RT")
    sleep(30)

    args = parser.parse_args()
    db.configure_connection(auth_token=args.auth_token)
    num_rt = db.execute_query(f"COUNT RECORDTYPE WITH name='{args.name}'")
    if num_rt > 0:
        return
    db.RecordType(name=args.name, description=args.description).insert()


if __name__ == "__main__":
    main()

This script can, e.g., be called from the web interface as explained in the server-side scripting documentation, will run for at least 30 seconds, and then insert a new RecordType if we don’t have one with the same name already. Note that this is more of a sketch rather than production code, e.g., it is missing any kind of user feedback regarding successful execution or possible errors.

While 30 seconds execution time usually don’t cause any timeouts, it is already a rather long time to wait in the browser without any feedback, so we want to call this script in an asynchronous way and then inform the user that it was called asynchronously and is running in the background. To achieve this, we create a second Python script that calls the above one. Assuming the above script was called create_rt.py, the calling script could look as follows.

#!/usr/bin/env python3

import os
import subprocess
import sys

from pathlib import Path

import linkahead as db
from caosadvancedtools.serverside import helper


def main():
    parser = helper.get_argument_parser()
    parser.add_argument("name", help="Name of the new RT")
    parser.add_argument("description", help="Description of the new RT")
    args = parser.parse_args()

    # log-in with the provided credentials and save auth token for further log-in
    db.configure_connection(auth_token=args.auth_token)
    db.Info()
    conn = db.get_connection()
    auth_token = conn._authenticator.auth_token

    # Provide the absolute path to the other script
    exec_path = Path(__file__).parent / "create_rt.py"
    cmds = [
        str(exec_path),
        "--auth-token",
        auth_token,
        args.filename,
        args.name,
        args.description
    ]
    # We can use the environment of the SSS setup, but we need to reset the HOME
    # so that the called script can find e.g., the pylinkahead.ini config file.
    myenv = os.environ.copy()
    myenv["HOME"] = str(Path(__file__).parent.parent / "home")
    # We also need to provide a cwd, otherwise Popen will default to cwd=None
    p = subprocess.Popen(cmds, start_new_session=True, env=myenv, cwd=str(exec_path.parent))
    print(
       f"Your job to create RT '{args.name}' asynchronously was started successfully "
       "and is running in the background."
    )


if __name__ == "__main__":

    main()

There are a few things to mention about the calling script:

• We use the authentication token provided by the server-side scripting environment to authenticate, then we copy the resulting login token and provide it to the called script which can then use this token to authenticate with the same permissions as the calling script.

• We reproduce the general environment of the server-side scripting API, but we reset the HOME variable since the SSS API uses a temporary directory.

• We need to additionally specify a current working directory since Popen’s default to None will result in errors when importing LinkAhead.