Python Notes - Part 1 of n

In my effort to learn Python, I have been picking up various techniques. These include parsing CLI arguments, dynamically loading modules and operator overloading. This article consists of lightly elaborated notes. It may not be of general interest and is primarily intended to help me recall these techniques at a later date.

Operator Overloading

Python has first-class support for operator overloading. This allows code to invoke methods using Python’s built-in operators, like +, < and * by specifying corresponding methods. Similarly, it is possible to implement indexing (using [n]). Given I was working on matrix multiplication, I thought I would give this a try. To simplify the example, I will show the technique with a vector. Here is the implementation.

class Vector:
    def __init__(self, vals):
        self._vals = vals

    def __mul__(self, vector) -> float:
        result = 0.0
        for i in range(len(self._vals)):
            result = result + self[i] * vector[i]

        return result

    def __getitem__(self, i):
        return self._vals[i]

    def __setitem__(self, i, val):
        self._vals[i] = val

Here is a test that exercises this implementation.

import unittest
from matrix import *

class VectorTestCase(unittest.TestCase):
    def setUp(self):
        self.unit = Vector([1.0, 2.0, 3.0])

    def test_multiply_multiplies_vectors(self):
        other = Vector([3.0, 2.0, 1.0])

        # Use the overloaded operator
        result = self.unit * other

        self.assertEqual(result, 10.0)

Note the use of the overloaded operator *. Also note in the function __mul__, each element of the vector is referenced by its index. Though not shown here, values can also be set by index, for example unit[0] = 3.14.

Parsing CLI Arguments

Python has built-in command line argument parsing. More than one, actually. The one that seems to be pretty feature rich in my limited experience is argparse This one interested because it has support for sub-commands.

#!/usr/bin/python3

import argparse

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Example CLI with subcommands')
    command_parser = parser.add_subparsers(dest='cmd', title='commands')
    args = parser.parse_args() 
    print(args)

But what do you actually do with these arguments once they have been parsed? You could write a bunch of conditional checks and then call a function based on what condition is true. I prefer an approach where the command execution can be encapsulated and loaded dynamically. This makes it easy to add sub-commands without having the change the main program. This is the next area I explored while learning this topic.

Dynamically Loading Modules

Java has the ability to create classes found on the classpath by name. How is this done in Python? Using the importlib library. Here is a solution to dynamically loading types the expose specific functions to initialize the sub-command parser and run specific operations when invoked from the CLI.

Create a Directory of Commands

First create a directory of commands.

mkdir commands

Then create files in this directory to represent each command. For example, the version command might be specified as follows.

class VersionCommand:
    def __init__(self, args):
        self.args = args

    def run(self):
        print("version 0.0.0")

    def init_parser(command_parser):
        command_parser.add_parser('version', help='print version information')

This class specifies the following.

1. a constructor that gives us a reference to the arguments

2. a function called run that is called if this command is invoked

3. a class function init_parser that initializes the parser with whatever arguments can/must be specified

The modules in the commands directory can be dynamically loaded as shown in the following code.

#!/usr/bin/python3

import argparse
import importlib
import os.path
import glob

def load_modules(command_parser):
    mods = {}
    for f in glob.glob('./commands/*.py'):
        name = os.path.basename(f).removesuffix('.py')
        mod = importlib.import_module("commands." + name)
        mods[name] = mod

        cmd_type = getattr(mod, str.capitalize(name) + "Command")
        cmd_type.init_parser(command_parser)

    return mods

def run_command(mods, args):
    constructor = getattr(mods[args.cmd], str.capitalize(args.cmd) + "Command")
    cmd = constructor(args)
    cmd.run()

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Example CLI with subcommands')
    command_parser = parser.add_subparsers(dest='cmd', title='commands')
    cmds = load_modules(command_parser)
    args = parser.parse_args() 

    if args.cmd == None or args.cmd == 'help':
        parser.print_help()
    else:
        run_command(cmds, args)

The function load_modules loads all the modules and returns them as a map. It is called in the main loop with the sub-parser, which is passed into the function <Cmd>Command.init_parser, thus initializing the parser. Once the CLI arguments are parsed, they are passed to the function run_command, which resolves the module from the module map. The command class is then instantiated and run.

File globbing, getting the filename and string editing are also illustrated.