Compiler

Build complex compilation pipelines

Compiler

Sometimes AGGREGATES may require complex initialization and be built from multiple sources, like database queries, API calls, or other external resources are required to build it. Each one with it's own lifecycle that may require some initialization method or cleanup after they are no longer needed.

In the context of neural networks, AGGREGATES not only should be built but also compiled. Compilation is the process of converting a high-level neural network model into a low-level representation that can be executed on a specific hardware platform, and can be seen as an integral part of the process of building the AGGREGATE. This process get even more complex when performing distributed training.

A Compiler is a class that compiles a pipeline of functions to be executed in sequence in order to build an a low-level representation of the AGGREGATE. Since some compilation steps sometimes requires runtime information, the Compiler provides a mechanism to inject dependencies into the pipeline.

Attributes:

Name	Type	Description
steps	list[Callable[..., Any]]	A list of functions to be executed in sequence.

Methods:

Name	Description
compile	Execute the pipeline of functions in sequence. The output of each function is passed as input to the next function. The compiled AGGREGATE should be returned as a result of the execution of the pipeline.
step	A decorator that adds a function to the pipeline. The function should take as input the output of the previous function in the pipeline and return the input of the next function in the pipeline.

Example

from logging import getLogger
from torch import cuda
from torch.nn import Module
from torchsystem.compiler import Compiler
from torchsystem.compiler import Depends
from torchsystem.compiler import compile

compiler = Compiler()
logger = getLogger(__name__)

def device():
    return 'cuda' if cuda.is_available() else 'cpu'

def epoch():
    raise NotImplementedError('Override this function with a concrete implementation')

@compiler.step
def build_classifier(model: Module, criterion: Module, optimizer: Module):
    logger.info(f'Building classifier')
    logger.info(f'- model: {model.__class__.__name__}')
    logger.info(f'- criterion: {criterion.__class__.__name__}')
    logger.info(f'- optimizer: {optimizer.__class__.__name__}')
    return Classifier(model, criterion, optimizer)

@compiler.step
def move_to_device(classifier: Classifier, device = Depends(device)):
    logger.info(f'Moving classifier to device: {device}')
    return classifier.to(device)

@compiler.step
def compile_classifier(classifier: Classifier):
    logger.info(f'Compiling classifier')
    return compile(classifier)

@compiler.step
def set_epoch(classifier: Classifier, epoch: int = Depends(epoch)):
    logger.info(f'Setting classifier epoch: {epoch}')
    classifier.epoch = epoch
    return classifier
...

compiler.dependency_overrides[epoch] = lambda: 10
classifier = compiler.compie(model, criterion, optimizer)
assert classifier.epoch == 10

dependency_overrides property

Get the dependency overrides. Dependency overrides are used to inject dependencies into the pipeline. This is useful for late binding, testing and changing the behavior of the compiler in runtime.

Returns:

Name	Type	Description
dict	dict	The dependency map.

Example

def device():...
...

compiler.dependency_overrides[device] = lambda: 'cuda' if cuda.is_available() else 'cpu'

__init__(*, provider=None)

Initialize the Compiler.

Parameters:

Name	Type	Description	Default
provider	Provider	The dependency provider. Defaults to None.	None

compile(*args, **kwargs)

Execute the pipeline of functions in sequence. The output of each function is passed as input to the next function. The compiled AGGREGATE should be returned by the last function in the pipeline.

Returns:

Name	Type	Description
T	T	The compiled AGGREGATE.

step(callable)

Add a function to the pipeline. The function should take as input the output of the previous function in the pipeline and return the input of the next function in the pipeline.

Parameters:

Name	Type	Description	Default
callable	Callable	The function to be added to the pipeline.	required

Returns:

Name	Type	Description
Any	Any	The requirements for the next step in the pipeline.