QUICK START GUIDES

Here for the first time? These quick start guides will help to get you up to speed.

API Documentation

blocks
datasources
dl_models
engines
pipelines

Adding custom components in DL model

This tutorial guides you through the steps required to publish custom built layers, metrics and cost function for DL model. Once the custom component is published, it will be available both in jupyter and DL model designer in IDE.

1. Building custom component

Like custom blocks, custom layers should follow certain directory structure so that the component can be published. A custom DL block can have two different scopes

  • project A component published with project scope will be available ony in the project from which it is published. All python code for a custom componet with project scope should be placed inside the following directory structure(replace <bundle_name> with the actual bundle name of the component)

    • __dl_blocks/project/custom_tf_code/<bundle_name> for custom tensorflow layer
    • __dl_blocks/project/metric/<bundle_name> for custom metrics
    • __dl_blocks/project/loss/<bundle_name> for custom loss function

  • org A componet can be published with scope org so that itis available in all projects. Custom component code with org scope should follow the below directory structure

    • __dl_blocks/org/custom_tf_code/<bundle_name> for custom tensorflow layer
    • __dl_blocks/org/metric/<bundle_name> for metrics
    • __dl_blocks/org/loss/<bundle_name> for loss

1.1 Example - Custom tensorflow layer

Create a python file __dl_blocks/project/custom_tf_code/CustomLayer/custom_dense.py and add the following code to it

import tensorflow as tf
import tensorflow as tf
from rztdl.dl.components.custom.annotations import Inputs, Output
from rztdl.dl.components.custom_component import CustomComponent

class CustomDense(CustomComponent):
    """e
    Custom Dense Layer
    """
    layer_input: Inputs
    units: float
    a_out: Output

    def builder(self, layer_input):
        self.kernel = self.add_weight("kernel", [int(layer_input[-1]), self.units])
        self.bias = self.add_weight("bias", shape=[self.units])

    def call_component(self, layer_input):
        # This method should return a dictionary containing tensors
        output_dict = {}

        weight = tf.matmul(layer_input, self.kernel, name="kernel")
        output_dict["a_out"] = tf.add(self.bias, weight, name="bias")
        return output_dict

1.2 Import the custom layer

Create a python file file __dl_blocks/project/custom_tf_code/CustomLayer/__init__.py and add the following python code to it

from .custom_dense import CustomDense

1.3 Publish the layer by running the following code

from razor.api import blocks
from razor.api.core import DLBlockType, BlockScope

blocks.publish_model_custom_blocks(bundle="CustomLayer", 
                                   code_type=DLBlockType.CUSTOM_TF_CODE,
                                   scope=BlockScope.PROJECT, 
                                   overwrite=True)

INFO:razor.api.impl.block_manager_impl:Publishing block bundle...
INFO:razor.api.impl.block_manager_impl:Imports statements are :
INFO:razor.api.impl.block_manager_impl:from razor.project.dlblocks.CustomLayer import CustomDense

'Block published successfully'

The block can be imported and used in jupyter. The block will also be available in DL model designer in IDE

from razor.project.dlblocks.default import CustomDense

Using default configs as no custom logging config was provided

2. Building a custom metric

2.1 Example - Custom metric

Create a python file __dl_blocks/project/metric/Absolute_Metrics/absolute_metrics.py and add the following code to it

import tensorflow as tf
from rztdl.dl.components.custom.annotations import Inputs, Output
from rztdl.dl.components.metrics.custom_metric import CustomMetric


class MAE(CustomMetric):
    labels : Inputs
    prediction : Inputs

    def build(self, input_shape):
        self.metric_function = tf.keras.metrics.MeanAbsoluteErrorr()

    def call_component(self, prediction, labels):
        return self.metric_function(prediction, labels)

    def reset_states(self):
        self.metric_function.reset_states()

2.2 Import the layer

Create a python file file __dl_blocks/project/metric/Absolute_Metrics/__init__.py and add the following python code to it

from .custom_rmse import MAE

2.3 Publish the custom metric

from razor.api import blocks
from razor.api.core import DLBlockType, BlockScope

blocks.publish_model_custom_blocks(bundle="Absolute_Metrics", 
                                   code_type=DLBlockType.METRICS,
                                   scope=BlockScope.PROJECT, 
                                   overwrite=True)

INFO:razor.api.impl.block_manager_impl:Publishing block bundle...
INFO:razor.api.impl.block_manager_impl:Imports statements are :
INFO:razor.api.impl.block_manager_impl:from razor.project.dlblocks.Absolute_Metrics import MAE

'Block published successfully'

1.4 Import the published block

from razor.project.dlblocks.Absolute_Metrics import MAE

3. Building a custom loss function

3.1 Example - Custom loss

Create a python file __dl_blocks/project/loss/CustomLosses/custom_mse.py and add the following code to it

from rztdl.dl.components.custom.annotations import Inputs, Output
import tensorflow as tf
from rztdl.dl.components.losses.custom_loss import CustomLoss


class Loss_MeanSquredError(CustomLoss):
    labels: Inputs
    prediction: Inputs
    out: Output

    def build(self, args):
        self.metric_function = tf.keras.losses.MeanSquaredError()

    def call_component(self, prediction, labels):
        return tf.math.sqrt(self.metric_function(labels, prediction))

3.2 Import the CustomLoss

Create a python file file __dl_blocks/project/loss/CustomLosses/__init__.py and add the following python code to it

from .custom_mse import Loss_MeanSquredError

3.3 Publish the custom loss component

from razor.api import blocks
from razor.api.core import DLBlockType, BlockScope

blocks.publish_model_custom_blocks(bundle="CustomLosses", 
                                   code_type=DLBlockType.LOSS,
                                   scope=BlockScope.PROJECT, 
                                   overwrite=True)

INFO:razor.api.impl.block_manager_impl:Publishing block bundle...
INFO:razor.api.impl.block_manager_impl:Imports statements are :
INFO:razor.api.impl.block_manager_impl:from razor.project.dlblocks.CustomLosses import Loss_MeanSquredError

'Block published successfully'

3.4 Import the published block

from razor.project.dlblocks.CustomLosses import Loss_MeanSquredError