ConvNet API

namespace convnet

Functions

ConvNetConfig parse_config_json(const nlohmann::json &config)

Parse ConvNet configuration from JSON.

Parameters:

config – JSON configuration object

Returns:

ConvNetConfig

std::unique_ptr<ModelConfig> create_config(const nlohmann::json &config, double sampleRate)

Config parser for ConfigParserRegistry.

class _Head
#include <convnet.h>

Public Functions

inline _Head()
_Head(const int in_channels, const int out_channels, std::vector<float>::iterator &weights)
void process_(const Eigen::MatrixXf &input, Eigen::MatrixXf &output, const long i_start, const long i_end) const
class BatchNorm
#include <convnet.h>

Batch normalization layer.

In production mode, so really just an elementwise affine layer. Applies: y = (x - mean) / sqrt(variance + eps) * weight + bias which simplifies to: y = scale * x + loc

Public Functions

inline BatchNorm()

Default constructor.

BatchNorm(const int dim, std::vector<float>::iterator &weights)

Constructor with weights.

Parameters:

  • dim – Dimension of the input

  • weights – Iterator to the weights vector. Will be advanced as weights are consumed.

void process_(Eigen::MatrixXf &input, const long i_start, const long i_end) const

Process input in-place.

Parameters:

  • input – Input matrix to process

  • i_start – Start index

  • i_end – End index

class ConvNet : public nam::Buffer
#include <convnet.h>

Convolutional neural network model.

A ConvNet consists of multiple ConvNetBlocks with increasing dilation factors, followed by a head layer that produces the final output.

Public Functions

ConvNet(const int in_channels, const int out_channels, const int channels, const std::vector<int> &dilations, const bool batchnorm, const activations::ActivationConfig &activation_config, std::vector<float> &weights, const double expected_sample_rate = -1.0, const int groups = 1)

Constructor.

Parameters:

  • in_channels – Number of input channels

  • out_channels – Number of output channels

  • channels – Number of channels in the hidden layers

  • dilations – Vector of dilation factors, one per block

  • batchnorm – Whether to use batch normalization

  • activation_config – Activation function configuration

  • weights – Model weights vector

  • expected_sample_rate – Expected sample rate in Hz (-1.0 if unknown)

  • groups – Number of groups for grouped convolution

~ConvNet() = default

Destructor.

virtual void process(NAM_SAMPLE **input, NAM_SAMPLE **output, const int num_frames) override

Process audio frames.

Parameters:

  • input – Input audio buffers

  • output – Output audio buffers

  • num_frames – Number of frames to process

virtual void SetMaxBufferSize(const int maxBufferSize) override

Resize all buffers to handle maxBufferSize frames.

Parameters:

maxBufferSize – Maximum number of frames to process in a single call

inline virtual int GetPrewarmSamples() override

Get how many samples should be processed for the model to be considered “warmed up”.

Override this in subclasses to specify prewarm requirements.

Returns:

Number of samples needed for prewarm

class ConvNetBlock
#include <convnet.h>

A single block in a ConvNet.

Consists of a dilated convolution, optional batch normalization, and activation.

Public Functions

inline ConvNetBlock()

Default constructor.

void set_weights_(const int in_channels, const int out_channels, const int _dilation, const bool batchnorm, const activations::ActivationConfig &activation_config, const int groups, std::vector<float>::iterator &weights)

Set the parameters (weights) of this block.

Parameters:

  • in_channels – Number of input channels

  • out_channels – Number of output channels

  • _dilation – Dilation factor for the convolution

  • batchnorm – Whether to use batch normalization

  • activation_config – Activation function configuration

  • groups – Number of groups for grouped convolution

  • weights – Iterator to the weights vector. Will be advanced as weights are consumed.

void SetMaxBufferSize(const int maxBufferSize)

Resize buffers to handle maxBufferSize frames.

Parameters:

maxBufferSize – Maximum number of frames to process in a single call

void Process(const Eigen::MatrixXf &input, const int num_frames)

Process input matrix directly (new API, similar to WaveNet)

Parameters:

  • input – Input matrix (channels x num_frames)

  • num_frames – Number of frames to process

void process_(const Eigen::MatrixXf &input, Eigen::MatrixXf &output, const long i_start, const long i_end)

Process input (legacy method for compatibility, uses indices)

Parameters:

  • input – Input matrix

  • output – Output matrix

  • i_start – Start index in input

  • i_end – End index in input

Eigen::Block<Eigen::MatrixXf> GetOutput(const int num_frames)

Get output from last Process() call.

Parameters:

num_frames – Number of frames to return

Returns:

Block reference to the output

long get_out_channels() const

Get the number of output channels.

Returns:

Number of output channels

Public Members

Conv1D conv

The dilated convolution layer.

struct ConvNetConfig : public nam::ModelConfig
#include <convnet.h>

Configuration for a ConvNet model.

Public Functions

virtual std::unique_ptr<DSP> create(std::vector<float> weights, double sampleRate) override

Construct a DSP object from this configuration.

Parameters:

  • weights – Model weights (taken by value to allow move for WaveNet)

  • sampleRate – Expected sample rate in Hz

Returns:

Unique pointer to a DSP object

Public Members

int channels
std::vector<int> dilations
bool batchnorm
activations::ActivationConfig activation
int groups
int in_channels
int out_channels
class ConvNet : public nam::Buffer

Convolutional neural network model.

A ConvNet consists of multiple ConvNetBlocks with increasing dilation factors, followed by a head layer that produces the final output.

Public Functions

ConvNet(const int in_channels, const int out_channels, const int channels, const std::vector<int> &dilations, const bool batchnorm, const activations::ActivationConfig &activation_config, std::vector<float> &weights, const double expected_sample_rate = -1.0, const int groups = 1)

Constructor.

Parameters:

  • in_channels – Number of input channels

  • out_channels – Number of output channels

  • channels – Number of channels in the hidden layers

  • dilations – Vector of dilation factors, one per block

  • batchnorm – Whether to use batch normalization

  • activation_config – Activation function configuration

  • weights – Model weights vector

  • expected_sample_rate – Expected sample rate in Hz (-1.0 if unknown)

  • groups – Number of groups for grouped convolution

~ConvNet() = default

Destructor.

virtual void process(NAM_SAMPLE **input, NAM_SAMPLE **output, const int num_frames) override

Process audio frames.

Parameters:

  • input – Input audio buffers

  • output – Output audio buffers

  • num_frames – Number of frames to process

virtual void SetMaxBufferSize(const int maxBufferSize) override

Resize all buffers to handle maxBufferSize frames.

Parameters:

maxBufferSize – Maximum number of frames to process in a single call

inline virtual int GetPrewarmSamples() override

Get how many samples should be processed for the model to be considered “warmed up”.

Override this in subclasses to specify prewarm requirements.

Returns:

Number of samples needed for prewarm

class ConvNetBlock

A single block in a ConvNet.

Consists of a dilated convolution, optional batch normalization, and activation.

Public Functions

inline ConvNetBlock()

Default constructor.

void set_weights_(const int in_channels, const int out_channels, const int _dilation, const bool batchnorm, const activations::ActivationConfig &activation_config, const int groups, std::vector<float>::iterator &weights)

Set the parameters (weights) of this block.

Parameters:

  • in_channels – Number of input channels

  • out_channels – Number of output channels

  • _dilation – Dilation factor for the convolution

  • batchnorm – Whether to use batch normalization

  • activation_config – Activation function configuration

  • groups – Number of groups for grouped convolution

  • weights – Iterator to the weights vector. Will be advanced as weights are consumed.

void SetMaxBufferSize(const int maxBufferSize)

Resize buffers to handle maxBufferSize frames.

Parameters:

maxBufferSize – Maximum number of frames to process in a single call

void Process(const Eigen::MatrixXf &input, const int num_frames)

Process input matrix directly (new API, similar to WaveNet)

Parameters:

  • input – Input matrix (channels x num_frames)

  • num_frames – Number of frames to process

void process_(const Eigen::MatrixXf &input, Eigen::MatrixXf &output, const long i_start, const long i_end)

Process input (legacy method for compatibility, uses indices)

Parameters:

  • input – Input matrix

  • output – Output matrix

  • i_start – Start index in input

  • i_end – End index in input

Eigen::Block<Eigen::MatrixXf> GetOutput(const int num_frames)

Get output from last Process() call.

Parameters:

num_frames – Number of frames to return

Returns:

Block reference to the output

long get_out_channels() const

Get the number of output channels.

Returns:

Number of output channels

Public Members

Conv1D conv

The dilated convolution layer.

class BatchNorm

Batch normalization layer.

In production mode, so really just an elementwise affine layer. Applies: y = (x - mean) / sqrt(variance + eps) * weight + bias which simplifies to: y = scale * x + loc

Public Functions

inline BatchNorm()

Default constructor.

BatchNorm(const int dim, std::vector<float>::iterator &weights)

Constructor with weights.

Parameters:

  • dim – Dimension of the input

  • weights – Iterator to the weights vector. Will be advanced as weights are consumed.

void process_(Eigen::MatrixXf &input, const long i_start, const long i_end) const

Process input in-place.

Parameters:

  • input – Input matrix to process

  • i_start – Start index

  • i_end – End index