Modifying the Software#

Implementing the ASR API#

Begin your ASR port by creating a new folder under example/speech_recognition/asr/port. Be sure to include asr/api/asr.h in your port’s main source file. The asr.h and device_memory.h files include comments detailing the public API methods and parameters. ASR ports that implement the public API defined can easily be added to current and future XCORE-VOICE example designs that support speech recognition.

Pay close attention to the functions: - asr_printf - devmem_malloc - devmem_free - devmem_read_ext - devmem_read_ext_async - devmem_read_ext_wait

ASR libraries should call asr_printf instead of printf or xcore’s debug_printf.

ASR libraries must not call malloc directly to allocate dynamic memory. Instead call the devmem_malloc and devmem_free functions. This allows the application to provide alternative implementations of these functions - like pvPortMalloc and vPortFree in a FreeRTOS application.

The devmem_read_ext function is provided to load data directly from external memory (QSPI flash or LPDDR) into SRAM. This is the recommended way to load coefficients or blocks of data from a model. It is far more efficient to load the data into SRAM and perform any math on the data while it is in SRAM. The devmem_read_ext function a signature similar to memcpy. The caller is responsible for allocating the destination buffer.

Like devmem_read_ext, the devmem_read_ext_async function is provided to load data directly from external memory (QSPI flash or LPDDR) into SRAM. devmem_read_ext_async differs in that it does not block the caller’s thread. Instead it loads the data in another thread. One must have a free core when calling devmem_read_ext_async or an exception will be raised. devmem_read_ext_async returns a handle that can later be used to wait for the load to complete. Call devmem_read_ext_wait to block the callers thread until the load is complete. Currently, each call to devmem_read_ext_async must be followed by a call to devmem_read_ext_wait. You can not have more than one read in flight at a time.

Note

XMOS provides an arithmetic and DSP library which leverages the XS3 Vector Processing Unit (VPU) to accelerate costly operations on vectors of 16- or 32-bit data. Included are functions for block floating-point arithmetic, fast Fourier transforms, discrete cosine transforms, linear filtering and more. See the XMath Programming Guide for more information.

Note

To minimize SRAM scratch space usage, some ASR ports load coefficients into SRAM in chunks. This is useful when performing a routine such as a vector matrix multiply as this operation can be performed on a portion of the matrix at a time.

Note

You may also need to modify BRICK_SIZE_SAMPLES in app_conf.h to match the number of audio samples expected per process for your ASR port. In other example designs, this is defined by appconfINTENT_SAMPLE_BLOCK_LENGTH. This is set to 240 in the existing example designs.

In the current source code, the model data (and optional grammar data) are set in src/process_file.c. Modify these variables to reflect your data. The remainder of the API should be familiar to ASR developers. The API can be extended if necessary.

Flashing Models#

To flash your model, modify the --data argument passed to xflash command in the Flashing the Model section.

See asr/port/example/asr_example_model.h to see how the model’s flash address is defined.

Placing Models in SRAM#

Small models (near or under 100kB in size) may be placed in SRAM. See asr/port/example/asr_example_model.h` and asr/port/example/asr_example_model.c for more information on placing your model in SRAM.

ASR API#

enum asr_error_enum#

Enumerator type representing error return values.

Values:

enumerator ASR_OK#: Ok.

enumerator ASR_ERROR#: General error

enumerator ASR_INSUFFICIENT_MEMORY#: Insufficient memory for given model.

enumerator ASR_NOT_SUPPORTED#: Function not supported for given model.

enumerator ASR_INVALID_PARAMETER#: Invalid Parameter.

enumerator ASR_MODEL_INCOMPATIBLE#: Model type or version is not compatible with the ASR library.

enumerator ASR_MODEL_CORRUPT#: Model malformed.

enumerator ASR_NOT_INITIALIZED#: Not Initialized.

enumerator ASR_EVALUATION_EXPIRED#: Evaluation period has expired.

enum asr_keyword_enum#

Enumerator type representing each supported keyword.

Values:

enumerator ASR_KEYWORD_UNKNOWN#: Keyword is unknown.

enumerator ASR_KEYWORD_HELLO_XMOS#

enumerator ASR_KEYWORD_ALEXA#

enumerator ASR_NUMBER_OF_KEYWORDS#

enum asr_command_enum#

Enumerator type representing each supported command.

Values:

enumerator ASR_COMMAND_UNKNOWN#: Command is unknown.

enumerator ASR_COMMAND_TV_ON#

enumerator ASR_COMMAND_TV_OFF#

enumerator ASR_COMMAND_VOLUME_UP#

enumerator ASR_COMMAND_VOLUME_DOWN#

enumerator ASR_COMMAND_CHANNEL_UP#

enumerator ASR_COMMAND_CHANNEL_DOWN#

enumerator ASR_COMMAND_LIGHTS_ON#

enumerator ASR_COMMAND_LIGHTS_OFF#

enumerator ASR_COMMAND_LIGHTS_UP#

enumerator ASR_COMMAND_LIGHTS_DOWN#

enumerator ASR_COMMAND_FAN_ON#

enumerator ASR_COMMAND_FAN_OFF#

enumerator ASR_COMMAND_FAN_UP#

enumerator ASR_COMMAND_FAN_DOWN#

enumerator ASR_COMMAND_TEMPERATURE_UP#

enumerator ASR_COMMAND_TEMPERATURE_DOWN#

enumerator ASR_NUMBER_OF_COMMANDS#

typedef void *asr_port_t#

Typedef to the ASR port context struct.

An ASR port can store any data needed in the context. The context pointer is passed to all API methods and can be cast to any struct defined by the ASR port.

typedef struct asr_attributes_struct asr_attributes_t#: Typedef to the ASR port and model attributes

typedef struct asr_result_struct asr_result_t#: Typedef to the ASR result

typedef enum asr_error_enum asr_error_t#: Enumerator type representing error return values.

typedef enum asr_keyword_enum asr_keyword_t#: Enumerator type representing each supported keyword.

typedef enum asr_command_enum asr_command_t#: Enumerator type representing each supported command.

void asr_printf(const char *format, ...)#

String output function that allows the application

to provide an alternative implementation.

ASR ports should call asr_printf instead of printf

asr_port_t asr_init(int32_t *model, int32_t *grammar, devmem_manager_t *devmem_ctx)#

Initialize an ASR port.

Parameters:

model – A pointer to the model data.
grammar – A pointer to the grammar data (Optional).
devmem_ctx – A pointer to the device manager (Optional). Save this pointer if calling any device manager API functions.

Returns:

the ASR port context.

asr_error_t asr_get_attributes(asr_port_t *ctx, asr_attributes_t *attributes)#

Get engine and model attributes.

Parameters:

ctx – A pointer to the ASR port context.
attributes – The attributes result.

Returns:

Success or error code.

asr_error_t asr_process(asr_port_t *ctx, int16_t *audio_buf, size_t buf_len)#

Process an audio buffer.

Parameters:

ctx – A pointer to the ASR port context.
audio_buf – A pointer to the 16-bit PCM samples.
buf_len – The number of PCM samples.

Returns:

Success or error code.

asr_error_t asr_get_result(asr_port_t *ctx, asr_result_t *result)#

Get the most recent results.

Parameters:

ctx – A pointer to the ASR port context.
result – The processed result.

Returns:

Success or error code.

asr_error_t asr_reset(asr_port_t *ctx)#

Reset ASR port (if necessary).

Called before the next call to asr_process.

Parameters:

ctx – A pointer to the ASR port context.

Returns:

Success or error code.

asr_error_t asr_release(asr_port_t *ctx)#

Release ASR port (if necessary).

The ASR port must deallocate any memory.

Parameters:

ctx – A pointer to the ASR port context.

Returns:

Success or error code.

asr_keyword_t asr_get_keyword(asr_port_t *ctx, int16_t asr_id)#

Return the XCORE-VOICE supported keyword type.

Parameters:

ctx – A pointer to the ASR port context.
asr_id – The ASR port keyword identifier.

Returns:

XCORE-VOICE supported keyword type

asr_command_t asr_get_command(asr_port_t *ctx, int16_t asr_id)#

Return the XCORE-VOICE supported command type.

Parameters:

ctx – A pointer to the ASR port context.
asr_id – The ASR port command identifier.

Returns:

XCORE-VOICE supported command type

struct asr_attributes_struct#: #include <asr.h>

Typedef to the ASR port and model attributes

struct asr_result_struct#: #include <asr.h>

Typedef to the ASR result

Device Memory API#

void *devmem_malloc(devmem_manager_t *ctx, size_t size)#

Memory allocation function that allows the application to provide an alternative implementation.

Call devmem_malloc instead of malloc

Parameters:

ctx – A pointer to the device memory context.
size – Number of bytes to allocate.

Returns:

A pointer to the beginning of newly allocated memory, or NULL on failure.

void devmem_free(devmem_manager_t *ctx, void *ptr)#

Memory deallocation function that allows the application to provide an alternative implementation.

Call devmem_free instead of free

Parameters:

ctx – A pointer to the device memory context.
ptr – A pointer to the memory to deallocate.

void devmem_read_ext(devmem_manager_t *ctx, void *dest, const void *src, size_t n)#

Synchronous extended memory read function that allows the application

to provide an alternative implementation. Blocks the callers thread until the read is completed.

Call devmem_read_ext instead of any other functions to read memory from flash, LPDDR or SDRAM. Modules are free to use memcpy if the dest and src are both SRAM addresses.

Parameters:

ctx – A pointer to the device memory context.
dest – A Pointer to the destination array where the content is to be read.
src – A Pointer to the source of data to be read.
n – Number of bytes to read.

int devmem_read_ext_async(devmem_manager_t *ctx, void *dest, const void *src, size_t n)#

Asynchronous extended memory read function that allows the application

to provide an alternative implementation.

Call asr_read_ext_async instead of any other functions to read memory from flash, LPDDR or SDRAM.

Parameters:

ctx – A pointer to the device memory context.
dest – A Pointer to the destination array where the content is to be read.
src – A Pointer to the source of data to be read.
n – Number of bytes to read.

Returns:

A handle that can be used in a call to devmem_read_ext_wait.

void devmem_read_ext_wait(devmem_manager_t *ctx, int handle)#

Wait in the caller’s thread for an asynchronous extended memory read to finish.

Parameters:

ctx – A pointer to the device memory context.
handle – The devmem_read_ext_asyc handle to wait on.

IS_SRAM(a)#

IS_SWMEM(a)#

IS_FLASH(a)#